Don't beat yourself up too hard.
I have noticed that there are certain things that can be challenging for me and I typically have no real problem with keeping a rhythm going. Certain things can be harder to hear and grapple with mentally. It took me a while to feel comfortable with 5/4 time and the metronome on my keyboard helped me to work through some of the timing stuff, like a triplet against a 5 beat measure and not lose the beat, skip a beat, or add a beat. Certain pieces of music might also sound quite standard when casually listening but aren't quite as simple when you get closer. "Linus & Lucy" is an example of that.
It helps that I normally play in a band with a drummer being one of the other members. Helps to keep me honest. Sometimes I feel a little limited by the lack of freedom to take as many liberties with the timing as I might want to. "Stardust" has a few places where a pause is nice, not a complete stop, but just plowing through doesn't quite feel right, either.
It's fun.