SouthPark K.S. I know what you mean. It was a long time ago - I thought the issue was me - but the issue is actually with the system. The way that people are teaching it - which is messed up.
For example, for intervals - people are often teaching things like ---- 'From C to E'. And students are always thinking the obvious -- and intuitively -- sure -- from C --- in C major scale, the 'E' is two notes 'away' from 'C', right?
There is a much bigger picture that I'd like to go into for teaching as a whole. I find that when things are taught, esp. theory, "basic things" are glossed over, or gotten at in a utilitarian superficial way. By "utilitarian", I mean let's give a quick name to this so that we can get back to playing music - or let's memorize these rules so the exam can be passed. The teacher may have learned in the same manner and nobody has ever really gone to the heart of anything. We can pass exams nicely without ever grasping the essence of a thing.
In 2006 I studied the whole RCM rudiments, did the middle and upper level exams passing the middle with 99.95% and felt quite dissatisfied. I restudied it all, this time at the piano or tapping rhythms to make it more real: then I taught the whole thing over 18 months, this time the way that I would have wanted to do it. I was still near the start of my own journey, but it went fairly well.
When I worked with that student, who had been playing for some years, one of the elementary concepts that had gotten mixed up involved time. There is note value, the relative note value of say half note vs. quarter note, beats, and meter. For some of this there were "what is this really?" moments to sort out. (Just as an example - I don't want to start a new topic. 😉 )
The elementary concepts are "so easy", so "babyish" - why spend any time on them. And yet we build everything on top of them and if the bricks are misshapen, we'll flounder here and there without ever knowing the cause as we advance. OR we actually succeed with high grades in exams, be able to pontificate to the next crop of students if we become teachers, without ever really connecting any of this to music.
In my own training as a teacher at the "formative" level (primary grades), I already had the concept of going from concrete to abstract. The "centimeter cubes" example is that. So I went in that direction. My own first experience in lessons, which happened as an adult, collapsed largely because the "baby things" were rushed through. While a lot of this was physical instruction, the same principle applied. When I went back to basics and the "real" version, problems in advanced music also often resolved themselves.
This has gone way past intervals, but the issue with intervals that you described I think is part of that picture.