I suspect the falling notes appear more intuitive because the music follows the orientation of the keyboard. Whereas the sheet music is more like the keyboard turned 90 degrees counterclockwise.
So I have a follow-up question. For people who eventually developed fluent score reading ability, do they somehow mentally turning the staffs to match the keyboard, or form some other kind of positional association that makes the link between notes on paper and keys under finger more direct?
ShiroKuro
I used Playground Sessions for a couple of months so that I would get instant feedback when an incorrect note is pressed. Playground Sessions takes the MIDI from the keyboard as input and displays incorrect notes in different color on the screen. It helped me some, but eventually I made a better association between the sheet music and the notes that I played. I also learned to hear incorrect notes meanwhile. Thus I do not use Playground Session any longer, except once in a while I check if new music scores have been released that I like.
iternabe
I do not turn/rotate the staffs in my mind to match the keyboard. Instead when I read music, I generally recognize in which octave range the note/chord is located, e.g. C2-C3, C3-C4, etc. while also recognizing the note(s) themselves. There is also some pattern recognition involved, e.g., recognizing chords, playing octaves. When playing octaves, I have a particular finger spread so that I only need to find the bottom note (or the top note) on the keyboard. The other note of the octave is determined by the spread of my fingers. Similar for playing an octave with a fifth in between. If I just find the root note, then the fifth and the upper note of the octave is "known" by a given distance of my hand, so there is no need to look at it.
iternabe For people who eventually developed fluent score reading ability, do they somehow mentally turning the staffs to match the keyboard, or form some other kind of positional association that makes the link between notes on paper and keys under finger more direct?
I am a very confident score-reader, and a pretty good (relatively speaking) sightreader.
I do not mentally turn the staff, I never did.
I'm trying to think if I have anything helpful to say in terms of how I got so comfortable reading....
Read a lot? Also maybe do some HS practice while reading, to reinforce the notes on the page with the place on the keyboard, without having to be overwhelmed by the two staves?
ShiroKuro I'm trying to think if I have anything helpful to say in terms of how I got so comfortable reading....
Read a lot?
That's pretty much it. I know this answer is not satisfying for us adult overthinkers but there isn't really a better one. Just read a lot of music and over the long term your reading skills will improve.
@BartK although I guess we could add to the recommendations a bit, to specify what kind of reading and what to do while reading.
If I were making a “how to” list for improving wading, this is what I would put on it:
Oh, and here’s something I’ve seen mentioned but that I am pretty sure I do not do:
Read from the bottom staff up, rather than from the top staff down.
Does anyone here actively do that?
I am almost positive that this is not how I read, but I can’t figure out why, although I have some ideas.
For example, I wonder if it’s because I learned to read the treble clef first, so that was easier. Another possibility might be because it seems intuitive to read from top to bottom.
Lastly, I wonder if it’s because most music is melody driven and it’s more likely that there are repeating patterns in the LH, which makes it easier to put the LH on autopilot such that, once you’ve mostly learned the piece, you don’t have to actively read the LH part as much as you do the RH part.
This last part may be where @Animisha is feeling challenged, because if you’re not actively reading the LH part, it can be easy to miss subtle changes, like a pattern that’s repeated but only one note in a chord is changed. I’m pretty sure I make that mistake a lot, missing something in the LH because I’m not reading it closely. IMO, that’s what red pens are for! 
I eventually get it corrected.
Which is to say, I’m not that worried about initially learning a wrong note.
Actually, I find the immediate feedback on mistakes (I see that Synthesia stopped moving) very beneficiary.
Also, not having a good memory, I see this strange chord, I figure it out, I try to remember it, next day, I need to figure it out again - it is so, so wonderful to immediately get feedback if I figured it out correctly.
Also, big advantage, I am more relaxed when figuring out difficult passages. No more worry I learn incorrectly.
ShiroKuro But if you want to improve your score-reading ability and improve at playing from a score, Synthesia will be counterproductive.
Personally, I think immediate feedback the moment I play an incorrect note, will help.
*
... feeling like the pianist on the Titanic ...
Actually, the subtle changes in the piece that I talked about ("C# is played together with E, or with D, or with F#, or even with F") were in the alto voice. Still, it is quite true that most of my attention goes to the voice that is the most important one, which often is the soprano voice.
I have never heard of your recommendation "Read from the bottom staff up, rather than from the top staff down" and I never sight read like that. In the easier sight-reading pieces that I do, I focus on getting the melody right, both the notes and the rhythm, and I hope that the single notes or chords in the bass clef somehow tag along.
By the way...
I did not recommend Synthesia for sight reading practice. In sight reading you need to play through your mistakes, so after one mistake, Synthesia would stop moving. So it is useless.
I do - still 
- recommend Synthesia (or a similar app) for the early stages of learning a new piece, when you want to make sure that you are not learning the wrong notes.
*
... feeling like the pianist on the Titanic ...
@Animisha I feel sorry for nudging the topic of this discussion from note/chord/error recognition to sight reading. They are different, and we already have a sight reading thread. But I am glad, as always, I elicited some very helpful insights on sight-reading from forum members. 
One thing @ShiroKuro mentioned strikes me as so true is the need to consciously read the notes on the sheet after the piece is already mostly memorized by hand. The reason I worry about committing wrong notes into subconscious memory is exactly because I only read them closely during the very early part of learning a piece. Pretty soon after there become a point where the fingers retained enough muscle memory (not by intention) that my brain no longer fully register and scrutinize those notes on the sheet even when my eyes are alway looking at them. It's like a "fake" state of reading while playing.
So, maybe add to her great advice: "Read a lot of new music"?
iternabe For people who eventually developed fluent score reading ability, do they somehow mentally turning the staffs to match the keyboard, or form some other kind of positional association that makes the link between notes on paper and keys under finger more direct?
I definitely have a positional association. If I see a chord on a grand staff, I will automatically feel in my hands how I'd play it, and I visualise where it would be on the keyboard. I don't consciously rotate the score in my mind, but I think this idea can help when learning to read piano music.
Animisha I have never heard of your recommendation "Read from the bottom staff up, rather than from the top staff down" and I never sight read like that.
Nor do I. I think I read more from the outsides inwards: if I'm sight-reading a tricky piece, it is usually most important to get the melody and the bass notes right. When things are flying by too fast, I fake what's happening in the middle.
iternabe One thing @ShiroKuro mentioned strikes me as so true is the need to consciously read the notes on the sheet after the piece is already mostly memorized by hand. The reason I worry about committing wrong notes into subconscious memory is exactly because I only read them closely during the very early part of learning a piece. Pretty soon after there become a point where the fingers retained enough muscle memory (not by intention) that my brain no longer fully register and scrutinize those notes on the sheet even when my eyes are alway looking at them. It's like a "fake" state of reading while playing.
I know many beginners learn like that (including myself several years ago) but muscle memory is very weak and very unreliable. A good way to practice is to go so slow that you disengage the auto pilot. Try to read the notes conciously and be aware of everything while going glacially slow. If you know some theory you can also play in blocked chords and say the chord you are on at the same time. All of that makes you rely less on muscle memory and makes it more likely that you will notice if you missed something.
iternabe For people who eventually developed fluent score reading ability, do they somehow mentally turning the staffs to match the keyboard, or form some other kind of positional association that makes the link between notes on paper and keys under finger more direct?
I am far, far away from fluent score reading ability, but when I sit at my desk and see a score without a title, I move my fingers and hum, and if I know the piece (also pieces I have never played, for instance pop songs), I can often "hear" which piece it is. This was something I absolutely could not do before I played the piano.
Now that I think of it, I am maybe better when I do this at my desk instead of at the piano, because at my desk I don't make mistakes!
PS I don't know if this some kind of answer to your question iternabe... But I don't make a mental rotation for this to work.
*
... feeling like the pianist on the Titanic ...
iternabe The reason I worry about committing wrong notes into subconscious memory is exactly because I only read them closely during the very early part of learning a piece. Pretty soon after there become a point where the fingers retained enough muscle memory (not by intention) that my brain no longer fully register and scrutinize those notes on the sheet even when my eyes are alway looking at them.
This is exactly what I also do! And that is also a reason why I want to get external feedback on the correctness of my playing, so I don't learn the wrong notes and the wrong sounds!
*
... feeling like the pianist on the Titanic ...
MRC When things are flying by too fast, I fake what's happening in the middle.
That is probably a very good skill, esp. for sightreading truly new music!
When reading a new piece, I have a hard time leaving out notes or, for example, an inner voice. If I'm learning something and decide not to play one chord note (say because it's beyond my reach) that's one thing, but I usually can't make that decision in the middle of prima vista sightreading.
BartK disengage the auto pilot.
This is very important!! And something I tried to do in the weeks leading up to the concert I played in last week, because, as you say, muscle memory is very unreliable. But auto pilot is very... insistent. It's like the brain wants to go into auto pilot, that's the default mode.
My advice, which is worth what you paid for it!, is to avoid Synthesia - it is teaching the wrong things. Might as well light up the keys you need to press on the keyboard. The fact that it tells you when you make a mistake is not helping you - you need to develop the skill to hear when you have made a mistake yourself. And the only way to do that is to make mistakes and correct them yourself. Practice reading music 15 minutes a day, every day, and your skills will improve.
I personally don't think the way Animisha is using Synethesia is harmful at all, and in fact I think the way she's using it sounds helpful. After all, one of the things teachers do for us is catch mistakes that we would otherwise be unaware of. And that's what it's doing.
Sam you need to develop the skill to hear when you have made a mistake yourself.
Sam, I am learning a new piece, and I am not familiar with its sounds. If I would play the alto voice as E in both measures, how could I hear it? This is an honest question, because I don't know, and I certainly don't hear it.
PS The piece is in A major.
*
... feeling like the pianist on the Titanic ...
