rsl12 BKN1964 For example, if I see 4 flats, I can say "oh, Ab major (even though it may really be F minor), but I can't then say "the notes are Ab, Bb, C, Db, Eb, F, G". Shan on JazzSkills emphasizes that you should be able to go even a step further, and be able to instantly bang out the chords; Abmaj7, Bbmin7, etc. As I stated in the OP, without having all the notes on instant recall, being able to add extensions while I'm playing is not really a possibility.
This is the kind of thinking the drill I described (rsl12 ) is meant to improve. Particularly if you include the arpeggio version in your practice.
I missed @BKN1964 's commend about not being able to name the flats based on how many there are.
I think this is a super important skill, and IMO more important than being able to name the key.
When I look at the key signature, I see all the accidentals as a glob 😅 But I can tell how many there are (i.e., three? four? whatever), and because I know the sharps/flats in order, I never have to look at the notes (lines, spaces) to figure out which are flat/sharp, I just know.
Long ago I memorized sentences for the order of flats and sharps (Flats: Battle Ends And Down Goes Charles Father; Sharps: Father Charles Goes Down And Ends Battle). Some people don't like this, so you could just remember that for the Flats it's BEAD plus GCF, and then the sharps are reversed from that, FCG DAEB. [Edited because I can't type]
Then, whenever I would look at the key signature, before starting on the score, I would just play and say the black keys in that key, in order. So if there were three, I would play the black keys and say "B E A" or if there were five, I would play those black keys and say B E A D G. (And I did it in both hands, but one at a time, looking at the keyboard and the black keys as I played them). Over time, doing this reinforced it significantly.
This got the order of flats and sharps embedded in my brain, and connected to the musical sounds. And now I never hesitate or have to think to figure out which notes are flat/sharp, I just look at the key signature, see "ok, four flats" and know which ones they are.
Hearing them in order as you play is important as well, so that's why I argue not to do it in keyboard order (physical order) but in musical order (i.e., for three flats, not "A B E," but "B E A").
Even now I sometimes do this (play the black keys for that key signature before starting on the piece), for example, two pieces I'm working on right now: Sept. Song has five sharps and Miracles has five flats. When I practice one and then the other back to back, I sometimes get the key signatures mixed up (I don't have this problem if I work on something else in between). So I play those black keys (in order) before starting the piece. The way a violinist or guitarist checks the tuning before starting....
Sorry this was kind of rambly... my point is, it's super important to know the order of the flats and sharps. If using a sentence helps, do that. If thinking "flats are BEAD GCF, reverse for sharps," then do that. And then it's easy to drill each set as you're playing pieces (i.e., learn it in context).