BKN1964 Okay, now were getting somewhere...
The first thing that jumps out, from what you've said, I think (and this is just my opinion) that you absolutely must know the correct scale fingerings. Otherwise, even if you could recite something like the 11th of A major by heart, your fingers won't get you where you want to go, when you want to improvise, without having those fingerings come easily.
Based on what you said, I would recommend you pick a few tunes, doesn't matter the key, just make sure they are tunes you really like, and work on them. Pick them apart, find the scales that go with the chords in the context of the key the song is in, and practice playing those scales over the changes. Maybe find the 2-5-1s, or turnarounds, and work on those. Once you're at the point where you can get around smoothly through those changes of those three or four tunes. Add another tune. But keep working on the others. You will subconsciously recognize the chord/scale structure similarities of these tunes, and future tunes.
If you haven't tried it, I suggest looking at "I Got Rhythm". This tune covers a lot of "jazz ground".
Jamey Aebersold has a Play-a-Long called "I Got Rhythm Changes In All Keys"
https://www.jazzbooks.com/jazz/product/V47DS
The play-a-long tracks are good, comfortable tempos. But the book's breakdown of the "original rhythm" and "jazz rhythm", with regard to the various scales that can be used to improvise is comprehensive. And if you can get around on "Rhythm Changes", you'll find those changes, or parts of the changes, in lots of other tunes.
If you plan on playing with horns, or having them want to play with you, you have to work on Bb (trumpet, tenor sax) and Eb (alto sax). I know a piano player who can play almost any tune and improvise to the point where he sounds good. But he can only play in C and A minor. Horn players (and female vocalists) hate playing with him.
