rogerch It sounds like we have a similar approach, though I'm betting you're much better at it than I am. 🙂
I'm learning improvisation in a blues and boogie context. I spend some time working on specific elements such as LH patterns, RH licks, intros, endings, breaks, etc. Then I spend some time mixing and matching them, and generally free-styling it. I don't record any of that.
My main goal is to play and sing songs that are mostly composed, but with some improv sections. That's the sort of thing I'm more likely to record. The solo sections are still, at best, semi-improvised: I'm mainly playing a lick, noodling on a scale for a bit, then playing another lick, ad nauseum. It still sounds pretty cut-and-paste, but I'm trying harder to make it sound more musical -- as you put it, to think about the listener. To that end, I try to think of it more like a vocal melody, something that would be pleasant to hear someone sing, even without specific words. That makes for much better phrasing and overall composition.
It's definitely a long-term process. I've been working on all these basic, isolated elements for quite a while now, but it's starting to come together and feel more organic. I have a couple of other sources for lesson material that I want to work on for a bit, but eventually I hope to just learn lots of lots of song and work improvisation practice into just plain playing tunes, rather than a separate, artificial activity. It's a life-long process, I'm sure!