Don't like to generalize. Think many people who quit music were enrolled by their parents at a young age. They were pushed to practice like exercises. After someone decides to quit music, it's unlikely he/she would return later.
I got into music in high school around age 17 when my sister asked for violin lessons. Coming from a non-musical family, mom & dad were ok with music lessons but not pushy. Neither of us knew anything about the instrument. There was 1 person in the neighborhood where we lived who supposedly had violin lessons. I navigated my way in a haphazard way.
When I bought my keyboard 2 decades later, I had some experience reading music and learned enough music theory from the violin teacher. Learning is still a struggle but not as much as before. Being an adult learner I pay for my own lessons. With online info to supplement my learning at least I'm not driving without a roadmap.
Even if I'm not making a lot of progress now, I have collected a library of pieces to keep me playing for years so no reason to quit.