I'm with excargodog's aggregate outlook of the next 5-7 years on this one.
If I was in the OPs shoes I'd keep plugging away at the LE job and building up that pension if able. Sometimes we're simply priced out of jobs, housing markets et al, by timing, personal, medical or family demographics, good bad or indifferent. It is what it is.
Private small airplane ownership has treated me very well; it can be done on a non-upper middle class income if you align your ducks at home right, especially experimental. Of course, that outlet being sufficient to appease the individual is my biased position due to having high overhead costs at this juncture in my life now as a head of household, and never been drawn to piloting transport-category flying. The bit of it I did in the military frankly almost cured me from wanting to continue flying professionally. So I'll defer to those who legitimately enjoy that kind of flying to more positively answer whether the alternative I propose is sufficient for ya. Good luck OP.