If you already have $ saved up, and are real serious about this, you could quit your job and dedicate yourself to nothing but flight training and be done in a year.
So by 42 or 43 you could be building flight time instructing. If you did this as your full time gig you could have enough hours to get a "legitimate" pilot job by 45. So if you wanted to be an airline pilot that would give you a 20 year career.
So no, you aren't to old. I don't think anyone would say that you are. It's a matter of are you wealthy enough, because if you are 41 now and made the switch you wouldn't be back to making a respectable income until you were close to 50. (flight instructuting will prob give you 15k a year) (Regional pilot might bring you closer to 20k) (regional pilot after 20 years seniority you'll be close to 6 figures)