No way!Don't do it!! If you're married with a family, especially don't do it. It would be about the most selfish thing you could do to your family. If you still have your med school loans, you'd be an idiot to do this, sorry, but I have to call it like it is.
I know several pilot/doctors in the Air Force. Guys who do both jobs at the same time. When I've spoken to them about their post AF career, they both said they'd practice medicine, without a doubt. By the way, if you were younger, I'd say go this route, pretty sweet deal. You might even get a reserve or guard unit to pick you up with an age waiver based on your background.
You're QOL (and finances) will be in the gutter compared to what you have now, for 6 to 9 years, maybe even indefinitely.And you will never make the kind of money you can make as a radiologist (I'm assuming you can make $300 to $900k a year). Believe me, this will become a JOB someday, and be about the money.
What you can and should do, is get your ratings at a local school, become a CFII on the side, maybe do some part 91 contract work, get jet type ratings etc. you could easily be a contract for hire corporate pilot on the side, picking up trips as you want. Then down the road, you'd have the option to jump ship. Y