I was a career changer myself. Went from engineering to flying. How did I know? I got my private license and flew lots of $100 hamburger trips. Then after getting stuck somewhere because of fog, I decided to pursue the instrument rating. In my search, a light bulb was lit and I decided to make a career out of it, because I loved it so much.
So go give it a shot! Get your private first and figure out if you love it enough to make it career. Have fun and good luck!