I think you should stay with IT, and do the flying stuff on the side. I went to a well-respected school and ended up with a degree in aviation. After flying and getting paid for it, it wasn't as fun as thought it would be.
I went back to school and finished up another degree that gives me a 9-5 job.
I can still go to the FBO and fly whenever I want....but I'm not tied to it. It's the best of both worlds.
Before you jump ship on your IT career, I'd get some more flight experience to get a better picture of things.