I understand your frustration with part 61 training and the long process that instructors can make of it. I left for 141 training to get all my flight training done quickly and with loans. Our situations were different.

You have a good job, I didn't.
You have an airplane to fly, I didn't.
You have instructors slowing you down. I really had money slowing me down, but my instructor was part-time.

My advice:
Keep your job.
Fly the Arrow.
Get a new instructor or more than one. You're the boss, fire your instructor and find a new one. The least you should do is talk to him about it.
Get your instrument, then commercial, then multi-eninge, then all your CFI ratings. Worry about building twin time later.

Have fun. Good luck.