Agreed...have another very experienced instructor that students love to fly with...it's one thing if he's experienced, but there is a difference between knowledge/skill, and the ability to impart it.
Maybe even a third opinion.
If you all agree that it is hopeless, then all should talk to him about it together, so he won't think one guy is singling him out.