I would switch asap. I switched CFI's after the first 5 hours of flight instruction when I was working on my PPL. My first instructor just didn't seem to know what he was doing. It made a huge difference switching CFI's. I learned more in my first lesson with a new CFI than I did in the first five with the old CFI. Why spend 160$ p/h to get yelled at?