Excellent choice on the degree! Definitely pursue it. I graduated college with EE and 1500 hrs with CFI, CFII, and two type ratings. The flight test field is NOT hard to get into so don't let people make you think it's so challenging. There are people fresh out of college with no pilot's experience that get FTE jobs, it's not a big deal. If anything you are overqualified and you WILL get calls if you apply to different manufacturers.

There is one kicker though, which I am sure you are familiar with, and that is simply the aircraft industry. I love aviation, but I chose not to study AE (although it is ideal for a test pilot position) because I didn't want my life to depend on the aircraft industry. If one aircraft manufacturer is laying off, chances are all them will too. EE (or even CSE nowadays, but coding is too boring for me) gives you a lot of flexibility. If an aircraft manufacturer lays you off, you can just look for a job nearly anywhere else (power distribution, RF, optics, etc.). You may not be the happiest elsewhere, but at least it provides decent income and food on the table.

I will be honest though, some AEs in the industry look down on other engineering disciplines (such as EE or CSE) because it doesn't exactly line up with what they perceive their line of work to be. What you will find out however is that a degree like EE is actually ideally suited for flight test today. Simply put, the biggest problems we face regarding testing, evaluation, debugging, etc is avionics, and that is where the field is growing. A lot of the "black magic" in aerospace engineering has largely been unveiled. Not to say that AE is unmarketable, but it will be a while (probably when commercial spaceflight starts becoming more commonplace) before it truly becomes an "emerging" field again. What I tell a lot of people is to just look at a Cessna 182 from 1978 and one from 2012. Airfoil is the same, airframe is largely the same (bigger engine but that's pretty much it)....what's different is the cockpit.

I've probably repeated a lot that has been already said, but hope you can take something from this and best of luck.