I believe you might be a bit misinformed on the amount of mil pilots doing the UAV thing. The vast majority still fly real airplanes.
Starting in 2010, over 50% of the new USAF student pilots bypassed UPT and went directly into UAVs.
In 2010 when I went through Upt 3 people out of my class of 30 went to UAVs. That number only decreased with the classes after me.