Long-term, I think most military UAV operators will be military personnel. This is one of those things which is as obvious as water flowing downhill. The AF or other services may try to entrench civilian contractors to save money...but the first time a UAV rolls out on or designates a baby milk factory, congress will step in and fix that. If a soldier does it, its collateral damage, if a civilian does it, it's a crime against humanity.

Savvy military leaders will recognize this and proactively shift to military operators. I think we seeing this now.

The point of this...you may not be able to make a career out of military UAV operations, although I'm sure operator experience would help you get in on the engineering side.

If you want an adventure...now's the time while you are young, single and have time to get back on your career track. But, like others have said, adventure usually consists of someone else far away having a really bad time You'll have cool stories to tell later, but of course they will all be classified.