Wow. I've used my military passports quite often -- lots of Visas too.

Gucci boys get better trips, I guess. In the Reserves we tended to avoid trips that did require passports as it was hard to organize reservists, confirm them on the trip, that sort of thing.

There are a few places that we go that insist we have the brown passport. Orders won't suffice.
Your unit's deployment office should know the answer.