Spoken like a bunch of people who have never needed to commute internationally.
As a rule of thumb, domestically, I suggest keeping your ID on but tucked into a shirt pocket. Other crews are smart enough to figure this out and you don't look like a douche to those who don't know. There is a noticeable difference in how you're treated.
Internationally (on a jumpseat or ticket) you need to play it by ear. But if you want to guarantee yourself a nightmare be sure to wear civvies going to KWI on a paid ticket. ME, South America, South Asian, and ASEAN countries definitely treat you much better in uniform. Other Asian countries do a little, but not much. In Europe always go in civvies or they get bent out of shape.
I wouldn't suggest this on personal travel though as a matter of protocol... just work travel (paid or jumpseat).