It's just not worth the hassle to travel in civvies.

Once past the tray-stacking sudoku ninjas I will throw on a light fleece, aka. my airport camo.

The trick is to keep it unzipped enough to get the discount at Starbucks, but not enough to be asked questions by helpless pax.