I usually commute in and out on the same day of my trip and I always commute in uniform. And yes, I take off my tie and insignia because I don't like wearing the tie and it's all or nothing.
Honesty, who cares? Do it if you want, don't do it if you don't want. If I have a long airport sit or an opportunity to go to a lounge and grab a meal or drinks then I will change. Otherwise, I just keep the uniform on out of either laziness or not wanting to iron a shirt in the lounge.