That's what I do. I don't even fully uncup my ear, I just twist the headphone to make a little gap in the front. I've done 7:15 FLL-SEA and never took it off other than bathroom break. Doesn't bother me in the slightest.

Sure, the NEO is "quiet", but by the time the other guy is raising his voice to have a convo and I have the speaker on loud enough to understand what ATC is saying, it's no longer nice and quiet. With a headset I can control how loud I hear ATC and how loud you talk to me.

If your A20's are too tight, leave them on a soccer ball on your days off at home.