I have done this before and you're just downgraded to "cabin only" status. I took SWA and their agents didn't have issues with my not having the passport.

Shaggy, I got my passport back in only 4 days.