Again, the FAA says ONLY YOU can determine your fitness to fly. You might wake up with a tickle in your throat knowing full well that in six hours you're gonna be in a full blown head cold.
If you are good to fly back to any base, I'd give them a heads up, get the plane there and go home.
But, that's just me.