Ya'll are missing the most important thing here. What does your company's Ops Manual say? Where I work, there is not much guidance for jumpseating, but working a flight and being accused warrants a call to the CPSC and removal from the flight until tested. Any accusation, even jokingly, is an accusation, and there can be ramifications from the company for just blowing it off.
Here's the scenario:
Joe Bagadonuts, being the dumbass that he is, makes a joking accusation about you drinking. HaHa, very funny. You give him the stinkeye and carry on.
Larry, the ambulance chaser, already a fearful flyer, doesn't get the subtle undertones of Joe's humor and thinks that it was a serious accusation that you just blew off. Next thing you know, Action News at Six is hyping the story and the FAA and CP is looking for you.
Don't think it can happen? It already has on numerous occasions and that is why there is company guidance on what to do when accused. If your company doesn't have any formal guidance, at a minimum, you should call the CP with all the pertinent info before flying.