I think the best thing to do is to confront the person in question nose to nose and say, "Are you making an accusation or a slanderous remark, because I will not be slandered by you or anyone else." Then watch how quickly they backpedal and say, "Jeez! I was just kidding!" Then you say, "I'm still not laughing and I await your apology."