You may want to consider a phone call to Tool's Anonymous (Pro. Standards), you are most likely not alone in your assesment, and that's probably why that good trip (with him on it) was in open time in the first place! At least you'll have a 'paper trail' if you do get stuck flying with him again sometime, and elect to get off the trip, or sick out of it.
Western used to keep a "No Fly" list, where all crew members could put down others they refused to fly with. This was good for two reasons; the obvious being if all the other crew members are putting in the same name, well, that guy needs an attitude adjustment.
BUT...if one crew member had half the base listed as "No Fly" then they knew who the problem child really was!