I would say that there is a definite correlation between height and g tolerance. That does not mean that very tall people can't fly fighters or are too g-intolerant to fly fighters.
The 'fuge is worse than the actual aircraft if you ask me. Many people have trouble with it and go on to be fine in the airplane. They also do a great job of publicizing the video's of the dudes who pass out, and that might skew your impressions.
At 6-4 you'd be pretty close to the max height allowable to fit the cockpit, but I've known more than one F-16 pilot your height. F-15 and A-10 cockpits are much roomier. Can't speak for the F-22 but judging by the rest of the airframe, I'm going to guess it's got plenty of room.
My advice, start smoking, lots of cheesburgers and hit the gym!