I've experienced vertigo numerous times. Some of my training seemed to be in situations that easily induced it. Sitting in the back cockpit with a blind pulled over the cockpit and the sunshine (especially on a day with broken layers) seem to induced vertigo easily. also, flying formation in the clouds was a sure way of getting vertigo especially if lead was maneuvering at some time.
Unless you have a different vestibular system then you don't have any type of super human immunity to vertigo I would guess, rather you just haven't been in IMC enough or encountered the right mix of conditions. One way that they would induced vertigo in our training in the spin-n-puke was to put us in a turn and then have us look down and left to a control panel to switch freqs or something and then look back up quickly. This was a sure fire way of making the world tumble.

USMCFLYR