Originally Posted by BlueMoon

Your not supposed to remove them from the tracks, there are some rubber pieces that are in the grip that help it grip the track better. Unfortunately, when people take the visor off the tracks (every airplanes, except some of the new ones), they sometimes/eventually fall out. That is why you can't swing it up.