A phrase from a time when women were referred-to politely, rather than the way rappers now do. It was another way of saying "women". "Fair" in its archaic usage meaning "attractive" or "beautiful". "Fairer sex" = "beautiful gender" = "women".