I've had 2 FA's removed from flights since I thought their illness would impair their ability to safely perform their job or cause a bigger problem. Didn't have to do a carpet dance. As the CA you are responsible for their safety/ability to perform their duties also....
You also have the responsibility of pulling a fatigued crewmember off the airplane.