Of course, if that is your company's policy. But the FAR are clear.

121.533:

(a) Each certificate holder conducting domestic operations is responsible for operational control.

. . .

(d) Each pilot in command of an aircraft is, during flight time, in command of the aircraft and crew and is responsible for the safety of the passengers, crewmembers, cargo, and airplane.

. . .

(e) Each pilot in command has full control and authority in the operation of the aircraft, without limitation, over other crewmembers and their duties during flight time, whether or not he holds valid certificates authorizing him to perform the duties of those crewmembers.

If you boot a pax off during boarding at most airlines, you are going to have to explain why.