No, not necessarily. You'd have to approve that prior with the guy in the left seat. If he's cool with not logging the time at all, then you can. The only way that both of you are gonna log that time is if one of you gives the other dual or if one of you has a hood on.
If something happened, the FAA or insurance may have a problem with you loggin with passengers on board. Some of the old farts who don't care about hours any longer may be more willing to let you log from the left on empty legs. Depends on the place.