You can log the whole time, so long as the 50 NM req. is met on at least one of the legs.

I am almost positive that 141 regs state that it must be from your "point of departure" though...