There is no "common interest" here. If you happened to be going on a flight and he wanted to hop along and you agree to a pro rata, then that's okay. But in your scenario, he initiated the flight, and you are being compensated. The Chief Counsel has a very broad definition of "compenstaion". There are also plenty of legal interpretations available to you from their office.

Refrain from doing anything that involves an exchange of money or anything of value (including flight time) with people/businesses while holding a private pilot certificate.