We don't define crew positions by what you don't do (PNF), but we do define crew positions by what you do (PM). As a PNF, I also am not serving drinks to the passengers, but you don't think of yourself as a PNFA, pilot not flight attending.
As a pilot, you are always "flying", or in better terms, responsible and in control of the aircraft, you might not be the one who is physically maneuvering it.
Perhaps if we had more PMs instead of Pilots Not Flying, we wouldn't have some of these stupid accidents.