Quite honestly I just avoid talking to passengers. In today's 'know-it-all' age people already have their minds set that pilots make some bank, but we all know that's not true. One thing that bothers me is when you do spit out the $40K number, some seniors and baby boomers think back to the 70's and 80's economy when $40K was decent money. They have a hard time understanding that in 2012 it isn't much money.
Save yourself some heartache and keep to yourself. Like someone else said start reading more books or pick up a new hobby. It'll make the trip go by quicker and save you some hassle.