I'm no financial or management expert, but I'd say that the lack of one collective bargining unit for all pilots is what really created this issue.

Although there is a "national" union that serves as the voice of pilots with regards to political matters, the national union plays very little role in the negotiations conducted on the local level at each airline. As such there has been no "standard of pay" set across the industry like there is with regards to other "union shops".

Since each individual company (and it's respective pilot group) negotiates their own contract, the interest of the pilots on each property is their focus. As such, the contracts are written to reward the employees that have been there the longest and commit to staying beyond their probationary year.

Of course that's a very simplistic reason, but that combined with the various reasons for unionization and other factors that affect the management/labor relationship at a company that employs a large group of employees has resulted in the seniority and reward system of benefits.

Outside of the airline/unionized world, most corporate and charter operators that are non-union still operate on a "pay based on experience" system.

Simply put- It is what it is... but only because of how things started out way back when ALPA started.