Many (most) pilots don't want to remove the age restriction.

At some point you're just too damn old and the older you get the harder it becomes to certify your fitness... you'd need more comprehensive (invasive) testing, and more frequently. If that sort of testing gets established, do you think it will be limited to age 65+? Not a chance, we'll all be subject to increased scrutiny and increased costs.

Even the most elaborate medical screening system would not prevent rising incapacitation rates beyond some age. Old folks are just prone to that, and that includes "soft" incapacitation where you're not firing on all cylinders.