I agree Timbo and good response. I linked to the article mostly for the stats.
I think the decline in GA is even more than that -- most of my colleagues can easily afford flying but only a very few do, even fewer than previous where doctor-pilots were common (and, incorrectly or correctly, regarded as somewhat dangerous.) A lot of them actually express surprising opinions like RhinoParrot and think that flying is incredibly complex or dangerous and would rather play with their smartphone in their precious spare time.
Studying the GA accident rate, it remains higher than it should / could, but not incredibly high. Also, there are easy things that one can do to mitigate their own personal risk.
Money remains a big factor. I remember one of my schoolteachers used to take a few of us grade-schoolers up in his Cessna if you got an A. It was wild and I still remember it. I doubt many schoolteachers can fly now days (and perhaps liability fears would prevent a joyride with random kids).