Do it!
While there's maybe a tiny bit of stigma for guys doing what you're doing, who cares. The only thing that's going to matter once you make it through training is are you safe and nice to fly with. I found most of the guys making the change like you to have a great attitude and be humble and hard working. A lot of the issue that guys have with the later in life career changers is that they feel that they're not on the same team because career changers tend to be willing to defend their company even when they're mistreating their pilot group and it doesn't sit very will with the guys depending on this career for their financial wellbeing. Be humble about your financial position and try to identify with the gripes people have with the company and you won't have any problems.
I would much prefer to fly with someone like you who is enthusiastic about what we're doing and positive than someone who is miserable and has a poor attitude. Everyone keeps hammering how poor you're going to be and how it would make more financial sense to keep doing what you're doing and they're probably right, but you only get this one life and you don't have too many more years to do what you want career wise.
It seems to me like you're probably somewhat wealthy and are just too polite to say it and a lot of the guys aren't picking up on that. It's cool to fly a jet and make the mission happen. There have been points in my career when I almost couldn't take being poor anymore and got tired of my company dumping BS on me, but I'm glad I chose what I chose. Skywest doesn't have a lot of patience for someone that isn't putting in the work in training, but as long as you're giving it your legit best effort, they're not unreasonable at all. IF you go there, make friends with your classmates and spend time quizzing each other. That was the way my class did it there years ago and everyone got through.
Before anyone asks, if it matters, my career path was CFI>Night freight>Trans States>Skywest>Continental>United.