I know the feeling. As a Mormon growing up outside of Utah, I had school friends whose parents wouldn't let them play with me. After I joined the AF and was stationed in TX/AL/FL, I had neighbors refuse to acknowledge my existence when they found out where I went to church. Wouldn't return my wave/smile/nod. Others gave me pamphlets or straight up told me I was going to hell. Whatevs, that's human nature and humans suck sometimes. Anyhow, Utah has probably changed a bit in the years (decades?) since you lived there. The last mayor of SLC was a lesbian Democrat - so that should tell you something.

To the OP, Mormons are now a minority in Salt Lake County - definitely true in metro SLC or Park City, if you can afford it. Up by Hill AFB (Layton/Ogden/Clearfield) and there's a huge integrated military community that's been established for generations - I've never spoken to an Air Force peep who has been stationed at Hill and felt ostracized by the locals - they all loved it. Like others have said, avoid West Valley City, Magna, Taylorsville, and other parts of the northwest portion of the SL Valley (gang activity, crime, rough schools). Areas around the South Valley have been growing like a wildfire with all the tech companies moving in (Draper, Bluffdale, Herriman, Daybreak, and spilling into northern Utah Valley). Lots of out-of-state transplants. Regardless of where you settle, there's always going to be some sort of LDS influence. I'd say the key to not being miserable is just accepting that there's a different cultural aspect and not getting offended when it doesn't align with your background or beliefs. For example, if you throw a birthday party for your kid on a Sunday morning, expect that a lot of their little Mormon friends are not going to show. Not because they're ostracizing you, but because to Mormon families "the Sabbath" is a big thing. You wouldn't bring bacon jalapeño poppers to a Jewish friend's dinner party, and likewise, you'll get some awkwardly raised eyebrows if you bring a 24 pack of Natty Light to your Mormon neighbor's BBQ. Feel free to DM me if you have other questions or even want to chat on the phone.