If you act like an adult and give them a “thanks but no thanks” you will be in good shape. To be honest, they will probably be shocked and extremely appreciative that you let them know instead of just no-showing.