The cash is not earning anything but the loan is costing you 6%. If you're worried about having a cash reserve for a rainy day, get a HELOC.

Pay off the loan unless you think investing the 100K will net you more than 6% (in rough terms). There's no point in paying the interest on the loan if you don't have to.