Just re-read the TSP website about loans:

"he interest rate you pay for the life of the loan is the latest available interest rate for the G Fund at the time your application is processed. The interest you pay on the loan will go into your TSP account, along with repayments of the loan principal. Visit Current Information on this Web site or call the ThriftLine to find out the current interest rate for TSP loans. Also, if you use the Loan Calculator on this Web site, the calculator will use the current rate automatically."

So, the interest goes back into your account but it is taxed by the Fed.