I'm not sure word conspiracy is the word I'd use as much as slander or liable. If the person writing the letters is still employed by your former employer, and there is no derogatory record of service on your part, I think I'd ask the advice of a lawyer on this one.

Good Luck