Put yourself in the shoes of your future dream employer. You have a candidate who left a company and walked away from a signed agreement, then threatened legal action to avoid paying it back. Would you hire said person?
Is it likely that they’d know you that you did that? Probably not, but I guarantee it would destroy your chances if it did, so why not just pay back what you owe?