Not unless they cross the legal definition of libel. From what you stated, their behavior looks bad on you because you 'vouched' for them.
It's a mistake in judgement on your part but not a cause for legal action.
If the co-worker had maliciously said or implied something untrue about you that caused you to lose a tangible item such as a promition or job, you might (and I mean might) be able to sue for libel but the rules there are pretty strict.
This is a no-go as you've described it.