Hold your head high and smile!
Let others think what they want of you. If you respond in any way shape or form, you are validating whatever gossip these people are spreading.
Whatever the gossip is, if you carry on about your business the way you always have with a smile on your face and your head held high, sure they'll judge you but they will get bored with this endless gossiping.
If you say or do something about it, not only do you validate what they say, but you give the gossipers more to talk about. Gossipers tend to twst stories and mince words, so correcting them or giving the gossipers the whole story will make a bigger problem for you.
Just smile, nod politely and let them talk, obviously they have nothing better to do with their meager, pathetic little lives. :)