In his as in all politicians' lives, we cannot expect perfection (as UnnamedHarald has already said). However, the different parts of people's lives cannot be compartmentalized to the extent that a person's character in the privacy of their own home has no relation to their public character. As the saying goes, "truth will out," and a person's underlying character will eventually be revealed. If the shock of discovering someone's sordid history and current misbehaviors can cause him (or her) to be impeached, forced to resign, or otherwise disgraced while IN office (think of Bill Clinton, Richard Nixon, John Edwards, Mark Sanford), should not those same considerations prevent him (or her) from attaining office at all?
Personally, I would not vote for someone who we already know could not show fidelity to his wife and family when no one else was looking. Why should his public character be any different?