Why wouldn't it be fair? If your lover/spouse has died, (s)he cannot be hurt by your actions on earth. And if said lover/spouse would be hurt by the possibility of you finding someone new in the event of your death, (s)he probably isn't worth having around anyone. How selfish is it to want to condemn the person that you love to unhappiness in order to assuage your own need for self-gratification?
Even in the event of a bad breakup where you were the one who acted badly (cheating, ect.), continually seeking to punish yourself over it won't help anyone. Learn, and more on. It's all that anyone can do.