It's a barrier that is slowly starting to break down now, but you are right; in most cases, men are still heavily frowned on for wearing dresses. I think it has to do with the subconscious attitude that many misogynist men have toward women. It goes like this: "It's perfectly fine for a woman to imitate a man, because a man is worthy of imitation. Any man who imitates a woman in his style of dress, however, is clearly less of a man." I do hope that this sort of primitive thinking will vanish in time; I know plenty of perfectly normal men (and no, not just homosexuals, though some of them, too) who enjoy wearing dresses or skirts because they allow greater freedom of movement, and because they draw attention, courtesy of the taboo surrounding men dressing that way.