In my opinion, sex appeal is skin deep, it's a matter of personal charisma and vibes that one glows. Sometimes my wife gets surprised when we are watching TV and I would say about a so-so pretty woman that I find her more attractive than some glamorous movie queen - I won't mention any names, but I could think of a half dozen off hand right now.
The same would go for men. Being an old timer I remember how girls used to be crazy about someone like Jean-Paul Belmondo, with no classically handsome facial features, while not caring that much about those "pretty boys" of the big screen - and there were quite a few in those times. Jean-Paul had that something spontaneous and boyish, vulnerable about him, although not a sissy but a manly kind.
Now, let's not talk about extremes here, and beauty will always be in the eyes of beholder. So we can't discuss "erotic tastes", while statistically it's probably true that a woman in a great shape will turn more heads than one out of shape. However, a most beautiful woman can be a turnoff if she doesn't have an attractive personality.