Great moral question. There is a very deep confusion over sexuality. We are all sexual beings, but so many hide from sex. Until one can admit sexual desire to oneself, there will always be that bias based on fear.
That said, I feel we are all born with, and develop over time, our particular talents. If one of yours is physical appeal, it is just as much a part of you as intelligence, diplomacy, or compassion. If there are ways you feel morally okay with using it, why not? You are the one who gets to live with the consequences, good or bad.