I would, because my happiness should matter to those around me. If those I'm near tell me that anything I can dream up isn't worth it, they don't deserve me.
And I would leave, and I have left, relationships (not just romantics ones) where the other did not respect anything that I did or cared about how I felt.
Of course, I care about the happiness of those around me, and I certainly value their opinions about people, things, and ideas. If they had valid reasons for not wanting me to pursue something, I would definitely look into it. If you can't trust those around you, you need new friends.
However, it would not be the basis of my rejection of anyone or anything. That decision is mine and mine alone, and I will make it as it pertains to my well being and happiness.
"I am only resolved to act in a manner that will ensure my own happiness without reference to you or to anyone so wholly unconnected with me." - Elizabeth Bennett to Lady Catherine deBurg, Jane Austen's Pride and Prejudice