I think that a good part of the time people most often tend to be most passionate about something (of things) that they're naturally good at. Being naturally good at something isn't always enough for success, because sometimes people are "naturally good at" but still not quite skilled/polished enough to really become successful. I'd use, say, learning to play an instrument or drawing as examples. People often have natural leanings and even natural talent, but some things can require additional training. Sometimes a person is SO good at something he doesn't really need training, but I don't think that kind of ability is anywhere near as common as the kind that people have who are "naturally good" but, maybe, not "phenomenal". :)
Of course, someone with only average talent can be trained and become reasonably (even very) successful; but those who are most successful are most often (I think) either naturals-with-training or else those rare "phenomenals-without-training" (provided its an area like medicine, which requires training in order to work at it in the first place). I think, too, that it can depend on what the area/field is when it comes to how important natural ability is. Using medicine as an example, I really think there are plenty of perfectly successful but not particularly gifted physicians. In a field like that, I having natural ability is very important if a person is to be among the best in his field.
There are other factors in success, though: How confident a person is in dealing with other people (because there can be a lot of "scrapping" required to "take one's show on the road". Whether a person's circumstances/life support, or hinder, his chances of success are also factors.
I suppose my answer to your question is that passion alone can certainly bring a certain amount of success. Giftedness, itself, tends to naturally combine with passion and lead to an even better degree of success. Natural talent without interest in the area generally leads to "having a nifty skill that comes in handy in one's other, preferred, field". :)