Depends what the "it" is. I'm assuming you mean something tangible such as a prestige job, a qualification, a fortune in money or real estate, or simply fame.
Generally the ACHIEVING is the long journey, and people who have set out on a long journey to achieve something do not like to suddenly back-track or find another road because they've perhaps discovered they're on the wrong one after all this time.
So what they 'feel' they have to lose is the time and effort they have put in. This includes, perhaps, the skills they've learned, the habits they've formed, and even their social network.
So, in effect, what we ' have to lose' is generally simply a thing of the mind. But don't under estimate 'things of the mind.' They are the things which generally motivate us and give meaning to our lives.