A couple things come to mind.
More times than not, people set their expectations too high. Especially in themselves. When they don't reach those expectations, they perceive it as failure.
Also, many people want to get from A to Z without dealing with all the letters in between. They want to go from being the company janitor to the C.E.O. without the process of working their way up. This is like getting to the finish line without running the race. This isn't failure. This is not being willing to do the work it takes to be successful.
And then there is the fear of failure, so they don't even try. You can't fail at something you didn't attempt, but some perceive it that way.
And finally, there is the lack of patience. Most time, success takes time to achieve. If you aren't willing to stick things out and quit too soon, you fail.