I think many people are under the impression that fantasizing is benign, but it can lead to problems down the line, depending upon the nature of the fantasy, its endurance, and its frequency.
In general, fantasies are an indication that something is missing or something in the relationship is going unfulfilled. Every now and then, one of the parties may fall deeper into "fantasy land" or begin to resent the other party for not being able to fulfill a particular fantasy or they are not willing to.
The danger is when the fantasy starts to take on a life of its own. And, if the other party is not able to partake in the fantasy...the projector of the fantasy may find someone who can. In any regard, most fantasies, if left unchecked, can lead to dissatisfaction and alienation within the relationship. It becomes imperative, therefore, to be open and honest with your mate.