Not always. Sometimes people observe a shift in behavior towards them and suspect there is something going on or not quite right. If and when they address it with their mate they are put off or told they are paranoid that only raises more suspicion.
Generally what happens is we tend to shower a new person with loads of attention, spend hours together, have lots of sex, respond to their phone calls, emails, and text messages fairly quickly, make weekend plans on Monday or during the prior the weekend...etc
When a partner suddenly becomes "less available" or less enthusiastic about getting together the other person suspects they are pulling away for some reason. The easiest thing to do is assume they are seeing someone else.
If their mate refuses to offer any clear explanations as to why the sudden change in behavior or reassurances that everything is as wonderful as it had been then things can get rocky.
There are instances however where a person has been cheated on in the past and they see the same signs.
Accusing someone of cheating has no bearing on your actions anymore than accusing someone of lying or stealing.
The accusation stems from a belief you are having about a particular situation.