They do, because they cannot see it and usually do not understand it. A person in a wheel chair has an obvious handicap. A bipolar person will look and act normal most of the time and when he doesn't most people assume he is on drugs.
I am speech impaired--not terribly--but impaired. People do not understand. Keep my mouth shut, no one notices. If I mispronounce a few words, everyone turns into an instant therapist.
Also, today many people blame all mental illness on drug abuse, when that is not the case.