Pastors are (human) too!
They put their pants on one leg at a time and deal with the same temptations and egotistical desires as everyone else.
Those who believe in God would probably be better off dealing with him direct as opposed to emotionally investing in a minster or organized religion.
Too many people are more invested in ministers than their faith.
There will always be church scandals, ministers committing crimes, church politics, and immoral behavior going on. Simplify is the key.
You don't need a "middle man" to connect with God.