My husband was a Lutheran pastor for years and had regular office hours. It was nice because when people wanted to chat with him or needed him, they could just drop in. But office hours, whatever they are, is only a small part of his time with his parishioners. They also would call him at home during all hours, but that was his job and if they needed him, he went.
A pastor must be available to his members as all times of the day or night because people's crises don't always fall during office hours. I remember many Christmas Eves where he had to go to someone's bedside or home due to their illness and on one occasion, death.