I have never unfollowed a hubber once I've committed myself to follow them. The only thing which would make me think about it is if I thought they were treating either my fellow hubbers or myself unfairly or rude. Some writers don't have as much time to be here as others do. If they still have hubs posted here, they could be back even if they have been gone a spell.
I never follow someone if I am not interested in reading their hubs. I never follow someone just to get them to read and follow me. Either they like my hubs or they don't. I expect the same from them. I know we are all here to learn. I know all my hubs aren't great, but I try just like everyone else does here. Every now and then I do come up with a good one.
I see some people follow hundreds and hundreds of fellow hubbers. I don't see how they keep up with this many. I tend to follow only those I'm sincere about wanting to read their hubs. As time goes on I'm sure I will follow more.