I don't have to worry about following anyone too much because I'm rarely spoken to on Hubpages. I guess it's because I had an account here for six months and I only posted four hubs. I'm extremely busy with running an events website and other writing jobs so no I don't have a lot of time to post as often as some people can. Personally, I don't think I would unfollow someone for inactivity because there is life outside of Hubpages and people can get tied up.
I would unfollow someone whom I felt made me feel out of place by only posting comments on the hubs of those who have the higher hubscore, the most accolades and the most followers. I try to treat people who take the time to read what I write with respect so I expect that in return. Another reason, I would unfollow a hubber is because sometimes people develop circles on websites like Hubpages and they act like they don't want to be bothered by "outsiders" so the only thing I can do is unfollow.