Doing some pruning? Probably not a bad idea.
I think I can answer half of this.
For the active part, yes you can filter.
Go to your account/hub statistics page. Click on Following in the links at the top. Then click on Hubbers You Follow in the links on the left side.
You shoulc now be looking at a list of everyone you are following. At the top of this it will say in bold letters Hubbers You Follow.
Beneath this it will allow you two sort options.
The first option is Hubbers which I believe is an alphabetical listing
The second option is recent hubs which sorts by the date of the last hub published. This is the one I think you want.
If you sort by recent hubs the most recently published hubs will be at the top. This means that you can scroll all the way to the bottom and see who hasn't posted in a year or two.
I'm not sure if there is a way to filter who you're following based on their subjects. At least if there is I haven't found it yet.