There is a good psychology argument for this: people are three times as likely to act on negative emotions as they are to act on positive emotions.
Meaning, the actually negative audience is overly represented in a vote-up, vote-down system.
However, I think the vote down should remain. When I give vote downs, it is for one of two reasons:
1. Extremely poor quality content: short, auto-translated from another language, etc. Or articles posing as experts, but with bad information or ineptitude.
2. Things that will harm the reader if they get into it ( e.g. witchcraft, etc)
So, there are good reasons for the vote down.
What I would like, though, is to know WHO it is that has flagged a hub. Twice, I have had someone start flaming me... and then had a hub flagged. I would like to have a system in which a false-flagger can be punished, to protect the innocent. One of the flagged hubs was booming in daily views. While it was offline for three days, it dropped, and never recovered.