I comment on hubs where the author has obviously put in a lot of effort, particularly if they go to the trouble of using original photos. I leave a comment on hubs by hubbers who pay me the courtesy of leaving comments on my hubs. I also make a point of commenting on a new hubber's hub, even if it is to be encouraging and perhaps make a helpful suggestion.
I always vote up if a hub seems worthy, but I don't think I have ever voted down. I simply don't comment or vote on hubs that cause me concern.
Sometimes I come across a hub that has been up for years but still has no comment. It may well have lots of traffic and be making money for the hubber, but I try to leave a comment there because hubs look kind of bare without any comments at all. :)