In my opinion, it's the amount of interaction that can get us more followers. Insightful answers to questions, participating in the forums, visiting and commenting on hubs make people aware of our existence.
Those who like our comments and answers may get curious enough to visit our profile page and read our hubs. Some may follow us if they really like our work.
You are right about one thing, the number of hubs really don't matter. Those who start to follow us cannot go through all our hubs, in fact some follow us solely based on comments or answers.
The other thing is, if someone is following us and commenting on our answers or hubs, it's good to follow them back and comment on their posts as well.