Hi there! I haven't actually heard that if you follow too many topics your score goes down? I follow quite a number of topics and yes in the main it does tend to be subjects that I know and enjoy writing about frequently.
However, I also follow other topics on Hub Pages that I know very little about and never write about. The reason is that I learn a lot of useful things from these hubs and the hubbers. For example things like SEO, Keywords and all that kind of stuff - which is an absolute mystery to me for most of the time - but I have learned quite a lot by reading the work of others.
I would honestly just go ahead and follow whatever topics you enjoy and/or you feel would be useful to you.