It's fine, I know people are busy. I know I have been guilty of approving a comment on the fly and then forgetting to go back and answer it. I didn't realize in my early days here that it was expected that you would reply to each and every comment.
I guess I would feel a little snubbed if someone answered everyone's comment except mine, but really even that is probably just because they overlooked it.
I comment because I want to respond to something I read, anything I get in return is just a bonus.