I was a bit confused too when they first came out with this feature. What I have noticed is that this feature is specifically designed for use when you have multiple comments on a single Hub that you want to reply to. You click "reply", type in your comment, then it will disappear once you publish your reply.
I find it useful because sometimes I have to walk away from my desk and I know exactly who I last replied to because of this feature. Two other things I noticed were that when you add a comment in the comment box instead of hitting "reply" the reply button does not disappear. The last thing I noticed is that even though the reply buttons disappear as you are commenting through them, once you close the page and reopen it, the reply buttons show up again. I am not sure why the team decided to do this and I agree that it can be a bit confusing. I hope this explains it a little bit more for you!