I had cataract surgery on one eye about eight weeks ago, with the other scheduled for a couple of weeks later but has been postponed twice for various reasons.
I suggest you ask your doctor what complications could developed and could they be aggravated by the delay to the other hospital or by the actual trip. If the answer is yes to either, I would consider being closer to the specialist.
As far as reviews go, I put little faith in them. All doctors do not have a great bedside manner and that get them a bad rating. Some seem to be in a rush when explaining things because their patient load is so high and that gets them a bad online review. If you have concerns about your doctor, check with your state Medical Society to see if there are any outstanding sanctions, complaints, etc. If they will not talk to you, go to the courthouse and have the civil suite section look up his name to see if there are any outstanding lawsuits. If you have been seeing him for years and trust him, go with that.
Your problem is geography. If you were in a major mainland city, the needed specialist might be close by, but by being in Hawaii, physicians are scattered over a large area.
As far as your brother possibly over-reacting--he just want the best care for his sister. You cannot fault him for that. Ask some more questions. Then decide.