Lots of thoughts here, somewhat connected: Power is systemic. A system permits power to go to someone's head. The l'il "peons" don't speak up in time to keep power in check.
Someone mentioned control. That's key too. A person might not have faith that another person would exercise power fairly, so they want to retain control over their own situation by staying at the top. It's a form of fear and self-preservation.
Some men have feared women acquiring personal power because they thought they'd suddenly be trampled just as women were/are. For example, this was expressed in suffrage-era cartoons of women smoking cigars and treating men like dirt. Some didn't imagine that maybe power could take another form. Power WITH instead of power OVER.
Power can be wielded in a synergistic manner that benefits everyone. A big problem is that newcomers to power might not have adequately good models of power to follow.