I think it is a human problem because people are more comfortable with others that are similar to themselves, which becomes almost a sum game, i.e. "if (fill in the blank)s are given more rights, that means less rights for me". This goes beyond race and encompasses sex, religion (or lack of), economics (class), ethnicity, and even regions. The issue comes when a particular group uses this acknowledgement of difference or is perceived as using this difference to dominate another group. This can go for the minority or the majority.
I am a White Christian straight healthy American male who minds my own business and tries to be helpful to every person I meet, yet because I am in the perceived "dominate" groups, I am labelled the bad guy by default. This does not sit well for me, so I do hold some animosity and possibly have been pushed to be a little more biased for self-preservation.This is just a small view into the race problem in the US by my perspective.