We should by all means challenge books if we don't agree with their content; the whole point of books is for people to have reactions to them and to express their reactions. The question of whether a book should be banned altogether is another matter. People arguably should have the right to write and read what they choose.
The question of when a school librarian decides that school students should not be exposed to a certain work of literature is perhaps one of the most controversial as it is very subjective and dependent on the value systems of that community.
At what level do we decide that objectionable views or language e.g. racism or sexism are outweighed by literary merit?
If we teach children to be discerning and questioning readers, they should be better able to be aware of and evaluate possible bias rather than being passively influenced.
For example, as a child, I was allowed to read Enid Blyton but it was explained to me that I would find racist and sexist views in her books and should bear those prejudices in mind.