I don't think I would not pick to speak or deal with someone based on the color of their teeth. There are many reasons someone could have horrible teeth. My husband has an extreme fear of needles. It takes medicating him just to get him to the dentist and then they have to sedate him even for cleanings. Some people can not afford to do all that. Also many health conditions cause bad teeth.
As somebody who worked in a dentists things are not as easy as one would think.
The yellow/brown that you see is behind the enamel and cannot be removed by ordinary brushing. So I would advise one to not be mean to people who may not have the funds, or the gum condition may not be good enough, to go for 'whitening' which may not be permanent anyway .