When someone stares at you it gives you the feeling of violation because that person is scanning you and they could be doing it for a reason other than wanting to harm you. For the most part the human eyes can also tell if someone is scared or just plain out of their mind. Some people have learned to use this as a way to comply fear in others.
For one reason why most of the time I wear sunglasses, I know this is going to sound bad, but with shades over your eyes people cannot tell where you are looking, who you are looking at and if they stare at you and you look back you can make them uneasy because they cannot see your eyes and what you might be thinking. This is also the big reason why law enforcement wear sunglasses because if they make a traffic stop or a bust, it is hard for the other person to intimidate them through the sunglasses and the fear that they might cause is reflected back on them.