Some types are in style right now. Justin Timberlake & co made them popular in an ironic way with the hipster crowd in the last decade. Dark waistcoats worn with ties are also popular. Men are very ego-centric and if they see someone wearing a style that they do not understand, they tend to get angry because it makes them feel out of touch. Women do not understand men's fashion. On the other hand, if it's causing that much of a stir, I wouldn't do it anymore unless you like the attention.