Yes, often. Many times a good reason that an answer has been voted down is that it is totally stupid or inaccurate.
But many questions are themselves stupid, particularly the relationship, motherhood, and religious questions, so the hidden answers may be politically incorrect. The weenies and crybabies and religious wackos will have voted them down. By looking at the question, I can usually tell that the answers voted down are probably worth reading.
Many of my answers have been voted down. So take a tip -- read them anyway. :-)