From the Bible-
"What causes fights and quarrels among you? Don't they come from your desires that battle within you? You want something but don't get it. You kill and covet, but you cannot have what you want. You quarrel and fight. You do not have, because you do not ask God. When you ask, you do not receive, because you ask with wrong motives, that you may spend what you get on your pleasures"
Religion isn't a "thing"- like a park bench or a cherry pie- it is a belief system. The person behind the religion is the responsible one, for he/she is responsible for what they choose to believe... and then what they do "in the name" of it. Religion wasn't holding the gun, knife, or bomb- a human hand was.
The logical answer to your question, based on your conclusion, would be to "outcast" the people responsible...but that doesn't seem very practical. We probably all would be "outcasted" at some point.
For what is war but an escalated fight. How many of us are guilty of fighting at some point, on some level in our life. How many have we "killed"-- maybe not physically but perhaps emotionally or spiritually?
Think of how many people fight with those they "love".
Religion is not the guilty party.