1. There's no way of knowing this. If they are a million light years away right now, they might as well not exist. If they died out a million years ago or are still in hunter / gatherer phase now, we'll never meet.
2. Life could be based on methane or nitrogen, but then any intelligent life from those biochemistries would have no interest in Earth. Then you don't get contact.
3. Larry Niven wrote a great piece that water worlds are likely abundant - and intelligent dolphins and octopi will never build space ships or make contact. And they could even be our neighbors and we might never know.
4. If the intelligent aliens are like koalas, limited to a small area of their planet, they'll exist but never build the technological base to travel.
5. If there is a tendency to study the worlds within, as we're seeing with virtual reality sapping interest in actual space travel, that alone could explain why many aliens stayed home.