Things that Internet hide from you

For each site that you can visit, there are at least 400 others who can not access. They exist, they are there, but they are invisible. They are trapped in a digital black hole larger than the Internet itself. Every time you interact with a friend on social networks, several others are ignored and have the messages buried in a huge cemetery online. And when you do a Google search, you do not receive the actual results - but a dressed up version, previously modified according to secret criteria. Yes, all this is true - and it's no big conspiracy. It happens every day without you noticing. Get your Indiana Jones hat and let's explore the lost web.

First stop: Facebook. When you access your account, the first screen that appears is called the News Feed - that list with the latest comments and links posted by your friends. This page is edited by Facebook, and only includes messages from people with whom you interact more. You can override this edition - just click on the "Most Recent" and Facebook show, in chronological order, all messages from all your contacts. The problem is that this lotará your feed waste, with loads of irrelevant updates (what matters is that his former colleague that you have not seen for years changed or girlfriend is going on vacation?). Conclusion: The content editing done by robots Facebook is good for you. Except when it is not.

The American writer Eli Pariser supports the Democratic Party, Barack Obama, but also has friends who vote in the Republican party. From one day to the other, Pariser noted that Republicans are gone from your Facebook. He was surprised and tweak the site configuration, thinking that he had done something wrong. Nothing: the robots is that they had decided that he did not have the right friends. Facebook took a political-ideological decision and imposed the user. "Customizing the internet reinforces stereotypes and beliefs that the person already has," says Viktor Mayer-Schoenberger, Internet researcher at the University of Oxford.

In other cases, robots Facebook can cause family conflicts. It happened with the systems analyst Rodolfo Marques. His brother, Diego, is a musician and a clip posted on Facebook. But Rodolfo nor heard - just because, as he used to not talk to Diogo online, robots deduced that it was not an important person. "I thought he did not like the video," says Diogo.

Google also handles what you see on the Internet: each person can get a different result for the same search. The seeker uses criteria such as the history of pages that the person visited, the place where it is and even the browser you use. In all, Google invests more than 100 variables (they are kept secret so that other search engines do not copy) to customize the results.

And that has profound consequences. In an experiment conducted by the University of London, the scientists created three fictional characters, who were baptized Immanuel Kant, Friedrich Nietzsche and Foucault - 3 of the greatest philosophers of all time. Each character used Google to do research on the books themselves. The intention was to induce the site to trace a psychological profile of each of them. It worked. After a few days, Google began to generate completely different results for the same search. And it happens to everyone, every day.

"Users can disable personalization," it is argued Kumiko Hidaka, global manager of Facebook. Google also allows it. But what the search engines conceal the user is only part of the problem. Another, perhaps even more, is what they themselves can see.

When you do a search on Google, it comes out not covering the entire internet looking for the information you want. It would be very time consuming. Google query your Index, a collection of copies of 46 billion web pages.

It is an enormity. But it is much less than it is out there. Nothing is posted on Facebook, which has 750 million users and is the largest social network of all time, it appears in Google results. It is estimated that Google and other search engines are able to access only 0.2% of all information actually contained in the network. All other pages, no one knows exactly how many there are and where they are, form the so-called deep web - the deep web. These hidden sites remain hidden for several reasons. If a page requires signature and is password protected (such as newspaper sites and magazines), Google robots crawlers can not get into it, not the copy to the Index. Facebook blocks the entry of Google bots, because they do not want your content to appear in the search engine (which could steal audience from Facebook). There are also online databases that are not in HTML - language that Google understands.

If Google can brave the deep web, life will be very different. There will be more specialized websites for hotels, real estate, airline tickets etc. You need not enter the recipe page to find out if released the tax refund - just type your Social Security number in Google - or access the health plan's website to see a doctor. All this, and all the rest, will be on Google itself.

He already has a team of researchers trying to explore this lost internet. The team is led by Alon Halevy, a computer scientist at the University of Washington. "We have developed software that can find the information more intelligently," says Halevy. As? One of the main tactics of the robots Google's kick.

Yes. When is a database that does not understand, the robot begins to search multiple terms: "apartment", "convertible" and "Lycra", for example. If the "flat" word is present, it is because that site contains information about real estate. If "convertible" work, it is because it is a table with car prices. And so on. Knowing what it is, Google can add the information to your Index - and put them within the reach of everyone.

The problem is that the information is scattered around the web chaotically, and find them is like finding a needle in a haystack. "We need a more efficient tracker," explains the Brazilian Juliana Freire, a professor at the University of Utah. She is the creator of DeepPeep, a project that aims to make accessible all the internet databases.
So lost or hidden information, the Internet is still far from reaching its full potential. It can, must and will get much better. While there is, make a habit of going beyond their favorite sites and set aside some time to explore the corners of the internet you do not know. If Nietzsche, Foucault and Kant were alive, they certainly would.