How to Find / finding Real Nuggets of Information in a Sea of Crap: 3 Crap Detector Tests You Should Use On the Internet
What Is a Crap Detector?
The term "crap detector" was probably coined by Ernest Hemingway, in an interview with "The Atlantic" magazine. The complete quote was:
"To invent out of knowledge means to produce inventions that are true. Every man should have a built-in automatic crap detector operating inside him. It also should have a manual drill and a crank handle in case the machine breaks down."
In other words, you should know what is a lie, and what is true. If you don't you should be able to find out with minimal research.
Howard Rheingold defined "crap" as "information tainted by ignorance, inept communication, or deliberate deception".
With the spamification of the Internet (otherwise known as Search Engine Optimization, or SEO) trying to 'game' the search engine results, simple search is no longer enough. Any one can produce a professional website in about 15 minutes by copying and pasting a lot of information and use existing templates. Hoax sites abound to make fun of the unwary (ever seen the "LASIK at Home" website?)
You must learn how to be a skeptic, and how to interpret the information available online. It is now more important than ever.
Internet and Proliferation of Crap
In the past, you can reasonably count on any non-fiction book in a library to be factual, since the books must be published, and must be popular enough or valued for a library to have it. It may be biased, but at least the author is backed by sources with enough money to get the book published and distributed. That barrier to entry means there's an automatic filter in place against "crap" getting too popular.
No such filter is available on the Internet. Crap is just as likely to be picked up and indexed as reputable sources. In fact, "search engine optimization" can be said to be techniques to make "crap" MORE relevant than real information. Thus, ANYTHING you find via a search engine must be filtered by your "crap detector".
To detect crap, you need to boil down the conclusion into the basics, preferably in the form of
a) someone said (source)
b) at a certain time
c) some evidence
Once you have distilled the crap down to the essence, you can try one of the three tests below.
The CRAP Test
In this case, CRAP is an acronym: it stands for Currency , Reliability, Authority, and Purpose/POV (Point of View)
The following are the criteria in the CRAP test
* Currency -
o How recent is the information?
o How recently has the website been updated?
o Is it current enough for your topic?
* Reliability -
o What kind of information is included in the resource?
o Is content of the resource primarily opinion? Is it balanced?
o Does the creator provide references or sources for data or quotations?
* Authority -
o Who is the creator or author?
o What are the credentials?
o Who is the publisher or sponsor?
o Are they reputable?
o What is the publisher's interest (if any) in this information?
o Are there advertisements on the website?
* Purpose/Point of View -
o Is this fact or opinion?
o Is it biased?
o Is the creator/author trying to sell you something?
The Bulls**t Test
Scott Berkun wrote an essay on how to detect bull**** by asking several questions.
1) How do you know what you know?
Most people simply take statements given to them at face value. Why is that so? If you are being fed a statement, you should ask for how did they know. If they are full of bull****, they will have a hard time explaining how they got to their conclusion. They will often sprout even MORE bull****, thus revealing their true nature. At the minimum, they will reveal that they did not think through their position, and thus, often reveal an admission that this is purely an opinion.
2) What is the Counter-Argument?
Any one who actually has studied all the facts available would have known both side of the argument, and what evidence support each one. If the guy who fed you the position have to think about the counter-argument, it is clear he came in with a bias. And if they claim there is no counter-argument, it's clear they are sprouting BS.
Other questions along the same vein:
- Who besides you shares this opinion?
- What are your biggest concerns, and what will you do to address them?
- What would need to change for you to have a different (opposite) opinion?
3) Why are you in such a hurry?
Bull****ers want you to make up your mind now, now, now! Scammers want you to commit now. "Ground level opportunity, join today!" Is always suspect. If there is an element of time pressure, you should suspect that someone is trying to "pull a fast one". Big decisions should never be hurried, and without additional consultation with professionals.
4) Why can't you speak plainly?
Bull****er often use a lot of fancy words, or rely on similar existing terminology. Such words are often lampooned in Dilbert and such cartoons. If you don't understand what they are saying, they are the one having problems, NOT YOU. Do NOT just nod and pretend you understand. Reply with:
- I refuse to accept this until I, or someone I trust, fully understands it.
- Explain this in simpler terms I can understand (repeat if necessary).
- Break this into pieces you can verify, prove, compare, or demonstrate for me.
- Are you trying to say “_____?” If so, can you speak plainly next time?
5) Why should I trust you?
Bull****ers are experts in making you agree to MORE than you should. How confident are you in them to trust them that much? It doesn't matter how confident they appear to be about their own position. It is how confident YOU are that counts. If they have no history with you worth your trust, then don't trust them. It really is that simple.
Trust can be delegated, but that would mean you need to find friends who are about as bull**** proof as you.
The FactChecked Test
FactChecked.org has a 5-step guide to check on facts. It is mainly meant for science and politics, but it works on all facts as well.
1) Keep an open mind.
All of us have a personal bias toward something, and we tend to believe something that fits within that bias, even though it was not backed up by facts or logic. Conversely, we tend to disbelieve information that does not fit within that bias, even if it's backed up by solid facts and logic. If you don't keep an open mind, you will end up believe in 'crap'.
Scammers take advantage of this bias often by gaining your confidence first, then use that to convince you to hand over money. That's why scam is also known as a "con game".
2) Ask the right questions.
Just because someone said so doesn't automatically make it true. They could be citing wrong information, outdated information, biased information, or even purely observation and opinions disguised as fact, or using logically fallacies instead logic. Who is that person doing the claim? Where did his/her information came from? How can it be validated / proven? Does the evidence actually prove the conclusion? Are there other evidence to consider?
Skepticism is asking questions and looking for answers in things we "believe" we know. Since you don't really know if what's on the Internet is really true, you should approach everything on the Internet with skepticism.
Just one source of information is never enough. In fact, even multiple sources can be faked by setting up false fronts, Internet personas, and more. Or there can be affiliations behind the scenes. What do others say about the information? When two, three or more reliable independent sources report the same facts or conclusions, that's far more likely to be "not crap". When multiple sources contradict each other, there is a deeper story for you.
When a ton of unknown "independent" sources all claim something, it's time to be suspicious, as it's likely "astro-turfed" (faked grass-root support).
4) Consider the source.
Direct information is worth more than first hand observation, which is worth more than second or third-hand observation. Why? Because observation can be influenced by bias, but direct information cannot be "spun" by the observer. Ever watch CSI? "The evidence does not lie." Physical evidence is always more reliable than eyewitness testimony, which can be biased. In most cases, second or third-hand observation, known as "hearsay", is not even allowed in court.
Something published in New York Times or the Wall Street Journal is much more reputable than something printed in a local newspaper with circulation of 5000.
In January 2011, a study on smartphone market share came out. Depending on the source, you get vastly different conclusions: Apple holds off Android (Pro-Apple source), Android surpasses Apple (Pro-Android source), Blackberry still leader in Enterprise (Pro-Blackberry source). You have to go to the data they cited to see their conclusions are "tainted".
5) Weigh the evidence.
Is the evidence from a real scientific study?
Are the evidence facts? Or are they just interpretations by unqualified individuals, in other words, "anecdotes"?
Does the conclusion follow logically from the evidence? Or is there some logical fallacies involved?
One of the simplest and most often used defense by a scam is "it's not a scam, it paid so-and-so." However, Ponzi scheme and pyramid scheme will pay SOME participants and use them as "bait" to attract even more victims. Thus, getting paid does NOT prove it's not a scam.
Some More Questions to Consider
Remember we divided up the statement into who, when, what, and why? Here's more crap detection:
Detecting Crap from the source
Who is the source? What is his/her qualification? Is it relevant to the subject being discussed? (NOTE: Lack of qualification does not automatically disqualify the source, and "ad hominem" attacks are against the qualification of the source, not the message/conclusion itself) You don't want a layman giving conclusions on international law, unless s/he cites facts and existing cases or such to back up his/her view.
Can the author be reached for comments, clarifications, and other interactions? Or is s/he a nameless entity behind a generic "feedback" form, or worse, no feedback at all?
What do other people think of the author? Keep in mind on-line reputation can be astro-turfed, i.e. faked, as it is easy to create online personas
Can the author's reputation be verified? How about qualifications? Keep in mind that lack of qualification or reputation is not necessarily a bad sign. Facts and citations speaks louder than any sort of fake reputation or fake qualification.
Is it published on a website? Is the site .edu or .gov? Those websites are generally more reliable than anything else available, as those only go to real educational institutions or government agencies. Except user webspaces. Anything else can be registered by anybody.
Who registered it? If "private registration" for a website is turned on, that's a MINUS for sure. It means the website owner does not wish to make his/her information public.
Does the author have a vested interest in the conclusion?
Detecting Crap from the Evidence
How much evidence is there to support the conclusion?
Is the evidence actually facts, or is it an opinion? If it's an opinion / conclusion, what is it based on?
Is the evidence current? Has anything changed since the publication of the evidence?
Are all the evidence given based on reliable sources or facts that can be verified?
What if you research the sources for the evidence? Do they lead to original information? Or was the source cited from some other place already "interpreted"?
Did other people link to the sources? Who are those people?
Do the sources appear to be unbiased or biased? Any disclosures about the sources?
Do the source have proper qualifications or reputations?
Are there multiple independent sources confirming the data / conclusion?
Detecting Crap from the Conclusion
Does the conclusion appeal to some basic needs, such as money, love, and so on? Are you aware of this appeal? Are you more disposed to believe or disbelieve the information, even if you have no facts pointing either way? Do you know why? Can you still keep an open mind knowing that?
Does the evidence lead logically to the conclusion, or is there some logically fallacies involved?
Is the conclusion trying to sell you something?
What is the counter-argument? Has the other side been considered at all?
Crap Detection Example
Recently Lifehacker did an "Idol White" run through. Let us use "Idol White" as a test case.
I searched on Google for Idol White and I found links "USFreeAds", "GoArticles", "SkinTreatmentGuide", "ArticleBase", "Squidoo", and so on.
USFreeAds is obviously an ad site, and worthless. In fact, WOT (Web of Trust) red-marked it as unreliable.
GoArticles and ArticleBase are "article repositories" where anyone can write anything and submit it to them to be archived and republished. Most of the stuff is outdated and no accuracy is assured.
Squidoo is similar to an article repository, in that writers create mini-webpages about a specific topic trying to compete for advertising dollars. Again, no accuracy is guaranteed. You must rely on the evidence given within the webpage.
Let's try SkinTreatmentGuide.
The first thing you notice about "SkinTreatmentGuide" link about Idol White is there is NO AUTHOR. The article was posted by "admin". Furthermore, comments are closed.
Second thing is it is filled with "FREE TRIAL OFFER!" No less than 5 times scattered throughout the article. And that link goes to a website with poor WOT reputation called MHLNK, which is clearly a link forwarder. MHLNK.com doesn't go anywhere, but searching DomainTools.com shows it belongs to HealthBuys.com and MarketHealth.com.
Third, endorsement allegedly from Kim Kardashian was right up on top with a long quote. That's an endorsement alright. When and where was this endorsement made? No idea.
Fourth, the picture of Kim Kardashian is swiped from Maxim magazine. The Maxim logo is clearly visible. Yet there is no attribution. Picture pirate!
Fifth, there's something about "special ingredients" but basically it's a bleach that whitens stains on your teeth. It claims to not damage your enamel, but there are no details.
Sixth, the author then relates personal anecdotes about preferring home treatment to visit to the dentist. Irrelevant.
So, how did the article do on the CRAP test?
Currency: FAIL there is no "last updated" date on this page, nor for any of the "endorsement" cited.
Reliability: FAIL. there are virtually no facts in the entire piece. Even the picture was stolen without attribution.
Authority: FAIL alleged personal testimony and alleged celebrity endorsement with zero reliability. Publisher is a merchant.
Purpose: FAIL it is a sales site, not a guide at all, just opinions.
Thus, this website failed the CRAP test.
A little more research shows that the main idolwhite.com has an affiliate program (which I will NOT link to), which points back to... (drum roll) Markethealth.com (hey, didn't we just saw that name? Yep... MHLNK.com That's why they all are singing praises... They get a cut from every kit sold.
With Internet and easy access to information, it is easier than ever to access crap, and you need to have your crap detector tuned up to make sure you did not get deceived by crap.
More about Scam and Crap Detection
- How Does a Scam Mess with Your Head: MLM, eWallets, ...
Do you know the three factors used in cost/benefit equation? It's cost, benefit (also called gain or value), and risk. Learn how a real scam mess with your estimate of those three factors.
- How to Research a Network Marketing Opportunity with...
Wondering if a Network Marketing Opportunity is good for you? Here are the topics you should to research on, and what sort of answers are good, bad, or outright ugly.