What is anecdotal evidence?
Case 1. Two unrelated people living on the same street are diagnosed with an extremely rare type of leukaemia. Their families appear on a live TV talk show and blame the power pylons above their homes. The host opens up the phone lines. Three people call in who have leukaemia. All three live under power pylons.
Case 2. A young man lies desperately ill in intensive care. Every treatment the doctors try is unsuccessful and his condition worsens every day. His family turns to unconventional treatments and over the objections of his doctors he is given an extremely high dose of a vitamin. Overnight his condition improves and in a few days he is out of intensive care and well on the road to recovery. This case is widely reported in the media.
These two accounts are examples of anecdotal evidence. What is anecdotal evidence? I once read someone define as "evidence not collected by scientists". This is not true at all. Whether evidence can be considered "anecdotal" has nothing to do with who collected it or who is relating it but rather the nature of the evidence itself. Anecdotal evidence, as the name suggests, consists of an anecdote or a descriptive story about an event or phenomenon. Another name for this type of evidence is a testimonial. Anecdotal evidence usually has one or more of the following features: it is unplanned, it is poorly-documented, it lacks appropriate controls and it lacks sufficient numbers of subjects for statistical analysis.
We are all familiar with this kind of evidence and commonly use it in everyday decision making. For example, when choosing a babysitter or plumber we would often ask after the experiences of friends and family. While it would be impractical and often impossible to demand higher standards of evidence for these day to day decisions, for more important ones it is best to avoid relying on anecdotal evidence if at all possible. This is because anecdotal evidence falls short of what is necessary for a reasonable standard of proof. To understand why this is so, we need to take a closer look at some of the pitfalls inherent in anecdotal evidence.
As it tends to be unplanned and poorly documented, anecdotal evidence often misses important information that would result in different conclusions. While scientific studies and experiments are carefully designed in attempt to ensure that all the relevant data is collected, this commonly does not occur with anecdotal evidence. For example, in case 1 above do the people with leukaemia have anything else in common that may be the cause of their cancers? While questioning of those involved may bring to light more information, often they may not be able to remember some important details. Furthermore, some important information may not be available at all because it was not collected. For example, in case 2 the vitamin would have to be absorbed by the sick man's body for it to be effective but how much actually was absorbed? If tests to measure the levels of the vitamin in his blood were not conducted at the time, we have no way of knowing.
Inaccuracies in anecdotal evidence is another serious problem. Often poorly documented, they rely on people's memories which are often incomplete and sometimes outright wrong. The people involved are usually not experts and may be mistaken about some of the salient details. Moreover, anecdotal accounts are commonly hearsay and mistakes may creep in and get compounded with each retelling. A scientific study or experiment is carefully documented by experts at every stage to ensure that all the data is as reliable as possible.
An exceptional event
Anecdotal accounts may be the exception rather than a "typical" experience and there is no way to tell whether this is the case or not. The stories that get remembered and reported are usually the "notable" ones. In relation to case 1, there may be a number of streets in the country with no power pylons where two or more people have come down with the same type of rare leukaemia. Would these people end up on a talk show? Part of the problem is that we are generally not very good at evaluating the likelihood of chance events without the help of statistics. The link between the leukaemia and power pylons may appear pretty convincing as it seems very unlikely that all these people could have come down with the same rare cancer by chance. But unlikely events do occur – someone always wins the lottery. Scientific evidence uses statistics and the appropriate controls (such the number of people living on streets without power pylons who also have the rare leukaemia) to determine whether something is occurring by chance. In anecdotal evidence, statistics can't be utilised due to insufficient numbers of subjects or lack of controls.
Anecdotal evidence in science and medicine
Scientists do not consider anecdotal evidence to be validating evidence, that is it is never used to decide if something is "true" or not. But anecdotal evidence is very useful in suggesting scientific questions or hypotheses and, thus, is a good starting point for further research. Case 1 suggests that a study into the incidence of the rare leukaemia around power pylons may be warranted. While case 2 suggests that research into the effectiveness of the vitamin therapy for the illness that the man was suffering from may be fruitful.
In medicine, the use of anecdotal evidence is much more prevalent. This is because proper scientific studies and experiments are so hard (and sometimes impossible) to carry out in a medical setting. New drugs are always subject to extensive scientific analysis before being allowed onto the market. But once on the market, these drugs may be used in ways that have not been scientifically evaluated (for different illnesses, in different doses or in new combinations with other drugs for example). In these cases, the physicians are relying on their own and their colleagues' anecdotal experience. This is also the case (and perhaps more so) for non-drug based interventions like surgery. Although anecdotal evidence will always play an important role in medicine due to the difficulty (and sometimes impossibility) of employing science to answer medical questions, many in the medical community believe that anecdotal evidence has been, and often still is, utilised more than it should be. This belief was the genesis of the movement called "evidence-based medicine".
As I noted earlier, we frequently use anecdotal evidence in our day to day lives. This familiarity is one of the reasons why we find it so compelling and convincing. In many cases, it may be the only evidence available and, despite its deficiencies, anecdotal evidence is definitely better than no evidence at all. But you should try to recognise anecdotal evidence as such and be aware its limitations. Then before you make a decision based on it, you will be able to ask yourself whether better evidence may exist and whether it would be worthwhile to track it down or even generate in yourself.