Add a Comment   Up to OJB's Blog List

Anecdotes

Entry 644, on 2007-11-20 at 21:34:51 (Rating 2, Science)

Yesterday I blogged about the Quackcast podcast. Another quote used on that show is that "the plural of anecdote is anecdotes, not data". Its a thought that can be applied to many areas where a proposition is being supported with poor evidence. What it is really saying is that quoting multiple instances which appear to support an idea really means nothing because bias and poor understanding of chance can lead anybody into false beliefs.

I accept that multiple examples of a phenomenon can't be ignored, but they should be used as a starting point for a real investigation instead of as an end result. For example, there are thousands of sightings of UFOs, but that can't be used as conclusive evidence they exist because when the reports are carefully investigated there are usually better explanations for the event, or the report might be found to be deficient in some way (for example, it could be a hoax).

There are many areas where research contradicts anecdote. Some of the more significant are: psychic powers, prayer, ghosts, UFOs, and cryptozoological phenomena (bigfoot, the Loch Ness monster, etc). In all of these cases commonly accepted anecdotes are not supported by thorough investigation. That doesn't prove that the phenomena doesn't exist, but it does mean we should accept the non-existence of the phenomena as an interim theory until possible further evidence contradicts it, and that's all we ever do in science anyway.

One of the most common problems with anecdotes is that the person presenting them tends to have heavily cherry picked. An event where someone got a psychic reading right is worth mentioning but 100 events where the same person got it wrong will most likely be ignored. This means that the possibility of a statistical fluke is often discounted. The person hearing the stories will often form a hasty generalisation on the topic without knowing all the evidence.

As I have said, anecdotes are a problem in pseudoscience and the paranormal, but this happens in politics as well. A recent campaign by conservatives here in New Zealand purported to present evidence against recent anti-smacking legislation, but at closer inspection it would be better construed as evidence supporting the new law because the incidents were so minor and rare.

Anecdotal information has lead science astray at times in the past but science's rigorous self correction mechanism has quickly fixed the error. Of course there could be scientific beliefs we accept now which are wrong but haven't been corrected yet. That's why we have to keep reminding ourselves that all conclusions are interim and we should constantly consider new evidence as its discovered. And we should always look at the data because anecdotes are as likely to obscure the truth as to reveal it.

Comment 1 (991) by Paul B on 2007-11-25 at 22:03:59:

Are you saying that things are only true if science experiments show they are true? Science misses out on a lot of things because it is too narrow to find out about some of the things people have already known for centuries.

Comment 2 (992) by OJB on 2007-11-26 at 19:09:38:

Its possible that science might take a bit longer to discover some things because of its insistence on thorough analysis and following rigorous protocols. But the advantage of its narrower focus is that it is far less likely to reach false conclusions. If the evidence really exists science will eventually follow it to the correct conclusion, although it might take a while.

You can leave comments about this entry using this form.

Enter your name (optional):
Enter your email address (optional):
Enter the number shown here:
Enter the comment:

To add a comment: enter a name and email (optional), type the number shown, enter a comment, click Add.
Note that you can leave the name blank if you want to remain anonymous.
Enter your email address to receive notifications of replies and updates to this entry.
The comment should appear immediately because the authorisation system is currently inactive.