Add a Comment   Up to OJB's Blog List

The Power of Anecdotes

Entry 1818, on 2016-11-02 at 11:26:04 (Rating 3, Skepticism)

I recently discussed a range of subjects with a quite intelligent and thoughtful religious person (yes, they do exist). These included topics such as whether "god did it" is a useful answer to questions we might have about the real world, what limitations science should have on the questions it attempts to answer, and the nature of morality.

Since I don't have any particular religious view to defend I am open to look at all possibilities, but because of this I recognise that if I took all the possible sources of knowledge (all the religions, all the paranormal claims, all the philosophies, all the informal logic, and all the rigorous science) I would never arrive at any conclusion.

I would have to spend a lot of time carefully examining claims with little physical, objective evidence supporting them. I would have to reverse direction after taking a wrong turn because I followed misleading anecdotal evidence. I would spend so much time trying to collate all the various claims that I would have no time left to evaluate them.

That's why anecdotes don't count. In fact, everyone knows that anecdotes don't count because they only look at a tiny proportion of them, specifically the ones which support the worldview the person favours. So a Christian will take a lot of notice of people who say they have been healed by Jesus but ignore claims of the power of crystals, or how a disease was cured after a person was abducted by aliens, or how the Asvins (Hindu gods of healing) helped a person who couldn't be cured by conventional medicine.

I'm not saying all of these anecdotes are untrue, or that in a perfect world they shouldn't be investigated. What I am saying is that an anecdote by itself has very little value. If we gave equal weight to all anecdotes in a fair way we would have to believe in a huge number of mutually incompatible ideas. We would have to believe dozens of gods were performing healings. We would have to believe in the power of crystals, of herbal remedies, of homeopathy, of alien interventions, and of hundreds of other things as well.

Just saying that (for example) Christian healing through the power of prayer is true but all the other stuff isn't is classic cherry picking. If you believe in using anecdotes as evidence then you should believe all the anecdotes, not just the ones which agree with your preferred religion or new-age belief.

Or, you could believe none of them. And that is the far more rational approach that I take.

But we shouldn't just totally dismiss anecdotes. If there is sufficient reason to think that a consistent pattern is emerging then the idea should be tested using more objective, systematic methodologies. I would suggest two approaches to testing whether the anecdotes have merit: first have an unbiased expert look at the evidence objectively; or second, set up a scientific experiment or trial of some sort.

For example, if a lot of people report that their health improves after friends and family pray to Jesus to help them (and remember that the Christian God will answer prayers according to numerous Bible verses such as John 15:7) then let's test that claim. We know that people sometimes get better spontaneously, that they sometimes feel better because they think they should, and that there are many other confounding factors, so let's test the claim using a double-blind trial.

And when we do we get very conflicting results. Most show no effect. Some show that the people prayed for get worse. Some show they get better. These are exactly the results we get when we test other ideas which have doubtful prior probability, such as homeopathy. Therefore, whatever the anecdotes tell us, we can say that our interim conclusion is that prayer offers no consistent solution to health problems. In other words faith healing and prayer don't work.

Or, if we hear of an apparently miraculous cure of some sort, such as that attributed to Saint Teresa of Calcutta, then let's have a closer look at the claims. It turns out in that case, that almost all the claims were untrue and that the conclusion that a miracle occurred is embarrassingly absurd (see my blog post "Sinner or Saint?" from 2016-09-07 for details). So we can reject that anecdote based on better evidence.

Remember, that these are interim results, but all results in science are interim so we shouldn't treat them as any less certain than other conclusions reached in the same general area of human knowledge.

You might object and say that by dismissing anecdotes as evidence in themsleves that I potentially miss out on new discoveries. Well that is a risk I must take because if only one in a million anecdotes genuinely represent something new and real then I really can't take any of them seriously and risk being mislead by the other 999,999. It's that simple.

There are no comments for this entry.

You can leave comments about this entry using this form.

Enter your name (optional):
Enter your email address (optional):
Enter the number shown here:
Enter the comment:

To add a comment: enter a name and email (optional), type the number shown, enter a comment, click Add.
Note that you can leave the name blank if you want to remain anonymous.
Enter your email address to receive notifications of replies and updates to this entry.
The comment should appear immediately because the authorisation system is currently inactive.