A personal story on cause and effect

I recently posted this comment on the "Autism debate" at Salon.com:

No doubt I will be accused of being an arrogant scientist, but here goes... Which of the following sentences makes sense to you?

"Cosmologists have found several planets rotating around stars quite a distance from earth, but parents disagree."

"Geologists have found that plate tectonics underlies the formation of mountains and the occurrence of earthquakes, but parents disagree."

"Medical scientists have found that thimerosal and vaccines have nothing to do with autism, but parents disagree."

This was a trick question. All 3 statements are nonsensical.

*That's* why rationalists call parents who campaign to ban vaccines the "mercury militia". Right or wrong, it's out of frustration with the lunacy of it all.

(As an aside, I'm a biomedical scientist and new parent. My son has been having *all* of his vaccines on schedule.)

Several commenters took me to task as being a lousy scientist, because I ought to know that parents watch their children and know their children well. I thought I would expand on this thought, as it's clear that many people don't understand how science arrives at particular conclusions. People are really, really lousy at arriving at cause and effect, and it takes very careful controls to demonstrate cause and effect.

As an example of how I (a scientist) arrive at a conclusion:

I have recently been diagnosed with occipital neuralgia. It's a rather painful disorder that's caused by damage to nerves in the back of my scalp (running up over my ear). Basically, the nerves fire inappropriately, and so I experience severe pain in my head like I'm being struck by lightning (repeatedly) even when there's nothing physically wrong with my head.

My neurologist has prescribed for me a drug called gabapentin that is supposed to help numb the nerves. On my last visit, he asked me the question, "Is the medication working?" Me being a scientist, I literally replied, "I don't know, I haven't done the control." My pain is mostly gone, but occipital neuralgia often spontaneously goes away. So I can't tell if my pain has been reduced because I'm taking the medicine, or if it's been reduced because the neuralgia is receding. (That said, given that the medicine has been shown in clinical trials to reduce the pain of neuralgia, I am going to continue with my dosage for a while before I experiment with reducing my dose). It may be that the medicine is no longer doing anything for me. Or it could be that it is the only thing between me and searing pain. Until I experiment with reducing the dose, I really have no idea.

It's an awful lot more difficult to determine cause and effect if you only have one sample (yourself). In science, we refer to this as n equals 1 (n=1). You have 1 sample. One. One sample tells you very, very little.

Same thing goes with determining causation with a disease in your child. If my son gets sick, I could say, "Well, he caught it at the daycare." He may have caught it at the daycare. Or at the grocery store. Or from the child next door. Or from his grandmother. I don't have enough information, even though I am his father and I watch him carefully. And when his cold resolves, I could say "It's because I gave him orange juice". Or vitamins. Or holy water. Or sunshine. But really, I don't know how quickly he would have gotten better if I had done none of these things. Or all of them.

This is why it is very important to determine things with larger groups of people. Larger groups allow you the power of controls. Using controls are how we determine causation.

Causation and correlation

I thought I would follow up my post about causation with this article about correlation. It's well worth a read. Merely seeing that two things happen together don't mean that one is caused by the other (and assuming that can lead to disastrous consequences).

A woman in Holland is spending time in prison because several people died while she was working as a nurse (the prosecution claims that it's just too unlikely, and she *must* be a serial killer - and it's not clear that they have any real evidence against her beyond the improbability of the event). While that was an unlikely event, given the number of hospitals in Holland, one would expect that at least one of those hospitals would have an unlikely event. Ben Goldacre gives a good treatment of the issue in Losing the Lottery:

Meanwhile, a huge amount of corollary statistical information was almost completely ignored. In the three years before Lucia worked on the ward in question, there were 7 deaths. In the three years that Lucia did work on that ward, there were 6 deaths. It seems odd that the death rate should go down on a ward at the precise moment that a serial killer – on a killing spree – arrives on the scene. In fact, if Lucia killed them all, then there must have been no natural deaths on that ward at all, in the 3 years that she worked there.

It seems very likely this is an innocent woman...

How does a scientist determine causation?

After talking it over with Mrs. Factician (who is also a molecular biologist), I realized that yesterday's post might not have made sense to everyone in the way it does to me. I would like conspiracy factory to be more than a place where scientists whine to each other that no one understands us. I'd like it to be educational to folks who don't do science every day. I'd like people like my Mom to come here and read it and find something useful for their day.

That said, I'm going to spell out the comic a little better. For those of you who are skeptics/scientists, this may seem a little like explaining a joke (if you have to explain it, it's not funny). But it should be educational, too. So go over, look at the comic, and come back if you have to. What does it have to do with autism? The joke is that scientists don't immediately assume causation if two events occur close together. The fact that the scientist gets zapped by lightning immediately after pulling the lever doesn't necessarily mean that the lever caused the lightning. (Granted, it may have caused the lightning, but until you've seen it multiple times under controlled conditions, you can't say for sure). Of course, doing an experiment that involves being shocked by lightning certainly wouldn't be performed by a scientist. It would be performed by her graduate student. ;)

What does this comic to do with autism? As it happens, the folks who claim that vaccines cause autism mostly rely on the fact that autism begins to show its ugly head around 2 years of age, right after children get a series of booster shots for their vaccines. Does this mean that vaccines cause autism? No. Especially once you realize that autistic children present with autism around 2 years of age whether they've had vaccines or not. It just happens that these things coincide in time with each other.

An experimentalist who wants to ask the question: "Do vaccines cause autism?" would examine a large group of children who get the vaccine and a large group of children who don't (who otherwise are living in similar circumstances) and see if a similar frequency of children get autism in both groups. As it happens, several groups have done this (National Academy of Sciences summarizes the studies here). And they do see similar frequencies. This pretty much kills the argument. (To further extend the analogy to the comic, this would mean that the scientist who pulls the lever a thousand more times would never get shocked again).