Friday, October 19, 2007

A personal story on cause and effect

I recently posted this comment on the "Autism debate" at Salon.com:

No doubt I will be accused of being an arrogant scientist, but here goes... Which of the following sentences makes sense to you?

"Cosmologists have found several planets rotating around stars quite a distance from earth, but parents disagree."

"Geologists have found that plate tectonics underlies the formation of mountains and the occurrence of earthquakes, but parents disagree."

"Medical scientists have found that thimerosal and vaccines have nothing to do with autism, but parents disagree."

This was a trick question. All 3 statements are nonsensical.

*That's* why rationalists call parents who campaign to ban vaccines the "mercury militia". Right or wrong, it's out of frustration with the lunacy of it all.

(As an aside, I'm a biomedical scientist and new parent. My son has been having *all* of his vaccines on schedule.)
Several commenters took me to task as being a lousy scientist, because I ought to know that parents watch their children and know their children well. I thought I would expand on this thought, as it's clear that many people don't understand how science arrives at particular conclusions. People are really, really lousy at arriving at cause and effect, and it takes very careful controls to demonstrate cause and effect.

As an example of how I (a scientist) arrive at a conclusion:

I have recently been diagnosed with occipital neuralgia. It's a rather painful disorder that's caused by damage to nerves in the back of my scalp (running up over my ear). Basically, the nerves fire inappropriately, and so I experience severe pain in my head like I'm being struck by lightning (repeatedly) even when there's nothing physically wrong with my head.

My neurologist has prescribed for me a drug called gabapentin that is supposed to help numb the nerves. On my last visit, he asked me the question, "Is the medication working?" Me being a scientist, I literally replied, "I don't know, I haven't done the control." My pain is mostly gone, but occipital neuralgia often spontaneously goes away. So I can't tell if my pain has been reduced because I'm taking the medicine, or if it's been reduced because the neuralgia is receding. (That said, given that the medicine has been shown in clinical trials to reduce the pain of neuralgia, I am going to continue with my dosage for a while before I experiment with reducing my dose). It may be that the medicine is no longer doing anything for me. Or it could be that it is the only thing between me and searing pain. Until I experiment with reducing the dose, I really have no idea.

It's an awful lot more difficult to determine cause and effect if you only have one sample (yourself). In science, we refer to this as n equals 1 (n=1). You have 1 sample. One. One sample tells you very, very little.

Same thing goes with determining causation with a disease in your child. If my son gets sick, I could say, "Well, he caught it at the daycare." He may have caught it at the daycare. Or at the grocery store. Or from the child next door. Or from his grandmother. I don't have enough information, even though I am his father and I watch him carefully. And when his cold resolves, I could say "It's because I gave him orange juice". Or vitamins. Or holy water. Or sunshine. But really, I don't know how quickly he would have gotten better if I had done none of these things. Or all of them.

This is why it is very important to determine things with larger groups of people. Larger groups allow you the power of controls. Using controls are how we determine causation.

Digg!

17 comments:

Kevin Scott said...

But did you PRAY? Because if you pray with enough faith your child will be well. So will your head.

Which is why I believe parents of autistic children have autistic children because of their lack of faith.

This is so easy to understand. It just makes common sense.

Chuck said...

Genetics will not be able to reach the scientific threshold of “causation” for ASD disorders. Is it a factor, yes.

Thimerosal will not be able to reach the scientific threshold of “causation” for ASD disorders as well. Is it a factor, yes.

Environment will not be able to reach the scientific threshold of “causation” for ASD disorders as well. Is it a factor, yes.

All have correlation, none have causation.

Anonymous said...

just to be a nitpicker: You can't experiment with reducing the dose and get meaningful results -- due to having a test group of 1. If you reduce the dose and your pain comes back, it could be because the medication had been working at the higher dose, or it could be a psychological effect wherein you expect the pain to come back, so it does, etc.

-james.

makita said...

I didn't read all the comments, but I was pleasantly surprised by the number of sensible ones.

When my sons' pediatrician told us she suspected he was autistic, we ultimately decided to go ahead and immunize him normally. This was several years after thimerosal was removed from the shots, but even if it hadn't I still would have gone ahead.

No, I'm not a crazy mother, but I do know the lesser of two evils when I see it, and I do understand the real risks of not immunizing.

Chuck said...

Kevin Scott,

I agree that parents of autistic children have faith. These parents have an unfounded faith in the hubris of the scientific community. This elitist belief that the scientific community displays has never helped these parents at all.

Aquinas Dad said...

Of course, some scientists express reservations about the studies ruling out thimerosal or vaccine reactions. Also, some parents are scientists.

Do I think there is a link? No. But to dismiss this controversy by, effectively, mocking the concerns of parents is, well, less than endearing. Especially since the investigation of possible causes for the increase in autism is a bit more complicated than looking for palnets around other suns. In the last decade alone Thimerosal has gone from being considered safe to being considered dangerous to being considered safe again... all by the same consensus of scientists that now state that there is no link.

Many parents point out - correctly - that new discoveries sometimes prove scientists wrong; their caution and desire for more testing before continuing exposure of children to chemicals that are likely to cause allergic reactions is not worthy of mockery.

The Factician said...

aquinas dad,

My intent is not mockery of parents. My intent is to mock journalists who compare the views of scientists with parents and equate them in value. They're not equal. As I point out in the little anecdote, I don't consider myself able to determine cause and effect of my own health issues, much less my child's. And I'm an excellent scientist. To prove cause and effect requires vastly more data than a parent will be exposed to, and requires a vast amount of training to interpret said data. For a journalist to equate these two is clearly ludicrous.

You're absolutely right that the search for a cause for autism should be compared in difficulty with sending probes into space. That said, training as a parent doesn't qualify one as an epidemiologist or a molecular biologist.

That is my only point here. Parents can disagree with scientific consensus all they want. But their experiences as parents don't qualify them to have meaningful disagreements. A scientist has a hell of a lot more access to meaningful data than a parent, and is able to make a lot more meaningful contributions in the search for cause and effect.

Chuck said...

A scientist has a hell of a lot more access to meaningful data than a parent, and is able to make a lot more meaningful contributions in the search for cause and effect.

That is a gross generalization that is only biased opinion, that is not a statement of fact.

The Factician said...

*sigh*

[sarcasm]

You're right, Chuck. The average parent has subscriptions to all of the academic journals where data are published.

They also have access to high tech labs, where they can test their latest pet hyptheses, in addition to the hundreds of thousands to millions of dollars required to do said experiments.

In addition, the average parent has decades of training in how to do controls and how to think about data in a way that they can make meaningful conclusions.

Finally, the average parent has the support network of dozens to hundreds of others of parents who also have decades of training in science, and are willing to pick apart the weakness of their arguments, honing it down until they have some close approximation of the truth.

[/sarcasm]

No, Chuck. It's a statement of fact. Scientists have better access to data and better training to interpret said data than your average parent.

*sigh*

Chuck said...

Fact

Scientist have "cherry picked" data that is mulled and refined to their unique laboratory specification that may or may not be communicated to other similar laboratories or institutions or oversight government entities.

Government oversight entities have vast amounts of scientific studies that may or may not be communicated to other government agencies, similar laboratories or institutions, and the general public.

If economic model of universal knowledge in the market were a statement of FACT, there would definitely be a significant reduction in the number of scientists, doctors, economist, and many many other professions.

Given that both you and I are currently gainfully employed proves universal knowledge of the marketplace to be a fallacy and your statement nothing more then opinion.

Fact,

There may be many parents with better scientific insight on this subject then you.

[sigh, shaking head]

The Factician said...

Fact

Proof, please?

Government oversight entities have vast amounts of scientific studies...

I see you agree with me, then, that scientists (both government-employed and academic) have better access to data, than your average parent.

There may be many parents with better scientific insight on this subject then you.

I suspect there are a few parents out there with better insight than me on this topic. But on average, parents don't know more about this subject than the average scientist who can educate himself on the topic. And an average parent certainly know *vastly* less than scientists actively engaged in research related to the health of children (like autism research).

Gotta run, Chuck. Farewell for now.

Chuck said...

Government oversight entities have vast amounts of scientific studies...

You forgot to print the ending of this sentence, which is

that are not worth the paper they are printed on.

Also, you make a logical fallacy in assuming that government scientist will have more access all of the necessary scientific information on a subject then a non-scientist would.

As a counter-point, maybe you have heard of Erin Brockovich?

Aquinas Dad said...

This statement is a bit, well, funny to me;

"My intent is to mock journalists who compare the views of scientists with parents and equate them in value. They're not equal."

Combined with the rest of your statements, I sense a tendency towards technocracy in your thinking. My wife is a biologist and a parent; does her opinion count, or does someone need to approve her journal subscriptions? I am a theologian; what if I were to demand that you stay out of discussions of ethics and morality unless you were a theologian or philosopher with the 'proper' credentials and training?

Here is a fact; there are a lot more parents than there are scientists. Journalists seem have uncovered another fact - it appears parents (in general) don't trust scientists (in general). Perhaps the real question should be 'why do scientists lack credibility with parents?'

I have my own suspicions.

The Factician said...

I am a theologian; what if I were to demand that you stay out of discussions of ethics and morality unless you were a theologian or philosopher with the 'proper' credentials and training?

Hmmm... I don’t think I’ve demanded anyone stay out of any discussions. Did you get that impression?

What I have said is that I think that a parent’s opinion and a scientists studying a child’s health are not of equal value. Does that make me a “technocrat”? If so, then count me as one. If a surgeon says a child needs a liver transplant, and a parent says that she doesn’t, do you think those opinions are of equal value? If an optometrist says a child has 20/20 vision and a parent says they are blind, whose opinion will you put the most weight on?

Perhaps you should re-read my post. In it I mention that I am a scientist (I’m also a parent, the plot thickens). I also mention that I think I am unqualified to diagnose cause and effect of my own illnesses. Does that make me a technocrat? Or does it mean that I have sufficient training to realize that having data from one person (me) and from one treatment (my gabapentin pills) means that I cannot make any conclusion at all. The only reason I can suggest that the gabapentin is working is that it has been shown to work in double blind placebo-controlled clinical trials in other people with the same condition as my own. My own experimentation with dose is extremely poorly designed experiments that have little hope of proving anything other than that I have searing pain some times and not at others.

Here is a fact; there are a lot more parents than there are scientists.

Hmmm... There are also a lot more parents than there are carpenters. Is this supposed to mean anything? Does that mean parents opinion of carpentry is more valuable than a carpenter?

Journalists seem have uncovered another fact - it appears parents (in general) don't trust scientists (in general). Perhaps the real question should be 'why do scientists lack credibility with parents?' I have my own suspicions.

As do I. Our modern culture glorifies the outsider. The maverick. The uneducated man who knows more than the physicist. The oil-drillers in space who save the world from a comet, because the dumb gits at NASA can’t figure it out. We no longer value training or expertise. It's rather a pity, isn't it?

Aquinas Dad said...

"My intent is to mock journalists who compare the views of scientists with parents and equate them in value. They're not equal."

As I said; based upon this, you must admit that my views are of superior value to yours in terms of morals and ethics, no? If you consistently apply this concept, you must also dismiss scientists who have doubts about ethics which conflict with the majority of philosophers.

"To prove cause and effect requires vastly more data than a parent will be exposed to..."

While I understand your attempted point, the fact remains that this is true and/or relevant of a rather small, even vanishingly small, sub-set of decisions a parent must make. Indeed, of the decisions of any person. While the scientific method is a fine tool, its power derives from its rather narrow focus and, frankly, limited usefulness outside of academia.

A parent that devoted their time to scientific investigations of cause and effect in regards to the source of each and every illness of their child would be a lousy parent; there would be little time to actually be a parent. Therefore parents must do what all of us must do - risk assessment and reaction, whether aware of the process or not.

While you may be frustrated that parents do not accept the conclusions of a majority of scientists, that is because you view those scientific opinions as knowledge-based assessments (i.e., it is an objective evaluation of facts). To the majority of people, however, this is an authority-based assessment (i.e., do I trust the veracity of the authority).

While you may blame a culture that tends to glorify the underdog, I disagree; for example, the trope of a lone scientist struggling against mass ignorance is just as prevalent in culture as the good ol' boy vs. the scientists. I posit that the use of emotional language by some scientists combined with public failures of scientists as authorities have led to a fear that scientists are no more or less likely to be accurate than non-scientists.

For example, Thalidomide was a serious blow to the credibility of scientists in the public sphere. The fact that decades after the Thalidomide issue resulted in changes to drug testing the Vioxx and cerivastatin issues arose further undermine the credibility of scientific statements on drug safety. Indeed, even such things as the recents studies pointing to transfats as a health risk - after decades of being touted as the safer alternative by medical professionals - have eroded confidence in the authority of scientists in the arena of public health.

Since experience has led at least some parents to critical doubts as to scientific authority, they *must* perceive statements about the link between autism and vaccination as subjective (as I mentioned, above, in the last decade alone scientific authorities has changed from declaring Thimerosal safe to declaring it unsafe to, once again, declaring it safe). Many seem to be deciding that caution is the best rule; avoid as many vaccinations as possible just in case the questionable authorities are wrong again. This is neither unreasonable nor irrational (at least in most cases), but is just solid risk assessment.

Let me revisit something; you state,

"I am a scientist (I’m also a parent, the plot thickens)."

and

"...I think that a parent’s opinion and a scientists studying a child’s health are not of equal value."

Let me ask you - if you disagreed on some point with a health scientist about the best course for your child, would you simply submit to the expert?

The Factician said...

Hmmm... Blogger ate my response. Let's try again...

As I said; based upon this, you must admit that my views are of superior value to yours in terms of morals and ethics, no?

Superior? Perhaps. More informed? Almost certainly. Are you suggesting that your training is of no value? Why does one train in a field if all arguments (informed or not) are of equal value?

While the scientific method is a fine tool, its power derives from its rather narrow focus and, frankly, limited usefulness outside of academia.

Hmmm... Have you ever taken any medicine? Have you ever driven across a bridge or flown in an airplane? I think you are deeply mistaken about how often the scientific method impacts your daily life.

On the other hand, I'm not suggesting that a parent use controlled experimental conditions for their daily decisions (did you get the impression that I had?). This simply isn't possible. But I am suggesting that if a controlled experiment tells you one thing (that thimerosal doesn't cause autism) and a parent tells you another (But I saw my child show signs of autism after he got a vaccine shot!), you'd be wise to put your money on the controlled experiment.

Since experience has led at least some parents to critical doubts as to scientific authority, they *must* perceive statements about the link between autism and vaccination as subjective (as I mentioned, above, in the last decade alone scientific authorities has changed from declaring Thimerosal safe to declaring it unsafe to, once again, declaring it safe).

Safe? Who has declared thimerosal safe? Everything is toxic. Everything.

http://conspiracyfactory.blogspot.com/2007/05/everything-is-toxic.html

Scientists asked the question if thimerosal caused autism. They addressed that question experimentally. And the answer was no.

Let me ask you - if you disagreed on some point with a health scientist about the best course for your child, would you simply submit to the expert?

Hmmm... I think you are under the impression that I am an authoritarian. I am not.

I am not suggesting that we submit ourselves to higher powers. I am suggesting that if a parent tells me that their child got autism from a vaccine shot and a scientist tells me that's simply not possible based on experimental data (that is available for me to see) that those two peoples' opinions are not of equal value. Without seeing the data, I would be more inclined to trust the scientist.

I am in the fortunate position of being highly trained and able to evaluate many types of biological data. If I had a further question, I would look at the data myself and decide based on that. If I were unable to interpret the data myself, I would be likely to trust the scientist, based on my trust in the scientific method. I know what I am able to discern with my own two eyes watching my child doesn't necessarily reflect true information, and I am aware of the multitude of ways that I can be deceived. That is the point I am trying to make. Science filters out as many of the deceptive bits of info, and gets closer to the truth than your average parent is capable of (through lack of data and lack of training how to look at the data).

You might enjoy reading this post at Bad Science:

http://www.badscience.net/?p=404

It points to how bad people are at finding patterns and cause and effect. Science filters these things out - imperfectly - but it is the best tool we have available for determining cause and effect.

Chuck said...

There is no way to “scientifically” prove that thimerosal, genetics, OR environmental toxins “CAUSE” autism due to the multiple combinations of diagnosis criteria that are subjectively determined in the DSM, the actual diagnosis that is subjectively measured, and the multiple combination of thimerosal, genetics, and environmental toxins that may cause ASD.

Thimerosal, genetics, and environmental toxins will never be able to reach the threshold of causation, but all are factors. The best tool we have available for determining cause and effect offers nothing that actually aids anyone with ASD.