The statistical method made me lie

14 Sep 2012

There’s a hubbub brewing over a recent study published in the Annals of Internal Medicine that compares organic food (as in ‘USDA Organic’) to non-organic food. The study, titled “Are Organic Foods Safer or Healthier Than Conventional Alternatives?: A Systematic Review” is a meta-analysis of about 200 previous studies. Their conclusion, which I have cut-and-pasted below, is

The published literature lacks strong evidence that organic foods are significantly more nutritious than conventional foods. Consumption of organic foods may reduce exposure to pesticide residues and antibiotic-resistant bacteria.

When I first heard about this study on the radio, I thought the conclusion seemed kind of obvious. It’s not clear to me why, for example, an organic carrot would have more calcium than a non-organic carrot. At least, I couldn’t explain the mechanism by which this would happen. However, I would expect that an organic carrot would have less pesticide residue than a non-organic carrot. If not, then the certification isn’t really achieving its goals. Lo and behold, that’s more or less what the study found. I don’t see the controversy.

But there’s a petition over at change.org titled “Retract the Flawed ‘Organic Study’ Linked to Big Tobacco and Pro-GMO Corps”. It’s quite an interesting read. First, it’s worth noting that the study itself does not list any funding sources. Given that the authors are from Stanford, one could conclude that therefore Stanford funded the study. The petition claims that Stanford has “deep financial ties to Cargill”, a large agribusiness company, but does not get into specifics.

More interesting is that the petition highlights the involvement in the study of Ingram Olkin, a renowned statistician at Stanford. The petition says

The study was authored by the very many [sic] who invented a method of ‘lying with statistics’. Olkin worked with Stanford University to develop a “multivariate” statistical algorithm, which is essentially a way to lie with statistics.

That’s right, the statistical method made them lie!

The petition is ridiculous. Interestingly, even as the petition claims conflict of interest on the part of the study authors, it seems one of the petition authors, Anthony Gucciardi, is “a natural health advocate, and creator of the health news website NaturalSociety” according to his Twitter page. Go figure. It worries me that people would claim the mere use of statistical methods is sufficient grounds for doubt. It also worries me that 3,386 people (as of this writing) would blindly agree.

By the way, can anyone propose an alternative to “multivariate statistics”? I need stop all this lying….