Stanford Study Spins Out of Control

In 2012, a team of Stanford scientists published a meta-analysis in the Annals of Internal Medicine, entitled "Are Organic Foods Safer or Healthier Than Conventional Alternatives?" In a Stanford press release summarizing the study, the senior author, Dena Bravata, MD, MS is quoted as saying: "There isn't much difference between organic and conventional foods, if you're an adult making a decision based solely on your health." The release then proceeds to summarize the findings of the study as follows:

• They did not find strong evidence that organic foods are more nutritious or carry fewer health risks than conventional alternatives, though consumption of organic foods reduce the risk of pesticide exposure.
• Although there is a common perception--perhaps based on price alone--that organic foods are better for you than non-organic ones, it remains an open question as to the health benefits.
• No consistent differences were seen in the vitamin content of organic products, and only one nutrient--phosphorous--was significantly higher in organic versus conventionally grown produce (and the researchers note that because few people have phosphorous deficiency, this has little clinical significance).
• Evidence from a limited number of studies suggested that organic milk may contain significantly higher levels of omega-3 fatty acids.
• While researchers found that organic products had 30 percent lower risk of pesticide contamination than conventional fruits and vegetables, organic foods are not necessarily 100 percent free of pesticides. What's more, the pesticide levels of all foods generally fell within allowable safety limits.
• As for what the findings mean for consumers, the researchers said their aim is to educate people, not to discourage them from making organic purchases.

On Sept 3, 2012, Kenneth Chang wrote a story in the New York Times entitled "Stanford Scientists Cast Doubt on Advantages of Organic Meat and Produce." It started a firestorm. The story seemed to pick up the Stanford press release as opposed to the original study, and pretty much parroted its findings. The author did, however, add his own editorializing, in particular that the study's conclusions "will almost certainly fuel the debate over whether organic foods are a smart choice for healthier living or a marketing tool that gulls people in overpaying." This single statement not only put the organic food industry on the defensive, but cast organic producers as possible shysters out for a buck. Even the findings demonstrating that organic produce and meat had benefits in excess of conventional farming methods were single-handedly deconstructed and declared clinically insignificant--the total effect of the article was to suggest organic food was likely a waste of money.

Other news organizations jumped on the bandwagon. On NPR, host David Greene told his audience:"I feel kind of duped. I mean, I was in a grocery store and was seriously thinking about buying organic raspberries the other day because I figured that, you know, organic, it must be better. I mean, how did this industry explode and become this big without someone at some point saying earlier, you know, we don't that it is any better." And the headline of the New York Daily News heralded: Organic food is not healthier...When it comes to nutrition, organic meats, produce and dairy are no better." Dozens more outlets headlined variations of the same.

Soon, bloggers from The Huffington Post made a solid, coherent effort to put the study in context. Most declared that the findings "missed the point." Here are a list of some of their more compelling criticisms:

• Food issues should not be discussed solely within a consumer frame, which ignores how the organic movement is a system that interacts with the environment and farm workers in a political way that transcends nutrients, but has much to do with biodiversity, the decline of pollinators, and effects of pesticides on farm workers, children and pregnant women.
• Many scientists pointed out the study's methodological flaws such as undercounting and the failure to meaningfully define terms.
• Nutrients and vitamins alone do not constitute "healthy." Reduction of exposure to pesticides and antibiotics contribute to overall health as well.
• The whole point of organic food is not that it's more nutritious, but that it's more environmentally sustainable. The study sets up a straw man issue only to knock it down.

Perhaps Amy Fitzpatrick's post best illustrates how spin can confuse the issues by framing them within questions that might not be the most important to consider. She points out that the NYT article begins with the question, "Does an organic strawberry contain more vitamin C than a conventional one?" She writes, "If you were hooked by this, you may have been lured into 'accepting the premise.' If you accepted the premise, you may have then thought organic food should be more nutritious and it should have more vitamins that non-organic food. It should not."

In fact, the Organic Food Production Act makes no mention of nutrition, nutrients or higher vitamin levels. It plainly states: "Neither the Organic Foods Production Act nor the NOP address food safety or nutrition."

Even Mark Bittman, a writer for the NYT, referred to the study as "flawed," calling it not only an exercise in misdirection, but a headline generator. By providing "useful" and "counterproductive" information about organic food, it "played right into the hands of the news hungry while conveniently obscuring important features of organic agriculture."

By October 15, Kenneth Chang wrote a followup in the NYT, now entitled "Parsing of Data Led to Mixed Messages on Organic Food's Value." In this article, he takes a much more balanced approach, and even cites a study ignored by the Stanford researchers that found organic food was more nutritious than conventional food.

Had Chang bothered to take more time to put together the thoughtful article rather than the controversial one, the flawed findings of the Stanford study may not have found their way onto dozens of Facebook posts and tweets spreading the misinformation.