You probably heard about the study published last week in the Archives of Internal Medicine which links red meat to a substantially increased risk of death from all causes. The story has spread through flurry of news reports and proliferated through Facebook and Twitter feeds everywhere. From the New York Times:

Eating red meat is associated with a sharply increased risk of death from cancer and heart disease, according to a new study, and the more of it you eat, the greater the risk.

… after controlling for [smoking, body mass index,] and other variables, they found that each daily increase of three ounces of red meat was associated with a 12 percent greater risk of dying over all, including a 16 percent greater risk of cardiovascular death and a 10 percent greater risk of cancer death. …

“When you have these numbers in front of you, it’s pretty staggering,” said the study’s lead author, Dr. Frank B. Hu, a professor of medicine at Harvard.

Judging from these results, on top of the fact that health authorities have been telling us for decades that red meat is unhealthy, there can’t be any doubt that pulling back on all that pork and beef is a good idea, can there? 

The study in question is what’s called an epidemiological study. This means that a large group of people was followed over time, and they periodically filled out detailed surveys reporting their diets and behavior such as exercise habits and alcohol intake. After the data was compiled the authors used statistical methods to correct for as many confounding factors as they could think of. For example, red-meat eaters are more likely to smoke, so the researchers corrected the results to eliminate the increased number of deaths from smoking.

Once the effects of all such confounding factors are eliminated, the increased risks from red meat have supposedly been isolated, and voilà, red meat has been proven to kill you. (No one has been quite irresponsible enough to actually use the word “proven,” but judging from the breathless press coverage and from the researchers’ own comments, they may as well have.)

Leaving aside the issue of the unreliability of self-reported data, the problem with epidemiological studies like this is that despite all the efforts to control for confounding factors, they can’t establish causation, only correlation. No matter how hard the researchers try, they cannot possibly untangle the complex web of factors that would lead a self-selected group of people to eat more hamburgers and hot dogs and fewer skinless chicken breasts and Tofutti Cuties.

This is why, as Gary Taubes notes in a blog post worth reading in its entirety, even “the best epidemiologists … think this nutritional epidemiology business is a pseudoscience at best.”

Taubes points to a further reason to doubt the results of this study: something known as the compliance effect, which was discovered during a drug trial when researchers found that those who faithfully took their medicine as instructed had substantially better outcomes than those who didn’t follow the prescribed regimen as closely. So far, so good, but it turned out that the those in the placebo group who most faithfully followed directions showed the same benefit.

What this tells us is that “whenever epidemiologists compare people who faithfully engage in some activity with those who don’t — whether taking prescription pills or vitamins or exercising regularly or eating what they consider a healthful diet — the researchers need to account for this compliance effect or they will most likely infer the wrong answer.” And when it comes to diet, the people who avoid red meat are the ones most likely to be faithfully following decades worth of establishment advice about diet.

So when we compare people who ate a lot of meat and processed meat in [the late ’80s and the 1990s] to those who were effectively vegetarians, we’re comparing people who are inherently incomparable. We’re comparing health conscious compliers to non-compliers; people who cared about their health and had the income and energy to do something about it and people who didn’t.  And the compliers will almost always appear to be healthier in these cohorts because of the compliance effect if nothing else. No amount of “correcting” for BMI and blood pressure, smoking status, etc. can correct for this compliance effect, which is the product of all these health conscious behaviors that can’t be measured, or just haven’t been measured. … When the Harvard people insist they can “correct” for this, or that it’s not a factor, they’re fooling themselves. …

That’s why you need randomized, controlled trials to establish causation. And no such trial has ever backed up the conclusions of this study. (For more debunking, go here and here.)

What’s more annoying than the conclusion of this silly study is the sensationalistic and uncritical press coverage it garnered. A responsible science journalist would examine reports like this with a skeptical eye rather than trumpeting its conclusions as scientific fact. The coverage of this study proves such skepticism to be depressingly rare.

Take, for instance, this follow-up article by Donald Bradley of McClatchy Newspapers headlined “Red meat mortality study leaves many carnivores unfazed.” It opens by comparing red-meat eaters to cigarette smokers and those who refuse to wear seat belts, and goes on to uncritically report the study’s conclusions at length. Finally, the author makes an attempt at “balance” with a brief quote criticizing the study — not from an objective critic, but drawn from a statement from the American Meat Institute, which only adds to the impression that no principled objection to the study exists.

And that’s how bad science and irresponsible press coverage lends the conventional wisdom on diet, however questionable, the aura of indisputable scientific fact. This inevitably affects government recommendations, school-lunch requirements, and ultimately, regulations aimed at changing the general public’s diet. The actual health consequences of all this are anyone’s guess.