Nutritional clinical trials vs. observational studies for dietary recommendations.

The Science Behind Dietary Recommendations Isn’t As Conclusive As You Think

The Science Behind Dietary Recommendations Isn’t As Conclusive As You Think

What to eat. What not to eat.
April 12 2015 7:05 PM

Nutritional Science Isn’t Very Scientific

The research behind dietary recommendations is a lot less certain than you think.

Heroes or villians?

Somchai Som/Shutterstock

The potato industry recently scored a big victory, in what sounds at first like a familiar story of Big Ag winning out over public health. Eight years ago, over allegations of insufficient nutritional value, potatoes were excluded from a government program that helps pregnant women and young children improve their diets. The rationale was that aid recipients in the Women, Infants, and Children program already ate plenty of white potatoes, a widely consumed vegetable in the U.S.

But in February, potatoes were back on the menu after a fierce multiyear lobbying campaign by the National Potato Council. Nutritionist Marion Nestle and other progressive reformers called foul, denouncing the change. “Really?” Nestle scoffed. “I have a hard time believing that WIC recipients are suffering from lack of potatoes in their diets.” Several watchdog groups and the national WIC advocacy group opposed the change, too. “It’s disappointing that politics has trumped science,” Margo Wootan, director of nutrition policy at the Center for Science in the Public Interest, told reporters.

But did science really lose? In this case, the potato industry had science on its side, the outcry from nutritionists notwithstanding. Despite the common belief that potatoes are nutritionally null, a report released in February by the Institute of Medicine, an independent nonprofit, shows that white potatoes are an inexpensive source of potassium, fiber, and other needed nutrients, and one that people actually enjoy eating.


The takeaway from the potato controversy is not that lobbyists sometimes base their campaigns on real science. Rather it’s that the David-and-Goliath narrative of science versus Big Ag may be blinding us to another, even bigger problem: the fact that there is often very little solid science backing recommendations about what we eat.

Most of our devout beliefs about nutrition have not been subjected to a robust, experimental, controlled clinical trial, the type of study that shows cause and effect, which may be why Americans are pummeled with contradictory and confounding nutritional advice. Nutritional bad guys that have fallen from grace in the national consciousness—white potatoes, eggs, nuts, iceberg lettuce—have been redeemed years later. Onetime good guys, like margarine and pasta, have been recast as villains. Cholesterol is back in the probably-won’t-kill-you column after being shunned for 40 years, as of the latest nutritional advice from the Dietary Guidelines Advisory Committee in February. (That advice was still too timid, according to Cleveland Clinic cardiologist Steve Nissen, who also wants the nutritional guidelines to admit our best evidence suggests fat isn’t bad for you either). And then there’s salt—don’t eat too little, says the newest research. You could die. 

Amid the growing concern that nutritional advice to avoid high-fat food led to overconsumption of carbohydrates and caused spikes in illnesses like Type 2 diabetes, more and more scientists are starting to worry publicly that the basis of our dietary advice is scientific quicksand.

How potatoes were born again is a telling example of the uncertain foundation of nutrition. Potatoes had a bad rap in part because they are usually eaten deep fried, but even when they are not, they have a high glycemic index, according to prominent Harvard nutritionist Walter Willett. A high GI means that foods quickly turn into sugar (glucose) in the body and may eventually lead to heart disease and other illnesses, especially among diabetics, according to an analysis of data from the Nurse’s Health Study at Harvard, which Willett oversees. He placed potatoes in the same naughty group as candy in his influential 2005 book Eat, Drink, and Be Healthy: The Harvard Medical School Guide to Healthy Eating. One of his studies concluded that white potatoes are worse than soda, leading the L.A. Times to dub spuds public enemy No. 1 in 2011.


So how, just a few years later, did white potatoes return to the list of healthy foods? The IOM committee found a lack of evidence supporting claims that the tubers are unhealthy. WIC only pays for fresh produce, or frozen or canned fruits and veggies with nothing added, so there is no worry women in the program are using their vouchers for potato chips. No one contests the positives of potatoes—much needed micronutrients and fiber—and there isn’t enough proof, they found, that the GI is important to health. So white potatoes got the green light.

Willett calls the IOM potato report “myopic,” because other vegetables have the same needed nutrients as potatoes without the possible dangers of high GI foods that his studies have shown. And because micronutrient deficiency isn’t our biggest dietary problem, Willett told me in a recent email exchange, “obesity is likely to be of greater concern.” In other words, why take the risk on a possible downside to eating potatoes when you don’t need to?

Many nutritional studies are observational studies, including massive ones like the Nurses’ Health Study. Researchers like Willett try to suss out how changes in diet affect health by looking at associations between what people report they eat and how long they live. When many observational studies reach the same conclusions, Willett says, there is enough evidence to support dietary recommendations. Even though they only show correlation, not cause and effect, observational studies direct what we eat.

Apart from their inability to determine cause and effect, there’s another problem with observational studies: The data they’re based on—surveys where people report what they ate the day (or week) before—are notoriously unreliable. Researchers have long known that people (even nurses) misreport, intentionally and unintentionally, what they eat. Scientists politely call this “recall bias.”


The coupling of observational studies and self-reported data leads some observers to the conclusion that we know neither how Americans do eat nor how they should eat. A recent PLOS One article even suggests that several national studies use data that is so wildly off base that the self-reported caloric intake is “incompatible with survival.” If people had eaten as little as they reported, in other words, they would be starving.

Peter Attia, a medical researcher and doctor, started questioning the basis of dietary guidelines when he saw that following them didn’t work for his patients. They didn’t lose weight, even when they virtuously stuck with their diets. When he took a look at the research supporting the advice he was giving to his patients, he saw shoddy science. Attia estimates that 16,000 nutritional studies are published each year, but the majority of them are deeply flawed: either poorly controlled clinical trials, observational studies, or animal studies. “Those studies wouldn’t pass muster in another field,” he told me.  

Attia co-founded the nonprofit Nutrition Science Initiative. NuSI has about $40 million from the Laura and John Arnold Foundation to support clinical trials in nutrition rigorous enough to tell us what to eat. Its goal is to answer one fundamental question about how we should eat: Are low-fat or low-carb diets better for health?

The NuSI projects are not the first clinical trials in nutrition, and Willett pointed out to me that previous big and expensive trials have failed because people don’t follow their randomly assigned diet—and that is one key reason that the field has depended so heavily on observational studies. NuSI-funded trials try to correct for this problem: In one of the studies, subjects are isolated in a clinical residence for eight weeks at a time and every morsel of food they eat is controlled. In another, 150 overweight or obese subjects, all college students, faculty, and staff, eat every meal in a special dining hall. In the third study, participants aren’t isolated but are assigned to either a low-fat or low-carb diet and record what they’ve eaten immediately after they eat to sidestep recall bias. In two or three years, Attia estimates, results will be available.

Even if the participants in the NuSI studies do follow their diets perfectly, and even if researchers establish whether low-fat or low-carb is better, that won’t necessarily establish whether we should eat more or less of individual foods. What kind of carbs are good for you, and should potatoes specifically be avoided? Which vegetables should you be eating more of? It would take hundreds of clinical trials to come up with solid advice about which foods to eat and which to avoid—especially since genetic variations mean that different people have different nutritional needs.

In the meantime, should dietary recommendations end until we have solid clinical trials to show how to navigate our bewildering array of food choices? Should the government stop creating random purchasing restrictions for people who need help getting food on the table? One interesting bit of advice came in the many discussions following the IOM’s recent reversal on cholesterol—cardiologist Steve Nissen suggested that any recommendations come with a side of brutal honesty. “We are all guessing,” he said. “And we have to be more humble about telling people what we know and what we don’t know.”