Nutrition research is always big news when science discovers some new tidbit about what you should and shouldn’t eat. It drives the rise and fall of entire industries. You would never have even heard of an acai berry had not University of Arkansas (Go Razorbacks!) scientists discovered acai’s antioxidant qualities which caused these to appear on supermarket shelves.
In this post, we explore why these discoveries almost never apply to you.
On September 12 2016, the American Medical Association’s Journal of Internal Medicine published the results of an analysis of 50-year-old correspondence between researchers at Harvard (motto “veritas but not the whole veritas”) and the Sugar Association ( motto “You’re not fat. You’re big boned”) which showed that the sugar association paid for a study that minimized sugar’s role in heart disease and emphasized fat’s role.
At the time, science was pointing to both being a factor but the Sugar Association had its act together more than the National Cattleman’s Beef Association, the Dairy Council, Pork:The Other White Meat People and the Bacon-and-Ranch-Flavored-Chocolate-Covered-Cheese-Puff Institute. They put their money where your mouth was and for the next fifty years, we’ve been eating nothing but low-fat skittles.
To this day, it’s a fact that eating too much of anything causes heart disease. It doesn’t matter if its fat, carbohydrates or protein but between the Healthcare Industrial Complex and the Healthy Lifestyle Militia, you’re being told to eat a crapload of one of these nutrients because the others will kill you. Meanwhile, whatever we are eating, we’re eating too much of it, so nothing really changes. But I digress.
Nutrition Research – You Are Not “N”
When you read the primary source for any research reported in the media, you’ll consistently see a reference such as “N=1700”, or “N= 102″, or N=”8”. In these instances, “N” stands for how many subjects participated in the study. So if you read a study and find that N = 50, that means that whatever they were testing was performed on fifty people, fifty mice, fifty rats, fifty wombats, fifty rathtars, and so on. And because you were probably not a member of the esteemed population of “N”, you can only guess how the study’s results apply to you but you can’t be sure.
There are a bunch of reasons why it’s only a guess. Those reasons have to do with things that make you unique as well as the study design and implementation. For the rest of this post, I rattle off some of the problems with nutrition research. If you want to skip to the conclusion, nutrition research is flawed. But it’s the best we got when it comes to deciding what to eat for optimal health. Expect inconsistencies in what you hear and plug into the Karma Sense Wellness Media Empire to cut through the crap and apply the concepts in a way that works for you.
1. Study Funding Drives Results Bias
The aforementioned caper between the Sugar Association and Harvard isn’t an isolated incident. For a bunch of reasons, money to perform most kinds of scientific research is increasingly scarce. This leaves scientists beholden to commercial interests who at best are looking for research to find positive qualities and at worst are looking to sweep negatives under the rug,
Marion Nestle, professor of Nutrition at New York University (Go Bobcats!), no relation to the Big Food company of the same name, monitors industry sponsored research and tallies conclusions favorable to the sponsor. By her calculations, over 90% of industry-sponsored research is flattering to the people who paid for it. I’m surprised it’s not 100%. Not because the sponsors are right but because of the natural inclination for repeat business. Do you think McDonald’s would pay for additional studies if Brazilian researchers didn’t conclude that fast food meals are as least as nutritious as the beans, whole grains, fruits, vegetables and unprocessed meats that Brazilians typically eat? Do you think that’s true?
Dr. Nestle does a much better job of parsing this subject than I ever could so if you’re interested, follow her here. The point is, whether its food, pharmaceuticals, supplements, medical devices or shake weights, the legitimate health claims you hear are probably paid for by the people trying to sell you their tchotchkes.
2. The Study Says One Thing and the Headlines Say Something Else
We’ve covered this before when I debunked a headline that said that bacon causes cellular jet lag. The study the article referenced didn’t prove that. It had no intention of proving that.
The Karma Sense Wellness blog also recently addressed an article that claimed a new study proves drinking black soybean tea melts away the fat. The study neither tested nor demonstrated that.
The big food companies love to use this phenomenon to their advantage whenever the headline overreach is in their favor. So I find it funny that when I go back to the Harvard study on carbohydrate vs. fat, that the Sugar Associations responded as follows:
“Most concerning is the growing use of headline-baiting articles to trump quality scientific research—we’re disappointed to see a journal of JAMA’s stature being drawn into this trend.”
They then go on to quote a report from the FDA that says:
“Randomized controlled trials examining cardiovascular risk factors, body weight, inflammatory markers and risk factors for type 2 diabetes demonstrate no effects of increasing sugars intake.”
If I go to the source report that includes the quote, I see that it’s pulled from a context that says although there is no demonstrable effect on these specific and dangerous health conditions, it is still accepted that excessive added sugar is detrimental to general health. In other words, the FDA study didn’t say what the sugar association claims. Pot, meet kettle.
3. Long-Term Randomized Control Trials for Nutrition Research are Nearly Impossible To Do
I won’t bore you with the details of what’s required to execute a well-designed study to discover long-term effects of any food choice. But to really do it in nutrition, it involves subjecting an unmanageable number of subjects (N!) to years of isolation, highly controlled and boring meals with limited variety and elimination of hundreds of other confounding habits that the subjects find enjoyable. Plus, anyone placed in the control group would possibly be exposed to unhealthy and even dangerous conditions. Who is going to sign up for that? Who is going to pay for that? Probably not the Sugar Association. Maybe just rathtars.
4. Study Design is Hard to Dissect
Was the study done on people or other creatures? Cells or whole animals? Was it a case report, a cross-sectional survey, case-control study, cohort study, quasi-experiment, random control trial or meta-analysis? Did all subjects complete the experiment? Were subjects given whole foods or isolated extracts? How were the control subjects managed? How did the investigators eliminate confounding variables? All of these things impact the strength of a study’s conclusion. Do you ever hear about this? Of course not. I almost fell asleep listing all those questions.
5. Peer Review Doesn’t Work
We’re not supposed to have to worry about any of those questions in #4, thanks to a process called peer review. People with no prior knowledge of the study, but expertise on the topic, analyze the study design and results to assess whether the conclusion has merit. But the niches in which these experts operate are pretty small. If they don’t know THE gal or guy who ran the experiment, they know a gal or guy who knows THE gal or guy.
The peer review process is not fun. It gets contentious. Nerd fight! And it’s volunteer work. Scientists who have better offers such as their own research, exotic lecture tours, or unlimited pasta offers at the Olive Garden almost always opt for those. That leaves the less experienced and qualified to do peer reviews.
6. Junk Journals and Paywalls
There are three types of scientific journals.
There are legitimate ones that hide behind paywalls. These have research that is on the up-and-up but you have to be willing to pay to actually read them. This is a problem for any poor schlub like me who wants to parse the study design and conclusions but can’t because I only have access to the abstract and I don’t have enough dough to get to the fully published version.
There are open scientific journals that don’t hide behind paywalls but still have legit research in them. A great example is the Public Library of Science or PLOS. Hopefully, PLOS will do to paywall based science journals what Wikipedia did to Encyclopedia Britannica.
Then there are so-called predatory publishers. These sleaze bags have a pay-to-play model. Give them your credit card and they’ll publish your crap science in fake journals with fancy names like International Journal of Food and Nutritional Sciences and International Journal of Herbal Medicine. These are real journals with fake science. To prove this point, Christophe Bartneck from the University of Canterbury in New Zealand (Go TBD! Seriously), recently had his paper accepted for inclusion as part of the International Conference on Atomic and Nuclear Physics. The problem? He composed the paper by randomly selecting words from his smartphone’s autocomplete feature.
A dude named Jeffrey Beall from the University of Colorado, Denver (Go Lynx!), used to publish a list of predatory publishers but he recently pulled it down. Speculation is that publishers on the list were threatening lawsuits. You can find an archive here.
7. Cherry Picking
The best example of cherry picking is the great debate of low-fat diets vs. low-carb diets. It’s a fight that’s been going on ever since Marco Polo discovered macaroni and cheese at the neighborhood pool snack bar. Or wherever it was that he first found pasta.
Each side in the debate promotes the studies that support their point of view and ignore the studies that weaken it. What they all fail to acknowledge is that the real problem isn’t too many carbohydrates or too much fat. The real problem is too much crap. Too much crap combined with a little you are not “N.” Reduce overall consumption of ultra-processed foods and still eat the foods you love. You will end up healthier and happier in the long run.
8. Nutritional Dogma Speaks Louder Than Science
Let me repeat what I said at the end of the description for problem #7. Reduce overall consumption of ultra-processed foods and still eat the foods you love. You will end up healthier and happier in the long run.
The problem isn’t fat and it isn’t sugar. The problem is how much of each we eat. Science consistently demonstrates this. But through a confluence of influence, the Dietary Guidelines for Americans chose to pick on fat in the 1980s and to this day those guidelines still demonize fat. They’ve lightened up a little but the warnings are still there. The next version of the guidelines is due in 2020. We’ll see how far they move the needle then.
9. Nobody Promotes the Status Quo
If you go to a database like PubMed or Google Scholar and search the term flossing, you’ll find thousands of studies showing the benefits. Some of those studies are more convincing than other, but they’re there. Research demonstrating the benefits of flossing continues. But you never hear it in the headlines. That makes sense, it isn’t news. But as soon as one study comes out and casts aspersions on this sacred ritual, it’s all over the news. And that’s what people remember. That one study doesn’t cancel out the thousands that already existed and the thousands more to come.
Looking at the details of that research, it raises valid questions. But I’m still going to floss. I hope you do too.
10. You Are Not “N”
No matter how well research proves hypothesis X there’s no guarantee that those results apply to you.
My Love-Hate Relationship With Nutrition Research
Although I just gave ten reasons why I hate nutrition research, my real relationship is love-hate. The research isn’t perfect but it’s the best tool we have. Just remember that like most tools, when they’re put in the wrong hands, it’s easy to turn them into a weapon (WordPress in the hands of semi-literate nutrition bloggers excepted).