
Are research findings misrepresented by funders, PR machines and the media?
Loading summary
A
Hello and welcome to More or Less on the BBC World Service, the lighter way to enjoy statistics. I'm Robert Cuff. Christmas is coming, which means we'll be getting fat. But as we put down the biscuits only to pick up that 10th bar of chocolate, our guilt will will be eased by headlines like Chock Full of Benefits.
B
This is why you should be eating chocolate every day. For the sake of your brain and heart.
C
To improve memory, consider chocolate.
A
Drinking hot chocolate could prevent Alzheimer's. Well, this week we have bad news for your confectionery guilt. We're tracing how good research evolves into the crumbliest, flakiest headlines about chocolate's health benefits and asking whether industry funding has a role to play in the hype. Extravagant claims about chocolate are nothing new. It was originally advertised in the 17th century for its medicinal properties. Thirsted after by people of all degrees, especially those of the female sex, either for the pleasure therein naturally residing to cure and divert diseases, or else to supply some defects of nature. Modern researchers focus on one particular element in cocoa beans. Flavanols. Dr. Joanne Manson, a professor at Harvard Medical School, told us about the research to date.
D
Some trials have shown that they lower blood pressure, they improve glucose metabolism. Others show reduction in inflammatory markers. So they have a number of potential mechanisms and pathways that are relevant to reducing the risk of heart attacks, strokes. But they haven't yet been tested for their ability to prevent clinical cardiovascular events.
A
Like any statistician, my two favorite things are randomized controlled trials and chocolate. So when I heard about a trial testing whether it could slow my memory's decline, my middle aged heart skipped a beat. The research was conducted at the University of Columbia and funded by grants from the US government, a charity, and Mars, one of the world's largest manufacturers of chocolate. But when we read the paper, the findings weren't actually as bold as the headlines. The researchers took 37 people in their 50s and 60s, giving half of them a daily high dose Flavanol supplement drink for three months. People receiving the drink were quicker to recognize wiggly lines they had seen before in, well, in a lineup, showing changes equivalent to improvements in cognition by approximately three decades of life. This language did not go down well with Henry Drysdale, a doctor who does his research at Oxford's Centre for Evidence Based Medicine.
B
What that communicates to the reader is that you'll have the brain of someone 30 years your junior just by drinking a supplement. Now that is an astonishingly bold conclusion to draw from a very modest result in a very small study measuring an Unvalidated surrogate marker of cognition at 12 weeks follow up.
A
By the time the press office for Columbia got their hands on it, the headline became Dietary Flavanols Reverse age related memory decline. Henry was not impressed at this slip from recognizing artificial lines to real world memory.
B
This study doesn't measure any real world memory outcomes, but in the Columbia article they mention the word memory. I think about two dozen times they talk about real world aspects of memory, like remembering names where you parked your car. But this isn't what they measured in the study, so that claim is not supported by the evidence.
A
The final slip came once the newspaper headline writers wanted to swap boring sciencey flavanol in the university prior for something that people would want to read. The New York Times recommended that those who want to improve a memory consider chocolate. But the study wasn't looking at chocolate.
D
Chocolate is not synonymous with cocoa flavanols. It would take an extremely high number of calories in order to provide this amount of cocoa flavanols. It would take 700 calories in dark chocolate, 5,000 calories a day in milk chocolate.
A
That's a kilo of milk chocolate or 300 grams of dark chocolate. Imagine my delight when Lizzie, our producer, suggested a practical demonstration and my face when she told me that I would be trying the most efficient way to eat your daily dose of 100 grams of unsweetened cocoa powder. Slightly surprised by the small print on the golden ticket that secured me this presenter slot, I complied with predictable results. So here goes. So I only got about half of it in. It's so dry, it feels like it's sucking all of the water out of my head. Oh my gosh. So I tried to eat 4 grams there. So one of the 25 spoons I'd have to take. There is no way that's happening. Back to the story. We can have a great time poking fun at one over egged study that was funded by industry. But are we picking on the weakest kid in the class? Well, Julie Beloz, the senior health correspondent@vox.com has examined the literature since the 1980s.
E
We're looking at the results of the research that's been sponsored or funded by some of these big chocolate makers. We found that they overwhelmingly drew conclusions that were favorable to the candy makers. You can't help but think that they're doing that to both better understand their chocolate and perhaps generate the kind of hype and press we read about in newspapers. It feels like on a weekly basis under headlines like Chocolate Fights Alzheimer's so
A
is all chocolate research junk. Back in Harvard, Joanne Manson is running the Cosmos study, looking at 20,000 people and examining the effect of flavanol supplements on heart attacks, strokes and major clinical events.
D
It is expensive to do a trial with 20,000 participants. Industry can help to reduce the expense for taxpayers in helping to support the trial.
A
But a major review of medical research showed that studies funded by industry are more likely to reach positive conclusions. Should we be worried about commercial sponsors manipulating research?
D
We're designing the study, we're conducting the study. The analyses are being done totally independently of industry and the results will be published regardless of whether they are favorable, unfavorable or totally neutral.
A
But there's so much at stake when the results could affect profits and chocolate is big business. Mars alone brought in $37 billion last year.
B
I would argue that actually academic incentives such as you want to get your next grant or you're hoping to get tenure at your university and you need more big exciting papers to back you up, I would argue that those academic incentives can be just as influential and can introduce just as much bias into the results themselves.
A
Julia agrees about the quality of research, but suspects that the bias works in a different way.
E
Not all of these studies are bad. Some of them are very high quality science. They've passed peer review, they've been published in top tier journals. It's more of a question of what types of lines of inquiry are being pursued. A lot of the studies were about very specific short term health questions. They're not asking questions that might lead to unfavorable conclusions. And they're not asking questions like, you know, what role has chocolate played in our obesity crisis?
A
And practicing researchers like Joanne and Henry know the difference between research that is real world and pure science. The difference between remembering where you left your keys and recognizing unusual wiggly lines. Their advice is to check whether the story is about a big randomised study that measures a real life event in people like you. But outside the scientific community, we're not always so sharp to these distinctions and that can be a sweet, chocolatey recipe for confusion. That's all we have time for. Thank you to Dr. Joanne Manson, Dr. Henry Drysdale and Julia Bellutz. If you spot a claim anywhere in the world that you think we should be taking a look at, please get in touch. Our email is more or lessbc.co.uk and we're on Twitter bcmoreorless. The show was presented by me, Robert Cuff, and Produced by Lizzie McNeil, whose allergies conveniently prevent her from tasting cocoa powder. Thanks for listening. If you like this show, you may also like this other podcast from the BBC World Service.
C
Hello. It's said that the unexamined life is not worth living. But what if you've got a question about life or the universe or what lies beyond the universe and the Internet just isn't giving you a satisfying answer? Well, crowdscience might just be the podcast to help you out. Every episode we take one of your science questions and travel the world or wherever to find the answers. It doesn't have to be epic. Maybe you just want to know ways to stop losing your keys every day. Or why is it so hard to wake up in the morning? Our podcast listeners keep their science queries coming and we love answering them every week. Join the gang by downloading our podcast. It's called Crowd Science. And our latest question comes from Rachel in Scotland. My question for this week's crowdscience is
A
can we resurrect the dodo?
C
Bring back the dodo. As it turns out, this is no walk in the Jurassic park. But might science one day revive long dead species? To find out, you're going to have to search for crowdscience or one word wherever you find your podcasts.
Podcast: More or Less, BBC Radio 4
Date: December 10, 2017
Host: Robert Cuff
Theme: Examining the truth behind headlines claiming chocolate has brain and heart health benefits, especially whether chocolate can "make your brain younger." The episode also explores the role of industry funding in shaping research.
This episode investigates scientific claims about chocolate's supposed health benefits, specifically regarding brain and memory enhancement. Host Robert Cuff breaks down how rigorous scientific findings morph into exaggerated media headlines and questions the influence industry funding might have on research and public perception. A mix of skeptical inquiry, expert commentary, and lighthearted demonstration illustrates the gap between sensational news and real research.
"Some trials have shown that they lower blood pressure, they improve glucose metabolism. Others show reduction in inflammatory markers... But they haven't yet been tested for their ability to prevent clinical cardiovascular events."
Study Design: 37 people aged 50s–60s received either high-dose flavanol drink or a placebo for three months.
Results: Those who took flavanols performed slightly better at recognizing visual patterns (wiggly lines), an "unvalidated surrogate marker" for memory.
Media Leap: The Columbia press release inflated the results to "Reverse age related memory decline," despite only minor, very specific improvements ([03:19]).
"What that communicates to the reader is that you'll have the brain of someone 30 years your junior just by drinking a supplement. Now that is an astonishingly bold conclusion to draw from a very modest result in a very small study measuring an unvalidated surrogate marker of cognition at 12 weeks follow up."
"This study doesn't measure any real world memory outcomes... they talk about real world aspects of memory, like remembering names, where you parked your car. But this isn't what they measured in the study, so that claim is not supported by the evidence."
"Chocolate is not synonymous with cocoa flavanols. It would take an extremely high number of calories in order to provide this amount of cocoa flavanols. It would take 700 calories in dark chocolate, 5,000 calories a day in milk chocolate."
"It's so dry, it feels like it's sucking all of the water out of my head..."
"We found that they overwhelmingly drew conclusions that were favorable to the candy makers. You can't help but think that they're doing that to both better understand their chocolate and perhaps generate the kind of hype and press we read about in newspapers."
"Industry can help to reduce the expense for taxpayers in helping to support the trial."
"The analyses are being done totally independently of industry and the results will be published regardless of whether they are favorable, unfavorable or totally neutral."
"I would argue that actually academic incentives such as you want to get your next grant or you're hoping to get tenure... can introduce just as much bias into the results themselves."
"It's more of a question of what types of lines of inquiry are being pursued. They're not asking questions that might lead to unfavorable conclusions...like, you know, what role has chocolate played in our obesity crisis?"
"An astonishingly bold conclusion to draw from a very modest result in a very small study measuring an unvalidated surrogate marker..."
"It's so dry, it feels like it's sucking all of the water out of my head. Oh my gosh. So I tried to eat 4 grams there. So one of the 25 spoons I'd have to take. There is no way that's happening."
"We found that they overwhelmingly drew conclusions that were favorable to the candy makers."
"The analyses are being done totally independently of industry and the results will be published regardless of whether they are favorable, unfavorable or totally neutral."
This episode of More or Less uses the case of chocolate and cognitive health claims to illustrate how nuanced research gets transformed into overly optimistic headlines, often fueled by industry funding and media simplification. Expert guests urge listeners to look beyond the hype—scrutinizing the scale, outcomes, and sponsors of health studies before drawing conclusions.
Not sure whether chocolate is making your brain younger? This episode suggests you might want to check the research… and don't believe everything you read on holiday chocolate boxes!