Loading summary
Transcript140 lines
- [00:00]
Helen Pearson
This is an iHeart podcast, Guaranteed Human
- [00:04]
ChatGPT for Business Sponsor
Today's episode is brought to you by ChatGPT for Business. As a listener of this podcast, you're looking for ways to help teams move faster, make sharper decisions, and turn scattered context into work they can use. ChatGPT for Business can help. ChatGPT for Business gives teams a shared workspace with admin controls, permissions, and access to work and codecs. In ChatGPT, this means your business can move from question to answer and code to rollout quicker. Join over 10 million business and enterprise users worldwide already using ChatGPT for work. Download the ChatGPT desktop app or contact Sales to learn more.
- [00:42]
Jacob Goldstein
Did you ever notice how you spend hours shopping online only to pause at checkout because you wonder if you trust it enough to hit buy? Now Agentic Commerce is testing that moment more than ever. That's where PayPal comes in. With 25 years of checkouts, 400 million consumer accounts globally, and the benefit of fraud protection. So no matter where a purchase starts, it ends with trust. Built for payments growth and agentic PayPal open built for all business.
- [01:11]
J.R. Martinez
Visit PayPalOpen.com this is JR Martinez for Medal of Stories of Courage. This episode is brought to you by Navy Federal Credit Union. At Navy Federal, they want you to know that if you need a positive sign towards achieving your goal of home ownership, this is it. Because even though today's housing market can seem discouraging, their Home Buyer's Choice Loan offers members down payment options as low as 0% with no required private mortgage insurance. Look for your sign today. Navy Federal Credit Union. Navy Federal is insured by NCUA Equal Housing Lender. Terms and conditions apply. Loan subject to approval and eligibility requirements. Learn more@navyfederal.org Zerodown.
- [01:56]
Jacob Goldstein
Pushkin. Jacob I'm Jacob Goldstein, and this is what's yous Problem? My guest today is Helen Pearson. She's a science journalist, she writes for the journal Nature, and she recently wrote a book called Beyond Belief How Evidence Shows what really Works. Helen's problem is how do you get people to make important decisions based on high quality evidence rather than hunches or habits? To a large extent, the book is a history of what Helen calls the evidence revolution, and this is a remarkably recent revolution in her telling. Fifty years ago, she writes, doctors did not typically try to sort through, you know, the high quality published studies to figure out what they should do. Instead, they often did whatever they'd been taught in medical school or what the senior doctor in the room said to do, even if the evidence said Otherwise. But a few doctors spent their careers trying to change that, sorting through all the evidence to figure out what worked best in different situations and trying to get their fellow doctors to pay attention. In more recent decades, that spirit has started to spread to other fields, fields like management and crime prevention and education. Later in the conversation, Helen and I talk about some of those fields, places where the evidence is still often unclear and where people often ignore the evidence even when it is clear. We also talk about how she uses evidence in her own life and about the prospects for the spread of evidence based decision making in the world. To start, we talk about one doctor who did a lot to get the evidence revolution going in medicine. His name was Ian Chalmers.
- [03:50]
Helen Pearson
Ian is really the reason I ended up writing the book. He was training to be a doctor in the kind of late 1960s, early 1970s. He was working in obstetrics and gynaecology in Wales at the time. And he noticed this sort of odd, which was that when doctors would hand out advice to women with the same condition, they would give different advice. So, you know, some doctors would think it was really important to have prenatal vitamins or not to drink during pregnancy, and other doctors would say no such thing. And this was kind of so extreme that Ian once told me that when he was woken in the night to treat a woman, the first thing he would say was, what's wrong? And the second thing he would say was, who's treating her? Because that was going to change, you know, what he did next. Right. So what genuinely happened was that everyone followed the advice of the most senior doctor in the room, which is sometimes now called eminence based medicine.
- [04:42]
Jacob Goldstein
Tell me about his work in Gaza and how that influenced him.
- [04:48]
Helen Pearson
Yeah, so that was another part of his training which was also kind of influential in him realizing that a lot of what he'd been taught in medical school was wrong. So he worked on a refugee camp in Gaza during his medical training, and during that time he was treating children who had measles. And he had been taught in medical school that you shouldn't give antibiotics to a child with measles, even if they were becoming quite sick, because antibiotics treat a bacterial infection, so why would you give it to a child with a viral infection? So he followed the advice he'd been taught and he didn't give the antibiotics. But then later, when he returned to the uk, he discovered that actually this advice he'd been given was tragically wrong, and that there were in fact about six controlled clinical trials that suggested it did help to give some children antibiotics prophylactically because they were developing a secondary bacterial infection along with the viral one.
- [05:44]
Jacob Goldstein
And in fact, as I understand it, like, he concluded when he went back to the UK and studied the advice, that children under his care had probably died or possibly died, because he had done the wrong thing.
- [05:56]
Helen Pearson
That's exactly right, yeah. So a huge kind of turning point for him of realising that.
- [06:02]
Jacob Goldstein
So let me ask the naive question. Why had he been taught something that was not consistent with the evidence?
- [06:10]
Helen Pearson
Because that was standard practice in medical schools at the time. Most doctors weren't really being taught about clinical trials in the 60s and 70s. Hard though it is to believe.
- [06:21]
Jacob Goldstein
It is weird, right? It is hard to believe. I mean, I guess is part of it the sort of formal, you know, randomized controlled trial where you have a placebo arm and a treatment arm and the people running the trial don't know which is which. That was actually relatively new still, right?
- [06:36]
Helen Pearson
Exactly. Yes. So we feel like randomized trials have been around for ages, but they are relatively new. I mean, around the time that Ian was having this realization that he hadn't been taught about trials, they were really only then becoming commonplace.
- [06:50]
Jacob Goldstein
So you have this sort of research changing and the practices of doctors are lagging. That is sort of the moment when Ian Chalmers walks onto the scene and it's this moment when there is now research, but doctors, weirdly, to our sensibilities, are not paying attention to it. What does Chalmers do when he realizes this?
- [07:15]
Helen Pearson
So what he did was he became very passionate about randomized trials, actually. And I discovered this with other people I spoke to, is that there's something about this method which is very captivating to people.
- [07:28]
Jacob Goldstein
It's so elegant, right? Like, intellectually, like it's blinded. There's a placebo control. Like, I recognize that they're imperfect and people love to talk about the problems with them, but it's wildly elegant and not intuitive. Right? People want to just give something to somebody and see if they get better and say it works, if they get better. And for so many reasons, a randomized trial is so much better than that.
- [07:52]
Helen Pearson
And that argument goes on all the time today. I mean, that's what mostly happens in policy, is we just introduce policies, right, without bothering to test whether they're effective and assuming they do good when some do harm. So I don't think that's an old argument.
- [08:05]
Jacob Goldstein
We will get to policy. Yes. So this is this moment people are falling in love with randomized trials. And more importantly for our story, Ian Chalmers is is realizing that this evidence is not making its way into medical practice. What does he do about that?
- [08:20]
Helen Pearson
So he decides to do this pioneering exercise where he is going to collect all of the controlled clinical trials. They weren't all randomized, but controlled trials which had been done, which had tested treatments in pregnancy and childbirth. And at the time. So this is in the 80s. I mean, that was an extraordinarily difficult thing to do because he couldn't just go online and search up these trials. So he eventually ended up recruiting hundreds of people to sort of go to libraries and scour through the medical journals and really exhaustively try and track down these clinical trials.
- [08:56]
Jacob Goldstein
It's amazing that it's almost an archeological exercise. Right. Like people have generated all this evidence not in ancient times, but in the last 10 or 20 years, and it's already been sort of lost. Right. He is an ob gyn. Right. And he's having to have people go look for the research in his field.
- [09:14]
Helen Pearson
Exactly, exactly. It's extraordinary. And that is still a real difficult problem today, which I think is absolutely mind boggling that a lot of research is published and yet people still struggle to find it on databases.
- [09:25]
Jacob Goldstein
So they're doing this archeological expedition to look for recent research and what happened.
- [09:31]
Helen Pearson
So it takes him about 10 years, him and his many colleagues, and eventually he doesn't just get the evidence together. I think this is what's so interesting about it, is he realizes that he needs to synthesize the evidence and not just to look at one trial at a time. Because if you look at one trial, it risks being wrong. Or you get a group of trials which may be conflicting, maybe some of them are poorly done, maybe it's just through chance that they point in different directions. And so he took this quite radical step of synthesizing the evidence using a method called a systematic review, and ended up doing hundreds of systematic reviews to look at all of these different approaches in pregnancy and childbirth, and eventually publishes all of this after 10 years in these two massive books which I also have on my bookcase behind me. So they revealed that a lot of practices in pregnancy and childbirth were not based on evidence or were actually doing harm. So, for example, even things like invasive episiotomies, which is a surgical incision which is supposed to give babies a bigger hole to get out, there wasn't really evidence to support these practices. And like shaving women's pubic hair in labor. So countless women had been subjected to these really quite invasive and degrading practices because doctors assumed they were the Right thing to do. And Ian's work by synthesis, synthesizing evidence in this way, helped bring an end to them. So he was sort of inspired by what he had done and thought, why can't we do this across all specialties of medicine, you know, synthesize the evidence in heart disease and diabetes and cancer and so on. And that led to this group which he helped found, he was seminal in founding, called the Cochrane Collaboration. And they set out and are still trying to do this today, to synthesize evidence which shows which treatments are help and do not harm across these different groups. And nowadays the word sort of like a Cochrane review, if you ever hear that, it's sort of like associated with doing the kind of highest standard, gold standard type of review of evidence.
- [11:40]
Jacob Goldstein
And I will say a lot of it at least is just online, right? Like you can go look if you're thinking of taking a supplement, say you can just go look. And there are a lot of Cochrane reviews of whatever, magnesium for something, something. And I find it useful. I mean, quite often they say there's not enough evidence to say one way or the other.
- [12:04]
Helen Pearson
Right.
- [12:04]
Jacob Goldstein
That's typically with the kinds of things that I'm looking at, you know, over the counter things or whatever. There just isn't enough research to have a clear answer. And that is what they often say, in my experience.
- [12:15]
Helen Pearson
It is often the conclusion. And I mean it's not just so. So Cochrane reviews now are a tiny fraction of the number of systematic reviews in the world. So Cochrane was really sort of seminal in sort of showing that a really good way to make sense of conflicting bodies of evidence. But now thousands are coming out every year. So in general, a systematic review is a better way to get an overview of what the evidence says than looking at just one study. Now, of course it's not completely straightforward because like randomized trials, some systematic reviews aren't very good either. So that's a challenge too.
- [12:46]
Jacob Goldstein
Well, it's always something. Yes. Well, I mean, Cochrane is useful as a brand in that way, right? Like I trust their ability to assess the evidence, even if what they say is we don't know, like we don't know. It's a very respectable answer. As a general matter, what's Ann Chalmers like?
- [13:07]
Helen Pearson
Quite extraordinary actually, just the most. I don't think I've ever met anyone who is so extraordinarily modest about his accomplishments and downplays them so much to the point where he denies sort of starting the Cochrane Collaboration. I think he says something along the lines of he came up with the idea. So. And also very exacting and critical. I mean, it was interesting when he retired or decided to step away. He didn't retire from Cochrane, he decided he wanted to move on. He actually set up a prize for the group that offered the most astute criticism of the Cochrane Collaboration.
- [13:42]
Jacob Goldstein
Oh, that's really admirable.
- [13:43]
Helen Pearson
Yes, exactly. And they still have that attitude of kind of inviting criticism because of course that's exactly what he and the group have done all these years, is sort of speak truth to power. So they need to invite that criticism themselves.
- [13:57]
Jacob Goldstein
Who won the prize and what did they say?
- [14:01]
Helen Pearson
I don't know. It's one every year.
- [14:03]
Jacob Goldstein
Oh, it's one every year, yeah.
- [14:04]
Helen Pearson
Yeah.
- [14:04]
Jacob Goldstein
I could imagine him having a pretty high tolerance for disagreeableness, say. Right. I mean, what he did was in a sort of cultural way, in terms of the culture of physicians, the culture of medicine. I don't know, antagonistic is maybe too strong a word. But he said things that presumably lots of people didn't want to hear. He said to doctors, doing it wrong, both in a specific way, you shouldn't be doing this particular intervention, and also in a general way, like he implicitly or explicitly was saying, you're thinking about the practice of medicine. Wrong.
- [14:39]
Helen Pearson
That's right. That's exactly right. I discovered this as I sort of was reporting the book and kind of speaking to these people who were quite seminal in bringing evidence based practice into their fields. They often had this sort of rebellious streak to them and are kind of willing to speak truth to power because, yes, it does. I mean, you're basically saying to people you know, or to doctors, you've been doing it wrong all these years. And he, he himself said he thought he was able to do that because he was sort of. He was a bit of an outsider actually, like. Like he worked in Oxford, but he wasn't sort of part of all the kind of, you know, highfalutin academic circles. He was a bit on the outside and he felt that that allowed him to maybe do this. Speaking of, of truth to power, which maybe others were unable to do.
- [15:25]
Jacob Goldstein
So where's medicine today? To what extent is medicine today evidence based?
- [15:31]
Helen Pearson
Well, more than it was then, I think. I used a figure in my book of about sort of 60% of health care is probably evidence based, or at least draws on evidence based clinical guidelines. So nowadays what happens is there are typically systematic reviews of clinical trials or other evidence, and on the back of that, then professional groups prepare clinical guidelines and the clinical guidelines are what might be used in the clinic to help actually make a decision.
- [16:01]
Jacob Goldstein
Right. So like the national association of Cardiologists will have guidelines on whatever, who should get statins or who should get what interventions.
- [16:09]
Helen Pearson
Exactly. And I mean, that's quite interesting because it means that the evidence is kind of baked into the processes, which I find quite interesting. It sort of made it. I mean, I wouldn't say it's easy, but it is relatively easy for doctors now to use evidence because that's the expectation. Whereas I'm sure we'll get onto this. But you look at other fields and that's just not the case. But of course there are huge challenges. You know, lots of criticism still of evidence based medicine. Some people criticize, for example, you know, this sort of special place that the randomized control trial has and point out that randomized controlled trials can be wrong or poorly done. And then of course, you know, pushback in the world against evidence based practices anyway and people wanting to go their own way. So in no way is the kind of evidence revolution that I write about, you know, obviously done. It's evolving, it's new in some way.
- [16:57]
Jacob Goldstein
So if you say it's 60% evidence based for the other 40, I could imagine two things. Right. Clearly there are a lot of settings like medicine is complicated, people are complicated, the body is complicated, lots of sick people have lots of things wrong with them. So you can imagine lots of cases where there just isn't evidence, where it's just not clear, what should I do for this patient right now? Well, there's lots of different conflicting things that I could do and there are trade offs. The other piece is there is evidence and people just don't follow it. So, like, how much of each of those is there?
- [17:27]
Helen Pearson
Yeah, so I can't give you a percentage on each of those. But I mean, just to speak to the first one though, I mean, what's really interesting about evidence based medicine is that evidence alone doesn't tell you what to do. So evidence is one part of the decision making process. So, and I think that's a common misunderstanding actually around it is that like you look at the evidence and it will tell you, you know, whether should take this drug. But that's not the case because what a doctor or whoever is, you know, a doctor, hopefully in patient together are doing is saying, okay, you know, this is the evidence on what treatments are effective. This is the clinical expertise of the doctor who absolutely, you know, they've built up all this expertise over the years. We're not throwing that out the window. And then you as a patient have values and preferences which have to feed into this decision. Right. If you've decided this isn't a treatment path you want, then that has to be respected. So. But that's still evidence based medicine.
- [18:15]
Jacob Goldstein
Yeah, that's still in your 60%.
- [18:16]
Helen Pearson
That's still in my 60%. That's an easy one.
- [18:19]
Jacob Goldstein
Tell me about the 40% that's not evidence based. What's going on there?
- [18:23]
Helen Pearson
Well, in some places evidence based medicine is less common or accepted. So people might choose traditional Chinese medicine, for example. So there may be other practices around medicine which people prefer. In some cases, you're absolutely right, the evidence isn't really there or it might be misapplied. So there's the kind of all the human factors that come into it where people may through choice or inadvertently not follow the the evidence based path.
- [18:55]
Jacob Goldstein
We'll be back in just a minute.
- [19:07]
ChatGPT for Business Sponsor
Quick one before you jump back in, you're listening for ways to help teams move faster, make sharper decisions, and turn scattered context into work they can use ChatGPT for business can help ChatGPT for business gives teams a shared workspace with admin controls, permissions and access to work and codecs. In ChatGPT, this means your business can move from question to answer and code to rollout quicker. Join over 10 million business and enterprise users worldwide already using ChatGPT for work. Download the ChatGPT desktop app or contact sales to learn more.
- [19:42]
Jacob Goldstein
Did you ever notice how you spend hours shopping online only to pause right before you hit buy? Now that tiny hesitation, the one where you wonder if it's trustworthy, can make or break the sale. Now, with AI changing the way we discover and compare things, that split second trust question matters more than ever. That's where PayPal comes in. For more than 25 years, PayPal has been a leader in online payments, and now they're at the forefront of agentic commerce, making it work for businesses, letting them maintain control of their brand and their customer relationships. So even as the way we shop changes, the moment that matters most still feels familiar and deeply dependable. Built for payments, growth and agentic PayPal open built for all business visit PayPalOpen.com
- [20:33]
Laura Maxwell
to get started Working as a dedicated Fidelity advisor, I always ask what's important to you. I'll look at those priorities and goals alongside your full financial picture to build a comprehensive plan designed to help grow and protect your wealth. With proactive portfolio insights and an eye on taxes, I'm here to help you make the most of your money I'm Laura Macro and I'm a dedicated Fidelity Advisor. Learn more@fidelity.com Wealth Investment Minimums Apply Fidelity Brokerage Services, LLC Member NYSE SIPC
- [21:06]
Jacob Goldstein
okay, so let's talk about evidence outside of medicine. A big chunk of your book is that, and it does seem to some extent at least that this evidence based revolution, as you call it, that Ian Chalmers helped us start, has directly influenced people in other fields. You know, and you, you write about economics and crime and management and parenting. And you know, I will say, like I covered economics for a long time and in economics they've gotten better about evidence. It seems to me that in a lot of fields outside of medicine, it's just frustrating. It's just frustrating. When you sort of try and study the evidence, it seems like there's often not good evidence. When there is, people don't follow the evidence. I mean, in a broad, broad way. What's your assessment of how evidence as a way of looking at the world is doing outside of medicine?
- [22:01]
Helen Pearson
So I, I think it really depends which field you're looking in, of how. Like I, I suppose I ended up thinking that each of these fields was, you know, weaving its own path through evidence and some of them had directly looked at medicine and gone, yeah, we want to do that too. Like we, we're, we're, we think our practices shouldn't be based on conventional wisdom and anecdote. And in others, like in economics, for example, there's kind of, you know, evidence that the use of randomised trials has really swept through international development, through development economics. But yeah, I think, I don't think any of them have advanced as far as medicine. And I think that that is partly because of the histories of these fields and the people who've been championing it and the different barriers that it's faced.
- [22:45]
Jacob Goldstein
It seems harder in most other domains to do useful studies that have real external validity where you can look at something in one place and say this same intervention is going to work in other places. The world is messy is a fundamental problem with trying to generate evidence.
- [23:05]
Helen Pearson
Yeah, it makes things really, really hard. So let's say you want to, I don't know, test whether giving people incentives to have vaccines works. And I mean, I talked about a trial in the book that was done where in India people were offered steel dinner plates and bags of lentils in order to incentivise parents to come to clinics to have their children vaccinated. And it seemed to be effective. Right now you can't just lift and shift that and put it into the uk you have to think about the different cultures and the different ways incentives might work and so on and so forth.
- [23:37]
Jacob Goldstein
I mean, that particular example, like you do see conditional cash transfers in other countries where, you know, if parents bring their children for vaccination and send them to school, then they get a monthly stipend from the government. Like those seem to work like those. That one does seem to generalize at least some.
- [23:55]
Helen Pearson
Yes, it has. I mean that's built up. That's unusual really, in having such a body of evidence to it. That's built up over time.
- [24:01]
Jacob Goldstein
Although it's funny like that one, I understand that we shouldn't rely on common sense is one of the themes of the book. But like if you give people money to do something that is actually helpful to them, they will do it is basically what that finding says. And it's true.
- [24:15]
Helen Pearson
Right. But it took evidence to convince policymakers of that.
- [24:18]
Jacob Goldstein
Right? Yeah, fair.
- [24:19]
Helen Pearson
So the counterargument, you know, you're absolutely right that this point about generalization is like a really big problem across all of these fields. Right. That what works in one place might not work in another. Now some people would say, well, why bother, you know, well, why bother getting, getting evidence at all? Like what's the point of testing it? But you generally, over time, you know, if, if something it might not work exactly the same, but if you start to build up a body of evidence that shows, yeah, actually this seems to like the conditional cash transfers have some effectiveness here and some effectiveness here, then you've come up with a robust policy that does seem worth pursuing.
- [24:54]
Jacob Goldstein
Tell me about evidence based policy making for like government policy, social policy, like what. How has the sort of march of evidence been there?
- [25:05]
Helen Pearson
So I think in line, I concluded in the end that there was a kind of what I call the evidence zeitgeist around the sort of 1990s to maybe 2010 when a lot of these fields were turning to evidence. And I think it's because these ideas of evidence based medicine were percolating out and people were coming across them. And then you see this kind of emerge in the policy language. So you know, we had, for example, Tony Blair and New labor were talking about evidence based policy or doing what works as it was called in the 1990s. Obama talked about it in his inauguration speech in 2009. So it sort of was filtering into policy and you know, particular people became very enamored by it. I mean, I would say it sort of, it succeeded in certain pockets, shall we say.
- [25:52]
Jacob Goldstein
And during that time People are championing it. Are you seeing policy changes based on evidence? Are they actually doing the thing they're talking about?
- [26:03]
Helen Pearson
In some cases there were, yes. So that there was a series of what were called, for example, in the Obama administration, a series of what were called tiered evidence initiatives which came in where in order for policy to progress, it was only allowed to, for example, have extra funding if it was able to produce evidence of effectiveness.
- [26:21]
Jacob Goldstein
Are there examples of things that got cut because they were sort of proven to not work? That's the real one, right? Are you going to say, hey, this doesn't work so we're going to stop doing it?
- [26:31]
Helen Pearson
That is much harder to find examples of, I would say.
- [26:34]
Jacob Goldstein
Yeah. So what are examples of things that got funded because there was evidence that they were good?
- [26:40]
Helen Pearson
One example was the Nurse Family Partnership, which is a program where first time mothers are given a lot of support in the early time of pregnancy and in the first two years of life. And because there was a lot of evidence that this really was helpful towards child outcomes that progressed and was awarded more government funding.
- [27:01]
Jacob Goldstein
And what's the status of that program now?
- [27:03]
Helen Pearson
Still going.
- [27:04]
Jacob Goldstein
Oh, still going. I didn't expect that. I'll admit in my jaded heart. I thought you were going to tell me it got killed. That's in the U.S. yep. Okay, well, that's good news. Tell me about. You have a chapter about evidence based management. The evidence seems particularly weak in that domain.
- [27:28]
Helen Pearson
Frankly, I think some of the evidence is weak and I think the culture of using evidence is very weak. So of the fields I looked at, I would say that that is the one where I felt evidence was struggling the most. I mean, there is a surprising amount of evidence on effective management practices and a lot of people are kind of really sort of surprised to discover this. Like you can, you know, find doctors who are absolutely evidence based in everything they do, but they think that maybe management is some type of, you know, magic or art or something. So yeah, there's evidence on, you know, whether feedback works or, you know, whether performance appraisals work, which by the way, they don't seem to be all that effective at improving performance, which I found quite interesting.
- [28:12]
Jacob Goldstein
Yes, I'm overdue. The performance appraisal software keeps bothering me that I'm overdue. So if you're telling me it's a waste of time, you're validating my behavior and I'm happy to hear it. So what does work tell me like one management thing where there's like really clear evidence that is actually Actionable, to use a management word that's like, here's a great thing and you should do it if you're a manager.
- [28:39]
Helen Pearson
So goal setting is very effective. And good feedback. Good feedback meaning fair feedback. Feedback which is specific to people, not generic feedback. So there does seem to be good evidence that those things are effective at improving performance.
- [28:57]
Jacob Goldstein
Fair enough. Why do I find that unsatisfying? Maybe because it seems obvious.
- [29:02]
Helen Pearson
Yes. It seems odd. No, but that's a really good point. So that, to me is one of the things that holds back evidence based management, because some of the principles which are supported by evidence feel obvious to people and they're not new. And what sells business book is exciting new ideas which aren't necessarily tested or proven to work. So there's financial incentives sort of working against evidence based management, in my view.
- [29:27]
Jacob Goldstein
You've been an editor and a writer on the sort of magazine side of the journal Nature. Right. And I was a newspaper reporter before. And I mean, I was thinking about evidence in our profession, right. And I don't mean evidence in the sense of like you want to write about what's true or you want to assess the evidence when you're writing about something. I mean, evidence for the way one should write or what kinds of stories we should do or what headlines we should write. And in my career there has been none of that, essentially. I mean, you had the Internet come along. I'm old enough that I remember when I was relatively new and people, you know, you had a dashboard and you started to see, oh, people are reading these kinds of stories and not reading those kinds of stories. And in a sort of quasi anecdotal way, you might shift course, but nobody wanted to use evidence. I mean, not only did people really not think that way, but to the extent they did, they didn't want to think that way. Right? Like, we like writing the kinds of stories we write. I learned some particular craft of podcasting that was like one guy who was really good at making podcasts we thought told everybody how to do it, and we did it, right? Eminence based podcasting. And then all these other podcasts come along that are nothing like the rules that I learned and are wildly popular. And I'm like, oh, the rules that I learned were just what one guy made up and we all liked. But I haven't really changed much based on that. I mean, do you use evidence in your craft?
- [30:53]
Helen Pearson
That's such a good question. I mean, really what you're talking about is using data, right, to inform what you do I think, well, you could
- [31:03]
Jacob Goldstein
do a randomized control trial. I mean, certainly a B testing of headlines. That is a thing people do.
- [31:09]
Helen Pearson
Yes, exactly.
- [31:10]
Jacob Goldstein
But you could do a randomized trial of different kinds of articles in nature if you wanted. Right. You could prospectively decide we're going to randomly assign somebody to do A or B or whatever. As far as I know, nobody's ever done that.
- [31:26]
Helen Pearson
That's a great point. Yep. Absolutely. Yeah. Mostly people are chasing audiences, but even
- [31:32]
Jacob Goldstein
analyzing the best way to chase audiences, like, why don't we do that?
- [31:36]
Helen Pearson
We should. Let's do it.
- [31:39]
Jacob Goldstein
But do you mean that? I feel like I don't wanna. At some level. Right. This is like part of the. It's like I like doing my job the way I want to. Like. Yes. I also would like to have more listeners and perhaps if I was better, you know, more analytical or more evidence based, I would have more listeners. But I don't know. I think there's something going on there that is interesting and that points to resistance to evidence. Like, I think of myself as an analytical person. I think I follow evidence in healthcare, and yet in this thing I do all day, every day, I sort of don't pay attention.
- [32:09]
Helen Pearson
Well, it depends what outcome you want. Right. I mean, if your outcome is very clearly that you want to grow listeners, then you might be incentivized and to do that experiment, if you're happy with the outcome and no one's being harmed by it, which I don't think they are, if you continue to podcast in the way you do, or I tell stories in the way that I do, then, you know, there has to be an incentive, I think, to do the test.
- [32:30]
Jacob Goldstein
How did writing the book change the way you think about evidence?
- [32:35]
Helen Pearson
I think it changed the way I work as a journalist to some extent. I mean, obviously I write features about complex bodies of evidence, so I find interesting questions and I want to go and look at what the evidence says about them. And I would say what's changed about that is I am much more likely to now look for evidence syntheses. So, for example, for systematic reviews when I'm diving into a new field. And I don't think I did that as much in the past. And I found that quite liberating, you know, because it. And of course evidence is just ballooning. I mean, in terms of the amount of research that's coming out. So that's been quite helpful.
- [33:09]
Jacob Goldstein
I mean, the classic bread and butter of science journalism is writing about a new study. Right. But one lesson of what you're Talking about is like, except in rare cases where the new study is like really giant or something, we should almost ignore new studies. Like we should add this one new study to this pot of a thousand studies and see how much it, you know, adjusts the totality of the evidence, which is usually going to be not that much.
- [33:34]
ChatGPT for Business Sponsor
Right?
- [33:35]
Helen Pearson
Yes. I mean, it's going to depend how big the body of evidence is that you add in the study to. So you're right. If it's a brand new trial, it's the first time it's tested a drug. Totally, we should write about that and report on it. But if it's another study saying, you know, red wine is good for you, that's probably best to put to one time. Now, the reason, of course, those stories still come out is because they get clicks on websites. But I'm with you. I mean, I have definitely reached this point of thinking, yes, I need to look at the body of evidence as a whole.
- [34:03]
Jacob Goldstein
And what about outside of science? Like do you use, is there any part of your life where you use evidence now, outside of work, outside of health, where you didn't use it before?
- [34:19]
Helen Pearson
Well, probably I use it more in parenting. I mean, that's a really interesting case where I feel it's a bit like management in that there's just, it's completely awash with, you know, fads and fashions and books and. And actually there is quite a lot of evidence around, you know, practices that can support child development, some of which is mirrored in standard parenting advice, but some of which might be slightly different. And I just think it gets lost in the kind of fire hose of information.
- [34:48]
Jacob Goldstein
What's a non obvious piece of parenting advice with robust evidence behind it?
- [34:52]
Helen Pearson
I don't know if it's non obvious. I would just say it's obviously a really live debate right around. I've looked specifically for nature. The evidence around is social media and screen use causing the rise in mental health problems for adolescents and young people. And that is a really complicated body of evidence which I think people are interpreting in different ways. But I do think that this general sense that that's been shown that this is causative and it's a huge driver of adolescent mental health problems is an overreach based on the people that I've. And having looked at the evidence myself.
- [35:32]
Jacob Goldstein
So are you saying screens and social media aren't as bad for teenagers as a lot of people think? Is that a fair reduction of what you said? Yeah, yeah, yeah. Good news, good news.
- [35:42]
Helen Pearson
Can I go back to Your point? Cause you asked me how it's changed, how I, you know, personal decisions.
- [35:47]
Jacob Goldstein
You let your kids stay on the phones more than you did before.
- [35:51]
Helen Pearson
To some extent, yeah. I mean, it just made me relax a little bit about it because, you know, we're really seeking these concrete answers, aren't we? Or what I would have loved the evidence to say was like, you know, two hours and that's it. Right. But it doesn't provide you with simple answers. Evidence generally doesn't. But sometimes I think that that can be quite empowering itself, you know, as a parent, to go, okay, well, I have looked at the evidence and it doesn't give me the crisp answers I would really love to have about what to do. But I understand that. And now I feel empowered to perhaps make decisions on my own, based on my common sense and weighing up the things which are important to me and my family.
- [36:30]
Jacob Goldstein
What's the future of evidence as an idea, evidence in the world? Do you feel like we're in this moment when there's a sort of evidence
- [36:37]
Helen Pearson
backlash to some extent? I literally published a big piece last week for Nature about trust in science because I wanted to look at this kind of like there's a sort of sense in the air that there's this big crisis of trust in science. People have lost trust in science, and I wanted to kind of scrutinize that claim. So I went off and looked at kind of data, and I've spoken to lots of people, and I would say the big picture, given that surveys are imperfect, but from lots of surveys, is that generally, you know, globally, trust in science is quite high. People generally trust science, and scientists as a profession is one of the most trusted professions. Like, it's up there with like, doctors and teachers, you know, and you've got journalists like me and politicians who go down the bottom of the list. So the global picture is kind of like glass half full. Everything looks fine. But then obviously, yes, they were big problems. So in the US Trust in science has gone down in the last few years, and there's specifically a political polarization going on around trust in science in the US with it falling amongst Republican leaning people, but remaining high amongst Democrats. And people feel concerned that the Trump administration is kind of capitalizing on this idea of distrust of science to now attack scientific institutions. So I don't want to give the impression that I came away from this exercise thinking everything's rosy with trust in science, because that's not the case. This big global trust in science survey I looked at in some countries, lower trust in science was associated with being right leaning, but in others it was associated with being left leaning. So it really kind of depends on the country.
- [38:11]
Jacob Goldstein
What's the future of evidence?
- [38:13]
Helen Pearson
I think in some ways, you know, looking at the evidence revolution, like going back to where we started, it's quite recent, you know, I mean, evidence based medicine as a term was only introduced in 1991, 35 years ago. So you, if you want to have the optimistic view, which maybe I do, you know, you could say that the current kind of setbacks are that there's always been opposition to evidence and what's happening now is just another setback in the kind of longer march towards rationality, which is going to take decades. So I suppose, yeah, I do come out hoping that's the case. It feels a little hard for me to believe that we're going to get to a point where we reject all of this knowledge we've created in the world and choose not to use that at all in decisions. I hope that we don't get to that point.
- [39:03]
Jacob Goldstein
We'll be back in a minute with the lightning round. Quick one.
- [39:15]
ChatGPT for Business Sponsor
Before you jump back in, you're listening for ways to help teams move faster, make sharper decisions, and turn scattered context into work they can use ChatGPT for business can help ChatGPT for business gives teams a shared workspace with admin controls, permissions and access to work and codecs. In ChatGPT, this means your business can move from question to answer and code to rollout quicker. Join over 10 million business and enterprise users worldwide already using ChatGPT for work. Download the ChatGPT desktop app, or contact sales to learn more.
- [39:50]
Jacob Goldstein
Did you ever notice how you spend hours shopping online only to pause right before you hit buy? Now that tiny hesitation, the one where you wonder if it's trustworthy, can make or break the sale. Now, with AI changing the way we discover and compare things, that split second, trust question matters more than ever. That's where PayPal comes in. For more than 25 years, PayPal has been a leader in online payments, and now they're at the forefront of agentic commerce, making it work for businesses, letting them maintain control of their brand and their customer relationships. So even as the way we shop changes, the moment that matters most still feels familiar and deeply dependable. Built for payments growth and agentic PayPal open built for all business visit PayPalOpen.com
- [40:41]
Laura Maxwell
to get started working As a dedicated fidelity advisor, I always ask what's important to you. A look at those priorities and goals Alongside your full financial picture to build a comprehensive plan designed to help grow and protect your wealth. With proactive portfolio insights and an eye on taxes, I'm here to help you make the most of your money. I'm Laura Maxwell, and I'm a dedicated Fidelity advisor. Learn more@fidelity.com wealth investment minimums apply Fidelity Brokerage Services, LLC. Member NYSE SIPC
- [41:14]
Jacob Goldstein
let's finish with a lightning round, if you would.
- [41:17]
Helen Pearson
Okay.
- [41:19]
Jacob Goldstein
What's a piece of really solid evidence that you ignore?
- [41:23]
Helen Pearson
Ooh, I think I try not to think too hard about the evidence on alcohol.
- [41:33]
Jacob Goldstein
Tell me more.
- [41:34]
Helen Pearson
Even though I've written a whole piece about it. Yeah. Because I know having reported on it, and I literally, like, calculated it all out and looked at my own drinking and I thought I probably should drink less. And I find it difficult to do that. I'm not saying I drink to excess. I'm just saying it probably should be less than it is.
- [41:51]
Jacob Goldstein
Well, the old, like one glass of wine a day is actually good for you. The evidence seems to have shifted on that claim, correct?
- [42:00]
Helen Pearson
Yes. In general, most people could benefit by cutting back and the risks go up from a really low level. Yeah.
- [42:06]
Jacob Goldstein
And was the problem with that evidence confounding that the people who drink one glass of wine a day tend to be have really healthy lives? They're like very moderate people. They don't do anything in excess. And that's why it looked like that was good for you?
- [42:20]
Helen Pearson
Partly, yes. Confounding is really hard to strip out of those studies. And also, of course, the other way is that heavy drinkers also tend to be smokers and do all the other unhealthy behaviors. So it is a difficult relationship to tease out.
- [42:34]
Jacob Goldstein
So you previously wrote a book about these longitudinal studies where researchers, as I understand it, followed a group of people for their entire lives. Extremely interesting. What were some of the things they learned from those studies?
- [42:50]
Helen Pearson
The big message from all of the studies. I mean, I set off to write this book that I thought was going to be about these fascinating sort of about science somehow. And I sort of ended up writing this book about inequality, which I didn't really realize till I published it, because really the big message was that children who were born into difficult circumstances tend to follow, on average, tend to follow, more difficult life paths. So they ended up in poorer health, less wealth, in less good jobs. And maybe that sounds really obvious, but the sad thing is that that's kind of been repeated, you know, generation after generation, and it's still being repeated today.
- [43:24]
Jacob Goldstein
So mobility didn't significantly increase over time?
- [43:29]
Helen Pearson
Not, no. I mean, there has been some changes. But that point about your, your early circumstances having this profound influence on the rest of your life, that remains true.
- [43:41]
Jacob Goldstein
Tell me about beer glasses in British pubs.
- [43:48]
Helen Pearson
I love this story. This is one of my favorite randomized trials that I came across when I was writing the book. So this is a story about a highly unlikely trial that was done by a researcher called Jonathan Sheppard, who was a, is still actually a facial surgeon. And he noticed that a lot of the people that he was treating with terrible injuries had come from glass. Often these awful incidents were glassings where two people in a pub get angry and somebody decides to shove what's in their hand, which is like their pint glass, into somebody else's face. And so they have these awful lacerations and life altering injuries. And so he started thinking, well, what if we had stronger beer glasses and maybe that would lead to less glass injuries.
- [44:34]
Jacob Goldstein
Just to be clear, why is it better if that glass that they're shoving into somebody's face is tougher?
- [44:41]
Helen Pearson
It's less likely to shatter.
- [44:42]
Jacob Goldstein
To shatter.
- [44:44]
Helen Pearson
So basically you might have been part of this unusual randomised trial of beer glasses if you bought a pint in a selection of pubs in the UK and Wales around 1998. Because he got a selection of pubs and bars to randomly, some of them were signed to install toughened glasses and the rest carried on with their normal glasses and managed to show. It was a bit of a complicated trial, but he managed to show that indeed, having tougher glass reduced the number of glass injuries. And this helped lead to momentum, which encouraged the pub industry to switch to toughened glass in the uk, leading to, or at least associated with a drop in glass injuries of like tens of
- [45:27]
Jacob Goldstein
thousands a year or something. Extraordinary.
- [45:29]
Helen Pearson
Yeah, that's a big number. Yeah, yeah. Amazing.
- [45:32]
Jacob Goldstein
If somebody gave you, say, $100 million to fund a study, any study you want, what would you do?
- [45:41]
Helen Pearson
Wow. Oh my God. Okay, that's trying to think of the question that I most want an answer to, for which there is no evidence. It has to be something around parenting. I mean, okay, I'll give you an answer, but I think it's a bit of a cheat because it's sort of happening, which is around these social media bans which are going on at the moment. I mean, the UK is just introducing a policy to ban all teenagers from social media and it's already happened in Australia. So I think the key question, which evidence curious people like me have and lots of researchers have, is what's the effect. So I think a careful study to understand the impact of those changes is really important.
- [46:26]
Jacob Goldstein
It was great to speak with you and I appreciate your work. Thank you.
- [46:31]
Helen Pearson
Thank you very much. No, thanks very much for your questions and thanks for having me on the show.
- [46:41]
Jacob Goldstein
Helen Pearson is the author of the book Beyond Poetry Belief. Please email us at problemushkin fm. Tell us what kinds of shows we should do more of or less of or particular people you think would be good on the show. You can also find me on x and on LinkedIn. Our show is produced by Gabriel Hunter Chang and Trina Menino. Our editor is Lydia Jean Kott and our engineer is Sara Bruguer. I'm Jacob Goldstein and we'll be back next week with another episode of what's yous probably.
- [47:19]
ChatGPT for Business Sponsor
Before you sign off, you tuned in for ways to help teams move faster, make sharper decisions, and turn scattered context into work. They can use ChatGPT for business can help ChatGPT for business gives teams a shared workspace with admin controls, permissions and access to work and codecs. In ChatGPT, this means your business can move from question to answer and code to rollout quicker. Join over 10 million business and enterprise users worldwide already using ChatGPT for work. Download the ChatGPT desktop app or contact sales to learn more.
- [47:54]
J.R. Martinez
This is J.R. martinez for Medal of Stories of Courage. This episode is brought to you by Navy Federal Credit Union. At Navy Federal, they want you to know that if you need a positive sign towards achieving your goal of home ownership, this is it. Because even though today's housing market can seem discouraging, their home Buyer's Choice Loan offers members down payment options as low as 0% with no required private mortgage insurance. Look for your sign today. Navy Federal Credit Union. Navy Federal is insured by NCUA Equal Housing Lender. Terms and conditions apply. Loans subject to approval and eligibility requirements. Learn more@navy federal.org 0down did you ever
- [48:31]
Jacob Goldstein
notice how you spend hours shopping online only to pause at checkout because you wonder if you trusted enough to hit buy? Now Agentic Commerce is testing that moment more than ever. That's where PayPal comes in. With 25 years of checkouts, 400 million consumer accounts globally and the benefit of fraud protection. So no matter where a purchase starts, it ends with trust. Built for payments growth and agentic PayPal open built for all business, visit PayPalOpen.com
- [49:01]
Helen Pearson
this is an iHeart podcast. Guaranteed human.