
Tim Harford examines the claim that NHS hospital patients are 45% more likely to die...
Loading summary
A
Thank you for downloading More or Less from the BBC. This is the version of the programme first broadcast on BBC Radio 4. Here's Tim Harford.
B
Hello and welcome to More or Less, the programme about the numbers which surround us in the news and in life this week. Fact checking none other than Sir David Attenborough. Yes, no one is safe from the firm thwack of our slide rule. This fella's been feeling it too.
C
I am very sorry. I'm very sorry.
B
And we tell the story of the most famous and most misunderstood experiment in industrial history. But first, maybe you heard this recent Channel 4 news story.
A
Shocking new figures given to this program reveal that the NHS is fundamentally failing. They suggest that death rates in hospitals in England have been far higher than. Than in other countries. And it's not just in the scandal hit trusts you read about in the newspapers, but we're talking here about the average NHS hospital. The figures compiled by the eminent doctor and statistician Professor Sir Brian Jarman, suggest patients are 45% more likely to die in hospital here than in America.
B
Loyal listener Liz Lloyd is one of several people who've been emailing us atmore or lessbc.co.uk.com to.
A
This sounds scary. Can you unpick the statistics on this?
B
It does sound scary, Liz. So what's going on? The basis of this comparison is the HSMR Hospital Standardised Mortality Ratio. We've looked at it before, in this very series, no less. The HSMR measures how likely you are to die in a particular hospital relative to the typical hospital. After adjusting to the best extent possible for just what was wrong with you in the first place. Channel 4 have asked instead how likely you were to die in one country's hospitals between 2004 and 2010 relative to another country's. Again, adjusting for how sick you are, as you heard Cathy Newman saying on Channel 4. The research behind this story was conducted by Professor Sir Brian Jarman of Imperial College. And with a title like that, you probably don't need me to tell you that he's an eminent sort of fellow. He's also the inventor of the HSMR. He worked with Channel 4 for months and after the broadcast and a storm of protest, he released further bits of information on Twitter. But there's a lot we don't know about his analysis. The uk, or strictly speaking, because this is the way the data are gathered, England is compared with six other countries and five of them aren't even named. The details of the analysis are a black box. So I began by asking Sir Brian why he hadn't published in a transparent fashion in a peer reviewed journal.
D
The English data that I got was in 1988, which was the data on every admission to hospital and I got that data. I had to sign the Official Secrets act and in other countries it is equally confidential and I have it on the basis that I will not pass the data to anyone else. I realized that I couldn't submit it for publication in a peer reviewed journal because I'm not allowed to give the information. On the other hand, I just felt that, that although I knew I would get the enormous level of attacks, it was just duty as a doctor, if you like, that one has to draw attention to possible problems now.
B
Other eminent statisticians aren't impressed. On the British Medical Journal website, Professor Martin McKee of the London School of Hygiene reeled off a list of research from around the world, concluding that HSMRs just don't tell you about the quality of care. When we hear that you're likely to die in an NHS hospital, we naturally think that the alternative is not dying. But we all die eventually. And so another possibility to take into account is dying somewhere other than a hospital. Professor David Spiegelhalter, a biostatistician at Cambridge, has pointed out also in the bmj, that some countries routinely discharge terminally ill patients so they can die at home or in a hospice. Other countries routinely admit such people so they can die in hospital. Only a third of all deaths in the Netherlands are in hospital, for instance, but more than three quarters of all deaths in Japan are. This doesn't tell us anything about whether Dutch hospitals are better than Japanese ones. In the United States, 45% of deaths occur in hospital. In England and Wales, 56% of deaths do. That's almost a quarter again, as many. Is that a comment on the quality of NHS hospitals? I have no idea. Sir Brian is unruffled.
D
We published a paper in the BMJ on that very subject in 2004 and we show that there are differences. There is the factor which is the discharge to what I call intermediate care. We discharge a higher proportion of people straight to home than Americans. But looking at those differences, the figures show that those factors are not sufficient to explain the differences that I found.
B
Opinion amongst medical statisticians on that claim is divided. I think it's fair to say so. Maybe US hospitals are better than English NHS hospitals and maybe they aren't. But on a purely anecdotal basis, Sir Bryan argues it could hardly be a surprise if English hospital care was behind the curve.
D
Now, from my own personal experience, I've seen when I was there that there was a very, very different availability of doctors, the length of time we were on call, the availability of resources, maybe that is still there.
B
Professor Sir Brian Jarman. If this research does stand up, it wouldn't be telling us that the US healthcare system outperforms the NHS in general. It would be telling us that the US system treats the elderly better because it's the elderly that are overwhelmingly the people who end up dying. And it wouldn't be the first study to suggest cause for worry about that. The Care Quality Commission and the Patients association have both raised concerns about the treatment of the elderly in NHS hospitals. You're listening to More Or Less, and I'm Tim Harford. I don't have a knighthood, but everyone else on the program today seems to. So let's now hear from a BBC institution, a man who is surely never wrong. Sir David Attenborough.
D
We just shut our eyes to the fact that the world population is increasing out of control. I mean, since I first started making programs 60 years ago, the human population has tripled three times as many people on Earth. Extraordinary, isn't it? And it is extraordinary. And we don't look at this fact in the face. There is a serious problem ahead of us and we owe it to future generations to face up to this.
B
Sir David Attenborough on the Today programme last week. But something worries me about that. It seems to conflict with something broadcast Elsewhere on Radio 4 by a slightly less renowned voice. Birth rates have been falling around the world. World population looks like it's going to stabilise. We have serious environmental problems. It's not at all clear that overpopulation is one of them. Dear me. That wasn't a more or less episode, by the way. That was me moonlighting to pay for my new calculator on my other series. Pop up ideas. Anyway, I appear to have fallen victim to one of the classic blunders. The most famous is never get involved in a land war in Asia. But only slightly less well known is never go in against Sir David Attenborough. Well, let's start by looking at Sir David's historical claim that since I first
D
started making programmes 60 years ago, the human population has tripled.
B
Is it true? Yes, it is. Or close enough. In 1950, around the time Sir David was starting his broadcasting career, there were about two and a half billion people in the world. 63 years later, and the latest estimates of world population put it at. Well over 7 billion. That's a little shy of tripling, but it's not far off as to whether that's a problem. Right now. That's a huge topic, and we're going to sidestep it. But I think it's reasonable to suggest that 7 billion people have a larger environmental impact than 2 or 3 billion people. So let's move to Sir David's next claim. Is the world's population increasingly out of control? If world population continued to triple every 60 years, in accordance with Sir David's experience, it would be almost 40 billion people. That's4.0 billion by the end of this century. But the latest United nations projection is that we will have fewer than 11 billion people. The rate of population growth has slowed dramatically, although, of course, 11 billion people is still 50% more than we have today. The UN population growth projections might not be at the forefront of Sir David's mind, but they are the basis for all kinds of other forecasts. For instance, about energy use or even corporate profits. No wonder they've been scrutinized and by
E
the likes of this chap Sanjeev Sanyal. And I'm Deutsche Bank's global strategist.
B
And as Sanjeev poured through the UN's figures, he thought that something was odd.
E
Unfortunately, when I looked at them, I discovered that they were almost certainly wrong about it.
B
So the UN predicts that population growth will slow enormously. Now, Sanjeev didn't think their forecast was too low. He thought it was too high.
E
If you look at the fertility rate, which is the number of babies that a woman has over the course of her life in very large parts of the world, those fertility rates are now below what is needed to replace the population. Much of Europe, Japan, large countries like China, even Brazil don't produce 2.2 or 2.3 babies. Some of them are way below that level. And as a result, it is almost certain that these huge countries are going to see rapidly declining populations within a few decades from now. So it's very difficult to justify the numbers that the UN is coming up with. In fact, if you look at the UN's own numbers, you will discover that they are generating their population growth with huge increases in population of, you know, small countries like Nigeria, where they expect population to grow from 150 million to almost a billion by the end of the century.
B
Let's just run through what your forecast actually is.
E
In my model, the world population would actually peak around 2050 at 8.7 and decline to about 8 billion by the end of the century.
B
But while Sanjiv believes the global population will peak around 2050, if we look purely at birth rates, we'll stabilize, he says, in just the next 15 years, we'll see a bit more growth overall because of the fact that we're living longer. So let's summarise what we've got so far. The UN predicts that the world's population is going to grow by around 50% over the next 90 years. Yet Sanjiv Sanyal thinks it will barely grow at all. They're both using the same data, which comes from national censuses in 2010. So what explains the difference of nearly 3 billion?
E
I took into account two or three things which I think are inadequately reflected in the UN. One is the effect of urbanization. I have probably accounted more aggressively for things like gender bias in countries like China and India. They aren't likely into the future to have enough women of childbearing age. Another thing that the UN does, which I think is quite awkward, is that in the long run they assume that all fertility rates converge towards a replacement rate. Now, this is a very odd assumption because we have not seen any country where fertility rates have declined very, very dramatically. To have seen them drift back up to the replacement rate. And I have simply just taken that assumption out.
B
So what does the UN say? Unfortunately, they didn't want to comment without first speaking to Sanjeev Sanyal. But they did explain that making these long term forecasts isn't an easy task, even with a great big team. Unlike the un, Sanjeev Sanyal is a one man band who's looked at around 25 countries, representing around 70% of the world's population. We really don't know what population will be in 2100, because even very small changes to fertility rates can have a huge impact decades down the line. The UN's own predictions offer up a huge range from less than 7 billion to almost 17 billion by 2100. Plenty of room for disagreement now from global population in general to the global population of my favourite kind of person. The Ginger Pride March in Edinburgh last month, the first of its kind. I have to confess an interest. I do have a penchant for redheads. I even married one. But there was a time when I thought that would never happen. A bit like poor Charlie Brown.
A
I'm just a nice sort of a guy who never gets to meet little red haired girls.
B
Well, perhaps Charlie will meet the girl of his dreams in Scotland. According to the BBC website, Scotland is said to be home to almost A fifth of the world's redheads. Hmm, that seems like a lot. Or as loyal listener Mark Hagen put it, implausible. Or as I put it, bobbins. Here's Mark Hagan.
F
If we assume that 1% of the world's population have red hair, and if we assume that the world population is about 7 billion at the moment, then this would mean that there'd be a world population of around 70 million redheads. For Scotland to have a fifth of the world's redheads, then it would need to have 14 million. The 2011 census showed a population of Scotland to be 5.3 million. And that would mean an impossible 264% of the population are red headed.
B
Well, Hannah Barnes. Raven headed. Hannah. Not really a redhead. Anyway, you've been looking into this for us.
A
I have, and I've traced the origin of this mighty statistical claim all the way back to an interview on Radio 4's Today programme with Sean Hitchens. He was the organiser of that gingerbread march.
C
It's a great opportunity to bring together all the redheads in the country that holds 19% of the world's population. Population of redheads.
B
He sounds very authoritative. Presumably he's done a great deal of research on the prevalence of redheads.
A
Not really, no. He's a Canadian comedian and self described proud ginger.
B
He was obviously basing that comment on something, though.
A
Yeah, no, not really. He admitted to me that he just got a bit confused during the interview and got his words muddled up.
C
Oh, gosh. Oh, I'm misleading the people. I am very sorry. I'm very sorry. This is what. When you let comedians be spokespeople, I have to say, during the festival, there was just so many things coming at me. So the number of redheads, the percentages, nobody really knows the exact one. And then there was somebody from Glasgow saying it was 19% in just the population of Scotland.
A
So what happened? You were doing a flurry of interviews and you got the 19% of Scotland confused with the world.
C
Yeah. It must have been in confusion. I was walking through Edinburgh with 200 redheads behind me. I was just mostly trying not to. To lead people into traffic. So that's probably what happened. Also, I'm a ginger, so I'm 10 times more likely to make up a statistic.
B
Well, fair enough. I think we can be very proud of ourselves, Hannah. We've had something that could have turned into a zombie statistic. We've stopped it in its tracks.
A
We have. And actually the online team have admitted that while they hadn't given it much thought at the time. On reflection, it does seem implausibly high. And they've now taken the 20% reference out of the story.
B
So we know that a fifth of the world's redheads are unlikely to be living in Scotland. Do we know the correct figure?
A
Well, we don't really. It's estimated that about 10% of the UK are redheads and the proportion in Scotland could be a little bit higher than that.
B
Wow. So that would be 6 million incredibly hot people in the UK as a whole and half a million of those in Scotland.
D
Maybe, maybe more.
A
Tim. Tim.
B
Sorry, I was miles away.
A
Now, most estimates put the number of redheads at somewhere between 1 and 2% of the total world population. So that would be between 70 and 140 million.
B
So Scotland is home to perhaps at most, 1% of the world's population of redheads.
D
Yeah.
A
And that's still a respectable amount. Northern European nations do have a greater proportion of redheads than anywhere else in the world. Now, I know you're going to ask me.
B
I am going to ask you why.
A
Well, let me hand you over to Jonathan Rees, professor of dermatology at the University of Edinburgh, and one of the people who identified the ginger gene.
F
When humans moved out of Africa, let's say, 50,000 years ago, skin lightened and people developed hair with different colours. And the best reason for that is there seems to be some advantage in having pale skin if you live in parts of the world where there isn't much sunshine. Conversely, the best skin to have if you live in an area with a lot of sunshine, is very, very dark skin. We know that there has been selection. What I mean by that is there's been pressure to develop light skin in northern Europe, and the probable reason for that is that we make vitamin D in our skin. We rely on sunshine to make vitamin D, and if you have very dark skin, it's harder to make that vitamin D. So many of us view red hair as just another way of developing pale skin, which enables you to make vitamin D in your skin.
B
In other words, redheads are more efficient at making vitamin D. They don't need so much sunlight in order to get the amount they need. So there's a big advantage to being in this minority, quite apart from the fact that redheads are, as a matter of scientific fact, extremely sexy. Well, thank you, Hannah. Now, last week we mentioned the Hawthorne effect and I teased you by saying there was a story to tell. You emailed in your droves, asking to hear more. So are you sitting comfortably? Then I'll begin. We're in 1920s Chicago. A major new industrial centre, the Hawthorn plant in the suburb of Cicero, was making telecommunications equipment and it became the venue for one of the most famous experiments in industrial history. The National Academy of Sciences began an experiment to see how the level of lighting affected how productive workers were. The findings from this simple experiment have become legendary because when fancy new lights were installed, the workers worked harder. And when the lights were removed again, the workers worked harder. Nine more years of research followed and the same results. Lengthened work breaks, the workers worked harder, Shortened work breaks, the workers worked harder, shortened the working day, the workers worked harder. Lengthen the working day, the workers worked harder. And perhaps that's not a surprise. Factory work was tedious and repetitive. Being experimented on was much more interesting. When they asked me would I like
E
to work in a test room, I
A
thought it was going to be something different than what they were doing.
B
Here's one of the original workers from Hawthorne being interviewed on an AT&T documentary the year they discovered People years later.
A
We liked it.
B
All of us liked it.
A
You know, every day we had to do something different. It wasn't the same thing over and over.
B
The grand conclusion was that it wasn't the details of the work environment that mattered. It was receiving close attention from the experimenters. We could more logically attribute the increase in efficiency to a betterment of morale than to any of the alterations made in the course of the experiment. We concluded that the same relationship might exist throughout the plant and that the best way to improve morale was through improved supervision. It was an extraordinary realization. Workers weren't just flesh and blood robots on the production line. They were humans and they responded to attention and to communication.
G
This was not only a very important scientific moment that some researchers would argue is a breakthrough as important as Galileo, but it also had an important effect on industrial relations and how employers actually treat employees.
B
This is John List, a professor of economics at the University of Chicago, and he's alluding to the fact that there were really two audiences for the Hawthorne effect, and it came to mean different things to these two groups of people. For social scientists like John List, the Hawthorne effect was a headache. John List specializes in field experiments, that is, experiments performed on people in their workplace or some other everyday environment. The Hawthorne studies showed that once experimenters show up and start fussing around, they can easily change people's behavior and spoil the validity of the experiment. But the other group of interested people were management experts, and for them, the Hawthorne studies weren't a headache, but an opportunity to improve productivity.
G
This was a time when people viewed the employee as an automaton, a person who you could turn on and off and they would work Monday through Saturday and have one day off on Sunday. And essentially you could treat them like they were computer chips. Now the Hawthorne experiments showed that that indeed was not correct and that employee relations is extremely important.
B
And you can still see the effects of these experiments today with even just a cursory surf through the management guru section of youtub.
A
This study gave rise to the human relations movement, which really shows the importance of not only communicating to, but communicating with your employees. So communication in organizations today needs to be a two way street in order for it to be the most effective.
B
And that's the story of the Hawthorn studies. And that's where the story ends.
G
You know, that's actually for me, where the story begins.
D
Oh.
G
What I found in these early studies is that they never presented data. It was simply a qualitative account of, for example, a few experts or eyewitnesses that led to these very, very strong statements about how productivity changed, regardless if we made the condition better or worse.
B
So John List thought he'd track down the data to see if more lessons could be learned from this seminal research. He searched through academic journal after academic journal. So did his colleague Steve Levitt, most famous as the co author of Freakonomics. But all reports were that the data had been lost. It looked like Levitt and List would have to give up until John List reached for a paper written in the 1970s and blew the dust off a footnote.
G
When I first read that passage, and I'm reading deep into the night, I had to read that sentence in that appendix, I think 20 or 30 times because I could not believe that those data could actually exist.
B
The footnote led Leavitt and List to a microfilm archive at the University of Wisconsin, Milwaukee. And after a search there and elsewhere, they uncovered the data from the very first illumination experiments at Hawthorne. And the amazement didn't end there.
G
What we found was something astonishing. And that's that when they ran these experiments, they made the experimental change on Sundays. Now they made the experimental change on Sundays because that was the day off for all of the workers in the plant. But when you look at the productivity of workers over the week, what happens is, is that on Mondays they're very, very productive, probably because they're rested from their day off. And by the time they get to Saturday, they end up not working as hard as they do on Mondays. And Tuesdays. So the empirical fact is after you control for just that, people work harder on Mondays. The entire Hawthorne effect result disappears.
B
A legend slain. In some ways this is a historical curiosity. There was no Hawthorne Effect in the Hawthorne illumination studies. But sexual scientists remain convinced that being experimented on does change our behavior, and nobody's more convinced of that than John List.
G
I do believe in Hawthorn effects. I do believe that what they originally argued to find, even though it's not in the original data, I think it's fair to say that decades and decades of research since has taught us the power of receiving attention or people observing us.
B
John List, economist and co author of a new book, the Y Axis and just time to reveal a further bit of evidence as to the flakiness of Deputy Prime Minister Nick Clegg's claim that plastic bags last a years, a statistic we debunked last week. Loyal listener Tim Blackwell emailed more or lessbc.co.uk to on the day my daughter
F
was born, I bought three newspapers to commemorate the event. I took them home and placed them in carrier bags which I carefully folded and sealed with a popular brand of sticky tape to be opened many years hence. A few weeks ago I was having a bit of a clear out and I came across the newspapers. I was disappointed to discover the carrier bags had completely disintegrated into a flaky mess. I can only assume they were biodegradable in a way I really didn't want them to be. My daughter is not 1,000 years old. She's just turned five.
B
This episode of More or Less was brought to you by the number 1444. And if you'd like to tell us which number will bring you the final episode in the series, please drop us an email. Here's a reminder of where we're up to so far. This week's episode of More or Less was brought to you by the number one. By the number 49100 784. The first person to send the correct answer will be rewarded with a reply to their email and perhaps a little bit of on air praise. The runners up and everyone else can get a free download of the program at our website, BBC.co.ukmoreorless until next week. Goodbye.
A
More or Less was presented by Tim Harford, the Financial Times undercover Economist. It was produced by Ruth Alexander and made in association with the Open University. You can Download many more Radio 4 programmes for free. Find these@BBC.co.uk Radio 4.
Date: September 27, 2013
Host: Tim Harford
This episode of "More or Less," presented by Tim Harford, investigates the numbers and statistics behind some of the week's biggest headlines and commonly held beliefs. The main topics include debunking scare stories about NHS hospital death rates, examining Sir David Attenborough's claims about world population, unraveling myths about redheads in Scotland, and retelling (and reanalyzing) the famous Hawthorne effect in industrial psychology. As ever, the show expertly cuts through the hype, confusion, and occasional statistical error in the news and popular opinion.
"I realized that I couldn't submit it for publication in a peer reviewed journal because I'm not allowed to give the information... it was just duty as a doctor, if you like, that one has to draw attention to possible problems now."
— Prof. Sir Brian Jarman [02:49]
“Redheads are more efficient at making vitamin D... there’s a big advantage to being in this minority, quite apart from the fact that redheads are, as a matter of scientific fact, extremely sexy.” — Tim Harford [18:15]
“After you control for just that — people work harder on Mondays — the entire Hawthorne effect result disappears.”
— John List [24:33]
Throughout, the episode is playful, witty, and subtly skeptical, with Harford’s characteristic dry humor (“firm thwack of our slide rule”), respectful but firm fact-checking, and clear exposition of statistical reasoning. The discussion is both accessible and incisive, inviting listeners to look beyond the numbers in the headlines and question the assumptions behind them.
In this densely-packed episode, Tim Harford and team debunk statistical myths spread by media, celebrities, comedians, and even classic social science. Listeners are reminded that apparently “shocking” health or demographic statistics are often much more complex—and dubious—than they first appear. Whether clarifying mortality ratios, population projections, or the realities of redhead genetics, "More or Less" exemplifies the value of skepticism, careful statistical reasoning, and a good sense of humor.