← More or Less
More or Less
The Death Toll in Syria
00:28:04
Tim Harford looks at the different claims made about how many people have been killed...
Loading summary
Transcript82 lines
- [00:00]
A
Thank you for downloading More or Less from the BBC. This is the version of the programme first broadcast on BBC Radio 4. Here's Tim Harford.
- [00:09]
B
Hello and welcome to More or Less the programme which applies the firm smack of numerical correctness to the outstretched palm of devious politicians and lazy journalists. Or is that lazy politicians and devious journalists? I can never remember. First, an apology to all the left handed listeners who were expecting to listen to our item last week. We'll get to it this time, but if you believe all you're told, the chances of you surviving to hear our conclusions are slim. Right handed people live on average nine years longer than left handed people. But first, the all consuming subject of the war in Syria and the prospect of Western intervention after a chemical weapons attack to the east of Damascus. How many people were killed by this attack? The US government says 1,429, including 426 children. France, which has committed itself to joining the US in military action, says 281 people died. The British Foreign Office hasn't given an official number at all, but the Joint Intelligence Committee says that at least 350 people were killed. And Moscow says it's ludicrous to say that the Syrian regime used chemical weapons at all. All very puzzling. I called up Kelly Greenhill, a professor of political science at Tufts University in the US and co author of a book about the politics of crime and conf statistics called Sex, Drugs and Body Counts. I asked her why the numbers varied so wildly, even between the US and France who've resolved to take military action together.
- [01:41]
C
The Americans acknowledge that they're not only relying on hospital sources and morgue reports, but also even more than some of the others, on activist testimony, on social media and on unidentified intelligence sources we aren't privy to. As well as there's more discussion of a reliance on mathematical models. And in part, so far as we can tell, some of the numbers that are coming out are based on numbers of rockets fired, wind patterns, dispersal patterns and other sets of expectations. The French, as I understand it, are relying on casualty counts and they're only counting people where they can identify a first and last name. That said, they have also that their mathematical models do suggest that casualty numbers might be as high as about 1,500, more or less consistent with the US claim. The British have been rather tight lipped, but so far as I can tell, the British are essentially arguing as well that they're only using numbers of people that they can actually identify as casualties.
- [02:51]
B
So these numbers then really they're actually answering a different question. So the French are saying we have a reasonably well Verified list of 281 named people who have died. And the Americans are saying we think it's plausible that 14, 1500 people could have died based on a collage of different evidence, including various estimates.
- [03:15]
C
Well, they're claiming to be answering the same question, but Tim, you're absolutely right. They're speaking to a different set of issues and based on a different set of expectations.
- [03:23]
B
Which I guess leads to your point, which is that this is all about politics.
- [03:28]
C
It's at least in large part about politics. And a large number is arguably particularly important to the Obama administration as it's resting its case for military action in no small part on the scale of the attack. And the argument is that there's something about this chemical attack on August 21 that is qualitatively different both in scale and scope than was previously the case.
- [03:51]
B
Professor Kelly Greenhill of Tufts University, the United nations says that around 100,000 people have died in the Syrian civil war so far. They get this estimate from the Human Rights Data Analysis Group, a non profit outfit that in turn relies on information supplied by a number of different groups on the ground in Syria. I spoke to Megan Price of the Human Rights Data Analysis Group and I began by asking her how we go about defining a conflict death.
- [04:17]
D
There is not an official definition. There's a spectrum of what is considered a conflict related death. The narrowest definition would be a very specific violent outcome of the conflict, someone who is shot or is a victim of a bombing or that sort of a violent episode in the conflict. But you could also consider someone who needed medical help and was unable to reach the hospital due to a checkpoint or due to a roadblock as also being a conflict related death.
- [04:53]
B
And are we counting deaths only on one side?
- [04:56]
D
The groups are doing their best to count all victims and it really varies depending upon the group's access and connections in the country. Some of the groups have stated that because the regime is not cooperating with them, they are unable to verify regime victims. And so they do not publish that data because they do not consider it to be sufficiently verified.
- [05:24]
B
How do you know when you, you receive a list from an organization in this field that the names on that list haven't been manipulated to meet somebody's political aims?
- [05:34]
D
In general, we do not. And as statisticians, our approach to that problem is we think of a sensitivity analysis, we look at the data and we ask ourselves how different would this data have to look to change our substantive conclusions. But I should also mention that there are a number of ways in which a record could be considered inaccurate. It wouldn't necessarily have to be quite as nefarious as being a completely fabricated record. It could be anything from an individual who was a victim of an accident, who was counted as being a conflict related death, or an individual who was missing following a violent event and whose name appeared on one of the lists and who later was discovered to in fact be alive. So there certainly is room for error in this process.
- [06:26]
B
Meghan Price. Her team have counted almost 100,000 deaths in the civil war in Syria between the beginning of the conflict and April this year. Because of the way these deaths have to be verified, she thinks that's likely to be an underestimate of the true death toll. You're listening to More or Less, and I'm Tim Harford. A bit of good news this week. The Daily Telegraph made a numerical claim on its front page on Wednesday, which usually makes for some entertainment. Cost of care forces 1 million families to sell homes. The Telegraph went on to explain that over the course of just five years, a million families have been forced to sell their home to meet the cost of paying for residential care. And this number was based on an Internet poll. Ruth Alexander's here. Ruth, you know about this kind of stuff from your illicit presenting on Moneybox Live. More than a million homes in five years sold to fund care for the elderly at the. That's a very high figure.
- [07:21]
A
It's huge. We know from the government's property sale figures that about four and a half million homes have been sold in the past five years. So if this research is right, one in four home sales have been to fund care.
- [07:34]
B
Wow. So every fourth for Sale sign, you see, that's somebody selling their home to fund care for the elderly. I mean, that's amazing. Or untrue.
- [07:44]
A
Well, yeah, and we don't think it's true. And neither does Philip Spires of the Elderly Accommodation Council. The figures reported in the Telegraph come from recent research commissioned by NFU Mutual, which found that more than 4 million people had gone into care in the past five years, which would be something like 800,000 a year. Now. This is way too high. According to Mr. Spires, the number of
- [08:05]
E
people entering care each year is round about 130, 150,000. Obviously varies. The latest figures come from language who provide statistics on this, and they're figures from September 2012 tell us that there were 432,000 care home beds occupied at that present time. That's a combination of both residential care, which is just personal care, and nursing homes. If you're talking about the millions entering care over the last few years, it's not possible. Definitely not true.
- [08:32]
B
So NFU Mutual, who produced this number, they've clearly bodged something somewhere. It's quite interesting though, to look at how they got it wrong.
- [08:40]
A
Yes, they'd commissioned icm, another company, to carry out an Internet survey on their behalf and they asked 2,000 of their panel of paid survey responders whether they or a partner had a parent go into residential care. 368 of these 2,000 people said yes. But when you look at these figures, there's something odd going on. The researchers found that people between 18 and 35 were more likely to have parents in residential care than people who are aged between 35 and 55.
- [09:12]
B
And Ruth, you mentioned that people were paid to complete these surveys and maybe that gives us a clue as to what was going on. Some respondents to such surve are called happy clickers. They're just trying to get to the end of the survey, collect their money. They don't really care.
- [09:26]
A
Yes, and when we made this point to the people behind the research, they did admit it does look like there's a problem with the answers given by some of their younger respondents.
- [09:34]
B
But Ruth, I think there's arguably an even bigger problem with the methodology here. My father, for instance, he's in good health, thankfully. But imagine if he was taken into care. Then I'd say yes when asked if I or my partner had a parent in care and my wife would say the same thing and my sister would say the same thing and her husband would say the same thing.
- [09:56]
A
How many sisters have you got, Tim?
- [09:58]
B
I've got three sisters and I've got two step siblings and of course their various husbands and girlfriends.
- [10:03]
A
How many girlfriends have you got?
- [10:04]
B
No, I mean my siblings have various girlfriends. There are actually nine of us who could say yes, my parent or the parent of my partner is in care and all nine would be talking about the same individual. So there's a lot of double counting
- [10:18]
A
here and a few mutuals. Research suggests 4 million people have a parent or parent of a partner going into care. They have, however, now seen the light.
- [10:28]
B
We now understand this is not the same as 4 million people going into care due to overlap between siblings and appreciate the points made by more or less. We love it when people are big enough to admit their mistakes. Thank you, Ruth. If you have a statistical claim you'd like us to investigate, please Send an email to more or less@BBC.co.uk. loyal listener Lex did just that, asking, could you explain the relative safety of the Super Puma helicopter that was involved in the recent crash off Shetland? The BBC transmitted reports of opinions, but I'd rather hear facts. Several other listeners had similar questions. I began by calling Robert Patterson, who's director for health, safety and Employment at Oil and Gas uk, the industry body, and asking him to explain what happened in the recent crash.
- [11:17]
F
On Friday 23rd August, about 06:00', clock, a helicopter crashed in the sea just south of Shetland on its approach to the Runway.
- [11:30]
B
Four people were tragically killed and we're
- [11:33]
F
in the process of trying to understand what happened there.
- [11:38]
B
There have been several accidents in recent years in the British area of the North Sea involving a particular group of heavy helicopters, which all fall under the description of Super Pumas, which has led to concerns that maybe there's a problem with the UK offshore oil industry's safety standards, or maybe there's a problem with this particular class of helicopter. Well, what about asking an informed observer of the uk, Toni Krakners of the Norwegian research group Sintef. Tony was part of a team that looked in detail at Norwegian North Sea helicopter safety between 1999 and 2009. The safety record in Norway, which has similar numbers of helicopters flying similar distances, often operated by the same companies, was better than in the UK, a lot better.
- [12:24]
G
In 1990 2009, there was one accident on the Norwegian side, not fatal. On the UK side there was 11 and three of them was fatal. With a total of 34 casualties, the statistical risk level would be zero for the Norwegian side and five point something on the UK side fatalities per million flight hours.
- [12:46]
B
The trouble is, says Tony, there just aren't that many helicopter crashes. Well, obviously that's good news, but it means that even a decade's worth of data can't distinguish the role of luck from something more systematic. Tony Krakenas and his team weren't able to say whether the superior Norwegian safety record was a lucky fluke or not. In the 1990s, the boot was on the other foot, with more accidents in the Norwegian sector. Several listeners asked us to compare the helicopter accident rate to that of a more humdrum mode of transport such as the car. As Tony Kraken has said, the death rate in Norway over the last decade was zero per million hours flown, or for that matter, zero per any number of hours flown, which is great, but statistically unhelpful. In the UK, the death rate was about 1 death per 200,000 passenger hours of helicopter flight. Now, I've just done some very rough and ready calculations to put that into context. If we look at cars in free flowing traffic in the UK, the back of my envelope says that there's 1 passenger death per 6 or 7 million passenger hours driven or ridden. Riding in a helicopter then is quite a bit more dangerous than riding in a car. But given what an unforgiving environment the North Sea is, it's hardly surprising that it's riskier than driving down the M5. If you look at the list of helicopter accidents in the British North Sea and there are few enough that you can look through every single case in a few minutes, you'll soon see that each one looks unique. There are pilot errors, technical problems, lightning strikes, gusts of wind, even water spouts. Helicopters may be more dangerous than cars, but Tony Krakinus thinks that under the circumstances, the industry has a good safety record.
- [14:27]
G
The general impression, based on our study findings, is that the helicopter transportation is safe and it keeps improving. We normally never hear about helicopter transportation except when there are accidents. So it is not strange that the general public get the impression that helicopters keep falling out of the sky. But most people working offshore consider helicopters as their everyday bus to work and think nothing of it. But we must keep in mind that helicopter transportation is the only way of traveling to offshore installations and the associated risk is part of the package of being an offshore employee.
- [15:02]
B
Is this really the kind of problem that one can examine statistically? It just strikes me, listening to your analysis, that maybe we just learn more by looking at accidents on a case by case basis than we learn looking at the statistics. Because the number of accidents is so low.
- [15:19]
G
Well, I think you're right. The data material is just too thin. We cannot really say anything. We can observe the differences, but we cannot conclude based on those numbers. So it's more fruitful to look into single accidents. I agree.
- [15:33]
B
Norwegian safety researcher Tony Krakeners of Sintev. You're listening to More or Less and I'm Tim Harford. Now, if you're left handed and you've been eagerly awaiting our item on left handed people, well, apparently you're lucky you've survived this long. Loyal lefty listener Claire Allen from Hampshire has been worrying about something she heard on, well, another radio station.
- [15:55]
H
I heard a Factoid on Radio 2 which suggested that right handed people live
- [16:01]
B
on average nine years longer than left handed people.
- [16:04]
H
I was absolutely horrified. I thought, that can't possibly be right.
- [16:07]
B
It is alarming what could the explanation be Claire's been scratching her head and so have we? Heavy machinery, maybe the awkwardness of driving a car, but surely not. Right handers would be an endangered species in mainland Europe and the U.S. maybe the threat to left handers lies closer to home. Here's Claire in her own kitchen.
- [16:27]
H
The knife is very awkward. Knives are designed for right handers. If you look at it, you can see that the blade is designed so that the sharp edge is this way around. So if you use it as a left hander, it cuts on the slant all the time. See, the knife is turning that way, it's turning to the left, whereas for a right hander it cuts straight.
- [16:52]
B
Knife accidents, It's a mystery. I turned to Chris McManus, professor of psychology and medical education at University College London and the author of Right Hand, Left Hand. The claim is that left handers die nine years earlier on average than right handers. How plausible is that? On a scale of completely plausible to
- [17:14]
F
vaguely plausible, it's not at all plausible. If this were true, it would be the largest single predictor we had of life expectancy. It'd be like smoking 120 cigarettes a day plus doing a number of other dangerous things simultaneously. So it really is highly implausible that an epidemiologist wouldn't have spotted it previously.
- [17:33]
B
Now, I have heard that claim though myself or something like it, so it's obviously doing the rounds. Is it pure myth or has anybody ever done any research on this?
- [17:42]
F
Well, the interesting thing is what Clare says when she quotes nine years is she's entirely accurate about an entirely wrong figure because the data originally said that there was a nine year difference. There was a paper in the New England Journal of Medicine, which is very prestigious, published by Diane Halpern and Stanley Curran, and they argued that they had data that showed this. And it followed up from another paper a couple of years earlier in the even more prestigious Nature, which was actually entitled Do Right Handers Live Longer? So there's a good pedigree for these claims. They're just utterly wrong. That's the problem.
- [18:13]
B
So what's the nature of the mistake?
- [18:16]
F
Well, it's worth looking in detail at what they did. They did this in Southern California and lists are published in Southern California of all the people who've died. So you can get a public list of the recently dead. And what they did was to send questions around to the recently bereaved families and asked them whether or not their relative was right or left handed, amongst other things. And when they totted up the 2,000 cases they managed to get data on, they found that the average age at death of the left handers was about nine years younger than the average age at death of the right handers. So they then concluded on that basis that the left handers died earlier.
- [18:53]
B
Which seems plausible.
- [18:54]
F
It does indeed, and this is a very subtle error. And their mistake was that they only looked at the dead.
- [19:00]
B
Now this sounds really weird. The issue is that left handers are more common than they used to be, and we'll explain why that is in a moment. But why does that make a difference? Let me give an exaggerated example. Imagine that there were no left handers at all born before 1973, 40 years ago. Now let's look at death records and ask who amongst the dead is left handed? Well, we know that in our exaggerated scenario all left handers were born after 1973. So when you look at the people who died in 2013, surprise, surprise, all of the left handers among them died at or before the age of 40. But that doesn't mean that left handers in general are at any higher risk of death. All of this raises the question, why are younger people more likely to be left handed than the elderly? Chris McManus explains the natural rate of
- [19:50]
F
left handedness is around 10 or 11%. But the rate was pushed down artificially during the Victorian period. You can see it going down from about 1800 onwards through to about 1900.
- [20:01]
B
People had it thrashed out of them
- [20:03]
F
partly, but I think also it was quite difficult to be a left hander. I think what happened was left handers became very conspicuous. Suddenly they went to work in factories using machines that were design for, you know, spinning jennies and all that sort of stuff designed for right handers. And the left handers looked awkward. And then compulsory schooling came along and they were obliged to sit in classrooms and try and write with their right hand using an ink pen and they made blotches and made a mess. And the result of all of this, that left handers became very visible suddenly and I think probably stigmatized. They were regarded as catcaddy stupid and all this sort of thing.
- [20:35]
B
And so people who were born left handers, identified as right handers, acted as right handers and when they died their families would have said yes, they were, they were right handed.
- [20:43]
F
That may be part of it. The other thing is that actually the left handers had fewer children in the 19th century. And the result of that was that the genes which make people left handed became less frequent. So fewer left handers were Born.
- [20:56]
B
So we've blown this nine year myth out of the water. But is there any danger to being left handed in a right handed world?
- [21:02]
F
I think it's quite clear there are some problems of being left handed and left handers will describe them. At most machineries designed by right handers for right handers. There is some evidence that left handers are more likely to have minor accidents, but it's pretty insubstantial and I doubt if it's affecting the mortality rate very much at all.
- [21:22]
B
Chris McManus now, from being left handed to wielding a computer mouse with your left hand and, well, who knows what
- [21:29]
H
with the other Parliamentary porn consumption laid bare in official figures.
- [21:35]
B
UK Parliament porn habit 300,000 adult site hits in last year. Interesting. And that story was among the most shared on the BBC website at times this week. But what do the numbers really say? The figure of 300,000 originally came in response to a Freedom of Information request from the Huffington Post. It corresponds to roughly one porn request per MP per day, which of course has caused a few chuckles. But there's something fishy about this. In one month, February, there were 15 attempts to access websites that had been blocked by an anti porn system. But in November there were 114,844 such attempts. Now, that vast disparity suggests that the headline of 300,000 isn't telling us quite what we think. I spoke to Fergus Reid, who's in charge of information technology at the Houses of Parliament, and I asked him what explained the variability.
- [22:32]
I
These figures are so high, as you say, 114,000 in November, that we're pretty confident that these are the results of malware on the system getting onto a device and issuing requests in an automated blizzard.
- [22:48]
B
So someone's smartphone or someone's computer has a piece of software that just keeps trying to access porn for some reason.
- [22:55]
I
Well, it's mischief and it's a bit of malware doing that and it's either on a device or it's on a, an otherwise standard website. They've put this malware on the site and respectable users of that site, then in fact their machine or their visit is hijacked by this malware. What I'm telling you today is that common sense really, with a 7,000 user base to have some subset of that user base trying to access sites that they're very clearly not allowed to do 114,000 times and getting blocked returns really suggests that they are automated and not a real person trying that. I mean, who has the time or tenacity to accomplish that, well, you should
- [23:36]
B
meet the editor of More or Less, but yes. So really, this headline that more than 300,000 attempts were made to access pornographic websites at the Houses of Parliament in the past year, that's one of these statistics that is simultaneously true and entirely unhelpful.
- [23:52]
I
I think that's probably right, because what's missing there is made by a system or made by a person, which really is a key bit of knowledge that you need to have.
- [24:00]
B
So, one month we had 115,000 attempts to access pornographic websites. In another month we had just 15. Do we have any sense of which of these two figures is actually closer to telling us about human beings trying to look at smutty pictures?
- [24:17]
I
I think the logic of me saying that that higher number is so incredible that it's almost bound to be automated systems, then I've got to say that the 15 looks more credible.
- [24:29]
B
Fergus Reid. And let's not even get into the question of how good the overlap is between websites that the IT system blocks because it thinks they're pornographic and websites that are in fact pornographic. Before we leave Parliament entirely, let's go to the House of Commons, where Fiona mactaggart, mp, was concerned about the amount of time we all spend on hold to the government, waiting for someone to answer the phone. It's a very long time indeed.
- [24:54]
J
Apparently, citizens of Britain have, in the last year spent 76 centuries, if you put them one after the other, waiting to be answered by government departments.
- [25:05]
B
Now, loyal listener Peter Daplin was surprised to hear this claim and skeptical. So we wrote on his behalf to Fiona MacTaggart herself. She fessed up immediately. She had looked the figure up from a reputable source. But then disaster struck.
- [25:20]
J
I did the maths and scribbled a figure on my report to read out during the hearing. When I calculated it again, following your query, I realised that I'd misread my handwritten calculation. We weighted 7.6 centuries rather than 76. But even though the figure's wrong by a factor of 10, I think all of us think it's just too long to be kept hanging on.
- [25:45]
B
We forgive you, Fiona. It's an easy mistake to make. I made pretty much the same mistake myself last series. But actually, there is another thing that bothers me about the claim that the great British public have spent 7.6 centuries on hold trying to reach government helplines. It just doesn't help us understand the problem. The first rule of more or less is always ask if it's a big number. And off the top of your head. It's really hard to know whether 7.6 centuries distributed across 60 million-odd people is a big number or not. Let me just type this into the nearest calculator. Damn. 7.6 centuries is 760 years. That's 6.6 million hours. 6.6 million hours divided by 60 million men, women and children is about six and a half minutes per person per year. Hanging on the telephone to the government, is that bad? I'm really not sure. Here's a more interesting question. When someone phones up the government, how long on average do they have to wait? But that's never going to make a parliamentary soundbite, even if you do accidentally multiply it by 10. And that's all we have time for this week.
- [26:56]
A
Tim, before we go, you should mention the award More Or Less has won.
- [27:00]
B
Ruth, we don't talk about the awards that more or less wins.
- [27:03]
A
But you have to mention this one. It's the Royal Statistical Society's award for statistical excellence in journalism.
- [27:09]
B
We don't mention awards. I told you when we won the same award in 2010. We don't talk about awards. We've been commended by the Royal Statistical Society for our journalism every year since then. And we don't mention that either. It just doesn't accord with our core value of enormous humility.
- [27:27]
A
Actually, you're right, Tim. I'm sorry.
- [27:30]
B
If there's a number you'd like us to investigate, email us at more or lessbc.co.uk. you can get our free download at our website, BBC.co.uk this week's episode of More or Less was brought to you by the number 49. Until next week, Goodbye.
- [27:47]
A
More or Less was presented by Tim Harford, the Financial Times undercover economist. It was produced by Ruth Alexander and made in association with the Open University. You can Download many more Radio 4 programmes for free. Find these@BBC.co.uk Radio 4.