← More or Less
More or Less
Big numbers
00:27:41
More or Less examines this week's claims and counter-claims about VAT, exposes some -...
Loading summary
Transcript87 lines
- [00:00]
A
Thank you for downloading this week's More or Less podcast. Here's Tim Harford.
- [00:05]
B
Hello and welcome to More or Less, the show that does to dodgy statistics what Alistair Cook does to the Australian Paceman. This week we'll be covering the Archers, the dangerous concept of proofiness. We'll revisit the Ashes predictions of Jack the Psychic Simeon, and we'll pull apart the claims and counterclaims about the rise in vat. But first, let's turn to a story on contraception, which was so badly reported on Channel 4 and elsewhere, it made us want to scream. Channel 4 News has discovered more than a thousand complaints about Implanon have been received by the medicine's watchdog at a time when the device is being hailed as the future of family planning. Figures obtained by this program reveal that 584 women who had the hormone filled tube inserted into their arms have reported unwanted pregnancies to the Medicines and Healthcare Regulatory Authority. Some women have been forced to have terminations and suffered the breakdown of their relationships. Well, there was an awful lot more like that. But one thing that was missing from that report and much of the BBC's coverage and that of the Guardian, the Daily Mail and the Telegraph was any sense of context. Obviously, 584 unwanted pregnancies is 584 too many. But before trashing the Implanon contraceptive, you might have thought Channel 4 would ask how its success rate compares with that of other contraceptives. And they didn't. But our very own Charlotte Pritchard did. What did you learn, Charlotte?
- [01:31]
A
I did, Tim. According to the Department of Health and the manufacturer, 1.4 million women have used Implenon since 1999 when it was first licensed. In order to work out the failure rate, we need to convert that to women's years of use, say 1 million women years, if those women used it for less than nine months, which is
- [01:48]
B
actually quite conservative because it was designed to be used for three years.
- [01:51]
A
Yeah, it's a three year implant. So if we take the 584 women who reported their unwanted pregnancies, the failure rate for Implanon is less than 0.06%, which is much lower than any of the reporting or even the manufacturer suggests.
- [02:05]
B
Okay, so I guess we need a relevant comparison. So a million women for a year using other contraceptives. Do we have any data on that?
- [02:14]
A
So let's take the Pill, for example. According to the association of Reproductive Health Professionals, which is in the States, women with perfect use of the pill, 0.3% of women have unexpected pregnancies in the first year, so that is around 3,000 women.
- [02:30]
B
What about with typical use of the pill?
- [02:31]
A
So typical use is much higher and it's 8% failure rate, which is 80,000 women. 80,000 who have unexpected pregnancies in the first year of use.
- [02:41]
B
And condoms.
- [02:42]
A
So if we take condoms, if you go for the typical use, the failure rate is 15%. Which means there are 150,000 unwanted pregnancies in the first year of use for every million women.
- [02:53]
B
Well, is there any method of contraception that works?
- [02:56]
A
There is male sterilization.
- [02:58]
B
I should hope so.
- [02:59]
A
But even that has a failure rate. We'd expect 1500 pregnancies per million women per year.
- [03:05]
B
So how does Implanon compare to all these?
- [03:08]
A
Well, according to the arhp, it's the most effective of all these methods. Three times more effective than typical male sterilisation.
- [03:16]
B
So in fact, the story is that Implanon is by some margin the most reliable contraceptive available.
- [03:21]
A
That's not completely fair. There are some concerns that the contraceptive has sometimes been improperly inserted and women have reported side effects. And also, while we know that 584 women have alerted the NHS of unwanted pregnancies, there may be others out there who haven't. But one thing's for sure, if 1.4 million people use a contraceptive, any significant failure of that contraceptive is going to produce thousands and thousands and thousands of unwanted pregnancies. The figure of 584 pregnancies without context is completely misleading.
- [03:53]
B
Thanks, Charlotte. You know, this is the kind of thing that really winds us up on more or less. Channel 4 and elsewhere presented this as some kind of shocking failure of Implanon without bothering to collect, even very roughly, the basic data that their anxious viewers needed to make any sense of the story. Oh, I hadn't realised we'd be this high up. For many Radio 4 listeners, the biggest story of the week wasn't contraception or vat, but something far graver.
- [04:23]
C
Roofs do tend to be on top of things.
- [04:27]
B
I'm careful, it's frosty. Yeah, it's so much easier in the light. That's the Archer's Nigel Pargetta on the icy roof of Lower Loxley hall in the middle of a gale. What could possibly go wrong? Oh, I guess that could go wrong. That's quite a scream. The program could have used the classic Wilhelm scream as heard in Star Wars, Batman, the Lord of the Rings, Raiders of the Lost Ark, Kill, Reservoir Dogs and Toy Story. That's brief and snappy. But instead, they allowed Graham Seed, the actor who played Nigel, to produce his own rather drawn out swan song, which prompted our listener, Giles Catchpole, to send us. Given that Nigel Pargetta's death yodel lasted five seconds, how high does that make the roof of Loxley Hall? I reckon it must be quite a house. Now, this same question has also been floating around the Internet and others have tried to answer it. For the record, here's the official more or less view. We actually timed the scream and it lasts about three and a half seconds, not five. If you remember your high school physics, you'll know that Nigel's flailing form would have accelerated at almost 10 meters per second per second. He'd have started at a speed of zero and finished the first second at a speed of 10 meters per second, covering about 5 metres. During the next second, he'd have fallen 15 metres. During the third, 25 meters. We've taken the liberty of ignoring air resistance because terminal velocity, the speed at which air resistance balances the force of gravity, is over 55 meters a second. According to our calculations, given 3.5 seconds, Nigel would have fallen a whisker over 60 metres. That's the height of York Minster, and it suggests that lower Loxley hall is as much as 20 storeys high. Quite a pile. You're listening to More or Less with me, Tim Harford, in association with the open university. I'm imharford on Twitter. Our website is BBC.co.uk more or less, and you can use our email address more or lessbc.co.uk to send us complaints about our lack of respect for the late Nigel Pargetter. Almost lost in the controversy about poor Nigel was Tuesday's increase of VAT from 17.5% to 20%. Needless to say, the Shadow Chancellor, Alan Johnson, thought it was the wrong move. As he explained on the day to the Today Programme's Evan Davis, in principle,
- [06:54]
C
you would like VAT not to go
- [06:56]
B
up and instead, at some point, not
- [06:58]
C
now, National Insurance to go up by more.
- [07:01]
B
Yes, that was the argument at the general election. That's still our argument now, that National Insurance actually is a big deal.
- [07:07]
C
The Chartered Institute for Personnel and Development
- [07:09]
B
say that the VAT increase is going to cost 250,000 jobs, whereas the national insurance increase would cost 75,000 jobs. So if National Insurance was a tax
- [07:18]
C
on jobs, then VAT is three times
- [07:21]
B
the tax on jobs.
- [07:22]
C
We appreciate your clarity.
- [07:23]
B
Thank you very much. As you heard, Alan Johnson's numbers come from the Chartered Institute for Personnel and Development, and their chief economist is John Philpott. So I asked Dr. Philpott whether Alan Johnson had quoted the CIPD correctly.
- [07:37]
D
Yes, I think so. We calculated first what was likely to be the impact of the VAT increase on the amount of money people would spend on the goods and services covered by the increase, and then what impact that would have in turn on on the demand for labour by employers who are producing those goods and services. Now, that suggested that the increase would probably cut consumer spending in the order of half of 1%. We then extended that to look at what kind of impact that might have on demand for labour and employment in the economy. And we calculated that the impact there would be a little under 1% of total employment, a small number in percentage terms, but nonetheless, 250,000 jobs in the economy as a whole.
- [08:28]
B
Just want to check that. We're comparing like with like here. You were comparing two different policies. So the coalition introduced this VAT rise. They scrapped a planned national Insurance rise. So, you know, you're saying 250,000 jobs could be lost because of the VAT rise. 75,000 jobs would have been lost if the national Insurance rise had been proceeded with. Would those two policies have raised the same amount of tax revenue?
- [08:55]
D
Well, in rough terms, the impact would have been comparable. Where it becomes uncertain is that, you know, clearly if you're looking at the impact of the VAT increase, you've got to look at what impact that has on spending, unemployment in the economy. And that, of course, will reduce some of the tax take. The impact of the increase in national insurance was quite interesting. People tend to think of it as a tax on the employer, one that raises wage costs, so it causes the employer to hire less staff. But we know that analytically what tends to happen with a payroll tax is that it gets passed on to workers over time in the form of reduced wage increases. And so the impact on wage costs of the National Insurance increase, although it can be considerable when it's first introduced, tends to subside over time. That's the reason why we see that having a lesser impact.
- [09:55]
B
Now, I wasn't really sure that the two tax rises, one in National Insurance proposed by the last Labour Chancellor, Alistair Darling, which the CIPD says would cost 75,000 jobs, and one in VAT introduced by George Osborne, which the CIPD says will cost 250,000 jobs, I wasn't really sure that these two tax rises would actually have raised the same revenue. So I asked again, were we comparing like with like?
- [10:19]
D
That was our estimate and assertion I mean, we were looking specifically really at the impacts of these things on the demand for labour rather than, you know, their broader revenue raising potential. But our assertion was that the National Insurance increase was going to be, you know, sort of an alternative to the VAT increase.
- [10:41]
B
Perhaps I should have done my homework a bit better before I interviewed John Philpott, but something was nagging at me. So later on in the interview I went back for a third try, thinking about this comparison between a VAT increase and the National Insurance increase. Now, according to your estimates, the VAT increase would cost 250,000 jobs over five years. The national insurance increase would cost 75,000 jobs. And these two taxes are, you say, roughly the same magnitude?
- [11:10]
D
Yeah, I mean, you'd have to check, double check on that fact, but that was our starting assumption.
- [11:17]
B
Okay, well, we did double check that fact. According to the official budgets published by the treasury, the VAT rise is expected to raise just over £12 billion in tax revenue next tax year. Alistair Darling's changes to National Insurance were expected to raise just over £3 billion, and George Osborne has broadly reversed that policy. In short, the George Osborne tax rise that the CIPD expects will cost 250,000 jobs is about four times as big as the Alistair Darling tax rise that was expected to cost 75,000 jobs. That wasn't the only VAT related controversy of the week. Here's the Chancellor, George Osborne, followed by the Labour leader, Ed Miliband. If you look at a distribution of the population by how much they spend, actually VAT is progressive and that when you're looking at an indirect tax, that is a fair, fairer approach to deciding its progressivity.
- [12:09]
E
George Osborne should stop treating the British people like fools.
- [12:12]
C
He claimed this morning that this VAT rise was fair and everybody knows that it's poor and middle income families that will be hit hardest.
- [12:22]
B
Well, is VAT progressive or not? You may recall that we discussed this with the Institute for Fiscal Studies back in June after the VAT rise was announced. And you can hear the full interview on our website, BBC.co.uk more or less. But to summarize, a tax is progressive if it takes a greater proportion of income from those with more income. Taking the usual measure of income, the VAT rise is not progressive. However, the IFS argued that it was appropriate to look not at the households with the lowest measured income, but at the households with the lowest measured expenditure. Their reasoning being that spending is a better guide to lasting poverty than measured income. If you look at the VAT rise in this way, The IFS told us that it is is progressive, which is the case George Osborne was making. But to be fair to Ed Miliband, that is not the most conventional measure of whether a tax is progressive. George Osborne also cited the International Monetary Fund as supporters of indirect taxes such as vat. Of course, there's a debate, but most economists, and certainly the best international evidence from the IMF and the OECD is that countries dealing with large deficits should increase their indirect taxes. That's true, but it's not the whole story. The treasury sent us an IMF report from November 2010 which does indeed advocate the use of VAT as a good way of raising revenue, particularly in the United States, which has no vat, and Japan, which has a very low rate. However, the IMF points out that VAT needs to be used alongside income tax because it's income tax that provides the progressiveness of the tax system. And the IMF argues that VAT can be a very efficient way to raise revenues if it's applied without lots of exemptions. The British VAT system has very broad exemptions. Just over half of consumer spending is subject to the full rate of vat. And the IMF says the UK system of VAT is one of the least efficient of the 16 countries it examined. If George Osborne really took the IMF advice seriously, he'd be introducing full rate VAT on food, children's clothes and domestic fuel. You can imagine what that would do for the tax's progressiveness. If you're a loyal listener to more or less, or even if you've only been listening for the last few minutes, you'll have noticed that we think numbers can all too easily be used to confuse or to mislead. And we aren't the only ones who think so. Charles Seif is a professor of journalism at New York University and author of a new book, Proofiness, which exposes what he describes as the dark arts of mathematical deception. When Charles Seif came on the program to tell us about proofiness, he began by recounting the tale of a once obscure and now infamous Senator who used a bogus number to devastating effect.
- [15:03]
F
The senator in question was Joe McCarthy, and he held up a sheaf of papers and said, I have here in my hand a list of 205 Communists working for the State Department, and that number sent a jolt of electricity through Washington. Even the White House, who suspected that McCarthy was lying, were surprised by what he was saying because the specificity of the number made it look like he really had a list of people.
- [15:28]
B
So when a number is added to a story, it suddenly gives that story weight. You distinguish between three different kinds of. Well, I suppose we should call them proofiness. Three different kinds of ways numbers can lie to us. You talk about Potemkin numbers and disestimation and fruit packing. What are those three things actually referring to?
- [15:50]
F
Potemkin numbers are the mathematical analogues of Potemkin villages. You probably know the old legend of Prince Grigory Potemkin, who set up these elaborate facades of villages throughout the Crimea to fool the Empress into thinking that the region was live and full of life. But if she bothered to alight from her carriage, she would have noticed that these were just fronts, that there was nothing to them. And if you look at an advertisement, you're likely to see them. One of my favorites playing in the US now has to do with a mascara, an eyeliner that says it will give your eyelashes 12 times more impact. 12 times more impact. I mean, that's a nonsense number. That means nothing. How do you measure impact? By how loud the lashes clunk together when the person blinks.
- [16:35]
B
So those are Potemkin numbers. Now, what about disestimation?
- [16:38]
F
All numbers are fallible. They represent measurements in the real world, and all measurements have errors associated with them. No matter how small those errors may be, those errors still exist.
- [16:50]
B
This problem came to the fore during the disputed presidential election of 2000.
- [16:56]
F
Every time there's a close election and 2000 race, you can't get much closer than that. Disestimation comes to the fore. In Florida, the situation was even worse because the errors inherent in the ballots, the ballots were poorly designed and poorly executed. You probably had an error on the order of a percent or so. It was utterly meaningless. If you count them once, it looks like Bush wins by a few hundred votes, you count it again, Gore wins by a few thousand. Count it again, Bush wins by a few thousand. These numbers don't reflect any reality. They just reflect the errors inherent in the measurement. So it was really impossible, given the precision that we can measure votes with, to tell the difference between these two candidates. It's like eyeballing two 6 foot tall people across the room, one of whom is the fraction of a human hair's width taller than the other. And saying, I know with certainty that the person on the left is taller. You just can't do it. You have to declare that to the best of my ability, I can't tell the difference between these two people.
- [17:58]
B
Do you think that's what the US Electoral authority should have Done.
- [18:01]
F
Absolutely. And in that case, there's a procedure that was encoded in Florida law of what should happen next. Strangely enough, the procedure was the victor should have been determined by lot. They should flip a coin.
- [18:14]
B
So your third example of how numbers can be used to mislead us is something you call fruit packing.
- [18:18]
F
Yes, yes. This is a series of techniques that people use to present data and make it look more appealing than it is, in the same way that a greengrocer might turn fruit that's semi rotten into something that's appealing. One type that almost everyone is familiar with is cherry picking. When somebody looks at a set of data and picks out data that supports an argument that they want to make and rejects or ignores data, that seems to undermine it. It happens all the time. And politicians are particularly fond of picking numbers that make it look like something is working when something isn't, or vice versa. Anytime there's a government program in the US or in the uk, people will trot out statistics to show, aha, this program is working wonderfully. When in fact, if you look at the full set of data, it's not so clear.
- [19:09]
B
Charles Seif author of Proofiness Now England are bringing home the Ashes, but here on More Or Less, our celebrations are tempered by a twinge of sadness. Earlier in the series, we asked a monkey, Jack, to predict the outcome of each of the Ashes tests. We did so in honour of Paul the Octopus, who famously correctly predicted the outcomes of eight football World cup matches in the summer. Unfortunately, Jack didn't do too well. Wesley Stevenson is our monkey correspondent. Wes, remind us how Jack got on.
- [19:42]
C
Well, Jack got the first two tests right, at which point I came onto the programme to present his impressive results and his forecasts. But then he got the next two wrong. He got the last final test right, but that was three out of five altogether.
- [19:57]
B
So he's not psychic?
- [19:59]
C
No. But after we broadcast news of his initial success, much of the media thought he might be. The sun reported that Jack the cotton top tamarind has stunned his keepers with his fortune telling skills and could become a massive star if his streak continues. Radio 1, BBC Newcastle Real Radio Metro Radio 5 live, ITV Tyne Tease News look north and Newsround covered the story right so well.
- [20:25]
B
Our patient listeners have no doubt been wondering what on earth we've been up to long enough. So, Wes, it's time to reveal.
- [20:32]
C
All right, well, we wanted to illustrate quite a useful point. It's if you only hear about the successes but not the failures, you get a distorted view. That's what Happened to Paul the octopus in the World Cup. We heard about him because he did well. But in fact there were lots of other animals trying to predict World cup matches. We just didn't hear about them because, well, frankly they were rubbish. If enough animals are making these predictions, we shouldn't be surprised that one is right. To make our point, we didn't just ask Jack to predict the outcomes of the first two tests. We asked 17 other animals too. Right.
- [21:10]
A
Well basically we've got three short clawed Asian otters in here. Their names are Bertie Splashing Gizmo.
- [21:14]
B
They're all five years old.
- [21:15]
A
As you can probably hear they're extremely hungry. 3, 2, 1.
- [21:23]
C
So Gizmo goes for Australia.
- [21:30]
A
Right, so in here we have a common marmosette called Charlie. And we're putting out his favorite breakfast which is porridge. Coming down now we have Mo. She is a cotton top tamarind.
- [21:40]
C
What are you gonna go for, Mo? She's looking at Australia. Although she looks a bit suspicious, I'm not surprised. Oh, she's eaten. Thanks Mo.
- [21:48]
A
In here we have a red eared terrapin called Bob and we're just about to feed him his favorite food which is juicy mealworms.
- [21:58]
C
So this is exactly the same type of octopus as Paul the octopus.
- [22:01]
D
It is, yeah.
- [22:02]
A
RIP Paul.
- [22:04]
C
So we should ate a score. Expect some kind of action from her. But it doesn't look as though there's a great deal of movement at the moment. Well thanks Otto. I'm not sure you've told us very much there, but maybe you know more than we think.
- [22:19]
A
Right, so in here we have a seal called Sophis and he's a common seal. We're going to put the buckets down after 3. 3, 2, 1 and seal straight away. Straight up there very fast. He's gone over to Anna, which indicates an Australian win.
- [22:35]
B
Oh, Sophis.
- [22:37]
C
So that means Sophus has predicted that Australia are going to win the first test at Brisbane. That's very disappointing, Sophus. Now all those animals made legitimate predictions. There was no cheating involved, I promise you. But by the time we'd finished, we had all nine possible combinations of results for those first two, two tests. So we knew one would be right
- [23:02]
B
and that was Jack. So we lied.
- [23:04]
C
We never said anything on air that wasn't true. We said Jack had got the first two tests right and he had. And after those first tests we then revisited Jack and asked only him to predict the outcomes of the final three tests. And he got one of those three right.
- [23:19]
B
Well, which is exactly what you would expect from chance. One in three. Thanks, Wales. Well, one of our regular contributors, David Spiegelhalter, the Winton professor of the Public Understanding of Risk at Cambridge University, has actually been following the story of Paul the psychic Octopus with some interest. And I asked David whether he thought there were any important lessons from our experiment with Jack the pseudo psychic. Simeon.
- [23:42]
E
I think they're really important lessons because all the time we're hearing claims on the radio or the TV about the latest medical treatment that's, you know, made some people better, cured some diseases. And the problem is that if we only hear about successes, it's impossible to judge the true chances. It's like only seeing film of a goalkeeper when he saves penalties and not seeing any of the times he's let them in. We just can't judge how good he is.
- [24:08]
B
And we systematically chose to survey lots and lots of animals to get a successful forecast. Now, you're not accusing the media of systematically deceiving people here. You're not accusing, say, Paul the Octopus, the famous psychic octopus. I mean, you're not accusing his handlers of having systematically gone and interviewed hundreds of octopodes to produce these results. I mean, it's a different process that's producing this sort of deceptive coverage.
- [24:37]
E
Well, it's not that different because the point is that Paul had become famous two years before in Germany, and so other German zoos, for example, there was a hippopotamus, there was a porcupine, all making predictions. Now, Paul got the first three right, but there are many animals making predictions, so it's almost certain one of them would get them right. And that's when he became started becoming famous. The first coverage appeared on the BBC just before the Germany England match, which was the fourth match that Germany were
- [25:06]
B
playing after he sprung to media prominence. Paul got what, five in a row right?
- [25:12]
E
Yes. Even if we ignore the first three that he got right, because we wouldn't have heard about him if he hadn't got them right, he still then got five right in a row. And we where each of the predictions was being announced in advance. And so this was really quite impressive. That's a 1 in 32 chance. That's a half times. A half times, a half, five times in a row. That's the chance he would get that if he was just guessing. And the interesting thing about that is 1 in 32 is about 3%, and that's less than the classical threshold used, 5%, which is generally used to indicate what's called a statistically significant result that is something that's really quite unlikely to have occurred just by chance alone. But, you know, why don't we believe he is psychic? Apparently, that's a statistically significant result.
- [25:58]
A
Yeah.
- [25:58]
B
So we should naively say there's a 31 out of 32 chance that he really is psychic. Now, is that the right way to reason about this?
- [26:07]
E
Well, no, definitely not. And that's exactly the point. We have to go into what's called Bayesian reasoning, which means that we have to combine the evidence from the data that we've seen with what it might have been reasonable to believe about this octopus before we saw that evidence. And the point about that is that I don't know about you, but my initial opinion, if I was told, is this octopus psychic or not, I would say, no, he's not, is absolutely ridiculous. It's an absurd idea. And so even if you present me with considerable evidence, if my initial beliefs is that, well, there's no evidence at all, well, mathematically nothing at all times, quite a lot is still nothing at all. So it hasn't shifted my opinion one little bit. And I think that's what people would generally think. And so what it shows is that we can't interpret just one piece of selected evidence on its own. We have to combine it with our background knowledge and information in order to come to a final conclusion about the plausibility that he really is psychic. I think there's some really deep lessons to get from such a ridiculous figure
- [27:04]
B
as Paul David Spiegelhalter of Cambridge University. That's all we have time for this week, but we'll be back next week. Until then, as always, our website BBC.co.uk more or less for that VAT interview with the Institute for Fiscal Studies, our podcast, our archive and much more. And you can email us with your questions and comments at more or lessbc.co.uk. please do. And until then, goodbye.
- [27:30]
A
More or Less was presented by Tim Harford, the FT's undercover economist. The producer was Richard Knight and the editor was Richard Varden. It was made in association with the Open University.