← More or Less
More or Less
Election and Adultery Special
00:23:08
Tim Harford and a panel of experts discuss pre-election polls and election fact checking.
Loading summary
Transcript87 lines
- [00:00]
A
Thank you for downloading this programme from BBC Radio 4. I'm Tim Harford. Hello, and welcome to an unprecedented live edition of More Or Less. We just thought the programme would sound a lot better unscripted and massively sleep deprived. Stay with us and let us know if you agree. The election is topmost in our thoughts. Of course. And I'm joined by some of the country's most respected fact checkers and opinion pollsters, if indeed the opinion pollsters still are respected. And we'll be discussing the numbers behind the campaign and the polls and the result. But for variety, we'll also be discussing the statistics of adultery. Now, before any of that, I am being told by our Trumpetonshire correspondent, Ruth Alexander, that the much anticipated declaration there may be imminent.
- [00:48]
B
Yes, Tim, it's been a tense wait here in Trumpton Town Hall. There have been counts and recounts. This is a key marginal, but I think, yes, the results are about to be called. Here we go.
- [01:00]
C
I, Sir Gordon Murray, being the returning Officer at the election of the Member of Parliament for Trumpetonshire constituency, do hereby give notice that the number of votes recorded for each candidate is as. Barney McGrew, Labour, 14,246. Farmer Bell, Conservative, 23,252. Dibble, UKIP, 7,133. Ms. Lovelace, Liberal Democrat, 1,827. Windy Miller, Green Party, 1,370. The percentage turnout.
- [01:44]
B
Well, he's done it. Farmer Bell's done it for the Conservatives. He's even increased his majority.
- [01:49]
C
Declare that Farmer Bell is duly elected as Member of Parliament for Trumpetonshire constituency.
- [01:55]
D
Ladies and gentlemen, on behalf of the people of Trumpton, I am delighted, absolutely delighted. Thank you all very much indeed.
- [02:06]
A
It's been a successful night for the Conservatives across the country and in Trumpetonshire too, apparently. I am now joined in the studio and down the line by some of the nation's leading polling experts. One of them is the BBC's very own David Cowling. He joins us from the epicentre of election watching BBC Millbank. I must remind David that he called Trumpetonshire for Labour last week. Ben Page, the chief executive of Ipsos Mori, is here and so is Deborah Mattinson from Britain thinks her research company has been running focus groups gauging opinions during the campaign. By no means least, David Spiegelhalter. Sir David Spiegelhalter, professor of Statistics at the University of Cambridge. And everybody is giggling because one of our research team, Hannah Moore, has just brought in a marzipan Hat.
- [02:55]
E
Paddy Ashdown, is it?
- [02:56]
D
Yeah.
- [02:56]
A
Well, you may remember.
- [02:57]
B
Yes.
- [02:57]
A
Lord Ashdown last night said that. Said that if the exit poll was correct, he would eat his hat and
- [03:03]
F
Alistair Cowboy was going to eat his kilt.
- [03:06]
A
And I think he said he was willing to do it as long as Andrew Neill procured a marzipan hat. So we have a marzipan hat here. Sadly, we don't have Lord Ashdown. We have got Ben. Ben, one of the country's leading pollsters. Should you be eating the hat, Ben?
- [03:21]
E
No, I don't think. Well, first of all, I haven't promised to eat it. I did promise actually to eat my socks on live television If UKIP achieved 18% of the vote share at the general election, prompted on by banter on Twitter with various avid UKIP supporters who felt that my polls were terribly, terribly biased against their dear party. But, no, I'm not going to be at the cable.
- [03:44]
A
They got 13%, we should say. So you were 5 percentage points away from sock eating.
- [03:49]
E
I was. And at the last general election, Ian Dale promised to run naked down Whitehall if the exit poll, which David commissions each time, David Cowling commissions each time for the BBC and ITV and sky as a consortium, was correct, because he's yet to do it. People have a bad record of not delivering on their promises or their days.
- [04:07]
A
But did you and your fellow opinion pollsters deliver before the election? Because people were telling us that this election was very, very close, that an overall majority was extremely unlikely. I mean, of course, everyone always hedges their bets and says it's difficult to call, and so on. Deborah Mattinson, you told us it's too difficult to call.
- [04:24]
F
I did.
- [04:24]
A
You also told us.
- [04:25]
F
That's right.
- [04:26]
A
Well, actually, in a way, you were right. But you also said Ed Miliband would be Prime Minister, so
- [04:35]
F
clear that that was. That wasn't based. That. That was assuming that we had a very close contest and an uncertain outcome, and it was because he has more. He had more options available to him to form a government.
- [04:48]
A
Yeah. But we didn't. But we didn't have it on certain outcomes. So, Ben, were your polls right?
- [04:54]
E
Well, I think it's. The poll, of course, asks people what they will do tomorrow. So if you're asking it three weeks away, that's about three weeks away. And now some pollsters refuse to predict. Polls don't predict pollsters, and the people running them do. And, of course, it's unprecedented to have an election campaign where the polls, particularly the two main parties, are so flat and so level with what seemed like just noise in the data. So it is an upset when you actually go through the gubbins of the polls, as I'm sure David has done and we in the industry will be doing. The exit poll again was pretty accurate. It slightly underestimated the Conservative number of seats, but overall it was big surprise because of course it showed the Conservatives well ahead, whereas virtually all the upholsters and all of the academics and other people running models were predicting, you know, everybody, both the big parties being well short of the majority. Now that clearly hasn't happened. Yeah, when we look at the. I'll look at, I'll give you my own opinion poll and my dissection of that. So the one that we did on Wednesday, we had the conservatives on 36, they got something like 37.8, so we were within the margin of error. But of course the expectation is that it will be perfect. But we overestimated labor, we were pretty accurate on Ukip, we were spot on on the Lib Dems, we were okay on the Greens, but we overestimated Labour and we are looking at that now. That's happened in the past. And I think the other difficulty in all of this with our electoral system is that very small percentage differences in vote share for each party lead to very large differences in the House of Commons, which exacerbates any problems or otherwise in the opinion.
- [06:31]
A
You make a small mistake and you get the wrong.
- [06:35]
E
Just to give an example of how finely balanced it is when Tony Blair got a 36% share of the vote and got the 64 seat majority the following election, Cameron gets what, perhaps 37 and is 20 seats short. So just 1 or 2% can make a huge difference. And with these opinion polls and these sample sizes, no respectable academic would ask you to go and do an opinion poll in sort of 24 hours to pinpoint accuracy because, you know, we don't do that for serious academic studies. We are taking months.
- [07:03]
A
We have a respectable academic in the studio. But before I turn to him, I wanted to ask David Cowling. David, you've commissioned all of these polls. You're the BBC's expert on trying to interpret the polls. Do you think the pollsters deserve blame for their performance?
- [07:17]
G
Well, I think they certainly need to provide answers. And I mean, forgive me, I mean there are plenty of times when journalists and myself requires to give answers. So I'm not saying that saying this from some great situation on the top of Mount Olympus, being pious about it, but I looked at the nine companies that were polling and I looked at their last poll before polling day and I think all nine of them understated. Sorry, eight of the nine understated the Conservatives and all nine overstated Labour by varying degrees, but it seemed to be sort of systemic. And we've had this in the past, as Ben rightly identifies, you know, for the four elections from 92, they overstated Labour and of course in 2010, just to make life confusing, they understated Labour. So. But yes, I think there is some, something that has to be answered and I'm sure, you know, the pollsters, when they get past today and all the, you know, the sort of half bricks that will be thrown at them will have a serious interest in doing this in any event, both as professionals, but also to ensure that much of the polling, most of the polling they do is nothing whatever to do with political opinion polling. It's much more serious and they'll want to make sure that their systems work for that research as much as for the political polling.
- [08:32]
A
David Spiegelhalter, do you think this margin of error thing is a reasonable excuse? The pollsters were within the margin of error?
- [08:38]
H
It could be, if it's used on a single poll. A thousand, you know, is about plus or minus 3%. But actually, as David said, when you put all these together, they were showing a systematic bias in a particular direction. So I don't think margin of error is something you can get away with in this case. And I also think this is a serious application of statistics. People take a lot of notice of it. I'm sure even the political parties, a lot of their strategies were being driven by these polls, because it wasn't just the polls. You know, if you look on the betting markets, William Hill were offering 9 to 1 against a conservative majority the day before the election. So, you know, it wasn't just the polls. How much that was influenced by the polls, I don't know. So I think there are really serious questions to answer about that. And there has been, as David said, a consistent overestimate of the Labour vote over elections. Last time, 2005, it was actually an overestimate of Lib Dem and an underestimate of both the Conservative and the Labour vote. I think they're really serious questions to answer. For example, can I ask, in a telephone poll, what's the response rate?
- [09:41]
E
Well, it's going to be maybe 30%. It depends how long you want to get. It depends how much time you've got, because obviously we can get with face to face surveys, 75 or even 80% response rate, we will go and wait outside your house for you to come home. But doing relatively low budget, fast turnaround surveys and with Internet surveys, of course, the response rate is going to be even lower. You've got the people who are online who like joining an Internet panel.
- [10:06]
A
So are we saying here that this is basically, Ben, you're saying it's quick and dirty stuff, people shouldn't take it too seriously?
- [10:12]
E
Well, we are trying to do it to the best that we can within the sort of confines, the time pressures and some of the budgetary pressures that we're working to. But also, of course, the polls, you know, they have a reasonable track record. I mean, I think you're all wanting it to be everything, to be perfect. We would probably say, you know, it's never going to be perfect.
- [10:30]
A
Deborah Murphinson?
- [10:31]
F
Yeah, I was going to say two other things that relate to that. One is this, this sense of the kind of herding of pollsters together. As Ben says, it's, you know, polls do what they do, but it's pollsters that make the predictions and they make the predictions with half an eye on what the other guys are saying and they feel very anxious if they move away from that.
- [10:48]
A
So I was very struck, actually, I wanted to ask you that. Survation published a blog post this afternoon saying that they had conducted a poll, they'd got the answer right and they didn't have the courage to publish it. Now, I don't know what to make of that.
- [11:03]
E
I was the only pollster to say there'd be a hung parliament in 2010. And then immediately I noticed that every other single pollster had said that there was going to be a Conservative majority in 2010. I said, well, of course the poll says that there'll be a hunger, but I'm sure that by Thursday, the day, you know, the numbers will have moved.
- [11:17]
F
Before I knew Survation had done this, I was saying, I wonder how many people are looking back at data they maybe, maybe had two weeks ago kicking themselves.
- [11:25]
A
But if pollsters then are self editing like that, wouldn't that potentially introduce a systematic bias?
- [11:30]
F
Of course it does.
- [11:30]
E
I think the absolute truth is, Tim, is that half the time, and I've been doing the job for 28 years, half the time the data is right, but the pollster is dubious, is skeptical about the data. The other time the data has some issues and the pollster may be right, but you never quite know which, which time it is.
- [11:45]
F
And then the other thing I would say that the media, I think are culpable here as well. We've talked about this, you know, the strive for perfection with data that often has been produced quite quickly and imperfectly in some instances. And yet, you know, it's often published in a way that doesn't talk about margins of error, that, you know, wrongfully compares 1% here or there as if there's a real movement when there isn't.
- [12:07]
E
Labour surges ahead, 1% lead, you know, let.
- [12:10]
A
Can I just ask David Cowling, does it matter if these polls are all wrong?
- [12:15]
G
Yes, I think it does because they're done seriously and as we've already rehearsed, as colleagues already rehearsed, they're taken seriously. They become an important part of the debate and the campaign. Anybody who could imagine they've gone through the last six months, last six weeks of the campaign, sorry, it felt like six months, but the last six weeks without thinking that the opinion polls were an important part of the political debate and discussion clearly hasn't gone through the same election as I have. So yes, of course they are serious and they're done seriously by serious people. You have them in the studio with you. They're not a bunch of cowboys who are just sitting there carelessly doing things for the fun of it.
- [12:53]
A
We have a hat. Nevertheless, they may not be cowboys. David Cowling, just very briefly remind us, what is it about the exit poll that made it so much more accurate than the opinion polls in general?
- [13:07]
G
Well, it's a very different kettle of fish and Ben's colleagues produced it with GfK NOP. They did all the hard work. And what it is, is it's sampling about 140 contact points out of 39,000 polling districts. And we're looking mostly at polling stations which we have used before. And what we're doing is measuring change. And so when I was, everybody was shocked at 10 o', clock, I was shocked at 2 o' clock because I was sitting there watching this stuff coming through.
- [13:35]
A
And of course you're able to ask people how they voted rather than how they might vote in a week or if indeed they might, because, you know
- [13:41]
E
they voted, which is a massive advantage.
- [13:43]
G
And you know they voted because the exit poll is what is the names on the T? We ask people who've literally just come out of the exit, out of the
- [13:49]
A
port to give it. David Cowling, thank you very much. Ben Page, Deborah Mattinson, Sir David Schapegelhalter, thank you all very much. You are listening to a special live edition of More or Less. Our email address is more or lessbc.co.uk. more politics in a moment, but first, here's Hannah Moore.
- [14:07]
I
I was in the office one morning working on a script when the emails started to arrive. Message after message telling me the same thing. How a sleepy haven in the East Midlands had become a hotbed of cheating and backstabbing. Beeston in Nottinghamshire, they said, is the most adulterous town in the uk. And Loughborough in Leicestershire, just a few miles from where I'd grown up, was also in the top 10.
- [14:36]
A
The newspapers declared 2.5% of Beeston's population are having a sexual relationship with somebody other than their husband or wife.
- [14:46]
I
941 affairs reportedly taking place right now. What was going on? A listener from Beeston had got in touch with a theory.
- [14:57]
D
When I heard that we were right at the top of this pole, I did think back to the people that I've known while I've lived here and actually quite a few have been involved in affairs. I wandered through Beeston on quite busy day and I was struck by just the sheer number of really attractive people in the town.
- [15:21]
I
Could Stephen be right? Are people in Beeston just too sexy to resist each other? Or is there something more sinister behind these figures?
- [15:33]
A
There we are, the most adulterous town in Britain. Hannah, more reporting. And here to make us help of this, help us make sense of those sordid goings on, is David Spiegelter. He hasn't left just yet. Not only a statistics professor, but also the author of a new book, Sex by Numbers. And I mean, you have that certain animal magnetism about you, David, and I understand you are actually a former resident of Beeston.
- [15:55]
H
I lived in Beeston when I was first married. But that does not give me any personal knowledge of these issues.
- [16:01]
A
I mean, is it. Is it famously adulterous town?
- [16:04]
C
I don't know.
- [16:05]
H
It's a universe where a lot of university people live, so that possibly could contribute. But other places, you know, almost as high as Beeston is runcorn. I mean, it's very difficult to find some pattern, I think, in these cases.
- [16:14]
G
Should we take.
- [16:15]
A
I mean, this sort of number is published in the media all the time, you know, gets attention. It's got a number, it's about sex. Should we take this kind of thing seriously?
- [16:23]
H
Yeah, not very at all. I mean, this is a great publicity for the site, but. And they assume that everyone's regular registered with their site, which you can do women register free. I was on there this morning, just checking up on. On what?
- [16:35]
A
On adultery.
- [16:36]
H
On Adultery and beaston and you know the 80,000 women that are registered there and in, in the country and, and, and men can get a free, a free trial on that. So it doesn't represent actual successful affairs
- [16:48]
A
represents lecturer statistics professors claiming to be doing research.
- [16:52]
H
Well, exactly. But I mean a lot of people do use dating sites and of course actually the to do serious statistical work on people having affairs or is also that term the cheating term is very judgmental when it's done seriously is considered the Americans use this term extra dyadic partnerships or you could just say concurrent partners that the sex survey modern sex survey will do. It's very difficult to do because it's not behavior that people admit to. But the good sex surveys can find out roughly how many people are having
- [17:20]
A
affairs but they are not being conducted via self registering websites. Thank you very much, Sir David Spiegelhalter. And we're going to go back to politics because for the last six weeks Ruth Alexander and I have been popping up on Radio 4's PM programme and shoveling away steaming mounds of politician maths. And we are not the only ones who've been trying to impose a bit of numerical discipline on the election campaign. I am joined in the studio by Paul Johnson, director of the Institute for Fiscal Studies, Will Moy, director of of Full Fact and Federica Coco from the Mirror's amped fact checking arm. Welcome to all of you. Do you think the election's been a good one? The campaign has been a good one for people who like numbers and facts, Federica?
- [18:05]
J
Yeah, definitely. I mean we've definitely seen a success in our push to introduce statistics and fact checking to a tabloid audience, which previously perhaps weren't that. It was previously something that tabloids didn't really cover and we've had a lot of fun with it.
- [18:28]
A
Yeah, I mean, the Daily Mirror isn't the obvious place to go for fact checking.
- [18:32]
J
Oh, well, yeah, I suppose you're right. But it is now, at least in the website. We've done our bit. We've tried to fact check all parties and as I said, we had a lot of fun with it, especially with our charts with the wrong baseline.
- [18:46]
A
Yeah, very retweetable. Have you had a big response?
- [18:49]
J
Yeah, I would say so. I mean, obviously we've had a big response on Twitter and Twitter is a universe in itself, but our charts have made it into the paper as well, so I would consider that a success.
- [19:01]
A
Let me turn to Will Moore. You are the director of Full Fact and you've been trying to basically live fact check the elections debates provide almost round the clock fact checking service. Do you think that fact checking is the new thing? That people are just really into it and that people have been relying on you for their data about what the politicians have been saying?
- [19:28]
K
Two halves of that question. Do I think that fact checking is the new thing? I really hope not. I hope that fact checking can clear away the confusion of politicians throwing numbers past one another so that people can get onto the real arguments, the real politics, which is what do we do with the facts? But we have seen local broadcasters, national broadcasters, international broadcasters coming to us saying we're doing an event, will you come and live fact check it? We've seen every newspaper from the Sun, Fruit of a Guardian come and use our stuff. So it's been extraordinary what the appetite for this stuff is both from individuals online sort of saying, I've found full fact, thank goodness to major broadcasters is beginning to expect that if we are putting politicians in front of a public, we want people there who can translate the claims into something that you have a means of judging.
- [20:16]
A
And very often you have been using as a source the Institute for Fiscal Studies. We have been using as a source the Institute for Fiscal Studies. Paul Johnson is here, director of the Institute for Fiscal Studies. You are almost the arbiter of claims about taxation and spending and deficits and so on. But I mean the. You have clashed with some political parties. You've been accused by, I think by the SNP of getting your sums wrong. What did that feel like?
- [20:41]
L
It was unusual. I mean, the parties don't tend to challenge what we say because we do tend to be right. And in this case we were right. And the snp, I think were just wrong to challenge the numbers. I mean, actually, to their credit, had they admitted that?
- [20:55]
D
No. Okay.
- [20:56]
L
But to their credit, I mean they said very clearly what their spending plans, what were in numeric terms. It's just that their numeric terms didn't meet their rhetoric.
- [21:05]
A
We had a similar thing from Ed Miliband, the Institute of Fiscal Studies said, well, the Labour Party proposed balancing the current budget and that implies an extra £30 billion of spending. And they just said, no, it doesn't. It was a very curious conversation we kept having with them.
- [21:23]
L
Well, it was a strange one. The issue with what the Labour Party was saying is that they were in a way, I think being something very cleverly but very deliberately vague about actually what they wanted to do in terms of balancing the books. They said at some point during the Parliament we want to get to current budget balance. That means current budget balance means you're happy to borrow in order to invest. And we invest about 25 to 30 billion pounds a year at the moment. So essentially the only assumption that we could make, because they, I mean, they were being deliberately vague, was that BY Somewhere around 2019, they would get the deficit down from the 90 billion it was last year to about 30 billion.
- [22:04]
C
Yeah.
- [22:05]
A
So, I mean, vagueness, I think, is obviously a weapon for the politicians. My impression was that often it wasn't that the politicians got their facts wrong, it was that their facts were just profoundly unhelpful or misleading. Do people agree?
- [22:17]
L
I thought there was a great example in the Budget, actually, where, you know, you have this perfect example of the Chancellor and the Leader of the Opposition talking across each other in exactly the way Will describes. George Osborne says we're all 900 pounds a year better off. Ed Miliband says we're all 1,600 pounds a year worse off. Now, I can tell you exactly what those numbers mean. They just mean two completely different things.
- [22:38]
A
They mean absolutely different things. Paul Johnson, Will Moy, Federico Coco, thank you all very much. That is all we have time for. If you have any numbers you would like, like us to take a look at, do please get in touch. Our email address is more or less@BBC.co.uk, our website, BBC.co.uk more or less. We will be back next week. Until then, goodbye. And just a reminder that you can download more podcasts@BBC.co.uk Radio 4.