
Is there any evidence to support the Beecroft Review's recommended changes to law? and...
Loading summary
A
This is more or less a slice of statistical lemon in the pleasant but confusing gin and tonic of media reporting. The programme now airs year round on the World Service, but this is a full length episode from BBC Radio 4. Hello and welcome to More or Less Bearing an Olympic sized torch for Numeracy in the news.
B
Your cheating heart
A
will make you weak. This week I ask whether it's more likely that I'm cheating on my wife or that my wife is cheating on me. Although presumably I should know something about at least one side of that equation. And we'll confess to our own statistical crime under the thousand yard stare of the Lord High executioner of statistical malpractice, Ben Goldacre. But first, the. The leaked Beecroft Report has been stirring up controversy all week. It's a series of proposals for reforming employment law with the aim of boosting growth, written by venture capitalist Adrian Beecroft. And there's been quite a fuss. Here's a tape of some men shouting at each other. The Business Secretary says it is the
C
last thing government should do.
A
Who does the Prime Minister agree with?
D
We need to make it easier for
C
businesses to grow, for businesses to take
D
people on, for businesses to.
A
The report was criticised by the Business Secretary, Vince Cable, who didn't like the proposal to allow employers to fire their employees at will. Mr. Beecroft, in turn, accused Mr. Cable of being a socialist. I should declare an interest here. I briefly worked for Vince Cable a long time ago when he was preaching his socialism as the chief economist of a major multinational oil company. Enjoyable as this personality politics is, what really interests us is the data. The logic is pretty clear on both sides. If employers are able to sack workers at will, they might do just that. On the other hand, if employers are able to sack workers at will, they might be much happier to hire them and give them a chance. But beyond laying out the theoretical argument, what's the evidence that employers should be able to fire their staff at will? And come to think of it, what's the evidence that they shouldn't? Well, I've got the Beecroft report here. At least I. I thought I had it, but my lazy producer seems to have only printed the executive summary. Rich. Rich, could you bring the whole Beecroft Report in here, please?
C
Tim, I've checked and that is actually the whole report.
A
Don't be ridiculous, Rich. This is only 15 pages long. There are no footnotes, there are no citations, there's no evidence. This isn't really a review.
C
Ok, then why don't you call it a pamphlet instead?
A
Well, the Beecroft pamphlet might be an evidence free publication, but perhaps there's evidence elsewhere. So we've been looking for it. One of the obvious sources is the oecd, an organisation which does a lot of work measuring and reviewing the economies and economic policies of rich nations. We spoke to Christophe Andre, acting head of the OECD's UK desk and asked him whether the UK was being held back by excessive labour regulation.
C
In comparison with OECD countries, employment protection legislation in the UK is fairly light. Only the US has lighter employment protection legislation for regular employment and for temporary workers. Unlike Canada, the US and the Slovak Republic has lighter regulation.
A
So the OECD thinks excessive labour regulation could cause a problem, but that the UK itself is a long way from being in that situation. We also called a couple of pro market think tanks, the Adam Smith Institute and the Institute for Economic Affairs. They welcome Adrian Beecroft's logic, but they aren't aware of any particular evidence which supports it. So I called John Van Reenen, the director of the Centre for Economic Performance at the London School of Economics. If anyone would know what evidence exists in this area, John Van Renan would.
E
The conclusion of the vast majority of the studies is that there is actually no significant relationship between employment protection and the overall level of unemployment. For example, one recent study looked at 14 countries over about 15 or 16 years and and found there was no significant relationship. In fact, they actually found there seemed to be a positive effect of tougher employment protection on lowering unemployment. So in fact the opposite of Beer Croft. But that relationship wasn't statistically significant. In other words, statistically you can't reject the idea that there was actually just a zero effect. Interestingly, one of the co authors of that is now currently George Osborne's special advisor.
A
That is interesting. Presumably that's not the only effect. Loosening employment regulation might boost growth, for instance.
E
It might do. But equally people have looked at say GDP per head and again found generally no relationship between the toughness of employment protection legislation and growth or the level of GDP per Headcroft seems to be completely wrong.
A
That's the big macro picture. Have there been any other studies done?
E
One of the criticisms that you can make of these macroeconomic studies is that so many things happening between different countries you can never really control properly for them. So one way to try and dig down into that, if you look at the kind of hundreds of thousands of firms and economies, do they react to employment protection legislation? And there you start to see some effects. So for example, in Italy, when firms get to be 15 employees large, there's a very big increase of employment protection legislation. And in France there's something similar when firms get to be 50 or more employees. I have a study which looks at France, which then looks at what firms do in response to that. And of course, just as you would expect, a lot of firms stay at around 48 or 49 employees to avoid paying the extra cost of crossing that threshold of 50 employees. But those effects are pretty small. So when you start adding those all up to see how much that really matters in the grand scheme of things, it doesn't come to a very big effect.
A
To summarise what I think I'm hearing from you, John, the evidence doesn't really show strong effects, although there are some small reasons to think Beecroft may have a point in certain circumstances. And yet very strong claims have been made on either side of the political aisle. Beecroft himself doesn't refer to any studies in his paper. The politicians don't seem to be terribly interested in evidence. How do you feel, as somebody who runs a center devoted to investigating the evidence on economic performance, how do you feel when this political debate takes place and no one seems to be terribly interested in the numbers?
E
Well, it's kind of profoundly depressing really, because for an issue which is very important how we can get more growth out of the economy, then it really is important, I think, for politicians on all sides to try and look as best we can at the evidence. And especially with an issue like this where there is substantial evidence that could be drawn on.
A
Professor John Van Reenen of the London School of Economics. On the subject of employment, we've been thinking about the latest instalment of the Eurozone crisis. One of the basic narratives is that the hard working Germans are being asked to bail out their lazy Greek neighbours. Is that true? It sounds like a job for our Greek God. Wesley Stevenson and some more Eurostats. Excellent. What have you got for us, Wes?
D
According to figures from the Organisation for Economic Cooperation and development, the OECD, the average Greek worker toils away for more than 2,000 hours a year, which is more than the workers in any other European OECD member. Out of the OECD's 34 members altogether, only the workers of Chile and South Korea come higher. German workers only manage 1,400 hours a year. Close to the top of the OECD's lazy chart, the average Greek is working 40% longer.
A
I'm guessing this can't be the full picture.
D
No. Pascal Mariana is a labour market statistician at the oecd.
A
In Germany, the share of employees working part time is quite high. This represents something like a more than one in four.
D
And it's not just the part time workers. There's another factor that skews the figures and that's the high number of self employed workers. In Greece these workers tend to work longer hours than those on fixed contracts.
A
Still, long hours are long hours. I mean this is not a statistical trick. I mean the Greeks really are working hard. The figures we have are correct.
D
If you accounted for these factors stripping away part time and self employed people and looked at just full time salaried workers.
A
Let me guess, the Germans work harder than the Greeks.
D
No, you take out the part time and the self employed workers and the gap narrows. But the Greeks are still working almost 10% more hours than the Germans. And this is because the Germans take more holiday, they take more sickness leave on average around four weeks more than the Greeks.
A
Now hang on, this part time worker thing has got me thinking. Let's say you've got a person who doesn't want to work full time. Maybe they have young children, maybe they're getting old, whatever. If part time work is available and it's widely available in Germany, they might well take it. And if it's not available, they won't work at all.
D
Yeah, I mean that's the sort of scenario that might mean that there's more work going on in total in Germany, but fewer hours per worker. And to figure that out you'd need to look not at the hours per worker, but the hours per adult citizen of working age. So this would include the unemployed people in early retirement, stay at home, parents and so on. Pascal Mariana says that even on this hours per adult measure, the Greeks still beat Germany, but the gap is even narrower because only 60% of the working age Greeks have jobs versus 72% of the Germans.
A
OK, so how is it that the Germans are bailing out the Greeks and not the other way around?
D
Well, yeah, that's a very good question. But what looking at long hours misses completely is a really important question of what all those long hours are actually producing. And a rough and ready way of doing that is to divide gdp, that's the country's entire production by the number of workers. And, and despite working fewer hours, the average German worker is more productive than a Greek. By this method, Germany ranks 8th most productive by worker out of the OECD countries, while Greece drops down to 24th.
A
Well, thank you Es, and if you would like to go to the BBC news website, you'll find an online calculator where you can compare your working hours with those of the Greeks, the Germans and a number of other countries. You're listening to more or less in association with the Open University and I'm Tim Harford. Now a brief update on a story we covered last week. We looked at the government's claim that 120,000 troubled families are responsible for a disproportionate share of society's ills. The Government's own explanation of its methodology shows that in fact, it counted 120,000 families who were very poor rather than very disruptive. We asked the Department for Communities and Local Government, home of the Troubled Families Unit, for an interview. They gave us a statement instead. And so naturally we interviewed the statement. We had two other questions, though, so here's the update. The Government didn't just say there are 120,000 of these troublesome families, but that they cost the nation £9 billion a year. Given that, we don't have much confidence in the 120,000 figure, we were concerned about where the 9 billion pound number had come from, so we asked the Department to tell us. No interviewee was on offer again this week, I'm afraid, but. But we did get another statement. Welcome to the studio. What do you have to say for yourself?
B
This estimate is based on data analysed in October and November last year by the Department for Communities and Local Government. It brings together information from across Government about the proportion of existing budgets which is currently spent on these families. This includes the cost of taking children into care, police callouts and the burden on the criminal justice system and out of work benefits, among other factors.
A
We replied to that, asking to see the actual data. The Department said no, sorry, the data
B
has not been published.
A
The Government says it has worked out where these 120,000 families live by local authority area and it's found nearly £450 million to spend on helping them. So our question is, is the Government using that data, data we have very little confidence in, to target its money statement.
B
The £448 million three year budget is allocated proportionately to upper tier local authorities in England based on the total number of troubled families in their area, as agreed by their Chief Executive, in the context of the criteria set out in the financial framework for the Troubled Families Programme.
A
Well, there you are. Well, thank you. Statement. Are you alright, by the way? You sound a bit dejected.
B
I am dejected. I hate coming on, more or less. You imbue me with human qualities and talk to me in A way I find deeply disturbing. You made a fool of me last week.
A
Well, I'm sorry if we hurt your feelings.
B
I'm just a statement, Tim. I'm only as good as the material they give me to work with.
A
You're right. You're right. I'm sorry. Let's. Let's change the subject. Do you have any plans this weekend?
B
I do, actually. I've got a date with a lovely young statement from the Department for Culture, Media and Sport.
A
I thought you were married. Infidelity. Well, it happens. Perhaps more often than many of us would like to imagine. The adulterer's journey is a predictable one. Moving from defiance.
C
I did not have sexual relations with that woman.
A
To acceptance.
C
Indeed, I did have a relationship with Ms. Lewinsky that was not appropriate.
A
But is it also predictable that men are more inclined to commit adultery than women? It's a very commonly held belief that men are less faithful than women. But it takes two to tango. So can this be mathematically possible? Fiona woods has been. Well, actually. Fiona, what have you been doing?
F
I've been looking at the numbers, Tim.
A
Alright, so that's supposed to explain all those late nights in the office?
F
Yes, it does. And be careful, because there is evidence that men are less faithful than women. This isn't just folk wisdom.
A
Indeed. There is also data. And you've been looking at the American General Social survey?
G
Yes.
F
Its 2006 study of 20,000 people found that nearly twice as many married men as women admit to having had sexual relations with someone other than their spouse.
A
Men, eh?
F
Well, quite. And the story is similar in the uk.
A
Yes. The National Survey of Sexual Attitudes and Lifestyles has also pointed the finger at men. And I've been talking to the head of analysis for the survey, Dr. Catherine Mercer.
G
The bottom line is that women are less likely to report that they cheat on their partners than men. But as it's a socially sensitive behaviour, we can't directly observe unfaithfulness. So we have to rely on what people tell us. And we know that there are gender differences in how people report sexual behaviours.
A
That makes sense. So let's take a step back then. When you are grilling people about how many times they've cheated on their husband or wife, what's the context? Is this face to face? Is it over a computer terminal? And what sort of questions are you asking?
G
So the National Survey of Sexual Attitudes and Lifestyles is a large probability survey of people aged 16 to 74 living in Britain. And people are around randomly selected to be invited to participate. People are asked less sensitive questions initially, for example, about how they learned about sex, their general health, contraception use. And then for the more sensitive questions, for example, about the numbers of partners they've had, the type of sexual practices they engage in, we use self interview. So the respondent reads the question from a laptop screen and answers their response directly into the computer.
A
What are the actual questions that you're asking in order to deduce that people have been unfaithful? Well, maybe you just ask them straight out.
G
Well, there have been two previous NATSAU studies and we didn't actually ask people outright, have you been unfaithful? Instead, we've used the dates of first and most recent sex with their most recent partners. And we look then to see if the dates overlap to see if there's any evidence that perhaps those partnerships were overlapping.
A
Is this the language that you guys in Natzel use? I mean, do you talk about overlapping sexual relationships rather than talking about infidelity Anyway, we do.
G
Infidelity is quite a loaded word. Whereas thinking about overlapping partnerships is more appropriate when we're thinking about the epidemiological context of these data, we're thinking, for example, about the importance of overlapping partnerships or concurrency for. For the transmission of sexually transmitted infections and hiv.
A
Ok, so what are the headline rates of concurrent sexual relationships as measured by the survey?
G
Using that method, we found last time in 2000 that 15% of men had had overlapping or concurrent partnerships in the last year, in contrast to 9% of women. So there is a gender difference and that is actually statistically significant. There is evidence as well to suggest that there could well be a gender difference in unfaithfulness when we think about the age difference between sexual partners. So, for example, we know that on average, men tend to be slightly older than their female partners. And so if you imagine a scenario where a married man has an affair with a younger woman who's perhaps more likely to be single because she's younger, then he would have engaged in unfaithfulness, but she wouldn't have done.
A
Dr. Catherine Mercer of the Centre for Sexual Health and HIV Research at University College London. Now, a few weeks ago, we looked at a story about the number of children in London primary schools who speak foreign languages at home. Malcolm Gillies, the Vice Chancellor of London Metropolitan University, was quoted in the Daily Mail as saying, the majority of our students in London primary schools now have a home language other than English. In other words, they come from a very diverse ethnic base. When we looked into the figures, we found this was an exaggeration in fact, only 35% of state primary schools in London had a first language known or believed to be other than English, although they make up the majority in 42% of schools. Now, loyal listeners may be wondering why they can't remember hearing this item. It was interesting, but in the end we thought we could broadcast other things that were even more interesting. And that decision got us thinking about the stories we choose not to include. If we on more or less were only to report statistical errors and never report statistical triumphs, for example, you could be forgiven for concluding that the world is full of numerical lies. Now, as it happens, we try to present more of a rounded view than that, but we still do have a bias towards, well, more interesting results. And we're not the only ones. Here's Ben Goldacre, author of Bad Science, to explain why this is a big problem in science.
C
Well, publication bias is the phenomenon whereby positive, exciting, interesting findings are much more likely to get published in academic journals than boring, negative findings. And at face value, that sounds kind of trivially true. It's a bit like the judgments that a newspaper editor might make. You know, when a dog bites a man, that's not news, but when a man bites a dog, that is news. But the problem with it is that overall, it completely pollutes the academic literature because it means that what we're seeing is many, many, many more flukes and we're not seeing the full statistical context.
A
And why does this matter? What are the real world effects of not publishing these negative results?
C
In the world of medicine, for example, findings aren't just kind of abstract. Findings in medicine relate to the real human world of suffering and flesh and pain. And so we can get ourselves in a lot of trouble if the published research findings aren't as true as we think they are. But when it comes to treatments in the real world, we've got an even bigger problem. So the best way we have to find out if an intervention like a pill, for example, works or doesn't work is by doing a randomised controlled trial. And the problem is, some randomised controlled trials will get published and some won't. In fact, it turns out that only about half of all of the trials that get conducted get published. And trials which report positive results for the pill being tested are about twice as likely to get published as studies reporting negative results.
B
So.
C
So that means that we are dramatically over exaggerating the benefits of the treatments that we give in medicine. The effect in the final analysis in the clinic room where you're seeing A patient is, we don't know the true effects of the medicines that we're prescribing. So there is a solution to this problem and it's called a register. So this has been done mostly in the area of clinical trials. You write down the protocol of exactly what you're going to do in your experiment and you post it somewhere in public. So the idea of that is we can then see all of the trials that have been conducted and completed and then we can go back and we can say, well, are there any that have been completed but never reported? So we can try and identify which trials have gone missing in action. The idea of setting these trials registers up was that essentially public shame would prevent people from hiding data. Unfortunately, it hasn't quite worked out like that. And people have continued firstly to register trials, but then leave them unpublished, but they've also continued to publish trials that weren't registered.
A
Okay, so we know what to do about this, but we aren't really sticking to it. Is there an example of publication bias in scientific publications more generally?
C
Back in basic science, it's an even more serious problem. Last year a chap called BEM published a paper showing that undergraduate students have some kind of power to predict the future with their minds. Right? And this paper was published in a very well respected psychological journal. And everybody said, well, this is probably a fluke finding, let's see if we can replicate it. And so far, at least two different groups have gone out, tried to conduct exactly the same experiment, not found evidence of precognition, and then submitted it to the journal and been told, oh, sorry, we're not interested in that finding. So that's a very immediate human illustration of how this problem is still ongoing.
A
Ben Goldacre. It's only fair then that we list the items that we looked into this series and for one reason or another discarded. Could it really be true, we wondered, that the Olympic torch route comes within 10 miles of 95% of the population? Well, it's hard to be sure, but we think it probably is true. We researched whether city shipping insurers are making more from Somali piracy than the actual pirates. The answer, probably not. We examined the uselessness of pandas in some detail and then couldn't quite make it all stand up, much like the pandas themselves. What else? Ah, yes. How do we know how many fish are left in the sea? Who knows? We got distracted. Sorry. And of course, there were many, many other items that we contemplated briefly, perhaps very briefly, before deciding to focus our statistical spotlights elsewhere and that's all we have time for this week and indeed this series. But please keep your comments and suggestions coming in to more or lessbc.co.uk. we will read them before the next series begins. And as you may know, your suggestions and questions are the basis for much of what we do on the programme. You might also want to surf on over to BBC.co.ukmoreorless to subscribe to our podcast, in part because it means you won't miss the start of the next Series series, and in part because a shorter version of the programme now runs all year round on the BBC World Service. But farewell to Radio 4 for now. We'll be back in July, and until then, goodbye.
C
More or Less he's presented by Tim Harford for Financial Times Undercover Economist producer is Richard Knight. The editor is Richard Boehner.
D
More or Less is made in association
A
with the Open University.
BBC Radio 4 | Aired: May 28, 2012
Host: Tim Harford
This edition of More or Less tackles the statistical and evidentiary foundations— or lack thereof— behind major claims in public debate. The focal point is the controversial proposal from venture capitalist Adrian Beecroft to allow "at-will" firing of employees in the UK, positioned as a measure to promote economic growth. Along the way, the episode examines work ethic myths, marriage infidelity statistics, the perils of publication bias in science and medicine, and reflects on the show’s own editorial choices in reporting numbers.
[00:45–02:34]
[02:34–03:47]
OECD Viewpoint (Christophe Andre):
Pro-Market Think Tanks:
Interview: Prof. John Van Reenen, LSE ([03:47–06:38])
No Significant Overall Impact:
Little Effect on Growth:
Firm-Level Nuance:
Why Politicians Ignore Evidence:
[07:00–10:16]
Key Insight:
Memorable Exchange:
[10:16–12:43]
Quote:
[13:14–17:35]
Societal belief: Men are more likely to cheat than women.
US Survey:
UK Survey (Dr. Catherine Mercer, NATSAL):
Methodological Notes:
[17:35–22:19]
Ben Goldacre (Bad Science):
Example:
[22:19–23:47]
This episode expertly weaves together critiques of data-deficient policy proposals, the misleading nature of widely-circulated numbers, and statistical pitfalls across science and society. Each segment emphasizes the need for robust, context-aware evidence— and a healthy skepticism about both what is reported and what’s left out.
The tone is characteristically witty, skeptical, and analytical, making serious points with a light touch. If you want to understand how numbers can be twisted— or ignored— in debates that affect us all, this episode is a masterclass.