Loading summary

Transcript79 lines

[00:00]
A
Thank you for downloading this week's More or Less podcast. Here's Tim Harford.
[00:05]
B
Hello and welcome to More or Less, in which we take a look at the numbers behind the week's news and in the world around us. The government launched its social mobility strategy this week. But are our views of social mobility being distorted by an old statistical problem? And as Portugal asks for a bailout, we'll try to fix the economy closer to home in Trumpton. Oh, dear. Whatever next? But first, we're frequently told that young people have never had it so bad they can't get on the housing ladder and it's increasingly difficult to get a job. The last official set of figures revealed that nearly a million 16 to 24 year olds were unemployed. The latest unemployment stats are out next week. But do they actually tell the real story? Well, Hannah Barnes has been looking into this and Hannah, there have been some pretty high level complaints about how the unemployment numbers are presented yet, notably from
[00:55]
A
Ian Duncan Smith, the Work and Pension Secretary. He's written to the Office for National Statistics who published the figures, saying he's concerned that they can be used to present a misleading picture. Chris Grayling's the Employment Minister and I asked him why he and his boss are worried.
[01:12]
C
Well, the issue we have is that the headline figure for youth unemployment in the country today is 974,000, but nearly 300,000 of that total is actually made up of people who are full time students and those people are telling the surveyors that they are interested in getting a job to top up their income, but the system is showing them up as being full time unemployed. Now that's clearly nonsense. Somebody who is full time at college I do not think should be counted and I think the system should automatically discount them from these numbers. To me, it's kind of pretty simple really. If you're a full time student at university who would like a Saturday job, you are not in the same position as an 18 year old who's left college, who is out of work, has little prospect of work and needs intervention to help them get into a proper sustained future. I don't think they're the same thing at all. The statisticians may disagree with me, but I think most public would think I was right.
[02:07]
B
So the problem stems from the fact that just because someone's a full time student doesn't actually exclude them from being counted as unemployed.
[02:14]
A
Exactly. 16 to 24 year olds are treated like everyone else in the Labour Force Survey, which collects this information on employment and they're put into One of three employed, unemployed or economically inactive. And those definitions are suggested by the International Labour Organisation, which is a UN agency. But the problem is when they're applied to full time students, it can lead to some quite strange results. I'm at Westminster University in central London. I'm in the canteen dean of the Marylebone campus and I've managed to find three young undergraduates.
[02:51]
B
Simon, Robin, Roberto.
[02:53]
A
How would you sort of categorise your status at the moment? What are you doing?
[02:58]
B
Well, I'm studying Westminster University and I'm hoping to go to my master's after that.
[03:03]
A
What about you, Robin?
[03:04]
D
Yeah, I'm an undergraduate as well, so
[03:05]
B
they're basically the same. I'm doing a business management degree and I'm planning to do my master's next year.
[03:11]
A
Roberto, do you do any paid work as well as your full time studies?
[03:17]
B
I do some part time working, sometimes maybe about 10 hours a week.
[03:22]
A
What about you, Robyn, do you do any work?
[03:24]
B
No, not at the moment. Basically I get support from my parents at the moment.
[03:29]
A
Have you looked for any work at all?
[03:30]
B
Yeah, I mean I've applied for graduate schemes but at the same time I'm also applying for masters.
[03:35]
A
Ok. And what about you, Simon, do you work?
[03:37]
B
No, I don't work. I used to work but because it's my last year, so I decided not to work at the moment.
[03:43]
A
Now, despite Simon, Robin and Roberto all being in full time education, they're all classed differently for the purposes of the official employment stats. Roberto's employed, Robin's unemployed because he's looked for paid work but he didn't get it. And Simon would be counted as economically inactive. I asked them what they thought about that.
[04:05]
B
Strange, because we're all similar situations, so how can there be so much change of so much different categories? I don't understand. The thing is that I don't really need to work at the moment. I think it doesn't make sense.
[04:15]
D
Let's say if you get the support
[04:16]
B
from your parents, you're a full time student, then you shouldn't be really considered as unemployed. So hasn't the government got a point here? It does seem a little odd to be counting people who are in full time education as unemployed.
[04:28]
A
Well, yes it does. And it's true that if you discount unemployed students, the headline figure falls quite dramatically. But if you agree with the government's line that students shouldn't be counted as unemployed, then by the same logic they shouldn't be counted as employed either. And there are an awful lot of employed students, 870,000 at the last count.
[04:50]
B
And I guess this matters because of the way that unemployment rates are calculated.
[04:54]
A
Yeah, exactly. All employment rates are calculated as percentages of the economically active population, not the total population. So in the case of students, if you say they shouldn't be counted as employed or unemployed, then the denominator that you're using to calculate the results is shrinking. And just a little basic number crunching tells you that if you class all young people in full time education as economically inactive, then the headline rate for youth unemployment stays the same. I put that point to Chris Grayling.
[05:26]
C
The issue is about unemployment and how we respond to that. I mean, I have to say I don't think that students doing a couple of hours a week of employment should show up in the overall employment figures either. And it does have, I think, a demoralising effect on young people if the headlines are all about record youth unemployment.
[05:44]
A
But as I said, if you count students all in the same way, the headline rate that one in five, while the overall number is fewer, you still end up with 20%.
[05:53]
C
Well, my concern is not about the overall numbers, it's not about the statistical debate, it's actually about dealing with a very human problem of people who do not have jobs to whom we need to be focusing real attention to try and help them get into the workplace.
[06:07]
A
Well, putting the statistics debate aside, even so, the figures are still pretty bad. Hundreds of thousands of young people are unemployed, even discounting students. But is it fair to call today's young people a lost generation? I've been speaking to Simon Briscoe from economic data company Timetric.
[06:25]
E
If we look at the unemployed youths as a proportion of all of the youths, then it's going to come in at about 12% rather than, let's say, about near 20%. You know, I'm not saying that one in eight unemployed youths is not a bad thing and is not really grim for those individuals, but I don't think that one in eight makes youths stand out from the broader population in the way that saying 1 in 5 it is very difficult because the labor force survey didn't exist 30 years ago. So we can't really compare like with like. But certainly when you do look at the number of young people on benefits of one sort or another related to unemployment, then the numbers were very, very much 30 years ago.
[07:08]
B
That's Simon Riskoe's for you. Have you managed to get anything out of the ons?
[07:11]
A
Yeah, we've asked the Office for National Statistics to give us a bit more data so we can Get a better picture of the work that's being carried out by 18 to 24 year olds in full time education. And Simon's helped us out with the analysis.
[07:24]
E
I think these figures are really quite revealing because it does show that there are over half a million people aged between 18 and 24 who are full time students who are working more than six hours a week. And a quarter of a million of these students are working more than 16 hours a week. And that's a doubling in the last decade.
[07:44]
B
Well, that's Simon Briscoe of Tim Metric and our own Hannah Barnes. Thanks, Hannah. Now, you may have heard Evan Davies clashing with Ed Balls, the Shadow Chancellor, on the Today programme on Wednesday. What is your line? If we increase the deficit by a pound, how much extra growth do you think we get?
[08:02]
F
Do we get a pound 50 of
[08:04]
D
a bigger economy or do we get 20p bigger economy?
[08:07]
F
What is the relationship between letting the deficit rise and the growth?
[08:14]
B
And if you were thoroughly confused by that exchange, you weren't alone. One of the key political debates of our age is whether the government must immediately and substantially cut spending and raise taxes in order to cut the deficit, which is what simple arithmetic would demand, or whether doing that will be counterproductive, undermining the economy just as it's slowly struggling out of recession. This is a debate about what economists call the fiscal multiplier, which is what Evan Davis and Ed Balls were talking about. If the multiplier is close to zero, there's no need to worry that cuts will undermine the economy. The private sector will fully step in to fill gaps that the public sector leaves. If the multiplier is more than 1, then government spending cuts will actually make the private sector smaller. If the multiplier is more than zero, but less than 1, the private sector will partly, but not completely expand to fill the space as the public sector shrinks. But how does this mysterious multiplier operate? How big is it and how do we know? It's something I've been agonizing about recently. You see, while more or Less was off air, I took on a very important responsibility. I'm proud to say that I've been elected Treasurer of Trumpetonshire County Council. Not everyone's heard of our little county, although it was famous in the 1970s when BBC Children's television broadcast a series of hard hitting documentaries about life in three local communities here, the genteel county town Trumpton, the village of Camberwick Green and the bustling industrial hub of Chigley. Life wasn't always straightforward in 1970s Trumpetonshire. But they seem like halcyon days compared with the fiscal crisis now facing our county council. We have an unprecedented budget deficit, meaning that each year we're spending substantially more than we take in in tax. Some blame overspending by the previous mayor. Others claim that the collapse of the Trumpton Building Society devastated the local economy. Whatever the reason, as Trumpetonshire treasurer, it's my job to fix the problem. I took firm action only this week. Trumpeton Fire Station. Hello, this is the treasurer of Trumpetonshire County Council. Is that Captain Flack? Yes, it is. I'm afraid I have some bad news. Captain Flack, can you please find Dibble? Find the what? Find Dibble, Please find him and tell him he's been made redundant. Oh, dear. Whatever next? I know it's hard to believe, Captain Flack, but this is a fiscal emergency. Cutbacks must be made. We're all in this together, you know. I'm sorry, but please will you or one of your team find Dibble and sack him? Yes, of course we will.
[10:58]
D
Right away.
[10:59]
B
Right away. It's one of the hardest phone calls I've ever had to make. It's not just the human cost, it's that I'm not even sure these cuts will help restore prosperity to Trumpetonshire. There are plenty of people telling me that making cuts is self defeating. So I've called in Chris Giles, economics editor of the Financial Times and the noted expert on the Trumptonshire economy, for advice. Chris, I've been making some tough decisions here, although I do feel obliged to point out that according to Wikipedia, the Trumpton Fire Brigade has never actually put out a real fire. Nevertheless, what effect might sacking Dibble have on the Trumptonshire economy?
[11:36]
F
He would have less money to spend now he's unemployed. And that money, he probably wouldn't be going to buy his biscuits from the factory in Chigley. And so the factory in Chigley would have less revenue, and then because they've got less revenue, they probably want to make fewer biscuits. And that would mean that maybe the wheat farmers in Trumpetonshire might have less demand and so on.
[11:58]
B
Windy Miller over in Camberwick Green.
[12:00]
F
Yeah, exactly. And so. And this process can continue and so it can go round and round in circles. So your original decision to fire the fireman damages the biscuit factory, damages the farmers, which then perhaps even means you have even less tax revenue, which means you have to then fire someone else and the process continues.
[12:18]
B
Well, that's all very plausible, but I guess there's another Way to tell the story. Imagine Jiggly Biscuit Factory is doing well. It's trying to hire good, reliable workers and it can't find anybody. And then when Dibble is fired from the fire service, suddenly he's on the labor market. They can hire him, they can expand their operations and that's good for everybody.
[12:41]
F
There's lots of ways you can tell the story. You can say that by firing Dibble, everyone else in Trumpton is actually much more reassured that you've got the finances of Trumpetonshire under control and then are more willing to spend themselves because they're no longer worried that the economy is going to implode because the public finances of Trumpetonshire are under severe threat.
[13:04]
B
Ok, so that's the abstract argument. I guess when we get to the specifics of Trumpetonshire, we have to actually understand some of the variables involved. So we have to know, I guess we have to know how deep the recession is in Trumptonshire. We have to know what sort of trading arrangements there are. Does it have a floating exchange rate, for example, or is it an open economy? Why might that make a difference?
[13:24]
F
If the fact is that your firing of Dibble means that he doesn't spend less in the Chiggly Biscuit factory, but he spends less in other parts of the uk, then actually that's not a problem for Trumpet and Shire. So you've actually exported the spending cut to someone else. It's a problem somewhere else, but not domestically. Equally, if unemployment is very low and demand elsewhere is high. So there's a lot of other things going on in Trumpetonshire. Then again, firing Poorwald Dibble means that he's quite likely to find a job elsewhere because there's enough momentum in the Trumptonshire economy.
[14:02]
B
Well, thanks, Chris. That's Chris Giles of the Financial Times. I also asked Ray Barrell for advice. Ray is the head of macroeconomics at the National Institute for Economic and Social Research. Ray's not an expert on the fiscal multipliers for Trumpetonshire, but he does know a lot about the evidence from around the world.
[14:19]
G
Well, in a country like the US, you might expect that if spending is cut by $100, then output might fall by about a hundred dollars. In the UK, you might expect if spending is cut by about £100, output will fall by about £70 because some of the spillovers go into imports. Country like Ireland, though, if you. Which Trumpeton is probably like. If you cut.
[14:41]
B
I hope not. Their banks are in a shocking state.
[14:44]
G
Well, we haven't inspected Trumpton's banks yet. But if you cut spending in Ireland by €100, then probably most of that will go into imports and output will only fall by about 10 or €15.
[14:55]
B
Well, thanks, Ray. If Ray Barrel is right, then I needn't worry too much about firing Dibble. I will have a smaller fire brigade, but I won't have a smaller economy. But for the UK as a whole, it's not so straightforward. Ray Barrel's estimate is that the spending multiplier in the UK is 0.7. That means that for every million pounds by which George Osborne cuts public spending, the private sector grows by just 300,000 pounds and the economy as a whole therefore shrinks by 700,000 pounds. This is over the short to medium term, over four to five years. Most economists would assume the multiplier is zero and the private sector will eventually fill any gap. But Chris Giles told me that the range of uncertainty is pretty wide. The fact that the UK is quite an open economy with a flexible exchange rate points towards a low multiplier and support for George Osborne's austerity position. But the fact the recession has been deep and unemployment's high suggests a high multiplier and support for Ed Balls position. Chris thinks that the consensus estimates of the fiscal multiplier are roughly halfway between what George Osborne would like us to believe and what Ed Balls would like us to believe. He also told me nobody can possibly know for sure. Whatever the fiscal multiplier really is in Trumpetonshire. I'm afraid it's too late for Dibble.
[16:12]
D
Pew.Pew.
[16:13]
B
barney McGrew, toughbut grub. You're listening to More or Less in association with the Open University, and I'm Tim Harford. You can email us on more or lessbc.co.uk and you can find me on Twitter simply as Tim Harford. If you have 20 minutes and you're unsure what to do with them, you could download an old edition of More or Less and listen to 72% of it. Or you could complete the Big Risk Test questionnaire on the PBC website. There's a link to it on our site, BBC.co.uk more or less the Big Risk Test hopes to be the biggest ever study of our perception and understanding of risk. My producer took it and he claims it's actually rather fun. I can't vouch for it myself, but then I never have 20 minutes to spare and he evidently does. Which figures? And our alternative to the census is still up on the Today programme website. And there's A link to that on ours. Hello. I have a problem and I would be most grateful if you could help
[17:11]
D
me to solve it.
[17:12]
B
Why, it's Lord Bellborough of Winkstead hall near Chigley. What can I do for you? It's the apple crop this year. It's too big. I'll see what I can do at once. Lord Bellborough. That man's so out of touch. We're not really all in this together, are we? The trouble with this country is there's not enough social mobility. And don't take that from me. That was the message this week from the Deputy Prime Minister, Nick Clegg, as he launched the government strategy on social mobility.
[17:38]
G
It is simply unacceptable, Mr. Speaker, that so many of our children have their life chances shaped by the circumstances of their birth. Gaps in development between children from different backgrounds can be detected even at birth. By the age of five, bright children from poorer backgrounds have been overtaken by less bright children from richer ones. And from this point on, the gaps tend to to widen still further.
[18:03]
B
In the strategy document, opening doors, breaking Barriers, a striking graph is prominently displayed. It caught the eyes of the BBC news website editors, who used it to illustrate their story about the strategy launch. The graph isn't that new. It's based on work by Leon Feinstein, now a professor at the Institute for Education, published as a discussion paper way back in 1998, studying the educational outcomes of children born in 1970. It's really about the performance of the British education system under the likes of Edward Heath and Jim Callaghan. It's an incredibly powerful graph and I've written about it admiringly in my own work for the Financial Times. But perhaps it doesn't show what I and Nick Clegg's team think it shows. We'll put the graph up on our website, but let me try to paint a picture of it in your mind. The graph tracks the performance over time of children who are set some tests of cognitive ability. The tests give a snapshot of how the children are doing at the ages of just under two years old, nearly three and a half, five years old and ten years old. Now, the graph splits these children into four groups very loosely. We could call them bright rich kids, bright poor kids, dim rich kids and dim poor kids. Although the real definitions are both more precise and more polite. You count as bright if you're one of the top performing toddlers and dim if you're one of the worst performers at the age of 22 months. What's appalling about the message of this graph? Is the way that by the age of 10, the talentless toddlers with rich parents have overtaken the bright poor kids. It's a profoundly upsetting illustration of the way natural talent is trumped by social privilege. Or is it? Daniel Reid, professor of Behavioral science at Warwick Business School, emailed us because he thinks the way the richer dim kids seem to catch up with the poorer bright kids might be at least in part due to a very old statistical problem.
[19:57]
D
There may indeed be some effect, but I think the majority of the effect that we see is due to a statistical artifact that's called regression to the mean.
[20:05]
B
Regression to the mean is simply this. Imagine you've got a sample of data. Maybe it's the quality of meals at restaurants. Maybe it's the number of traffic accidents at traffic junctions. Maybe it's the test scores of toddlers. Now pick out the very best restaurant meals, the very safest junctions, the highest test scores, and think about the effect of sheer good luck. Sometimes the chef at a restaurant happens to find some really exceptional ingredients. Sometimes the users of a traffic intersection are lucky and avoid accidents when someone could easily have had an eye out. And sometimes toddlers have a particularly good day. In our sample of the very best performances, then some of them really are top performers and some have enjoyed good luck. If we come back another day, we might find that the restaurant has gone off the boil, the drivers at that safe junction have become careless, the bright kid has somehow got stupider, and all that might be happening is that our original measure reflected good luck. What relevance does this have for the social mobility graph? Well, think back to what Nick Clegg said to Parliament. He said that there was already an ability gap detectable at par birth between families of richer and poorer children. Certainly there seems to be a gap at the age of 22 months. According to the British cohort survey on which Feinstein's analysis is based, the richer toddlers are more likely to be in the high ability group at the age of 22 months. But if the typical poor kid is born at a disadvantage to his richer peers, then if you take a snapshot of toddler ability at 22 months, the poorer kids in the high ability group are disproportionately likely to be there because they happen to have a good day. And the richer kids in the low ability group similarly are more likely to be there because of bad luck than because their cognitive ability really is lower. It's not that it's all fluke, but if there's any element of luck at all which There surely is, because we're talking about ability tests for toddlers, then we have to allow for what we'd expect to happen when that luck fails to last. And what we'd expect to happen is pretty much what the graph in the government social mobility strategy shows, which is that the next time you test the children, all the high performers have dropped off, but especially the poorer kids, who, remember Nick Clegg says, were disadvantaged from birth. And all the lower performers have caught up, but especially the richer kids. And then as you continue to test the richer kids, gain on the poorer kids at a very much less dramatic pace. Daniel Reid thinks it's an open and shut case. A lot of what looks like social unfairness is actually just this statistical effect of regression to the mean.
[22:47]
D
There's one feature of the graph which is very diagnostic, which is that if you look at it, you see that most of the shift that we Observe occurs from 22 months to 40 months. So from the first observation of the second, and what's happened is that in the first observation, you've selected the kids. So you said, well, these are the guys who did really well in 22 months. Let's look at them. And here are the people who've done very poorly at 22 months. Well, let's look at them. And now the next observation we make, what happens is we don't expect the error which pushed the measures up or down to reoccur, and so the measurements tend to get closer to the mean. And there's this very big effect from 22 months to 40 months. And if you look at the graph, you see that after that there's not really much going on at all. There's a little bit still, but there's not very much. So almost all of it is due to that shift. And that shift, I think, certainly due to regression to the mean.
[23:35]
B
And the implication of that, presumably is we're horrified at the idea that these very talented poor kids are overtaken by these extremely untalented rich kids.
[23:45]
D
Right.
[23:45]
B
But actually the initial measure of talent was pretty noisy.
[23:49]
D
That's right, the initial measure was noisy. And by choosing the top performers on that noisy measure and the bottom performers on the noisy measure, you actually augmented the degree of error. And you also put a bias in the error, making the error positive for the high performing kids and negative for the low performing kids.
[24:06]
B
I realise this is not something you've looked in tremendous detail at the data, but is there any way that this can be cleaned up and we can allow for regression to the mean and we can actually figure out whether there's still a real effect. And you do think there is still a real effect?
[24:19]
D
Well, I think there is some effect. I mean, the simple way to do it is to say, well, let's look at the graph and let's simply block out the first observations and now look at what happens for the remaining observations. And what you see then is that from the age of 4 to the age of 10, you find that the poor kids who are performing the best, their performance goes down a little bit, and for the rich kids who are performing the worst, their performance goes up a little bit. There seems to be an effect there where the rich kids are benefiting over time and the poor kids, perhaps slightly less, are getting worse over time. That's only true for the high performing poor kids and the lower performing rich kids. Now for the poor kids who are low performers, their performance doesn't change over time. And for the rich kids who are high performing, their performance doesn't change over time. If I can say as well, there's further evidence that it's regression to the mean, because the thing with regression to the mean is it doesn't just work forward. If you have an exceptional experience, the previous experience was probably more mediocre than the one you're currently having. And actually in the original paper describing these results, there is a figure showing children sorted at 40 months and looking back at their performance at 22 months. And there you get the same pattern as when you go forward showing that that's a regression to the mean effect and doesn't have anything to do with forward looking causality. And again, if you block out that particular observation, you find that there's a small effect left over, but it's quite
[25:42]
B
small because the naive interpretation of that graph would be that the talented poorer kids suddenly, dramatically catch up with the talented richer kids by the age of 14 months and then they fall away again.
[25:54]
D
That's right.
[25:54]
B
And that seems a very weird thing to happen. And so that's sort of evidence that this is partly a statistical artifact.
[26:01]
D
I think I would call it proof.
[26:02]
B
Daniel Reed of Warwick Business School. Now, Wesley Stevenson's here. Wes, you've been trying to track down Leon Feinstein, the man behind the original graph.
[26:09]
H
Yes, he's in a slightly tricky position at the moment because he's currently doing some work at the treasury and he couldn't just drop everything to come on the programme. But he did tell us that the measurement error could explain a small part of the pattern. But he's very skeptical that Regression to the mean is the main influence on the results. He thinks it doesn't explain why poor kids with high test scores are more likely to be having a good day than their rich counterparts.
[26:34]
B
So presumably you put that to Daniel Reed. What was his response?
[26:37]
H
Well, yes, he stands by his critique. He thinks Feinstein's research shows clearly that rich kids do better on average than poor kids in these tests. There are three times as many rich kids in the study getting top scores than there are poor kids. He thinks those few poor kids are outperforming the average for their group much more than the rich kids are. And so you'd expect them to fall back more than the top performing rich kids do, which is exactly what the graph shows.
[27:03]
B
Well, thank you, Wes. Incidentally, we're not claiming there's no problem with social mobility. Lots of different data sources here, lots of reasons to think that social mobility may be low in the uk. But this one rather famous graph may be, at least in part, a statistical anomaly. And that's all we have time for this week. Don't forget, you have one week left to fill in our alternative census. There's a link on our website, BBC.co.uk more or less. We've had over 10,000 responses so far and we'll be broadcasting the results on the 22nd of April. We'll be back next week. Until then, goodbye.
[27:40]
A
More Or Less was presented by Tim Harford, the FT's undercover economist. The producer was Richard Knight and the editor was Richard Varden. It was made in association with the Open University.

Summary8 min read

More or Less – "Youth Unemployment" (BBC Radio 4, 8 April 2011)

Host: Tim Harford
Theme: Examining and challenging the statistics behind youth unemployment and social mobility in the UK.

Episode Overview

This episode of More or Less investigates the numbers shaping political debates on youth unemployment and social mobility in the UK. The show scrutinizes how youth unemployment statistics are constructed, whether they misrepresent the reality facing young people, and draws on expert voices to dissect the impact of student status on employment stats. The programme also explores a graph central to the government’s social mobility strategy, raising questions about statistical artifacts such as regression to the mean.

Key Discussion Points & Insights

1. Youth Unemployment: Are the Headline Numbers Misleading?

[00:05–06:25]

Government Concern:
The episode opens with government ministers (notably Iain Duncan Smith and Chris Grayling) expressing concern that youth unemployment figures are misleading due to the inclusion of full-time students.

"The headline figure for youth unemployment ... is 974,000, but nearly 300,000 of that total is actually made up of people who are full time students..."
— Chris Grayling, Employment Minister [01:13]
Labour Force Survey Explained:
The Labour Force Survey, following International Labour Organisation definitions, classifies 16–24-year-olds as employed, unemployed, or economically inactive. The system treats full-time students the same as others in this age group, regardless of actual job-seeking status.
Student Perspectives:
Three Westminster University students (Simon, Robin, Roberto) are interviewed to demonstrate the confusing impact of survey classifications:
- Roberto: Working part-time = counted as employed.
- Robin: Looking for work but none found = counted as unemployed.
- Simon: Not job-seeking = counted as economically inactive.
"Strange, because we're all in similar situations, so how can there be so much change of...so much different categories? I don't understand."
— Student [04:05]

"If you're a full time student ... you shouldn't really be considered as unemployed."
— Student [04:15]
Statistical Implication:
Removing all students from both the unemployment and employment categories shrinks the denominator used to compute unemployment rates, resulting in little change to the overall unemployment rate (~20%).

"Just a little basic number crunching tells you that if you class all young people in full time education as economically inactive, then the headline rate for youth unemployment stays the same."
— Tim Harford [04:53]
Ministerial Focus:
Chris Grayling emphasizes the importance of focusing support on those truly out of work and needing intervention, rather than fixating on statistics.

2. Is There a “Lost Generation”? Historical Context and Data

[06:25–07:44]

Expert Perspective:
Simon Briscoe (Timetric) suggests that discounting students, the true unemployment figure for 16–24s is closer to 12% (around 1 in 8), which mitigates claims of a uniquely ‘lost’ generation.

"If we look at the unemployed youths as a proportion of all the youths, then it's going to come in at about 12% rather than...near 20%."
— Simon Briscoe [06:25]
Working Students Trend:
ONS data reveal over half a million full-time students aged 18–24 now work more than 6 hours per week; a quarter-million work over 16 hours per week—a doubling over the past decade.

"...over half a million people aged between 18 and 24 who are full time students who are working more than six hours a week. And a quarter of a million...are working more than 16 hours a week. And that's a doubling in the last decade."
— Simon Briscoe [07:23]

3. The Fiscal Multiplier: Spending Cuts and Economic Fallout (Trumptonshire Analogy)

[08:14–14:55]

Political Context:
The segment pivots to a debate on government spending cuts vs. stimulus (the fiscal multiplier). The radio drama of “Trumptonshire” is used to illustrate the issue.
Mechanics of the Fiscal Multiplier:
Chris Giles (Financial Times) explains that spending cuts reduce incomes and demand, causing ripple effects throughout the local economy. However, alternative scenarios (such as fired workers finding new jobs) introduce countervailing effects.

"You can say that by firing Dibble, everyone else...is actually much more reassured that you've got the finances...under control and then are more willing to spend themselves..."
— Chris Giles [12:40]
How Big Is the Multiplier?
Ray Barrell (NIESR) gives rough estimates:
- US: A $100 cut leads to $100 less output.
- UK: A £100 cut leads to £70 less output.
- Ireland: Only €10–€15 less output for a €100 cut (because of import leakages).
"In the UK, you might expect if spending is cut by about £100, output will fall by about £70..."
— Ray Barrell [14:18]

For the UK, the consensus multiplier is estimated as 0.7.

"...for every million pounds by which George Osborne cuts public spending, the private sector grows by just 300,000 pounds and the economy...shrinks by 700,000 pounds."
— Tim Harford [14:55]

4. Social Mobility: Does a Famous Government Graph Mislead?

[17:11–27:02]

The Social Mobility Graph:
Analysis shifts to a striking graph (from research by Leon Feinstein) showing that, by age 10, low-ability rich kids surpass high-ability poor kids in educational achievement—a figure used in the government’s social mobility strategy.

"...by the age of 10, the talentless toddlers with rich parents have overtaken the bright poor kids. It’s a profoundly upsetting illustration..."
— Tim Harford [18:02]
Regression to the Mean – A Statistical Warning:
Daniel Reed (Warwick Business School) argues that much of the dramatic shift shown in the graph is explained by regression to the mean: the tendency for extreme observations to 'move' closer to average on repeated measurements, especially when initial measures are noisy (as is likely with toddler cognitive scores).

"The majority of the effect that we see is due to a statistical artifact that's called regression to the mean."
— Daniel Reed [19:57]

"...from the first observation to the second...the measurements tend to get closer to the mean. And there's this very big effect from 22 months to 40 months. ...that shift, I think, certainly [is] due to regression to the mean."
— Daniel Reed [22:46]
Can We Separate the Real Effect?
By ignoring the initial ‘noisy’ measure and observing performance from age 4 onward, a smaller gap remains, suggesting some true effect alongside the statistical artifact.

"...block out the first observations and now look at what happens for the remaining observations...you find...there seems to be an effect there where the rich kids are benefiting over time..."
— Daniel Reed [24:18]

"...the naive interpretation...would be that the talented poorer kids suddenly, dramatically catch up...by the age of 14 months and then they fall away again."
— Tim Harford [25:42]

"That seems a very weird thing...that's sort of evidence that this is partly a statistical artifact."
— Tim Harford [25:49]

"I think I would call it proof."
— Daniel Reed [26:00]
Original Researcher Responds:
Leon Feinstein, the graph’s author, is skeptical that regression to the mean fully explains the results, noting differences in the likelihood of children being outliers across income groups.

"He did tell us that the measurement error could explain a small part of the pattern. But he’s very skeptical that regression to the mean is the main influence..."
— Wesley Stevenson [26:09]

5. Conclusions and Nuanced Takeaways

Youth unemployment headline statistics can be misleading—both overstating and, when adjusted naively, understating the true scale of the problem among non-students.
Statistical categories for young people can produce “strange” and counter-intuitive results, complicating straightforward narratives.
Social mobility data, especially when visualized in powerful graphs, must be read with caution—statistical artifacts like regression to the mean can produce misleading stories about the triumph of wealth over ability.
There remains a real, though possibly smaller, problem with social mobility in Britain, even after correcting for such artifacts.

Notable Quotes & Memorable Moments

Chris Grayling on headline youth unemployment:
"Somebody who is full time at college I do not think should be counted..." [01:20]
Student response to employment status classification:
"Strange, because we're all similar situations, so how can there be so much change of...so much different categories? I don't understand." [04:05]
Tim Harford on denominator-changing trickery:
"Just a little basic number crunching tells you that if you class all young people in full time education as economically inactive, then the headline rate for youth unemployment stays the same." [04:53]
Simon Briscoe's cautious optimism:
"I don't think that one in eight makes youths stand out from the broader population in the way that saying 1 in 5 does..." [06:25]
Daniel Reed's clear diagnosis:
"The majority of the effect we see is due to a statistical artifact that's called regression to the mean." [19:57]
Insight on graph myths:
"That seems a very weird thing to happen. And so that's sort of evidence that this is partly a statistical artifact."
— Tim Harford [25:49]
Feinstein's skepticism:
"[He] thinks it doesn't explain why poor kids with high test scores are more likely to be having a good day than their rich counterparts." [26:33]

Timestamps for Key Segments

Youth Unemployment: Statistical Issues, Student Experiences: [00:05–06:25]
Historical Comparison & Working Students Trend: [06:25–07:44]
Fiscal Multiplier Debate (Trumptonshire Story): [08:14–14:55]
Social Mobility Graph and Regression to the Mean Discussion: [17:11–27:02]

Tone & Language

The episode is marked by Tim Harford’s trademark clarity, skepticism, and humor. Explanations are lively and accessible, with the use of practical illustrations (interviews, analogies, and radio dramatizations) blending statistical precision with human narrative. There is an insistence throughout on questioning surface-level statistics and interrogating the truth behind the numbers.

For Listeners: The Bottom Line

This episode is a call for statistical literacy. Youth unemployment and social mobility remain crucial challenges, but popular figures and government graphics can be misleading if interpreted at face value. Definitions, denominators, and statistical phenomena like regression to the mean must always be considered when interpreting 'headline' numbers or compelling visualizations—lest policy, media, or public debate get "more" wrong than "less" right.

Loading summary

Transcript79 lines

[00:00]
A
Thank you for downloading this week's More or Less podcast. Here's Tim Harford.
[00:05]
B
Hello and welcome to More or Less, in which we take a look at the numbers behind the week's news and in the world around us. The government launched its social mobility strategy this week. But are our views of social mobility being distorted by an old statistical problem? And as Portugal asks for a bailout, we'll try to fix the economy closer to home in Trumpton. Oh, dear. Whatever next? But first, we're frequently told that young people have never had it so bad they can't get on the housing ladder and it's increasingly difficult to get a job. The last official set of figures revealed that nearly a million 16 to 24 year olds were unemployed. The latest unemployment stats are out next week. But do they actually tell the real story? Well, Hannah Barnes has been looking into this and Hannah, there have been some pretty high level complaints about how the unemployment numbers are presented yet, notably from
[00:55]
A
Ian Duncan Smith, the Work and Pension Secretary. He's written to the Office for National Statistics who published the figures, saying he's concerned that they can be used to present a misleading picture. Chris Grayling's the Employment Minister and I asked him why he and his boss are worried.
[01:12]
C
Well, the issue we have is that the headline figure for youth unemployment in the country today is 974,000, but nearly 300,000 of that total is actually made up of people who are full time students and those people are telling the surveyors that they are interested in getting a job to top up their income, but the system is showing them up as being full time unemployed. Now that's clearly nonsense. Somebody who is full time at college I do not think should be counted and I think the system should automatically discount them from these numbers. To me, it's kind of pretty simple really. If you're a full time student at university who would like a Saturday job, you are not in the same position as an 18 year old who's left college, who is out of work, has little prospect of work and needs intervention to help them get into a proper sustained future. I don't think they're the same thing at all. The statisticians may disagree with me, but I think most public would think I was right.
[02:07]
B
So the problem stems from the fact that just because someone's a full time student doesn't actually exclude them from being counted as unemployed.
[02:14]
A
Exactly. 16 to 24 year olds are treated like everyone else in the Labour Force Survey, which collects this information on employment and they're put into One of three employed, unemployed or economically inactive. And those definitions are suggested by the International Labour Organisation, which is a UN agency. But the problem is when they're applied to full time students, it can lead to some quite strange results. I'm at Westminster University in central London. I'm in the canteen dean of the Marylebone campus and I've managed to find three young undergraduates.
[02:51]
B
Simon, Robin, Roberto.
[02:53]
A
How would you sort of categorise your status at the moment? What are you doing?
[02:58]
B
Well, I'm studying Westminster University and I'm hoping to go to my master's after that.
[03:03]
A
What about you, Robin?
[03:04]
D
Yeah, I'm an undergraduate as well, so
[03:05]
B
they're basically the same. I'm doing a business management degree and I'm planning to do my master's next year.
[03:11]
A
Roberto, do you do any paid work as well as your full time studies?
[03:17]
B
I do some part time working, sometimes maybe about 10 hours a week.
[03:22]
A
What about you, Robyn, do you do any work?
[03:24]
B
No, not at the moment. Basically I get support from my parents at the moment.
[03:29]
A
Have you looked for any work at all?
[03:30]
B
Yeah, I mean I've applied for graduate schemes but at the same time I'm also applying for masters.
[03:35]
A
Ok. And what about you, Simon, do you work?
[03:37]
B
No, I don't work. I used to work but because it's my last year, so I decided not to work at the moment.
[03:43]
A
Now, despite Simon, Robin and Roberto all being in full time education, they're all classed differently for the purposes of the official employment stats. Roberto's employed, Robin's unemployed because he's looked for paid work but he didn't get it. And Simon would be counted as economically inactive. I asked them what they thought about that.
[04:05]
B
Strange, because we're all similar situations, so how can there be so much change of so much different categories? I don't understand. The thing is that I don't really need to work at the moment. I think it doesn't make sense.
[04:15]
D
Let's say if you get the support
[04:16]
B
from your parents, you're a full time student, then you shouldn't be really considered as unemployed. So hasn't the government got a point here? It does seem a little odd to be counting people who are in full time education as unemployed.
[04:28]
A
Well, yes it does. And it's true that if you discount unemployed students, the headline figure falls quite dramatically. But if you agree with the government's line that students shouldn't be counted as unemployed, then by the same logic they shouldn't be counted as employed either. And there are an awful lot of employed students, 870,000 at the last count.
[04:50]
B
And I guess this matters because of the way that unemployment rates are calculated.
[04:54]
A
Yeah, exactly. All employment rates are calculated as percentages of the economically active population, not the total population. So in the case of students, if you say they shouldn't be counted as employed or unemployed, then the denominator that you're using to calculate the results is shrinking. And just a little basic number crunching tells you that if you class all young people in full time education as economically inactive, then the headline rate for youth unemployment stays the same. I put that point to Chris Grayling.
[05:26]
C
The issue is about unemployment and how we respond to that. I mean, I have to say I don't think that students doing a couple of hours a week of employment should show up in the overall employment figures either. And it does have, I think, a demoralising effect on young people if the headlines are all about record youth unemployment.
[05:44]
A
But as I said, if you count students all in the same way, the headline rate that one in five, while the overall number is fewer, you still end up with 20%.
[05:53]
C
Well, my concern is not about the overall numbers, it's not about the statistical debate, it's actually about dealing with a very human problem of people who do not have jobs to whom we need to be focusing real attention to try and help them get into the workplace.
[06:07]
A
Well, putting the statistics debate aside, even so, the figures are still pretty bad. Hundreds of thousands of young people are unemployed, even discounting students. But is it fair to call today's young people a lost generation? I've been speaking to Simon Briscoe from economic data company Timetric.
[06:25]
E
If we look at the unemployed youths as a proportion of all of the youths, then it's going to come in at about 12% rather than, let's say, about near 20%. You know, I'm not saying that one in eight unemployed youths is not a bad thing and is not really grim for those individuals, but I don't think that one in eight makes youths stand out from the broader population in the way that saying 1 in 5 it is very difficult because the labor force survey didn't exist 30 years ago. So we can't really compare like with like. But certainly when you do look at the number of young people on benefits of one sort or another related to unemployment, then the numbers were very, very much 30 years ago.
[07:08]
B
That's Simon Riskoe's for you. Have you managed to get anything out of the ons?
[07:11]
A
Yeah, we've asked the Office for National Statistics to give us a bit more data so we can Get a better picture of the work that's being carried out by 18 to 24 year olds in full time education. And Simon's helped us out with the analysis.
[07:24]
E
I think these figures are really quite revealing because it does show that there are over half a million people aged between 18 and 24 who are full time students who are working more than six hours a week. And a quarter of a million of these students are working more than 16 hours a week. And that's a doubling in the last decade.
[07:44]
B
Well, that's Simon Briscoe of Tim Metric and our own Hannah Barnes. Thanks, Hannah. Now, you may have heard Evan Davies clashing with Ed Balls, the Shadow Chancellor, on the Today programme on Wednesday. What is your line? If we increase the deficit by a pound, how much extra growth do you think we get?
[08:02]
F
Do we get a pound 50 of
[08:04]
D
a bigger economy or do we get 20p bigger economy?
[08:07]
F
What is the relationship between letting the deficit rise and the growth?
[08:14]
B
And if you were thoroughly confused by that exchange, you weren't alone. One of the key political debates of our age is whether the government must immediately and substantially cut spending and raise taxes in order to cut the deficit, which is what simple arithmetic would demand, or whether doing that will be counterproductive, undermining the economy just as it's slowly struggling out of recession. This is a debate about what economists call the fiscal multiplier, which is what Evan Davis and Ed Balls were talking about. If the multiplier is close to zero, there's no need to worry that cuts will undermine the economy. The private sector will fully step in to fill gaps that the public sector leaves. If the multiplier is more than 1, then government spending cuts will actually make the private sector smaller. If the multiplier is more than zero, but less than 1, the private sector will partly, but not completely expand to fill the space as the public sector shrinks. But how does this mysterious multiplier operate? How big is it and how do we know? It's something I've been agonizing about recently. You see, while more or Less was off air, I took on a very important responsibility. I'm proud to say that I've been elected Treasurer of Trumpetonshire County Council. Not everyone's heard of our little county, although it was famous in the 1970s when BBC Children's television broadcast a series of hard hitting documentaries about life in three local communities here, the genteel county town Trumpton, the village of Camberwick Green and the bustling industrial hub of Chigley. Life wasn't always straightforward in 1970s Trumpetonshire. But they seem like halcyon days compared with the fiscal crisis now facing our county council. We have an unprecedented budget deficit, meaning that each year we're spending substantially more than we take in in tax. Some blame overspending by the previous mayor. Others claim that the collapse of the Trumpton Building Society devastated the local economy. Whatever the reason, as Trumpetonshire treasurer, it's my job to fix the problem. I took firm action only this week. Trumpeton Fire Station. Hello, this is the treasurer of Trumpetonshire County Council. Is that Captain Flack? Yes, it is. I'm afraid I have some bad news. Captain Flack, can you please find Dibble? Find the what? Find Dibble, Please find him and tell him he's been made redundant. Oh, dear. Whatever next? I know it's hard to believe, Captain Flack, but this is a fiscal emergency. Cutbacks must be made. We're all in this together, you know. I'm sorry, but please will you or one of your team find Dibble and sack him? Yes, of course we will.
[10:58]
D
Right away.
[10:59]
B
Right away. It's one of the hardest phone calls I've ever had to make. It's not just the human cost, it's that I'm not even sure these cuts will help restore prosperity to Trumpetonshire. There are plenty of people telling me that making cuts is self defeating. So I've called in Chris Giles, economics editor of the Financial Times and the noted expert on the Trumptonshire economy, for advice. Chris, I've been making some tough decisions here, although I do feel obliged to point out that according to Wikipedia, the Trumpton Fire Brigade has never actually put out a real fire. Nevertheless, what effect might sacking Dibble have on the Trumptonshire economy?
[11:36]
F
He would have less money to spend now he's unemployed. And that money, he probably wouldn't be going to buy his biscuits from the factory in Chigley. And so the factory in Chigley would have less revenue, and then because they've got less revenue, they probably want to make fewer biscuits. And that would mean that maybe the wheat farmers in Trumpetonshire might have less demand and so on.
[11:58]
B
Windy Miller over in Camberwick Green.
[12:00]
F
Yeah, exactly. And so. And this process can continue and so it can go round and round in circles. So your original decision to fire the fireman damages the biscuit factory, damages the farmers, which then perhaps even means you have even less tax revenue, which means you have to then fire someone else and the process continues.
[12:18]
B
Well, that's all very plausible, but I guess there's another Way to tell the story. Imagine Jiggly Biscuit Factory is doing well. It's trying to hire good, reliable workers and it can't find anybody. And then when Dibble is fired from the fire service, suddenly he's on the labor market. They can hire him, they can expand their operations and that's good for everybody.
[12:41]
F
There's lots of ways you can tell the story. You can say that by firing Dibble, everyone else in Trumpton is actually much more reassured that you've got the finances of Trumpetonshire under control and then are more willing to spend themselves because they're no longer worried that the economy is going to implode because the public finances of Trumpetonshire are under severe threat.
[13:04]
B
Ok, so that's the abstract argument. I guess when we get to the specifics of Trumpetonshire, we have to actually understand some of the variables involved. So we have to know, I guess we have to know how deep the recession is in Trumptonshire. We have to know what sort of trading arrangements there are. Does it have a floating exchange rate, for example, or is it an open economy? Why might that make a difference?
[13:24]
F
If the fact is that your firing of Dibble means that he doesn't spend less in the Chiggly Biscuit factory, but he spends less in other parts of the uk, then actually that's not a problem for Trumpet and Shire. So you've actually exported the spending cut to someone else. It's a problem somewhere else, but not domestically. Equally, if unemployment is very low and demand elsewhere is high. So there's a lot of other things going on in Trumpetonshire. Then again, firing Poorwald Dibble means that he's quite likely to find a job elsewhere because there's enough momentum in the Trumptonshire economy.
[14:02]
B
Well, thanks, Chris. That's Chris Giles of the Financial Times. I also asked Ray Barrell for advice. Ray is the head of macroeconomics at the National Institute for Economic and Social Research. Ray's not an expert on the fiscal multipliers for Trumpetonshire, but he does know a lot about the evidence from around the world.
[14:19]
G
Well, in a country like the US, you might expect that if spending is cut by $100, then output might fall by about a hundred dollars. In the UK, you might expect if spending is cut by about £100, output will fall by about £70 because some of the spillovers go into imports. Country like Ireland, though, if you. Which Trumpeton is probably like. If you cut.
[14:41]
B
I hope not. Their banks are in a shocking state.
[14:44]
G
Well, we haven't inspected Trumpton's banks yet. But if you cut spending in Ireland by €100, then probably most of that will go into imports and output will only fall by about 10 or €15.
[14:55]
B
Well, thanks, Ray. If Ray Barrel is right, then I needn't worry too much about firing Dibble. I will have a smaller fire brigade, but I won't have a smaller economy. But for the UK as a whole, it's not so straightforward. Ray Barrel's estimate is that the spending multiplier in the UK is 0.7. That means that for every million pounds by which George Osborne cuts public spending, the private sector grows by just 300,000 pounds and the economy as a whole therefore shrinks by 700,000 pounds. This is over the short to medium term, over four to five years. Most economists would assume the multiplier is zero and the private sector will eventually fill any gap. But Chris Giles told me that the range of uncertainty is pretty wide. The fact that the UK is quite an open economy with a flexible exchange rate points towards a low multiplier and support for George Osborne's austerity position. But the fact the recession has been deep and unemployment's high suggests a high multiplier and support for Ed Balls position. Chris thinks that the consensus estimates of the fiscal multiplier are roughly halfway between what George Osborne would like us to believe and what Ed Balls would like us to believe. He also told me nobody can possibly know for sure. Whatever the fiscal multiplier really is in Trumpetonshire. I'm afraid it's too late for Dibble.
[16:12]
D
Pew.Pew.
[16:13]
B
barney McGrew, toughbut grub. You're listening to More or Less in association with the Open University, and I'm Tim Harford. You can email us on more or lessbc.co.uk and you can find me on Twitter simply as Tim Harford. If you have 20 minutes and you're unsure what to do with them, you could download an old edition of More or Less and listen to 72% of it. Or you could complete the Big Risk Test questionnaire on the PBC website. There's a link to it on our site, BBC.co.uk more or less the Big Risk Test hopes to be the biggest ever study of our perception and understanding of risk. My producer took it and he claims it's actually rather fun. I can't vouch for it myself, but then I never have 20 minutes to spare and he evidently does. Which figures? And our alternative to the census is still up on the Today programme website. And there's A link to that on ours. Hello. I have a problem and I would be most grateful if you could help
[17:11]
D
me to solve it.
[17:12]
B
Why, it's Lord Bellborough of Winkstead hall near Chigley. What can I do for you? It's the apple crop this year. It's too big. I'll see what I can do at once. Lord Bellborough. That man's so out of touch. We're not really all in this together, are we? The trouble with this country is there's not enough social mobility. And don't take that from me. That was the message this week from the Deputy Prime Minister, Nick Clegg, as he launched the government strategy on social mobility.
[17:38]
G
It is simply unacceptable, Mr. Speaker, that so many of our children have their life chances shaped by the circumstances of their birth. Gaps in development between children from different backgrounds can be detected even at birth. By the age of five, bright children from poorer backgrounds have been overtaken by less bright children from richer ones. And from this point on, the gaps tend to to widen still further.
[18:03]
B
In the strategy document, opening doors, breaking Barriers, a striking graph is prominently displayed. It caught the eyes of the BBC news website editors, who used it to illustrate their story about the strategy launch. The graph isn't that new. It's based on work by Leon Feinstein, now a professor at the Institute for Education, published as a discussion paper way back in 1998, studying the educational outcomes of children born in 1970. It's really about the performance of the British education system under the likes of Edward Heath and Jim Callaghan. It's an incredibly powerful graph and I've written about it admiringly in my own work for the Financial Times. But perhaps it doesn't show what I and Nick Clegg's team think it shows. We'll put the graph up on our website, but let me try to paint a picture of it in your mind. The graph tracks the performance over time of children who are set some tests of cognitive ability. The tests give a snapshot of how the children are doing at the ages of just under two years old, nearly three and a half, five years old and ten years old. Now, the graph splits these children into four groups very loosely. We could call them bright rich kids, bright poor kids, dim rich kids and dim poor kids. Although the real definitions are both more precise and more polite. You count as bright if you're one of the top performing toddlers and dim if you're one of the worst performers at the age of 22 months. What's appalling about the message of this graph? Is the way that by the age of 10, the talentless toddlers with rich parents have overtaken the bright poor kids. It's a profoundly upsetting illustration of the way natural talent is trumped by social privilege. Or is it? Daniel Reid, professor of Behavioral science at Warwick Business School, emailed us because he thinks the way the richer dim kids seem to catch up with the poorer bright kids might be at least in part due to a very old statistical problem.
[19:57]
D
There may indeed be some effect, but I think the majority of the effect that we see is due to a statistical artifact that's called regression to the mean.
[20:05]
B
Regression to the mean is simply this. Imagine you've got a sample of data. Maybe it's the quality of meals at restaurants. Maybe it's the number of traffic accidents at traffic junctions. Maybe it's the test scores of toddlers. Now pick out the very best restaurant meals, the very safest junctions, the highest test scores, and think about the effect of sheer good luck. Sometimes the chef at a restaurant happens to find some really exceptional ingredients. Sometimes the users of a traffic intersection are lucky and avoid accidents when someone could easily have had an eye out. And sometimes toddlers have a particularly good day. In our sample of the very best performances, then some of them really are top performers and some have enjoyed good luck. If we come back another day, we might find that the restaurant has gone off the boil, the drivers at that safe junction have become careless, the bright kid has somehow got stupider, and all that might be happening is that our original measure reflected good luck. What relevance does this have for the social mobility graph? Well, think back to what Nick Clegg said to Parliament. He said that there was already an ability gap detectable at par birth between families of richer and poorer children. Certainly there seems to be a gap at the age of 22 months. According to the British cohort survey on which Feinstein's analysis is based, the richer toddlers are more likely to be in the high ability group at the age of 22 months. But if the typical poor kid is born at a disadvantage to his richer peers, then if you take a snapshot of toddler ability at 22 months, the poorer kids in the high ability group are disproportionately likely to be there because they happen to have a good day. And the richer kids in the low ability group similarly are more likely to be there because of bad luck than because their cognitive ability really is lower. It's not that it's all fluke, but if there's any element of luck at all which There surely is, because we're talking about ability tests for toddlers, then we have to allow for what we'd expect to happen when that luck fails to last. And what we'd expect to happen is pretty much what the graph in the government social mobility strategy shows, which is that the next time you test the children, all the high performers have dropped off, but especially the poorer kids, who, remember Nick Clegg says, were disadvantaged from birth. And all the lower performers have caught up, but especially the richer kids. And then as you continue to test the richer kids, gain on the poorer kids at a very much less dramatic pace. Daniel Reid thinks it's an open and shut case. A lot of what looks like social unfairness is actually just this statistical effect of regression to the mean.
[22:47]
D
There's one feature of the graph which is very diagnostic, which is that if you look at it, you see that most of the shift that we Observe occurs from 22 months to 40 months. So from the first observation of the second, and what's happened is that in the first observation, you've selected the kids. So you said, well, these are the guys who did really well in 22 months. Let's look at them. And here are the people who've done very poorly at 22 months. Well, let's look at them. And now the next observation we make, what happens is we don't expect the error which pushed the measures up or down to reoccur, and so the measurements tend to get closer to the mean. And there's this very big effect from 22 months to 40 months. And if you look at the graph, you see that after that there's not really much going on at all. There's a little bit still, but there's not very much. So almost all of it is due to that shift. And that shift, I think, certainly due to regression to the mean.
[23:35]
B
And the implication of that, presumably is we're horrified at the idea that these very talented poor kids are overtaken by these extremely untalented rich kids.
[23:45]
D
Right.
[23:45]
B
But actually the initial measure of talent was pretty noisy.
[23:49]
D
That's right, the initial measure was noisy. And by choosing the top performers on that noisy measure and the bottom performers on the noisy measure, you actually augmented the degree of error. And you also put a bias in the error, making the error positive for the high performing kids and negative for the low performing kids.
[24:06]
B
I realise this is not something you've looked in tremendous detail at the data, but is there any way that this can be cleaned up and we can allow for regression to the mean and we can actually figure out whether there's still a real effect. And you do think there is still a real effect?
[24:19]
D
Well, I think there is some effect. I mean, the simple way to do it is to say, well, let's look at the graph and let's simply block out the first observations and now look at what happens for the remaining observations. And what you see then is that from the age of 4 to the age of 10, you find that the poor kids who are performing the best, their performance goes down a little bit, and for the rich kids who are performing the worst, their performance goes up a little bit. There seems to be an effect there where the rich kids are benefiting over time and the poor kids, perhaps slightly less, are getting worse over time. That's only true for the high performing poor kids and the lower performing rich kids. Now for the poor kids who are low performers, their performance doesn't change over time. And for the rich kids who are high performing, their performance doesn't change over time. If I can say as well, there's further evidence that it's regression to the mean, because the thing with regression to the mean is it doesn't just work forward. If you have an exceptional experience, the previous experience was probably more mediocre than the one you're currently having. And actually in the original paper describing these results, there is a figure showing children sorted at 40 months and looking back at their performance at 22 months. And there you get the same pattern as when you go forward showing that that's a regression to the mean effect and doesn't have anything to do with forward looking causality. And again, if you block out that particular observation, you find that there's a small effect left over, but it's quite
[25:42]
B
small because the naive interpretation of that graph would be that the talented poorer kids suddenly, dramatically catch up with the talented richer kids by the age of 14 months and then they fall away again.
[25:54]
D
That's right.
[25:54]
B
And that seems a very weird thing to happen. And so that's sort of evidence that this is partly a statistical artifact.
[26:01]
D
I think I would call it proof.
[26:02]
B
Daniel Reed of Warwick Business School. Now, Wesley Stevenson's here. Wes, you've been trying to track down Leon Feinstein, the man behind the original graph.
[26:09]
H
Yes, he's in a slightly tricky position at the moment because he's currently doing some work at the treasury and he couldn't just drop everything to come on the programme. But he did tell us that the measurement error could explain a small part of the pattern. But he's very skeptical that Regression to the mean is the main influence on the results. He thinks it doesn't explain why poor kids with high test scores are more likely to be having a good day than their rich counterparts.
[26:34]
B
So presumably you put that to Daniel Reed. What was his response?
[26:37]
H
Well, yes, he stands by his critique. He thinks Feinstein's research shows clearly that rich kids do better on average than poor kids in these tests. There are three times as many rich kids in the study getting top scores than there are poor kids. He thinks those few poor kids are outperforming the average for their group much more than the rich kids are. And so you'd expect them to fall back more than the top performing rich kids do, which is exactly what the graph shows.
[27:03]
B
Well, thank you, Wes. Incidentally, we're not claiming there's no problem with social mobility. Lots of different data sources here, lots of reasons to think that social mobility may be low in the uk. But this one rather famous graph may be, at least in part, a statistical anomaly. And that's all we have time for this week. Don't forget, you have one week left to fill in our alternative census. There's a link on our website, BBC.co.uk more or less. We've had over 10,000 responses so far and we'll be broadcasting the results on the 22nd of April. We'll be back next week. Until then, goodbye.
[27:40]
A
More Or Less was presented by Tim Harford, the FT's undercover economist. The producer was Richard Knight and the editor was Richard Varden. It was made in association with the Open University.