How useful is GDP?

Summary

More or Less: "How Useful is GDP?"

Podcast: More or Less, BBC Radio 4
Host: Tim Harford
Date: April 22, 2011

Overview

In this episode, Tim Harford and guests tackle the relevance and usefulness of GDP (Gross Domestic Product) as a measure of national progress and well-being. The discussion moves from defining and critiquing GDP to considering alternatives that could better reflect societal well-being. The episode also covers the limitations of international educational comparisons using PISA statistics and explores a probability puzzle involving fathers named "John Charles."

---

Key Discussion Points and Insights

1. What is GDP and How is it Measured? (00:32 – 02:44)

• Joe Grice, Chief Economist at the Office for National Statistics (ONS), explains:
  • GDP measures a country’s output, income, and expenditure—ideally, these should align, though practical differences arise from varied data sources.
  • The ONS collects around 80,000 returns from producers and uses direct estimates for less tangible outputs, such as health services (measured by number of patients treated and quality of care).
  • Quote: “It really measures what as a country we produce... it's also a measure of what the income from that output is, ... an estimate of what we have available to spend.” — Joe Grice [01:50]

2. GDP as an Economic Indicator: Strengths and Limitations (02:44 – 04:34)

• Martin Weale, Economist and member of the Bank of England’s Monetary Policy Committee:
  • GDP is a “good, fairly general measure of economic activity,” and serves as the primary tool for comparing economic performance over time.
  • Early GDP estimates, despite imperfections, receive significant attention from policymakers, including those setting interest rates.
  • Quote: “For all their shortcomings, I do give rather a lot of attention to the GDP numbers...” — Martin Weale [03:30]

3. Should GDP Be Replaced? Well-being and the 'Easterlin Paradox' (03:59 – 06:24)

• Andrew Oswald, Professor of Economics, University of Warwick:
  • Argues that GDP’s usefulness as a progress indicator is waning:
    • Early UK conditions—scarcity, simple needs—justified prioritizing economic growth.
    • In modern times, extra GDP does not correlate strongly with increased happiness (Easterlin Paradox).
  • Advocates for a shift toward measuring happiness and mental health, predicting GDP “will look increasingly out of date, maybe even useless” in coming decades.
  • Quote: “Once politicians and researchers grasp that ... GDP as a measure of our overall well-being and contentment, that's going to look increasingly out of date, maybe even useless.” — Andrew Oswald [04:34]

4. Measuring Well-being: Subjective and Objective Approaches (05:27 – 07:04)

• Oswald notes the evolution of well-being research:
  • Simple happiness surveys are surprisingly informative, but blending subjective data (“How happy am I today?”) with objective/physiological measures (e.g., cortisol/stress levels, air quality) yields robust results.
  • Advocates a gradual transition to well-being metrics to ensure empirical soundness.
  • Quote: “We're trying to blend subjective answers ... with objective and in some cases physiological measures.” — Andrew Oswald [05:37]

5. The 'Other Census': Well-being, Loneliness, and Optimism in Britain (07:04 – 10:56)

• Tim shares results from More or Less’s alternative census (with >14,000 self-selecting respondents), comparing them to official data:
  • Respondents report high happiness, frequent laughter, optimism (especially among the over-65s), but noted loneliness was prevalent among those aged 16–24.
  • Quote: “What is perhaps surprising is that the loneliest age is apparently 16–24, where nearly half ... said they were lonely.” — Tim Harford [08:39]
  • Regional findings: Northern Ireland residents laugh the most; Wales, the least.
  • On what would make them happier, answers included higher income, better health, love, and more sleep—with touching personal comments such as “Norwich City winning promotion to the Premier League,” and “Return of my wife from Alzheimer's.” [10:47 – 11:27]

6. Limitations and Pitfalls in Interpreting Self-Selected Data (07:49 – 09:39)

• The sample is not representative, heavily skewed by age and Radio 4 listenership.
• Re-weighting attempts don’t solve the core selection problem—self-selecting young Radio 4 listeners may not represent the wider young population.

---

7. Comparing International Education: Problems with PISA Statistics (11:53 – 22:12)

• PISA (Programme for International Student Assessment): Run by the OECD, ranks countries in education every three years.
  • Andreas Schleicher, PISA head, notes the UK’s 2000 and 2003 data were excluded for poor methodological quality—only comparisons between 2006 and 2009 are valid.
  • Despite this, UK Education Secretary Michael Gove cited rankings showing a dramatic UK decline, using PISA data in a way PISA itself disavows.
    • Quote: “In the last 10 years, we've plummeted in the rankings from 4th to 16th for science...” — Michael Gove [15:20]
  • Alan Smithers, Professor of Education, criticizes misuse of PISA stats due to changing test populations, response rates, and the number of countries participating. [16:37]
  • Selection bias and lack of comparability undermine reliability of simplistic country rankings.

---

8. Are PISA Rankings Themselves Flawed? The Danish Critique (18:02 – 22:57)

• Svend Kreiner, Danish statistics professor, claims PISA’s methodology is fundamentally unreliable due to Differential Item Functioning (DIF)—the idea that test questions have varying difficulty across countries.
  • Says country rankings could vary wildly based on test composition: “For UK, I can get them up at number 8 or down as number 36.” [19:47]
  • The statistical model (the Rasch model) presupposes the absence of DIF; Kreiner argues this is not met.
  • Schleicher (PISA) counters that the test pool ensures overall comparability and disputes Kreiner’s conclusions, saying: “The model is always an approximation of reality. The question is... does the model fit the reality such that there is no distortion of the results.” [21:44]
  • Tim Harford concludes that this methodical debate is highly technical and unresolved, pending peer review of Kreiner’s paper.

---

9. Probability Puzzle: Five Friends, Five ‘John Charles’ Fathers (24:16 – 27:20)

• Matt Parker, mathematician, tackles a listener's probability question:
  • The initial approach suggests astronomical improbability—1 in 40 billion billion—but assumptions of independence (names, friends) are false.
  • Real-world name probabilities and social selection biases make such coincidences far more likely than raw math suggests.
  • Takeaway: Seemingly mind-blowing coincidences are less surprising in the context of large populations and multiple opportunities for discovery.
  • Quote: “It's amazing if it happens to you, but it's not amazing that it happens in general.” — Matt Parker [27:20]

---

Notable Quotes & Memorable Moments

• Joe Grice on GDP’s conceptual breadth:
  “So GDP... is also a measure of what the income from that output is, how much income we earn, and... what we have available to spend.” [01:50]
• Andrew Oswald on GDP’s limits:
  “A kind of smooth transition from the old obsession with income measures... to our new concern for measuring mental well-being.” [04:34]
• Personal, touching survey responses about happiness:
  • “Norwich City winning promotion to the Premier League...” [10:47]
  • “Return of my wife from Alzheimer's...” [11:05]
• On PISA data misuse:
  “The government is presenting an over simple message which actually fits in with the narrative...” — Alan Smithers [17:51]
• On the John Charles coincidence:
  “It's amazing if it happens to you, but it's not amazing that it happens in general.” — Matt Parker [27:20]

---

Timestamps for Key Segments

• Definition and challenges of GDP: 00:32 – 04:34
• Debates about replacing GDP with well-being measures: 03:59 – 07:04
• Alternative Census results and British well-being insights: 07:04 – 10:56
• Comparing international education: PISA explained and critiqued: 11:53 – 22:57
• Probability puzzle—John Charles: 24:16 – 27:20

---

Tone and Style

The episode maintains Tim Harford's witty, gently skeptical, and accessible tone, combining technical clarity with humor and thoughtful analysis. Guest experts speak plainly but with academic precision, and listeners' contributions add color and humanity throughout.

---

In Summary

This episode explores the strengths and pitfalls of GDP as a measure of national progress, the emerging potential of happiness and well-being statistics, the dangers of over-interpreting comparative education metrics, and the quirks of probability in everyday life. It encourages critical thinking about numbers, their limits, and the very human stories embedded within statistics.

Transcript

[Tim Harford]

Thank you for downloading the More or Less podcast from the BBC. You can find out more about our programme on our website, BBC.co.uk radio4. But before you do that, here's Tim Harford.

Hello and welcome to More or Less Statistically proven to be the best program about numbers on Radio 4. This week we'll find out who the Education Secretary Michael Gove did describes as

[Michael Gove]

the most important man in the British education system. Much more important than me.

[Tim Harford]

And a listener writes, I was at dinner with my wife and three friends, we discovered that all five of our fathers were called John Charles. What are the chances of that? The mysterious case of Mr. John Charles and news to hearten the numerati everywhere. It's nearly time to reveal the results of our other census. But before we tell you what you or a non random sample of more than 14,000 of you told us, let's take a step back. Our census tried to measure some elements of happiness or well being. We're not the first. It's a project which has caused considerable excitement. The Prime Minister, David Cameron, has asked the Office for National Statistics to investigate the prospects for collecting meaningful well being data. And yet, for now, when it comes to putting a number on national progress, there still seems to be only one game in GDP or gross domestic product. The Office for National Statistics will release the latest GDP estimate next week and we can be sure that whether the economy grew, shrank or stayed the same, it will be big news. But what exactly is GDP and should we be relying on it as a measure of. Well, of what? Joe Grice, the chief economist at the ons, explains what GDP actually does measure.

[Joe Grice]

It really measures what as a country we produce, what's the value of the output we produce. So it's also a measure of what the income from that output is, how much income we earn, and it's also an estimate of what we have available to spend. In principle those numbers should be identical, but since we get the information on output, income and expenditure from different sources, in practice they're not always identical. So, so we have to balance that information up to find a central estimate of what has actually happened. Roughly speaking, 80,000 returns we receive from producers about what they've produced, we also have information on these kind of more difficult outputs. If you take the health service as an example, direct measures of the output. We know how many patients are treated and we have information about the quality of the treatments they receive. So we're able to use that kind of information to get a direct estimate of these. If you like more difficult, more esoteric outputs.

[Tim Harford]

Joe Grice from the Office for National Statistics. The economist Martin Weil has written extensively about GDP and now sits on the bank of England's Monetary Policy Committee. I asked him why GDP had become the measure of choice.

[Andrew Oswald]

Well, we've discovered that it's a good, fairly general measure of economic activity, of the size of what the economy is up to. And if people want to know whether there is more or less economic activity compared with last year or five years years ago, GDP is the best measure that we have.

[Tim Harford]

You now sit on the Monetary Policy Committee. How useful are these early releases of GDP and the revisions of GDP and other data to you and your fellow committee members when you're trying to set interest rates?

[Andrew Oswald]

From my perspective, the early estimates of GDP that we have are the best indicators available of the activity in the economy. For all their shortcomings, I do give rather lot of attention to the GDP numbers when they emerge.

[Tim Harford]

Martin Weil but not everybody thinks gross domestic product is such a great measure. I spoke to Andrew Oswald, professor of Economics at Warwick University.

[Andrew Oswald]

GDP in its time was a very useful measure. Many people had outside toilets, there weren't many automobiles. There was a real shortage of food for some of our citizens. But now many of those things have been turned on their head. I want to see us move away from measuring gross domestic product to measuring the happiness and mental health of our citizens. That seems to me logical and the next step for the coming century.

[Tim Harford]

So you would specifically like to stop calculating gdp, or do you think it's going to be useful? But it needs to be kept in perspective and you'd like to add other

[Andrew Oswald]

measures in 50 years from now. I can't see GDP being very valuable. And a key reason for that is something called the Easterlin Paradox, where there's a strong lack of link at this point in our richness as a country between extra GDP and extra happiness. It appears that happiness is running flat. Once politicians and researchers grasp that, say over the next 10 or 20 years, then GDP as a measure of our overall well being and contentment, that's going to look increasingly out of date, maybe even useless. A kind of smooth transition from the old obsession with income measures. That's what GDP is. To our new concern for measuring mental well being. That ought to happen steadily. We can't just jump straight away to well being measures. So I think there'll be a slow change and that's a sensible step.

[Tim Harford]

So let's talk about the ways we could think about measuring well being. What sort of Advances have been made and are we doing better than simply asking people how happy they are? Or maybe is that enough?

[Andrew Oswald]

It's not enough to ask people just how happy they are, though I've been surprised how persuasive some of the data patterns are, even from very simple kinds of questions like that. We're trying to blend subjective answers how happy am I today? With objective and in some cases physiological measures. The latest incarnation of research is going to blend subjective answers with cortisol readings, for example, the stress hormone is going to blend them with objective measures, how clean is the air, how much sunshine is there, and so on. We're finding a strong match between the subjective answers and the objective data, and that's very encouraging. And we can use that kind of match for all sorts of purposes in social science.

[Tim Harford]

When we began this conversation, I was assuming that you would be very keen on the Prime Minister David Cameron's effort to push measures of well being. But actually, hearing you talk, it seems that David Cameron's a bit behind the curve.

[Andrew Oswald]

I admire what Cameron is doing. I do think this trend is unstoppable. But of course, primarily I would be quite happy to to see the policy making go a little more slowly and allow us to get, let's say another decade at least, of really sound empirical knowledge about the foundations of mental health and happiness. We don't have to rush into policy yet.

[Tim Harford]

Professor Andrew Oswald. Andrew is, I suspect, in a minority in imagining that GDP will be replaced by a wellbeing index. But many people think measures of well being can tell us something useful about ourselves. It was in that spirit that more or less, together with our colleagues on the Today programme, launched our own alternative census three weeks ago. Unlike the real census, which deals in good hard facts, we tried to get at more subjective information. Are we lonely? Selfish? Happy? And unlike the real Census team, which won't start publishing results until July 2012, we've completed our analysis in under a week. Thanks again to Peter Lynn of the University of Essex for helping us get set up. First, the small print.

The other census is not a representative sample of the British population. It is instead a self selecting sample from a population of listeners to the Today programme and to more or less who have access to the Internet, terms and conditions apply.

In other words, if anyone else had done this and used the results to sell package holidays or shampoo, we'd be the first in the queue to tear strips off them. But we're not selling anything and well, go easy on us. Now to the results. Over 14,000 people completed our survey, of whom 54% were men. You seem to be a happy bunch. Over 10,000 laugh two or three times a day, or many times a day. Almost 8,000 of you are fairly happy, and almost 4,000 are very happy, with fewer than 400 not at all happy. And over 10,000 of you consider yourselves to be optimists. And the over 65s are the most optimistic of the lot. Most of you aren't lonely, but nearly 4,000 of you are. What is perhaps surprising is that the loneliest age is apparently 1624, where nearly half of respondents said they were lonely. And the same age group were the most likely to consider themselves selfish people, over 40% compared to about one third overall. Perhaps this tells us something about what it is to be young and British today. Or perhaps the real message is if you're under 25 and you listen to Radio 4, you're in a bad place. We asked whether you're considerate and kind to almost everyone. Nearly 12,000 of you say that you are an overwhelming majority, and kindness is almost uniformly distributed across the ages. Good for you. We also have a regional perspective. Bad news for the people of Wales, I'm afraid you laugh the least. According to our survey, the people of Northern Ireland laugh the most. On the subject of age and location, one statistical trick that can be used by pollsters is to reweight a sample to better match the underlying demographics. Ipsos Mori, the opinion poll company, very kindly agreed to do just that to our data. But they kept warning us that this wouldn't correct this sample selection problem, and we can see what they mean. For instance, as I mentioned, young survey respondents were more likely to be lonely. But compared to the UK population as a whole, our survey was light on young people and heavier on older respondents. The reweighting would put more emphasis on the responses from young people, leading us to conclude that overall, the UK population is lonelier than our sample would suggest. But is that reweighting a good idea in this case? Only if young Radio 4 listeners are just like young people everywhere. And this seems like a big assumption to make. Finally, we asked you what might make you happier. Almost 2,800 said a higher income would do the trick. But almost as many named better health, love or more or better sleep. But as so often, it wasn't the data that spoke volumes, but the additional comments.

[Matt Parker]

Norwich City winning promotion to the Premier

[Tim Harford]

League, a baby, fewer stupid people for

[Matt Parker]

my wife to indicate in any way that she loves me, having my student

[Tim Harford]

loan paid off, a girlfriend, return of my wife From Alzheimer's, being 20 years

[Matt Parker]

younger, guilt free extramarital sex, better health

[Tim Harford]

for those I love, less hassle for

[Matt Parker]

my ex husband not being widowed, my

[Tim Harford]

children getting a job.

[Matt Parker]

Sunshine.

[Tim Harford]

Thanks to everyone who took part in our alternative census. You're listening to More or Less with me, Tim Harford, in association with the Open University. Now, last week we talked about changes to the tuition fees system. We didn't make it clear that the changes apply only to England, because education policy is devolved and of course we should have done.

[Andrew Oswald]

Sorry.

[Tim Harford]

But now let's talk about an education story that does affect the whole of Britain. How well are our school children doing compared to their international peer group? It's an important question and there's a way of answering it, using a set of tests called pisa, the Programme for International Student Assessment, which is run every three years by another acronym, the OECD, or Organization for Economic Cooperation and Development. According to PISA's latest figures, for example, the UK was ranked by 28th for maths out of 65 countries. But Hannah Barnes has been looking at the PISA data because, Hannah, there have been rumblings of discontent from a strange and unexpected quarter.

Yes, Denmark. Now, Tim, although you're right, this is the story of a statistical model, it's also a rather touching story about two men. Andreas Schleicher from pisa and the Education Secretary, Michael Gove.

He's actually the Education Secretary for England, isn't he?

Yes. Now, Michael really, really likes Andreas. Listen to this. From the Education World Forum.

[Michael Gove]

Yesterday, you heard from a man that I recently described as the most important man in the British education system. Much more important than me, but he could equally be the most important man in world education.

[Tim Harford]

He really does like him.

Yeah. And there's more.

[Michael Gove]

Andrew Schleicher is a German mathematician with the sort of job title that you wouldn't wish on your worst enemy. He's head of the Indicators and Analysis Division, Directorate for Education at the Organisation for Economic Cooperation and Development. On the face of it, a job description like that might seem like the title of the bureaucrat's bureaucrat, but in truth, Andreas is the father of more revolutions than any German since Karl Marx.

[Tim Harford]

High praise indeed.

Enough to make Andreas blush.

[Andreas Schleicher]

Well, I can't comment on that. All we do is we publish comparative results and provide a mirror in which countries can see how they fare with respect to what the best performing systems show can be achieved.

[Tim Harford]

Now, the thing is, Michael Gove doesn't just like Andreas Schleicher, he loves his statistics. He and his department have quoted them to suggest declining standards in schools. One reason he'd like to shake up the education system. And Andreas is very happy for Michael to use his numbers. He just has one little rule.

[Andreas Schleicher]

We have very strict criteria. When, for example, a country doesn't meet very high standards of response rates or the quality of the survey samples isn't appropriate, or there are some other technical deficiencies, then we won't include the data for that year for that country. It happened to the UK, for example, in 2003, the data were not meeting the high quality standards of the oecd, so we chose not to publish them.

[Tim Harford]

And it's the same for 2000, isn't it, in the UK that that your report says they're not considered here because of methodological problems that invalidate those comparisons over time.

[Alan Smithers]

Yeah.

[Andreas Schleicher]

It's unfortunate because, of course, there's been a lot of effort invested in collecting those data, but we need to ensure that those data are comparable across countries. Basically, when in doubt, we don't use the data.

[Tim Harford]

So the PISA numbers are only calculated every three years and they started in 2000. But Andreas Schleicher says the data for the UK wasn't good enough in 2000 and 2003 and. And should be ignored. You can make a comparison between 2009 and 2006, he says, but that's it.

OK, fair enough. Very clear.

Yeah. So imagine how poor old Andreas felt when he heard this.

[Michael Gove]

In the last 10 years, we've plummeted in the rankings from 4th to 16th for science, 7th to 25th for literacy and 8th to 28th for maths.

[Tim Harford]

Oh, dear. These are based on the PISA numbers. Now, you asked us to imagine how Andreas Schleicher felt. Did you ask him?

I did.

[Alan Smithers]

Well, yes.

[Andreas Schleicher]

The OECD does not compare performance for the UK across the years. Many researchers do so. And basically, you know, this is a matter of judgment. I mean, we have made the judgment. When we are in doubt, we don't publish data. Quite frankly, there's probably little doubt that the performance of the UK has at best been flat. But, you know, we don't want to get, as the oecd, into methodological debates.

[Tim Harford]

And what has Michael Gove said to that?

Well, we invited him to come on, but his department sent us a statement instead, saying, among other things, that their reforms are based on wide ranging domestic and international evidence. But despite asking a number of times, the department hasn't directly addressed this question of comparing PISA numbers in a manner PISA itself doesn't like.

Well, perhaps it was just an innocent Mistake?

Well, maybe I asked that question of Alan Smithers, a professor of education at the University of Buckingham. He's not so sure.

[Alan Smithers]

We had a very low response rate in 2000, and we were almost excluded. And among the people who were responding, there was a disproportion from independent schools and successful state schools. 2009, a much greater effort was put in to get schools to participate. There were actually some financial rewards for participating. Another factor is that the tests really aren't that very important to schools. No special effort has gone into doing well in the PISA test. On the other hand, the national tests are extremely important to schools. And so on our national test, the scores have gone up and up, but in pisa, our scores haven't changed that much. And then there were 32 countries tested in 2000. There were 65 tested in 2009. So we're not really comparing like with like. And the government is presenting an over simple message which actually fits in with the narrative, enabling it to do what it wants to do.

[Tim Harford]

Now, misusing the data is one thing, but what if the PISA data itself is flawed?

I see. So this is the Denmark thing. Is this where I get to make a joke about something being rotten in the state of Denmark?

I think it just did well, but you're right. We've been sent a new and as yet unpublished paper by a Danish statistics professor called Svend Kreiner. He's really worried about the PISA rankings.

[Svend Kreiner]

I don't think it's reliable at all. I'm sorry. That's a sorry fact.

[Tim Harford]

Now, to understand what led Kreiner to say that, it's probably helpful to explain a little bit about how the PISA tests work. There are tests in maths, science and reading. Svend has taken the data from the reading tests only. Each 15 year old taking part in the test is given a booklet with 28 reading questions. And crucially, those items are meant to be equally difficult for everyone, including in every country.

Okay, how do you make sure that a question in Danish is as hard for a Danish kid as a question in Spanish is going to be for a Spanish kid?

Well, it's not going to be easy. And Sven thinks that PISA hasn't been able to do it.

[Svend Kreiner]

There's in particular one problem that is really serious for what they're using PISA data for something called differential item functioning or item bias. And this is a problem we have when items in educational tests have different degrees of difficulty in different countries. And in this case, I'm not actually able to find two items in pisa's. Test that functions in exactly the same way in different countries. It looks as if the ranking of countries depend very much upon the composition of their tests. In terms of items that are difficult in some countries and items that are easy in some countries.

[Tim Harford]

How big a difference can that make?

[Svend Kreiner]

It's an extremely big difference for UK. I can get them up at number 8 or down as number 36.

[Tim Harford]

I put that to Pisa, though, and they said it's impossible to do that. They said simply nonsense.

[Svend Kreiner]

Oh, but I have the data. I'm doing this using their own data.

[Tim Harford]

If Professor Kreiner's right and there's this thing called differential item functioning, or dif, to such a high degree, then it's really important because the statistical scaling model that PISA uses, the Rasch model, is meant to be used only when this problem diff is eliminated. Sven Kreiner says there's no evidence PISA has been able to do that.

This is a pretty serious attack on pisa. I mean, Kreiner is basically saying the entire exercise might be useless.

Yeah, he is. So I put what Professor Kreiner told me to Andreas Schleicher from pisa. He says they'll prepare a full response when Kreiner's paper is published. But in the meantime, he had this

[Andreas Schleicher]

to when you choose five items, you see that impact. But no sensible person would test a country based on five tasks. We have two hours of testing time, 270 minutes of material on the item pool, precisely to construct a test that, on balance, is equally fair. You see, I'm not saying that you cannot possibly find a task in which Denmark does significantly better than England and another task in which Denmark does worse than England. The question is, will your test as a whole appropriately compare British and Danish performance? And that is the question that we have demonstrated in every possible way. The problem is, though, the point I

[Tim Harford]

make that Professor Kreiner claims that the fact that you have differential item functioning means that you are using the wrong model. The statistical model requires that there isn't differential item functioning. And he says that there is not one single item that is the same for all countries, not one across the 56 countries, and therefore you cannot use this model.

[Andreas Schleicher]

Good. I mean, again, you know, we have published the item fit statistics and we do that on every assessment. You know that PISA is completely transparent about it. The model is always an approximation of reality. The Rasch model and any scaling model is an approximation of the reality that we observe. The question is not, does the model exactly fit the reality? The question is, does the model fit the reality such that there is no distortion of the results.

[Tim Harford]

So he's come out fighting basically, hasn't he? I mean, he's dismissed the critique entirely.

Yeah.

So who's right?

I hope she wouldn't ask that, Tim, because to be honest, I'm not sure. I've shown Kreiner's paper to other academics in this field and I've been told it's sound. As I say, PISA told me they wanted to wait for Kreiner's study to be published in a peer reviewed journal before commenting. But then just before I came in to record this, PISA sent me a rebuttal written by the man who carries out their statistical analysis, Ray Adams. That's also unpublished, of course, and it really lays into Criner, but it's very hard to judge who's right. The truth is very, very few people understand this area of statistics well enough to be definitive. I certainly don't.

Ok, well, as you say, best to be honest. Thanks, Hannah. Guess we might have to wait until Criner's work gets published if it does and the reaction to it before we can get much further with this. You're listening to More or less. I'm Tim Harford, I'm on Twitter as Tim Harford and you can reach us via More or [e-mail] or via our website, BBC.co.uk more or less. As many of you have emailed to point out, our website is difficult to navigate at the moment. Sorry, our website is in transition between two systems at the moment and we are hoping to improve things soon. Thanks, by the way, to the many, many of you who wrote in to comment on our interview with the libertarian philosopher Jamie White. Jamie argued that progressive taxes are unfair, and while some listeners agreed, most didn't. And some have reacted as if we invited Beelzebub himself into the studio. Much as we welcome your emails, be careful, that kind of criticism only encourages him. Now, among all those emails about progressive taxes, we found this from Richard. I was at dinner with my wife and three friends. We discovered that all five of our fathers were called John Charles. What are the chances of that? Well, search me. This sounds like a job for Matt Parker of Queen Mary University of London, otherwise known as the stand up mathematician.

[Matt Parker]

Well, I approached this problem the way most people approach a probability question. I broke it down into little bits, worked out the probability of each, and then multiplied them all together. And our basic unit we're looking at here is the probability. You have a dad whose name is John Charles and we are of course, assuming that the dad's name is John Charles, generic surname. So I had to work out the odds of someone being named John Charles. And in terms of dads, I'm going to assume that people who are dads of people who would then be at a dinner party, that they're going to be distributed around the 40s and 50s in terms of births. Now, I looked into the statistics because you can download them from the Office of National statistics. In the 1940s, John was the single most common name to give a boy. And in the 50s it was the second most common name to give a boy. Charles, however, comes in at about the 38th most common name. Even though I could get the rankings, I couldn't get the distribution. So I compared this to modern names. And currently the most common boys names account for about 2% of all births, and 38th most common names are about 0.6%. And so if you take your 2% and your 0.6%, the probability is in general, your dad's name would be John Charles is about 0.012%. If you've got five people or with their dad's name John Charles, multiply those together and it ends up being a probability of one in about, well, 40 billion billion, which is a ridiculous number. Even if you had 9 million dinner parties a day since the beginning of the universe, we would have only just had 40 billion billion of them. Something's gone wrong. In fact, there are two things that have gone wrong with this approach to probability. You can't assume things in the real world are independent. And if you look at names, the names are not independent. People with different family backgrounds tend to choose different names. And if you choose the first name, John, it's not independent. You choose a second name, Charles, and on top of that, they're at a party with their friends. And you don't choose your friends at random by rolling a dice.

[Tim Harford]

Well, most people don't.

[Matt Parker]

Friends are picked because they're people you know, and they're people that generally have similar lifestyles. So if we actually wanted to get the real probability of this, we'd have to look much deeper into the statistics and pretty much conduct some surveys at dinner parties. But short of doing that, we can still assume, even though it's much more likely than we expect, it is still fairly unlikely. Which is where we come across our second problem, the number of chances this has had to happen. There may have been more people at the dinner party and just five of them happened to match. What if their mothers names had matched, would that be equally amazing? Because that doubles our chances straight away. And the father's name could have been any name and we just looked at the chance that it's John Charles. And what if they discovered this at a football game or any other situation? In fact, there are loads of opportunities for people to realize they all have a significant person that they all know with all the same name. And once you start looking at that, you realize even though it's still amazing it happened to them, it's really not amazing that it happened in general. So if you look across the entire population, things like this are expected. It's a bit like winning the lottery. It's amazing if it happens to you, but it's not amazing that it happens in general.

[Tim Harford]

Matt Parker that's all we've got time for this week, but please keep your emails coming in. Our email address is more or lessbc.co.uk. i'm Tim Harford. On Twitter, the website BBC.co.uk more or less is the place to read more, listen again and fulminate about how difficult the website is to use at the moment. We'll be back next week. William and Kate will miss the fun, but hopefully they subscribe to our podcast. Goodbye.

More or Less is presented by Tim Harford, the Financial Times undercover economist. The More or Less team is Richard Varden, Richard Knight, Wesley Stevenson and me, Hannah Barnes. More or Less is made in association with the Open University.