Ryanair punctuality; mistakes in academic papers

Summary

---

Podcast Summary: More or Less

Episode: Ryanair punctuality; mistakes in academic papers
Date: May 27, 2013
Host: Tim Harford, BBC Radio 4

---

Overview

In this episode of More or Less, Tim Harford and his guests critically examine two main themes:

• The validity of Ryanair’s punctuality claims compared to other European airlines
• The prevalence and impact of errors and questionable practices in academic publishing

The episode combines data-driven scrutiny with lively, skeptical discussion, focusing on how numbers and statistics shape public debate, business claims, and scientific reputation.

---

1. Ryanair’s Punctuality Claims

The Claim and Its Repetition

• Ryanair’s Claim: Over 90% of its flights arrive on time, outperforming all other European airlines.
• Frequent Use: Ryanair routinely repeats this claim in-flight and in public communications.

  “Anyone who flies on Ryanair... will have heard this claim because they [have] been repeating it for years.” – Tim Harford (00:21)

Data Reality Check

• Independent Analysis: UK Civil Aviation Authority data (via Flightontime.info) reveals only 83% punctuality for Ryanair at 10 UK airports—below the stated 90% and not the best among competitors.

  “Based on that data... Ryanair is falling some way short of 90%, with only 83% of flights arriving on time. Nor is Ryanair the most punctual airline...”—Charlotte Macdonald (00:32)

• Limitations of the Data: Only covers 10 UK airports, not Ryanair's whole European network.

Possible Explanations & Data Gaps

• Ryanair’s Secrecy: The airline did not release more detailed figures.

• Expert Insight: Jim Paton, Cranfield University, points out Ryanair’s network includes smaller airports, possibly improving their averages.

  “A big proportion of their network in Europe is operations to small airports and airfields that don't suffer from air traffic congestion...” – Jim Paton (01:32)

• Questionable Comparison: Ryanair references statistics from the Association of European Airlines (AEA). However, those stats:

  • Haven’t been updated since 2009
  • Cover only 32 member airlines (of over 200 in Europe)
  • Exclude Ryanair itself

  “So I don't think that data really backs up their boast.” – Charlotte Macdonald (01:55)

Schedule Padding — The "On-Time" Paradox

• Listener Question: How is "on time" even measured?
• Industry Practice: "Schedule padding" gives airlines artificially long flight times to boost on-time arrival stats.
  • Ryanair claims not to do this; comparative analysis at Edinburgh mostly supports this (Ryanair schedules longer flights than competitors on only one route).

  “There is a term for this in the industry. It's called schedule padding...” – Charlotte Macdonald (02:42)

  • Historical Note: Flight times have lengthened over decades, in spite of technological advances.

    “In 1958, when it was using planes with propellers, it was taking 70 minutes [London-Amsterdam]. Now, it's 75 minutes.” – Charlotte Macdonald (03:19)

Memorable Quote

• “It's only a matter of time before they start charging us for listening to it.” – Charlotte Macdonald, joking about Ryanair's penchant for add-on fees (00:38)

---

2. Mistakes in Academic Papers

The Reinhart & Rogoff Scandal

• Background: Earlier series covered a pivotal economics paper by Reinhart & Rogoff which had misinformed major policy debates due to spreadsheet errors spotted by grad student Thomas Herndon.

  “I immediately saw the spreadsheet error... seeing is believing, but I almost didn't really believe my eyes.” – Thomas Herndon (04:01)

Prevalence and Types of Error

• Types of Issues:
  • Honest mistakes (e.g., miscalculations, spreadsheet errors)
  • Deliberate fraud (e.g., fabricated data)
• Incidence Rates:

  “The best estimate… is that it's perhaps between 1 and 2% of researchers commit the most serious forms of misconduct... In the US, 1% would be 3,000 articles a year.” – Dr. Elizabeth Wager (04:43)

The Replication Crisis in Social Psychology

• Example: Social priming study (elderly word priming makes people walk slower) fails to replicate unless experimenters expect it.

  “They only did so if the experimenters themselves expected them to do so.” – Ed Yong (05:31)

• No Fraud Alleged: Errors may stem from inadvertent experimenter bias, not misconduct.

Not Just Fraud — Questionable Research Practices

• Description:

  “Fairly common ways of basically massaging data, not fabricating it, in order to get positive results.” – Ed Yong (06:39)

  • Examples:
    • Stopping analysis when a "significant" result emerges
    • Testing many variables and only reporting those with positive findings

Structural Incentives and the ‘Publish or Perish’ Trap

• Pressures: Scientists incentivized to publish novel, positive results (not careful replications).

  “They're symptomatic of the pressure to publish or perish... What is good for science is not necessarily what is good for individual scientists.” – Ed Yong (07:14)

• Recognition Gap: Replication and error-checking are undervalued in academia.

  “Somebody who’s checking someone else’s idea... isn’t the person with a new idea. And as a result, what graduate student wants to choose something where they know they’ll never really excel?” – Unnamed Graduate Student / Economist (07:58)

Broader Importance

• Flawed research can influence public policy, waste taxpayers’ money, and mislead society at large.

  “If you think academic work has any value... then surely catching fraudsters, checking results, and correcting errors is valuable too.” – Tim Harford (08:22)

---

Timestamps for Key Segments

• Ryanair punctuality claim explored: 00:05 – 03:45
• Academic paper mistakes & replication crisis: 03:45 – 08:22

---

Tone and Style

The episode adopts a witty, skeptical tone—balancing serious inquiry with dry humor, especially when discussing opaque statistics and airline practices.

---

Notable Quotes

• “It's only a matter of time before they start charging us for listening to it.”
  – Charlotte Macdonald (00:38)

• “I immediately saw the spreadsheet error... I almost didn't really believe my eyes.”
  – Thomas Herndon (04:01)

• “They're symptomatic of the pressure to publish or perish... What is good for science is not necessarily what is good for individual scientists.”
  – Ed Yong (07:14)

---

Conclusion

This episode of More or Less unpacks how both commercial claims and academic research numbers can be misleading—whether through selective measurement, lack of transparency, or systemic incentives that discourage critical validation. Listeners are given not just the facts, but a lesson in skepticism and the real-world importance of verifying statistics and research.

---

Transcript

[Charlotte Macdonald]

This is the short edition of More or Less, first broadcast on the BBC World Service.

[Tim Harford]

Hello and welcome to More or Less on the BBC World Service. I'm Tim Harford. This week we test the veracity of this claim.

[Graduate Student / Economist (unnamed)]

Welcome.

[Charlotte Macdonald]

You've arrived in yet another on time flight. Last year, over 90% of Ryanair flights

[Tim Harford]

landed on time, beating every other European airline. Anyone who flies on Ryanair, a near ubiquitous, no frills airline operating all over Europe, will have heard this claim because they been repeating it for years. Charlotte Macdonald's here. Charlotte, it seems a bit odd that Ryanair keep using this statistic and surely

[Charlotte Macdonald]

it's only a matter of time before they start charging us for listening to it. Now, we only have Ryanair's word that this claim is true. The UK Civil Aviation Authority collects data on flights arriving and departing from 10 UK airports. Based on that data, a website called Flightontime.info worked out that Ryanair is falling some way short of 90%, with only 83% of flights arriving on time. Nor is Ryanair the most punctual air airline operating out of those airports.

[Tim Harford]

Well caught red handed then.

[Charlotte Macdonald]

It's hard to be sure of that. This is just data from 10 UK airports. There are lots of airports in Europe. It could be that Ryanair's average is much better when you take them into account.

[Tim Harford]

And is it?

[Charlotte Macdonald]

I just don't know. Ryanair refused to give me a breakdown of their figures, but Jim Paton, Senior Lecturer in the Department of Air Transport at Cranfield University in the uk, agrees. Ryanair's average could be being brought up by other routes.

[Jim Paton]

A big proportion of their network in Europe is operations to small airports and airfields that don't suffer from air traffic

[Ed Yong]

congestion, such as would be the case

[Jim Paton]

around London, for instance, or Paris.

[Tim Harford]

Right, so Ryanair are arriving on time 90% of the time, according to super secret proprietary data held by Ryanair. What about the claim that they beat every other airline?

[Charlotte Macdonald]

Now, Ryanair's press office says they're comparing their figures to the most recent association of European Airline Statistics, but that organisation hasn't published any new figures since 2009. And more to the point, that organisation has only 32 members. Over 200 airlines in Europe aren't members, including Ryanair itself. So I don't think that data really backs up their boast.

[Tim Harford]

Now, we were alerted to Ryanair's proud announcement by Fiona, a loyal more or less listener from Edinburgh in Scotland, who emailed us at more or lessbc.co.uk. she's also wondering how one would objectively measure being on time.

[Jim Paton]

I do wonder if they've simply designated such long flight times that it would

[Tim Harford]

be practically impossible for them to be late.

[Charlotte Macdonald]

Now, there is a term for this in the industry. It's called schedule padding, and it's been alleged not only in Europe, but in the United States as well. If you push back your scheduled arrival time, leaving extra time for the flight, you can still say you're on time. I asked Ryanair if it does this. It said no, and in this case, I'm going to defend them. I had a quick look at Fiona's home airport of Edinburgh to see how Ryanair flight times measured up to its rivals. I found eight routes where competitors were making the same journey, and in only one case did Ryanair give a longer flight time than its rivals.

[Tim Harford]

Ok, that could just mean, though, that all the airlines are exaggerating their flight times.

[Charlotte Macdonald]

Well, that may be true, Tim. And just to put this into perspective, today British Airways reports a flight time of 75 minutes from London to Amsterdam. In 1958, when it was using planes with propellers, it was taking 70 minutes to do the same journey. And Jim Paton told me that schedule padding is common for a number of reasons, including the fact that customers feel cheerful when the pilots tell them that despite a late departure, they're going to make up some time.

[Tim Harford]

Thank you, Charlotte. Earlier in the series, we brought you the story of a highly influential economics paper written by two Harvard professors which had helped shape the debate on austerity in America and Europe. And the story of the graduate student who'd spotted some problems with it.

[Graduate Student / Economist (unnamed)]

I immediately saw the spreadsheet error. They left out five countries and, you know, they say seeing is believing, but I almost didn't really believe my eyes.

[Tim Harford]

That's Thomas Herndon and the issues he and his professors found with growth in a time of debt, the paper by Carmen Reinhart and Ken Rogoff have become a cause celebre. But there are those who are warning that this paper is by no means the only one out there which has mistakes in it. Some, as in the case of Reinhart and Rogoff's paper, are due to honest error, others outright fraud. Here's Dr. Elizabeth Wager, a freelance publications consultant and a member of the advisory board for the UK Research Integrity Office.

[Jim Paton]

The best estimate that we have is that it's perhaps between 1 and 2% of researchers commit the most serious forms of misconduct. And some people argue, well, then 99% is fine. What are you worrying about? But actually, if you do the maths and look at the numbers, they're quite worrying. Researchers in the US publish around 300,000 articles in the peer reviewed literature every year. One percent of that would be 3,000 articles affected the situation.

[Tim Harford]

In one field of psychology, an area called social priming, has prompted a warning from Daniel Kahneman, a psychologist who also has a Nobel Memorial Prize in Economics. Kahneman wrote an open letter to those working in social priming after a number of attempts to replicate original findings had failed. Science writer Ed Yong tells the story of one such attempt.

[Ed Yong]

In the original study, volunteers saw a set of scrambled words and they had to create a sentence from them. Some of the words related to being old. The volunteers weren't consciously aware of that fact. According to the study, those who saw those words relating to old age after they left the room where they did the test, then walked more slowly down a corridor. So those words related to old age primed their behaviour and made them walk more slowly after they left that testing room. Now, this attempt to replicate that study made a couple of small changes, but notably it found that the people who saw those old age words did not in fact walk more slowly down a corridor. They only did so if the experimenters themselves expected them to do so.

[Tim Harford]

There's no suggestion that there was any fraud in the original study. It may just be that those conducting it were inadvertently themselves being influenced by the experiment. But what are the wider concerns about unreliable results based on some instances of fraudulent results have been uncovered recently. But Ed Yong says more subtle problems may be more widespread.

[Ed Yong]

One of the main issues in psychology at the moment is what's known as questionable research practices. These are fairly common ways of basically massaging data, not fabricating it in order to get positive results. So it might be things like carrying on your analysis until such point as you see significant results and then stopping or analyzing lots of different variables together until you find one that gives you a significant result and then only reporting that.

[Tim Harford]

And both of these, to be clear, the test isn't actually valid if you handle your data in this way.

[Ed Yong]

Absolutely. They're symptomatic of the pressure to publish or perish the incentives that scientists have to produce a lot of papers in high impact journals that produce novel and interesting findings. And that is the root of the problem. What is good for science is not necessarily what is good for individual scientists.

[Tim Harford]

Ed Yong, whose blog is called Not Exactly Rocket Science, Ed, points to the fact that the career concerns of individual scientists are pushing them to damage scientific endeavour in general. We've got the incentives wrong. I spoke to Betsy Stevenson, an economist at the University of Michigan in the United States. She told me that these perverse incentives mean that problems often go Unidentified Economists

[Graduate Student / Economist (unnamed)]

really like the person with new ideas who is inventive, creative coming up with a new thing, and somebody who's checking someone else's idea, who's validating or showing that someone else's idea is wrong and isn't the person with a new idea. And as a result, what graduate student wants to choose something where they know they'll never really excel?

[Tim Harford]

Now, does this matter to anyone other than academics? It should do. If you think academic work has any value and taxpayers do pay for chunks of it, after all, then surely catching fraudsters checking results and correcting errors is valuable too. And that's all we have time for, I'm afraid. And in case you were wondering where our long awaited dogulator had got to, it broke. We called the IT Help desk. They said they'd fix it shortly. And it turns out that shortly in IT Help desk years means in a fortnight. But the dogulator is now fixed and you can calculate your dog's age in Human Years on bbcworldservice.com moreorless until next week. Goodbye.