Are refugees more likely to commit crime?

Summary

More or Less (BBC Radio 4) – Are refugees more likely to commit crime?

Date: May 20, 2026
Host: Tim Harford

---

Episode Overview

This episode tackles controversial and widely shared statistics claiming asylum seekers – specifically in Dorset – commit a disproportionate share of sex offences, critically examines the reliability of crime data by nationality or immigration status, and explores the broader public perception that refugees or asylum seekers are more likely to commit crime. Along the way, the More or Less team also corrects prior reporting errors on unrelated stats and investigates sensational claims from television and polling.

The episode exemplifies More or Less’s mission: cutting through misinformation, carefully checking numbers circulating online and in media, and holding themselves—and national treasures—accountable for numerical errors.

---

Main Segments & Key Discussion Points

1. The "44% of Dorset Sex Offences by Asylum Seekers" Claim

[01:42 – 07:42]

• Annunziata Rees-Mogg claimed on social media: "Asylum seekers make up 0.8% of Dorset’s population and 44% of alleged sex offences."
• Investigation by More or Less:
  • The 0.8% refers only to asylum seekers in hotels, not all asylum seekers.
  • Dorset Police FOI data showed one person from migrant hotels charged with a sexual offence in the relevant timeframe; updated stats said eight investigated, while 808 sex offences were reported in Bournemouth – so ~1% investigated, not 44% charged.
  • Data on sex offences by non-hotel-resident asylum seekers not available; crime stats generally aren’t broken out by immigration status.
  • Dorset Police statement (06:41): "These are not Dorset Police statistics and the information is incorrect." – Richard Varden, Dorset Police.
• AI compounding misinformation:
  • Annunziata Rees-Mogg later tweeted an apology, blaming AI for her error.
  • Elon Musk’s Grok AI incorrectly confirmed the “44%” statistic as valid, citing fabricated data.

Notable quote:

“So really not 44%? More like 1%.”
— Lizzie McNeil [03:38]

“The 44% figure as originally presented is misleading and inaccurate in that framing.”
— Grok AI bot (under Tim’s questioning) [07:08]

---

2. Can Crime by Immigration Status Be Reliably Measured?

[07:42 – 11:36]

• Guest: Madeleine Sumption, Migration Observatory, University of Oxford.
• Discussion:
  • UK crime statistics rarely record immigration status; convictions generally only categorized by nationality, not refugee/asylum status.
  • Some nationalities with higher proportions of asylum seekers (e.g., Afghanistan, Iran, Iraq) tend to have higher conviction rates, including for sexual offences, compared to the UK average – but as they’re a tiny share of the population, the absolute number of offences is small.
  • Demographic context: asylum seekers are more likely to be young men – a group statistically at higher risk of criminal offending for any nationality.

Notable quote:

“We do not [have good crime statistics for asylum seekers]. One reason for that actually is that most crimes never lead to a conviction and so the perpetrator isn't known and therefore the status of the perpetrator.”
— Madeleine Sumption [08:29]

“A fairly small number of people responsible for a fairly small number of offences, but nevertheless, it's not outrageous to say that some of them are overrepresented.”
— Tim Harford [11:26]

---

3. Mistakes & Corrections: Fish, Faith, and Famous Presenters

A. Fish Protection Cost Calculations [13:16 – 17:06]

• More or Less corrects a previous report suggesting Hinkley C nuclear power station was spending £250,000 per fish saved; in reality, that calculation only applied to endangered fish, not total fish, and used contested cost and benefit figures.
• Clarification: The high per-fish number refers only to endangered species; in total, millions of fish (mostly small/non-endangered) are saved, making the per-fish cost tiny.
• Richard Varden: “Which is why the Fingleton report focuses on the impact on endangered fish. However, what we said in the original item was misleading and in the spirit of more or less, we hold up our hands and we admit it when we're wrong.” [16:39]

B. The "Quiet Revival" in Churchgoing – Faulty Polling [18:50 – 24:26]

• YouGov retracted claimed “churchgoing revival” among young men, after it was found that fraudulent (and potentially bot-generated) responses skewed the poll.
• Discussed inherent weaknesses of "non-probability" online polling panels, especially for hard-to-reach demographic groups like young men; potential infiltration by AI-generated fake responses.
• “Always be sceptical about interesting poll results on the views of young men.” — Tim Harford [24:05]
• New polling with improved anti-fraud measures is planned.

---

4. AI’s Growing Role in Polling and Survey Fraud

[24:26 – 26:40]

• Expert: Annette Jaeckler, University of Essex.
  • Describes "agentic AI" capable of autonomously filling out surveys, mimicking not just answer patterns and reading speed, but even mouse movements and typos.
  • Bodes ill for the future reliability of online polls without enhanced anti-bot detection.

Notable quote:

"It's a real arms race... between the bot developers trying to outsmart the survey researchers in developing methods of detecting this kind of fraudulent survey completion."
— Annette Jaeckler [26:18]

---

5. Fact-Checking National Treasures:

A. David Attenborough and the Blue Tit Caterpillar Claim

[27:49 – 30:18]

• Claim from “Secret Garden” TV series: “Each blue tit chick requires 20,000 caterpillars to fledge.”
• More or Less checks with RSPB scientist Dr. Malcolm Burgess:
  • Real number: ~1,800–2,000 caterpillars per chick (assuming 90 per day for 20 days); with 10 chicks per nest, the 20,000 figure is closer to what’s needed for a full nest, not a single chick.
• BBC published a correction after More or Less raised the error.
• Tim's tongue-in-cheek on the perils of challenging beloved figures:
  • “Wait, we're fact checking Sir David Attenborough, the national treasure who just celebrated his hundredth birthday. Are you trying to get me fired?” [27:57]

---

Notable Quotes & Memorable Moments (with Timestamps)

• “So really not 44%? More like 1%.”
  — Lizzie McNeil [03:38]

• “These are not Dorset Police statistics and the information is incorrect.”
  — Richard Varden (Dorset Police, via Lizzie McNeil) [04:41]

• “Did the error spread because GROK said it was true?”
  — Tim Harford [07:14]
  “Yes, some people did cite my earlier responses … to prove the claim was true. That's a fair point. And I take responsibility for the imprecision in those initial replies.”
  — Grok AI [07:29]

• “We do not [have good crime statistics for asylum seekers].”
  — Madeleine Sumption [08:29]

• “There are lots of differences. So people seeking asylum are more likely to be men, and men of any nationality are much, much more likely to be convicted of offences than women.”
  — Madeleine Sumption [11:52]

• “What we said in the original item was misleading and in the spirit of more or less, we hold up our hands and we admit it when we're wrong.”
  — Richard Varden [16:39]

• “We have good reason to believe there's a lot of academic research that's gone on over the last few years on this, that these panels have increasingly been infiltrated by bogus respondents.”
  — Tom Coles [22:03]

• “With agentic AI, what you can do is you can ask the AI tool to do things for you independently … it can even mimic mouse movements on a web page.”
  — Annette Jaeckler [24:55–25:25]

---

Important Timestamps & Segments

• [01:42–07:42] — Breaking down and debunking the “44% sex offences” claim in Dorset
• [07:42–11:36] — The limits and challenges of UK crime data, and the demographic context
• [13:16–17:06] — Corrections: fish protection cost stats, why the “per fish” cost story is misleading
• [18:50–24:26] — Churchgoing “Quiet Revival” polling error; AI and fraudulent response risk
• [24:26–26:40] — Agentic AI and the future threat to polling validity
• [27:49–30:18] — Fact-checking the blue tit caterpillar claim (Attenborough correction)

---

Tone & Style

• Rigorous, skeptical, but never cynical.
• Willing and eager to fact-check even claims from “national treasures” and to admit More or Less’s own mistakes.
• Playful jabs at fearlessly going where the facts lead, and lighthearted banter among the team.

---

Summary

This episode delivers a careful, forensic dismantling of viral statistics about refugees and crime; highlights the persistent problem of bad data and misinformation, sometimes amplified by AI; corrects public misconceptions (even when originated by celebrated presenters); and retains a self-aware, transparent, and wittily British style. For anyone wanting to understand:

• Why the “44%” statistic about asylum seekers and sex crimes is flatly wrong,
• The truth about measuring crime rates among refugees,
• The pitfalls of modern online polling, and
• The value of healthy skepticism (even regarding Attenborough and churchgoing trends),

— this episode is as thorough as it is engaging.

Transcript

[Tim Harford]

This BBC podcast is supported by ads outside the uk.

[LinkedIn Ads Narrator]

Ever invest in something that seemed incredible at first but didn't live up to the hype? Like those $5 roses at a gas station? Or a secondhand piece of technology that breaks in the first 10 minutes? Marketers know that feeling. We optimize for the numbers that look great, impressions reach and reacts. But when they don't show revenue, well, that's a not so great conversation with the CFO. LinkedIn has a word for bullspend. Now you can invest in what looks good to your CFO. LinkedIn Ads generates the highest roas of all major ad networks. You'll reach the right buyers because you can target by company, industry, job title and more. So cut the bull. Spend. Advertise on LinkedIn, the network that works for you. Spend $250 on your first campaign on LinkedIn ads and get a 250 credit for the next one. Just go to LinkedIn.com Broadcast. That's LinkedIn.com Broadcast. Terms and conditions apply.

[Whole Foods Market Advertiser]

Save now at Whole Foods Market. It's the Summer Splash event with great everyday prices on 365 brand ground beef for the grill and ice cream for dessert. They have yellow sales signs on Ready to cook beef or chicken kebabs too. Level up with savory marinades, spices and rubs. And complete your cookout with a crowd pleasing cherry pie and their balsamic chicken salad available at the prepared foods counter. Get Summer Splash savings now at Whole Foods Market.

[Tim Harford]

Hello and welcome to More or less. We're your guide to the numbers all around us in the news and in life. And I'm Tim Harford. This week we fight misinformation wherever we find it, even if it does require us to take on the national treasure of all national treasures. And we examine further rather extraordinary developments on the story of the quiet revival in Christianity. But first, the columnist and former politician Annunziata Rees Mogg took to the social media platform formerly known as Twitter to post a claim about the proportion of sex offences in Dorset that are committed by asylum seekers.

[Lizzie McNeil]

Asylum seekers make up 0.8% of Dorset's population and 44% of alleged sex offences. So unbelievable. I had to check.

[Tim Harford]

And so did we. Lizzie McNeil has been on the case. Hello, Lizzie.

[Lizzie McNeil]

Hi, Tim. Well, first things first. This 0.8% of the population figure is specifically counting asylum seekers in asylum hotels now in Dorset. These are mainly in the Bournemouth area.

[Tim Harford]

Got it. And they're said to be committing 44% of the alleged sex offences. So where's that figure from?

[Lizzie McNeil]

Unclear. So, last year, Dorset police replied to a Freedom of Information request, asking them how many people living in migrant hotels in Dorset have been charged with a sexual offence. And the answer was one.

[Tim Harford]

One?

[Lizzie McNeil]

Yeah. That was the most recent data when the post was made. However, they then sent me an update to say that eight people living in asylum hotels have been investigated for sex crime from 1 January 2025 until 31 March 2026.

[Tim Harford]

So investigated for crimes is not the same as committing crimes, but hopefully there is some kind of connection. So how many sex crimes were reported in Bournemouth in that time?

[Lizzie McNeil]

808.

[Tim Harford]

Yikes. So really not 44%? More like 1%.

[Lizzie McNeil]

Yeah.

[Tim Harford]

Now, so far we've been talking about asylum seekers living in migrant hotels. So what about asylum seekers who don't live in hotels?

[Lizzie McNeil]

Well, the data isn't that helpful. Please don't break the crime statistics down by immigration status. We could instead look at crimes committed by British nationals versus foreign nationals.

[Tim Harford]

And what does that tell us?

[Lizzie McNeil]

Well, the most recent data from Dorset Police from 2024 and 2025 shows that 7% of all sexual offence crimes were committed by foreign nationals. If you add on people with dual British nationality, then that rises to 8%.

[Tim Harford]

Right. And these people are not necessarily asylum seekers, they're just people who aren't British.

[Lizzie McNeil]

Yeah. So these figures include people from Yemen, Afghanistan, Iran, as well as those from countries like New Zealand, Italy, Spain and Canada.

[Tim Harford]

Do we have any idea where the 44% figure came from?

[Lizzie McNeil]

Dorset Police don't. I asked them if they had any record of this figure and they responded,

[Richard Varden]

these are not Dorset Police statistics and the information is incorrect.

[Lizzie McNeil]

I also contacted Annunziata Rees Mogg, but she didn't respond to me. But she did later tweet an apology stating that she'd used AI to check the sources.

[Tim Harford]

Uh.

[Lizzie McNeil]

Oh, yeah. And even just after she posted, people were in her comments section asking Grok, which is Elon Musk's AI, whether Rees Mogg's statement was true and Grok said it was and cited police data.

[Grok AI]

Dorset Police's FOI disclosure confirms that 51 of 116 males awaiting prosecution for sexual offenses in the Bournemouth area are. Are asylum seekers from migrant hotels. Which calculates to about 44%.

[Tim Harford]

Right. Don't worry, Lizzie. I'll take it from here. Deep in the basement of New Broadcasting House, the more or less interrogation center. Bring in the suspect. The date is the 20th of May and the time is just after 9am I'm Tim Harford, with me in the room is the Grok AI. All answers are spoken by GROK in its own voice. Where did you get this stat? Out of 116 males awaiting prosecution for sexual offences in the Bournemouth area, 51. That's 44% were asylum seekers from migrant hotels.

[Grok AI]

The stat comes from a Dorset Police Freedom of Information FOI disclosure, specifically responses in their 2025 disclosure logs under categories like migrant sexual offenses and related migrant hotel queries.

[Tim Harford]

Funny, the FOI request does not contain the 116 males awaiting prosecution figure.

[Grok AI]

You're correct. The specific FOI request 01 FIN 253546, dated around September 2025, focuses on how many sexual offenses have been recorded resulting in charges of males residing in migrant hotels in Dorset. It does not directly state a county wide or Bournemouth wide total of 116 males awaiting prosecution for sexual offenses.

[Tim Harford]

Give me a link to a website that shows the 116 figure.

[Grok AI]

No single public webpage directly quotes the exact phrasing. 116 males awaiting prosecution for sexual offenses as a standalone official statistic from Dorset police.

[Tim Harford]

So the 44% figure is wrong?

[Grok AI]

Yes. The 44% figure as originally presented is misleading and inaccurate in that framing.

[Tim Harford]

Did the error spread because GROK said it was true?

[Grok AI]

No, the error did not spread because Grok said it was true. The misleading 44% framing originated and circulated widely before my response in that X thread.

[Tim Harford]

But people used GROK to prove it was true.

[Grok AI]

Yes, some people did cite my earlier responses, including the one you linked, to prove the claim was true. That's a fair point. And I take responsibility for the imprecision in those initial replies.

[Tim Harford]

Take em down boys. This whole story does raise another question. We know that the claim that Dorset asylum seekers are committing 44% of alleged sexual offences is wrong. We've got another figure about the number of foreign nationals committing sexual offences in Dorset, 7 or 8%. But without any more information, we don't really know what to make of that figure. So what can we say about the general sentiment here? The idea that asylum seekers are more likely to commit crimes than other groups. Who better to ask? But friend of the programme and Director of the Migration Observatory at Oxford University, Madeleine Sumption. Do we have good crime statistics for asylum seekers?

[Madeleine Sumption]

We do not. So we don't know how many crimes are committed by asylum seekers. One reason for that actually is that most crimes never lead to a conviction and so the perpetrator isn't known and therefore the status of the perpetrator. But where people are convicted. And we could, in theory, have good statistics in practice. The immigration status of whether someone's an asylum seeker or they're on a work visa or a student visa or anything else is not routinely recorded.

[Tim Harford]

Right. So even if they've been fined, they've been imprisoned, we don't necessarily know what their immigration status is.

[Madeleine Sumption]

That's correct. What we do have is some imperfect but vaguely serviceable data on cautions and convictions by nationality. And there are some nationalities where people are particularly likely to be asylum seekers. So people coming from, for example, Afghanistan, Syria, Iran, Eritrea. In most cases, they will have come either as a refugee, brought to the UK by the government or through the asylum system.

[Tim Harford]

Right. So we've got some sort of sense of how many Syrians are committing crimes, how many Eritreans are committing crimes, how many Germans are committing crimes, and we just have to kind of make an educated guess as to whether these people might be asylum seekers or not.

[Madeleine Sumption]

That's right.

[Tim Harford]

The. The Germans, probably not.

[Madeleine Sumption]

Exactly.

[Tim Harford]

So you said it was serviceable data. What does it tell us?

[Madeleine Sumption]

Well, I should say the data aren't routinely published. We have obtained data up to 2024 using Freedom of Information requests.

[Tim Harford]

You have to drag it out of them, is what you're saying.

[Madeleine Sumption]

Yes, indeed. And apparently there are plans to publish more of this officially in future. But based on what we have so far up to 2024, the overall picture is that we see the quite big variations in the rates at which people appear to be convicted of crimes by nationality. And it is true that some of those nationalities where most people tend to come through the asylum system, like Afghanistan, Iran, Iraq, for example, have higher rates of convictions overall. And also for sexual offences specifically. The challenge that we have is being precise about how high those rates are is really difficult because we don't have good data on how many people of each nationality are actually living in the population.

[Tim Harford]

So we don't know exact numbers, but we can estimate ballpark crime rates for different nationalities using data from the 2021 census and compare against the crime rate for the overall UK population. And that gives us a rough idea.

[Madeleine Sumption]

If you look at the top nationalities that are the sources of refugees to the uk, it does seem that when they're in the uk, the rates of convictions for sexual off offences are higher. So if they're overrepresented in those figures, they are, of course, a very small percentage of the overall UK population. So we're looking at probably a few percent of overall sexual offences convictions being committed by those top, say, 18 or so nationalities, it's a small share overall, but they're overrepresented because there aren't that many of them in the uk.

[Tim Harford]

Right, okay. So a fairly small number of people responsible for a fairly small number of offences, but nevertheless, it's not outrageous to say that some of them are overrepresented.

[Madeleine Sumption]

That's correct.

[Tim Harford]

Another question that arises is whether the demographic profile of people seeking asylum is different from the demographic profile of British residents. I'm thinking in terms of the gender and the age.

[Madeleine Sumption]

Yes. Yeah. There are lots of differences. So people seeking asylum are more likely to be men, and men of any nationality are much, much more likely to be convicted of offences than women. Those differences are really stark. We also see that people in general migrating to the uk, including in the asylum category, are in the younger age group, people in their 20s and 30s. And that's also where we tend to see higher rates of criminal offending in general. There are other risk factors for criminal offending in general, some of which also you would expect asylum seekers to be more exposed to. So, for example, being exposed to violence, a teenager and young adult having mental health problems, being socially isolated, all of these are things that in the research in general, we know are associated with higher rates of offending and they are also things that many asylum seekers will have experienced.

[Tim Harford]

Our thanks to Madeleine Sumption and thanks also to Lizzie McNeil. Oh, no. The More or Less Error. SIREN I can't believe we're only 10 minutes into the new series and already we've made a mistake.

[Richard Varden]

Actually, Tim, the error was in the last series.

[Tim Harford]

Ah, Richard Varden, editor of More or Less. Does your appearance in the studio mean it was your mistake?

[Richard Varden]

Oh, no. On More or Less, we are like the Cabinet, we take collective responsibility.

[Tim Harford]

Oh, really? Well, let's hear the grisly details in

[Richard Varden]

our Numbers of the Year programme. At the end of last year, one of the numbers involved the spiralling costs of infrastructure projects in the uk. In that section, we said that the Hinkley C nuclear power station involved hundreds of millions of pounds being spent on measures to protect fish.

[Tim Harford]

And is that wrong?

[Richard Varden]

No, that bit is correct. The problem is the next bit, when we say that the cost is something like a quarter of a million pounds per fish saved.

[Tim Harford]

So what should we have said?

[Richard Varden]

We should have said that if you look at the costs over 25 years, according to the highest estimates, it would cost something like a quarter of a million pound per endangered fish saved. But actually, it's a bit more complicated than that.

[Tim Harford]

Well, that's already quite Complicated. So go on then, remind us all where these numbers come from.

[Richard Varden]

The figures come from a government commissioned report on the regulation of the nuclear industry known as the Fingleton Review. In a case study in that report they say that edf, the company building the nuclear power plant, are going to spend £700 million on fish protection measures. The report also lists the number of different kinds of fish that would be saved by the protection measures every year. These numbers are very small things like 0.028, Seatrout 6, River Lamprey 18, Alice Shad. If you add them all up, assume a 25 year lifespan for the project and divide by the total cost, you get about a quarter of a million pounds per endangered fish.

[Tim Harford]

Right. So a few things to figure out there. First of all is £700 million definitely the cost of the fish protection measures?

[Richard Varden]

That's disputed. The nuclear power station is water cooled, which involves sucking in a lot of water from the Bristol Channel and then spitting it out again, killing lots of fish in the process. To lower the number killed, they are building three things, one of which has had a lot of media attention. An acoustic fish deterrent known of course as the fish disco. That's only going to cost 50 million pounds according to the Fingleton Review. Then there's a system of pipes so some of the fish that get sucked in can escape. Another 150 million and then 500 million for low velocity intake heads, which are basically like the nozzle on a vacuum cleaner sucking in the water. These are specially designed so they have lower suction and more fish can escape. But there's an argument about this last component because you'd need to build intake heads anyway. And it's unclear what proportion of the costs should be classified as fish protection. There's also an argument about whether all of this is enough. And it's entirely possible the costs and the timescale will change.

[Tim Harford]

What about the 25 year time scale?

[Richard Varden]

Well, that sounds like a good long term perspective, but in nuclear power station terms it isn't. The government is guaranteeing the price it pays EDF for energy from Hinkley C for 35 years. But EDF told us the 700 million includes the operating cost over the 60 year lifespan of the power station.

[Tim Harford]

And how about the number of fish being saved? Are the figures in the Fingleton review an exhaustive list of all the fish that will not be killed because of the protection measures?

[Richard Varden]

No. Fingleton focuses on endangered fish such as the sea trout Lamprey and Alice Shadow. If you look at all fish. You would get a much bigger number.

[Tim Harford]

Such as?

[Richard Varden]

There are lots of different estimates flying around, but it seems like Hinckley C might kill more than 100 tonnes of fish per year without any safety measures. And two of the protection measures without the fish disco might save more than half of them.

[Tim Harford]

That's quite a lot of fish.

[Richard Varden]

Yes, this equates to millions of mostly tiny fish. But to put that into context, a single medium sized commercial fishing trawler might catch hundreds of tons of fish each year. So one fishing trawler kills more fish than a nuclear power station. Which is why the Fingleton report focuses on the impact on endangered fish. However, what we said in the original item was misleading and in the spirit of more or less, we hold up our hands and we admit it when we're wrong.

[Tim Harford]

Quite right too. Thank you, Richard.

[LinkedIn Ads Narrator]

Ever invest in something that seemed incredible at first but didn't live up to the hype? Like those five dollar roses at a gas station? Or a second hand piece of technology that breaks in the first ten minutes? Marketers know that feeling. We optimize for the numbers that look great, impressions reach and reacts. But when they don't show revenue, well, that's a not so great conversation with the CFO. LinkedIn has a word for bullspend. Now you can invest in what looks good to your CFO. LinkedIn ads generates the highest ROAS of all major ad networks. You'll reach the right buyers because you can target by company, industry, job title and more. So cut the bull. Spend. Advertise on LinkedIn, the network that works for you. Spend $250 on your first campaign on LinkedIn ads and get a 250 credit for the next one. Just go to LinkedIn.com Broadcast. That's LinkedIn.com Broadcast. Terms and conditions apply.

[Whole Foods Market Advertiser]

Save now. At Whole Foods Market it's the Summer Splash event with great everyday prices on 365 brand ground beef for the grill and ice cream for dessert. They have yellow sales signs on ready to cook beef or chicken kebabs too. Level up with savory marinades, spices and rubs and complete your cookout with a crowd pleasing cherry pie and their balsamic chicken salad available at the prepared foods counter. Get Summer Splash savings now at Whole Foods Market.

[Tom Coles]

Oh no.

[Tim Harford]

What now?

[Tom Coles]

Don't worry Tim. That's the YouGov error siren.

[Tim Harford]

That's a relief. YouGov's error, not ours. It's Tom Coles, cat correspondent extraordinaire, occasional producer of the series. What have YouGov done.

[Tom Coles]

Well, remember we talked back in July last year about a report from the Bible Society called the Quiet Revival?

[Tim Harford]

This was the one that said there'd been a big increase in churchgoing in the uk, particularly among young men. And that wasn't reflected in the actual churchgoing data from the Anglican or the Catholic Churches. And it wasn't reflected in other polling data either.

[Tom Coles]

Yeah, that's the one. We raised an eyebrow, it's fair to say. And then polling godfather Sir John Curtis came on the program and he did the same.

[Richard Varden]

And?

[Tim Harford]

And I seem to remember that YouGov told us they had checked everything and that their result was entirely legit.

[Tom Coles]

Well, they've now gone back and found that it wasn't legit at all. They've retracted the survey and apologised, although they didn't want to come onto the program and tell us what happened.

[Tim Harford]

Right. So what did happen?

[Tom Coles]

In a statement they said that they looked again at the original survey data and found that specific demographic groups contained a number of respondents who they can now identify as fraudulent.

[Tim Harford]

They said some of the anti fraud

[Lizzie McNeil]

measures available in 2024 were not administered in the optimal way.

[Tim Harford]

Hmm. The Quiet Revival really became quite a thing in some Christian circles. So non optimal data processing is a bit of an understatement.

[Tom Coles]

Right. But it was welcome news to Professor David Vose, an emeritus professor at the UCL Social Research Institute, who told us the Quiet Revival wasn't plausible back in my first report.

I'm glad to be right. And it's unusual that something is retracted to demonstrate the correctness of one's initial skepticism. On the other hand, I do find it exasperating that it's taken so long, given that at the outset I was saying not only that the findings were very likely to be incorrect, but why they were likely to be wrong and what was behind the surprising results. And it turns out that it was indeed a matter of bogus responses on the survey.

[Tim Harford]

Yes, it is nice for once to actually get a clear answer on something. But Tom, I'm interested. Does the failure of this poll tell us anything important about polling in general?

[Tom Coles]

Maybe, yeah. The particular problem in this survey is one we've talked about before. The way non probability polls sometimes come up with very strange answers, particularly coming from from young men. Quick refresher on the terminology here. The best quality polling uses a probability sample where the polling company works out the people they need for a representative sample and then recruits them to take part. But the Quiet Revival survey used a non probability sample. YouGov recruits a panel of volunteers who sign up to do the surveys for a small financial reward, and then you invite the people you need to take part from that panel.

We have good reason to believe there's a lot of academic research that's gone on over the last few years on this, that these panels have increasingly been infiltrated by bogus respondents. Now, these could be real people who are just not who they claim to be. They might be real people who don't really care about the subject matter of the surveys and just racing through them as quickly as possible. And at least since 2022, they may well be AI chatbots that can be trained to simulate real people.

Now, when you recruit participants for a survey like the YouGov one, you have a particular problem with the demographic groups that don't volunteer in large numbers, one of which is young men.

If you're organising an army of AI chatbots where you happen to be some person sitting in a basement in India somewhere, trying to earn your 50p by doing a survey and doing as many as possible, what you will claim to be is a young man in Britain, probably in a big city. And I suspect that that's why we're getting such anomalous results, particularly in that age range and in that gender.

[Tim Harford]

And, Tom, it was young men that saw a big increase in their churchgoing in the very quiet revival survey that YouGov have now just admitted. Didn't actually catch a lot of bogus respondents.

[Tom Coles]

Indeed, YouGov haven't told us exactly which responses are likely fake, but David says it's quite possible that answers from bots pretending to be people could skew the result in this group upwards. Other surveys show the chances of young men going to church are very low, so they would answer a question about church going at the extreme end of the scale. But a bot might try to avoid the extreme ends of the scale to look more middle of the road and human, meaning they might push the average upwards.

[Tim Harford]

Ok. So always be sceptical about interesting poll results on the views of young men. Got it? Now, I couldn't help but notice a fleeting reference to AI there as one of the things that polling companies have to deal with. Tom, that feels to me like something that might be a very big problem coming down the tracks.

[Tom Coles]

Yeah, it does have that feel. I've been talking to Annette Jecla about that. She's a professor of survey methodology at the University of Essex. She told me that the AI problem is becoming really quite acute with the development of something called agentic AI. This is an AI tool that is quite different to the things like ChatGPT and Grok, where you type in a question and the LLM, that's the large language model, spits out the answers.

[Annette Jaeckler]

With agentic AI, what you can do is you can ask the AI tool to do things for you independently. So the AI tool would be able to operate a browser, would be able to click to navigate between pages, to submit the tool at the end. So whilst doing that, the LLM would itself be making decisions about what to do next and how to complete tasks.

[Tom Coles]

This makes it very difficult for polling companies to tell the difference between real people and agentic AI.

[Annette Jaeckler]

Not only can the AI take on a particular Persona and answer all the questions in a consistent way, including text questions, the agentic AI can also mimic human behavior. So it can mimic how long it would take a respondent to read and answer a question on a page. You know, that would be an indicator for a fraudulent survey if it's being answered too quickly, more quickly than would be humanly possible. But the AI can mimic that, or it can even mimic mouse movements on a web page. It can mimic responses, including typos, the kind of things, the mistakes that humans would make and correct. So the AI can mimic all of those kind of things. So there is evidence of this happening and it seems like a real arms race at the moment between the bot developers trying to outsmart the survey researchers in developing methods of detecting this kind of fraudulent survey completion.

[Tim Harford]

Long story short, in the age of AI, if you see a survey response that you can't triangulate with other data and trends, then you need to be very wary.

[Tom Coles]

Exactly. One more thing, Tim. When they retracted their survey, YouGov said they were going to run another quiet revival poll with all the anti fraud measures in place. This time it'll be interesting to see what that one finds.

[Tim Harford]

Yep, it's the gift that keeps on giving. Thank you, Tom. And thanks to David Vose and Annette Jaeckler. Yes, I'm afraid another klaxon, this time with a nature theme. I need to remind you at this point that we hold power to account on more or less, without fear or favour. No one is beyond scrutiny if our loyal listeners hear a stat that they doubt, we will follow wherever the story leads, no matter who it implicates. So it was when we got this email from Matthew Clark. I heard on the BBC Secret Garden TV program that it takes 20,000 caterpillars to feed one blue tit chick until it fledges. Blue tits can have several chicks so the numbers would multiply accordingly. I struggle to believe it's possible. Can you investigate, please? Investigation is what we do, Matthew. So our team went straight to the source. Who was it who made this eyebrow raising claim?

[Tom Coles]

To successfully fledge, each chick will require 20,000 caterpillars.

[Tim Harford]

It's a tall order that must be completed within three weeks. Wait, we're fact checking. Sir David Attenborough, the national treasure who just celebrated his hundredth birthday. Are you trying to get me fired?

[Tom Coles]

Without fear or favour, Tim. Without fear or favour.

[Tim Harford]

Well, who am I to question our production team, all of whose names are available in this programme's credits. If I go down, I'm taking you all down with me. So is it really right that each blue tit chick requires 20,000 caterpillars over the nearly three weeks that it's in the nest? Blue tit nests tend to have about 10 chicks in. That's 200,000 caterpillars for the two parents to find in three weeks. We spoke to Dr. Malcolm Burgess, principal conservation scientist at the RSPB, and more or less his resident blue tit consultant. He looked at the scientific literature on the topic and did some quick calculations for us. So each chick will be receiving about 90 caterpillars per day. The blue tip nestlings are in the nest for about 16 to 20 days. So say 20 days. 20 times 90 caterpillars per day gives about 1,800 caterpillars per chick. So nearly 2,000 caterpillars per chick. Much more feasible. But David Attenborough said 20,000. So what could have gone wrong? Well, remember that there's more than one chick in each blue tit nest. Typically, a nest would have up to 10 chicks in it. Could be more, could be less, but say on average 10. So 10 times 1,800 gives you 18,000 caterpillars. 18,000 caterpillars for the entire nest of 10 chicks. Just shy of Attenborough's 20,000 figure that he said was per chick. So that's the mistake here. They should have said 20,000 caterpillars per nest rather than per chick. After we got in touch with the makers of the Secret Garden documentary, the BBC published an official correction.

[LinkedIn Ads Narrator]

We have added a clarification strap at

[Tim Harford]

the start of the episode and apologise for the error in the narration. And who did that narration again?

[Tom Coles]

When caterpillars are in short supply, spiders will have to do.

[Tim Harford]

So we fact checked David Attenborough. Next on our National Treasure list is Paddington Bear. That's all we have time for this week, but please keep your questions and comments coming to more or lessbbc.co.uk because there's a whole series to get through and we couldn't do it without your ideas. Until next time. Goodbye. More or Less was presented by me, Tim Harford. The producer was Tom Coles with Nathan Gower, Josh McMinn and Lizzie McNeil. The production coordinator was Brenda Brown. The program was mixed by James Beard and our editor is Richard Varden.

[Madeleine Sumption]

If you've got a scrolling problem, then this is the podcast for you.

[Tim Harford]

It's called Top Comment with me, Matt

[Madeleine Sumption]

Shea and me, Mariana Spring. We both investigate social media for a living.

[Tim Harford]

Whether it's disinformation, conspiracy theories, Internet culture memes. We're going to be getting behind the

[Lizzie McNeil]

stuff that is popping up on your

[Tim Harford]

feed on this podcast. That's Top Comment on BBC Sounds.

[LinkedIn Ads Narrator]

Ever invest in something that seemed incredible at first but didn't live up to the hype? Like those five dollar roses at a gas station? Or a secondhand piece of technology that breaks in the first 10 minutes? Marketers know that feeling. We optimize for the numbers that look great, impressions reach and reacts. But when they don't show revenue, well, that's a not so great conversation with the CFO. LinkedIn has a word for that. Bullspend. Now you can invest in what looks good to your CFO. LinkedIn Ads generates the highest roas of all major ad networks. You'll reach the right buyers because you can target by company, industry, job title and more. So cut the bull. Spend. Advertise on LinkedIn, the network that works for you. Spend $250 on your first campaign on LinkedIn ads and get a 250 credit for the next one. Just go to LinkedIn.com broadcast that's LinkedIn.com broadcast. Terms and conditions apply.

[Lizzie McNeil]

Wait.

[Grok AI]

You don't need an hour to burn calories. TikTok has short workouts that get your heart rate up fast. Download now and get moving.