WS MoreOrLess: Does politics make us get our sums wrong?

Summary

---

More or Less: Does Politics Make Us Get Our Sums Wrong?

BBC World Service • Aired: November 9, 2013
Host: Ruth Alexander
Co-Host/Reporter: Ben Carter
Guest: Prof. Dan Kahan (Yale University)

---

EPISODE OVERVIEW

This episode of "More or Less" explores how political beliefs can interfere with our ability to interpret statistics and mathematical information accurately. The discussion centers on recent research showing that even people with high numeracy skills often misinterpret figures if the data contradicts their political or ideological leanings—a phenomenon the researchers call "motivated numeracy." The episode features an engaging quiz, in-depth explanation of the Yale study, and direct insights from lead researcher Dan Kahan.

---

KEY DISCUSSION POINTS AND INSIGHTS

1. The Skin Cream Scenario—A Neutral Numerical Puzzle

[00:28–02:13]

• Ben Carter presents Ruth Alexander and listeners with a basic numeracy challenge involving strawberry and banana skin creams, setting up a simple, neutral numerical comparison.
• Result:
  • Strawberry group: 200 improved, 100 worsened → 66% success rate
  • Banana group: 80 improved, 20 worsened → 80% success rate
• Quote:
  • "Well, 80% is bigger than 66%. So the banana cream wins." – Alexander [02:11]
• Takeaway:
  • It's crucial to consider proportions, not just raw numbers, despite gut reactions based on totals.

2. The Research Study: Numeracy vs. Bias

[02:22–03:21]

• Ben outlines Yale research where a similar numerical puzzle was posed to participants, who had been previously tested for numeracy skills.
• Outcome:
  • Only 4 in 10 got the correct answer overall.
• Prof. Dan Kahan:
  • "People who are relatively low in numeracy, this skill we measure separately... They're very unlikely to get the answer right. But people who are high in numeracy, they're very likely to get the answer right." [02:49]

3. Injecting Emotion/Identity: The Political Twist

[03:21–05:20]

• When the neutral puzzle is reframed to involve national identity (English vs. French women), Ruth jokingly picks "English," exposing instinctive bias.
• Key Point:
  • Even clever people are prone to letting opinions override reason, especially when a topic is emotive or identity-related.
• Quote:
  • "Clever people get stupider when they let their opinions get in the way." – Carter [03:48]

4. The Gun Control Experiment: Motivated Reasoning in Action

[04:17–06:21]

• The Yale study's main experiment recasts the problem with data about gun control, touching on a hot US political topic.
• Set-up: Participants evaluate crime statistics after a handgun ban, using EXACTLY the same numbers as the neutral test—but now outcomes challenge participants' political identities.
• Findings:
  • High-numeracy individuals solve the test correctly 70–90% of the time if results align with their ideology, but just 35% when results contradict their beliefs.
• Notable Quote:
  • “The people who were high in numeracy, they were about...70–90% likely to get it right, but only if...the data...supported the position consistent with their ideological predisposition. If [not], then only about a 35% chance..." – Prof. Dan Kahan [05:33]

5. How Politics ‘Turns Off’ Our Brains

[06:21–07:45]

• Why does this happen?
  • Kahan suggests people unconsciously use reasoning to defend their group identity, not to seek truth, especially about high-stakes political topics.
• Quote:
  • “Their critical reasoning has been disabled by the temptation of finding the politically correct result.” – Kahan [07:13]
• Underlying mechanism:
  • On political issues, high numeracy doesn’t grant immunity—everyone risks motivated reasoning.

6. Motivated Numeracy: A New Name for an Old Problem

[07:45–08:30]

• Motivated numeracy: People reason well until ideology triggers, then even the math-savvy slip.
• Kahan warns this is particularly dangerous when political positions serve as social identity badges.
• Quote:
  • “When positions on these issues become basically badges of membership and loyalty to groups, then people are going to apply their reason to get the positions that affirm that they belong to that group. That's a bad situation to have…” – Kahan [08:06]

7. Are the Right or Left More Biased?

[08:30–08:55]

• No significant difference: Both conservatives and liberals showed equal susceptibility to motivated numeracy.
• Ironically, even interpreting the study itself caused people to reaffirm their biases.
• Quote:
  • "People's political views are clouding their judgment when they look at this study, a study which is about how people's political views cloud their judgment." – Alexander [08:48]

---

QUOTABLE MOMENTS

• On human rationality:
  • "So politics makes us stupid then?" – Alexander [06:34]
• On the power of identity:
  • “…It's almost as if their critical reasoning has been disabled by the temptation of finding the politically correct result.” – Kahan [07:13]
• On the circularity of bias:
  • "That's another example of motivated numeracy. People's political views are clouding their judgment when they look at this study..." – Alexander [08:43]

---

IMPORTANT SEGMENTS (with Timestamps)

• Introducing the neutral numeracy puzzle: [00:28–02:13]
• How numeracy impacts accuracy on neutral tasks: [02:22–03:21]
• Demonstrating bias with the nationality twist: [03:21–04:06]
• The gun control scenario and motivated numeracy findings: [04:17–06:21]
• Kahan on why numbers don’t always help: [06:21–07:45]
• What is motivated numeracy and why it happens: [07:45–08:30]
• No ideological immunity: Right vs. Left: [08:30–08:55]

---

EPISODE TAKEAWAYS

• Political identity can profoundly impair our reasoning—even among people with strong numerical skills.
• On neutral topics, high numeracy leads to accurate answers, but on "identity-laden" topics, bias can override ability.
• Both sides of the political spectrum are equally vulnerable.
• This phenomenon, called “motivated numeracy,” warns us to be vigilant when interpreting statistics in political debates.

---

Hosts: Ruth Alexander & Ben Carter
Guest: Prof. Dan Kahan (Yale University)

Transcript

[A]

This is the short edition of More or Less, first broadcast on the BBC World Service.

[B]

Thank you for downloading from the BBC. For details of our complete range of podcasts and our terms of use, go to bbcworldservice.com podcasts

[A]

hello and welcome to More or Less on the BBC World Service. I'm Ruth Alexander and Ben Carter's also

[B]

here with a question for you and the listeners.

[A]

Oh, go on then.

[B]

You might want to take a pen and jot down some notes.

[A]

Okay. That kind of question. Right. Okay.

[B]

Okay, imagine this. Medical researchers have developed two new creams for treating skin rashes. One made out of strawberries, one made out of bananas.

[A]

Alright, unusual.

[B]

Okay, we need to test how these creams work. So we have two groups of people who all have a skin rash. One group is given the strawberry cream, the other group is given the banana cream. Okay, now let's look at how the two groups of people got on with their skin creams. In the group who used strawberry cream, 200 people got better and 100 got worse.

[A]

Okay, 200 better, 100 worse in the strawberry group. Yep, yep.

[B]

And in the group who used banana cream, 80 got better and 20 got worse.

[A]

80 better, 20 worse in the banana group.

[B]

Okay, so Ruth, which cream is better, strawberry or banana?

[A]

Okay, well, so 200 people got better using the strawberry cream.

[B]

Yep.

[A]

Compared to 80 who got better using the banana cream. So at first sight, the strawberry cream looks a winner. But then what about the number of people who got worse?

[B]

So it's 100 strawberry cream people got worse compared to 20 banana cream people.

[A]

Okay, then let's look at what proportion of people got better and worse in each group and then compare them. So the strawberry cream group, 200 people got better, 100 people got worse. So that means two thirds got better or 66%. And Banana Cream Group, 80 people got better, 20 worse. So. Well, that means that four fifths, or 80% of people got better. Well, 80% is bigger than 66%. So the Banana cream wins.

[B]

Correct. You've got it. Even though there were a bigger number of people getting better in the strawberry

[A]

group, there was a bigger proportion of people getting better in the banana group.

[B]

Right. Now, why am I talking talking to you about this?

[A]

I have no idea.

[B]

Well, this question, or a very similar one, has been put to people by researchers at Yale University in the United States. People whose numeracy they'd already tested, by the way. Now, this question is a bit tricky, and the version of the question they asked was even trickier. Dan Kahan, the Professor behind the experiment says on average, just four in 10 people got the question right.

[C]

It's difficult. And so people who are relatively low in numeracy, this skill we measure separately. They. They're very unlikely to get the answer right. But people who are high in numeracy, they're very likely to get the answer right. And the skin rash cream, that's what we found. Version of the problem. That's what we found.

[B]

So the people who scored well in other number tests the researchers had devised for them scored the best in this test. Not surprising, really. But that's not where the story ends.

[A]

Oh, a twist. I love a twist. Go on.

[B]

Well, the researchers then asked another question. So before I explain that, let me ask you another question, Ruth.

[A]

Okay.

[B]

Two groups take a test. A group of English women and a group of French women. 200 of the English women pass and 100 fail. 80 of the French women pass and 20 fail. So who are the cleverest, English or French women?

[A]

Oh, obviously the English.

[B]

Okay, so you made no attempt to think through the numbers at all. You just voiced your opinion and you are wrong. Now, obviously you're just joking around. But this is exactly what the researchers discovered. People do. Clever people get stupider when they let their opinions get in the way when they're faced with the reasoning test. Their judgment becomes impaired if they feel strongly about the subject.

[A]

So we let our opinions cloud our judgment. Well, we've all experienced this, I suppose, mainly observing it in other people, of course.

[B]

Of course. But what's new here is that the Yale team think they've actually measured this bias.

[A]

Oh, so they can see it. They've measured it. They can see it happening in the data wall. How have they done that?

[B]

Well, they asked people another question. This time it wasn't a neutral question about skin cream. It was a question about something the people really cared about, gun crime. The scenario the researchers outlined was a city government was trying to decide whether to pass a law banning private citizens from carrying concealed handguns in public. They were unsure whether the new law would decrease crime by reducing the number of people carrying handguns or increase crime by making it harder for law abiding citizens to defend themselves.

[A]

And, okay, so this is touching on a big political issue in the US if you're a liberal, you think guns are a huge problem for society. If you're from the right, you think guns keep people safe.

[B]

Exactly. So now they ask people to compare what happened with cities that brought in a gun ban and what happened in those cities where they didn't Bring in the ban. What was really smart about what they did was that they used the same figures as in the skin cream test. Sometimes those figures suggested crime increased, but sometimes it actually implied it decreased.

[A]

Okay, I see. So they were trying to work out what would happen to people's maths ability when the data suggested a result that clashed with their political opinion. So what happened then?

[B]

Here's Dan Kahn again.

[C]

The people who were high in numeracy, they were about somewhere between 70 and 90% likely to get it right, but only if, in fact, the the data, properly interpreted, supported the position consistent with their ideological predisposition. If they saw numbers that were inconsistent with their ideological predisposition, then there's only about a 35% chance that they're going to get it right. So those are numbers comparable to what you're seeing in the skin rash treatment when you're comparing high numeracy and low numeracy people. So all of a sudden the high numeracy people, they're not any smarter or they're not doing any better than their low numeracy counterparts if the data in the political version of the problem is something that would disappoint their ideological predispositions.

[A]

So the people who were good at maths got the question right more than 70% of the time when the numbers agreed with their political views and only 35% of the time when the numbers went against what they believed. So politics makes us stupid then?

[B]

Well, yes. And this result is exactly what Dan Kahan had suspected people do.

[C]

Our hypothesis was that people who know that they're supposed to use this more conscious, deductive way of thinking about the problem, if they see an answer that heuristically seems very appealing, they just look at it and they say, hey, look, crime is going down. That's what I expected, because they're looking at the treatment condition, gun control, man condition, and they're seeing that if that's what they want to see, they're kind of lulled into it by the heuristic. They stop thinking as much as they would if they were doing something that wasn't political. So, you know, in effect, it's almost as if their critical reasoning has been disabled by the temptation of finding the politically correct result.

[A]

So this explains why arguments and debate over political and social issues sometimes persist, even in the face of clear, incontrovertible scientific evidence. If it doesn't suit your worldview, you may not interpret the data properly.

[B]

And Kahan thinks that's exactly what he's found evidence of he calls it motivated numeracy.

[C]

There's something strange about these problems that are political in nature that effectively disables the people who are high in numeracy. So you're not getting the benefit. Those individuals aren't getting the benefit of their high numeracy. And the people who listen to them, they would be misled in the gun control ban. We should be worried that there are certain kinds of problems that are depriving us of the benefit. When positions on these issues become basically badges of membership and loyalty to groups, then people are going to apply their reason to get the positions that affirm that they belong to that group. That's a bad situation to have, and we should figure out why that happens and prevent it.

[A]

So the big question then, who's the most biased? People on the right or people on the left?

[B]

Well, some people looked at the data and said that it shows that the right wing are more biased. However, Dan Kahan says there's no evidence of that at all.

[A]

So that's another example of motivated numeracy. People's political views are clouding their judgment when they look at this study, a study which is about how people's political views cloud their judgment.

[B]

That's right.

[A]

I love it. Okay, well, Ben Carter, thanks very much. Ben was talking to Dan Kahan. If there are any numbers you'd like us to investigate or explain, do please email us. Our address is more or lessbc.co.uk and you can also sign up for a free download of this program at our website, bbcworldservice.com more or less.

[B]

There are dozens of different podcasts now available from the BBC, including news, documentaries, science, business, arts and sports. The details of them all go to bbcworldservice.com podcasts.