Podcast Summary: Choiceology with Katy Milkman

Episode: To Know What You Know
Guests: David Dunning & Andrew Flack
Date: August 26, 2024
Host: Katy Milkman

Main Theme

This episode explores the pitfalls of overestimating one’s own knowledge—a phenomenon captured by the Dunning-Kruger Effect. Through real-life stories and expert interviews, host Katy Milkman uncovers how blind spots in self-awareness can lead to poor decisions, public gaffes, and even viral moments. The episode illustrates these concepts using both the infamous art restoration mishap of Cecilia Jimenez and insightful commentary from psychologist David Dunning.

Key Discussion Points and Insights

1. Destructive Confidence: The Ecce Homo Story

Timestamps: 00:10 – 13:00

• Story Overview:
  The episode opens with a tale from Silicon Valley, where criminals were outsmarted by the very GPS trackers they stole—introducing the theme of overconfident decisions without the requisite knowledge.
• The Cecilia Jimenez Incident:
  • Cecilia Jimenez, an amateur painter in Borja, Spain, attempted to restore a deteriorating fresco of Jesus (Ecce Homo) despite lacking proper training and materials.
  • Her restoration went awry, resulting in a comical, widely-mocked version that became famous online and inspired memes around the world ([04:23]).
  • Andrew Flack, librettist for the comic opera “Behold the Man,” shares that Jimenez’s intentions were pure—she loved the painting and wanted to help ([06:57]).
  • The event highlights how sincere overconfidence can lead to public mistakes and social consequences.
• Unintended Benefits:
  • The botched restoration, though initially humiliating, brought international attention and tourism to Borja—more than 300,000 visitors—helping revive the local economy ([12:33]).
  • “She had ruined something that wasn’t really valuable in the first place. But the resulting tourism and...mania...people take pilgrimages to see this amazing little small fresco...” — Andrew Flack ([11:51]).

2. Understanding the Dunning-Kruger Effect

Timestamps: 13:45 – 23:14

• Definition and Description:
  • David Dunning, co-creator of the effect, explains:
    • “Those who don’t have expertise don’t have the expertise to recognize that they don’t have expertise. They think they’re doing just fine.” — David Dunning ([14:21])
• Natural Occurrences:
  • The effect is found in everyday life and critical decisions—first-time home buying, marriage, and even voting.
    • “We all think we’re doing just fine when we don’t have expertise, because we don’t have the knowledge or the skill to recognize we could be doing better and that we’re making mistakes.” — David Dunning ([14:21])
• Research Studies:
  • Classic lab study: Students overestimated their performance on tests, even with financial incentives to be accurate. No one won the prize for accuracy ([15:48]).
  • Medical student assessment: Poor-performing students significantly overestimated their grades, while high-performers slightly underestimated themselves ([16:39]).
• Difference from General Overconfidence:
  • Dunning distinguishes their effect from other forms:
    • “The Dunning Kruger effect is just a specific version...if you’re less skilled, you’re less skilled at knowing how skilled you are.” — David Dunning ([17:33])
• Root Causes:
  • The inability to self-assess comes from lacking the very skills needed for accurate self-evaluation ([18:58]).
  • “To know how well you’re doing... you have to have skill in the first place. To know if you’re a good chess player, you have to be a good chess player already. If you’re not, you won’t know.” — David Dunning ([19:06])
• Personal Reflection:
  • Dunning describes how his research changed his approach: he now seeks more advice and prepares more extensively for consequential decisions, advocating confidence only after careful preparation ([22:29]).

3. Practical Implications and Takeaways

Timestamps: 23:16 – 25:53

• The Classic “Confidence vs. Knowledge” Curve:
  • As knowledge increases, confidence often dips (“valley of despair”) and only rises again with real expertise ([23:20]).
  • Milkman emphasizes: “The more you know, the more you know you don’t know.”
• Humility as a Solution:
  • Urges listeners to add “a dash of humility” when tackling new or challenging tasks.
  • Advice: Add a generous buffer to time and effort estimates for unfamiliar projects, and seek expert advice before diving into tasks outside one’s domain ([24:58]).
    • “If you’re not a professional artist, maybe ask an expert or two if they agree it’s a good idea before volunteering to repair that portrait...”
  • Famous quote cited: “To know what you know and what you do not know, that is true knowledge.” — Confucius ([25:40])

Notable Quotes

• On intention vs. outcome:
  “She didn’t do this on purpose. This wasn’t a willful act. Something went wrong here…” — Andrew Flack ([04:23])
• Defining the Dunning Kruger Effect:
  “Those who don’t have expertise don’t have the expertise to recognize that they don’t have expertise.” — David Dunning ([14:21])
• On gaining insight:
  “To know how well you’re doing... you have to have skill in the first place.” — David Dunning ([19:06])
• Personal practice:
  “When it comes time to do something, be confident. Absolutely. But when you’re preparing, be a pessimist, be defensive. I’m a little bit more of a preparer than I used to be.” — David Dunning ([22:29])
• Katy Milkman’s Reflection:
  “Don’t trust yourself to accurately assess how hard something will be if it’s not in your wheelhouse. Add a little humility to your judgments just when you need it.” — Katy Milkman ([24:58])

Important Segment Timestamps

• 00:10 — Opening story: GPS theft and foiled criminals
• 02:14 — Introduction of Andrew Flack and the “Ecce Homo” story
• 14:17 — Introduction of David Dunning; defining the Dunning-Kruger Effect
• 15:48 — Key experiments and evidence for the effect
• 17:33 — Distinguishing Dunning-Kruger from general overconfidence
• 19:06 — Discussing the root cause and skill requirements
• 22:29 — Practical changes in Dunning’s personal life due to his findings
• 24:58 — Host’s actionable advice for listeners

Episode Tone & Style

The conversation is thoughtful and often light-hearted, particularly in recounting the comic opera about Cecilia Jimenez. Both host and guests use accessible language and vivid examples—making even complex behavioral science approachable and relevant.

Conclusion

This episode masterfully blends quirky real-world stories with scientific rigor to expose how easily any of us can fall into the trap of assuming we know more than we do. Through memorable anecdotes, clear explanations, and actionable advice, the podcast encourages listeners to question their confidence and approach unfamiliar tasks with humility, preparation, and a willingness to consult experts when needed.