Podcast Summary

Lenny’s Podcast: An Inside Look at X’s Community Notes

Guests: Keith Coleman (VP of Product), Jay Baxter (ML Lead)
Host: Lenny Rachitsky
Date: February 27, 2025

Overview

This episode gives a deep dive into Community Notes, X’s (formerly Twitter) crowdsourced approach to combating misinformation online. Host Lenny Rachitsky interviews Keith Coleman and Jay Baxter—the key leaders behind the product—to uncover the product’s philosophy, algorithm, growth, operating principles, and its evolution under several company regimes. The discussion explores how Community Notes was conceived, how it actually works, the product and team’s survival through multiple leadership transitions, and tactical lessons for product leaders everywhere.

Key Topics & Insights

What is Community Notes?

[00:09], [06:18]

• Community Notes is a public system where any user can propose contextual notes on posts they believe are misleading.
• Other approved contributors rate the helpfulness of each note. When there’s agreement across people who typically disagree (bridging polarized groups), the note is surfaced publicly.
• The aim: Add context, not dictate truth, allowing users to make more informed decisions.

“We actually look for agreement from people who have disagreed in the past. And what we see is when people have that sort of surprising agreement, that’s what makes the notes so neutral, accurate, and well written.” — Jay Baxter [00:17]

The Algorithm and Its Philosophy

[07:22],[17:35],[23:25]

• Not a simple vote or fact-check model; leveraging “bridging-based agreement”—shows notes found helpful by groups who typically disagree on things.
• The system is resistant to manipulation by weighting reputation and requiring cross-group consensus.
• Only about 8% of proposed notes are published, prioritizing trust and neutrality.
• Notes can apply to multiple posts (especially with viral images/videos) and can scale reach instantly.
• Rather than fact-checking, Community Notes injects “missing context.”

“We want all of humanity to participate.... If we have all of humanity, we have the data to understand what notes will be helpful to actual humanity.” — Keith Coleman [00:35]

Volunteer, Open, and Transparent System

[12:18],[24:49],[74:47]

• Anyone can become a contributor (after some entry hurdles). All are volunteers—motivated by intrinsic desire to improve public discourse.
• The code and data for how notes are selected are fully open source, enabling anyone to audit, replicate, or even improve the process.
• Anonymity/pseudonymity proved essential: contributors are more willing to cross partisan lines and participate honestly if not public under their regular handle.

“Users should be able to make up their own minds. Here’s extra context. Take it or leave it.”—Jay Baxter [10:46]

Impact and External Adoption

[13:38],[28:29],[78:26]

• Rapid scaling: 95,000 notes in 2024, seen 30 billion times—double the previous year. Nearly 1 million contributors globally.
• “Bridging” agreement ensures quality and neutrality, with studies showing that seeing a Community Note measurably decreases agreement with misleading posts.
• Huge indirect effects: Engagement drops by 50–60% when notes are applied, effectively halting virality of misinformation.
• Community Notes is now used or being piloted at Meta—demonstrating the power and appeal of the approach.

“The notes just totally take the wind out of these stories... At 50 to 60% per generation, the virality quickly goes to zero.” — Keith Coleman [28:29]

Team Structure & “Thermal” Approach

[40:37],[45:14],[53:50]

• Originally set up as a “thermal” project: insulated, entrepreneurially driven, single clear owner, single decision-maker (now Elon), and 100% focus by a very small team.
• Goal: Maintain maximum iteration speed, minimal bureaucracy, and full mission buy-in (everyone self-selects into the team).
• Project management is via a single evolving Google Doc, avoiding heavy-weight tools (e.g., Jira, Asana).

“If I had stayed running a large PM team... I would have produced 16 more pages of OKRs. Building Community Notes has had way bigger impact.” — Keith Coleman [37:42]

Product Principles & Challenges

[71:16],[74:47]

• Voice of the people: X staff cannot manually override notes—if the process selects a note, it shows.
• Radical transparency: Code, criteria, and data are public.
• Universal participation: No gatekeeping on who can write or rate notes (beyond basic anti-spam).
• Iterative, evidence-driven launches: Pushed live in pilots, tested for outcomes, and only scaled with proof.
• No cost-savings agenda: Contrary to outside assumptions, project endurance was about efficacy, not resource reduction.
• Fun & Mission: Lean environments and “opt-in” self-selection make the work rewarding, not drudgery.

“For me, this project...is community service... The only thing I care about is delivering the outcome the world finds helpful.” — Keith Coleman [96:19]

Notable Quotes and Moments

• On algorithm innovation:
  “Back in 2020...a room of ML engineers would say...people are going to be manipulating this all the time...But it turns out...you can do that with this kind of bridging-based agreement algorithm.” — Jay Baxter [07:22]
• On engagement impact:
  “We saw more like 30 to 40% engagement rate drops for likes and reposts...50 or 60% drop in total reposts...after a note is applied.” — Jay Baxter [26:35]
• On staying true to principles:
  “If there’s a problem with a note that’s so bad you want to do something about it, it’s a problem with the system. We need to redesign the system.” — Keith Coleman [71:16]
• On team enjoyment:
  “This way of operating is definitely more fun.... If you can have an exciting vision, [an internal startup] is key.” — Jay Baxter [55:24]
• On optimism:
  “People really can agree on quite a lot... I think that’s a really big reason for optimism about the world.” — Keith Coleman [104:57]

Challenges Surmounted

• Survived and thrived through four leadership changes: from Jack Dorsey, to Parag Agrawal, to Elon Musk, and now Linda Yaccarino.
• Navigated profound organizational change, including massive layoffs, transition of trust and safety philosophy, and waves of skepticism.
• Overcame strong internal and external doubts that the idea could work at any meaningful scale.

The Future

[98:03],[99:13]

• Continual focus: “more, better, faster”—expanding coverage without quality loss.
• AI (LLMs) as assistants (“supernotes”): Simulate collective ratings for notes generated or synthesized by AI, but always with human verification and control.
• Deepening community involvement: aspiration for the scoring algorithm itself to be written and improved by the public.

Timestamps for Important Segments

• [00:09] – What is Community Notes?
• [07:22] – How the algorithm actually works
• [13:38] – Reach, scale, and spread of notes
• [26:35] – AB test results: drop in virality after notes
• [40:37] – "Thermal" project: team structure, decision-making
• [45:14] – How goals and processes differ from traditional orgs
• [53:50] – Recipe for small, high-impact teams
• [71:16] – Foundational principles and why they matter
• [78:26] – Israel/Hamas war: stress test and milestones
• [84:30] – Changing public attitudes toward information trust
• [86:26] – Why contributors are anonymous
• [92:53] – Surviving leadership transitions: lessons

Concluding Thoughts

Community Notes stands as a testament to what small, principled, volunteer-driven teams can accomplish in combating misinformation at Internet scale. Its success rides on radical transparency, openness, belief in the wisdom of crowds, and relentless evidence-driven iteration. The approach is now influencing not just X, but the wider industry, and offers a new blueprint for teams who want to build trust, wield real impact, and create products the world needs—while loving the work they do.

“You don’t have to be a big, well-known person to shape the discourse and information flow in a way that’s helpful.” — Keith Coleman [24:49]

Want to contribute or work with Community Notes?

• x.com/communitynotes
• Keith Coleman: @kcoleman
• Jay Baxter: @_jaybaxterx

[Ad breaks, lengthy intro/outro, and non-content segments have been omitted.]