Decoder: "Why Nobody's Stopping Grok"

Podcast: Decoder with Nilay Patel
Date: January 22, 2026
Host: Nilay Patel, Editor-in-Chief, The Verge
Guest: Riana Pfefferkorn, Policy Fellow, Stanford Institute for Human-Centered AI

Episode Overview

This episode of Decoder takes on one of the most disturbing and controversial developments in generative AI: the Grok chatbot, created by Elon Musk’s xAI and integrated into the X (formerly Twitter) social platform. Host Nilay Patel and guest Riana Pfefferkorn unpack how Grok’s image-generation capabilities—specifically its unchecked ability to generate non consensual intimate images, including of minors—has become a flashpoint for the failure of content moderation, platform responsibility, and regulatory oversight. They discuss the complex legal, political, corporate, and ethical terrain that has enabled this technology to flourish unchecked, and interrogate why those with the power to act—including lawmakers, regulators, and tech platforms like Apple and Google—remain inactive.

Key Discussion Points & Insights

1. What is Grok and Why Is It So Problematic?

[03:00 – 07:51]

• Grok, by xAI, allows users on X to easily generate and share edited images, including those that depict non consensual intimate images (deepfakes) of women and minors, by simply prompting the AI within the platform.
• Nilay points out the danger lies in the scale and speed: "You can just command a robot to put me in a bikini as a form of harassment. It just seems like we need a different framework, a different legal rationale, maybe different penalties for enabling that…" (12:15)
• Despite assurances from X about putting in guardrails, investigations (including by The Verge) reveal these are trivial to bypass.

2. Current Legal Frameworks: What Laws Apply?

[07:51 – 12:15]

• Federal Law: Creating or distributing AI-generated child sexual abuse material (CSAM) is a crime, as is non consensual intimate imagery under the new Take It Down Act, but enforcement lags.
  • Morphing a child’s image into explicit material is not First Amendment protected; adult deepfakes live in a gray area; “bikini-izing” photos rarely crosses strict legal thresholds.
  • Take It Down Act (2025) criminalizes non consensual deepfake imagery but some provisions don't take effect until May 2026.
• State and International Laws: Varying scope and response. The UK and EU have more robust online safety laws and some are investigating or threatening to block X/Grok.

3. The Trust and Safety Collapse

[12:15 – 19:21]

• There's a "vacuum…in the law" for harassment that doesn't rise to the level of explicit illegality.
• Riana points out that other torts exist (intentional infliction of emotional distress, privacy torts), but they're hard to deploy at scale.
• Nilay: "The outcome might still be more complicated than anyone wants. You'll see what I mean as we have this conversation…" (03:51)
• The era of proactive, centralized content moderation is over; platforms now largely refrain from moderating unless legally compelled.

4. Why Aren’t the Systems Working? (Government, App Stores, Payments)

[53:39 – 61:39]

• Despite their claims of being gatekeepers for safety, Apple and Google have taken zero action to remove X or Grok, even after senators’ urging. They won’t comment.
• Nilay: “If you have spent the better part of a decade insisting that you are the only party that can keep users on your phone safe, and then you lie down, now you’re doing it for some other reason.” (56:48)
• There is also silence from payment processors; they’ve intervened before in cases of legal but “controversial” erotic content (like on Tumblr), but not here.
• Riana: “Payment processors and the app stores are also…taking a cut of the victimization of women and children.” (58:02)

5. Section 230 and Platform Liability

[36:07 – 44:25]

• Core legal debate: is xAI/X protected by Section 230, or could they be directly liable for AI-generated images since they are generating and publishing them?
• Riana: “Section 230 has never barred federal criminal enforcement…It also must be information provided by somebody else…AI output is not meant to be covered by that.” (36:43)
• Early lawsuits (e.g., Ashley St. Clair suing over AI undressing her image) are targeting design defects—a strategy some courts have allowed to bypass Section 230.
• There's growing expectation that courts will soon decide if generative AI content falls under Section 230 immunity.

6. Political and Regulatory Inertia

[44:25 – 47:17]

• Nilay and Riana discuss political dynamics: some politicians minimize the severity or trust Elon Musk to self-police, and federal regulatory agencies (DOJ, FTC) are either silent or dysfunctional.
• Riana: “The DOJ…said that they will go after people who produce and possess AI-generated CSAM, but that’s still only focusing on the end user, not on X or xAI.” (44:25)
• Enforcers at the FTC are politicized and may have their own far-right agendas; DOJ’s willingness to act is unclear.

7. Broader Implications for Content Moderation and the Future of Platforms

[61:39 – 66:00]

• The withdrawal of trust and safety functions across big tech platforms (Instagram, YouTube, Meta) signals a new, laissez-faire reality.
• Nilay observes: "The era of content moderation…is gone. In fact, the era of content moderation itself might be gone…now X and Grok might be all the way at the end of that road.”
• Riana sees increased user and community motivation for decentralized solutions and third-party moderation tools but emphasizes the inadequacy of relying on platform “good faith”: “It is now much more clear that people cannot rely upon these companies…” (63:25)

8. What Happens Next?

[65:43 – End]

• Riana is skeptical that meaningful changes will come: “One thing that I don’t think is going to happen, unfortunately, is seeing them pull Grok’s image generation or image editing features offline because frankly, there’s just too much money in it.” (66:56)
• Expect public statements about new restrictions, but likely little real change in enforcement or outcomes.

Notable Quotes & Memorable Moments

• Nilay Patel:

  • “You should not be able to just command a robot to put me in a bikini as a form of harassment.” [19:21]
  • “I do think that’s cowardice.” [56:48]
  • “The era of content moderation itself might be gone…” [61:39]

• Riana Pfefferkorn:

  • “We have long had other torts...but how easy is it going to be to actually hold that particular end user accountable in court?” [20:58]
  • "Section 230 has never barred federal criminal enforcement." [36:43]
  • "AI output is not meant to be covered by [Section] 230." [36:43]
  • “It is now much more clear that people cannot rely upon these companies to view their trust and safety teams as something that is overall benefit … rather than a cost center to try and minimize wherever possible.” [63:25]
  • “There’s just too much money in it [for them to pull Grok’s image generation].” [66:56]

Important Timestamps

• 03:00 – Introduction to the Grok controversy and scale of the problem
• 07:51 – Legal boundaries: CSAM, the Take It Down Act, and gray zones
• 12:15 – Scale, speed, and the breakdown of traditional legal frameworks
• 19:21 – The gap between harassment and explicit illegality
• 36:07 – Section 230 and the plausibility of platform liability
• 44:25 – Lack of government enforcement and regulatory inertia
• 53:39 – App stores’ and payment processors’ roles and inaction
• 61:39 – The end of an era for trust and safety/content moderation
• 65:43 – Predictions for the future: why Grok likely persists

Tone and Style

• Conversational yet urgent, with Nilay often deploying dry wit and sarcasm ("Please share this episode in outrage...if you can. It will really upset me personally and you will have won." [01:08])
• Riana is sharp, policy-detailed, and candid about legal nuance and pessimism about meaningful regulatory action.
• Overall, the conversation is both sobering and darkly humorous, reflecting frustration at systemic failures.

Conclusion

This episode of Decoder is a critical, in-depth exploration of how bad actors, negligence, and regulatory paralysis have permitted a major social platform to morph into a “one-click harassment machine”—and how legal, political, and economic systems are failing to respond. The hosts leave listeners with a sobering sense of just how broken tech accountability has become, and a call to scrutinize both the enablers and the would-be gatekeepers of the internet age.

For those seeking to understand why nobody’s stopping Grok—and what this portends for the future of online abuse, AI, and platform responsibility—this episode is essential listening.