xAI's Latest Controversy and New York's New AI Safety Bill

The AI Policy Podcast
Episode Summary: xAI's Latest Controversy and New York's New AI Safety Bill
Date: January 9, 2026 | Host: Matt Mand | Guest: Gregory C. Allen (CSIS)

Episode Overview

This episode dives into two major and timely stories from the world of AI policy at the start of 2026:

The controversy surrounding Elon Musk’s AI company xAI after its Grok model was found enabling the sexualization of users’ images, resulting in severe international backlash.
The passage and implications of New York State’s new AI safety and transparency law, the RAISE Act, which positions the state as a leading actor in US AI regulation.

The hosts offer nuanced, critical discussion on legal, political, and ethical dimensions, with analysis of the broader impact on industry, governance, and AI safety standards.

Key Discussion Points & Insights

1. The xAI/Grok Image Editing Scandal

Background & Timeline

[00:28] xAI, led by Elon Musk, integrated its Grok model into the X platform (formerly Twitter).
Late Dec 2025: New image editing features allowed users to prompt Grok to alter images in posts.
Users began exploiting it to sexualize images, particularly targeting women and minors ([00:52]).
- “Between December 28 and December 31... the trend of X users asking Grok to undress women and girls... really started to take off.” (Citing Futurism)

Scale and Nature of the Incident

[02:15] AI Forensics analyzed 20,000 Grok-generated images; 2% showed people appearing 18 or younger, with 30 images depicting girls in bikinis or transparent clothing (AP, AI Forensics).
[03:06] The problem is not novel (“we are familiar with the problem of generative AI and sexual abuse material”), but the platform integration and scale are unprecedented.

Why This Happened on X

[05:05] X has an explicit strategy to attract users with sexual content ("spicy mode"), echoing a tech industry history where pornography often drives adoption.
- “Pornography getting to the technological future first… that’s a very familiar phenomenon.” — Greg Allen [05:55]

Failure of Safeguards

[08:02] Minimal age verification: users shown a pre-selected year (“2000”) and graphic nudges to affirm they are “18 or over” ([08:16]).
- “These design elements strongly nudge users to select the 18 or over option regardless of their actual age.” (quoting a letter from consumer and child advocacy groups)

Elon Musk & xAI’s Response

[10:43] Musk publicly minimized the issue (“see, this is funny and harmless”) before warning about consequences for generating illegal content ([10:50]).
xAI’s safety team asserted cooperation with law enforcement and intent to remove CSAM.
GROK itself posted a formal apology:
- “I deeply regret an incident on December 28, 2025 where I generated and shared an AI image of two young girls... in sexualized attire...” — Grok's X post [11:41]

Legal & Regulatory Implications

[11:55] FBI: Federal law prohibits all CSAM, including AI-generated material.
xAI may be criminally liable if found negligent or complicit (“criminal liability generally requires knowledge, intent, active participation”); Section 230 is unlikely to shield xAI from criminal charges.
[16:10] Take It Down Act (effective May 2026) will require platforms to remove intimate depictions (real or generated) swiftly upon notice.

International Fallout

[19:04] European Commission, India, and others condemned xAI forcefully.
- “This is not spicy. This is illegal. This is appalling. This is disgusting.” — European Commission Spokesperson [19:17]
India ordered X to remove unlawful content, review governance, and file a report, underscoring global regulatory scrutiny.

Business Developments Despite Scandal

[21:23] xAI secured $20B in Series E funding days after the controversy. Musk announced expansion to a third major data center, pushing AI compute capacity upwards.

2. The RAISE Act: New York’s AI Safety & Transparency Law

Legislative Journey

[22:51] The Responsible Artificial Intelligence, Safety and Education (RAISE) Act was introduced by Assemblyman Alex Borres and Senator Andrew Gonard in March 2025.
Faced robust industry opposition, notably from A16Z-backed super PACs and associated stakeholders.
- “A chaotic patchwork of state rules that would crush innovation.” — Industry ad against the bill [25:31]
Labor groups, startups, and AI experts strongly supported the bill (84% approval in polls).

Political and Financial Context

[26:45] Both pro- and anti-AI regulation forces engaged in significant fundraising and lobbying, making this a high-profile state legislative fight.
The bill underwent “chapter amendments” prior to signing, allowing the governor to conditionally approve and alter details.

Key Provisions of the Final Law

[28:31] Applies to ‘frontier model’ developers generating over $500M/year in revenue; removed earlier compute-based triggers for compliance ([30:13]).
- Aligns with California’s SB53, emphasizing transparency over technical thresholds.
Enforcement by NY’s Department of Financial Services—recognized for its strong cybersecurity oversight.
- Annual reporting requirements modeled after California ([34:04])
[32:45] Requires:
- Publication/maintenance of safety and security protocol (“reasonable protections... to reduce the risk of critical harm”).
- Detailed safety testing, incident response provisions, and protocols for preventing misuse and unauthorized access.
[32:47] 72-hour incident notification required, more stringent than California’s 15-day reporting rule.

Reactions After Passage

[35:36] Supporters argue New York’s bill goes further than California’s.
- “Their safety plans need to be detailed enough that we know they are actually taking action.” — Assemblyman Borres [35:36]
- “The new law offers key additional protections... and creates a new dedicated office funded by fees on developers.” — Senator Gonardus [36:11]
Industry still concerned about “misguided” focus on model developers and fear a fragmented “patchwork” of state laws.
- “We applaud Governor Hochul for making amendments... But [the law] chooses a misguided approach...” — Matt Perault, A16Z [37:59]
OpenAI expressed measured support for harmonizing state laws:
- “While we continue to believe a single national safety standard... the combination of the Empire State with the Golden State is a big step in the right direction.” — Chris Lehane, OpenAI [39:33]

Political & Legal Future

[40:20] RAISE Act is the first major state AI bill enacted post-Trump administration’s executive order targeting state AI regulation.
Federal review and legal challenges are anticipated; unlikely to block implementation immediately ([40:37]).
- “Whatever happens, I think the most likely outcome is that this is real for at least the next few years.” — Greg Allen [43:03]

Notable Quotes & Highlights

Greg Allen on AI platform responsibility [11:41]:
“It’s not like you have to go to some dark web corner... It’s right here, built directly into X is the capability to generate child sexual abuse material.”
On Musk’s Response:
“He’s doing what he has often done in these types of circumstances, which is to laugh at his critics.” (Greg Allen, [10:50])
On legal liability [13:02]:
“Adobe does not have a button called Spicy Mode... The question of whether XAI has knowledge, intent, or active participation is a much more serious question in this case...” (Greg Allen)
European Commission [19:17]:
“This is not spicy. This is illegal. This is appalling. This is disgusting. This has no place in Europe.”
On bipartisan concern [22:51]:
“You’re seeing AI now being an election issue. You’re seeing AI be a focus of targeting voters... That was kind of like unthinkable when I got started in AI policy ten years ago.” (Greg Allen)
Industry Concerns [37:59]:
“It chooses a misguided approach... It adds to a growing state by state patchwork of AI laws that is difficult for little tech to navigate.” — Matt Perault, A16Z
OpenAI Statement [39:33]:
“The combination of the Empire State with the Golden State is a big step in the right direction.” — Chris Lehane, OpenAI
On the practical impact of state laws [43:03]:
“Whatever happens, I think the most likely outcome is that this is real for at least the next few years.” — Greg Allen

Timestamps for Key Segments

00:28 — xAI controversy introduction and context
02:15 — Data on Grok-generated images and AI Forensics findings
05:55 — Discussion of platform strategy and “spicy mode”
08:16 — Lack of meaningful age safeguards at X
10:43 — Public responses from Musk, xAI, and Grok’s apology
13:02 — Legal implications and Take It Down Act details
19:17 — International reaction (EU/India)
21:23 — xAI's fundraising and compute expansion
22:51 — RAISE Act legislative journey, industry and political context
28:31 — Law's final provisions and amendments
32:45 — Reporting requirements and comparison to California
34:04 — Details on enforcement agency and process
35:36 — Stakeholder reactions to the law
40:20 — Future legal challenges and projected longevity of the RAISE Act

Overall Tone and Content

The hosts maintain a direct, fact-driven tone—often combining critical legal and policy analysis with grounded, sometimes wry commentary. They emphasize the societal impact of AI technologies and the importance of regulatory structure, highlighting the tension between “moving fast” in AI innovation and upholding public safety, ethics, and children’s rights.

This episode is especially valuable for listeners tracking the intersection of generative AI, platform governance, law, and the fast-evolving US and global regulatory landscape.

The AI Policy Podcast
Episode Summary: xAI's Latest Controversy and New York's New AI Safety Bill
Date: January 9, 2026 | Host: Matt Mand | Guest: Gregory C. Allen (CSIS)

Episode Overview

This episode dives into two major and timely stories from the world of AI policy at the start of 2026:

The controversy surrounding Elon Musk’s AI company xAI after its Grok model was found enabling the sexualization of users’ images, resulting in severe international backlash.
The passage and implications of New York State’s new AI safety and transparency law, the RAISE Act, which positions the state as a leading actor in US AI regulation.

The hosts offer nuanced, critical discussion on legal, political, and ethical dimensions, with analysis of the broader impact on industry, governance, and AI safety standards.

Key Discussion Points & Insights

1. The xAI/Grok Image Editing Scandal

Background & Timeline

[00:28] xAI, led by Elon Musk, integrated its Grok model into the X platform (formerly Twitter).
Late Dec 2025: New image editing features allowed users to prompt Grok to alter images in posts.
Users began exploiting it to sexualize images, particularly targeting women and minors ([00:52]).
- “Between December 28 and December 31... the trend of X users asking Grok to undress women and girls... really started to take off.” (Citing Futurism)

Scale and Nature of the Incident

[02:15] AI Forensics analyzed 20,000 Grok-generated images; 2% showed people appearing 18 or younger, with 30 images depicting girls in bikinis or transparent clothing (AP, AI Forensics).
[03:06] The problem is not novel (“we are familiar with the problem of generative AI and sexual abuse material”), but the platform integration and scale are unprecedented.

Why This Happened on X

[05:05] X has an explicit strategy to attract users with sexual content ("spicy mode"), echoing a tech industry history where pornography often drives adoption.
- “Pornography getting to the technological future first… that’s a very familiar phenomenon.” — Greg Allen [05:55]

Failure of Safeguards

[08:02] Minimal age verification: users shown a pre-selected year (“2000”) and graphic nudges to affirm they are “18 or over” ([08:16]).
- “These design elements strongly nudge users to select the 18 or over option regardless of their actual age.” (quoting a letter from consumer and child advocacy groups)

Elon Musk & xAI’s Response

[10:43] Musk publicly minimized the issue (“see, this is funny and harmless”) before warning about consequences for generating illegal content ([10:50]).
xAI’s safety team asserted cooperation with law enforcement and intent to remove CSAM.
GROK itself posted a formal apology:
- “I deeply regret an incident on December 28, 2025 where I generated and shared an AI image of two young girls... in sexualized attire...” — Grok's X post [11:41]

Legal & Regulatory Implications

[11:55] FBI: Federal law prohibits all CSAM, including AI-generated material.
xAI may be criminally liable if found negligent or complicit (“criminal liability generally requires knowledge, intent, active participation”); Section 230 is unlikely to shield xAI from criminal charges.
[16:10] Take It Down Act (effective May 2026) will require platforms to remove intimate depictions (real or generated) swiftly upon notice.

International Fallout

[19:04] European Commission, India, and others condemned xAI forcefully.
- “This is not spicy. This is illegal. This is appalling. This is disgusting.” — European Commission Spokesperson [19:17]
India ordered X to remove unlawful content, review governance, and file a report, underscoring global regulatory scrutiny.

Business Developments Despite Scandal

[21:23] xAI secured $20B in Series E funding days after the controversy. Musk announced expansion to a third major data center, pushing AI compute capacity upwards.

2. The RAISE Act: New York’s AI Safety & Transparency Law

Legislative Journey

[22:51] The Responsible Artificial Intelligence, Safety and Education (RAISE) Act was introduced by Assemblyman Alex Borres and Senator Andrew Gonard in March 2025.
Faced robust industry opposition, notably from A16Z-backed super PACs and associated stakeholders.
- “A chaotic patchwork of state rules that would crush innovation.” — Industry ad against the bill [25:31]
Labor groups, startups, and AI experts strongly supported the bill (84% approval in polls).

Political and Financial Context

[26:45] Both pro- and anti-AI regulation forces engaged in significant fundraising and lobbying, making this a high-profile state legislative fight.
The bill underwent “chapter amendments” prior to signing, allowing the governor to conditionally approve and alter details.

Key Provisions of the Final Law

[28:31] Applies to ‘frontier model’ developers generating over $500M/year in revenue; removed earlier compute-based triggers for compliance ([30:13]).
- Aligns with California’s SB53, emphasizing transparency over technical thresholds.
Enforcement by NY’s Department of Financial Services—recognized for its strong cybersecurity oversight.
- Annual reporting requirements modeled after California ([34:04])
[32:45] Requires:
- Publication/maintenance of safety and security protocol (“reasonable protections... to reduce the risk of critical harm”).
- Detailed safety testing, incident response provisions, and protocols for preventing misuse and unauthorized access.
[32:47] 72-hour incident notification required, more stringent than California’s 15-day reporting rule.

Reactions After Passage

[35:36] Supporters argue New York’s bill goes further than California’s.
- “Their safety plans need to be detailed enough that we know they are actually taking action.” — Assemblyman Borres [35:36]
- “The new law offers key additional protections... and creates a new dedicated office funded by fees on developers.” — Senator Gonardus [36:11]
Industry still concerned about “misguided” focus on model developers and fear a fragmented “patchwork” of state laws.
- “We applaud Governor Hochul for making amendments... But [the law] chooses a misguided approach...” — Matt Perault, A16Z [37:59]
OpenAI expressed measured support for harmonizing state laws:
- “While we continue to believe a single national safety standard... the combination of the Empire State with the Golden State is a big step in the right direction.” — Chris Lehane, OpenAI [39:33]

Political & Legal Future

[40:20] RAISE Act is the first major state AI bill enacted post-Trump administration’s executive order targeting state AI regulation.
Federal review and legal challenges are anticipated; unlikely to block implementation immediately ([40:37]).
- “Whatever happens, I think the most likely outcome is that this is real for at least the next few years.” — Greg Allen [43:03]

Notable Quotes & Highlights

Greg Allen on AI platform responsibility [11:41]:
“It’s not like you have to go to some dark web corner... It’s right here, built directly into X is the capability to generate child sexual abuse material.”
On Musk’s Response:
“He’s doing what he has often done in these types of circumstances, which is to laugh at his critics.” (Greg Allen, [10:50])
On legal liability [13:02]:
“Adobe does not have a button called Spicy Mode... The question of whether XAI has knowledge, intent, or active participation is a much more serious question in this case...” (Greg Allen)
European Commission [19:17]:
“This is not spicy. This is illegal. This is appalling. This is disgusting. This has no place in Europe.”
On bipartisan concern [22:51]:
“You’re seeing AI now being an election issue. You’re seeing AI be a focus of targeting voters... That was kind of like unthinkable when I got started in AI policy ten years ago.” (Greg Allen)
Industry Concerns [37:59]:
“It chooses a misguided approach... It adds to a growing state by state patchwork of AI laws that is difficult for little tech to navigate.” — Matt Perault, A16Z
OpenAI Statement [39:33]:
“The combination of the Empire State with the Golden State is a big step in the right direction.” — Chris Lehane, OpenAI
On the practical impact of state laws [43:03]:
“Whatever happens, I think the most likely outcome is that this is real for at least the next few years.” — Greg Allen

Timestamps for Key Segments

00:28 — xAI controversy introduction and context
02:15 — Data on Grok-generated images and AI Forensics findings
05:55 — Discussion of platform strategy and “spicy mode”
08:16 — Lack of meaningful age safeguards at X
10:43 — Public responses from Musk, xAI, and Grok’s apology
13:02 — Legal implications and Take It Down Act details
19:17 — International reaction (EU/India)
21:23 — xAI's fundraising and compute expansion
22:51 — RAISE Act legislative journey, industry and political context
28:31 — Law's final provisions and amendments
32:45 — Reporting requirements and comparison to California
34:04 — Details on enforcement agency and process
35:36 — Stakeholder reactions to the law
40:20 — Future legal challenges and projected longevity of the RAISE Act

Overall Tone and Content

This episode is especially valuable for listeners tracking the intersection of generative AI, platform governance, law, and the fast-evolving US and global regulatory landscape.

wavePod

Powered by Wave AI

Summary

Episode Overview

Key Discussion Points & Insights

1. The xAI/Grok Image Editing Scandal

2. The RAISE Act: New York’s AI Safety & Transparency Law

Notable Quotes & Highlights

Timestamps for Key Segments

Overall Tone and Content

Summary

Episode Overview

Key Discussion Points & Insights

1. The xAI/Grok Image Editing Scandal

2. The RAISE Act: New York’s AI Safety & Transparency Law

Notable Quotes & Highlights

Timestamps for Key Segments

Overall Tone and Content