Decoder with Nilay Patel (Guest Host: Hayden Field)
Episode: How AI safety took a backseat to military money
Date: September 25, 2025
Overview
In this episode of Decoder, guest host Hayden Field (senior AI reporter at The Verge) speaks with Heidi Klaff, Chief AI Scientist at the AI Now Institute and former OpenAI systems safety engineer. Together, they explore the dramatic pivot among leading AI companies from championing ethics and safety to aggressively pursuing military contracts. They discuss the relaxation of bans on military AI applications by companies like OpenAI and Anthropic, the motivations and risks behind this shift, and what it means for AI safety both technically and politically—especially as these models become entangled with high-risk and high-stakes defense operations.
Key Discussion Points & Insights
1. Rapid Shift in AI Company Policies on Military Use
(04:16–07:27)
- OpenAI and Anthropic drop military use bans: Both had long-standing bans on military and warfare applications, but rolled them back in early 2024 in anticipation of major Defense Department contracts.
- "Clean slate" approach: Companies have not publicly addressed their prior warnings about LLMs being unsafe for military use, instead reframing their missions to align national security and AI safety.
“It was almost like a clean slate was being created where they behaved as if this was always aligned with their mission.”
—Heidi Klaff (06:11)
- Financial and strategic incentives: The pivot is aimed at securing government subsidies and lucrative contracts amid the unprofitable reality of current generative AI business models.
- National security narrative: Companies adopt the rhetoric of a US–China arms race to justify rapid AI deployment while sidestepping critical safety and security scrutiny.
“Now their reliance on this narrative of a US China AI arms race ... allows them to sort of avoid safety and security scrutiny within military applications.”
—Heidi Klaff (07:15)
2. Risks in Military Adoption of Commercial Foundation Models
(07:27–10:00)
- Senator Warren’s concerns about XAI’s DoD contract: Raised the issue that XAI, and similar companies, lack adequate safety audits before deploying to military contexts.
- Expanded risk surface: Use of commercial models in military settings expands attack vectors and threatens national security due to inadequate vetting and traceability.
- Data security: Military applications risk exposing sensitive personal or private data used to train these models, which could be compromised or manipulated by adversaries.
“Commercial models are unvetted. They don’t have a supply chain that follows the typical military supply chain, and they can be compromised in a lot of ways.”
—Heidi Klaff (08:17)
3. Military Contracts as a Financial Lifeline and Regulatory Workaround
(10:00–15:25)
- Financial imperative: Pre-profit AI companies chase military contracts for revenue and perceived legitimacy.
- Bypassing standards: Traditional military procurement standards—rigorous, multi-year, safety-focused—are not met by commercial generative AI models.
- Laxer evaluation: Newer deals push military systems built on commercial models that are much less robust or secure than previous defense technologies.
“The thing with AI systems ... is they do not meet the sort of very basic threshold that’s typically expected for a military system.”
—Heidi Klaff (11:35)
- Explanation of military procurement: Normally requires air-gapped, traceable supply chains and extensive testing. Recent shortcuts for AI highlight an erosion of these standards.
4. Are “Gov” AI Products More Secure? Not really.
(18:54–22:50)
- Air-gapping and fine-tuning aren’t enough: While government models may be somewhat more isolated, their foundational risks remain due to training on large, public datasets.
- “Sleeper agents” and prompt attacks: Models could be compromised at the training stage, with adversarial data poisoning leading to unpredictable responses later.
“A lot of research has shown that … you can implement what’s called a sleeper agent, which is given a specific prompt or a command. It will then behave in a harmful way … based on something that was implemented in the training data.”
—Heidi Klaff (20:32)
- The myth of control: Once the military takes delivery, neither the vendor’s terms of service nor their ethical promises reliably determine how the technology is used.
5. Ethical Dilemmas and Loss of Vendor Control
(22:50–25:52)
- Post-sale, no oversight: Vendors lose all control, and militaries are not bound by vendors' terms of service or intended use policies.
- Ambiguity of “direct harm”: Even non-lethal tools (e.g., information sorting) can assist targeting and surveillance, blurring lines about what constitutes “harm.”
- Procurement is not commercial contracting: Unlike familiar software sales, military procurement involves deep integration with state power and confidential, unaccountable use cases.
“Militaries do not follow terms of service ... this is not how military procurement works, period.”
—Heidi Klaff (24:24)
6. The CBRN (Chemical, Biological, Radiological, and Nuclear) Hype
(25:52–28:49)
- Little evidence for CBRN-capable AI—so far: No proof that current models can autonomously develop or deploy such weapons, though training on sensitive data could enable future dangers.
- The real concern: Military use of AI with low accuracy in high-stakes domains (e.g., nuclear command and control) is deeply irresponsible.
“If you’re thinking about nuclear command and control … it certainly shouldn’t be AI systems, because ... these systems are highly flawed and they’re always going to have inaccuracy.”
—Heidi Klaff (27:19)
- Projection vs. reality: Companies and regulators focus on hypothetical existential threats, sometimes at the expense of addressing immediate operational risks and misuse.
7. Redefining “AI Safety”—A Convenient Erosion of Standards
(30:34–41:32)
- Technical safety defined historically: In traditional domains, “safety” means protecting humans and the environment from harm—even in worst-case scenarios.
- “Safety revisionism” by AI labs: The meaning of safety is redrawn to mean “alignment,” “existential risk,” or vague generalities—departing from the established, technical, sector-specific understanding.
“[AI companies] use the same safety terminology ... but instead redefine those safety techniques with washdown alternatives that actually accelerate the deployment of inaccurate AI in high risk scenarios.”
—Heidi Klaff (32:00)
- Delegitimizing democratic control: By letting companies dictate risk thresholds and definitions, society loses time-tested, democratically determined standards for what is “safe enough.”
- General vs. specific safety: Military and safety-critical domains require tailored, highly reliable solutions—not generalized or “one-size-fits-all” safety frameworks favored by big AI vendors.
8. Why Current AI Safety Efforts Fall Short
(39:38–43:56)
- Risk assessment frameworks co-opted and misapplied: Originally designed as a step toward better regulation, such frameworks are now misused as catch-all justifications for broad AI deployment.
- Excessive focus on unproven hypothetical “existential” risks diverts attention from ongoing, measurable harms.
- Regulation at risk: If regulation focuses on future hypotheticals rather than current dangers, then “you might as well not have regulation at all.”
“If you’re not able to mitigate for the lack of safety and security of AI models today, then you have no chance of mitigating again these hypothetical risks that people like to bring up.”
—Heidi Klaff (43:07)
Notable Quotes & Memorable Moments
-
On the abrupt pivot to military work:
“It was almost like a clean slate was being created … they behaved as if this was always aligned with their mission.”
—Heidi Klaff (06:11) -
On compromised military-grade AI:
“Commercial models are already compromised from the day they’re built because they’re based on public data.”
—Heidi Klaff (21:18) -
On vendor control after sale to military:
“Militaries do not follow terms of service … this is not how military procurement works, period.”
—Heidi Klaff (24:24) -
On the myth of existential risk dominating real regulation:
“If regulation focuses on future hypotheticals rather than current dangers, then ‘you might as well not have regulation at all.’”
—Heidi Klaff (42:30) -
On safety standards in traditional critical infrastructure vs. AI:
“It even goes up to 99.999% ... if you allow zero risk, we’re never going to build anything ... with technological systems, there is some risk, but you have to mitigate for when systems fail.”
—Heidi Klaff (36:14–36:19)
Important Timestamps for Segments
- [04:16–07:27] — Industry shift to military applications
- [07:27–10:00] — XAI and Security Concerns of Military AI Contracting
- [10:00–15:25] — Military procurement, standards, and commercial AI limitations
- [18:54–22:50] — Are “government” AI models more secure?
- [22:50–25:52] — Vendor ethical statements vs. military oversight reality
- [25:52–28:49] — The hype and reality of CBRN weapon risks
- [30:34–41:32] — Redefinition of “AI safety,” loss of democratic risk threshold setting
- [39:38–43:56] — Misapplication and inadequacy of current risk frameworks; real vs. hypothetical risk
Tone and Takeaways
The conversation is clear, well-informed, and critical. Heidi Klaff challenges prevailing narratives from both AI companies and governments, emphasizing the seriousness of deploying untested AI models in defense while pointing out how the meaning of “safety” has been quietly revised to suit profit and policy expediency. The episode ends on a note of caution: true safety must be measured, verifiable, and rooted in established engineering practices—not redefined on the fly for financial or political motives.