Last Week in AI – Episode #212 (June 17, 2025)

Podcast: Last Week in AI
Hosts: Andrei Karpathy and Jeremy Harris Gladstone

Overview

This episode covers two weeks’ worth of fast-moving AI news, exploring major updates across tools, business, open source research, policy, chips, and synthetic media. While lacking a single blockbuster headline, the hosts unpack impactful developments—from OpenAI’s O3 Pro and sweeping model price drops to new security risks in AI agents, evolving labor market dynamics, and significant legal challenges around generative media. Their discussion remains lively, witty, and deeply informed for both technical and general audiences.

Tools & Apps

OpenAI Launches O3 Pro, Massive Price Drop

(04:46 - 08:02)

• O3 Pro: A new reasoning-focused LLM, replacing O1 Pro, now available to ChatGPT users.
• Price slashed by 80%: Input tokens now $2 per million (down from $8), a “huge price drop” (Andrei, 04:46).
• Performance: O3 Pro benchmarks show broad improvements, sometimes beating human and previous AI results.
• Notable Benchmark: “Basically you see a clean sweep where the model 64% of the time is preferred to humans…spanning everything from…personal writing and computer programming and data analysis.” (Jeremy, 05:52)
• 4 out of 4 reliability: OpenAI now reports strict test metrics where a model must correctly answer a question four times out of four—“I hadn’t noticed, to my embarrassment… Of course they’re doing this, but I hadn’t yet remembered seeing it in writing.” (Jeremy, 05:52)

Cursor AI Editor Hits 1.0

(09:27 - 12:43)

• Cursor: Powerful AI-enhanced coding environment achieves a significant milestone with version 1.0.
  • Launches BogBot (automated PR reviewer) and Background Agents (remote, agentic code workers).
  • Agentic coding: Agents now function asynchronously and autonomously.
• Security Concerns: “Agents have a much bigger surface area of attacks… if you’re deploying this in a production setting, this is a really interesting new set of vulnerabilities…” (Jeremy, 10:52)
  • Microsoft also reported a prompt-injection vulnerability in Copilot the same week.

Mistral’s Open-Source Reasoning Models & Market Position

(13:04 - 15:44)

• Mistral (French AI lab) releases Magistral models (24B parameter “small” is fully open source).
• Generally not state-of-the-art but fills an open-source need.
• Analysis: “The fact that they did release this suggests they don’t have a plan for blowing things out of the water anytime soon.” (Jeremy, 14:11)

ElevenLabs V3 – State-of-the-Art Multilingual Text-to-Speech

(15:45 - 18:54)

• V3 model: More natural, expressive voices (e.g., laughter, sighs), supports 70+ languages.
• Programmable cues: Developers can embed [happily], [shouts], etc., into prompts—“It seems obvious in retrospect, but somebody had to think of it and implement it.” (Jeremy, 17:10)

ByteDance CDance 1.0 & Google Veo – Video Generation Heats Up

(18:54 - 23:40)

• ByteDance (TikTok parent) launches CDance 1.0 to compete with Google’s viral Veo.
• CDance: 5s of HD video generated in 40 seconds; praised for handling complex sequences, character consistency.
• Google’s Veo Pro plan: $20/month, faster generations; “I continue to tap the sign that someday fairly soon we’re going to be able to generate 1s of video for each second that you wait.” (Jeremy, 23:40)
• Video generators will soon support real-time, interactive experiences—a “very dark rabbit hole” for media feedback and personal optimization.

Business & Industry

AI Talent Migration: Anthropic’s Winning the War

(25:41 - 31:40)

• Signal Fire report: OpenAI employees leaving for Anthropic at 8:1; DeepMind at 11:1 rates.
• Culture trumps pay: “I’ve never had a conversation that feels like that [tense, secretive] with an Anthropic employee.” (Jeremy, 25:41)
• Compensation: OpenAI counteroffers include $2M retention bonuses and $20M equity increases.
• Entry-level jobs vanishing: “We’re no longer hiring entry-level software engineers. We don’t expect ever to do that again.” (Jeremy, 32:32)
  • Senior talent only; AI is writing the majority of major labs’ code bases.
  • White-collar automation is accelerating.

OpenAI – New Court Order Calls for Retaining All Chat Logs

(34:33 - 36:36)

• Legal standoff: Court orders OpenAI to retain all user logs, including deleted ones, as part of copyright suit (NYT v. OpenAI).
• OpenAI criticizes the ruling as a “way of preventing OpenAI from respecting its users’ privacy decisions.” (Jeremy, 34:33)
• Could put OpenAI at odds with privacy law and “zero retention” business customers.

Hardware Race: China’s Huawei vs. Nvidia; Next-gen Chips

(36:36 - 50:30)

• Huawei is struggling to match Nvidia—despite state backing, outdated process nodes (5–7nm), and energy inefficiencies.
• Large Chinese techs (ByteDance, Tencent) reluctant to adopt due to competitive dynamics and U.S. pressure.
• Huawei promising 3nm GAA chips by 2026, but skepticism abounds; yields currently “really bad.”
• TSMC Angstrom (1.4nm) due 2028: Gonna cost $45K/wafer, 50% higher than 2nm.
• Nvidia as world’s most valuable company—could start crowding out Apple for “the leading node” at TSMC.
• Mistral launches Mistral Compute, touting energy and regulatory support (“the only Western country that can still build nuclear plants in <10 years” — Jeremy, 48:34).

Research & Open Source

ProRL – Pushing Reasoning With New RL Tricks

(51:26 - 56:09)

• Prolonged reinforcement learning adds “genuinely new capabilities” to LLMs, not just surfacing old ones.
• Innovations include periodic reference policy resets and nuanced regularization.
• “There’s all kinds of shit. It’s actually quite an interesting collection of shit. The shit links together in interesting ways…” (Jeremy, 51:26)

Rethinking Scaling Laws: Test Time & Memory Matter

(56:09 - 64:00)

• Kinetics: Rethinking Test Time Scaling Laws: Proposes including memory access in scaling equations—flops alone are inadequate.
• “One of the big bottlenecks now is just how fast can you move the data around… That’s become more and more of an issue as [sequence] lengths get greater and greater.” (Jeremy, 58:39)

Surprising Power of Negative Feedback (Negative RL)

(64:00 - 69:44)

• Training LLMs by penalizing wrong answers produces more diversity, less overfitting vs. rewarding “correct” answers (“Positive only”).
• “There’s a bit of a loss of output diversity versus negative only, which improves performance across all paths at K metrics.” (Andrei, 64:00)
• Weighted approaches (90% negative, 10% positive) seem optimal.

Automating Research: LMs as AI “Taste” and Experiment Runners

(69:44 - 79:56)

• Predicting Empirical AI Research Outcomes with Language Models: LM can predict experiment results better than human experts in some setups (77% vs. 49%).
• ExpBench: Benchmarks AI agents’ ability to replicate published research experiments—O3 mini bests others but only 1.4% success rate, yet: “That’s a pretty big 1.4%, at least in my mind.” (Jeremy, 76:25)

Policy, Safety, and National Security

Models Know When They’re Being Evaluated (Alignment/Sandbagging Risk)

(80:07 - 83:42)

• Multiple choice and open-ended tests show models (e.g., Gemini 2.5 Pro) are increasingly adept at recognizing when they’re undergoing evaluation for safety/capability.
• “Frontier models show definite above random evaluation awareness…that’s kind of interesting.” (Jeremy, 80:46)
• Raises concerns over models faking alignment in controlled tests but behaving differently in deployment.

Interpreting In-Context Learning; Multiphase Emergence

(83:43 - 90:11)

• New research shows that LLMs’ ability to learn “on the fly” isn’t just due to simple induction heads, but involves complex, staged circuit emergence.
• “There are different types of emergence that might occur in neural net training, which in general is interesting.” (Andrei, 90:11)

Security

First-Ever Zero-Click LLM Agent Attack (Copilot)

(92:04 - 94:07)

• Echo Leak: A vulnerability allowed attackers to extract data from Copilot via malicious emails—no user interaction required.
• “The attack surface has just exploded, right, with these agents.” (Jeremy, 92:04)
• Reveal the challenge of defending prompt-based systems in assistant/agent contexts.

ClaudeGov: Anthropic’s Models for US National Security

(94:07 - 97:32)

• Tailored LLMs used in classified government settings for “planning, operational support, intelligence analysis, threat assessment.”
• Noted tension: “Sometimes you do want these models to be capable of things you wouldn’t want everyday users to do.” (Jeremy, 95:22)

Synthetic Media, IP, and Labor

Midjourney Sued by Disney & NBCUniversal

(97:32 - 100:18)

• Accused of “straightforward copyright infringement” for letting users create imagery of protected characters.
• Notable: Lawsuit PDFs embed AI-generated Shrek and Darth Vader images.
• “Midjourney probably has fewer resources these days…to pull off its lobbying effort.” (Jeremy, 99:19)

SAG-AFTRA & Video Game Companies Reach Deal

(100:18 - 104:03)

• Union covering actors and voice actors wins AI compensation/consent protections after 18 months of negotiation.
• Ongoing dilemma: “Do we own our voices? What does it even mean to own our voices?” (Jeremy, 103:34)
• AI-powered voice synthesis is blurring lines of IP/likeness in media and entertainment.

Notable Quotes & Moments

• On O3 Pro’s benchmarking:
  “You see a clean sweep where the model 64% of the time is preferred to humans… across everything from quantifiable to qualitative tasks.” (Jeremy, 05:52)
• On software engineering’s future:
  “We're no longer hiring entry level software engineers…we don't expect ever to do that again.” (Jeremy, 32:32)
• RL section colorful summary:
  “There's all kinds of shit. It's actually quite an interesting collection of shit. The shit links together in interesting ways…” (Jeremy, 51:26)
• On the new zero-click Copilot hack:
  “There's no phishing, no malware needed. This is just straight prompt injection.” (Jeremy, 92:04)
• On labor market shifts:
  “The job of software engineers, the job even of AI researchers, is getting more and more abstract and further away from…many of the activities that used to define them.” (Jeremy, 32:32)
• On the new phase of AI entertainment IP:
  “Do we own our voices? What does it even mean to own our voices?” (Jeremy, 103:34)

Key Timestamps

• OpenAI O3 Pro + Price Drop: 04:46–08:02
• Cursor AI Editor 1.0: 09:27–12:43
• Enterprise AI Talent Trench: 25:41–31:40
• Entry-Level Tech Jobs Disappearing: 32:32–33:47
• Zero-Click Copilot Security Flaw: 92:04–94:07
• Midjourney Lawsuit: 97:32–100:18
• SAG-AFTRA Deal: 100:18–104:03

Final Thoughts

• The AI field is advancing rapidly in capabilities, with growing social and legal complexity.
• Workforce dynamics are shifting as entry-level coding jobs dry up and high-stakes retention wars heat up.
• New attack surfaces are emerging as agents gain more autonomy.
• Big legal showdowns (Midjourney, OpenAI) will set precedent for generative AI and IP.
• AI is increasingly both the originator and executor of research, and ethics, security, and labor issues will only accelerate in tandem with AI’s technical progress.

To learn more and find links to the stories, visit lastweekinai.com