Podcast Summary: City Journal Audio – "The AI Arms Race"
Host: Jordan McGillis (Economics Editor, City Journal)
Guest: Judd Rosenblatt (Founder/CEO, AE Studio)
Date: February 6, 2025
Overview
In this episode, Jordan McGillis welcomes Judd Rosenblatt to discuss the accelerating global race in artificial intelligence, spurred by China’s Deepseek AI firm releasing its competitive R1 model and the U.S. response with the massive "Stargate" initiative. The conversation centers on the strategic importance of AI, the open versus closed source debate, technological and geopolitical risks, and the urgent need for better alignment of AI with human values and U.S. interests.
Key Discussion Points & Insights
1. The Significance of Deepseek’s R1 Release
- Deepseek’s R1 Model: Comparable to top American models but reportedly much cheaper to train.
- Algorithmic Advances: Deepseek achieved significant improvements with lower compute, challenging the narrative that only massive resources matter ([01:23]).
- Negative Alignment Tax: Improvements in AI alignment can paradoxically increase overall capability:
"Improving alignment techniques... not only mitigates risks, but also enhances capabilities...it uses reinforcement learning to induce chain of thought reasoning." – Judd Rosenblatt ([01:23])
- Origins of Deepseek: Started as a hedge fund passion project, leveraging existing compute resources to enter the AI race ([02:30]).
2. Open Source vs. Closed Source Debate
- Context: Open source allows rapid innovation, knowledge sharing, and alignment advancements, but also increases the risk of abuse or embedded 'sleeper agents' ([03:00–05:13]).
- Risks of Open Source:
"You can put sleeper agents into an open source model and then there is no way to know that they are there and they can get activated anytime." – Rosenblatt ([03:40])
- Major Players:
- Meta: Pro open source ([05:21])
- Mistral: Open source, U.S.-invested ([05:21])
- OpenAI: Closed source, with Sam Altman recently questioning if that was the right strategy ([05:41])
3. U.S. vs. China: Hardware, Export Controls, and Geopolitics
- AI Chips Controversy: Evidence suggests Deepseek used American chips via loopholes, with major compute centers in China and possibly Singapore ([06:43]).
- Export Controls: New U.S. and Dutch regulations aim to tighten access to advanced chips and manufacturing tools.
- Chinese Self-Sufficiency: Despite efforts, experts doubt China can catch up quickly on indigenous chipmaking ([09:07]).
- Compute as Leverage:
"If we have very stringent export controls and we continue to invest in compute ourselves, they probably can't [outcompete us]." – Rosenblatt ([07:40])
4. Exponential Progress & Alignment Challenges
- Power of Exponential Growth: AI capabilities are projected to increase by 10,000 to a million times in five years ([11:36]).
- Human Blindness to Exponential Growth:
"Humans didn't evolve to be able to understand exponentials. There's something called exponential slope blindness." – Rosenblatt ([10:32])
- Alignment as Both Risk Mitigation and Capability Booster:
"Investing in alignment itself yields... advancements that can do things like cure aging and solve diseases..." ([12:16])
Alignment investment not only addresses existential risk but fosters new capabilities and economic value.
5. Policy Recommendations for U.S. AI Leadership
- Security & Espionage: Enhanced protection for AI labs from hacking, as leaks aid adversaries ([14:16]).
- Avoid Overregulation: Especially on state legislation like Texas’s Responsible AI Governance Act targeting "algorithmic discrimination," which could hinder innovation ([15:39]).
- Alignment Investment: Substantial new funding for AI alignment, ideally through public-private partnerships ([17:00]).
"Substantially increase investment in AI alignment. That would be the thing that would be the best thing you could possibly do." – Rosenblatt ([16:32])
- Monitoring and Compliance: Challenge lies in credibly verifying firms’ commitment to alignment research ([19:13]).
6. China’s AI Alignment Stance
- Concern at the Top: China’s Turing Award-winning scientist reportedly worried about AI risk. U.S. (Henry Kissinger) possibly pressured Xi Jinping to adopt a cautious approach ([19:38]).
- Diplomatic Leverage: America may influence China to slow or align with U.S. norms by capitalizing on shared fears of losing control ([20:49]).
7. U.S. Government Coordination and Political Outlook
- System Integration: No clear method yet to coordinate AI policy across agencies; much awaits executive branch direction ([21:29]).
- Conservative Leadership: There’s historical precedent for Republicans to lead on "unthinkable" risks, referencing nuclear threat analogies ([22:58]).
"There needs to be realistic people thinking deeply about how this technology is going to change everything..." – Rosenblatt ([23:59])
- Public Awareness: Trump administration and allies (including family members) are already signaling concern about advanced AI risks ([22:14]).
Notable Quotes & Memorable Moments
-
On the Danger of Open Source:
"China could eventually create some open source model…and it could turn out to have a botnet that could take over all infrastructure in the west." – Rosenblatt ([04:12])
-
On Exponential AI Development:
"The consensus seems to be somewhere between ten thousand and a million times more powerful five years from now." – Rosenblatt ([11:36])
-
On Alignment Investment Being a Win-Win:
"The biggest investments in alignment to date have actually advanced AI capabilities in the first place." – Rosenblatt ([13:34])
-
On Policy Balance:
"You don't want to overregulate and then therefore thereby not be able to solve the alignment problem because you cripple yourself." – Rosenblatt ([14:50])
-
On China and AI Doomerism:
"Supposedly Henry Kissinger is…tried to make Xi an AI doomer." – Rosenblatt ([20:06])
-
On U.S. Leadership:
"There’s a strong history of good conservative leadership on difficult problems like this." – Rosenblatt ([23:16])
Important Timestamps
- [01:23] – Deepseek’s R1 and negative alignment tax
- [03:00–05:13] – Open source vs. closed source, sleeper agents risk
- [06:43] – American AI chip involvement in Deepseek’s model
- [09:07] – Chinese chip self-sufficiency and export controls
- [11:36] – Exponential AI capability projections
- [14:16] – Policy prescriptions for U.S. AI leadership
- [15:39] – Pitfalls of overregulation; Texas law example
- [16:32] – Strong call for investment in AI alignment
- [19:38] – China’s top AI thinkers and possible U.S. leverage
Where to Find More
Summary
This episode provides a detailed, nuanced exploration of the global AI arms race, emphasizing the twin imperatives of American innovation and effective risk management. Rosenblatt calls for significant investment in AI alignment, both for safety and competitive advantage, while cautioning against regulatory overreach that could stifle progress. The conversation illustrates both the strategic stakes and the complexity confronting policymakers as AI rapidly evolves.