#247 – Anton Leicht on how middle powers avoid losing everything in a post-AI world

Summary

80,000 Hours Podcast: How Middle Powers Avoid Losing Everything in a Post-AI World | Anton Leicht (July 14, 2026)

Host: 80,000 Hours Team
Guest: Anton Leicht (Fellow at Carnegie Endowment, author of "Threading the Needle" policy blog)

---

Episode Overview

This episode explores the bleak prospects for "middle powers"—countries that are not the US or China—in a world dominated by rapidly advancing AI. Host and guest discuss the geopolitical, economic, and security challenges these countries face, and examine policy options for securing a stake in the post-AGI (artificial general intelligence) world order. Anton Leicht proposes practical strategies for middle powers, analyzes the feasibility of building their own frontier AI models, and discusses the risks of relative economic and strategic disempowerment.

---

Key Discussion Points & Insights

1. The Default Bleak Future for Middle Powers

• Global AI development is “winner-take-most”: countries not building their own frontier models risk capturing all the risks of AI but few of the benefits ([01:13]).
  • Anton: “Most countries in the world that don’t build their own frontier AI models… capture all the risk… and yet they minimize all the benefits.” ([01:13])
• The risk of middle powers becoming economically and strategically irrelevant unless they actively secure participation in global AI supply chains.

2. Policy Options for Middle Powers

• Integration into U.S.-aligned AI supply chains: Find and protect bottleneck assets (e.g. manufacturing, data centers) that major AI developers need, creating mutual dependency ([01:36], [03:21]).
  • Anton: “The middle powers do what they’re good at, the US does what it’s good at, and then we… combine that.” ([02:38])
• Compute-for-access deals: Building data centers or providing critical infrastructure in exchange for guaranteed commercial access to frontier models ([04:39]).
• Direct deals with labs vs. US government: Negotiating with labs is more agile and effective than relying on U.S. government promises, which can be volatile ([06:37], [08:22]).
• Bottleneck risk: Without assertively securing a role, middle powers may be shut out from critical AI capabilities, deepening their dependency ([09:35], [12:17]).

3. Challenges of Building Sovereign Frontier Models

• Extremely high cost and risk: $500 billion+ over several years just to reach parity ([12:34], [13:16]).
• Political and practical roadblocks: Pooling resources from several countries is challenging; U.S. may retaliate ([12:58], [15:03]).
• Pragmatic realism: Even if a large coalition formed, U.S. export controls on chips and coding agents could strangle such efforts ([25:01], [30:07]).

4. Limitations of the "Fast Follower" Strategy

• Open source “fast follower” models risk antagonizing the U.S. and undercutting core U.S. firms; difficult to maintain a tight competitive gap ([16:00]).
  • Anton: “If you think the fast follower can stay four months behind, you… put [the Americans] on a clock for every defensive operation because… everyone in the world is going to have it [soon after].” ([00:11], [16:00])
• Realistically, fast followers are “just a money drain”—expensive, always behind, and with no exclusive benefits ([16:48]).

5. Ideal Structure for a Middle Power AGI Project

• Should be a coalition with upfront, irrevocable funding and clear “czars” from each country who serve as translators between technical and political realms ([18:55]).
• Need political “buy-in” and readiness to treat AGI like other strategic assets (e.g., military equipment), justified on national security rather than ROI ([21:03], [22:33]).
• Anton: “A warplane is also a money sink. A warplane doesn’t make its money back. It’s something you buy, so you have it.” ([21:03])

6. Risks of U.S. Retaliation & Bottlenecks

• U.S. may retaliate not just in tech, but via broader sanctions, intelligence cut-offs, etc. ([25:01]).
• Export controls on chips/coding agents are major vulnerabilities; need for sophisticated bargaining using Europe’s control over key supply chains (e.g., ASML, Japan, South Korea) ([25:01], [28:22]).

7. Economic and Strategic Disempowerment

• Even if European economies benefit from U.S. AI products, over time the loss of relative power puts them at risk—unable to shape global rules, vulnerable to U.S. strategic whims ([38:03], [41:04]).
  • Anton: “At some point it's really difficult to look at these European countries and still think that they're sovereign nations in a relevant sense.” ([41:46])

8. Policy Recommendations for Middle Powers

• Defend & modernize bottleneck assets: Screen foreign direct investment (FDI) to prevent U.S. or others from acquiring critical industrial assets ([48:00]).
• Make assets AI-compatible: Modernize factories and labor markets to ensure seamless integration with AI-driven supply chains ([48:00]).
• Maintain flexible labor markets: Accept short-term pain for long-term gain; rigidity leads to larger disruptions ([52:45]).
• Avoid short-sighted protectionism: Temptation to lock-in jobs or resist AI transition will harm competitiveness ([54:45], [55:34]).

9. Equitable Taxation & Social Policy

• Corporations should be the tax base for disruption mitigation, avoiding token taxes that stall necessary AI adoption ([60:18]).
• Arguments against government equity stakes in labs: risk of conflicts of interest and inadequate revenue scale ([62:41]).
• Corporate tax coordination (e.g., global minimums) is needed to avoid race-to-the-bottom, given that AI-based firms remain somewhat geographically anchored via data centers ([64:56], [66:01]).

10. Forecasting the Long-Term AI World Order

• U.S. likely to pursue “soft” nationalization of labs via regulatory influence rather than outright ownership ([68:41]).
• Ambitious “pause” or “superintelligence ban” ideas are operationally ill-defined and geopolitically fraught ([70:58], [77:51]).
• The best-case scenario is a broad Western coalition with checks and balances, rather than unilateral U.S. dominance ([81:16]).
• Involving middle powers in access and governance increases policy “sanity” and reduces risks of catastrophic loss of control ([84:21]).

---

Notable Quotes & Memorable Moments

• On the bleak status quo:
  "Most countries in the world that don't build their own frontier AI models… capture all the risk… and yet they minimize all the benefits." — Anton ([01:13])

• On fast follower risks:
  "If you think the fast follower can actually steadily stay four months behind, you do two things to the Americans. You put them on a clock for every defensive operation.” — Anton ([00:11])

• On building your own model:
  “Building your own frontier model is just really extremely hard and extremely expensive… Realistically, it's maybe like $500 billion over four years.” — Anton ([12:58])

• On U.S. retaliation & leverage:
  “You do risk just like sort of… retaliation from the US… tariffs on unrelated things, seizing intelligence cooperation…” — Anton ([25:01])

• On sovereignty risk:
  "At some point it's really difficult to look at these European countries and still think that they're sovereign nations in a relevant sense." — Anton ([41:46])

• On labor market flexibility:
  “If your labor market isn’t very flexible, then people just stay in their jobs, even if AI is better at their jobs. And then you have huge efficiency losses… and eventually you just risk being disrupted.” — Anton ([52:45])

• On government stakes in labs:
  "If the U.S. treasury and also just general U.S. economic incentive… is directly tied to the stock of AI providers, what kind of incentive does that place on the US government in terms of regulating models?" — Anton ([61:18])

• On strategic advantage & coalition governance:
  "If you slow down specifically on the AGI development, that is just a huge asymmetric hit to the US...” — Anton ([74:55])

• On risk management:
  “Whenever you concentrate power within a single volatile actor, the variance of the decision quality just obviously gets higher... the risk of just doing something monumentally stupid with a very intelligent system is... much higher...” — Anton ([84:44])

• On outside analysis:
  “The Republic of Letters is sort of getting starved... as more of these things enter national security relevant domains... But this still needs doing. All of this needs explaining. All of this needs analysis...” — Anton ([88:44])

• On Germany’s future:
  “In a way it's very easy to solve because you just have to do all the things that hundreds of thousands of people are thinking and writing about anyway. But in a way it's also much harder to solve...” — Anton ([91:08])

---

Important Timestamps

• [01:13] - Why the future looks bleak for non-frontier AI countries
• [03:21] - Policy moves for smaller middle powers
• [04:39] - "Compute for access" concept explained
• [08:22] - How deals with labs can insulate middle powers from U.S. government whims
• [12:58] - Cost and feasibility of building sovereign frontier models
• [16:00] - Limitations and risks of the "fast follower" open-source approach
• [18:55] - Blueprint for a multinational, middle power AGI project
• [25:01] - Ways the U.S. could counter foreign AGI efforts
• [38:03] - How open markets with U.S. AI might still leave Europe strategically disempowered
• [41:46] - On the existential threat of relative power divergence
• [48:00] - Strategic FDI screening and factory modernization
• [55:34] - Social policy to cushion labor market shocks
• [60:18] - Taxation responses (corporate tax vs. AI-targeted taxes)
• [68:41] - The likeliest U.S. approach: “soft nationalization” of AI labs
• [70:58] - Realism and complexities of an AI development “pause”
• [81:16] - Would a decisive American strategic advantage be net positive?
• [84:21] - Involving more stakeholders lowers policy volatility and risks

---

Summary Table: What Should Middle Powers Do?

| Challenge | Policy Recommendation | |-----------------------------------|----------------------------------------------------------------------| | Lack of sovereign models | Negotiate “compute for access” deals with labs, not just U.S. gov. | | Losing strategic leverage | Defend bottleneck assets, screen FDI, invest in AI-compatibility | | Labor market disruption | Promote flexibility, resist reactionary protectionism | | Revenue/tax base erosion | Rely on coordinated corporate taxation (not AI-specific taxes) | | U.S. retaliation or indifference | Strategic coalition-building, leverage chip/tool supply chains | | Social unrest | Subsidize displaced workers, accept short-term pain for long-term gain| | Geopolitical instability | Push for multi-stakeholder alliances and checks/balances |

---

Closing Thoughts

Anton Leicht paints a sobering but constructive picture of the options facing middle powers as AI transforms the global order. While the path to building their own AGI is daunting, there are critical strategic moves that can secure a stake in the future: defending industrial bottlenecks, modernizing for AI, forging “compute for access” partnerships, and keeping labor and regulatory frameworks adaptive.

Engagement in the governance of AI is not just a technical or commercial matter; it is one of sovereignty, political stability, and global safety. The episode combines hard-nosed policy analysis with a keen appreciation for the political and moral stakes of the coming AI paradigm.

---

For more insights and expert policy discussion, subscribe to the 80,000 Hours Podcast.

Transcript

[A]

Feels to me that you could pitch this to the US's. We're going to create fast follower open source models such that the best open source model in the world is not Chinese. This is a good thing. We're not trying to eat your lunch. Like, why not go for that strategy?

[B]

If you think the fast follower can actually steadily stay like four months behind, you do two things to the Americans. You put them on a clock for every defensive operation because you tell them, well, whatever crazy dangerous stuff you come up with in four months, everyone in the world is going to have it. Every terrorist, every non state actor, every rogue nation in the world. Also you have these like $1 trillion companies that base their entire sort of revenue strategy, like selling these exclusive frontier models. And you only ever get 4 months to sell them because after that we'll just undercut them with an available open source model. I'm not quite sure you could sell the Americans without being a great idea.

[A]

Today I'm speaking with Anton Leicht, who is a fellow at the Carnegie Endowment for International Peace and writer of the excellent policy blog Threading the Needle. Among, like many different strengths. I think one of Anton's core appeals is he has this gift to write things that somehow sound both secret and obvious. Thanks for coming on the show, Agontorn.

[B]

Thank you so much for having me.

[A]

Why do you think that by default, countries that are not the US and China look like they have a pretty bleak future?

[B]

I think when we talk about AI policy, usually we talk about sort of minimizing the risk while capturing all the benefits. I think the default outcome for basically most countries in the world that don't build their own frontier AI models is the exact opposite, which is they capture all the risk that come from these models going through society, and yet they minimize all the benefits they could potentially have from AI happening.

[A]

Okay, what can middle powers do to avoid this future?

[B]

Well, I think find some stake and find some participation not only in this broader AI supply chain, but in what you expect the sort of world order with advanced AI to look like. Because anthropic didn't promise us just geniuses in a data center. They said they would cure cancer. So how do you do that? You actually have to sort of test the cures you make. You actually have to build the production plants to actually make the medicine and so on. The same thing goes for robots. The same thing goes for all the military assets that you think you might want to produce with that for us. China, competition, lots of middle powers, especially the Europeans and South Koreans and Japanese are pretty good at occupying these bottlenecks. And I think if you have something like that and if you can integrate it into the AI supply chain sufficiently effectively, then that is just sort of long term asset that gives you some economic contribution and that creates some minimal mutual dependency. There is a very realistic sort of second best, worst equilibrium where the middle powers are sufficiently useless allies that don't really build up the capabilities that would feed into this broader US AI model supply chain that would give them this edge, but instead decide to focus on whatever their own models or shutting off their markets from American models or just moving away from this mutually beneficial arrangement. And I think on a very high strategic level, what the goal here should be is to avoid these second best equilibria and just move toward the actually sort of best broad alliance wide supply chain integration. The middle powers do what they're good at, the US does what it's good at, and then we sort of combine that. But it's not obvious. And I think there are a lot of factors pulling away from that and into the worst outcome.

[A]

What policies do you pursue right now to not waste that window? Like it's kind of easy for, I don't know, Japan, South Korea or the Netherlands. But what if you're not one of those countries? What are you doing right now to make sure you're moving towards that partnership future?

[B]

I think there are a lot of narrow policy interventions that are very sort of country specific. Right. You want to think about very specifically integrating parts of your industry with American development, development in a way that is mutually beneficial. The American labs want close cooperations with legacy industries for a bunch of reasons, between financing and just data and just ability to very deeply integrate into broader supply chains and actually deliver on the things they said they built. I think these partnerships are attractive to the American labs. And then I think there's the much broader question of, well, what are the actual input assets you need as a middle power right now to enable sort of going down that path in the future as AI models become more important. And, and I think the most important thing for that is just access to the frontier models. You just need sufficiently good access to sufficiently good AI to then think about how to structure your administration and your business and even your consumer market around that. And for securing that access, there are not that many options. I think the best general version we've come up with is compute for access, which is you build out data centers to alleviate the inference crunch and the sort of sort of meet the demand for inference that these leading Labs have, and in return you ask them for guaranteed access to Frontier models, at least at parity with the US commercial market.

[A]

Has anyone done this yet? Have Norway and the uae, do you think they've got anything in return for their domestic compute build out?

[B]

I think those are different cases that were in a position to ask for different things. I think the UAE deals were in many ways an uphill battle for the UAE where they couldn't ask for much. I think it was already quite a stretch for the UAE to even convince the administrations that sort of negotiated this deal to even let them export the chips to the UAE as is. And so I think that was already pretty contentious. So I think they weren't in a position to ask for much more. I think Stargate Norway, which is now like a Microsoft buildout instead of an actual Stargate, because OpenAI has partly gone back on some of that, had the beginnings of that. But I also think the negotiations around that also preceded this acute need for Frontier access and guaranteeing it. So I think this is not yet part of any of the deal frameworks that are around. I do expect this to be part of the deal frameworks that come up as bigger data center build outs and middle powers get negotiated. But it is an open question whether that works, because the draw for the Frontier labs in building these data centers anywhere else that isn't the US is, well, can you get them up faster and can you get them up at least somewhat efficiently? And then the geopolitical considerations of, well, maybe the labs would like to reduce political concentration in the U.S. maybe they would like to reduce the geopolitical risk that comes with just being completely sort of having all their infant supply in just one country. Maybe they'd like to reduce the domestic political risk of lab. Like local political backlash against data centers, like national political backlash against the AI industry. They'd probably like to diversify out of that a little bit, but they can really only afford to diversify out of it if it doesn't take four years to build a data center. Right. I think the benchmark here is something like 18 to 24 months, realistically for a big data center. And so if countries can offer something like a gigawatt built in something like one to two years, I think that's a great basis. I think if you pitch that to a lab and say, well, we'd like to access parity in response, you can probably make that deal work.

[A]

Are they striking these deals with the labs directly or through US government? How do you envision this playing out

[B]

So I think this is all.

[A]

Is it George Osborne?

[B]

Somewhat hypothetical. I think generally what we've seen is that direct deals with the US Government are both somewhat volatile and somewhat susceptible to spreading out horizontally in a way that isn't great for the host countries. We've seen the UK US Tech Prosperity deal that was in theory a model for some deepened tech cooperation that also included data center buildup as one pillar of the deal. That didn't work very well, all things considered, because then last December the US brought food safety standards into the negotiations and then OpenAI sort of reneged on the Stargate aspect of it because inciting copyright and energy price reasons. And so that all didn't really work out the way they would have hoped. And I think there's the more general risk of, well, if you strike a deal with the US government, then the US can go back on that deal if you do things that the US government doesn't like in any other domain. And that doesn't sound that great if you're a middle power. I think making the deals with labs is easier, both because the lab demand for diversifying the inference compute is more acute and because I think the better way of political influence in some way and of making sure that the access restrictions aren't as hard and making sure that US government doesn't intervene in your access parity is incentivizing the labs to themselves lobby the US government. So if you have a lab that has guaranteed you access parity and the lab has put in writing, well, you gave us this data center. We're now giving your market the access to the same models that the US market gets. And I think that's the realistic case. Right. I think government contract parity is a lot harder to negotiate for because that's national security relevant. But if we're just talking about market access, I think this is a perfectly reasonable ask. Right? If Fable 5 is on the US market, it also comes on the UK market, also comes on the European market, comes on any market that has also come up with a data center for anthropic.

[A]

Right?

[B]

That's the basic deal structure. And then if the US government decides to come up with some access restriction, then the American firms that are involved in these deals and that have a stake in these deals and that have invested a lot of their money into these deals now have an incentive to tell the US Government that you'd have Nvidia and Anthropic and ideally sort of other labs as well that have maybe even better standing with the US Government. Sort of going to the government saying, well, we actually have all these contracts and all these gigawatts of inference compute online other countries. Can you guys not restrict access in this way? Can you at least give a carve out to this country that we have a data center in? Or can you make a carve out for allies only or something? Because we'd really like to not default on our contracts with them and we'd really like to sort of honor these agreements so we keep access to this inference and so you get a much stronger advocate in dc.

[A]

Does that account for that much though? Because I feel like Wired, I think were reporting that the Fable export control was in part prompted by them giving Mythos access to I think SK Telecom, the South Korean company which the admin alleged was like had ties to China or something. Like, it just feels like the labs, especially in the scenarios where things go crazy and where you might most want access to the inference compute, the labs will be powerless if US government decides that they should just pull the compute. How do you make sure that deal's not reneged on?

[B]

Well, so first of all, I think the Fable thing is I think a red herring in this conversation. Just because there is no restricted access for Fable. Right? There's just no access for Fable. And so I think Fable doesn't violate access parity in a strict sense. Right? The UK doesn't have any less access to Fable than the US market. No market has access to Fable. So this is perfectly compliant with access parity in some way. I think the specific scenario that you want to avoid is the US saying well, we're fine with you guys giving access to the US market but not with giving access to other markets. And I think these are largely not the scenarios where there are super strong and super restrictive security concerns. Because then why would you even give access to the US market? This is mostly insurance against the economic divergence specifically. Right, because what you are worried about on the economic divergence is one market has access to abundant Frontier AI and another market doesn't have that. And I think for these cases the security concerns are never all that high. Because if you are giving access to the entire US market to all US firms that you aren't really doing super intense KYC and very high defense contractor style security checks for, then it can't be that bad. So that's not the situation where things are going crazy. That's not the thing that commercial access parity insulates against. If we're talking about glasswing level access parity or even government Access parity. I think for that only the inference compute incentive doesn't work out quite as much. I think for that you also need cooperation on security architectures. Which is to say if I'm South Korea, for example, and I want to be included in something like Project glasswing, then the firms that I propose my Project glasswing is extended to have to pass some security standards that will probably be handed down from the U.S. government. And these security standards will be about cybersecurity architectures. They will also be about standard, just general security of the firms and they will also be about China ties, inevitably. And I think that will have to be part of the cooperation. If we think about closed access parity or even sort of government level access parity, you just have to come up with some shared amount of security standard and only firms, and only governments and only agencies that clear the security standard get access parity. And I think for that the inference leverage still matters a little bit because I think giving and expanding access once the security is guaranteed is not that big of an ask. These carve outs aren't particularly rare to get and I think it's not particularly difficult to lobby US government to come up with narrow covers for allies. And you just have to create some marginal incentive for anyone in the room being interested in doing that. So I think the hedge against that is just being like, well, the Australians or the UK or the Japanese or whatever, we know that this is a pretty secure country. We're not that worried about the security implications of exporting to them or expanding to them. You just need someone in the room to think, well, maybe we don't screw over our allies in this moment. And I think for that you just need a marginal incentive of the inference thing, for example. But most of it is about the security cooperation instead.

[A]

This still feels like a really fragile position to be in. There's all these problems where you need need to build loads of compute. And obviously your citizens are going to be unhappy with that anyway because that's going to suck for their life. And you need to depend on the US government being reliable or dependable. Why not just build your own frontier model? Is that not the best option?

[B]

Why does building compute suck so much? I think perfectly fine for the citizens, especially if we get the hyperscalers to pay for it. And I think there is a realistic chance we do that. A data center isn't all that bad for most of the citizens. It's just a box with a computer in it that sits somewhere in your countryscape. And Amazon paid For okay, but you're

[A]

also going to have to relax a lot of regulations if you want to be at all competitive with time, time to power and like the cost and all these kinds of things. Yeah, this is going to be hard to sell to the.

[B]

Yes. And maybe you do like narrow carve outs and yes, you get like some ideological groups opposed to it. But I think you don't have to give up on your entire regulatory framework. I think there is a way to sell it is just what I'm saying. But yes, I think that's right. Building your own frontier model is just really extremely hard and extremely expensive. If they really wanted to go for it, I think there is a way to do it. I wrote about this recently. I sort of did some educated guesses on how much that might cost. I think realistically, if you wanted to, for a four year project split between different middle powers that would work together to throw in the talent and the compute and the funding and the supply chain leverage and everything you would need to come up with an actually competitive way to build a frontier, frontier, frontier model. It's maybe like 500 billion over five years or over four years. That's a lot of money. I think if you go into a room right now, especially if you go into a room in a Treasury in a middle power country and not in a digital ministry, and tell them you'd like to have $500 billion to build a highly speculative enterprise, invest into American infrastructure, build huge data centers and then try to train a model on them. And you don't have a business case for where you'd sell that model. You don't have a clear way to get the investment back. And this entire sort of approach is highly vulnerable to both domestic political pressure and to US coercion to stop doing this. I think there is probably no treasury in the world right now that would tell you that they'd happily buy their share. And so I just don't quite know how you pay for it. I don't quite know how you do it. There's a long list of political challenges to it. I will say if there was sufficient motivation to do it and if there was sufficient funding to do it, 500 billion isn't actually that much money for x of the G7 Europe plus Canada plus the UK plus Australia plus Japan plus South Korea. If you could get the best version of the Allied Liberal Democracy alliance set up and you got all of them to pitch in, they could afford 500 billion for that kind of project. It's not an absurd idea on paper. And I think if you're sufficiently bold into this specific idea that the frontier matters a lot, I think this is the only realistic play because you're right. If the frontier matters as much as we've speculated it might, and if the Americans are as volatile as some people fear they might be, especially after the fable incident, then I think any other conclusion really doesn't work that much. They're all hedges and they're all bets that at some point the Americans will be wrong, or at some point the commodification will kick in, or at some point the fast followers kind of stay around, or maybe the econ just works out in a way where you capture a lot of the revenue downstream. This all might still happen, but any other sovereignty strategy that is not building a frontier model is basically a bet on one of these things. I think you can make enough hedged bets that you probably end up fine. But if you actually want to be certain that you don't get screwed over by frontier model progress specifically and by sort of super effective AGI systems taking over the world, then you do need your own AGI. And that's the only way to do it.

[A]

Why not pitch this? It feels to me that you could pitch this to the US's. We're going to create fast follower open source models such that the best open source model in the world is not Chinese. This is a good thing. We're not trying to eat your lunch. Why not mix, go for that strategy? Why say we're going to deliberately go for the frontier itself?

[B]

So first of all, I don't think the Americans will be particularly happy. I mean this is in a very, very real way does eat the lunch of the Americans, right? Because if you think the fast follower can actually steadily stay like four months behind, you do two things to the Americans. You put them on a clock for every defensive operation because you tell them, well, whatever crazy dangerous stuff you come up with in four months, everyone in the world is going to have it. Every terrorist, every non state actor, every sort of, you know, every rogue nation in the world. Everyone is going to get access to, however dangerous this capability is. Like if just actually committing to just throwing resources into having a fast follower that open sources things. I think that doesn't sound great to the Americans, not only because of the security part, but also just because of the commercial part where it's like, well, you have these $1 trillion companies that base their entire sort of revenue strategy on selling these exclusive frontier models and you only ever get 4 months to sell them because after that we'll just undercut them with an available open source model. So I'm not quite sure you could sell the Americans on that being a great idea. I'm also just not sure it actually works because what does it actually mean to build a fast follower? I think there's basically two versions of saying this is a fast follower. The one is it's just code for we are building a worse frontier model builder, someone who's just four months worse at building a frontier model, but does all of it independently. So they invest almost the same amount of money, almost the same amount of talent. They're just like four months slower. So they never get any sort of exclusive window where they get to capitalize on a lead. No one ever has any incentive to go for their models instead of any other models. So it's just, just a money drain where you just spend a lot of money to be worse at something than someone else. And there is no real niche for that that you can always do. And then there's fast follower in the way that is actually dependent on frontier progress, right where you're following. And you're not only sort of in parallel moving up the same gradient, but what you're doing is you're following because you know what is happening at the frontier. And because you can emulate some of the things happening at the frontier, you can maybe even distill some of the things happening at that frontier. And that requires the frontier to be available, to be able to fast follow it. And just, just not so sure that that's going to remain the case. And then I think the final question about the fast follower is why do you think that helps in this specific scenario? I think what we've talked about until now, why you need the compute for access strategy, why you need these bottlenecks, was the specific scenario where you were very worried about this four month gap, that the Americans have to use their frontier model very well, where you said, well, we also want the same exclusive access to this frontier model. And we think that if you give any other nation a reliable sort of 4 month, 6 month, I think it might realistically rather be 12 months lead, then they will use that to generate a compounding advantage in R and D, in software development, in AI progress, in all these domains, that these models might be good at, that eventually disempowers us relatively. If you think that that's the case, then the fast forward doesn't actually help you in the limit. In this scenario you actually need the frontier model itself and I think then you actually have to go all in on buying the resources you need for that and actually developing the model itself.

[A]

So if the middle powers decide they want to go for building a frontier AI project themselves, what would that project actually look like?

[B]

Yeah, so I think what you do is you set up the biggest possible thing you can. You don't start small, you don't start with giving money to any existing institution. What you do is you build an actual coalition that has enough money, that has enough leverage, that has enough talent, and that has enough general geopolitical heft to actually back this up. You get all of them to put in their money right away and they commit their money on day one of the project. They don't get it back under any scenario they can't affect. You put it all in one pot and then what you do with that is you fund a private company like vehicle that is structured just like a frontier lab. This is the thing that builds your model and that builds your frontier system. And then this system, this vehicle only corresponds with the funding governments and only takes instructions through a middle layer of people that get what's going on. Every country that is involved in this coalition designates a sort of, of. This is like our TSAR for the project. And this is the guy who talks to the board of the project or talks to the founders of the project and sort of communicates what we say in political language to them in technical language. And what the project says in technical language sort of goes back to the coalition in political language translated by the czars. I think this is the structure you have. This is what you throw all your money into. And then this vehicle just gets maximum freedom and maximum sort of buy in to actually do its thing in the way that we know that works, which is like the structure of a normal frontier lab, which is you pay a lot of very, very, very good engineers from other labs a lot of money. So they do work on foundational model research that is powered by a large, large, large amount of compute. And so I think you don't need to reinvent the wheel. In fact, you mustn't reinvent the wheel in any of this. Because there is only one way we know that works how to build a frontier model. And we just exactly replicate that. We give them all the resources they need, we stay out of their hair as much as we possibly can and we just sort of let them cook. We buy in as much as we can. And I you get into this, the more countries you get into cooperating on this, the Broader the coalition you get that actually buys into this, the better that is just because you will need a lot of geopolitical resistance to American pressure. You will need a lot of funding to buy in. So I think the broader you go as far as you can get, the buy in, the better that is.

[A]

And do we just expect the countries involved to accept that this will just be a huge cost sink for a couple of years?

[B]

What I expect is that this does not happen well, but I think this is one of the reasons, I think there are smaller scale versions of this where basically countries have been okay with making this kind of bet. Like Airbus is one obvious example. I think framed and understood as this is just a critical strategic capability we need to have. This would not be understood by countries as like a money sink or not a money sink. A warplane is also a money sink. A warplane doesn't make its money back. It's something you buy, so you have it. And I think in some way this sort of AGI project would be the same thing, which is tough to swallow because obviously the Frontier labs in the US also make a lot of money, but they have to justify their cost to private market investors as opposed to governments. It's just the American capital market that is invested into the Frontier labs and soon the broader US and global stock market that's going to be invested into them. Whereas the project needs to be funded by Treasuries, mostly because the private capital doesn't exist in the rest of the world. So that's a bit of a disanalogy. So you do have to make the strategic asset pitch first and foremost. But that said, once you have developed a frontier model as a strategic asset, that is also something you can probably sell sell. You can do just the government procurement of this Frontier model and just people buying the Frontier model like governments buying the Frontier model and domestic firms buying the frontier model surely gives you some revenue back. And then I think there are some arguments to be made why the European project, if successful, would be somewhat competitive in broader markets. Right. I think there's some concerns about American capriciousness that might make people less excited about buying American models for all these

[A]

people still buying from Misrule.

[B]

Yeah, Misal still has like a decent American market share. And so I think in an international market you could think about competing. In some ways I'm pessimistic about that because I think you can perhaps brute force your way to building a good frontier model, building a good software product company that actually manages to sell a lot of its product has been historically almost harder for Europe to do than it has been to build big infrastructure projects. So that's going to be really tricky. So I think to some extent you should expect this to be a money sink for some time, but so are militaries. And I think they still sort of in some way justify themselves. Yeah, I don't think anyone ever looks at like, well, you know, this aircraft carrier is just sitting there and it's not making any money. Why don't we sort of rent it out? And why are we so bad at like productizing aircraft carriers? They're not supposed to be productized. They're national security assets and that's what we pay for them. And I think you would have to pitch the coalition on a similar logic for why they should go for the project.

[A]

Who do you think should be in the project?

[B]

So who I think should be in the project is the eu, the uk, Canada, Australia, New Zealand, South Korea and Japan. And if you can get Taiwan into that, that's even better. But I think that's really stretching the limits of how realistic this is. I think the general logic here is somewhat aligned liberal democracies, fairly even sort of value floor for coalition of the willing, good talent, good compute bottlenecks that will help you in terms of leverage, just enough capital to build the actual thing. I think that's the ideal version. I think realistically you're not going to get that because I think there are strong incentives to defect from this coalition. And I think countries that still have a very good bilateral relationship with the US or that still think that their bilateral relationship with the US their best play to sort of secure access to models and to secure participation in this economic order in basically the way that we've just described.

[A]

Right?

[B]

People that have countries that have the best access to this allied scale play, why would they instead sort of throw in with the European Union that has time and time again failed at building the things it announced it would build. That's a pretty hard pitch to make to a country like the UK to Korea to Japan who are currently still feeling pretty good about their bilateral, all things considered. So I think realistically you have to to cut those out and hope that they maybe join later if things deteriorate further, or maybe they join if the project seems to be going well. But I think for the time being, you probably start with Canada, as much of Europe as you can get. You try to get Australia and New Zealand in there as well, and then you start from there.

[A]

How do we keep the US from blowing this up. Why wouldn't they just be extremely antagonized by this attempt at a project?

[B]

They would be. This is part of the reason why it's very difficult to pull this off. I think if you were in a completely free and open and fair market and you had abundant access to every U.S. asset and no U.S. retaliation to fear, this would not be as difficult. But the thing is a, you do risk just like sort of broader retaliation from the US on whatever the US might choose to do, tariffs on unrelated things, seizing intelligence cooperation on all kinds of issues, like sort of just hitting the security architecture of especially the Europeans hard. For example, in the context of Ukraine, in the context of intelligence sharing, there are a lot of broader things the US could do to just stop you from doing things that have very little to do with the project or with building AI itself. And they just have a lot more to do with whatever other issue the US has leveraged in. So this horizontal escalation is the first thing to worry about. And I think there is no clever 300 IQ AI policy intervention that deals with that. What you need to deal with that is just, you just need to be generally more strategically powerful. You need generally more geopolitical leverage. This is an open problem for a lot of countries in the world. We're not going to solve it in the specific AI policy comp. But then there are two specific AI related threats that the US has against anyone trying to build a frontier model outside of the US the first is you need US chips to build this project, to build this frontier model. And the second is you probably need at least to start with using US coding agents to build this model. Right. I think much of the talent base of a frontier lab today are already coding agents. And the best coding agents by far in the world are the US coding agents. If you actually poach a bunch of the US researchers, if you put them in to a lab in Europe, if you give them all the compute they want and you tell them to start building a frontier model, but you don't give them codecs or Claude code, they're going to be pretty annoyed and they're going to be a lot slower at building these models. And as the labs keep using their best version of their internal coding models, and the project still doesn't have access to even basic versions of the coding agents, the gap just widens even more. And the more pronounced this gap is at the start of the project, the harder it is to ever get it off the ground. So you do need access to Some version of the coding agents, ideally, which is something that both the US and the US labs can restrict. So you have to find a way to maybe do like a carve out for some coding agent use at the very start of your project, and then you bootstrap very quickly to your own model and you build your own coding agent scaffold around it. And you just have that be your first step of business. And you just hope that that works well enough that you're eventually not that far behind on coding agents. And maybe you can get a lab to agree to that. Maybe you can get a lab to sort of not take you very seriously and think, well, the Europeans are never going to get anywhere anyway, so we might as well sell them like 3,000 Codex seats for like a billion euros or something. And then you just spend a lot of money on getting coding agent access as long as you possibly can. But it is going to be hard. You could try to build around Chinese open source coding agents in that case and try to bootstrap from that, but I think that makes it even more of an upper battle. And the second thing that's even worse are the chips you need. I think my very general guess is something like 3 million Blackwell class chips for building this thing. That's a lot of chips. There aren't even that many chips just to buy at the moment. Like Nvidia. Supply is pretty limited. So you would actually need to sort of offer sufficient money and buy out contracts and give sufficiently good incentives to Nvidia to actually want to sell you these chips. That's not trivial, but that's the easy part because Nvidia is interested in having lots of different.

[A]

They love selling customers.

[B]

Yeah, yeah, they love selling chips, they love money. But what they specifically love is selling chips to a bunch of different people. They don't actually want to be reliant on just like one or two buyers. And so I think they generally be excited about, about selling a lot of chips to this project. I think the US would be somewhat less excited about Nvidia selling chips to this project. There is this argument that, well, they're selling chips to China. Surely they would also be interested in selling chips to the Europeans. I think that's sadly not really true. Not to get into the export control discussion itself, but I think this specific style of argument only applies to China, because if you don't export the chips to Europe, they're not going to build their own chips in five years. They're just not going to have any chips. So it's A lot less geopolitically risky to not export the chips to Europe. And it's a lot more geopolitically risky to export the chips to a project that actually says we're going to reach frontier parity. Right. I think the basic motivation here is just, well, they just said that Fable and the cybersecurity capabilities, that is so scary that they don't want it on the open market because they're scared it's going to get jailbroken. So why exactly would they look at a European project that says, well, we're going to build our own Fable and then we're going to use it for whatever the hell we want and you guys can't control it anymore and then export the chips to them instead? I think that's just, just very, very unrealistic in the long term. So you do need some leverage to secure access to both the coding agents and the chips. I think there are ways to do that. I think the main way to do that is to just use the semiconductor supply chain links that these countries currently have. So there are most obviously the thing everyone always talks about is ASML in the Netherlands. If you can get Japan and South Korea on board as well. You get some high bandwidth memory, you get some more semiconductor manufacturing equipment into that, you get all these things that Nvidia and the US are very reliant on, eventually build the chips they need and sort of get the chips online they need. There's probably some dealing and threatening to be done around that because there is eventually a mutually beneficial setup to be had here. Right. The Europeans keep expanding and the coalition keeps expanding its supply of these semiconductor manufacturing equipment and all these sort of upstream inputs continues giving them exclusively to the US and the US turns them into chips and then in return, the coalition gets to buy some of these chips for its own use. I think that's basically a pretty positive outcome. I think it's less good for the US than the outcome where the coalition gives them all the manufacturing equipment and then the US just uses all the chips it builds with that just for itself. But I think it's a better outcome for the US than if the coalition just decides to turn off the supply to limit the supply or even the limit starts threatening to export some of the supply to China. But I think there are things that you can do as a coalition to threaten around the supply of these semiconductor manufacturing things that don't only target the US but that also target Nvidia. So they just create incentive on the US And Nvidia side. To keep the favorable equilibrium of exclusive access to these semiconductor assets. And then in return they give you some of the chips. And I think if you're sufficiently coordinated to actually play hardball on some of this stuff, I think there is an incentive and the path of least resistance for the Americans to keep giving you the chips. And then realistically, if you look at sort of what the American appraisal of so sort of middle power government led project to build AGI would be, I don't think a lot of people in D.C. or SF would take this very seriously. To the extent that they think we have to stop this at all costs, I think they will realistically think this would probably go the way of all the failed catch up projects before. So they might just not care that much about stopping the chip exports early. And then by the time that the chips are online and the project is going and things are actually going well, it might be too late to claw things back back. And then the leverage just needs to get you over this initial phase where you buy and get online the chips.

[A]

What would convince you that okay, it's go time, let's actually pursue the project now rather than settle for compute access deals or anything else.

[B]

So I think there are two parts here. The first part is political willingness to do this, and the other part is just government and policymaker interest in doing this. I think right now I'm not sold that this is the right thing to do. Just because I still am a big fan of the sort of allied scale integration comparative advantage. It just appeals to the sort of efficiency maximizing part of my brain very much to just have the US do what it's good at and then have the allies do what they're good at. And then we put all that together and then that's just the best possible outcome that we could come up with for sort of an alliance wide post AGI economic and strategic structure. I just very much still like to do that. And I feel like it's not dead yet. And I think is needed now is for the allies to make a sufficient contribution to incentivize that and then for the US to respond rationally to the incentives that are set by actually sort of effective allies that are doing well at building out these capacities. So I would very much still like to try that and I would very much still like to see whether that works. And sort of cutting off the bilateral before we've really even tried, that would be pretty sad. That said, things are going a lot faster than I thought. I thought we'd have another year to set up this compute for access stuff at least and get some of these deals online, get some of these corporations online. The fable thing, even if it is a red herring, has shaken up a lot of people and has made a lot of people more worried about this. And it has at least now very much made this idea of export controlling models, whether it's weights or whether it's outputs a much more realistic prospect in the policy discussion in dc. So the conversation on that is moving a lot faster than I would have hoped for setting up this mutually beneficial alliance. I think that's one of the problems. Problems. And then the other problem is, well, how much political awareness do you really have to get started on this? And I think realistically the political awareness is always going to lag what is going to be necessary by at least a few months. That's the problem right now. I've written this piece from the perspective of this week. I think if we started tomorrow, we'd need the 500 billion. If we started in six months even, and if we started a big tour through all the capitals now and we met with all the treasury ministers and some somehow sold them to take like 100 billion of sovereign debt each and put it into a project that they don't know will work, which is pretty big ask. If we only took six months for actually authorizing this budget and getting the project started, I think we'd already have to increase the price by at least 100 billion again because it gets harder and harder. It gets harder and harder to compete for fewer and fewer chips that are going to be more and more shipped by then. It's going to be harder and harder to convince the labs to give you access to somewhat leading coding agents to build things as the labs pull further and further and further ahead. And it's going to be harder to avoid U.S. retaliation as U.S. political awareness of this risk and of the strategic relevance of these systems keeps and keeps increasing. So 500 billion is the price tag now if we convince people to do it, we're going to have to tell them in six months that well, now it costs 600 billion. Then we have to do another round of negotiations. And by the time we've sold them on getting 600 billion online, we're going to need a trillion. And I think there is just a very real way in which the political awareness is just lagging the necessary moves too much. And I think that is part of why I think it's probably just not going to happen. It's probably not going to work. And what would convince me on the political side if is, well, people actually want to do this. And what would convince me on the strategic tactical side is, well, the compute for access allied scale type of thing is not actually working. The US doesn't want to go for it. The US stops these deals from happening. The US restricts the outflow of chips even to sort of allied data center structures, this kind of thing. So if our play fails and if political awareness magically fixes itself, we should go for this. Until that happens, I think I'm still, still fine or at least hopeful to do the alternative play instead.

[A]

Okay, one question is why is this so catastrophic? So the US has frontier AI before the rest of the world does, but the US already outmatches most countries on most domains and they're doing kind of fine. Why is this so bleak for France or England or anywhere else?

[B]

Well, I think the question is what can you do with two or three or six month lead time on all the secrets of the world, right. I think think if you just have the cyber capabilities now, the effect is somewhat limited. But you could already see the Europeans freaking out about this, and I think rightly so. And then I think we expect models to be able to do more things well than just cyber, right? And I think a lot of them could matter a lot economically. I think there's an open question here, which domain specifically in which domain specifically the frontier matters that much. But I think R and D is one of the other obvious examples, right? If you have a few months lead time on innovation and you find out what material you're supposed to use a few months before the Euros do, and then the Euros find out as well and they build their thing and sort of it takes 24 months for anything to get off the production line anyway, for anything to get shipped anyway. So really what do the few months matter? I think that's a realistic near term future where the lag is just not all that relevant for the R and D stuff, for example. But the faster the world moves and the more integrated these systems get, the more automated manufacturing get. Maybe there is ways to sort of lock in these innovations in a way. For example, you can register patents for all the innovation that your AI finds before the Euros get around to them. Or you just get quicker and quicker and quicker iteration cycles on building things and testing based on them. And the, the faster the world starts moving as we integrate these systems and as we automate more and more things, the more you would expect these initial advantages sort of compound because Then you build a better thing, then you check how well it works. If you build a better thing, then you feed the data back into the system and you try what the system makes of that. And then you feed in the innovation again and again. And so you just get to start the flywheel and you get to start the exponential of this innovation just a few months earlier. And you're always a few months earlier. So your product is always better by a few months. And I think if the world just moves sufficiently fast forward for that, that does compound into a pretty substantial advantage.

[A]

But I mean, the world you paint where, okay, if most of the marginal returns on intelligence are in areas like cyber or R and D, then, I mean, Europeans just get access to better and better products from American firms. Our purchasing power probably goes up if these products are also being able to be produced at cheaper levels. Again, that doesn't sound catastrophic to me. What's bad about that situation?

[B]

I mean, I think this is not catastrophic in the new near term. I think just general increasing economic growth and general increasing innovation and productivity usually starts out being pretty good for the world in an open market. I think there are two conversations here. The first conversation is, well, how does that lead to adverse economic effects? And the other question is how does that lead to sort of bit by bit, relative disempowerment? And I think the economic part of it is not as catastrophic as you say say. I think the relative disempowerment stuff is a problem though. So these frontier AI systems themselves become a really important input into your economy. You would think that they pull away further and further and further from the AI systems you yourself built in your country for the same reason that the R and D effects compound the software R and D and the AI R&D effects also compound. Right? The Americans not only build better models, they also get much better at building their own AI models because they can use their own AI models, AI to both sort of broadly improve their broader AI supply chain from everything from chips to data to everything. And because of the sort of more narrow recursive self improvement effects that might start within the labs. So between those you just expect the American software to pull further and further away.

[A]

Why is the open source models not compounding at the same time, at the same rate?

[B]

I mean, A, you already have a growing divergence in compute. That just means that the American labs can move faster, can explore more new innovation, can sort of, you know, can just spend a lot more compute on building better models. And then B, because there are these recursive effects inside the labs. They can reinvest all the revenue they make into building more infrastructure, and then their models get even better. They can reinvest all the usage data, they get into making even better models. And of course they can just use their superior models, deploy them internally, use them to make their models better, but then prevent other competitors from using the models in the same way. We already see this with Fable, which is limited for frontier model research specifically. Specifically, which is, I think, something that we're going to see more broadly, because why would you build a tool that helps everyone else make models that disrupt your own business model? I think at some point they will stop doing that. And then I think the question is, and the challenge is exactly the other way around, which is like, why on earth would you expect this four month gap in open source to remain in place forever? I think the specific mechanism that has led to this in the past was just there was a lag because you could see where the frontier was. And the frontier was sort of moving four months ahead. And you could sort of see, well, what's the shape of the frontier, what kind of models are they building at the frontier, how quickly is it moving, what are the capabilities they're focusing on, what are the specific capabilities they're throwing all their RL resources on, what benchmark shape comes out when they build a new frontier model? And from that you can sort of guess like, what are we exactly chasing here, what are we going for? What are the next architectural breakthroughs that we have to find as a fast follower and so on. To say nothing of the fact that I think a lot of fast following is also about outright distillation, which is even harder to do. And so if you get to this point where the frontier A pulls away even further and also B becomes a lot less visible because some of it gets restricted, restricted, then it's just a lot harder for a fast follower to even know what they're supposed to be doing and just find the road to the frontier. And if that's the case, and they also have less and less compute relatively to the frontier, I would just expect that gap to open up further and further. And so I do think we should expect the rest of the world not to have their own fast followers that can really keep up. So the frontier models stay relevant input to these economies. So you have a lot of reliance from that because you need at least the frontier or the sub frontier models from America as just like your basic economic input, but you also just need all the AI produced goods that they're giving you, right? To the extent that your economic mechanism is, well, we'll be fine because we have something to offer to the Americans. Like, you know, like the Amalfi coast or like sort of Milanese suits or like whatever you might come up with, right? Or like maybe even like sort of very sophisticated machinery that integrates the AI very well. There is still asymmetric leverage, right? And the US could cut you off at much lower cost to the US than you could sort of be able to threaten the US and, and who's to say you can go into pretty crazy futures from there on. And I think at the limit, there are just very few things that you, as a relatively disempowered country can do to stop this power concentrated country from doing basically whatever it wants. Maybe land on Earth becomes very scarce and they decide they really want more land on Earth, and then what do you really do about that? Maybe they would like your country as a military base on the way to somewhere else. And then you can't really say anything about that. Maybe they want a natural resource you have, and then you can't really say all that much about that either. I think if the relative leverage becomes pronounced enough, then at some point it's really difficult to look at these European countries and still think that they're sovereign nations in a relevant sense. And I think that is a realistic threat if you just have a relative growth divergence. Even if the Europeans are fine on an absolute level, as you describe, I think at some point of divergence, the relative geopolitical leverage is just bad enough. That is still a pretty existential threat.

[A]

Let's go back to the world where you don't go for the project, but you have ensured continued access to the frontier model in that world. How is your economy capturing value when most of the labor is done by AIs created by American AI firms?

[B]

I mean, maybe the labor is just sort of commodified in a way that makes the labor not matter all that much. Right? I think the AI agents are just like one input into your economy in that world. Maybe it's a high revenue input, maybe it's a low revenue input, maybe it's a high margin input, maybe it's a low margin input. But there surely are going to be other things that are also important to contribute to this. Right? The AI agents do the labor. Labor, but what do they need? They probably need, at least for the start, a lot of proprietary data that you feed into them. And as they continue to do the work, they probably Want to interface with production that generates more data as they try things out. The very classic loop that you might imagine here is you have a super modular, super automated factory that builds whatever, let's just say it builds robots and it's connected to US Frontier models. And ideally you have this US Frontier data center here, you have this robotics manufacturing plant here. And they sort of correspond and coordinate very closely. And the model comes up with, well, this is how we should ideally sort of build this component. And then ideas of the AI are sort of implemented in the factory and then a lot of sensors and a lot of monitoring in the factory sort of captures how that exactly happens, sort of feeds back into the model and tells the model, well, this has worked. This hasn't worked. The model comes up with another way to optimize the supply chain. It sort of feeds back in the factory. The factory sort of corresponds to that and sort of makes the modifications and there's a feedback in. And at the end you have just very good robots coming out of this process. And that's just obviously a valuable economic asset. And the question is like, how big is the role that the AI company has played in that and how big is the role that the manufacturing has played in that? And I think it turns out it's almost as hard to build a factory that can do this as it is to build an AI model that can do that. And I think a lot of the world is going to be bottlenecked on factories that can do this, because you need a lot of inputs for that, right? You need the factory to be staffed by people that are good enough at making the modifications ad hoc. That AI system is telling you to. You need a factory that is good enough on sensors and all these reporting loops that it can actually give information on how well the AI ideas are working. Back to the AI. This is just a very rare asset and very important asset. And I think if you have that, it probably generates a lot of revenue for you. This is the idealized version of industry that is highly AGI relevant, that is integrated into a lab. But I think you can come up with a bunch of other small ways to do just the same thing. People that adjust providing the way that they provide services, the way that they advertise services based on their integration with AI labs. In the same way, you can come up with things that are maybe not quite as AGI built as robotics, but they are still just important to build. Maybe it's drones, maybe it's just random hardware devices that are used to whatever you want to do basically any industrial output in the world you can integrate into AI in the same way. And it turns out these industrial assets could just turn out to be an important bottleneck. And I think the same story in a slightly broader sense can be told for a lot of other downstream bottlenecks that can be integrated with AI. AI and that maybe just captures a lot of the value. And then there are a lot of things that just don't interface with AI that much that also capture a lot of the value. This goes back to the Baumld stuff that we've talked about, right? I think once you have your economy isn't super disrupted. Once your country is somewhat stable, once your governments are sort of doing fine, you can just come up with things that are scarce. As AI becomes important, you can create products that people still want. You can sort of goes back to the tourism and the music and the artisanal goods and all these things. All of these things still capture substantial value. I will say it's sometimes overrated the extent to which that is important. Even in countries that we think of as very tourism heavy tourism rarely makes up more than 10% of their GDP. So it turns out you just need to do all the legacy things that your economy is still doing. In some ways I think that's going to be somewhat harden the limit. I think there is just a lot of services industry that is susceptible to being displaced by AI events potentially. And I think there the question of saving some of your production and saving some of your economic activity is more about comparative advantage. Why exactly would it be worth it for AI agents to completely replace some random aspect of the services industry in Germany or in Greece or whatever? Instead of investing this marginal inference budget you have this marginal RD you have. Why are you spending this on displacing random German services industry instead of doing things that countries have geniuses in a data center like crazy innovation they could come up with. It's just very unclear that that's the thing you primarily do. And so I think a lot of this is also just a bet on the division of labor shaking out that way and some of these activities still staying around. And I think in the limit if they all get displaced and if all the service industries just get done by cheap and abundant AI, I think there is no good answer to that. But then there is also no good answer to what the US economy does all day. Yes, they probably get to tax the agents or whatever so it's not as economically catastrophic for them. But as sort of as a matter of economic structure disruption. I think if the AIs just do all the jobs, there is no good answer to what does your economy do? Because currently economists still do hinge on doing all the jobs.

[A]

Okay, what policies do you think this nets out in right now for middle powers? So if what you want is to own the robot factory, does that mean Germany should be banning Bezos new company from buying out new German firms?

[B]

Like definitely, definitely Screen for the fdi because. Because I think this is a very risky situation right now where I think a lot of outside companies are starting and a lot of US based actors specifically are starting to become aware of how important the bottlenecks are. But the countries in the rest of the world haven't internalized how valuable and how important these things are just yet. So there is a lot of incentive for outsiders to come in and buy up all these bottlenecks and then sort of productively redeploy them once the assets and hub become more important. The Bezos fund, that Prometheus thing that you've mentioned is I think one obvious example. They just buy all the industrial cap and then they sort of redeploy it when AI systems are stronger. And obviously you shouldn't sell these things right now. Obviously they're just like if you think about them now, and if you think about selling them now, a price that something like the Bezos fund would be able to offer or something that basically any sort of investor that would be interested in running this play would be able to offer. Sounds great. If you're not sufficiently aware of what is coming in AI, you get a few billion for an asset that you thought was only worth a few hundred million in the current economy. That sounds crazy good to you. You sell those data. Well, this is just a win for your family, set for life. That sounds amazing. But it turns out alternatively this would have been your slice of the light cop. This is a super important piece of manufacturing. This would have secured your stake in the AGI economy for decades to come. And it would have been worth so much more than a few billion. And this awareness gap and how much these assets are worth is a very, very threatening phase of the AI economy transition right now. So yes, I think you just need to get ahead of that. You need to screen for foreign investments into this. You need to make sure that your important assets aren't taking over. I think think that's the very baseline of what you should be doing as a country. I think you can also think about how you can make these assets AI compatible because As I said before, I think the risk here is it's just become so hard to work, for example, with a manufacturing that is AI compatible, that it's just more efficient to just rebuild this thing in the US to begin with. And I think to make it efficient and almost sort of bindingly interesting to the AI labs to cooperate with your robotics factory somewhere else in the world, rather than just rebuild it in the US US is to just make it AI compatible in all the ways we just went through. So make it sufficiently modular that it can quickly react to what the AI input is, have workers around that is sufficiently skilled to adjust the production process to whatever the new idea of how we're doing this now is. Actually manage to not regulate yourself out of providing the data to the US developers in a way that creates these fast feedback loops, make sure that the plans are sufficiently set up with sensoring equipment, and all this kind of stuff that actually reports the data back. There are are AGI compatible production plans for a bunch of things you can build right now that are extremely enticing and attractive for cooperation, both on a geopolitical level for the US Government and on a commercial level for the AI labs. If you make these changes now, you have a very, very good position to then ask for these corporations and integrate. If you don't do that right now and you just have a basically defunct plant somewhere that is still staffed by people that don't really know what they're doing, that isn't really modernized in ways, then, then the pitch becomes a lot harder and then the only thing you might be able to do is sell some of your IP you still have, sell the physical plants, and then have the US take them over, reintegrate them, eventually indigenize them. And so I think the way you stay alive is just anticipating where this future is going and making sure that whatever your production process is actually becomes compatible with that. I think that on the specific industrial policy stuff is I think, the most obvious intervention. This is the sort of micro example for the broader economic structure, because that's the sort of cleanest and most direct integration example. I think one slightly more abstract, abstract version of this is what you do about your labor market. I think currently labor markets in a lot of middle powers aren't very well set up to deal with disruption and quick iteration based on what AI systems do to the labor market. To the extent that there is a future labor market that still includes humans in important roles along bottlenecks and things that AI can't do very well, and you have this idea of you have this very jagged frontier, and the humans sort of fill out the spots where the AI isn't particularly good and the AI doesn't. The thing that the AI is very good at. Moving into the sort of dips in the jagged frontier of AI capabilities requires your labor market to be extremely flexible, right? If your labor market isn't very flexible, then people just stay in their jobs, even if AI is better at their jobs. And then you have huge efficiency losses. People don't move into the things that they would otherwise be good at quickly enough. Your firms become uncompetitive because you're not doing a particularly sort of efficient distribution of the workforce across the task profiles that your human workers would actually be good at. And eventually you just risk being disrupted in a few ways. Either fast competitors that are just integrating AI agents entirely vertically just start disrupting you because you are just not using AI to its full effects. And then at some point, just these quick and more agile competitors just start out competing you. And then your entire business model crumbles. And then you actually, just like you have to do mass layoffs, your company just goes under, things got really bad. Or you just get a scenario where are, well, other countries in the world that are better at having a more flexible labor market that are allocating their workers along the jagged frontier more efficiently. These firms just get more efficient, and then they outcompete you internationally. And then you have the same effect. Your firm gets under pressure, maybe it gets outcompeted, maybe you get forced to just put up higher entire trade barriers to compete to protect that firm. But either of those are pretty bad outcomes. So what you need to do is you need to make sure your labor market is flexible enough that the workers can move to the sections of the jagged frontier where humans will still be required. So you get the general economic efficiency gain chains, and you don't expose your workforce to the broader international disruptions from AI agents just being deployed everywhere. And that's a pretty counterintuitive finding, right? Because in a world where AI agents get really good and you get really worried about the workforce displacements, the first thing you might want to do is lock up the labor market and make sure people can't get fired for now. And make sure that people stay in their place and stay in their positions, and make sure that the disruption doesn't get too bad too quickly, because that's politically really risky, that's socially really disruptive, and that is having mass layoffs and having people go to 6% unemployment for some time. 7, 8, 9, 10% unemployment. Some of the labs are calling that is extremely politically scary. But what's even more scary is, well, we lock this labor market in place, we stay at 5% unemployment for another year or two until our firms go under, then they do mass layoffs, then we end up in 15% unemployment. But it's also the case that none of these workers can really reallocate into more useful jobs after that. And so I think you have to bite the bullet on the current count. Intuitive conclusion of make your labor market more flexible as the AI disruption comes in. Otherwise you risk sort of going to the second best, very inefficient structure in the short term and then get really disrupted in the long term. I think maybe one framing to think about this is that this is sort of race between automation and augmentation, where you have this threat of this impending automation. Labs are building these agents. You might use these agents to build entirely vertically integrated firms that need very few humans. And what you want to do is you want your firms that still employ a lot of humans to stay competitive with these vertically integrated, sort of very agent heavy firms. And you need to do that by moving towards the most efficient distribution of labor between human workers and AI agents. And if you don't do that, you lose the race between the augmented firm and the automated firm. The automated firm wins and you have no use for the workers anymore. And so rather try to win the augmentation automation race. Try to incentivize a flexible labor market that, that makes humans move into the parts of the jagged frontier that are actually human favored. And then you can deal with some of the disruption.

[A]

How do we do this without causing too much social unrest? Is there anything else we can offer?

[B]

Well, I think you just have to buy off the losers of this trend. Basically. I think this is going to lead to localized disruptions. There are inevitably going to be people who are currently very good at the things that AI agents are going to be very good at in the future and that are only good at these parts of the narrow task profile. They're not very good at selling software. Maybe they don't have, have lots of great research tastes, but they're very good at executing on building software. And these people you need to compensate in some way. You just need to buy them off. Basically these are the people that generate your social unrest. It's not trivial how you would buy people off. It's not entirely clear. You can do something like wage guarantees and wage insurances has been done in Past globalization shocks, digitalization shocks to economies. That has worked okay, but it still has created these pockets of political resentment. You can't actually get around with it just by giving people money. I think some people have floated the idea of giving them job guarantees instead and just ensuring that these jobs stay around, but basically just subsidizing them into existence. I think in the short term that is not the worst idea. But I think there are real limitations to how well that works in this specific case, which is mostly because you can do wage guarantees and you can do job guarantees fairly well if the only thing that people care about is their actual job as they have it right now. But a lot of the disrupted workers we're talking about here here are in their early to mid 20s. They're not interested in their job because they want to do their specific job. They're interested in their career trajectory and they want to get somewhere with this. And if you just give them their wage and their job forever now, they're not going to. No one who's 23 years old and doing software engineering is not good. I just want to do this forever. I just want to sit here and execute doing this code. No, no, no. I don't ever want to sort of climb up the ladder. I don't want to ever make more money. Just give me my wage guarantee. It's very different than a 55 year old guy who works in an industrial plant that you can just pay off and be like, well, you get the wage you got, you can stay at home for most of the day, you can work for 20 hours. This is the deal that a lot of western economies try to make with displaced industrial workers. It also didn't work great, but it kind of worked because those weren't upwardly socially mobile workers that really wanted more out of their career. So I think the tricky thing here is you don't only need to buy off people's current stuff state. You need to give them something that is sufficiently attractive to slot in for the career they used to have. I think one of the things we can do about this is hope that if you just keep them around by force with something like a junior job like subsidy, which incentivizing firms to keep them around on top of the AI agents they use, they just pick up enough along the way, they just learn enough along the way that they eventually become good at other things and they become good at things that will be valuable in the broader context of this augmented economy. And for that you just need to get them through these first few years where they probably don't have a lot of useful skills to offer and they probably primarily have things to offer that are good for AI agents. So maybe you can brute force your way through that with some sort of subsidy. I think that is a start and I think that gets rid of some of the disruption. And then the other question is how much of the disruption just happens as a result of of generally bad vibes. And I think that you just can't get around if you have these AI systems entering the economy at the same time where you just have bad macroeconomic indicators and bad macroeconomic sentiment and you just have high ish unemployment rate, even if it's for unrelated reasons, people are just going to hate this trend. No matter how good or how bad the specific safety nets for the AI disruptions are, the vibes are going to be bad, the stories are going to be bad. There's going to be political incentive to point at AI systems as the reason for these distributions disruptions. And if that happens, I think there's just not all that much you can do. You have to try to counterbalance with some social policy interventions. You have to try to counterbalance with some economic, I think at the limit. This is just going to take courageous leaders that say, well yes, this is hard, this is economic hardship. We're still going to have to go through this because if we don't do this, the equilibrium we get to if we don't do this is just even worse. And so just grit your teeth. This is pretty bad economic situation, but this is the only path to a better economic situation. And then you just hope that these leaders don't get sort of politically out competed by someone who has a better message of well no, this guy's selling you guys out to the AI companies. We're just going to shut down our market. We're going to keep the AI companies outside. Screw these guys. We're just going to do the firms as we always have and we're just going to put frictions in. We're going to pass human in the loop laws, we're going to pass laws against AI agent adoption. We're going to institute a particularly restrictive token tax that disincentivizes AI agent uptake and you can come up with all these restrictive policies that ultimately just make you lose this automation augmentation race and mean you get really disrupted in like two or three years. But these are going to be attractive political solutions and getting around them is just a matter of beating them in the political arena. That's going to be hard, but I think it's going to be necessary.

[A]

What's your preferred way to pay for these junior job subsidies or whatever else it is that you're offering? I think you said you prefer a corporation tax to something like a token tax.

[B]

Yeah. So I think, I think the general worry here is anything that narrowly targets AI use makes us just more sort of makes us lose this automation augmentation race even faster. Right. Because what you want is you want maximal uptake of these AI agents at the things that they're good at in the legacy firms that exist right now. Because otherwise these firms get less and less efficient and ultimately do get displaced by either international or national competitors. So anything like a token tax that basically tells a corporation, well, if you use more AI, we're going to tax you more. I think that makes it very hard and I think that just disincentivises the kind of adoption we want. The other thing you might think about is can you tax the AI companies and the AI developers specifically? And I just think maybe that works. The maximum version of that is taking the equity stake in the AI developers that the US government has recently started thinking about. Bernie Sanders has written about a lot and then the President has also started, are looking into this, I suppose and calling meetings on this and they're having conversations on this. So we'll see. No, I think it's a very bad idea because a. I think it's a bad idea for, on narrow economic grounds, mostly because we're just not sure whether it works. Even if these AI companies become extremely valuable, like 10x or like 100x their value in the next few years, that's still not even remotely enough to pay for large scale social disruptions. The economy is a lot larger than the AI labs right now. The economy is still going to be a lot larger than even AI labs that become 10 times as big as they are right now. So whatever equity stake you have in these labs is not actually going to be enough to pay for any of this. So if it isn't enough to pay for any of this, you still need alternative modes of taxation to pay for part of it. But then by taking the equity stake, you just create a massive misincentible for the government in basically doing anything to the AI market and regulating AI in any way. I know we've mostly talked about the economic part here before, but we also do have, have, we also do want AI models to be regulated in a reasonable way for other sort of risk related factors, obviously. And if the U.S. treasury and also just general U.S. economic incentive. And maybe a sort of Trump account filled with AI equity stakes or something is directly tied to the stock of AI providers. What kind of incentive does that place on the US government in terms of regulating models, in terms of making sure these models are safe? You're going to give a lot of power to these labs to say, well, you can do this, but then the Trump accounts are going to crash and how does that look politically? And I think you don't want to create this incentive basically ever.

[A]

What about other countries?

[B]

Well, I think the question is do you even get the equity stake? You can buy it. That's I think one way to do it. I think in the US they're still thinking about just ceding some of the stakes to the U.S. government. I don't think there's any other country where any ally would be interested in giving them stakes for free. So you just have to buy them. At this point, this is just a sovereign wealth fund with extra steps, but in a sovereign wealth fund that is much less diversified than any other sovereign wealth fund. So yeah, this is like sort of the stupid I only buy anthropic stock version of the sovereign wealth fund. And yeah, I think in the limit that works. That's hard to sell politically. Right. It's already hard to sell the idea of we'll take on a lot of debt and build a sovereign wealth fund, even though it is a good idea because turns out stock markets are pretty good at generating a lot of revenue. But then maybe you just invest into the broader stock market and you just do an actual sovereign wealth fund instead. And you probably put in the AI labs as a hedge. But you need a pretty big sovereign wealth fund to pay for widespread social disruptions like that. And so I think in the limit you need something that captures wherever the AI growth accrues. And I think it's a pretty good guess that corporations just like corporate income tax is just like a pretty broad net that captures a lot of value accretion at all kinds of different steps of the supply chain. Maybe it's the chips and then the tax hits Nvidia. Maybe it's the model development and then the tax hits the model developers. Maybe it's the downstream users and then the tax hits the downstream users. But we get some of it anyway. We just increase the corporate income tax. That is what we have right now. That is how we capture, I think a lot time of the the value. I think there are other ideas around that are even more elegant in some ways. I think a consumption tax, just like a very broad based consumption tax for whatever consumption is one of the other things that people have talked about. I just think I just have a political issue with that which is, well, it works elegantly, but then people open their doordash app and they look at their sort of order and then there's a $1 consumption tax on the breacher they order.

[A]

It's kind of aggressive anyway as well.

[B]

Yeah, that's just like. I think that's just not a great vibe for a new tax. And it makes it feel like the burden is on the consumers. Even though most of the consumption tax probably does fall on the companies instead. I think it's just politically more elegant to just campaign on this as well. Raise the corporate income tax. And that's just the thing.

[A]

Isn't there a big risk of capital flight also? If you have vertically integrated firms, they're not tied to a workforce in a particular country. Doesn't that just make it capital a lot more mobile than ever? How do you deal with that situation?

[B]

Yeah, I think as with many of these things, a lot of the basic low hanging fruit solutions that we already have for general issues in corporate tax and capital flight just become a little bit more important. You probably need global or at least between liances you need sort of minimum floors that just work. We got pretty close to that. That seems like a tractable problem comparatively to me. Just a 10, 15% minimum corporate tax rate floor across major allies that could potentially host this. I think these firms are mobile, but they're not that mobile mobile compared to just post box somewhere because it's just. Well, they do need a physical anchor. A firm that uses a bunch of AI agents needs a data center that runs these agents somewhere, needs access to a data center that runs these agents somewhere. So the fact that the most disruptive firms, we would expect them to have huge compute spends and the compute spends need to be anchored in a data center somewhere. You do get a revenue anchor for these firms in more general terms because they have to be linked to a data center somewhere. And, and then they can't really escape the sort of gravity of this data center in terms of where they're located.

[A]

But you also want to incentivize them to build your data center in your country too. So doesn't that cut against that somewhat?

[B]

Well, that's great, right? I mean you will probably be incentivized to give them some carve out to build a data center in your country, but it also means that they can Never entirely run away from the test. You get like minimal race to the bottom dynamics as you always do with company headquarters. But that's still like we currently have that sitting. And the floor is, is yes, corporations don't pay as much tax as in the ideal scenario where they all sort of sit in like the high tax countries or whatever. But it's also not a pure race to the bottom. It turns out not every company is listed on the Cayman Islands because there are a lot of factors that pull against it. And not that many countries can build a data center and not that many countries are willing to muster the political energy and the capital backstops and everything to build huge data centers. So you will have the data centers hopefully diversified across a spread of countries that isn't entirely incentivized to just put the corporate tax rate to zero. Especially if you have global minimum floor. And then you can't actually escape the pull of the data centers. Having the data center gives you some minimal tax revenue anchor. And the fact that this is also infrastructure bound means the capital can't actually fly away from the minimal tax rate anywhere. And so you would think that the economy just sort of smoothly distributes across the countries that have data centers that can serve as revenue anchors. And I think things probably shake out somewhat fine. It's still going to be hard to capture a lot of this value, especially if you compare this to labor tax taxes. It's just very easy to tax labor because laborers aren't particularly mobile and because we just have pretty high taxes on labor. All things considered, it's both direct income taxes, but then also all the costs that come with payroll, especially in some of the European countries. So if we move a lot of the revenue away from labor and towards basically capital, which is to say AI agents and the infrastructure they run on, we do get much fewer taxes. In total, you have rates of 25% maybe on corporate income and everything to do with the capital. We also give a lot of tax breaks to capital investments compared to labor investments. So we do lose. If we just naively move one unit of revenue from labor to capital and AI agents, I think we lose maybe half or more of the tax revenue we get off of that. So yes, there is going to be a massive revenue crisis for most Treasuries in the world. Yes, there is no good tool to think about this right now. And yes, at minimum, corporate income tax changes are probably going to be necessary at maximum. We have to basically rethink how we do tax revenue on private sector activity. All things Considered because right now it all hinges on fairly high taxes on human labour that we are not going

[A]

to be able to sustain within the us. To what extent or what kinds of nationalisation do you think are in some way inevitable? Like how do you expect anthropic to be governed by the time Claude is running robot factories?

[B]

Well, I think my mainline scenario, especially in the current administration, is there is no strong framework. There is, there isn't at any point like an EO or a bill that says, well, this is how we're soft nationalizing or hard nationalizing anything. It's just you have slightly more embedded observers that are linked to the US government within the labs that get what's going on. Maybe these are third party evaluators that report to the US government, Maybe these are just directly sort of placed assets from, for example, the intelligence community that just sit in the labs and monitor what's going on. And then you just have these feedback loops where the labs communicate more and more of what they're doing to the U.S. government specifically. And then there are people in the U.S. government that think about whether they like what's happening or not. And if they don't like what's happening, then the executive calls up the lab and says, well, have you guys considered not doing that? And would you like guys not like to do this differently? Because if you don't, then we have all this range of potential threats that we could come up with, but we never quite do. Whether that's export controls on the models or on the model weights, or whether that's sort of Defence Production act related interventions into the data center, or compelling the labs to turn over their model models, or compelling the labs to give information as to what they're up to and what they're doing. You have all these theoretical policy backstops that are already available to the executives today. And I think most of the nationalization that happens is just going to be the admin sort of vaguely gesturing at, well, you guys don't want us to pull on any of these levers, right? So maybe just do what we want in some unspecified way and then there's going to be push and pull, there are going to be sort of commercial forces pulling against that and the labs are going to say, well we'd like to, but this is pretty bad for our market position. And then the admin is partly going to say, well then maybe don't do the worst version of this and we're going to see a lot of messy negotiations like around the fable thing happening time and Time again, I don't think we're going to see a big framework for any of that. It's just going to be this continuing push and pull of the admin threatening to use some of these tools and then the labs just sort of acquiescing in some way and the labs sort of doing what the government kind of doing what the government wants in some way. And we just sort of try to figure out what the balance of power there is. And then at some point we're probably going to come to some sort of law. There is going to be executive orders that sort of come up with some minimal version of this that are going to sort of somewhat standardize some of these processes. But I think a lot of it is just going to be the soft back and forth between the admin and the labs.

[A]

Under what conditions would you advocate for something like a pause or a slowdown? Are you excited by this idea?

[B]

I'm not. I think there are two questions here. The first is what is the politically realistic version of a pause that we would see happening? And then there's the question of, well, what kind of policy intervention do we like in the abstract and I think think on the sort of politically realistic version. I think especially if this comes from the sort of general, oh, well, this stuff is really dangerous. We've got to get a handle on this. We've got to get all this coalition of people who all vaguely don't like AI for somewhat unrelated and uncorrelated reasons. We just got to get all of them together and we'll build this constituency. They're going to ask and push for the pause and then we're going to do the pause. I think this is very likely to not come up with a very good pause idea. And I think you need a very good pause idea for a pause to work right. If you just pause in the US in a sort of naive way, then of course the first thing you is you crash the stock market because that is just super dependent on the infrastructure investments that would sort of run directly counter to a pause. It's contingent on the IPO expectations of the labs and everything. So this is like domestically, the naive version is pretty harmful policy. And then you need all this international sophistication for the pause to not actually mean that all the compute re concentrates somewhere else, that China doesn't catch up in terms of semiconductor manufacturing in the meantime and then just builds the thing later on. There are all these complications that require you to have a very small sophisticated treaty in place and not Just the domestic pause. I think to their credit, a lot of the most intelligent pause advocates know this, and they have developed treaty architectures that they'd like to see. But I think politically speaking, it's going to be very, very difficult to get this kind of treaty done right. You have this broad coalition of everyone who doesn't like AI you get into the boat, they're worried about the job impacts, they're worried about the environmental stuff. And then you get them to the point where they pass a bill or where they have a majority for a bill that says, well, shut it down, let's stop this. Some vague sense of no more new data centers. Maybe let's tear down some of the old data centers, no more new AI models. And then the question is, does this coalition stick together to actually also negotiate the treaty, to actually advocate doing the most sophisticated version of this, to keep compelling policymakers? No, no. The domestic policy is not enough. You now have to get under the treaty. You have to get the Chinese on board, you have to fly there again. You have to do another round of negotiations. And if they're not going for it now, you have to give up more to get them into the treaty. Is this coalition really going to be interested in the nuances of the safely relevant part of the pause treaty, or is the broad coalition only there for the vague symbolic policy of we're doing the pause, and does the rest of the coalition jump off the ship at the moment that they've gotten the pause, and then you're just left negotiating for the treaty with. You have to get 60 senators on board for ratifying some crazy treaty. And that's just.

[A]

Isn't there a version of a pause, which is a bit more like essentially a slowdown, where what you're really doing is just reallocating compute? So you're making. Making commitments about reducing the amount of compute that you're using for R and D, increasing the amount of compute you're using for inference for state capacity or societal hardening or something like this. This is something you'd support more.

[B]

Definitely. I think that's where we get into. That's less of a pause, that's more of a slowdown. And I think that's also like. That is like sort of what sophisticated vehicle would be like in the abstract, which is just like, let's find some elegant way to reduce the pace of frontier capabilities specifically, and increase. Increase adoption instead, and increase rollout and societal hardening instead. I think that probably works pretty well if you can coordinate around it in some Reliable way. I think there is still a geopolitical issue to that, in that a symmetric pause in frontier capability development is not a symmetric pause in US China competition for AI. Because right now, China doesn't actually do a lot of frontier development. Obviously what they do do is a lot of diffusion, and what they do is a lot of catching up on semiconductor manufacturing. So if you pause specifically frontier development symmetrically throughout the world right now, and if you slow it down in some way, in some coordinated way, I think most of the Chinese ecosystem is extremely thankful for that, because the slower we play this, the quicker their semiconductor manufacturing catches up, the quicker their diffusion plays sort of work out through all of the economy, and then you have to

[A]

catch up pretty fast.

[B]

The question is like, how long is the slowdown? Like, what does the slowdown give you? What do they think? The question is, how much would the pause three years ago when they asked for it for six months would have really helped you? We would have still been like, well, we really have built that much resilience. What we really have in that time anticipated the kinds of things the systems can do. Now would we have passed the policies that would have worked. Now I think you have to do a pretty long slowdown to actually get on top of all the things and then also make the predictions of the things you would like to in the future get on top of. So the slowdown has to last for a while. And I think right now as it is, it would be an asymmetric sort of geopolitically asymmetric slowdown, slowdown, which is an extremely difficult task to make of the US Government in the context where they do see themselves in a geopolitical competition. You might think that AI is not only something that the US needs to narrowly win on, but AI is the advantage that the US has to win the broader great power competition. And the frontier lead is the one thing they really have ahead of China, plus the compute lead. And so if you institute a policy that specifically hits the one thing that American America is very much in the lead on, I think that's a very hard geopolitical sell. So that's probably something that the US Government is going to be very skeptical about. If that's something the US Government is very skeptical about, then you'll need broader Chinese concessions to make up for that. For this to be a fair deal, geopolitically speaking, you instead have to ask China to also slow down on semiconductor indigenization and on AI diffusion, because that's how you get a geopolitically even version of this case. And good luck getting China to do that, I think. And good luck very verifying not only the fact that they're not using their frontier development very well, but verifying all these unrelated industrial policy efforts. I think a lot of good work has been done on verification of a narrow pause treaty or a narrow slowdown treaty, where it's like, well, do we have some way to figure out whether the Chinese and the US are both sticking to this narrow idea of what you described, which is like, we put more compute in the deployment, we put less compute in the training runs, we don't exceed certain sizes of training runs, we don't put that much into the RL and everything. I think maybe we can come up with a way of verifying that. But can you really verify what kind of progress the Chinese semiconductor industry is making while the slowdown is happening and make sure that the geopolitical parity of the deal works out that way? I think that's a very hard case to make. I think sometimes the discussion about this underrates basically just makes the case, oh well, the US will still be ahead and the US is ahead and there's no way that China would catch up in that world. But I think that misunderstand the geopolitical dynamic here. The geopolitical dynamic is China is better at most other things that matter for great power competition right now. And the one asset that the US has right now is that basically much of its economy and also some of its strategic environment and an increasing amount of its strategic environment is a leveraged bet on AGI development. And if you slow down specifically on the AGI development, that is just a huge asymmetric hit to the US and if you ask that of the US I think it's just not going to work.

[A]

Okay, why not rally around a slightly more stupid but maybe still appealing idea of just ban on superintelligence. So it's not a pause, it's just like a permanent thing. And there's enough, this coalition's broad enough, the US is interested, China's interested.

[B]

Yeah, I just honestly have no idea what that means. I think that is just an incredibly underspecified thing. Systems. That doesn't mean that systems aren't allowed to be like super intelligent at everything. No alphazero or anything, because systems are definitely super intelligent at some things right now. Obviously they see human capabilities at some things things. So that narrow superintelligence band seems silly. I mean, I don't want to. This is like the lazy version of this Is do you want to bend calculators or anything? But even if you go into slightly more broader human domains, there are already important task profiles, even sort of strategically relevant task profiles, like sort of large scale data and image analysis and this kind of thing where AI systems are already superhuman. So we have narrow superintelligences, even strategically relevant tasks already. And what we don't have yet is like a broad superintendent that's better at humans at everything. But how do you operationalize that? Well, you just keep one thing behind the jagged frontier. It never gets quite as good as, I don't know, coloring, coloring books or something. And then it's not a super intelligence because it doesn't have the broad capability set. I think the operationalization of this is just very, very difficult. And I think you can just, even under a general superintelligence ban, build extremely dangerous sort of very jagged systems. And I'm not entirely sure that you're in much better place with that. I think maybe the most charitable way to operationalize something like is there something we can ban? Is maybe banning closed RSI loops within labs. Specifically basically ban anything that would realistically lead to an uncontrollable and unsupervisable software only intelligence explosion, or even that would just even ban anything that, even if it's not software only, that just bans closer and closer RSI loops. That just makes systems much better, much quicker than we used to to have. I think there's something to like about this kind of policy in that it doesn't only ban a thing that seems vaguely scary. There's a more principled reason for why you would want to do this, which is retain visibility into what these systems are doing ever quicker. RSI loops are just extremely difficult to monitor. It's very difficult to come up with a way that a government would actually have reasonable insight into the risks that emerge from this as the RSI is getting faster and faster and faster. So I think there's a good argument to be made that this should not be something with our current evaluation infrastructure, with our current oversight infrastructure that Labs should really be doing at the moment. Again, this is not going to be as obvious and as easy of a sell because this is one of the theories of victory for the US AI developers and therefore will also become one of the theories of victory for broader US China competition for the US like closing this loop and actually making these systems go exponential in their capabilities much faster than they other would, is one of the few ways you actually translate vague advances like vague few month advances in AI capabilities into very broad geopolitical leverage by actually getting to super, super, super intelligent systems that can both carry out all these important tasks, but then also maybe by themselves prove to be some version of a decisive advantage. And convincing the US to not click the decisive strategic advantage tomorrow button is I think, still going to be extremely difficult. But yeah, I think especially the strong closed loop versions of RSI are maybe the most easily operationalized ways of thinking about what we should ban or slow down or prevent. And I think I'm sympathetic to doing that.

[A]

How do you personally feel about the prospects of an American decisive strategic advantage, whether or not it's possible? Is that something that would actually be net positive for the world in your view?

[B]

Yeah, I think I'm still unsure about what exactly, exactly a decisive strategic advantage is. And I think you can come up with softer and harder versions of it that work or do not work in the real world, but taking the strong framing of it, which is basically just like you basically get unilateral power over doing to any sort of strategic rival or any other country, basically sort of enforcing your will on them in any potential way. I think I'd feel a lot better about a Western alliance with a strategic advantage with the DSA that than just an unconstrained US I still have, I think, a little bit more faith in the ultimate balance that will shake out between the US and the labs. But I think there are a lot of ways that US decision making around how to deploy this could go very badly. I think we've seen the administration is very volatile in how it chooses to deploy its assets. I wouldn't be entirely sure that it deploys this asset very well. I think we've also seen unfortunate tendencies in the securitization of the labs that would lead to this. I think there are a lot of, of unfortunate amalgamations between the labs and the US security apparatus specifically that you might come up with that are maybe favorable to some narrow version of American strategic interests as determined by whatever the political mood of the day is in the administration, but that aren't necessarily furthering the world very much. I think, that being said, if you think this advantage is something that is slightly constrained within the context of both a separation of power within America between both the sort of classic constitutional separation of power, also the separation of power between the private businesses that build these models and the government that sort of oversees how they're used, and then also a broader sort of separation of power within the broader Western alliance that controls its advantage. And there are some more checks and balances on how exactly it's deployed. And it's mostly sort of like an ultimate enforcement stick and sort of like a last piece of leverage that just wins geopolitical competition. I think that is a much more tenable future that I'm much less worried about because then you get rid of all the sort of. You still get all the positive effects which are sort of. You get AGI deployed or ASI deployed in a way that is sort of ultimately backstopped and backed up by functional, somewhat functional liberal democracies that sort of control how it's used. That seems pretty good. And that seems like a better way to control how these systems are deployed than basically any other way we can come up with. It also seems like broadly geopolitically good and also geopolitically stable for this advantage to exist, but to rest in the hand of a broad coalition that has a decently high bar of adversarially using it. So the best version, the alliance and separation of power constraint version, I feel good about the narrow version where you have a closer and closer amalgamation of Labs and US government and then this big public private partnership or whatever of Labs and usgs. You just can use this on a day's notice. I think that I'd rather not happen.

[A]

And from the perspective of risks from loss of control, is there a reason to believe that middle power is being involved, having access to compute? Is that going to be helpful? From a safety perspective, I would think so.

[B]

My general view of this is what it takes for us to lose control over these very powerful systems is fundamentally some people making stupid decisions. Well, just on the policy front, I know that the technical problems are hard, and I think the technical problems don't only come down to people being stupid. I think we will have to actually figure out a lot of the technical stuff that we haven't figured out yet. But on the policy side, we need

[A]

to stop people from releasing Mecha Hitler,

[B]

choosing to deploy a system that's not safe, giving it abundant access to compute and the Internet and all these things, all these classic preconditions to loss of control. They do also come down to just bad policy decisions being made around both the government governance of the developers and also the sort of governance of the specific systems. So the question is, in what environment do you get the least density of stupid decisions that can really screw the violence? And I think the chances are, I think whenever you concentrate power within a single volatile actor, the variance of the decision quality just obviously gets higher. Right? It gets less constrained, you get more daring sort of moonshots and cool things you might deploy these systems with. So the, the upside of just letting one guy run well with the ASI is maybe even higher because it's less constrained by process and whatever. And you can do risky bets, but the risk of just doing something monumentally stupid with a very intelligent system is also much lower. So buy is much higher if you have the crazy guy, and it's much lower if you have the constraint version. So I think just like if we kind of narrow the option corridor a little bit by involving more people in the decisions of how these systems are deployed and more people can sort of unilaterally shut them down and some way or unilaterally threaten to blow up the structure that makes these systems possible by withdrawing a bunch of the compute, withdrawing a bunch of the supply chain inputs or whatever. If there is more mutual control over what is happening with these systems, I think the band of outcomes just gets a little bit more narrow and we avoid the stupid policy decisions that seem to me to make up a lot of the policy risk for loss of control. So I think just broadening the inputs just seems robustly good in that case. And then on a much more general note, I feel like just having middle powers in the room and just having sort of geopolitical leverage more evenly distributed just is a less chaotic and volatile world in a lot of ways. I think if the US pulls ahead, but the US pulls ahead alone, and the rest of the world sort of gets disrupted in all the ways that we've talked about. The labor markets get disrupted, the strategic position gets seriously threatened, people lose their jobs, people get hit by AI misuse every day. Every day they start protesting on the street. There's mass migration flows happening because they want to move to countries that actually have access to the defensive AI that keeps them out of that countries get into conflicts because one side of the conflict has access to good AI systems and the other side doesn't have access to these systems, and these temporary technological advantages reopen old conflicts and make for an opportune window for one country to strike against the other. Of all these scenarios where a world that doesn't have equitable and good and sort of mutually beneficial access to AI systems just becomes a volatile and chaotic and messy world. And I think just from a very sort of general perspective, that just doesn't sound like a world where we'll be more likely to make good technical and policy decisions than the world where we have pretty sort of complex and well functioning supply chains. People are generally not entirely discontent and disrupted by the situation. There is less AI misuse happening. The world just generally seems less like it's going to hell outside the border of the us. I think that is just robustly. If you just think about which world would you rather make the ASI policy in and make the important decisions in? In what world do you think the people in the room are going to respond to better incentives? In what world do you think the decisions that are going to be made are going to be more sane and less reactive to a world in crisis then? I think you just want the more stable world. And I know that that's not the sort of finished causal model that people usually like when they talk about these things and when they talk about the impact that work specifically has. You usually want this very step by step by step. And we built the compute in Japan and then therefore the ASI rather does this. I can't give you that link, but what I can give you is just like, well, I can give you an offer to build a much more stable world that does much better on any basic categories of what we would want the world to look like and what world we would think good decisions might be made in. And then I put to you, I think that sounds like a much better world to make these important decisions in. And I think that's good enough reason to work on this.

[A]

And what are you doing in this world? Are you just going to be writing all the way up to the singularity?

[B]

Do you have a plan if people keep reading? I think there is a real open question that anyone who's doing writing and work in the outside game right now is facing, which is how is the outside game changing relative to the inside game, where I think a lot more information that is basically only available within the labs and within the governments is becoming more and more interesting, relevant. The labs have exclusive access to the best frontier models and they're going to like the gap between internal and external models is maybe going to increase. The information siloing is going to get worse and worse you would expect as more of these things enter national security relevant domains. So I think fewer of the discussions are going to happen on the outside. Dean Ball had this piece, I think it must have been two years ago now, about the sort of republic of letters that we had when the SB 1047 was being discussed and everyone was sort of writing their little essays and writing their little tweets and people were going back and forth and it was all an open discussion and that was there was a battle of ideas. And you could make a real contribution by writing about this and saying things about this. And one by one, the people writing the letters are choosing the inside game instead. And the Republic of Letters is sort of getting starved of both its core input, but also of its core output in that the decisions are just moving further away from the idealized discussion on the outside. So I think that does ask a real question about the value of the outside game on all these things. But I think there is also something pulling in the other direction, which is, which is this still needs doing. All of this needs explaining. All of this needs analysis by people that have sufficient access to the information, by having the conversations with the people that are on the inside, but still not looking at this from the biased institutional view of any one specific place. And it also just needs doing from the perspective of, well, you need to make these points and these findings credible to the rest of the world. And I think especially in the rest of the world, where people are still skeptical of the narrative that might come from inside securitized environments or inside the labs, there is real value in making this case and making this argument in the context of the outside game and not just in a sort of privileged conversation between intelligence officials between different countries or in a pitch that reads like a marketing pitch made by a lab to a foreign government. So I think especially for the work I do, there is still real value in doing it from the outside. But yes, I think there is an ongoing trend that, that pulls against the outside game. And I think that is something to be, not for myself, but for the ecosystem to be somewhat worried about. And I think something to at least look at.

[A]

Are you long term bullish or bearish on Germany?

[B]

Well, I think bullish on the sort of macroeconomic basics and on just like sort of. I think there is also the sort of regression to the mean argument to be made here in that this has long been a very successful economy and a very sort of, at least in the context of the west, pain been obviously strategically important country. And maybe you just think that, well, incidentally, this is not going so well right now. It used to go better 10 years ago, maybe it will go better 10 years in the future. And maybe this is the very high level historical view. And then the more you zoom into the details of what is actually happening, the more worried you get about some of the specific policy features. Not to expand this to the general domestic politics conversation, but just keeping it to the AI policy conversation. It's just Germany Germany has a lot of great assets for doing well in AI in very general terms like very skilled workforce, less service heavy economy than most of the other, most other comparable economies. Still a pretty good industrial base, I think a pretty good network of both bilateral and organization based relationships that are, I think the right set of relationships to have. If you think that US is building AGI, but you still need some counter leverage against that. I think all of these are a great position to be in. And the only things that Germany needs to do is make all the politically hard decisions to get back to 3% economic, 4% economic growth, a flexible labor market, all these things. In a way, if Germany only solved all of its other problems, it would be fine on AI, which is a fairly contentious thing. There are a lot of countries that have just bad assets for AI specifically. And if they do well right now, it doesn't mean they do well with AGI. If you have a very service, say maybe Singapore or Hong Kong or maybe even the uk if you have a very service heavy, the economy that is very susceptible to disruption by AI agents specifically, it just seems much scarier than if you have economic assets that matter in a broad set of AI futures. So a lot of German AI policy and I think a lot of European AI policy in general is just much less about AI and much more about general policy. So in a way it's very easy to solve because you just have to do all the things that hundreds of thousands of people are thinking and writing about anyway. But in a way it's also much harder to solve. Right, because you can't go there and you can't give them a two page memo of here are all the clever ways to pull the rope sideways, come up with all these very specific interventions and then you fix your AI policy and then things are good. No, you actually need to go through the messy politics of doing the labor market reforms and getting the industrial base back on track and all these things. And that's really hard in a way. But that's at least a known and well scoped challenge. And I think if Germany can get a hand handle on that, then it's in a great position for doing the AI stuff well as well. And TBD and honestly, way out of my depth on whether that is going to happen and how that should happen. But yeah.

[A]

Well, it's been a pleasure having you on Anton, and we'll see you next time.

[B]

Thank you so much for having me.