← 80,000 Hours Podcast
80,000 Hours Podcast
#238 – Sam Winter-Levy and Nikita Lalwani on how AGI won't end mutually assured destruction (probably)
01:11:03
Loading summary
Transcript103 lines
- [00:00]
Nikita Lalwany
There's a lot of uncertainty with this fast takeoff scenario. If it does happen, that could really change the calculus on the nuclear deterrence question.
- [00:08]
Sam Winter Levy
In a nuclear world, once you have nuclear weapons, the winner of contests and these kind of bargaining dynamics is determined less by the balance of power, like which side has more weapons, a stronger economy, more resources, and more by the balance of nerves. Which side has more resolve, which side is more willing to run a higher risk of nuclear escalation, which side cares more about the particular outcome.
- [00:28]
Nikita Lalwany
Imagine sort of going to the President today and saying this AI system, system can tell us with 100% certainty where all of China's nuclear weapons are. Would he accept that statement unquestioningly, or would he have some doubt as to whether the system was foolproof?
- [00:42]
Sam Winter Levy
I think the AI community, AI experts are really best placed to understand the technologies that are emerging, but they are not expert on nuclear weapons and nuclear deterrence. And conversely, the nuclear community knows that field of work, but they're just not necessarily entirely on top of the frontier of AI breakthroughs.
- [01:03]
Podcast Host
Today I'm speaking with Nikita Lawany and Sam Winter Levy. Nikita is a lawyer and policymaker. Previously she served as Director for Technology and National Security at the White House, where she focused on issues related to US China technology competition, and before that as a Senior Advisor to the director of the Chips Program Office at the U.S. department of Commerce. Sam is a fellow at the Carnegie Endowment for International Peace where he focuses on the intersection of national security and AI. They've both just co authored a really great piece for Foreign affairs called the End of Mutual Assured what AI Will Mean for Nuclear Deterrence. Thanks for coming on the podcast, Nikita
- [01:44]
Nikita Lalwany
and Sam, thanks so much for having us.
- [01:46]
Sam Winter Levy
Great to be here.
- [01:47]
Podcast Host
You think AI experts need to pay more attention to nuclear deterrence? What is the case for that?
- [01:54]
Sam Winter Levy
Yeah, so the basic case here is that nuclear deterrence is a key check on the ability to of even technologically very powerful states to impose their political preferences on their less technologically advanced adversaries. And so long as nuclear deterrence remains in place, we think the economic and military advantages produced by AI, while very significant, will not necessarily allow states to kind of completely dominate one another. States will continue to temper their actions for fear of nuclear reprisal, and there will remain pretty significant limits on states ability to, to impose their political preferences on their kind of major nuclear armed adversaries. But conversely, if AI does undermine nuclear deterrence, then the technology really could make a state unrivaled in its capabilities to threaten, coerce, and dominate its adversaries. So we think this kind of question, whether nuclear deterrence persists or erodes in this kind of age of AI that we're entering, will fundamentally shape both the risks and the stakes of the AI competition.
- [02:51]
Podcast Host
Yeah, it seems like a really big deal. Which way this question goes. I think there are a bunch of kind of nitty gritty details to get into to really understand the role of nuclear weapons in a world where one country or maybe multiple countries are building AGI. But to understand those, yeah, I want to quickly run through some key concepts. So nuclear deterrence and mutual assured destruction, I guess. First, can you give just a very quick refresher on what exactly nuclear deterrence is and how it works?
- [03:22]
Nikita Lalwany
Yeah, absolutely. So very quickly, deterrence refers to the practice of dissuading an adversary from taking certain undesirable actions up to and including a nuclear attack. And the way deterrence works is by making your adversary think it should not act, either because its objectives are too costly or too uncertain, or because the penalties it will incur from acting outweigh any benefits. The concept of deterrence has been at the heart of international relations theory and international politics for much of the last century. And I'll just highlight two features of deterrence that I know we'll come back to later in this discussion. The first is that for as long as nuclear deterrence has been in place, it has been difficult for states to fully impose their political preferences on nuclear armed rivals for fear of nuclear retaliation. And the second is that the central pillar of nuclear deterrence really comes down to what is known as the second strike capability, which is a state's capacity to respond to a nuclear attack with a devastating nuclear strike of its own.
- [04:29]
Podcast Host
I want to come back to the second strike capability. First, I guess I'm interested. To what extent does nuclear deterrence actually prevent states from coercing and pressuring their adversaries? It seems like clearly this is true to some extent, but it also seems like there are big exceptions like North Korea. What exactly are the kind of limits of deterrence?
- [04:55]
Sam Winter Levy
Yeah, so nuclear deterrence definitely doesn't end the possibility of coercion, intense competition, pressure between states, or even the risk of outright war. But in the conventional view of nuclear theorists, people like Thomas Schelling and Robert Jervis, nuclear weapons change the nature of these competitions between states. In a nuclear world, once you have nuclear weapons, the winner of contests and these kind of bargaining dynamics is determined less by the balance of power, like which side has more weapons, a stronger economy, more resources, and more by the balance of nerves, which means basically the idea of which side has more resolve, which side is more willing to run a higher risk of nuclear escalation, which side cares more about the particular outcome. And so what this means is that there's generally an upper bound on the kinds of coercion that states can exercise against their nuclear armed adversaries. Because if you attempt to coerce them on matters that are sufficiently important to them, so things like territorial integrity or regime survival, the kind of core political interests of a state, you run the risk of nuclear escalation. And so just to give one example, to try and motivate this, if you think about the US and Russia right now, the US is economically far superior. Its economy is like 15 times Russia. It's conventionally far superior as well. But there are real limits on how much leverage the US can exercise over Russia, because on issues that the Russians care about sufficiently, they can always threaten moves that can add risk of nuclear escalation or add risks of nuclear accident. And that puts a pretty significant limit on how much influence the US can really wield over them.
- [06:23]
Podcast Host
Yep, that feels important. And I think it's going to come up again for now. Yeah, I want to understand the second strike better. So it's not actually the kind of number of warheads or even that much about how much power the nuclear weapons in an arsenal have, like how big the bombs are. Basically it's this secure second strike. So can you explain kind of what exactly it is and why it's important?
- [06:52]
Nikita Lalwany
Yeah, for sure. So a secure second strike, as I was saying before, is sort of the cornerstone of nuclear deterrence. As you said, it's the ability of a state to retaliate after absorbing a nuclear attack. And so long as two nuclear powers can credibly maintain a second strike capability, that can inflict unacceptable damage on their adversary, a first strike would be suicidal. To maintain a secure second strike capability, your forces need to be survivable, which means that a substantial portion of your nuclear force should be able to survive any potential adversary attack and endure throughout crises and conflict. States, by and large, pursue different strategies to ensure the survivability of their nuclear forces. But to take the United States as an example, we have a triad of strategic nuclear forces. So we have land based intercontinental ballistic missiles, submarine launched ballistic missiles which are carried by nuclear powered submarines, and we have an airborne force of bombers. And there's a lot of redundancy built in and different approaches to survivability. So, for example, keeping weapons in hardened silos or moving them around or making them extremely difficult to find underwater. And we do all this to try to ensure we maintain a retaliatory capability.
- [08:07]
Sam Winter Levy
It might be worth flagging that different countries have emphasized different approaches to securing a second strike capability. So the US has the triad that Nikita mentioned. UK Only has nuclear submarines. Russia and China have road mobile launchers, which are basically large trucks carrying nuclear missiles that drive around roads and highways. That was kind of considered politically unviable in the US because people will be so freaked out to see nuclear weapons just driving around their cities. So the US does not have road mobile launches. So different states have taken different approaches, but they all revolve around this idea of resilience, redundancy, and survivability.
- [08:42]
Podcast Host
Nice. Yeah. I'd never really known why the US doesn't have road mobile launches, but it makes sense just because it's not politically viable. Yeah. I'm surprisingly unsettled by the idea of nuclear weapons driving down the highway in the US So I'm kind of glad we don't do. What exactly would it mean to undermine the secure second strike, practically speaking?
- [09:09]
Nikita Lalwany
Yeah. So this question, I think, really gets at the heart of what we've been interested in exploring and helps set the stage for kind of more rigorous exploration of how AI might or might not undermine nuclear deterrence. There are probably many ways to think about this, but the way we've thought about it is that sort of broadly speaking, there are three ways that a state could undermine an adversary second strike capability. The first is that a state could destroy a rival's entire nuclear arsenal in what is called a splendid first strike. This would require pinpointing all of an adversary's nuclear weapons, including the locations of nuclear submarines and mobile launchers. The second way that you could undermine the second strike capabilities, the state could prevent a rival from launching a retaliatory strike by disabling nuclear command, control, and communications networks. And those are the networks that support nuclear decision making and communicate those decisions to nuclear forces in the field for execution. And then finally, a state could strengthen missile defenses such that a rival could no longer kind of credibly threaten nuclear retaliation. So, you know, the question overall, then, is whether advances in AI will give states these capabilities. And there's a kind of second question that I'm sure we'll get into as well, which is whether and how these capabilities would then translate into geopolitical dominance.
- [10:31]
Podcast Host
Okay, I'm interested in getting into the details. I do find it super plausible that AI could be really good at kind of solving the kinds of problems that humans can't really solve, and therefore that kind of secure this second strike. But I can also imagine, maybe intuitively it feels like AI could kind of magically solve everything, but maybe it's harder than it sounds. So I'm interested in kind of going through different elements of different countries, nuclear arsenals, one by one, and trying to kind of see how realistic is this, starting with nuclear submarines. So nuclear subs, if I remember correctly, are extremely, extremely survivable because it's really hard to track things that are moving very slowly and quietly in the ocean. Can you explain why that is?
- [11:25]
Sam Winter Levy
Yeah. So you're definitely right that nuclear subs are generally viewed as the sort of most survivable leg of the triad, particularly for the US. The US's subs are extremely hard to find. That's really for kind of three core reasons. The first is just that the ocean is huge. I think it's easy to underestimate how big the ocean is. The scale of the search problem here is just enormous, like tens of millions of square miles of ocean that you're trying to find these relatively small objects in. Second, as you said, water kind of blocks a lot of what we generally use to detect things. So seawater is almost totally opaque to electromagnetic radiation. So techniques such as radar that work pretty well for surface and airborne targets are just much less useful for finding submarines, which, remember, they can remain submerged for months at a time, potentially when they're out on patrol. Instead, states are generally limited to acoustic methods, but only low frequency sound waves propagate significant distances in water. And in the process, they interact with all sorts of kind of complicated oceanographic phenomena that distort sounds. A lot of the ocean just contains a lot of ambient noise, and oceans are actually getting noisier over time with increased commercial shipping. And submarines just produce very low levels of signal relative to this background noise. And third, and finally, modern subs are engineered for silence. So their propeller designs, their hull coatings, their internal machinery, it's all just designed to minimise noise. And nuclear powered submarines can be so difficult to detect from other subs that they sometimes collide underwater.
- [12:50]
Podcast Host
Yeah, that's an incredible fact. I had never heard that before. And it does feel like it really helps my intuition understand how hard subs are to find in the water. I also think there's something about. Yeah, I agree. That it feels really difficult to truly, truly understand how massive and something like opaque oceans are. Like, part of it is that we see them on the map and we see the surface area and we don't see the depth and the volume of it is a big part of why this is such a difficult problem. It's not just the surface of a country where you have road mobile missiles.
- [13:32]
Sam Winter Levy
Exactly.
- [13:34]
Podcast Host
Yeah. This much, much larger area. Exactly.
- [13:38]
Sam Winter Levy
It's like really finding needles in a haystack where those needles are designed to be as difficult to find as possible. And we need to find every single needle simultaneously at the same moment with zero margin per hour. So yeah, extremely difficult search problem.
- [13:53]
Podcast Host
Yeah, that really helps there. I think the other thing that I kind of forget is the oceans. They have mountains in them, they have things. I think I kind of picture it as mostly empty except for fish. And if you wanted to find a submarine that would look different from a fish and you'd use these techniques and it just couldn't possibly be that hard. But they're actually very, very complex environments that, yeah, I guess on the day to day that is not that salient to me. But when I actually think about it, I'm like, ah, that is why this is such a hard problem.
- [14:36]
Sam Winter Levy
Yep, that's right.
- [14:37]
Podcast Host
What is the most compelling story that you could give for how AI would allow adversaries to track and target nuclear submarines? So I guess just like, yep, seems like the problem is hard, but what is the version that people who are really bullish on AI think is going to explain how AI does this successfully?
- [14:56]
Sam Winter Levy
Yeah, so there's maybe two ways to think about this and we can kind of bracket the first one and come back to that later potentially. First one is going to be true of many of these different types of questions, which is maybe AI can invent things that we can't anticipate in advance, just like totally new methods that it's just very hard to think through now. So we can maybe kind of come back to that set of answers. The other way of thinking about this is just what are the methods that AI could use that are kind of most continuous with what states kind of currently do? And that just seem most plausible. And there the story would go something like you can use machine learning to just integrate data from thousands of sensors filtering out noise, identifying these faint signatures that humans would miss. In particular, you can use AI to just fuse noisy data from multiple different sensors, including sonar, but also what are called magnetic anomaly detectors. So subs have steel hulls, which means they disturb local magnetic fields, which you might be able to pick up. And then also satellite based synthetic aperture radar, which might be able to identify very small wake patterns generated on the ocean surface by passing submarines. So you can fuse all of that data. Maybe you could also use autonomous underwater vehicles that can kind of patrol, continuously coordinate with satellites and build a kind of persistent ocean wide surveillance net. And if you combine all of that data with signals, intercepts and human intelligence and all these other sources of data that states have, maybe you could use AI systems to kind of track submarines more effectively. And you might also be able to use AI systems to hack into the systems the states themselves use to track and communicate with their own submarines. So I think that's the version of the story, that's kind of one plausible version of the story for how states could use AI to track nuclear submarines.
- [16:38]
Podcast Host
Nice, nice. It sounds compelling to me. But also my sense is that there are some reasons to think that this wouldn't actually work. Yeah. Can you go through some of those reasons?
- [16:50]
Sam Winter Levy
Yeah. So I think first it sounds easy when you say, like, oh, just integrate all these different data from different sensors. But this is just an extremely hard technical problem and there are probably limits to what these improvements can yield. Again, should bracket that. There is a lot of uncertainty in every part of this. Like we're talking about future technological developments, there's uncertainty about what state's current capabilities are, there's uncertainty about the nature of technology. So huge bracket for just big uncertainty here. But that said, this is just like a really, really hard technical problem. The physics of the undersea domain are brutal. Any system will struggle to kind of continuously identify, track and monitor multiple targets amid ocean background noise, especially as submarines get quieter and oceans noisier, which is currently happening. The number of uncrewed underwater vehicles you would need to render even part of the ocean transparent is just going to be enormous. Most of these vehicles have pretty low endurance, pretty limited in their covert communication range. And if you wanted to deploy millions of sensors around the oceans, that's also going to be challenging because if you want universal coverage, you're going to have to put some of those sensors in contested waters, which will incur risks of sabotage and interference of its own. So yeah, I think the first reason to think this is difficult is just hard technical problem. Second, it's just unlikely that States are going to do nothing in response. And this is going to be true for every element of this nuclear, every element of this discussion here. This is going to be a move, counter, move dynamic. And states have a lot of countermeasures that they can use to tip the scales in their favor. So they can jam signals, they can manipulate sensor data with deceptive decoys, for instance, they can just play recordings of a submarine into an amplified underwater acoustic source. And states can also use a lot of these technologies to protect their own submarines. So Russia and China, for example, can use these underwater sensors. They can use underwater networks to protect their submarines, which generally operate in waters closer to their own home territory where it's easier to protect them. I think there are just a lot of ways that states can add uncertainty to every step of the process of detecting, tracking and targeting submarines. And so it will likely remain a kind of probabilistic affair. And states are probably not going to want to target their adversaries, nuclear submarines, unless they're very confident that they can take them all out at once, because each submarine could carry as many as 200 warheads on them. So if even one submarine escapes, that's probably enough to deprive you of any kind of meaningful notion of victory. So yeah, it's a really hard problem. I would just want to add one good recent paper on this is from Tom Stefanik, which I'm sure we can put in the show notes, but in that paper he goes through a lot of these kind of move, counter move scenarios that are likely to play out. And in his view he thinks that ballistic missile submarines are likely to remain reliable second strike nuclear forces over the next 20 years and beyond. That's his take on this.
- [19:43]
Podcast Host
Yeah. Okay, so maybe one thing to put a pin in is like this assumes that progress is meaningful but not so fast that an adversary won't have any time at all to respond and build up some kind of defense capability. That's. Yeah, that's meaningful. So yeah, pin in that and maybe we'll talk about some scenarios where actually progress is too fast for adversaries who don't have AI or AGI to respond to. I guess in the world where like there is this where things are moving at a pace where countries can respond with kind of countermeasures. Is it plausible that there is still a sudden enough breakthrough in something like acoustic modeling or real time data fusion to create a short lived kind of offensive window where nuclear subs are identifiable? And maybe it doesn't need to be forever. It's just like if there's a window, a state could choose to use it knowing with some confidence that their adversary hasn't been able to build up defences yet?
- [20:57]
Sam Winter Levy
Yeah, so it's possible maybe. One interesting piece of history here is that there were periods during the Cold War, particularly during the 60s to the 80s, where the US was actually pretty good at Tracking Soviet submarines, that was partly for reasons of geography and partly for reasons of technology. So Soviet submarines weren't as quiet as American ones. And they also had to pass through very narrow choke points, in particular, most famously, the gap between Greenland, Iceland, and the uk they have to pass through these narrow choke points to get out to the open ocean. And so there was this window where the US Navy was actually pretty good at tracking Soviet submarines. The US didn't have this problem because the US Just has direct access to kind of tens of millions of miles of ocean just from the west coast and the East Coast. So it is possible that you get Windows where states will feel more confident at different times. But again, it kind of comes back to this whole issue here, which is just the level of confidence you need and the degree of redundancy, the degree of different systems that you need to take out simultaneously. It's not just enough to get significantly better at tracking other state submarines. You need to know that you have every submarine and then every road mobile launcher and every icbm, which I'm sure we'll come on to in a little bit. So definitely don't want to rule out the possibility of Windows emerging, and we can kind of come on to that later. But it remains a just very, very challenging situation where states are likely to be spending a lot of resources kind of staying on top of this problem.
- [22:13]
Nikita Lalwany
Okay.
- [22:13]
Podcast Host
Yeah, yeah, okay. Let's come back to kind of the uncertainty. And actually, what would countries do if they even thought maybe they had these kinds of opportunities? Let's move to road mobile missiles. So, yeah, you said in China and Russia, there's more of an emphasis on road mobile missiles, which, again, sounds actually really terrifying. If that's a place where you live, what makes these so hard to find?
- [22:39]
Nikita Lalwany
Yeah, so the basic story here is very similar to what Sam was just talking about with submarines. Road mobile launchers, like subs, are concealed, camouflaged, and they don't stay in one place for very long. They're carried on vehicles that can hide under netting, under bridges, in tunnels, and they're driven from one concealed location to the next. Taking a step back, the survivability of these launchers really depends on the competition between sort of a hider's ability to keep them concealed on the one hand and a seeker's ability to locate and track them on the other. And at least historically, hiders have had some real advantages. They can send mobile launchers to remote locations. They can move mobile launchers in short bursts at times that are selected specifically to make Them difficult to track. So, for example, at night or under extensive cloud cover. And finders, by contrast, have to overcome several obstacles. They have to track all mobile launchers kind of deep within enemy territory. During a crisis, they'd have to find the launchers and destroy them all more or less simultaneously, in coordination with attacks on the rest of a country's nuclear forces. And then they'd have to do all of this over a relatively short period of minutes or hours, all while the adversary is kind of doing everything it can to thwart surveillance and information gathering capabilities.
- [24:04]
Podcast Host
Yeah, so, I mean, that both sounds extremely hard. And also I again have this like, oh, this just sounds like the perfect problem for AI to solve. And maybe can you start actually by laying out that case, the case that plausibly AI has a shot at this?
- [24:22]
Sam Winter Levy
Sure, yeah. So in many ways, this is a kind of proven application to AI systems. The United States probably already has large amounts of data from satellites, signals, intercepts, aircraft, and so on. But that data currently kind of outpaces the ability of human analysts to digest. And states probably also have a significant stock of images of these mobile launches. So many of them are just paraded in public through Belly Shane, you can find images on the open source of these launches, along with information about their signatures and how fast they drive and their weight and so forth. But you could plausibly use these images and these signatures to train machine learning algorithms to just dramatically speed up the processing of intelligence to make it easier to conduct operations against these vehicles. So in many ways this is, like you said, this is sort of a classic pattern recognition within a big data set type problem that in many ways could be suited to AI. And the key thing here is that AI can reduce the area you need to search dramatically and similarly reduce the area you would need to attack by orders of magnitude, potentially, because just to kind of bring home what this entails. In most of the kind of modeling on this, the unclassified modeling, once you find these launches, you then basically just bombard that area with nuclear warheads of your own. So states only have so many nuclear warheads. So you have to kind of pick your targets relatively carefully. And so if you can use these AI systems to kind of dramatically shrink the search area and the targeting area, it could possibly increase the possibility that you can pull off a splendid first strike with a nuclear first strike of your own against these road mobile launchers.
- [25:52]
Nikita Lalwany
Yeah.
- [25:53]
Podcast Host
Okay, so that's the case. Four, what options would the defending countries have for keeping their road mobile launchers hidden?
- [26:00]
Nikita Lalwany
As with Every element of nuclear deterrence there are likely to be sort of move, counter move reactions. You know, there's another great paper that we can put in the show, notes Thomas McDonald, previously at Carnegie, now at Lawrence Livermore. He has a great recent paper on this where he describes some of the countermeasures that hiders could adopt to keep their launchers hidden. Some of which are very straightforward and sort of low tech, and others of which are more complicated. Most simply hiders can sort of adopt old fashioned low tech solutions, for example, covering roads with netting or constructing decoys. Decoys being vehicles that are made to look and act like mobile launchers, but aren't actually mobile launchers. And that would both increase the number of vehicles finders have to monitor and potentially strike and decrease their certainty of any given detection more dramatically. In a crisis, states could use anti satellite weapons to destroy or impair satellites, which would create holes in coverage that hiders could exploit to then move their launchers. This would obviously be a provocative measure, so less likely during peacetime, but could be used during a conflict scenario. The big picture here is that although it's impossible to predict exactly how states will react, there are countermeasures available to them. And if they care about the survivability of their mobile launchers, they have every incentive to use them. Of course, finders can then innovate as well. So one might expect a kind of measure countermeasure cycle that could potentially lead to greater instability.
- [27:33]
Sam Winter Levy
Yeah, I mean, maybe one thing to take away just from this whole section is that although AI systems may make it easier to track submarines and to track road mobile launches, defending states have a lot of options that they can take to try to shore up the survivability of their second strike capabilities. Or at the very least to inject enough uncertainty into the belief of a state that might be considering launching a first strike, that they're going to think very, very hard about doing so. And many of those options available to them do not require equivalent levels of technological sophistication. Many of these are pretty low tech measures to just massively expand the search area or massively increase the number of options you would need to target through things like making your launchers drive faster. That's not like a high tech move, but it can make launching a first strike just significantly more difficult to pull off.
- [28:19]
Podcast Host
Yeah, I do think that the low tech ness is striking. To me. It doesn't have to be the case that other countries have AI as good as kind of the leading country. It just has to be the case that AI continues to struggle to see through chicken wire over a highway, which, yeah, isn't really. Yeah, it feels like a surprise. Let's turn to missile defense. So in theory. I remember when I first learned about missile defense, it sounded like it could be a huge deal. If you can kind of defend against incoming nuclear missiles, you can basically make it impossible for them to kind of achieve this second strike, even if they have plenty of missiles left. After decades. My impression is that no one can do missile defense reliably. And I feel like there are famous analogies to what is it.
- [29:11]
Nikita Lalwany
It's like shooting a bullet with a bullet.
- [29:14]
Podcast Host
Yeah, it's like shooting a bullet with a bullet. Can you explain what exactly makes it so hard?
- [29:19]
Nikita Lalwany
Yeah. So missile defense systems must kind of do the nearly impossible, which is detect a launch, track potentially hundreds of missiles traveling through space at 20 times the speed of sound, estimate their future trajectories, and destroy them with interceptors all in less than 30 minutes, which is the rough flight time for most land based missiles from traveling between the United States and Russia or China. This was actually, I think, just dramatized pretty effectively in House of Dynamite, the new Kathryn Bigelow movie on Netflix. As some others have pointed out, it's likely that US missile defense would ultimately be effective against a lone icbm, but its hit rate would fall dramatically if faced with a sort of barrage of missiles. As you say, missile defense has sort of been likened to trying to hit a bullet with a bullet. In actual fact, it's probably a little bit easier than that. Hitting a bullet with a bullet is obviously incredibly difficult, but it's still very, very hard.
- [30:13]
Sam Winter Levy
It might be worth just clarifying the distinction between missile defense, where you're trying to defend the continental United States against a nuclear strike from Russia or China, distinguishing that from things like Iron Dome in Israel, which listeners might, you know, have a sense is very effective. Defending the United States against a kind of full scale nuclear strike from Russia or China is just a vastly harder proposition than protecting small areas of a small country from pretty small rockets launched from Lebanon or from other neighboring states. In the Middle Eastern context. And just to put some context on how expensive and how difficult this is, right now, the US is talking about potentially spending around $3.5 trillion over 20 years to build a missile defense system that might be able to block North Korea's ICBM arsenal, but definitely not one the size of Russia's. Just to give you a sense of just how expensive and difficult missile defense is.
- [31:04]
Podcast Host
Wow. Yeah. That is Striking. Okay, so the thing that I want to make sure I understand is it sounds like a very, very limited strike. It is plausible that the US could defend against using its missile defense system. And that's because of something like it doesn't have 100% success rate. But if there were a small enough number of missiles incoming, it is good enough that it would have a decent shot at taking all of those down. And the issue is that it doesn't have enough of those. And the success rate isn't quite high enough that a massive strike, say, from Russia or even China, which has in the hundreds of missiles that would overwhelm the US Missile defense system. Is that right?
- [31:49]
Sam Winter Levy
Yeah, I think that's right. And I think there's two issues that kind of complicate that in addition. One is that you don't know where a strike is coming from necessarily. Like in the case of submarines. It's not the case that you can just put all your missile defence systems around one particular city or to protect against one attack trajectory. These strikes could be coming from anywhere. And the second is that states can use decoys alongside warheads. Distinguishing between these decoys and the real warheads is a very challenging problem. And so that's another way in which you can overwhelm missile defence systems, because it's often much cheaper to build additional decoys than it is to build additional interceptors. So the economics combined with the physics in missile defense is just very challenging.
- [32:26]
Podcast Host
Makes sense. Yeah. How plausible do you think it is that very advanced AI could solve some of these problems?
- [32:34]
Sam Winter Levy
So, yeah, as with every element of this, I think AI can certainly help with elements of this problem. Software advances can make it easier to predict a missile's trajectory and speed up decision making. Once you detect a launch, machine learning algorithms can potentially rapidly analyze data from multiple sensors to distinguish actual warheads from decoys. These things that are kind of built to mimic the radar and heat signatures of real warheads. They're much cheaper to deploy. And maybe through advancements in material science, AI might produce lighter, more agile interceptors that make weapons kind of cheaper and more maneuverable in flight. But AI will probably still face pretty big limits here. There are lots of really hard technical challenges. In particular, an AI hardened system of missile defense is going to depend on machine learning algorithms that are trained on large, reliable data sets regarding decoys and countermeasures. And that's data that US adversaries just have every incentive to obscure. And US rivals could also try to confuse AI algorithms by manipulating missile tests, including disturbances or perturbations intended to poison data sets for machine learning if they believe they're going to be observed. And then even if AI can harden missile defence systems and improve them, none of these developments will take place overnight. Missile defense architectures take years to develop. And US adversaries will not just kind of stand by and watch that play out. Again. Not to bombard you with papers to put in the show notes, but Laura Grigo has a pretty good paper on how this offensive defence competition is likely to play out in the missile defence domain. And attackers just maintain really significant advantages. So they can launch from unexpected directions. They can use hypersonic missiles which are much more manoeuvrable than ICBMs. They can overwhelm defences with coordinated salvo attacks using large numbers of decoys. They can directly target those defence systems if they really wanted to. So this remains. I think most experts in this area think that missile defense remains a, a domain that is really tilted against the defender. Attackers just have a lot of options here for getting around missile defence. Yeah, and I think one last thing to note is even in a world where you have perfect missile defence, so unlikely world, but even if you're in that world, I think states can still resort to kind of more creative delivery methods, so they can use uncrewed undersea vehicles released from submarines near important ports. So Russia is already developing exactly this type of exotic means of nuclear delivery. They have this nuclear powered autonomous torpedo called Poseidon, capable of travelling supposedly thousands of miles. And states could also try and kind of smuggle or pre position small nuclear devices in enemy territory, along the lines of the sort of recent Ukrainian attack on Russian airfields, where they kind of smuggled in weapons deep into Russian territory. So even with perfect missile defence, states may still have options to kind of deliver nuclear weapons if they really want to.
- [35:20]
Podcast Host
Okay, so that's missile defence. The last big question mark is around nuclear command, control and communications. So in theory, one country could undermine an adversary's ability to launch a second strike. Kind of not by finding their adversary's hidden nuclear weapons or kind of solving the problems of missile defense. What are the key components of nuclear command, control and communications? And how do current nuclear powers currently keep those components survivable?
- [35:49]
Nikita Lalwany
Yeah, so as just a quick refresher, nuclear command, control and communications systems are designed to monitor the conditions of nuclear forces, develop and update nuclear plans, gather and understand information about adversary forces and possible targets in the context of an attack scenario. NC3 must enable decision makers to assess information, consult with other parties, and then direct US Forces and personnel to implement nuclear decisions to do all of this. NC3 infrastructure, at least in the United States, consists of more than 150 different systems, including infrared satellites that look for the hot flare of missile launches, ground based early warning radars, air surveillance radars, nuclear detonation detectors, fixed and mobile command centers, and then communications facilities, both ground based and space based, that connect civilian leadership to US military forces and others. Many parts of the NC3 system are already vulnerable to attack, and we should definitely talk more about cyber vulnerabilities here. But the basic way that states have tried to keep their systems survivable is by building in resilience. So, for example, some command bunkers are buried like 700 meters underground, which is deep enough to survive even a direct hit from a large nuclear weapon. In space, nuclear powers have sent hundreds or thousands of satellites into orbit. And in the air, the curvature of the earth limits the distance at which surveillance radar can sort of track airborne command posts.
- [37:26]
Podcast Host
Okay, so in theory, how might AI get closer to decapitating an adversary's command and control systems? I guess it seems like at least some of these components seem just pretty invulnerable. It's hard to imagine how AI solves this problem of a nuclear weapon not being able to reach a very, very hardened bunker. Yeah. What's the best case?
- [37:50]
Sam Winter Levy
I think AI might make it easier to track some of the kind of mobile command posts on land. It might make it easier to more precisely target airborne command posts. Maybe AI enhanced anti satellite weapons could make it easier to target satellites that provide early warning of incoming nuclear attacks. But other aspects of State's nuclear command and control systems are pretty immune to or are pretty robust against AI developments. In particular, for instance, bunkers that are 700 meters underground. Like, tracking them is not the problem. The problem is that even if you drop a nuclear bomb right on top of them, they're likely to survive. So that part of it seems less vulnerable to AI breakthroughs. I'd say the kind of biggest area of uncertainty and the most plausible pathway through which AI could have an impact here is the cyber domain. So you could envisage sophisticated cyber operations just supercharged by AI that might allow states to penetrate the rival's command and control networks, disable early warning systems, disrupt the transmission of orders. Huge uncertainty here about assessing this. But potentially there are vulnerabilities that could allow one state to penetrate another's nuclear networks, and those potentially already exist.
- [38:55]
Podcast Host
Right? Yeah, I'm interested in that because it does seem like if there was enough Redundancy with digital and analog parts of the system, the cyber thing wouldn't be decisive, but maybe there are kind of entire components that have been made digital and where there just aren't analog kind of redundant components anymore.
- [39:17]
Sam Winter Levy
Yeah. So I think there's definitely going to be redundancy. I mean, maybe one way you could think about cyber here is that it potentially buys time in coordination with everything else you have to do. So potentially you can delay a state's response and that might give you time to track their submarines and track their road mobile launches. And so even buying time there may be helpful. But as you said, states invest a huge amount in resilience and redundancy in different systems that run on different frequencies and use different software to the extent they even use digital technology at all. So, yeah, I think resilience and redundancy are going to be baked into most aspects of Estate's command and control systems.
- [39:53]
Podcast Host
Yeah. Okay. And then I guess the other thing that sticks out to me here is that cyber in particular feels like a place where AI is having a big impact already and will probably have an even bigger impact than other parts of this triad question.
- [40:12]
Sam Winter Levy
Sure. Though it's worth maybe flagging that the question of whether AI will ultimately benefit cyber defenders or cyber attackers is obviously a kind of big open question. Big arguments on both sides there. But states will most certainly use AI systems to try to shore up their cyber defences as well.
- [40:25]
Podcast Host
Okay, and what defences might keep nuclear command, control and communications survivable?
- [40:34]
Sam Winter Levy
Yeah, so I think the key challenge here, as with almost every part of the splendid first strike that we've been discussing, is, is that a state needs to destroy really as much of every component of nuclear command and control at the same time with as little warning as possible. And so that's the kind of key thing that makes this such a hard challenge. And as we said, a few kinds of these assets, really deep bunkers, command and control aircraft, those are so difficult to destroy or disable that they're likely to remain survivable even given plausible improvements in strike capabilities. I think that the cyber domain is the hardest to assess here, what the defences would look like. Vulnerabilities almost certainly exist, but so do defences, so do patches. And I think there are still going to be big challenges for an attacking state in the cyber domain. You can just name three quickly. I think first, nuclear armed states are just likely to expend a ton of resources trying to defend their NC3 systems, which are probably among their best protected networks. They'll use multiple redundant networks, they'll use different software, they'll use their own AI cyber defences. Second, cyberops would probably require kind of persistent access to an adversary system. But whenever you have persistent access, that risks detection and possible retaliation. Anything that looks like a cyberattack on another state's nuclear command and control systems could trigger this whole escalatory spiral that states are going to want to be very cautious about. And third, and this is kind of a recurring theme with a lot of these elements of a splendid first strike, it's very hard to test how well your malware actually works here in a counter NC3 campaign. You can't test it against the actual target networks because that might look like the beginning of a splendid first strike. So you'll need to use kind of virtual or perhaps even physical models of the target. But although simulations will only be as good as your available knowledge, and given how secret and classified and protected these networks are, your knowledge is going to be limited. So this testing issue, I think it affects every part of this. Like, you can't test pulling off a splendid first strike against every nuclear submarine, but you also can't test it in the cyber domain.
- [42:35]
Nikita Lalwany
Just taking half a step back, even if you do destroy a command and control system, it may not be enough to kind of prevent retaliation. The uk, for example, has adopted procedures to allow its submarine commanders to assess whether a nuclear strike has destroyed the country. And if they determine that it has, they then open what's known as a letter of last resort, which is carried on the submarine and may include instructions from the Prime Minister to launch a nuclear response.
- [43:03]
Podcast Host
Whoa.
- [43:03]
Sam Winter Levy
Yeah. And the Russians have something similar. Famously, during the Cold War, they had what was called the dead hand system, also known as perimeter, which was designed to automatically trigger retaliation if it judged that a state's national command authority had been destroyed. So even if you can take out a state's nuclear command and control system, they may well still have measures in place to try to make sure that retaliation still takes place.
- [43:25]
Podcast Host
Yeah, that just feels really huge to me. I guess a thing that is striking to me about all of this is both the extent to which it really depends on kind of this offensive, defensive back and forth between attackers and defenders, and also the extent to which a lot of this rides on how much uncertainty a country is willing to accept in attempting splendid first strike. So I want to ask some questions about that. Yeah, I guess the first one just being if this kind of survivability ends up depending a bunch on whether rivals invest thoughtfully and sufficiently in adaptation and Defence. How confident are you that that will definitely happen?
- [44:10]
Sam Winter Levy
Yeah, so I think as we, as you'll probably get the sense, like a recurring theme here is that this is a kind of move, counter move dynamic, where states are kind of responding both to technological changes and to moves of their adversaries. And that's really been the story of nuclear deterrence really for the past 80 years. This is not new with AI. AI may increase the speed at which that takes place, but this move, counter move dynamic has really played out since the early years of the Cold War. Now, states may adapt at different speeds from each other. So Austin Long and Brendan Rittenhouse Green have a great paper that we can put in the show notes again on anti submarine warfare during parts of the Cold War. In their view, there were these windows potentially from the 1960s to the 1980s, where the Soviets were a little bit slow to adapt. Both the ways in which they used their nuclear submarines and also the technology that they adopted in terms of how quiet those submarines were. And that gave the US Navy an advantage for a brief period during the Cold War. Never to the extent that you could pull off a splendid first strike, but to the extent that the US was better than is commonly acknowledged at tracking Soviet submarines. So it's definitely not a guarantee that states will adapt or that they will adapt at the same speed. But given the stakes in the nuclear domain, given this is likely a top, top priority of any major nuclear armed state, things would have to go wrong for them not to be paying significant attention to this problem set. Especially given that in many of these domains where we've been, in many of these areas where we've been discussing, they don't necessarily need to be at the technological frontier to shore up the survivability of their second strike forces. And the onus is entirely on the attacker to be able to get close to 100% certainty to pull off this kind of remarkably difficult sensor fusion challenge where you're trying to target submarines and mobile launchers and hardened silos all at once with no room for error and no margin for testing. So, yeah, a lot of this does depend on states acting. It does depend on equilibrium behavior shaking out. And there is the possibility that windows could emerge, but the balance is tilted against the state that might want to launch a splendid first strike.
- [46:19]
Podcast Host
Yeah, I guess just on this, that seems good. From the perspective of nuclear deterrence holding, it seems potentially bad from the perspective of just like arms races seem dangerous. How worried are you about this?
- [46:37]
Sam Winter Levy
Yes, I think when we describe these move countermove dynamics. I think that gives us a degree of confidence that nuclear deterrence, in the sense of a secure second strike capability, will survive. But as you say, there are big costs associated with these dynamics that will play out in response to technological change or in response to fears of technological change, because. Exactly. This move countermove scenario is essentially a form of nuclear arms racing. As states feel more insecure about their second strike capabilities as they disperse launches, build up more warheads, build more decoys, try to take moves to shore up their second strike capabilities. That's very expensive, not just in terms of just the sheer economic costs, but also in the political costs, in terms of the kind of mutual hostility and distrust that will play out in this kind of security dilemma type story.
- [47:29]
Podcast Host
Yeah, the Cold War wasn't a good time.
- [47:31]
Sam Winter Levy
Exactly. The Cold War wasn't a good time. So even if second strike capabilities survive as a result of this kind of move countermove scenario, that's still a very kind of destabilizing, potentially kind of scary world to live in. So we definitely don't want to seem sanguine about the impact of AI on nuclear stability, even if truly undermining second strike capabilities is a hard lift for states to pull off. And it also just increases the risk of accidents. So if you're having your robile launchers drive faster to make them harder to target, that's just a kind of more accident prone type of move to take if you're reducing their ability to, if you're reducing signals, communications because you're worried about a state intercepting them, or if you are delegating the launch authority to lower levels because you're worried about a state interact interfering with your nuclear command and control systems, those are all moves that might shore up sector strike capabilities, but also increase the risk of accident. And so that's also quite a scary world to live in.
- [48:27]
Podcast Host
Totally.
- [48:28]
Nikita Lalwany
Okay.
- [48:28]
Podcast Host
And then it seems like another one of the big considerations here is that advanced AI will likely make kind of destroying more of an adversary's missiles very possible. But it still seems like it's extremely difficult to kind of guarantee success of a 100% splendid first strike where you take out all of their missiles and you're confident in advance that you will take out all of their missiles. So if an adversary has even a few nuclear weapons left and can still kind of communicate with their launchers like, yes, we want to launch a second strike, that can be enormously costly for the attacking country, even if it isn't like a full scale second strike. So how close to a splendid first strike could a country get? I guess both in terms of, yeah, kind of like percent of arsenal destroyed, and also in terms of being confident enough in advance to being willing to take on this risk.
- [49:29]
Sam Winter Levy
Yeah. So maybe the question here is how important, Even if you can't take out 100% of an adversary's nuclear arsenal, how important is it if you can get much closer to 100% than in the absence of AI capabilities? I would say this is just an area of huge debate in the nuclear policy field. I'm just going to gloss over a lot of nuances here. Listeners read more about this, but broadly speaking, there are kind of two views in the nuclear policy debate. One view is generally dominant in the academic community. And this is the view that's associated with the kind of classic theories of mutual issue or destruction. And in this view, unless you have very high confidence that you can track and destroy all of an adversary's nuclear weapons, which they generally think is infeasible, then the relative size of countries survivable nuclear forces doesn't really matter. So even if AI could help the US take out 90% of Russian or Chinese nukes so that it could sort of quote, unquote, win a nuclear exchange, that's not super meaningful because 10% of Russian or Chinese nukes getting through is just still going to be absolutely devastating for the US and in this view, using these kind of AI capabilities to target other states second strike systems is itself dangerous and destabilizing and sort of pointless. Then there's a second view. This view is generally known as the kind of damage limitation approach. It's long been US Government policy and received some support from the academic community, but more limited support there. And in that view, it's generally seen as good strategy to deliberately hold at risk in adversaries nuclear forces in an attempt to limit the damage they can do to you. In this view, there really is a huge difference between losing two American cities and losing 20American cities. And so they think that even if AI capabilities can't guarantee 100% success rate, maybe you can't pull off a splendid first strike in the sense of taking out every single Russian or Chinese nuke. They do think that you can meaningfully limit damage, or at least that if the other side thinks that you can meaningfully limit damage, then that will make US threats to escalate more credible, which could contribute to deterrence. So this question of kind of what meaningful damage limitation is ultimately that's not a technological question, that's a political one. And it's going to be very difficult to define that ex ante what will constitute unacceptable damage to a nation state. And it's going to depend on mistakes and the leaders and psychology and all sorts of factors that it's hard to reason about in advance. But I would just say this is like a live area of debate in the nuclear policy field.
- [51:54]
Podcast Host
Yeah, I do find it pretty compelling that a 300 nuclear weapon attack is extremely different to a 5 weapon nuclear attack. And if a country could take out 295 missiles with reasonable confidence, and I guess reasonable is still very, very high because of the stakes, I can imagine that being a pretty different decision to opening oneself up to that 300 missile attack. So that does feel like a pretty important cruxy debate. Pushing on. Let's assume that AI does enable states to find all of an adversary's nuclear weapons. So like a proper splendid first strike, would we expect a state kind of with that capability to be able to impose its will on other states?
- [52:50]
Nikita Lalwany
Yeah, so I think this is a really important question because it sort of gets at a lot of the inherent uncertainties here and also the difficulties of predicting whether and how technological advantages will give states decisive strategic advantages. We could probably have a whole discussion just on this. But let me just offer a few observations. The first, to make the somewhat annoying move of sort of fighting the hypothetical. I just think it's sort of hard to imagine a circumstance where a state has 100% certainty that it knows the location of all of an adversary's nuclear weapons. And that's because that would require the state to also have 100% certainty that they've seen through any countermeasures. And as like the Rand scholar Ed Geist has written, AI tools can be harnessed, could potentially be harnessed to optimize military deception in ways that offset perceived advantages or perceived advances in situational awareness. The second is that even if AI could get a state to a 100% find rate, there's uncertainty as to what a state would do with that information. So imagine sort of going to the President today and saying, this AI system can tell us with 100% certainty where all of China's nuclear weapons are. Would he accept that statement unquestioningly or would he have some doubt as to whether the system was foolproof? Even if he did accept that statement, would he be certain enough to use that information to attempt a splendid first strike? Keep in mind that these are not capabilities that could be tested in advance. So there is sort of a lot of I think uncertainty there.
- [54:23]
Sam Winter Levy
And just to put a finer point on it, launching a splendid first strike here involves launching hundreds, potentially thousands of nuclear weapons at another state based on a belief that you have 100% probability of pulling it off. That's a huge gamble to. That's just a huge move to act on. And I think, yeah, this kind of. And I think there's kind of still this broader question of willingness to act on a capability. So the United States had nuclear weapons for a period of time before the Soviet Union did, but it didn't act on that advantage for various kind of historical reasons. But the broader reason is that there are ethical, political and international norms that often constrain states from using the kind of full extent of their available power. So US Leaders at the time did not want to, did not think the public would tolerate launching a pre emptive nuclear strike on the Soviets. They didn't want to trigger another war. They didn't want to be seen as the aggressor in a new conflict. So there are a bunch of other considerations that might continue to constrain state's capabilities even in a world where a state had the potential to pull off a splendid first strike. And I think one final point here is that even if you do have unchallenged technological advantage, that doesn't always translates straightforwardly into the sort of political dominance some people talk about when they talk about AI giving you the ability to just impose your political preferences worldwide. Complete dominance and control of the type that people like Dan Hendricks have written about. So just to give one example, the US had clearly had unquestionable technological dominance over Vietnam and over the Taliban, and just suffered an unambiguous defeat in both cases after a couple of decades of trying to impose its political preferences. So just this whole question of the relationship between technological power, political power, it's just a little bit more complicated than the most straightforward stories might imply.
- [56:13]
Podcast Host
Yeah, this point about the US Kind of having an opportunity to achieve something like a decisive strategic advantage when it had nuclear weapons before everyone else did, and kind of not really doing anything with that advantage seems important to me.
- [56:31]
Sam Winter Levy
I think it gets at the kind of underspecified nature of this concept. It just kind of conflates a lot of different things in one phrase.
- [56:41]
Podcast Host
Yep, that makes sense. Okay, pushing on. So far we've assumed that AI evolves somewhat gradually, so it's been moving quickly, but not at the kind of extremely, extremely fast pace that some people think it could move at. So I think people who really think that AI could basically give one country a decisive strategic advantage by undermining nuclear deterrence. I think are mostly imagining this world where there's really fast or recursive AI takeoff. So I guess where an AI system goes from subhuman level to human level to superhuman level in a matter of weeks or months. How should we think about fast takeoffs in the context of nuclear deterrence?
- [57:30]
Sam Winter Levy
Yeah, so I think, taking a step back, the critical question here, I think is likely to be the relative speed of two different quantities or two different properties. The first is what is the speed in sort of calendar months or years at which AI progress proceeds and translates into advantage? And the second is what is the speed at which other states, whose nuclear arsenals might be newly threatened, adapt? And so if the first of those, how fast AI progress is taking place is faster than the second, how fast states are able to adapt? Which could be the case either because you own one of these kind of very fast takeoff worlds that you just described, or because states are just kind of slower to respond because of bureaucratic, bureaucratic reasons or political reasons or any number of other reasons, then you get these windows of vulnerability and instability with year to year fluctuation, which can be kind of particularly dangerous. And I think a fast takeoff just kind of exacerbates some of those issues. But even if AI progress is not so fast, as we've kind of discussed, if it kind of outpaces the ability of states to adapt, then you get these kind of dangerous windows of opportunity. I think one factor that kind of complicates this is that in the case of AI enabled intelligence processing, US adoption of AI capabilities could be relatively invisible to adversaries. So if it's just like you suddenly have a kind of discontinuous leap, or maybe it's continuous but just very rapid leap in the ability of states to use AI systems to process intelligence that they're already collecting, to process signals, data they're already collecting, then other states may not know that this breakthrough has occurred. And so potentially you could get more kind of significant windows of opportunity that open up as opposed to in the sort of industrial explosions where we're coding the ocean in underwater sensors and we're building massive missile defence architectures in that world, there are just going to be visible changes to the physical environment that other states are going to be able to see and respond to. And I think that will likely give states more time to respond with countermeasures of their own.
- [59:25]
Podcast Host
Okay, so I guess I'm still curious you've Kind of outlined all of these constraints that mean that even if AI is progressing significantly, it's still pretty difficult to get anywhere near a certain splendid first strike. How many of those constraints still hold? If we're talking about this super fast takeoff world, sure.
- [59:47]
Sam Winter Levy
Yes. I think even in these fast takeoff scenarios, some constraints are likely to remain. So first, on the technical side, some technical constraints will surely remain. I mean, as we've discussed, these are very hard technical problems to solve, and powerful AI systems won't be able to evade the laws of physics. But let's say a lot of these technical constraints evaporate. I mean, I would just flag that Edward Geist has an argument that some of these problems may just be computationally intractable, even for extremely advanced AI systems. I don't know if I would go that far, but the technical problems are very hard to solve. But let's say the technical constraints evaporate. There are still going to be a lot of political and institutional constraints that remain that will slow a state's ability to respond. Because even if technology changes overnight, states don't generally integrate advanced technology at the same speed. That rarely takes place. Doing the kind of testing that you need to do, updating doctrine, updating bureaucratic systems, all of this stuff takes much longer in general than just like a technological breakthrough, Especially when the stakes of getting it wrong are so high, when you need to avoid triggering a preemptive response, rehearsing thousands of steps with no room for error. And so even if technological breakthroughs occur overnight, I think the political constraints, the institutional constraints, some of the normative constraints about whether leaders are actually going to act on these powers when they're thinking about legitimacy and public sign off before rolling the dice, I think many of those constraints likely to last longer. How long is like an unknowable question. Huge variation maybe between states, between leaders in how quickly they respond, but that's certainly going to be a lagging factor behind the speed at which technology on its own is advancing.
- [61:31]
Podcast Host
Yeah, I do find it helpful to kind of reframe. I guess the thing that feels compelling to me is like the stakes will feel so high for a country that feels like they could gain this, this decisive strategic advantage, but they will also feel so high for the same country in terms of getting the possibility that they get this wrong. And then they face this existential risk of a second strike that is much bigger than they hoped it would be, even though they thought they maybe had this splendid first strike capability. So, yeah, I guess that's just. Yeah, that's feeling quite salient to me.
- [62:10]
Sam Winter Levy
And that latter risk is just much more salient and visible. States know what nuclear war looks like, I think, knowing what living in an AGI world where another state has won this race, I think there's just so much more uncertainty about what that looks like, whether it really is that bad an outcome. There's just huge uncertainty there.
- [62:29]
Podcast Host
Yep, that makes sense. Yeah. How should governments account for this possibility of fast takeoff in the context of nuclear deterrence?
- [62:38]
Nikita Lalwany
So I think the first thing to say is that governments should just take this possibility seriously, even if they think it's low probability in terms of how to respond. I think the lowest hanging fruit is probably just to increase state capacity. You need people in government who understand AI capabilities and how they're evolving and who are able to translate that knowledge into sort of actionable nuclear policies. Ideally, you'd also have dialogue between nuclear experts and AI experts, including between people in government and people in the frontier AI labs, to understand how the technology is evolving and also what that means specifically for nuclear deterrence.
- [63:18]
Sam Winter Levy
Yeah, I think building dialogue between these two communities is really important because I think the AI community, AI experts are really best placed to understand the technologies that are emerging, but they are not expert on nuclear weapons and nuclear deterrence. And conversely, the nuclear community knows that field of work, but it's just not necessarily entirely up to. Entirely on top of the frontier of AI breakthroughs. And so just building dialogue between these two states, I think between these two communities, both in government and outside government, I think is really important. If you're worried about kind of fast takeoff scenarios where things could start to. To change very quickly, we just want
- [63:58]
Nikita Lalwany
to emphasize that there's a lot of uncertainty with this fast takeoff scenario. And if it does happen, that could really change the calculus on the nuclear deterrence question. And so, as we've written in our foreign affairs piece, and as we've tried to say throughout this podcast, it's something that government should be taking seriously and monitoring so that if it does seem like we're approaching a fast takeoff world, there are things that we, you know, there are actions we take to kind of reduce the risks.
- [64:28]
Podcast Host
Yeah, I buy that. It does seem from the bit of learning I've done for this episode, that there's just, like, no dialogue at the moment, but that they do seem like incredibly overlapping kind of fields in this way that seems super important for people to be tracking. Okay, so that's fast takeoff speeds. So we've kind of talked about how one reason to think nuclear deterrence won't prevent one country from having a decisive strategic advantage, is that AGI might threaten adversaries, secure second strike. But there are also people that argue that AGI could give a country a decisive strategic advantage without even having much of an impact on nuclear survivability. So I think roughly the argument here is something like nuclear weapons as a kind of technology that determine, at least in large part, some parts of global balance of power are eclipsed by kind of faster, smarter, non nuclear forms of power. So I guess the question is, is it plausible to you that a state with super intelligent AI could coerce its nuclear peers, and I guess non nuclear peers, but through kind of economic or information or kind of cybernetic channels without needing to use nuclear arsenals?
- [65:48]
Sam Winter Levy
Yes, I think it's certainly possible that new forms of competition may emerge, new technologies may emerge, that kind of root around nuclear weapons. Potentially AI is going to matter for national security in all sorts of ways, many of which are kind of hard to foresee in advance. But so long as states retain second strike capabilities, as long as they can still credibly threaten to just unleash devastation on an adversary's cities, I think there are good reasons that nuclear deterrence is likely to still matter and is still likely to kind of constrain a state's action. I mean, you mentioned economic growth. I mean, the United States and South Korea together right now have a combined 1000 fold economic advantage over North Korea, but pretty limited ability to coerce the North Koreans to do things they don't want to do on issues they care about significantly. So AI could turbocharge a state's economic power. But so long as these systems of nuclear deterrence remain in place, there may still be limits on how much they can act. As for information operations, cyber operations, these are all things to monitor. But this track record of states using economic sanctions, information operations, cyber operations to coerce their nuclear armed opponents is extremely mixed. To put it very generously. It's just really hard to coerce nuclear armed states to do things they don't want to do on issues they care about sufficiently. And I think that's likely to persist so long as nuclear deterrence remains in place.
- [67:14]
Podcast Host
Yeah, it does feel important to notice these cases where countries that are radically dominated economically are still just completely unwilling to bend on the issues that are most important to them.
- [67:30]
Nikita Lalwany
In addition to things government should do in a fast takeoff world, there are sort of certain, no regrets, moves that they can take regardless of what world we're in. I think ensuring that policy Processes include AI experts alongside nuclear ones to encourage kind of dialogue to between these two sometimes disparate communities. Two, I think conducting rigorous reviews of nuclear systems to check for vulnerabilities that could be exploited by advanced AI, especially in cyberspace. I think Herb Lynn has a great book about cyber vulnerabilities within the kind of nuclear system. I think kind of shoring those up would be very helpful. Nice. I think. Three, carefully calibrating any statements about the need to race to advanced AI or the importance of being the first to develop a sort of wonder weapon that runs some risk of exacerbating risky and costly nuclear competition. And then finally, it's kind of more important than ever, I think, to maintain channels of communication and pathways to reduce the risk of inadvertent escalation. Calamity. There should be, maybe it's important to kind of double down on arms control dialogues and to sort of strengthen the significant ethical, political, legal constraints on the use of nuclear weapons in the first place.
- [68:53]
Podcast Host
Okay, that was a lot. And it was all pretty heavy. I'm curious if you guys have kind of a book or a movie recommendation that you think our audience might enjoy.
- [69:02]
Nikita Lalwany
Yeah, I mean, just a quib from a colleague who gave this answer in another podcast. A book I really loved this year is George Saunders's book, A Swim in a Pond in the Rain. He basically takes five short stories by Russian short story writers and then writes essays about those stories. And I think it's just a sort of wonderfully human endeavor, you know, telling stories about the world and then trying to understand them. And it ends up being this really beautiful, I think, meditation on kind of the value of reading and writing and sort of just understanding how human beings kind of think and operate in the world. So would highly recommend that book.
- [69:42]
Podcast Host
Nice. Okay, great.
- [69:43]
Sam Winter Levy
And maybe I can just give one TV recommendation, which is a British TV show called the Alps series, which basically started in the 60s. And they follow a group of kids.
- [69:53]
Podcast Host
This is my favorite.
- [69:55]
Sam Winter Levy
Yeah, sorry to interrupt.
- [69:57]
Nikita Lalwany
We're obsessed.
- [69:58]
Sam Winter Levy
The first episode in this show is called Seminar up and they get a group of seven year old kids in the 60s, I think in the 60s, and then they check back in on them every seven years. So 7 up, 14 up, 21 up. And you just see all the way through to, I think 63 up is the most recent one. And you just see these people's lives unfold over the course of many hours of documentary footage broken down into these kind of seven year increments. And there's just like, so much humanity conveyed by just the passage of time and the stuff of life that these people's lives are filled with, even if they're not like super dramatic lives, but just you just realize how much drama and tragedy and triumph is in every individual's life.
- [70:43]
Nikita Lalwany
Yeah.
- [70:43]
Podcast Host
I could not recommend that series more highly to Massive plus one. And also I could talk about it for hours, but we have used up all of the time. We have. So thank you so much for coming on. My guests today were Nikita Lalwany and Sam Winter Levy. I really appreciate it. Thank you.
- [71:01]
Nikita Lalwany
Thanks so much for having us.