← The Lawfare Podcast
Loading summary
Transcript49 lines
- [00:01]
Kevin Frazier
The following podcast contains advertising to access an ad free version of the Lawfare Podcast. Become a material supporter of lawfare@patreon.com lawfare that's patreon.com Lawfair also check out Lawfare's other podcast offerings, Rational Security, Chatter, Lawfare, no Bull, and the Aftermath.
- [00:32]
Advertisement Speaker
They say opposites attract. That's why the Sleep Number Smart bed is the best bed for couples. You can each choose what's right for you whenever you like. You like a bed that feels firm, but they want soft. Sleep Number does that. You want to sleep cooler while they like to feel warm. Sleep Number does that too. Why choose a Sleep Number Smart bed so you can choose your ideal comfort on either side. Sleep Number Smart beds start at $999. Price is higher in Alaska and Hawaii. Exclusively at a Sleep Number store near you. See store or sleep number.com for details. I want my dog to live a long, happy life. Maybe even hit 19. So I feed them Ollie. Ollie's fresh and nutritious human grade meals are made to support their health and happiness with protein packed recipes dogs go crazy for like beef with sweet potatoes, turkey with blueberries, or lamb with cranberries. Honestly, you might start thinking, dang, my dog eats better than I do. And that's probably true when it comes to ollie. Head to ollie.com pup and use code Healthy Pup to get 60% off your first box of meals. Plus they offer a clean bowl guarantee on the first box, so if you're not completely satisfied, you'll get your money back. That's o l l I e.com healthypup and enter code Healthy Pup to get 60% off your first box. Feed your forever friend with Ollie.
- [01:54]
Dan Hendricks
AI is if if you have a super intelligence, it creates super intelligence. Broadly speaking, it can make many types of weapons research. You could have some breakthroughs in there and that could stabilize it. Now, you know, now we could come up with a scheme for a good anti ballistic missile system which reduces adversary second strike capabilities, which totally gets rid of mutual assured destruction, which is totally destabilizing.
- [02:15]
It's the Lawfare podcast. I'm Kevin Frazier, the AI Innovation and Law Fellow at the UT Austin School of Law and a contributing editor at LawFair, joined by Dan Hendricks, director of the center for AI Safety.
- [02:30]
Torts will add some restrictions to these models compared to where they are now, but I think that's totally reasonable and what you would need for longer term economic stability.
- [02:42]
Today we're discussing his paper Superintelligent Strategy, co authored with Former Google CEO Eric Schmidt and Scale AI CEO Alexander Wang. As AI capabilities accelerate, we're not just talking about automation or economic shifts. We're facing potential paradigm shifts in military deterrence, cyber conflict, and global power dynamics. Some experts argue that an AI arms race is beginning to resemble the nuclear age, but with even greater unpredictability. Our guest today has taken that analogy a step further, introducing a provocative new concept, Mutual Assured AI malfunction, or mame, a deterrence regime where great powers might sabotage each other's AI advances rather than risk unilateral domination or catastrophic loss of control. Dan and his co authors, Eric Schmidt and Alexander Wang argue that we need a structured approach to superintelligence strategy, drawing on historical lessons from nuclear deterrence, non proliferation and technological competition. So, Dan, there's so much to unpack there, but one thing I want to start with is why now? Why did you and Eric and Alexander choose this moment to release superintelligence strategy?
- [04:00]
Well, great question. I think we're entering a different paradigm, the reasoning paradigm of AI. So they're improving quite rapidly behind the scenes, and we're seeing some of that publicly. But basically it's improvement in software engineering and other STEM skills and other domains that require a lot of reasoning is going so quickly. So it could be the case that even in the middle of the year, AI becomes much more salient. So when people are thinking, what should we do? There's at least something lying around. So that's some of the motivation as well as the competition between the US and China has become substantially more close. So last year there was a strategy proposed to have a Manhattan Project project where the US just increases its lead and then they build super intelligence, and then they get to tell all the other countries what to do, because that gives them a monopoly on intelligence and potentially a strategic monopoly on power. So consequently, this strategy of doing a Manhattan Project of, well, we'll just race ahead of them isn't looking like that'll be as robust. We can't assume that one will have a substantial lead over the other.
- [05:14]
The moat is gone, so to speak. And I think perhaps the drawbridge is up too. After the Paris AI Action Summit, where Vice President Vance made clear that, yes, we are racing ahead, and so it's hard not to read that paper in light of these racing dynamics and with super intelligence strategy in mind. We have so many conversations about AGI, artificial general intelligence to transformative AI. What does super intelligence mean with respect to these other terms and how should we think about it?
- [05:48]
Yeah, yeah, great. Question. So maybe let's just first take what is AGI? AGI people define this differently. Some people would say, like, you know, when it can do the things that a typical person can do. Intellectual tasks or digital types of tasks, virtual ones, that's a definition. But it keeps changing. Transformative AI is often meaning when there's high economic impact. So it's not just do you have the technology, but is it diffused? Because sometimes there might be a lot of, there might be a lot of red tape that limits diffusion or procurement processes in the government may slow down diffusion substantially. And superintelligence is when it's basically vastly better than all the world's experts. It's just at everything intellectually. So it's not just at human level. It's for virtually everything is just way, way, way better. Weapons research, AI research, running a business, everything. Those are three different distinctions. It's made a fourth that I think for in the year coming years, it's sometimes more useful to look at specific capabilities. When is an AI capable of doing a particular risky thing? So is it, does it have expert level virology capabilities? That's quite relevant because that can be used for both good and bad. That's very dual use or cyber capabilities for cyber attacks. So those come at different times, those capabilities. So it's somewhat difficult to think about terms like AGI because they're all encompassing. It can be very good in some ways and very bad than others. The intelligence frontier is quite jagged. So it would not Surprise me if AIs are publishing papers or doing the heavy lifting and publishing mathematics papers before I have a robot that can fold my laundry at home and have that diffused throughout the economy. So thinking of it in all or nothing way tends to get people caught up. And I think for the geopolitics of this and, and for when we need to be coordinating, it's often useful to speak about specific capabilities. But I do think these notions are all useful, including the notion of superintelligence, because it's defined in a way that if you actually do get it, that would be plausibly quite destabilizing.
- [08:15]
Yeah. And what I think is really valuable about this framing of superintelligence versus AGI is it seems like everyone wants to debate precisely when and how we may reach AGI. But to your point, if we have a super intelligent system with respect to cyber or with respect to creating biological weapons, that's all we need to be concerned about to then trigger this strategy of mutual assured AI malfunction and so.
- [08:42]
Well, it would depend, it would depend, it's for some different capabilities there. So for bio, bio would be very risky from rogue actors, but be less interesting to say a superpower for the same reason that we have the Biological Weapons Convention, that these are chemical and biological weapons are poor man's atom bombs, so to speak. And so it would make sense for countries to, or superpowers to cooperate so that those destructive capabilities are not widely proliferated because rogue actors could then more easily threaten their survival. So for those capabilities, those are relevant for trying to coordinate. But in terms of incentivizing sabotage, I don't think those would necessarily do that. Maybe some form of cyber could, but I think other types of capabilities could be more destabilizing, like AIs that help make oceans more transparent so that you can see where nuclear submarines are, know where the mobile launchers are, or AIs. If you have a super intelligence, it creates superintelligence, broadly speaking, it can make many types of weapons research. You could have some breakthroughs in there and that could destabilize things. Now, you know, now we could come with a scheme for a good anti ballistic missile system which reduces adversary second strike capabilities, which totally gets rid of mutual assured destruction, which is totally destabilizing. So those are all possibilities with a superintelligence. And so you need to be betting that actually we've kind of explored technology to its fullest and there aren't going to be much returns from extra intelligence and there won't be breakthroughs that are destabilizing. Is, is what you would need to be betting on if, if a super intelligence is achieved. And if you're thinking it's not going to be destabilizing, which I think is not something people will buy. So.
- [10:36]
Right, right. So thinking of this term mutual assured AI malfunction, obviously folks are probably quickly drawn to the idea of mutual assured destruction. What is the nexus between these two terms and what is the distinction that you want to draw and emphasize with MAME as opposed to mad?
- [10:56]
Yeah, so I, I largely use the acronym just to, to allude to, to mad. There are of course many differences. This isn't just a repeat of nuclear history. I mean, I'm not advocating for a Manhattan Project for, for instance, and I think that it would just be extremely escalatory. You can't do a Manhattan Project secretly in today's day and age. You can't build a trillion dollar data center out in the desert. And there's these things called satellites and people would Notice if several hundred top researchers are suddenly missing.
- [11:29]
I don't know, Dan. Have you been watching paradise on Hulu? I mean you just blow up a rocky mountain and dig a big hole and there you go, you put a data center down there.
- [11:39]
Yeah. So there are some just analogies primarily throughout the paper be treating this as a potentially catastrophic dual use technology. Because that's what it is, it is dual use and definitionally it is potentially catastrophic. If it's expert level in virology, if it has these sorts of cyber capabilities, that's potentially catastrophic. And it has both civilian applications and weapons applications. And so I think seeing what the case was for chem and bio and nuclear, which were potentially catastrophic dual use technologies, we had to do a game of managing, of capturing the civilian benefits for energy or for products or for health care and reducing the downsides from the weaponization part of it. I think we're turning to, to Maine versus mad. Yeah, I think there's, I think there's a deterrence regime. We have, we have deterrence regimes for a lot of things for nuclear, but also for cyber attacks on critical infrastructure between superpowers, meaning destructive ones like China won't or is not taking down a power grid. They could, it would not be the most difficult thing for them to do, but we would probably do the same back to them and trying to destroy each other's economy. It's like there's so much economic interdependence or complex interdependence that's not very incentive compatible. And with nuclear as well, there are substantial deterrence to have people stave off from doing things that would just collectively harm everybody else. So in the case of AI, if there is an opportunity to get super intelligence, if one's on the verge of it, if one were to initiate a process such that one might have it, say a year later, this process could look like the following. Imagine that you've got an AI that can do AI research really well. Then you can make 1,000, 10,000, 100,000 copies of these things and run them in parallel. OpenAI maybe has got 200 or so researchers. Now they've got 100,000 world class ones. Wow. And they're operating at 100 times the speed of a normal human. They can wear around the clock. That can be quite explosive. This isn't a new point. This is a point many, many decades old. People like Alan Turing and others pointed this out and others have for years said, you know, that this would be quite explosive. Potentially an intelligence explosion. The at most three or top three most cited AI researchers. That is number one, Number two, number three in terms of citations, Ben Geo Hinton and Elius Eskever all think that you might lose control of this process that would put lots of other states at risk. So they wouldn't like the sound of that. But if. If a state does successfully control it, that would also be something that would threaten their survival because then they could use it against them and potentially reduce their second strike capability. So it could be quite disruptive and destabilizing. And so I think that states wouldn't get away with that. As AI becomes more salient, then states would think to deter this type of possibility and forestall it, unless it were done under more agreeable conditions, that is with like clear benefit sharing or something more multilateral. But that would of course be much later on. So that's sort of the gist of it. But the analogy to Matt is just pointing at the shared vulnerabilities, as there is for critical infrastructure and for financial systems too. So it's a broader phenomenon, I think, fairly precedent. I think we'll just see a similar thing for AI.
- [15:07]
And I think when folks read your paper, even the kind of layperson version, you all do a great job. You have a layperson version and an expert version, which I love and I hope more people follow because we should all be having these conversations. But I will admit, anytime in a paper you see a game theory diagram come in, I think a lot of people are like, all right, well, Dan, sorry, you lost me. I'm not going back to economics 301, but I do just want to highlight that you all have some very practical ideas of what MAME would actually manifest in. So can you walk us through, for example, why would we want nations to move their data centers outside of urban areas? What does that have to do with MAIM as a theory and maybe describe some more practical steps? You see?
- [15:58]
Yeah, so imagine that states were wanting to create credible deterrence. They want the deterrence to be credible, and they want it to be minimally escalatory as well. You know, an extremely escalatory thing would be a kinetic strike. You don't need that, though. By default, you can just disable the surrounding power plant. If the surrounding power plant is affecting cities, this would be much more irritating. So in the nuclear age, there's the concept of city avoidance, where later on it was, let's just have our nuclear weapons point at each other's military bases instead of at each other's cities. Initially that was a good idea. So it would be nice if we would have sort of city avoidance for this. So that mutual assured AI malfunction isn't sort of mutually assured human destruction. You don't want to overlap between those. You want it to be more, more surgical. So that, that's an example type of thing that states could do to reduce the escalation potential of sabotage, given that they will have incentives to make a conditional threat of sabotage against each other. There are other things that states could do such as increase their espionage programs of each other's projects. I don't think China really would need to. The US is basically an open book in terms of AI. I mean, here's how they could do it. You can just do a zero day on slack, the slack that communicate workspace communications thing, and they can see basically everything that's going on, very easy. And so that would get you Xai, that would get you OpenAI, that would get you anthropic, that would get you Google DeepMind. They also have a double digit percentage of the top researchers at all of these organizations are of Chinese nationals. And they are extortable. This isn't saying, oh, they're spies from the beginning, I'm not saying that, but they're extortable. They have family back home. Whenever they go back home, it might be a condition before they can leave if they divulge some stuff. So it's the information security I don't think is going to be fixed in that way. But it would make sense for the US to improve its ability to spy on programs abroad as well as develop cyber attacks forged for the data centers and for the surrounding power plants to make the deterrent more credible so that one doesn't need to resort to more hostile actions such as kinetic strikes, which I think would be pretty needless. Now I should note that when I'm describing this, I'm describing where the world might be at later. I'm not saying that currently China's talking about this sort of thing. I mean the paper is new. I mean it came out just very recently. So this I don't think was on their radar. I'm not describing that. This is how they're thinking. And here's the first introduction paper. So I think that AI will become more salient this year as there's maybe some big AI agent advance. And then people start thinking through the implications and becoming more important. Decision makers will be convinced of what my co author Eric Schmidt calls the San Francisco school, the school that AI is very powerful. AI is potentially this decade and Mass automation or huge automation waves is potentially this decade not something that's very far off, but potentially on the horizon. Even in the next, even in the next few years you could get AGI, so to speak. So I think more decision makers will be buying into the San Francisco school later. Whenever that happens then then I think the strategic implications would be that potential intelligence explosion giving you a super intelligence or some really rapid AI research and development giving you a super intelligence is something that we're not going to let one of each other do because that could suddenly alter the balance of power in a destabilizing way and so it would be prevented.
- [20:00]
We're speaking in mid March right after the request for information for the AI action plan closed and I'm hoping that somebody included in their submission hey Slack, you need to really bolster your cyber.
- [20:16]
Whoa, wait, I don't know if I was not claim that therefore the companies should be trying to improve their infosec substantially to be state proof. The reason is I don't think that's very tractable. Are we going to fire all the Chinese nationals? I think that would totally undermine U.S. competitiveness. Are we going to require all the people to get security clearances as well? So I think you just get rid of your workforce and then transplant them in some desert or have them work out of a skiff. And so I just don't really see that type of thing happening. So what happened in the in the nuclear age is things like the open Skies treaty where actually for stabilizing things they allowed mutual inspection. They reduced information problems and didn't try keeping things some select things as secret from each other to improve stability. So I think very strong state level proof or super power proof information security if you'd work in it would take would be very difficult, would take many years, would undermine your competitiveness and would also further I think make things less stable as it happens. So there are other routes. You'd also need to call on iPhone to reduce the zero days there. You could just spy on some key decision makers on their iPhones and that'd be another way you could get a sense of what's going on.
- [21:42]
Advertisement Speaker
But yeah, this episode of the Lawfair podcast is brought to you by Wild Grain Wildgrain is the first bake from frozen subscription box for artisanal breads, pastries and pastas. Wildgrain's boxes are fully customizable with great options no matter what your tastes or dietary restrictions. They even have gluten free and vegan options. Best of all, all items. Bake from frozen in 25 minutes or less. I love to bake, but keeping up with national security developments often gets in the way. Wild grain gives me fresh from the oven sourdough in 25 minutes when I can't plan a day ahead to make my own. Are you ready to bring all your favorite carbs right to your doorstep? Be sure to check out Wildgreen so you can begin building your own box of artisanal breads, pastas and pastries. For a limited time, Wildgreen is offering our listeners $30 off the first box, plus free croissant in every box. When you go to wildgrain.com lawfair to start your subscription, that's wildgrain.com lawfair or you can use promo code lawfair at checkout.
- [22:47]
Kevin Frazier
Did you know? Fast Growing Trees is the biggest online nursery in the United States with thousands of different plants and more than 2 million happy customers. They have all the plants your yard needs like fruit trees, privacy trees, flowering trees, shrubs and so much more. Whatever plants you're interested in, Fast Growing Trees has you covered. Find the perfect fit for your climate and space. Fast Growing Trees makes it easy to get your dream yard. Order online and get plants delivered directly to your door in just a few days without ever leaving home. Their alive and thrive guarantee ensures your plants arrive happy and healthy. Plus, get support from trained plant experts on call to help you plan your garden. I've been redoing my cabin in the woods recently and gutting a house is pretty expensive. Instead of paying thousands more for expensive landscaping, I have a honey crisp apple tree. Coming from Fast Growing trees to plant, I couldn't be more excited about the shade and frankly the apples and the cost savings is really hard to beat. You don't have to drive around to nurseries and big garden centers and choose plants from a limited assortment. Fast Growing Trees makes it easy to order online and your plants are delivered to your door in just a few days. Whether you're looking to add privacy, shade or natural beauty or apples to your yard, Fast Growing Trees has in house experts ready to help you make the right selection. With plant experts on call to offer growing and care advice, you can talk to a plant expert about your soil type, landscape design, how to take care of your plants and everything else you need. No green thumb required. This spring. They have the best deals for your yard, up to half off on selected plants and other deals and listeners to our show get 15% off their first purchase when using the code Lawfair at checkout that's an additional 15% off at fast growing trees.com using the code lawfare at checkout. Fast growing trees.com code lawfare Now's the perfect time to plant. Use Lawfare to save today. Offer is valid for a limited time. Terms and conditions may apply.
- [25:26]
Advertisement Speaker
Thumbtack presents the ins and outs of caring for your home out Indecision, overthinking, second guessing every choice you make in plans and guides that make it easy to get home projects done out beige on beige on beige in Knowing what to do, when to do it, and who to hire. Start caring for your home with confidence. Download Thumbtack today. This is a mini meditation guided by bombas. Repeat after me. I'm comfy.
- [25:59]
Dan Hendricks
Comfy.
- [26:00]
Advertisement Speaker
I'm cozy.
- [26:01]
Dan Hendricks
Cozy.
- [26:02]
Advertisement Speaker
I have zero blisters on my toes. Blisters. And that's because I wear bombas. The softest socks, underwear and T shirts that give back. One purchased equals one donated. Now go to bombus.com acast and use code acast for 20% off your first purchase. That's B O M b-a dash.com acast and use code ACAST at checkout. If you wear glasses, you know how hard it is to find the perfect pair. But step into a Warby Parker store and you'll see it doesn't have to be. Not only will you find a great selection of frames, you'll also meet helpful advisors and friendly optometrists. Yep, many Warby Parker locations also offer eye exams. So the next time you need glasses, sunglasses, contact lenses or a new prescription, you know where to look. To find a Warby Parker store near you or to book an eye exam, head over to warbyparker.com retail.
- [26:56]
Dan Hendricks
So listeners may be thinking, okay, Dan, you know I'm I'm bought in. You've outlined this idea of maim and called a lot on states to think through these specific escalatory steps, making sure we don't get too high on the retaliatory measures too quickly. They may be thinking, what's the responsibility of the labs here? Do you all call on the labs to have any role in maybe scaling things back or slowing things down? Or are these racing dynamics between the companies, between these countries, something that's kind of just baked into the idea of MAME being a useful strategy or a necessary strategy?
- [27:35]
I'm assuming the competition will increase at the speed that the companies can proceed at, so I'm not thinking they're going to voluntarily halt there. Instead is channeling competition later stage into other domains so instead of competing to have the first we have super intelligence should be totally destabilizing. You wouldn't get away with it. They can compete in other things like in, you know, they could compete in robotics for their economy. They can compete making AI chips. They can, you know, they can, they can compete in drones or what have you. So there are many axes for competition, but the competition doesn't look like the destabilizing sort. So there was competition between the US and Soviet Union, but you know, they end up having agreements and they weren't trying to, you know, trying to get a nuclear monopoly relative to the other. Later on when both of them had it, this wasn't a, so they, then it was just a focus of containment. And with China, I think it's just strategic competition and continued strategic competition, but avoiding things that could, could destabilize that, that competition now for, for companies. So what are they to do? I mean, I think clarifying or I think they should, you know, have to test their models for malicious use applications. That seems like it takes like a week or something like that before the release. This is not. Adding the safeguards are pretty easy for the catastrophic forms of malicious use like bio safeguards. So I currently think for chatbot safety and I speak as advisor for xai, so we have our risk management framework and I don't think it's very difficult to implement as it happens. So I think the claim that there's a huge tension is overwrought between competitiveness and safety. Adding the safeguards is pretty easy. If people think it's not, it's probably because they just don't know what they're doing. That said, this could change later. Imagine we get AI agents. There could actually be some trade off between getting the technologies out there as quickly as possible for everybody, in between diffusion and safety, largely due to torts. So if AI agents can scurry around on people's behalf and if AI companies get their way that, oh, we have no responsibility for what people instruct our agents if they tell them to go harass someone or go hack or something. Although it was on our server and we built the technology, we had nothing to do with it, there will actually be some trade off from tort law which will put some restrictions on it and make them have to watch things more carefully, this might mean that the costs will go up a bit because they're going to try and shift that liability onto an AI insurance company or something like that. So torts will add some restrictions to, to, to these models. Compared to where they are now. But I think that's totally reasonable and what you would need for longer term economic stability. But there's a, you know, a potential tension but I think it's pretty, I think it's pretty reasonable if, if AIs are having to abide by the reasonable person standard if they're acting as a fiduciary on behalf of individual people.
- [30:53]
So, so we've, we've analyzed one part of the three pronged superintelligence strategy, deterrence via mame. But you all also discuss non proliferation and competitiveness. I wanted to drill down a little bit more on non proliferation. What does that mean in this context and why do you think this is an important part of that overall strategy?
- [31:15]
Yeah, so for other potentially catastrophic dual use technologies, Chembio Nuclear, there was some precedence of like inspection and making sure it's not. The inputs to create the weaponized versions of those are restricted. You've got the US select agents list, you've got the Australia groups Schedule 1 chemicals for instance and you have the IEA others that do verification for the application of fissile materials. So this is a pretty common thing. And I think for, for AI as well, you would want safeguards making sure you're not proliferating the capabilities like imagine in the future extreme cyber capabilities to random rogue actors because cyber offense or cyber attacks are offense dominant with respect to critical infrastructure or extremely high virology capabilities and non proliferation of rogue actor or two rogue actors of the things that can make those sorts of capabilities, namely AI chips. And these chips are, you know, $30,000 plus a piece. So it's just like don't, don't, don't sell it to them. Let's make sure we know where the AI chips are at. And I think this is a place where superpowers can coordinate such as the US and China, that I don't think it serves either of their interests to proliferate those sorts of capabilities to rogue actors. Rogue actors I'm using as a shorthand for like imprudent or like terrorists or ones without self preservation property or ones that are not particularly prudent or rational.
- [32:56]
Advertisement Speaker
Right.
- [32:56]
Dan Hendricks
And just to press on that for a second though about whether or not for example China might have an interest, one could argue in I don't know, allowing some pretty advanced AI systems to make their way into the hands of, let's say some, some bad actors, some non state actors who want to wreak havoc in the Middle east and therefore draw the US into a Middle east conflict. Is that a potential threat scenario that you think has some that might detract from that sort of coordination between China and the U.S. so this is why.
- [33:29]
It'S useful to think about specific capabilities here. So for like bio, I don't see why that would be an incentive of theirs, because those can evolve. I think China's particularly vulnerable, relatively speaking, given the homogeneity of their population to two bioweapons relative to more diverse countries like the U.S. which is, you know, an extremely grim thing to talk about. But that's bio. It's a real horror show to think about. But it could, it could depend, I think, in terms of its incentive compatibility. And I think it'd be probably pretty imprudent given that a lot of the advanced capabilities are very tangled up with each other. So it'd be difficult for strategically enabling them with and get proliferating some of the capabilities to them while not others. So I don't quite see that. But it'd be an interesting thread to pull on, but I think it's probably not the most concerning compared to.
- [34:28]
We'll save it for a future tabletop exercise. But for now, I'm curious, you know, you release this paper with Eric Schmidt, with Alexander Wang, certainly folks that other people are aware of. Is your phone just blowing up? Are senators texting you? Thanks, Dan. We've got Mame now. It's all figured out. Or what's the reception been like?
- [34:50]
I think the reception of the national security establishment has been, has been good. Among people who are interested in AI stuff, of course there are people who are like AI or you know, what hype or something like that. And so there's nothing worth taking seriously there. And I think that will just like not be increasingly harder and harder to defend position as time goes on. But among people who are more into this, I think the previous state of the art strategy was the sort of Manhattan Project which had just like a lot of flaws and is like looking too late and would be like extremely escalatory, is the lowest impact that it would have and the. The highest. The hastening other. Other states demise or humanities demise. It's a really risky business. So I think it's a very clear improvement over that. So yeah, I was like in D.C. speaking to people about it last week and at the relevant places like in Eisenhower building and so on and so it seems, seems. I think, I think it'll. I think the main thing is socializing it and then there's some moment where people are aware of the salience of AI has increased and that's the main thing lying around and it's well understood and confusions are taken care of beforehand.
- [35:59]
So, yeah. So is it your assessment that generally policymakers in D.C. are still lacking a little bit of nuance in diving into the specifics of the threats posed by AI, where maybe that sort of SF school of AGI is coming, super intelligence is coming. That message hasn't reached the Hill in a meaningful way.
- [36:21]
So I would definitely distinguish between different agencies and different parts of government. So I think like Congress, for instance, is generally more behind on this and it's a lot harder for them to do stuff in terms of the executive office of the President or the. We'll say that the White House and surrounding parties, I think they're generally more technically with the program. If they're handling tech, they know this stuff fairly well. I think in the National Security Council in the prior administration as well, was very plugged into AI and many of their key people assigned plot were, you know, roughly in the San Francisco school, so to speak. Yeah, I mean, I'm, I'm, I'm very optimistic currently about the, the competence on the geopolitics of AI of the current domain administration, but, you know, different parts of different agencies that aren't really supposed to be on top of AI. I think a lot of them, you know, still don't know as much, but I think a lot of key actors do.
- [37:24]
And a prominent part of the paper centers on the idea of compute governance. And with respect to that, I think there's a connection to things like chip manufacturing at home, domestic chip manufacturing. Are you concerned about some of the pivots potentially away from the CHIPS act, for example, diminishing the sort of political and financial support for domesticating US production of chips right here in Arizona, for example, and elsewhere?
- [37:54]
So the, the China spends on the order of a CHIPS Act a year, so roughly 50 billion in increasing their domestic semiconductor manufacturing capabilities. I think if China invades, blockades Taiwan, there goes the US's compute advantage. Right now in terms of the competition between them, they're roughly tied on pretty much all axes. Like they've got similar models, they both have access to the Internet to scrapes, they've got similar data that they, similar quantities of data that they have access to for text data they've got. China has somewhat of an energy edge because it can build power plants and things like that more quickly, but that's not necessarily the most decisive factor. And the US's main advantage is on chips and compute AI. Chips so because 90 plus percent of the value added, the AI chip supply chain is in Western allied countries, countries, but the endpoint of the supply chain is in Taiwan. So if there is an invasion, whoopsies, there goes the US's advantage there. So I think it would make sense to have a more secured supply chain. Maybe other allies, for instance, could start doing domestic manufacturing themselves too, if the US would do less. But I would separate between the CHIPS act as implemented and a broader push to try and increase supply chain robustness. I think that's, you know what, a lot of the Trump administration is actually focusing quite a bit on reducing economic interdependence on key things there. So that could be possible. But I mean, the CHIPS act, for instance, didn't actually focus on AI chips that much. A lot of that money went to intel, which doesn't even make serious AI chips. So it was largely a reaction to Covid, where supply chains got messed up or AI chip, or excuse me, where chip supply chains got messed up. But I'm talking about targeting things on AI chip in particular. And I think there, there might be more that might become more salient and there might be more possibility. But yeah, that'll be important. Otherwise, you know, it's 20% plus chance that they invade this decade according to most or cringe many forecasters. So some put it higher. So that's kind of a clear loss condition that we're sleepwalking into. So I should hope that something's done about it. We're not really good at risk management.
- [40:31]
Well, speaking of nightmare scenarios, folks are probably familiar with your name as the executive director for the center for AI Safety, and you all have done a lot of work spreading awareness of potential catastrophic and existential risks posed by AI. I'm curious, we're in 20, 25, three years removed from the sort of chatgpt moment. Are you sleeping better at night or worse at night? Are you more panicky? How have things evolved over time?
- [41:04]
Yeah, I think it's been two years. But the, it's a great question. I mean, I think there's more of a strategy now, so that's nice. I think overall, things considered speaking, I don't know, maybe I'd be more like if this is a catastrophe or ends up extremely poorly for all of us or something, maybe I'll put that at more like 50, 50 as opposed to like 80% as it was some years ago. But I think we'll gain substantial information even in the next month, months of how this is going to be shaking out. In terms of the capabilities of these AI systems with this new reasoning paradigm, much faster rate of development than with the previous generation. So we'll see if there start to be some substantial economic impacts on software development this year. And that could really affect the conversation a lot. So yeah, so that's when I give has high uncertainty and a lot can change and will change quickly.
- [42:08]
And before we let you go talk more about these important new strategies and frameworks. I'm curious if we see Congress struggle to pass meaningful legislation around some of these key risks. Do you continue to think that states have, have a role in maybe putting in some more robust safeguards? Just with respect to the things you emphasized earlier about just even week long testing of a model to ensure certain capabilities aren't too likely?
- [42:39]
I think most of the risk reduction will come actually at this international or geopolitical level. And I'm not necessarily meaning through like the UN but just like the US China and Russia and their sort of dynamics and whether they stabilize things so one won't be bottlenecked by Congress for this. There still would be useful basic stuff like reporting what the capabilities are to the government. I mean, I think be very difficult. Imagine being in the Situation Room and you heard you got news from CIA that oh, China is a new big project or something, what should we do? And then they're like, well, where are companies at? Well, I don't know, we stopped requiring that they tell us what's going on, you know, two years ago. Okay, fantastic that, you know, China probably has more insight as to what's going on at the AI companies than our own government. You know, I think that could happen at the, the state level. I think even Congress requiring that whatever you're reporting to the eu, you also report to us, wherever you're reporting to foreign governments, you report to us. Seems pretty reasonable to me. But you know, obviously some political factions will say this is you're trying to take over the world or something, you know, something hyperbolic like that basic information stuff. So. But that could be done through other means. You wouldn't necessarily need Congress for this sort of reporting. So like through the Defense Production Act, I think many of them would give it willingly, but there's just even having the appropriate agencies asking for it. But I don't think, I think that there is a distraction or there is a. People are really fixated on the AI companies for affecting this, but I view them as basically most of their actions and behaviors are roughly predetermined by their overwhelming, overwhelming incentives and competitive pressures. So they should be like, you should think that they're going to spend 90 plus percent of their resources and effort on securing a next huge race, that they can buy more computer. And that's their primary activity. Whatever's required for that. They're solving for X. Whatever lets them raise that amount. That's their main objective. And then there's some safety noises that some will make along the way in varying degrees, but those won't really make much of a difference. So I think people get very fixated on performative safety things like, oh, they put out more safety research, more safety blog posts. These don't actually make much of a difference. What make a difference is whether states deter each other from the superintelligence type of stuff. Which countries have an edge in AI, like supply chain ones, if they have guaranteed supply chains or not. And if we limit proliferation of potentially catastrophic use capabilities to rogue actors, I think that's a much higher thing. So there's some role for the companies to play. But I, it kind of feels like TMZ or something like that where it's like some celebrity things of, oh, Demis versus Sam versus this is, this is not. They, they do not have much ability to act otherwise. They are, they're very constrained by competitive pressures. Their personalities are interesting, but they're, they're, they can't, they can't go up against those competitive pressures in any very substantial way. So viewed as largely preset.
- [46:00]
Yeah. It draws to mind the idea of the transparency reports released by social media companies isn't exactly changing the game from a content moderation standpoint.
- [46:10]
Yeah. They've got a better filter on their cigarettes or something like that. So they're the good one. I think people, in terms of their pressure, they should be getting on their governments to get on top of this. That's really. Instead of whining at OpenAI or whatever to DeepMind, anthropic, xai, etc. To do more safety stuff or something. Something which, you know, maybe they'll allocate 1% more of their budget at best if they'd pressure their governments to, hey, you guys can kind of sleep at the wheel here and you should get on top of this. That I think would be a more effective thing to push for if people are thinking to do advocacy.
- [46:52]
All right, folks, well, you've got your homework from Dan. Go get it. And hopefully he'll give you a good grade. But for now, Dan, we'll have to let you go. Thanks again for joining.
- [47:01]
Yeah, thanks for having the paper. Is at National Security AI and we.
- [47:05]
Will be sure to include that in the show notes. Until next time, folks. The Lawfare Podcast is produced in cooperation with the Brookings Institution. You can get ad free versions of this and other Lawfare Podcasts by becoming a Lawfare Material supporter at our website, lawfairmedia.org support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you get your podcasts. Look for our other podcasts including Rational Security, Allies, the Aftermath and Escalation. Our latest Lawfare Presents podcast series about the war in Ukraine. Check out our written work work@lawfaremedia.org the podcast is edited by Jen Pacha. Our theme song is from Alibi Music. As always, thank you for listening.
- [48:04]
Advertisement Speaker
If you wear glasses, you know how hard it is to find the perfect pair. But step into a Warby Parker store and you'll see it doesn't have to be. Not only will you find a great selection of frames, you'll also meet helpful advisors and friendly optometrists. Yep, many Warby Parker locations also offer eye exams. So the next time you need glasses, sunglasses, contact lenses, or a new prescription, you know where to look. To find a Warby Parker store near you or to book an eye exam, head over to warbyparker.com retail.