A (5:52)

So the brief summary is that In June of 2024, during the Biden administration, Anthropic and the Department of Defense at the time the Department of Defense signed a contract for the use of CLAUDE in classified settings. So this is in, you know, intelligence gathering and analysis, active combat operations, things like this and that contract had usage restrictions that applied to a number of different things. But the two ones that you know are now the subject of discussion have to do with mass domestic Surveillance and autonomous lethal weapons. Autonomous Lethal weapons are weapons that can autonomously close the kill chain, which is to say they can autonomously identify, track and kill human targets. So with no human intervention whatsoever, that contract was expanded by the Trump Department of Defense in July of 2025. When I worked for the administration, though I will say I had no role in negotiating that deal, but it was expanded at that time, and basically it had the same restrictions. The terms changed very slightly, but those restrictions did not change. Then in the fall of 2025, the Emil Michael, who is the undersecretary of War, the department was changed, the name was changed to Department of War. The undersecretary of War for research Engineering was confirmed by the Senate. And he, as a part of a broader review of the operations of his office, had AI under his remit, looked at these contracts for Frontier AI. And as far as I understand it, he determined that these contracts had usage restricted restrictions, which were onerous. And so this is not what the previous political officials in the Trump DOD before him thought. It's also not with the many lawyers who I'm sure signed off on this agreement, both for Biden and Trump thought. But he decided that these terms were obviously onerous, and so he sought to renegotiate them. This was starting several months ago. He began to try to renegotiate them. And Anthropic, I think, did renege on some of its red lines, but not particularly on the red lines related to mass domestic surveillance or autonomous lethal weapons. And so rather than just canceling the contract, they have canceled Anthropic's contract. They now have designated CLAUDE a supply chain risk. And what this means is that no DoD contractor can use Claude, but we don't know the exact details of what that means. Does it mean at all? Does it mean just when they're doing DoD business, only particular subsets of DoD business, we don't exactly know those details will probably be available in the relatively near future, but they're not at this exact moment that I speak to you. And so, yeah, that's kind of where we are.

B (8:54)

And just to jump in with a few points here, one is that is a really extraordinary step that to say, look, we don't agree over these usage restrictions, we're going to go search different commercial partner is something that maybe wise or unwise, but that would be a relatively normal step to designate Entropic, a US Company, a supply chain risk, which is a designation usually reserved for companies like Huawei, which are effectively controlled by the Chinese state, is really quite extraordinary. And I guess we can expect a. It matters a lot whether that is just in kind of dealings with the Department of War or whether it's in general. That would be an even broader escalation. I suspect that we can expect certainly Entropic to sue against this designation and quite possibly to prevail, given how unprecedented the use of this designation is in this context. But I kind of want to get to a couple of the broad implications here. The first is talk about what the nature of these user restrictions is. We talked a little bit about autonomous weapons and the sort of complete sort of kill cycle. What about the fears about domestic political surveillance? Why is that one of the major sticking points here? It's one that's been talked about less in the debate over this. How is it that these new AI models facilitate domestic political surveillance? And why is that such a concern?

A (10:15)

Well, there's two things that are worth mentioning here. The first is that it is illegal for the government to directly collect sort of private data about American citizens. Right. So the government can't, without a warrant, without some extraordinary circumstances, the government can't wiretap my phone. It can't put cameras in my house and record what's going on inside my house. But it can often acquire data from commercial vendors who, for example, like, might have, maybe you have, you know, commercial. Well, it can acquire data from commercial vendors who might have private sensitive information about Americans or might have data sets from which private facts can be inferred using analysis. And it can also conduct that analysis. The conduct, in other words, doing the analysis on the data is different from actually directly collecting it yourself as the government. And doing the analysis doesn't count as surveillance for purposes of, you know, the relevant national security laws here. So this is something that's existed for a while. Privacy advocates, civil rights, you know, civil libertarians have. Have talked about these issues for. For a long time. How does AI change this dynamic? Well, frontier AI means that all of a sudden I don't need some expert. Yeah, I don't need to pay some data scientist or some intelligence analyst to go track, you know, I want to go track. The government wants to track my movements, let's say. Well, in the past, it might have been, like, awfully expensive to do that because, you know, why would you. To pay. To pay a human to specifically track the movements of, you know, me and my family or something would be quite expensive relative to the intelligence value of my life. But all of a sudden, if you get to a world where the government has instead of having a few thousand intelligence analysts, it has, you know, a few million or tens of millions because they're autonomous agents. Well then all of a sudden the cost and, and whose, you know, whose marginal cost is effectively zero. All of a sudden, within existing law, nothing changed about existing law. Nothing really even particularly changed about existing intelligence practices. All that changed is that AI made essentially the value of expert attention much cheaper than it used to be. And that in and of itself enables quite significant, the potential for quite significant surveillance.

B (12:55)

And one way of putting this is that the laws haven't changed, but the laws were put in place for a reason, which is to make it impossible for the government to survey innocent Americans at scale unless there's, you know, probable cause of some very serious crime and there's some kind of judicial sign off, usually, and so on. And now suddenly the technology just means that the laws, even though they haven't changed, really are no longer effective to the purpose that they were trying to secure. And so that obviously is a very large concern. Now that gets, I think, to a larger set of things I've worried about for the last week. So in this particular fight, I think it's very easy to be sympathetic to Entropic. And you've written quite strikingly about your disquiet, and that's really probably a too softer word with how the administration has acted. You yourself served in the Trump administration, helped to formulate or really the lead in formulating the administration's AI policy. And you have said and written that you find the way in which we're treating antropic to be very counterproductive and irresponsible and dangerous to our republican institutions. But we also need to go beyond this current situation, the particular way in which this fight has shaped out, to the sort of broader question of who do we actually want to control those technologies. And my fear is that this is a dilemma that really doesn't have a very good answer, as we're seeing at the moment. You don't want the government to have control over technologies which would, for example, allow it to engage in this mass surveillance of ordinary citizens. But of course, contrariwise, you also don't want a bunch of private individuals to have control over governments that have huge national security implications that could, for example, hack into nuclear weapons programs potentially or in other ways are going to be necessary for us to compete with competitor nations and keep our country safe. And so, and another element of this, it seems to me, is that our analogies don't really work very well. Right. You know, The President of the United States has a nuclear football and he can wipe out a big part of the world at his whim within 20 minutes. That is a very, very scary and dangerous fact. But even though the President is able to lead to the destruction of much of the world in that way, nuclear weapons actually aren't terribly useful in engaging in political manipulation domestically, in mass surveillance, in, in manipulation of public opinion. These new AI tools are both potentially very dangerous in perhaps inspiring a war if they fire on the wrong target or misinterpret a rocket launch as an incoming nuclear weapon or other kinds of problems that could arise. But they're also really powerful in these domestic ways that these previous technologies were not. And so the sort of dilemma of AI control is really sharpened in this context. So you write what I think is the most interesting, one of the most interesting newsletters about sort of the intersection of artificial intelligence and public policy. One of the problems of trying to grapple with AI, whether that is from a perspective of political science, from a perspective of public policy, from a perspective of economics, is just how do you govern something? How do you think about how to influence the development of something, you know, the effects of which are still so radically uncertain? Both because it's still so hard to figure out, for example, whether AI is currently leading to huge job loss among entry level professionals, or whether actually it's a bust and it's not really having an impact on the economy and it's over. Harder to know. Ten years from now, is AI going to look roughly what it looks like today, or is it going to be some form of super intelligence that is just, you know, incredibly able to do things, you know, of its own accord, whether good or bad. You know, how before we get into some of these sort of questions about how to actually govern AI, like, like how do you think about, you know, trying to answer some of those questions under these conditions of radical uncertainty?

A (16:59)

One of. Well, one of the things I really like about AI is that it is such. I mean, it is maybe the most general general purpose technology that has ever been conceived. Definitely it's a general purpose technology. Maybe it is like truly the most general. I think it's competition is electricity. It has this very uncertain capabilities trajectory. Very hard to know exactly what the world with it, you know, a world with AI diffused in it. Well, it's very hard to know, you know, the, the quote, unquote, positive vision is very hard to articulate. And it presents so many. Such a wide combination, such a wide variety of things that implicate the public interest in public policy that you kind of end up going to like political theoretic first principles much more than you do. Like, when we're debating, like, the details of like, telecoms policy or like health care subsidies or something like, you know, Edmund Burke and Friedrich Hayek and, you know, philosophy philosophers and political theorists, like, their work doesn't come up in the same way that it does with AI, because AI is like it. It is almost in some sense like a literary device for thinking about the future in addition to being a very real technology. And what should the disposition of government overall be? So my view on this is that, first of all, there are like high level incentives, like, very, very deep incentives in our society. One of them was of market system and the price system, you know, like. And that structures the incentives of everyone in various ways. Another that's really important, though, on the sort of more, you know, protective side, we might say is liability. You know, we have this system of common law liability that has existed for centuries and that allows for people harmed by someone else by the actions of someone else to seek redress in court decided by a jury of their peers if they so choose. And those two things in and of themselves are like, they structure the incentives of every actor in our society in ways that are like, quite deep and quite, like, quite useful. And I like to sort of look to those things first and think about, okay, what are those things going to get me? What are they not going to get me? The price system maybe gets you. And like, capitalism gets you, like, aggressive development of the technology. The liability system gets you some harm reduction. And, you know, it causes people to. It causes big companies to incorporate, you know, oh, well, if this hurts someone, we'll get sued. And so we have to, you know, build in safeguards or whatever. You get some of that. And then it's like, all right, what are the. What are. We have those two big things. What. What are the gaps that come in, in between? And there definitely are going to be gaps, but how do we identify what those gaps will be? Well, we can try to sit around a table in Washington or in Brussels or for that matter, in Beijing and speculate about what the harms will be that we need laws to address. And we'll do that with varying degrees of success. My personal disposition is that most of the time we will not be very successful at that. Or what we can do is we can basically have fundamentally reactive public policy that basically looks at harms as they emerge and decides ways to deal with them. You know after they've happened, and passes narrowly tailored laws to avert specific kinds of harms that for whatever reason are not adequately addressed by existing law or the liability or the liability system. So that's like my very broad impression, like what I always say is like AI is already regulated in many ways, so let's like rely on the huge base of existing laws that we have before passing a bunch of new stuff that's speculative because we're usually wrong in our speculations. Even the smartest of us, maybe especially the smartest of us, have this tendency to be like quite wrong about things. Like the EU Air act doesn't talk about a lot of the things that people are actually worried about with AI today, like in the American electorate. None of the regulations from a year or two ago actually address those things like at all. So yeah, it's all, I think it's all like very, I'm basically articulating here some combination of like a Burkean view and a classical liberal view. So like a traditional conservative view, but also a classical liberal view about being like very open to change, which maybe Burke is not. So that's basically how I think about it.

B (21:37)

So there's a lot of things that are interesting in there and I want to drill down to some of them in a moment. But let's stay for a moment at this philosophical level because there's something sort of surprising about invoking, whether it is the classical liberal thinkers or perhaps particularly something like Edmund Burke in the discussion of this 21st century cutting edge technology that is going to transform the world. And you've written in a number of ways about how conservative thinkers, whether it's Burke or Oakeshott and others, can sort of help give us the right kind of orientation towards this. I think the point that you've just just made that is obviously right is that your general conception of politics and of the good governance of society is going to give you instincts about how to react to artificial intelligence. And so there's going to be some people in a more European status tradition who think there's a new thing happening. It has lots of potential benefits, perhaps also lots of potential risks. Let's just pass a bunch of regulations that really go heavy on sort of, you know, reigning in the risks. Right. You know, the role of the state is protected citizens from, from harm. It's to govern the economy and to make sure that there's a kind of primacy of our collective will over the private economy. So, so let's really lean towards trying to figure out the ways that we're going to minimize the potential harm and, and, and, and sort of become master of this. Right. Sort of very different kind of set of attitudes. But I think, you know, listeners to this podcast will also instinctively understand whether they agree with it or not, was sort of more libertarian. People should have the freedom to do whatever they want unless or until there's very clear, you know, proof of harm. And so until we actually see AI doing really terrible things, we should really be on the light side of regulation towards it. It seems to me that the conservative thinkers, you know, have more of an attitude towards continuity and change, where they think that there's lots of elements of our society that work for reasons we don't fully understand. We want to make sure that we don't impose a huge rational design on the world in a way that might stop those things from working. And that includes those forms of regulation that something like the EU might be tempted to pass. But at the same time, we also do want to ensure that change doesn't engulf everything that, you know, we don't perhaps have this libertarian instinct that say, let's just see what happens if there is a potential risk of AI just completely transforming society in such a rapid way where we can't keep track of that. But perhaps there's a reason to sort of slow that down a little bit. So tell us about, you know, when you invoke Burke, you invoke Oakeshot. What kind of sensibility do you think that should give us about how to respond to this, you know, potentially revolutionary technology that is spreading at, you know, incredible speeds throughout society and has a potential of, you know, leading to, you know, a degree of change and upheaval not experienced since, you know, the, the first large scale factories in Manchester in the, you know, early 19th century and perhaps not even then.

A (24:42)

No, I mean, I think this is the, this is the fundamental tension and it's a tension that exists in my own writing. Right. Because it's its attention in how. I think there's one part of me that is intrinsically skeptical of change. I, I like the, you know, I, I like things about my life the way they are. I have a what, what Michael Oakeshott would have called a disposition to be conservative, particularly with respect to, though, what government does. And the reason I am about what government does is because ultimately, like conservatism is this kind of beautiful and ultimately tragic philosophy which, and I say tragic in the sense that conservatism is about conservatism done right, I think is not about resisting change. It is about a disposition toward change and a. And a way of reconciling yourself to the dynamism of. Of reality. And then, of course, there is the, you know, there is a more libertarian or classical liberal impulse which says, like, that dynamism is often good, which is also true, though it's not always true. So, like, one thing that I think that I ultimately come down on is what's worse than create. Like, creative destruction has its problems, right? I mean, the word is destruction. It's not fun to be destructed, even if it's creatively. But, like, stagnation is worse. Stagnation is much closer to death. And at least from creative destruction, new things can be born. And birth is the thing that we should. We should want things to be born and we should want growth. Those things are healthier and better from all dispositions. They're, like, morally better, I would say. And stagnation is the worst possible thing and is an inevitable outcome of stagnation is a. Is gradual death. So what regulation does is it can freeze things in place, it can create red tape. But inside of every law that is written about AI is a complex of implicit and explicit assumptions about what AI is, what kind of a thing it is, what kind of a role it will play in our society, and, like, what is even technologically possible, right? There are just all these assumptions, unstated and stated, that I think AI will itself combust. And so what concerns me very much is that we will pass all these laws that will freeze in place a status quo with which I think, if we're honest with ourselves, I don't know that many people who are satisfied with the current status quo in Western civilization, right? Like, I think everyone is of the opinion that something's got to change, and we have different intuitions about what. But, you know, I think if I think about this, to use a specific example, you know, five years ago in 2020, during the pandemic, a lot of Americans and especially a lot of conservatives got kind of, like, outraged by, you know, when their kids came home and they were on Zoom class and they heard what the schools were teaching their kids, they got kind of outraged and they said, we urgently need reform in the education system. Oh, my gosh, this is horrible. And then a couple years later, a technology comes around that has, like, obvious, profound implications for the institution of the schoolhouse and the institution of education. And all of a sudden it's. And. But also, I think, creates, like, huge new opportunities in education. And all of a sudden, the mentality from, again, especially conservatives, but a lot of people other than beyond conservatives is like this. This technology is such an imposition. Why do we have to accommodate all this change just for this technology? And it's like, well, I thought change was what you wanted. So there's this tension all the time in, I think, a lot of popular discourse about AI, where all of a sudden AI puts us in this posture of defending these institutions, which I think, again, if we're all being honest with ourselves, we're showing their age for most of my lifetime, and I'm 33.

B (28:45)

Well, I mean, there's a broader issue that I think is confusing a lot of politics at the moment, which is that it probably never was true that progressives wanted history to speed up and conservatives wanted to stand up top, you know, history, and yell stop. But there was something to that. You know, I think today in many respects, the roles have reversed. You know, a big part of a Republican coalition today is sort of accelerationist and wants to go all out on innovation and change. And I don't know if that's the dominant faction, but it's a significant faction. And a significant faction of the left today is people who just instinctively are against any form of change. You know, I'm really struck by the fact that one of the most viral essays about AI on the left was this piece by Jack Tarantino in the New Yorker, which I mentioned in a piece I wrote about this. You know, to my ear, she just sounds like a kind of like 19th century priest denouncing the evils of trains before going on to say, you know, I've never used AI, you know, trains are evil, but of course I would never ride on one. And, you know, and I think, you know, again, I don't know if that's the dominant faction of the left, but there is this kind of, you know, really deeply entrenched, smallly conservative instinct on the left of, you know, if it's changed, then it must be bad and let's somehow stop it. Even as the left often criticizes the status quo as being very negative, perhaps let's make this a little bit more concrete because I think that I still struggle to get my head around different kind of options for how to regulate things and what the actual proposals are on the table. And you really know this in quite a lot of detail. So it sounds like you're quite critical of what the European Union laws and regulations about this are going to be. My understandings that some of these laws have been suspended for at least a year, but they're going to be implemented at some point, at least supposedly. You know, what assumptions about AI and about the right way to governing it do you think are implicit in the set of rules that the EU has passed? What actual rules are at stake here and why do you think that they are a mistake?

A (30:51)

So it's a great, it's a fantastic question. The AI act has kind of two. You can conceptually divide it into two halves. One is regulations put upon developers of AI systems and another is regulations that are placed principally upon deployers of AI systems in quote unquote high risk contexts, which would be like governments and bank financial institutions and healthcare and you know, things education, things like this. Some of the most vital sectors in our society may be some of the ones most in need of some institutional dynamism. The AI act perfectly exemplifies the assumptions thing because most of the text of the AI act was written in the early 2000 and twenties, prior to ChatGPT. And so the assumption of what AI was at that time was that the AI systems that predominated at that time were principally basically narrow machine learning based systems that would like for example, computer vision used for facial recognition. But that's all it can do is facial recognition. Or you know, I'm going to have a computer, I'm going to have a camera on the bottom of my tractor that's going to look at images of crops for defects in the crops. But that's all it can do, can't do anything else. And so I think, and maybe even most famously, they're like algorithms to like you might take a loan application and process them through a machine learning system, basically a statistical model based upon the previous loan data that that financial institution had and then make a prediction about whether this person's going to pay back their loan based on the historical data and you know, kind of, you know, then decide whether or not to issue the loan based on that, often with human review. So the interest was in regulating systems like this. And there's a couple of things about that. First of all, it's like mostly regulated institutions that are doing that kind of thing. Like I myself in that world where that's what AI is, I as a consumer, I'm not gonna like use AI. AI will actually be used on me is the model. And it's an already regulated institution that is like, presumably like working with contractors assembling data sets like that sounds like it probably costs like a lot of money and takes time. So maybe the marginal increase in paperwork is not Actually that significant compared to what they were already going to do anyway and cost, et cetera. I think that's debatable, but you could at least have that intuition. And then finally, like, maybe also it's the case that like that bank's historical loan data has real biases in it. Like maybe there was a period in the US context, maybe there was a period where it was like de facto or de jure illegal for certain kinds of demographic groups to even get a loan. And so that might bias the data. And if you just purely take the historical data and put it into a machine learning system, that machine learning system may well be unjustly biased against certain demographic groups. I think that like there is a fair argument to be made. Obviously you can also get into like disparate impact arguments where like, it kind of turns into like race Bolshevism where like, you know, everyone has to be treated equally about everything. And that's like also a problem. But I think there's a nut of like legitimate critique there. And then of course, like right at the end of the AI act, like final months, final innings of the, the AI Act's legislative process, comes the generalist language models. ChatGPT, Claude, Gemini, etc. These are totally different, right? Like, is bias an issue in these things? Yes, but like a, anyone can use them. They're adoptable extremely quickly by both enterprises and consumers. And there's this whole issue of like, the like. One of the fundamental constructs in the AI act is this notion of the consequential decision. When you make a consequential decision that is like a highly regulated moment, but with a language model which is like structuring your information environment in all these subtle ways and like doing research for you and writing software, you're writing code and it's generally, it's a general intelligence, right?

B (35:16)

The idea of a consequential decision is sort of like you're an insurer and you're deciding whether or not to accept this person for insurance. And that moment is the consequential decision. And if you want to outsource this so that human beings don't reveal the files, you have an automated system system that you have programmed especially for this, then it's very clear what that automated, you know what that consequential decision is, right? If in fact you now incorporate ChatGPT or whichever AI model into all kinds of business processes, where does the consequential decision start and stop? And if this regulation suddenly applies to all of those business practices, Then you basically just make it impossible to use AI.

A (35:57)

Yeah, like, to a first approximation, the insurance company has one use of that narrow system, which is to review the application and provide a recommended, recommended decision. In the case of a language model, an insurance company might have 10,000 uses of the same neural network. So this is exactly, you know, what I'm referring to where, like, there's just this complex of assumptions that just ended up being wrong. And now we kind of have to, like, live with that. You know, we have to live with the consequences of that. So that's a, that's a great example of what I worry about. And we've seen, we've witnessed that play out in, well, under, like, in like half a decade. Right. Took, took three years, really took three years for us to go from the framework that the Europeans had to like the generalist models, completely obviating it. I bet you a lot of the laws we think of today will be obviated by something that comes down the road for which we don't yet have a word.

B (36:44)

Now, when you say that we have to live with those consequences, I have a question, so somewhat narrowly about the European Union, but I've been in a number of debates about this and Europe and would love your view on this. Europe really now believes in the idea of a Brussels effect, that because Europe is a very significant market, when they pass certain rules and regulations about, for example, the fuel efficiency of cars, that ends up constraining even what Ford does in the United States because they can't afford not to produce for the European market. And it isn't really worth it producing two completely different sets of cars for the US and Europe. And so therefore, a rule passed in Brussels can in fact, you know, transform the car industry even outside of Europe. And they have now applied this idea to AI and really think that, you know, Brussels is able to steer the development of these frontier AI models which, you know, to a close approximation, do not happen in Europe at all. They happen mostly in Silicon Valley and to some extent in China and perhaps a few other places. But there's really not many significant AI players at all in Europe at the moment. But because ChatGPT is going to want to sell its products in the EU, perhaps it somehow really constrains OpenAI and how it produces those Frontier AI models. Now, I have a few things to say about this. Somebody who's both a US and a German passport. You know, the first is that it just seems to me like an incredibly low level of ambition for Europe to say, you know, Fine. We are not going to be players in the development of the most consequential technology of 21st century, but we're going to be able to regulate it. I mean, it just seems like a somewhat sad ambition. The second question is, you know, when I think about what the real risks of AI are, some terrorists being able to design a biological weapon that kills, you know, tens, millions or tens or hundreds of million people of people, or perhaps billions of people. You know, complete change in military technology where autonomous drones just transform modern warfare and perhaps change the, you know, trade off between offense and defense in such a way, the balance between offense and defense such a way that it just becomes much more enticing to start wars, you know, up to, you know, just a huge percentage of the middle class losing their jobs. Because AI just becomes better able than most human beings at doing white collar jobs and perhaps eventually blue collar jobs. Once we've developed more skilled, embodied AI, up to the existential risk of perhaps some AI model that's misaligned becomes so super intelligent that it just, you know, runs over humanity. It's just none of these things are going to stop at the border of Europe, right? It just seems absolutely naive to me to think that any of those four developments will somehow not happen in Europe because at the European border, suddenly the European AI act comes into play. On the other hand, it sounded, from what you were saying, we now have to live with the consequences of European Union AI act from an American perspective. So how relevant or irrelevant are those EU rules to the CEOs you talk to in Silicon Valley? Do you. Are they a little bit of a headache that might make it harder to monetize some products in the short run, or does the EU actually have some power to influence the trajectory of that technology?

A (40:06)

Well, I think we're at an interesting turning point right now because I think the Europeans have really backed down from the posture that they had toward regulation even just two or three years ago. So, in fact, as I speak to you right now, I am at the office of the Delegation of German Industry and Commerce in dc. I was speaking at an event with a bunch of sort of US German business type interests. And I think the posture really has changed and the Europeans have realized that they need to soften some things. In particular with respect to the AI act. There are rules on AI development. Most of those rules are actually like they, they, they gave themselves enough leeway in the text of the AI act that the implementation has been considerably softer than I might have predicted two or three years ago. When the law was first drafted. That being said, like, I think it is a real concern for industry because we don't know where those rules are going to develop. And yeah, it's also like, I think it's a concern for, for both Europe and America that most of the rules I mentioned were the rules about deploy on deployers. Those are regulations. They're regulations of any company with operations in Europe. So any European country, but also any, you know, multinational company. And that mostly is bad for their economy. Like, you know, if there's some multinational American firm that has a big research presence in Europe but also a big research presence in the United States. And like, that's also being coupled with like, well, there's all those tariffs and there's this build and build stuff here and we want to like, you know, want to show the Trump administration that we're building in America. There's all those incentives. And then it's also like, oh, the Europeans also have these laws that like make a lot of my fundamental work harder. My adoption of AI is much harder in Europe. Like maybe on the margin you start to start relocating facilities and relocating people. And that's like, that's in the short term that's great for America, but in the long term it's not so great for either Europe or America because I think America does fundamentally depend upon like a strong Europe. I mean, we don't depend upon it, but I think we will be weaker without a strong Europe, let me put it that way. So, yeah, I mean, but also to your point about the catastrophic risk, that's one thing I didn't mention when it comes to my disposition toward how we govern AI. But like, in some sense government is just, it's, you know, it's, it is this big, you know, risk management type of enterprise and like tail risks, catastrophic tail events, unlikely events with very, very high cost are the kind of things that, the sort of market based systems, the two incentives that I talked about were market incentives and liability. And liability in many ways is inspired by modern liability, is inspired by market based systems. In some ways imperfectly so, but nonetheless inspired market based systems do tend to be subpar in the management of catastrophic tail events in particular. So from a technocratic perspective, this is the area where proactive regulation of certain kinds, or at least proactive steps taken by government can make a lot of sense. And so this is why, you know, I have been supportive of things like SB53 in California, which are light touch, you know, I think even, even if, even if there's Room for government involvement doesn't mean the government should be unrestrained. It's a light touch bill that requires frontier AI companies only to disclose how they measure and evaluate and mitigate these types of catastrophic risks posed by their models, cyber autonomy, bio, things like this. And you know, I am supportive of that. If there's a federal standard, I hope that that's something like that at least is in the federal standard. It's worth noting that the EU code of practice, it's a lot of things, but one of the things that's in the code of practice is stuff pertinent to this too. And so this seems like a good thing to do. And it's one of many things you should do, most of which are non regulatory in nature, most of which are more like proactive, defensive and resiliency building. But in fact, I think what the transparency regulation does in this case is it allows us to all get a glimpse of what are the catastrophic potential risks that are emerging at the frontier and what kind of resources and steps should we be taking in areas far afield of AI, often to mitigate those risks. So I think that's actually quite prudent and is potentially an area of US and EU collaboration. We'll see.

B (45:14)

Well, it's promising that the UN US might still be able to collaborate on this or anything else. Let me push you a little bit on how we should think about regulating AI in a better way. So you're saying that the problem with a lot of the approach to regulation is, you know, we see what the technology is right now. Let's make a set of rigid rules around this technology in order to minimize risks and reassert our control over it. But that's really going to start being unhelpful very quickly if two or three years from now the nature of technology has changed so much that those rules end up being far too specific to a particular state of a technology and they either end up being overly constraining or underly constraining, or just mis tailored for what the state of the world is a few years later. And since it takes a lot of effort and time and so on to change and reform legislation, you're then stuck with a regulatory regime that really doesn't make sense. Now, you know, I have two questions. One is, so what's the alternative? I mean, what kind of spirit or approach to regulating AI makes more sense? You know, just inherently trying to regulate how AI works when we don't know what this technology is going to look like in free, it's Going to be really hard. So what's the alternative? And you know, in keeping with that, when we think about some of those bigger risks, is any alternative going to work to actually constrain that? Right. Unless we're saying, hey, you know, a certain kind of form of capacity we're not allowed to develop or needs to be developed, you know, and a very, very strong security regulations, etc, how can we avoid something like, you know, a super intelligent AI we can't control is, is a kind of set of dispositions ever going to be enough to, to facilitate that?

A (46:59)

Yeah, so I think you, you need to have a system that first of all, like, like I said, I think step one is probably transparency because like having information about these, like, I don't know, a lot of things about the world are going to change in, in 10 years. But like one of the ones that I think probably won't change is that like we probably won't want there to be like, you know, AI engineered pandemics. Like we'll probably still want to avoid those in 10 years even if many other things are quite different. So you know, Jeff Bezos always says like the best way to predict the future is to think about what won't change. And that's also the best way to write a law, try to like write for the evergreen thing. So that, that's one another error.

B (47:48)

Let's dig down on this. Right, so yes, absolutely. I assume that in 10 years and 100 years and a thousand years when not going to want to put technologies in the hands of everybody that make it very easy to cause a pandemic. But then the obvious follow up question is, okay, so what do we have to do in order to make sure that the ability of AI to teach people to do things in general and perhaps to give them knowledge about biology in specific and even more specifically to actually figure out new DNA sequences or other things that would allow someone, you know, with basic skills and laboratory technology to create a really potent bioweapon, you know, that it can't be used for that purpose. But that truly will depend so much on the state of the technology. That sort of knowing that we share the goal of let's avoid people being able to engineer these bioweapons is great, but you know, knowing, you know, what kind of regulation is necessary in order to carry out a task is much harder. Imagine that AI was so good at inventing really simple lab techniques and so good at inventing technologies that allow me to just sequence DNA and create a virus and so good at inventing incredibly deadly viruses that just anybody who has access to an all purpose chatbot powered by AI would be able to do that. You know, and it turns out that we're just not going to be very good with even basic alignment so that, you know, reliably programming these chatbots to stop people from being able to, you know, from, from giving that advice to people. Like in that kind of world, we would just have to make sure these ibots basically don't exist. Right? Because the moment they exist, somebody is going to be able to exploit them in such a way that, you know, hundreds of millions of people die. Now, I don't think that's likely to be the state of the technology, right. But the policy approach we should take, including whether it should be pretty permissive or whether it should be incredibly restrictive, surely depends in part on what the state of technology is. And so how can we pass regulation that is completely agnostic to predicting the future. Future in that way?

A (50:06)

Yeah. So it's not about, I mean, I, I would say it's not entirely agnostic because like, you know, something like SB53 does specifically talk about bio risk. Right? So it's like this is a risk that the state is now acknowledging and we're saying you have to do. There are rules that say you, if you are developing a model of this size, you have to go, you have to measure it and you have to like, tell us how you're mitigating it. And of course, you know that in and of itself, how a company does that, given that it's now public, that is going to, and how a company does that, by the way, compared to how other companies do it, that will factor in if there's ever liability, if there's ever a, you know, legal situation involving something like that, how well they mitigated those risks and what they did and how they measured and all that stuff, the thoroughness with which they did so they will be compared to their peers. And that is how we will assess a duty of care. The common law can allow that to be dynamic. It's also very possible that the things you need to do to mitigate this kind of a risk, just purely at the AI model developer layer, that they'll change. In fact, I think it's likely that they'll change quite a bit over time. So you want the companies to be incentivized to do a good job in mitigating these things while also not telling them, not being super prescriptive because your prescriptions might well become outdated soon. So like, for Example, let's say that there are, you know, there's all these safeguards that you can put on the models to prevent them from being jailbroken, to monitor usage. You can have like, oh, there's a specific kind of set of features of this model that has been activated and that immediately triggers human review to decide whether, okay, well, is this a virologist at a university who is doing legitimate scientific work or is this a potential bioterrorist. Right. Things like that that you would want to do. And you also might, you could see a world where you have technical standards for exactly how reliable those safeguards need to be. And so that's the kind of thing that, you know, or various organizations can be involved in making those private organizations, government organizations, the center for AI Standards and Innovation can do that kind of work. Plausibly, it can be some hybrid of all those things. Maybe eventually the risks get severe enough that we want there to be something like supervision or auditing that go on so that it's not just the AI companies sort of grading their own homework, but instead there's some either government or blessed by government process, private actor who is independent and who is sort of making sure, sort of testing, in particular testing their safeguards against some kind of pre existing spec that is not created by the model. You could see if the risks develop in a. I'm not confident we need to go in that direction, but that's one direction you could go if the downside risks become, seem like they're going to become much higher. And then the other thing though that I think is important to say is that there's a lot of other things that have to happen downstream for this to go well. One example would be something I worked on when I was in the Trump administration was something called the nucleic acid Synthesis screening framework on biosecurity. Where this is, this is a regulation, is a mandatory regulation on the companies that provide nucleic acid synthesis screening services. So if I want to make a virus, I might have its genome and I might send its genome to a company. This is not actually how it works because the technology is not quite there yet. But bear with me, I send a protein, you know, the sequence for a protein or of a genome to a company and then they will synthesize it for me. You can impose regulations that require those companies that do the synthesis to screen the sequences for pathogenicity and toxicity. We actually already have a early version of these regulations in place. You can have them do kyc so they can make sure, oh, hey, is This a. A lab? Is this a person in a lab doing legitimate work, just like the AI company.

B (54:12)

So KYC is know your customer.

A (54:14)

Know your customer. Sorry. The AI company has these safeguards. Then the nucleic acid synthesis company also has these safeguards. And then at the same time, you might also have biosurveillance programs. So we're constantly measuring the wastewater and other things in the environment for novel threats that are emerging. And maybe in the very long term, not the very long term in AI, very long term is like anything more than five years. Like, if the AI models are that good at synthesizing, you know, path pathogens the way you described, then they're probably also that good at synthesizing treatments for those things. And so, you know, there's a. Maybe there's a future where we all have little devices, you know, on our bodies that like, automatically, like they're connected to the Internet and a new pathogen emerges and like every person with that device just automatically gets injected with a cure to it that is developed by AI and sent down to the device automatically. Right. That's the kind of thing that you can imagine in the future.

B (55:12)

And given how universal the acceptance of MRNA vaccines was, I'm sure nobody would object to a machine like that. That's beside the point. I want to ask you about one element of this because you've touched on liability a little bit, and I think liability is actually something really important that we don't talk that much about in the broad now, you know, I can see how a liability framework can often be more useful than just straight up regulation, but I think it depends on a few boundary constraints. Right. So when you have a car company, you know, the risk here is that some design flaw in a car they produced is going to kill some number of people. There's not going to be millions of people. People. Right. But it is what we consider an unacceptable number of people who die as a direct result of this kind of design flaw. And when that happens, we have liability rules in place which mean that those customers are going to be able to sue the car companies. And most likely this should create the economic incentives such that together with bad publicity and other things, the incentives make it the case, such that the car company would, you know, rather spend more money on research and development and safety tests and so on than to have to pay out those financial fines than to take that kind of publicity hit. Right. But part of this is that there's no catastrophic risk involved here. Right? There's obviously catastrophic to individual customers, but not at Scale. And so that makes it kind of work. Right. The problem, I think, comes when, you know, the likelihood of a bad impact for any one company is very low. Right. It's very unlikely that any one particular AI lab is going to create, you know, the frontier model that causes the big global pandemic of 2030. You know, that it's going to create this super intelligent AI that kills all of humanity. So they might think, you know, each individual company might think the risk of us doing this is relatively low, well below 1%. We are not going to worry about this too much because most likely it's not going to impact the long term future of our company. But if it happens, then sure, perhaps liability somehow means that this company then goes bankrupt. But the pandemic has already happened. The superintelligent machine has already killed all of humans. So the fact that this one company ends up going bankrupt really doesn't matter in that grand scheme of things. Now, of course, it may be that if you have 300 companies building AI models at some point, the risk of any of them bringing about this really bad outcome is so low that for each one company, this liability concern is not front of mind. But cumulatively, the risk from those 300 companies is actually very high or at least significant. And in that kind of world, you would think that, you know, the liability framework really doesn't protect us in the kind of ways we need. So is the liability framework just not well suited to this kind of problem or do we need to complement it with other kinds of things?

A (58:23)

Yeah, it is. In fact, I alluded to earlier the notion that liability is not good at catastrophic tail risk. Traditionally that is an area where liability fails. And so that is for that reason. That is why I specifically concentrate my efforts to develop public policy on those things, because that is what I trust that liability like I have less trust in liability for that specific area. Couple things I would say. First, I want to complicate it slightly and just say that the liability system does incentivize companies to have like a wide variety of different safeguards and like jailbreak protection methods for things that are like normal consumer harms that would be dealt with by the liability system. Well, and that work probably is general purpose work that has benefit for the catastrophic risk side. But it is generally true that this is why you need the public policy like to step in here. And so that's why I'm supportive of things like SB53 in California. It's why I'm supportive of work downstream for biosecurity Policy in the biotech industry, mostly those are regulations.

B (59:44)

And tell listeners who are not as up on this field as you are what's in something. SP 53, the law in California.

A (59:53)

Yeah. So SB 53 is a law passed by the California government that was signed by Governor Newsom in September of this year. So it's brand new law. Basically what it says is that if you are a very large AI model developer, you know, basically the top five or six companies is the way it's designed. You have to publish your, what is called the safety and security framework. Every major company already does do this. They go by different names. Anthropics is called Responsible Scaling Policy. You know, they all have, they all have different names but actually they already all for the most part do this. So this is really just like codifying an existing practice of industry. And that document has to contain, must contain details about how you measure certain areas of catastrophic risk, including bio, how you like what the results of those measurements were and what you do to mitigate against those risks. So that would be the idea and also the notion that just in general like as we move up in different levels of capability as we go from the models really aren't that much more useful than a Google search would be in terms of biorisk to well now the models are like if you know the right questions to ask. So if you're an expert you can kind of get the models to be useful to you to like the models can help a complete novice design a bio weapon. And like we're probably somewhere between the first and the second right now in terms of like they're providing uplift to experts, but not necessarily like it's, it's not like I could go design a bioweapon right now. It's not clear when, when or if we'll get to that point. But like as we go up these different scales of, of of capability then you know, because the thing is is these capabilities are also really useful because you can design the same intelligence allows you to design all sorts of novel treatments and drugs. And so like it's like hugely useful and you don't just want to say you can't do it. It also probably doesn't work like to say like a model may not pose biorisk is like not going to be an effective law for any number of reasons. So but the notion, you know, is contemplated in SB53 that like as you go up to different like qualitative levels of capability, then different qualitative levels of safeguard are also required which seems reasonable to me. So I think you do have to take this stuff quite seriously. My view of, of the, of these issues is that they are quite tractable issues on which we can make a lot of progress. It's not like they're not threats at all. But also I think that sometimes, particularly early on in the AI safety, like sort of early after ChatGPT, there were AI safety advocates who I think exaggerated how overwhelming these problems really are. My view is like, especially having sat in government, like when you sit in government, a new catastrophic tail risk comes across your day, your desk, every single day. And so, you know, you have to. There's no 100% solutions to these things. There are compounding 95% solutions that kind of like as I described, with the mitigations that you might do on both biosecurity and the AI model, safeguards and wastewater treatment, biosurveillance and developing cures more rapidly, doing all those things together, that's the way we deal with these sorts of issues. And I think they're tractable problems. Do we have a 100% solution? No, but it was never realistic to expect one. So I think there's tractable problems. But also that doesn't mean, oh, we should relax and not do anything. It's like, no, there's quite a bit of work to be done and a lot of it needs to be done with urgency.

B (63:56)

So I feel like you've given us a little bit of a sense of how you do want to think about regulating AI, but perhaps you can now pull those threads together a little bit for listeners. I mean, if you had to answer in kind of 60 seconds, what forms of regulation can help to mitigate some of those very serious risks without slowing down innovation in AI that could really have very positive impacts for the world? Very briefly, give us a list of some of the things we should do and some of the things we shouldn't do.

A (64:32)

I do think that transparency and facilitating insight about these problems is step number one. And we're, we're kind of in the process taking that step right now. Another step is to think about, you know, for threat models that we believe are real, it's to kind of examine the downstream, like, okay, what would we need to do? What would the, the way I would put this is like, what's the victory condition here if we're worried about autonomous cyber attacks? Well, we know that autonomous cyber defense is also possible. So what does the world look like? What is the set of policies and institutions and technologies that we need in Order for us to feel like actually not only are we, you know, we're not just treading water, not only are we mitigating this risk, but we're actually going to get better at cyber defense than we were before. Bio. Very similar thing. We know that biosecurity is a problem. We just lived through a multi year pandemic. So like you think about what the victory condition looks like essentially and then you, you know, build all the, in parallel you have all these different societal actors. You're channeling philanthropic resources and government resources and corporate resources all toward trying to build that institution. And I think, you know, what I view as my role is like A helping to, to channel those resources and B, like trying to articulate like what, what does that institutional arrangement look like? Like what, what is the win condition here? Right? Or at least trying to amplify the voices of those who are already, who have already articulated such things. You do that and very frequently. The last thing I will say, and the reason that regulation is, is very much a double edged sword, is that all of the things I've mentioned, bio, cyber and I think even the challenge of AI governance itself, AI is a general purpose technology. It is useful for the defensive side and the resilience side of all of those things. We will not govern AI without AI. That's a weird fact, but it's like also kind of trivially obvious if you think about other general purpose technologies. Imagine trying to govern computers without computers. You know, governing computers with pen and paper doesn't seem like it would work very well. Similar kind of thing. So if you regulate AI too much, not only do you stifle capabilities, growth and innovation, but you also stifle often the ability of people in the kinds of like critical infrastructure. Say we regulate critical infrastructure a lot. But if we regulate AI use in critical infrastructure because we're worried about AI risks in critical infrastructure, then we also might regulate by accident the ability of critical infrastructure owners to adopt AI in defensive ways that we are want them to. So that is the like, that is the challenge, that is the balance you have to strike. And like I don't, unfortunately, you know, I don't have a way of. I'm not a good politician and I don't have like a way that I could summarize all of this in like one sentence or two. But that's basically how I think about it.

B (67:32)

This starts to get a little bit into another point you've made that I find interesting, which is about how AI is going to transform our institutions. So Far I think this conversation has been quite focused on what regulation is helpful or harmful, what kind of framework to use for regulation. There is a kind of more fundamental question where our institutions were built in the 18th century and already there's all kinds of technologies, including just the Internet, that I think lead to a little bit of a felt mismatch between our democratic institutions and how people are now using, used to having an impact on the world. Some of the basic assumptions that we can't all come together and decide to govern and deliberate together are not really true with web 1.0 or 2.0 technologies. Now there's other constraints. I think one of the reasons why actually you don't want direct democracy is that most people just aren't that interested in politics. And that would end up with the people most obsessed with politics, who are often the most ideological, having the biggest voices, all kinds of other problems. But there is a kind of fundamental question already about if we really are serious about the project of self government, why are we self governing ourselves with 18th century technology rather than using late 20th century, early 21st century technology? You could imagine ways in which artificial intelligence poses the question of how we should radically transform our institutions in quite fundamental ways as well. Well, and part of that may be about the institutions of self government, right? Like how is it that we can use AI to actually translate popular views into public policies, which I think is kind of one of the core points of a democracy. But also just, you know, how can we just radically change our idea of what the DMV look like? And beyond government, how is it that universities are going to look different, that research labs are going to look different, that corporations are going to, you know, produce goods in a different way? There's obviously no firm answers to any of those questions. But how should we think about keeping what's valuable in our institutions not becoming a weird kind of AI utopian who thinks all these institutions no longer have value because they're somehow going to be replaced by Gemini 3.0 or something, and yet remaining open to intelligent and smart ways to rethink our institutions such that they can seize upon these things. Technologies.

A (69:49)

Yeah, no, I mean, this is in some sense this to me is the fundamental question. Like, so I wrote a piece a couple of months ago, it was called the Building Company. And it was a, a speculative sort of short story about a company that has developed the capability to robotically construct buildings like warehouses and stuff. And you can think about in that world, world where that technology exists and it's widely diffused you can imagine that like, you know, there might be like something like building, you know, there's like building codes in, in local government that like, you know, are enforced by departments of buildings and they send inspectors out periodically to go like, look at sites. And it's often a very cumbersome process. It can be very expensive. Well, like when you have end to end autonomy of like the construction of a building, well, you can kind of just have the robots, like they have all this telemetry data that they're collecting, all this visual data, all this stuff about what they're doing, right? And that is just like perfectly encapsulated all information that can be streamed basically in real time to a regulator who can be monitoring constantly with their own eyes for potential, you know, risks, bad practices, whatever else, violations of the code. You can imagine that that would like, change how banks and insurers would think about like, you know, writing policies for the, for buildings. You know, they would like they knew everything if they knew every step that was taken in construction. You can imagine like all sorts of changes that would come downstream of that. And you can imagine that like mostly being better now. You can also imagine that being that same idea being used to like constantly surveil everyone and be sending everything constantly to surveillance and you get into Skynet. So we have to have a set of principles by which we decide like, probably way, way, way more sophisticated than like data or private data. We probably need like more sophisticated abstractions for thinking about what are the moments when we actually want there to be like heavily AI enabled governance and what are the areas where we very much don't. And I think the answer will be like, there will often be like very binary decisions, but where we are like, no, we do not want AI enabled governance of that. And some where we were like, yes, we want to aggressively adopt that. So that would be one thing. I think, you know, you mentioned, you brought up democracy and you know, I mean, I think like the institution of democracy itself is the kind of thing where it's like, well, you know, the notion of going to a polling place and casting a vote on a specific day is like a very nice notion. It is also technologically contingent notion. And maybe there are better ways that are way more profound than like, I vote online now that we can use digital technologies to like bring the, you know, the perspective of the public to bear on the creation of public policy and make sure that the public has basically ultimately retains sovereignty over like the direction of, of things of public interest. Very, very possible. But like you know, you have to be extremely careful and things like that, of course. And I think like, you have to be cognizant. Like, I think like you, you have to have very firm principles. And this is why I, like, this is what one of the other areas where like AI does go back to being very political, theoretic and philosophical in nature, where like, you need a really, really robust set of principles to decide, like, when are we going to do this kind of thing and when are we not? And like, what are the things we do want to preserve? Because all of a sudden governments will be able to do like much more than they currently can. In some sense they'll be able to surveil and incorporate much more information than they currently can. And in some ways that will be great, and in some ways that will be very bad. And if we go into it with like mealy mouth principles, we'll probably come out with very subpar outcomes. So this is why I think that classical liberalism is so important and I hope that it has a serious resurgence not just in America, but across the world, because I think it does give us a very like, individual liberty, preserving, privacy respecting way of thinking about this stuff.

B (74:17)

Yeah, I mean, a few thoughts on this kind of reimagination of government. You know, I feel really torn on the subject because, you know, the first thought I have is, yes, of course we should try and figure out ways to do this. It is kind of strange to run our institutions, you know, along a model that was available in the 18th century and that, you know, sidesteps all of these technologies that we can use now. And it's not like we're doing very well at delivering on the basic promise of our political system. But one of the key problems of our political system is that it is government by the people, that we are translating popular views into public policies. And most citizens do not feel like that. And that's something that citizens on the left and the right and the center probably have in common. But most of them feel like those people in Washington are not really listening to me. So of course we should be open to this. Now the second point is that any actual set of ideas about how to do that, it's just very easy to pick the holes in. And it's because they usually do have holes. You know, I used to teach a class on democracy in the digital age at Harvard, you know, 10 years ago now. And even then there was these kind of utopian ideas about liquid democracy and different ways to have kind of smart setups to. And like they're kind of cool. Like, they're fun to talk about. They were a good teaching tool in that class, but, like, it was kind of obvious that none of them would work. Right. And then the third is that, as you're pointing out, one of the reasons why they don't to want work are things that actually have quite fundamental reasons that are going to apply at any stage of a technology. One beautiful text from a very different political tradition than yours, but I think is relevant, was written in the 1960s when there was a huge fashion for basis democracy, for participatory democracy in a kind of somewhat socialist vein by Michael Walzer in Dissent magazine. And he called it A Day in the Life of a Socialist Citizen. And the idea of the piece was, you know, if we manage to achieve socialism, but we do it in a form of basis participatory democracy, what would the day of the socialist citizen actually look like? And the kind of joke of the piece was that where Marx promised that people would be, you know, fishing in the morning and hunting in the afternoon and being critical to critics in the evening, you know, the day of a socialist citizen basis democracy would be, you know, sitting on the, you know, fishing license committee in the morning and debating laws about hunting in the afternoon and sitting on a literary prize committee in the evening. So actually it would completely alienate you from the. From. From the things that this utopian system is. Is supposed to deliver for you. Right. And, you know, the debate today is very different, has very different contours. But I think the basic question of, you know, yes, we have a technology to involve everybody in the design of the details of regulation about AI and we could have public comment in a much more fluid way. And we could have, if we want, every US citizen voting on this stuff.

A (77:11)

Right.

B (77:11)

And this is stuff that we could do before AI we could have done that 10 years ago. But most Americans don't have the interest, don't have the expertise, you know, in participating in this process. And we open it to everybody. The only people who like to would really participate are special interests, people with extreme ideological views, weirdos. It would not, in fact, make the system more representative of ordinary citizens. And so some of those constraints on why our political system isn't responsive to popular views in the ways we like it to look like they're technological, but actually ultimately are probably more rooted in basic facts about human psychology and politics that are unlikely to change.

A (77:50)

Yes, no, I mean, completely. And, and by the way, I myself any of those kinds, especially to things as fundamental as Democracy, any kind of institutional evolution in, in these fundamental things, like I am a, like my Burkian comes out and my inner Burkian comes out in a very strong way and I'm like quite skeptical and like, oh, I'd like to see a lot more, I'd like to see the technology diffuse and develop in like way, way, way more areas until we even began to think about things like that. One of the other things here though is that, you know, if you think I really have like a quite like biological conception of institutions and I often use the language of biology in analogical ways to like think about, about institutional evolution. And I think it is true that like institutions that are insufficiently supple will in the long run. I mean, we're probably already living through that right now. That is why I think probably the, the insufficient suppleness of Western governance institutions is probably a big part of why many citizens of Western societies do not feel as though their government represents their interests. And so, you know, what are the institutions that are more capable of evolving and that have been competently, you know, like that are sort of functioning well in, in our society today? Well, I would argue like, probably like big technology companies are a fairly good example of that of like, you know, what went right during COVID Well, we did produce MRNA vaccines that, that worked. That was controversial, but we did in fact do that. And we were all able to go to remote work like kind of overnight, massively increasing our usage of cloud computing. And that kind of just worked, you know, like it kind of just happened. And like we owe AWS and Microsoft and Google and other companies, like you know, basically we owe them thanks for like allowing us to keep our economy afloat during this very fraught time. So like, I guess what I'm saying is that the risk you run if of being like highly burkean with respect to everything that involves the government sort of creative, transformative adoption of AI in the institutions of governance is that they just simply don't evolve and the other, you know, other institutions do. And like there are things about current big tech companies that are already quasi governmental in nature. And in some sense this has been an old, this has been an old feature of America, right? The financial institutions, the banks have quasi governance aspects to them. There are many private organizations that have quasi governmental function. So it's not inherently bad that they will. But you can imagine a world if AI really pans out in some of the promises that it has. Like you can. One of the questions that I think about all the time is like, what kind of an institution is the frontier laboratory, right? Like the Frontier AI Lab, the OpenAI, the Anthropic, the Google deep minds of the world. What kind of a thing is this? And like, to what extent and in what ways will it be governmental in nature? And how will that challenge the existing structures of the nation state? Because, I mean, I guess, you know, to, to put an exclamation point on it. Like, I don't think the nation state is necessarily the end of history. Now. The evolution of something like that, even, you know, if AI moves very quickly, will still take decades, if not centuries. But nonetheless, I think we need to be prepared for, like, it might just be a structural reality that the tectonic plates here are moving beneath our feet in such a way that the nation state itself becomes challenged over time. I don't have high confidence in that, but I think it's certainly a plausible possibility.

B (81:46)

Thank you so much for listening to this episode of no Good Fight. In the rest of this conversation, Dean and I talk about existential risk. Is it realistic that we would be able to govern AI if it comes to have a capacity and a desire to destroy us? Spoiler alert. Dean is a little bit skeptical about that, as am I. And should we be more optimistic when some people like Eliezer Yudkowski and Nick Sorry's about the likelihood of that happening? Or might the latest model released by Entropic give us reason for optimism that we could one day have a wise AI to hear his answers to these questions? To support this podcast, to stop hitting this annoying paywall, to do what you probably should have done a while ago. If you regularly listen to this podcast and feel good about yourself, Please go to yashamonk.substack.com and support the work we do here at siaschalmonk.substack.com.