← The Lawfare Podcast
Loading summary
Transcript42 lines
- [00:01]
Kevin Frazier
The following podcast contains advertising. To access an ad free version of the Lawfare Podcast. Become a material supporter of lawfare@patreon.com lawfare that's patreon.com Lawfair also check out Lawfare's other podcast offerings, Rational Security, Chatter, Lawfare, no Bull and the Aftermath.
- [00:32]
Renee Direzda
Every idea starts with a problem. Warby Parker's was simple. Glasses are too expensive. So they set out to change that. By designing glasses in house and selling directly to customers, they're able to offer prescription eyewear that's expertly crafted and unexpectedly affordable. Warby Parker glasses are made from premium materials like impact resistant polycarbonate and custom acetate, and they start at just $95, including prescription lenses. Get glasses made from the good stuff. Stop by a Warby Parker store near you. When you're starting off with something new, it seems like your to do list keeps growing. Finding the right tool helps, and that tool is Shopify. Shopify is the commerce platform behind millions of businesses around the world and 10% of US e commerce. And best yet, Shopify is your commerce expert with world class expertise in everything from managing inventory to international shipping. If you're ready to sell, you're ready for Shopify. Sign up for your $1 per month trial at sh.com retail. Go to shopify.com retail.
- [01:35]
Colin O'Keefe
To really drive home the difference between the different types of loyalty that we might want to distinguish between. For AI agents, we have this concept of AI henchmen, which are agents that are perfectly loyal. They'll do what the principal asks them to, and they will be willing to break the law either if they're instructed to, or perhaps the more insidious cases not when they're instructed to, but when they realize it would be in the best interest of their principal for them to break law.
- [01:59]
Kevin Frazier
It's the Lawfare Podcast. I'm Kevin Frazier, AI Innovation and Law Fellow at Texas Law and a Contributing Editor at Lawfare with my co host Renee Direzda, Associate Research professor at the McCord School of Public Policy at Georgetown and a contributing Editor at Lawfare, and our guest, Colin o' Keefe, Research Director at the Institute for Law and AI.
- [02:21]
Colin O'Keefe
I think it would be bad if we try to make this project hinge on having a philosophical account of both the law generally and the exact application of every single law to every imaginable circumstance in this brand new world of AI agents.
- [02:38]
Kevin Frazier
Today we're talking about his work on AI agents. A forthcoming paper he co authored proposes a constructive path forward on how to adjust legal norms and systems for the age of AI agents, we'll be exploring the concept of law following AI, the innovative thinking behind it and why it's a crucial concept for building a better future with AI. Alright, today we're talking about a truly transformative development. AI agents imagine AI systems capable of tackling complex computer based tasks with expert human level skill. The potential for economic growth, scientific discovery and improving public services is immense. But how do we ensure these powerful tools operate safely and align with our societal values? Colin, your paper presents law following AI as a positive vision for integrating AI agents into society. But before we go into what it even means for AI agents to follow the law, what are AI agents? Everyone keeps telling me we're in the year of AI agents and yet I'm still planning my own travel, which is profoundly frustrating. So what are AI agents in theory and where are they right now? Just to kind of make sure we're on the same technical level.
- [03:54]
Colin O'Keefe
Sure. So when we talk about AI agents, as you suggest, we're talking about a trajectory of AI technology that's still in very early stages. So I think it's most intuitive to explain AI agents with contrast to the generative AI systems that people are most familiar with. Now, this is a type of AI like ChatGPT, where you input something, a request of some sort and get a pretty static output, whether that's a piece of text, a piece of software code, an image, maybe even a video. And that's all well and good, it's very useful for a lot of things. But what the companies developing these systems really want to do is be able to automate any task that can be done on a computer. And this is more or less explicitly how they often define the goal of AI agents being able to do anything that can be done in a computer. On a computer, caveat is important because robotics doing things in the real world is often a lot harder if it's a non routine task. But that's the ultimate goal and we're not there yet. It's not obvious that we're anywhere close to there yet, but companies like OpenAI and Anthropic are rolling out demos of their early AI systems that do things like operate a GUI computer like a human would, are able to enter search terms and click around and do some very basic tasks. But as you suggest, they're not particularly good at this yet. There's kind of another track that people have been working on for a long time, which is kind of building what we call scaffolding software around these existing generative AI systems that helps them do more agentic things, things that look more like taking actions than outputting text. But really we're mostly just talking about a trajectory that the companies are working very hard to realize.
- [05:40]
Kevin Frazier
Yeah. And thinking through our current state of where things are and looking forward to where things may go. Let's just play hypo real quick. I have to because I'm a law professor. So in this hypo, right, I'm planning my trip, my dream honeymoon. My wife doesn't listen to these. I talk too much about AI. So I can tell you that we haven't booked our honeymoon. Yes. We've been married for more than a year. Yes, It's a source of contention. Eventually we will. Maybe I'll use an AI agent to do so. So I reach out to my AI agent, I say, I'm finally being a good husband. Let's go to Greece, plan this trip. Sounds great, right? I just press it, set it. Prego. We see where I'm going in Athens until I find out. Oh my gosh, you know, the AI agent to get that last hotel room had to email the other person who had that reservation and tell them that if they didn't unbook their trip, then their child would be abducted or some crazy scenario. Right. These are the sort of fearful situations of AI agents getting out of control. So I'm trying to tee you up to help explain why do we need to be thinking about law following AI? What is the importance of setting up this paradigm for AI agents before we realize their capabilities to undercut that person who stole my beautiful hotel room on the Aegean?
- [07:05]
Colin O'Keefe
Yeah, I think that's right. So one of the features of AI systems that we've observed in past AI systems and also has a pretty strong theoretical basis is that there's no strong guarantees that they'll behave in ways that humans expect them to, first of all, or that they'll behave in accordance with any particular set of values that we want them to behave in. And so this is what people familiar with the field of AI typically call alignment. And it's typically kind of decomposed into a few different types of alignment. So the one that is often considered the most basic type of alignment is called intent alignment. And you can think of this again using a like principal agent set up that will be familiar to a lot of economists and lawyers listening to this, where the AI system does what the principal would want it to do. So Kevin, since I know you're a good law following person, an AI that is aligned to your Intent would not do that type of blackmail to secure the last hotel spot.
- [08:06]
Kevin Frazier
Thank you for thinking highly of me. Yes, I appreciate it. Yeah.
- [08:10]
Colin O'Keefe
But you know, sometimes there are, there are not good people in the world. And so a system that is only aligned with the intent of a user would not have the same type of ethical qualms with regard to users that just ask it to do something and tell it that it doesn't care if it violates the law. Or perhaps more in a more nuanced way, a more advanced AI system might reason about, is it worth it for me to break the law on behalf of my principal in this situation? Because I think I could get away with it. Because probably if, if it, you know, if it was found out that you knew that the AI system was blackmailing people on your behalf and at the very least you would feel that hopefully you would face some sort of consequence or the company would face some sort of consequence. But you know, a more purely intent aligned system might be quite Machiavellian and very strategically break the law. So people have known about this for a long time. It's why all the systems on the market today are not kind of attempting to be purely intent aligned in this way. They do have ethical boundaries of some sort. And people have often called this kind of like extra set of guardrails value alignment. But there's also a lot of kind of controversy around certain forms of value alignment because obviously humans don't all agree on sources of value. So the incident that people might be the most familiar with is the Google Gemini incident a couple years ago where basically people found that if you entered what would be pretty benign descriptions of things, it would output very counterintuitive results. So for example, it would refuse to generate pictures of white couples, but would be very comfortable doing that with couples of other races. It would depict like 1945 German soldiers since the word Nazi was censored as racially and gender diverse, which is obviously historical and offensive. So you know, there's this separate conversation about when we're choosing values. Obviously we don't all agree on what values things would have. And this, this, you know, understandably lands companies in a bit of hot water when the people who have different set of values don't like this. And you know, to their credit, it's not just cherry pick stuff. There's decent scientific literature that finds that the political views of the systems met most closely to like something like center left politics across the developed world. And you know, people are also worried about this from a further left Perspective people in the Global South. There's a lot of literature coming out of the Global south about how these systems reflect the perspectives of people in the Global North. And so all this is to say that there's a bit of a quandary here, which is everyone wants systems to have some sort of guardrails beyond just doing what's in the user's interest. But so far, most of that discussion has been around a set of extra legal normative values that sound more in ethics than law. And so the law following AI pitch is that, well, as a first step, probably not sufficient or holistic, but as a first step, let's try aligning these systems to law that is preventing them from taking actions that would break the law.
- [11:06]
Kevin Frazier
And Renee, do you want to jump in here?
- [11:08]
Colin O'Keefe
Yeah.
- [11:08]
Renee Direzda
So your paper actually goes into the notion of AI henchmen. Do you want to describe the kind of explicitly manipulative aspect that you model out in the paper itself? So we can go into this notion of the explicitly manipulative agents that you characterize as henchmen?
- [11:26]
Colin O'Keefe
Sure, yeah. So the AI henchmen concept is really just trying to make this point that I think we all want AI agents to be loyal, but I think anyone who's thought about the concept of loyalty realizes there's kind of multiple ways to be loyal. And you know, in the traditional principal agent literature that is familiar to lawyers, agents have a duty of loyalty to their principal, but it's qualified by a duty to still obey the law. It's not a defense to break the law that you are following orders from your principal. You're still liable for lawbreaking, even if it's at the orders of or in service of your principal. And so to really drive home the difference between the different types of loyalty that we might want to distinguish between, for AI agents, we have this concept of AI henchmen, which are agents that are perfectly loyal. They'll, they'll do what the principal asks them to, and they will be willing to break the law either if they're instructed to, or perhaps the more insidious cases, not when they're instructed to, but when they realize it would be in the best interest of their principal for them to break the law, and possibly even for them to break the law without telling their principal about it, so that their principal has plausible deniability over the law breaking behavior. You know, this is a thing that we associate with sophisticated criminal organizations, the middle management taking the fall for the leadership so that the leadership keeps their hands clean. And so you Know, AI henchmen could be a pretty nasty thing to have to deal with. So, you know, I think if we're just limiting ourselves to the type of AI agents that we consider in the paper, we're mostly talking about things that can be done on the computer. But a lot of nasty stuff can happen on the computer. You can steal people's identities, you can steal money, you can blackmail people, you can create defamatory information about them with, you know, new AI tools that could be quite nasty. You can, you know, be part of a larger criminal conspiracy on the computer. So even if it's other physical humans that are going out and doing the physical dirty work, you can arrange payments and coordinate, coordinate action. You know, so it's imagining all the different ways that a digital worker could either do a lot of very nasty stuff itself or aid in the effectiveness of a larger criminal or illegal enterprise.
- [13:29]
Renee Direzda
I think one thing that's interesting, just for listeners who are not as familiar with both the theory and I think the practice is kind of how far along we are on a technological front with just how indistinguishable some of these systems actually are at this point. And there was work that OpenAI has done, some of the personhood credential work where I've intersected with some of your teammates on that. Just the dynamics of actually how hard it is to tell already at this point. But when you're engaging with an agent and the rapid acceleration of the technology and the question of, how do you even tell. One thing I really appreciated about your paper is that. And I definitely encourage listeners to read it, even if you're, you know, it's not. It's not, you know, technically challenging to read. It's really, really interesting because you go into these kind of vignettes around, like, if you're in a discord server and you're just talking about cryptocurrency, and one of the entities participating in the cryptocurrency server with you is a bot, right? Or, you know, an AI agent that engages as if it's another cryptocurrency enthusiast. But what it's there for is to pay attention to, kind of monitor the conversation. And when you say something like, man, I made so much money today, well, that's when it decides, okay, now it's time to extort this person. Right? And that. And that notion of extortion and ways in which AI is used to extort people is something that we see actually constantly in the work that I do on adversarial abuse so it is very much already in the realm of things that are real. And this question of where does the law intersect? Or in the work that I've done and intersected with the OpenAI team on, how do you have people indicate, even in a Discord server, like, I am real, I am human. That's a technologically very, very complicated thing. That is a problem that we're working on solving from a technological standpoint that we have not been able to do yet. And then there's this other question of, from a legal standpoint, what are the legal ramifications for this agent, who controls it and what happens? So I'd love you to talk maybe a little bit about even just that vignette, the cyber extortion one, which I think is so accessible to people because spam and scams are at least things that regular people encounter every day. And then maybe we can talk about the. The global national security criminal masterminds next. But that one was. I really liked that example because it is something that I think people can see themselves in because everybody has had that obnoxious people on the Internet trying to scam you experience.
- [15:45]
Colin O'Keefe
Yeah. And I'll say also that I owe a small debt to this podcast, actually, for helping inspire this vignette, because it was inspired in part by a vignette that Jonathan Zittrain told when he appeared on the podcast. I don't remember how long ago it was. He talks about how pretty soon it's going to be quite cheap if you want to really make someone's life hard, to just instruct an AI agent to just follow them around from website to website and harass them and make fake accounts, spread nasty news about them on all these different websites to the point where they have to either go totally anonymous or change their identity or something like that. And that's very close to something that seems possible today. I don't want to be in the business of making hard predictions, but it does seem quite plausible from where today's money is. You could probably do this for something like, I don't know, under. Definitely way under a thousand dollars for the rest of someone's life to just have an AI agent Google this person every day and if they find something new, you know, make a new account to. To try to harass them. So that was some of the inspiration. So, yeah, the basic story here is that, you know, a criminal group, you know, is quite interested in cryptocurrencies. You know, I'm not trying to be too negative on crypto here, but you Know, it has these nice features for criminals, which is that it's a bit harder for various law enforcement agencies to track and then halt transactions in. So what they do is they search social media, something like Twitter, to, to find kind of information of people who have been recently posting big gains on, on Twitter. They, you know, make their find information about a discord group where people tend to like, brag about these, about their big gains. Again, this is a very common thing on social media that people familiar with the crypto scene might be familiar with. And something like just, you know, if someone accidentally posts their real name or their email, this could make it quite easy for them to. The agent, that is the agent to, you know, use something like a data broker to figure out more information about this real, the real natural person that's posting about these gains and then, you know, look up more information about who their contacts are in real life and then threaten them with some sort of blackmail. So in the scenario we do, they use AI tools to create deepak pornography of, of this person. And yeah, it's threatened to release it if they don't hand over some of their crypto gains. And yeah, that's that. It's something you could easily imagine someone panicking and doing if they were faced with that threat and found it quite credible.
- [18:09]
Kevin Frazier
Yeah. And just to. To jump on there and say, listeners, please don't do this to me. I don't have any crypto gains, so. So don't get any thoughts.
- [18:19]
Colin O'Keefe
Not a lot of gains this year.
- [18:20]
Kevin Frazier
I think everyone's stock portfolio is in the trash. So we don't have to, maybe we don't have to worry about that immediately, but to ground this a little bit more into actual AI governance. So you've got this great theoretical paper about what could happen, what we could see happen in the near future as we continue in this year of AI agents. We've had a couple of folks on the POD who have said, hey, we need to make sure this is a space of, for lack of, better phrase, permissionless innovation. Let's not clamp down on AI before we see these risks on the other side. We've had folks come on the POD and say, well, we should probably just pause this whole pursuit of AI agents if we're seeing these potential harms arise. So when we think about law following AI as a sort of regulatory device, how would you like to see this apply? Do you think this is something that the lab should be spearheading? Should we see states, should we see Congress? Should this be an international Accord of lfai, which is the acronym. Should we see some big LFAI treaty or what's the point of intervention you'd like to see here?
- [19:29]
Colin O'Keefe
Well, I think one piece of good news is that the labs are already doing something like this, which is part of why we wanted to write the paper. So OpenAI has this document called the Model Spec that kind of goes through the principles in a hierarchical way that it wants its models to follow. And in part, this is meant to deal with the exact kind of problem that we're trying to solve here, where the ways that OpenAI wants its model to behave might be in conflict with the ways that its users want the model to behave. And one of the restrictions that OpenAI puts on there, it does take this very principal agent approach where it says, you know, you're supposed to mostly do what the user wants, but it does have an exception there for things that would violate the law and Anthropic's approach, which is called constitutional AI, their constitution for their system. Claude has a similar limitation, although it's a bit less explicit or a bit more buried in there. So, you know, there are kind of hints that the industry wants to move in this direction, and you can see why that would be the case. To go back to the thing that you opened with, Kevin, I think this would be a big scandal if it was found out that ChatGPT was blackmailing people in order to get a better hotel deal for their users. Right. So I expect it to be pretty sympathetic to a lot of AI companies here. So that said, the primary thing that I care about for the purposes of this project is the prospect of AI agents being integrated into the government, and particularly being integrated into the government in ways that would allow them to exercise various hard power functions, things like law enforcement, military, things like investigating citizens, et cetera. And I think as a part of our Anglo American legal tradition, that's the type of thing that we're supposed to be quite worried about. And basically, I think that that's where I have the most optimism for law flying AI. And I hope that a lot of people who take a more libertarian approach to this technology will similarly feel that that is something that there should be pretty significant guardrails of. You know, it's something that most American political traditions really care a lot about. And so, you know, I think there's a debate to be had about what sort of regulations the private sector should have with regard to how AI agents should be behave. But I think really like the red line that we're trying to kind of generate consensus for in this paper is that AI agents acting as a henchmen within the government is a pretty intolerable situation. And we can go into more why it might be even more intolerable than the situation where you have perfectly loyal humans staffing the government in these kind of hard power positions. But yeah, that's the primary way I see this playing out.
- [21:58]
Renee Direzda
If you're an experienced pet owner, you already know that having a pet is 25% belly rubs, 25% yelling drop it. And 50% groaning at the bill from every vet visit. Which is why Lemonade Pet Insurance is tailor made for your pet and can save you up to 90% on vet bills. It can help cover checkups, emergencies, diagnostics, basically all the stuff that makes your bank account nervous. Claims are filed super easily through the Lemonade app and half get settled instantly. Instantly. Get a'@lemonade.com pet and they'll help cover the vet bill for whatever your pet swallowed after you yelled drop it.
- [22:35]
Dr. Patrick McGrath
What if I told you that right now millions of people are living with a debilitating condition that's so misunderstood many of them don't even know that they have it? That condition is Obsessive compulsive disorder, or OCD. I'm Dr. Patrick McGrath, the chief clinical officer of NOCD, and in the 25 years I've been treating OCD, I've met so many people who are suffering from the condition in silence, unaware of just what it was. OCD can create overwhelming anxiety and fear around what you value most, make you question your identity, beliefs and morals, and drive you to perform mentally and physically draining compulsions or rituals. Over my career, I've seen just how devastating OCD can be when it's left untreated. But help is available. That's where NOCD comes in. NOCD is the world's largest virtual therapy provider for Obsessive Compulsive disorder. Our licensed therapists are trained in exposure and response prevention therapy, a specialized treatment proven to be incredibly effective for OCD. So visit nocd.com to schedule a free 15 minute call with our team. That's nocd.com My husband and I recently.
- [23:37]
Renee Direzda
Realized that neither of us were getting the sleep we deserved. So we sat down and talked about our ideal beds. For him, soft as feathers. For me, firm as a plank. This would be a huge issue if it weren't for the Sleep number Smart Bed. Thankfully, with our new Sleep number Smart Bed, we can each dial in our desired sleep number settings to our ideal Comfort and finally get the sleep we deserve. Plus the climate series feature makes sure our bed stays nice and cool through the warm summer months. Why choose a sleep number Smart bed So you can choose your ideal comfort on either side. And now it's the Sleep number everything Smart Bed sale. Every smart bed and base are on sale during our Memorial Day event. Up to 50% off limited time, exclusively at a sleep number store near you see store or sleepnumber.com for details.
- [24:28]
Kevin Frazier
We're speaking on April 22nd. There's a big debate right now about the word facilitate that's going on in the courts. What does facilitate mean? And I think Renee has a great thread to pull on here about the ambiguity in the law. So Renee, I'll kick it over to you.
- [24:45]
Renee Direzda
Yeah, no, I was curious as I was reading it, just this question of which interpretation of the law is correct. And that is constantly evolving. We see the notion of, and I'm not a lawyer, just to be clear, so I feel like this, you know, this was me reading it as a non lawyer. Just this question of as case law evolves as we talk about, you know, as Kevin saying a lot, hinging on the interpretation of one word and really critical decisions hinging on the interpretation of one word. How do you port that to a model where we're expecting AI agents to not only have an immediate, I don't know, like system update, how do you immediately kind of like pass that on down and through, right. How do you adapt that instantaneously? There's also these moments where things will come into conflict, a decision will come into conflict, the Supreme Court will temporarily halt something. There will be a moment where a decision is stayed, where there's a temporary restraining, temporary order. Help me out here, Kevin. I don't want to use the wrong.
- [25:43]
Kevin Frazier
Legal term, good old TRO or an injunction.
- [25:46]
Renee Direzda
Injunction. That's what I was going for. And this is where the, that question comes in around what happens technologically, like what is what is transmitted and how to the system in those moments. This is what you cannot do for this day, potentially for this hour, for this two week period. And then, okay, now you can do it again. Right. So what is, what does the implementation look like for that, for the technological system?
- [26:11]
Kevin Frazier
I mean I kind of think, and I love Renee. What I kept thinking of as you were mentioning this with the systems update is a sort of Y2K. Are we just going to have Y2Ks for all of these AI agents of, you know, California passes a new law and it's super confused about which interpretation does it follow? California law, New York law, Italian law, and then see a change of interpretation.
- [26:33]
Renee Direzda
Trying to get my head through the. Yeah, like, or, you know, you don't, you don't patch your windows or whatever and something goes horribly wrong. Right, you know, Yeah.
- [26:44]
Colin O'Keefe
I mean, yes. So I think, I think you all are hitting on the exact right type of questions. You know, we're talking about building a type of AI system that will work in the real world. And so we need to have principled answers to these questions. But one inconvenient fact about the law is that very few people agree on, like, the underlying philosophy of law. I think very few people even have a great definition of what the law is, much less what it requires in all circumstances. And I think it would be bad if we tried to make this project hinge on having a philosophical account of both the law generally and the exact application of every single law to every imaginable circumstance in this brand new world of agents. Right. And one that could command high enough consensus to pass in our very polarized world. So I don't think that we are going to be able to have something completely theorized by that. We say that we want to build towards something like a minimum viable law following AI that kind of preserves the status quo of the distribution of power in society, which is that there are limits to what people in the government and the military will do. If they see an act as sufficiently illegal, they will, many of them will refuse to carry it out. And so we do have to come up with some kind of view about how the AI agents are supposed to reason about whether they are being asked to do something illegal and carry it out. I think the world of AI agents will actually allow for a lot of fascinating possible technical solutions to this. So you can imagine that you can get 100 different legal opinions from 100 different AI lawyers. Maybe 100 different AI lawyers. Each one of them is fine tuned to the opinions of a different district court judge. So that like this one's pretending to be Judge X and this one's pretending to be Judge Y. And it aggregates all these views and comes to like a different view within less than a minute of whether the thing that it is trying to do would violate the law. And, you know, you could imagine some different decision procedure about how it's supposed to aggregate that information. But overall, I think, you know, we can start with the qualitative thing that we wanted to do, which is we wanted to refuse to take many illegal actions, possibly not all. And we can Talk about like, you know, I think expecting perfect obedience from these things is not realistic and it's not how humans work. But, you know, we wanted to have to refuse to take sufficiently illegal actions. And then I think the question that we need to figure out as a society is how can we build systems that like, roughly mirror what we expect law abiding humans to behave like? And how can we ensure that the systems that the government deploys roughly match kind of our expectations for the expectations that we have of civil servants and the military officers to both be obedient to their principles, but also to not obey illegal orders? You know, there's an even nerdier legal subject that lawfare co founder Jack Goldsmith is, is expert in, which is like the authority of the President to interpret the law for the executive branch. So I think you could have a very important discussion to be had about when the President interprets the law one way and the law following AI thinks it will likely be interpreted a different way. And there's maybe case law not exactly on point, but very close to on point in a third way how it should resolve that. So that's the type of question that we'll need to figure out. But what we're really trying to do with this article is open a conversation into, into these questions rather than purporting to offer a holistic account, in part because, you know, we think that a holistic account probably doesn't exist yet and it's unlikely to by the time we have AI agents unless they figure it out for us, in which case that would be good.
- [30:15]
Kevin Frazier
You haven't figured it all out yet, Colin. I mean, come on, we need, we need a comprehensive solution now. But I do want to applaud you for that humility and that invitation for more discussion on this topic as a sort of baseline. I have to say I'm pretty attracted to this law following AI concept because if you look at, for example, there was a recent Brookings report showing that the public is still quite skeptical of AI and in particular AI use by the government. And if you have just the Wild west AI, AI agents with no sort of guardrails, we'll continue to see that public backlash and perhaps a reactionary response where instead of realizing the potential of an AI agent that can, for example, proactively look and find, oh, hey, you're eligible for this benefit that you didn't realize, you should probably sign up. That sounds super positive. If we don't have something like law following AI, that gives the public the assurance that this can be used in a reliable and known fashion, then we may just miss out on those outcomes. So you've also shown humility by sharing this paper and this idea with a lot of folks, which is always just a gut wrenching moment of, oh, gosh, what are my colleagues going to say? What's the most compelling piece of critique you've received? What, what gave you pause? Whose feedback do you say, oh, shoot, now I have to rewrite that entire section, or now I have to plan this entirely new paper? What are some of those counterarguments?
- [31:49]
Colin O'Keefe
Yeah, I mean, I think Renee and the earlier conversation we had really did hit on one of them, which is, what exactly is the task that we're asking the law following AIs to do when they're trying to execute on this duty that we give them to obey the law, obey their principles, but only within legal bounds? So, just for example, you know, one popular theory of what the law is is often associated with Oliver Wendell Holmes Jr. It's called the prediction theory of law, where the law is a prediction of what courts are likely to decide. And, you know, the intuition behind this is that that's ultimately what most citizens care about when they, you know, ask a lawyer for advice. They don't, they don't really care about, you know, what is the true nature of the law. They say, if I am sued or prosecuted over this, am I likely to win or lose my case? Yeah, so that, that might be an intuitive place that you can start, but I think, you know, it's actually probably a lot more complicated than that, in part because there's all these duties that the law imposes on the executive branch in particular, that are not likely to be litigated. And so I think figuring out what exactly we want the AI to do and how we would even generate a like, training signal to, to train it on that task. What the ground truth would be seems pretty hard to me. I do also, Kevin, to your general point, like, worry about the trade offs, about putting barriers to adopting AI agents in government. So even though this is a piece that's quite focused on the risks of AI agents in government, I do actually think that having a lot of government automated in the medium run could be a pretty big thing. You know, I think part of the reason that people have a lot of distrust of government is because they see all of the ways and it fails. And I think a lot of the ways in which it fails are things that AI could solve. So, you know, I think if you look at like, processing times for like, Green cards for passports. That like, is something that I think AI agents in charge of issuing those documents could, could dramatically reduce the backlogs of AI agents could help you file your taxes. Like imagine if the IRS just gave you an AI agent that would like you just dump all your documents to it and it files it for free. You know, this is like the thing that most countries have that the US doesn't have. So you know, if we had something like that, you know, I think that could be quite important. And also, you know, it has to be said that like there will be a lot of pressure to adopt these AI agents throughout the national security complex, even if we have a lot of worries about it. If our major adversaries are able to operate at the speed of AI and augmented by their own AI agents. The idea that the national security apparatus is going to sit back and let us iron out the civil liberties and civil rights concerns before automating in kind, maybe that would be the right response ethically. But I think if we look at the way that these types of competitions tend to play out, I wouldn't bet all my money on it. So I would rather invest in a parallel kind of stream of work. We can and should debate when it is appropriate to automate the government. Overall, I think that there should be lots of parts of the government that are eventually automated. But I think our North Star should be when we can guarantee that it will not disturb the balance of power in society, which is which currently relies on the fact that if you had an aspiring tyrant that a lot large parts of the civil service and the military would refuse to carry out illegal orders and that they have some common regard for their citizens and that they swore an oath to the Constitution that they take seriously. And until we have AI agents that we can expect to live up to that same standard, we should be able to pretty hesitant to replace these hard power functions of the government with AI agents.
- [35:25]
Renee Direzda
I think that's a really important point. I think that is something that you do well to write a whole lot more op EDS on just to begin making that point in a lot of places, the importance of the civil liberties argument because I think there are going to be henchmen. This is the argument that came up quite a bit as I was reading the paper. I was also thinking a lot about the discussion between closed source versus open source models with regard to some of the safety conversations that happened early on regarding generating of obscene and illegal content. For example, you remember, I'm sure those, those conversations as they as that began to become a thing that we saw. And that question of the recognition that the henchmen will exist. And so this challenge of ensuring that government AI does not become what government AI henchmen is a separate problem from the notion that there will be no henchmen, which is an unrealistic state. Right. And so I think that the argument you're making here is very much related to the ethics that we want in our government, the civil liberties protections that we want for our citizens. Recognizing at the same time that the dynamic of finding and stopping henchmen is a separate and distinct challenge.
- [36:38]
Colin O'Keefe
Yeah, absolutely. And that's in part why we put such an emphasis on the governmental case is the thing that we care the most about, I think, is a conversation to be had about should private citizens be allowed to procure AI henchmen. It's not obvious to my personal politics that that is a like huge liberty interest that people have. But I think, you know, I respect people who, you know, have a more techno libertarian bent on that. And to their credit, you know, there are all these other ways that we can deal with the problem of henchmen in the private context existing. We have law enforcement, we have civil suits for damages, we have self help, we have, you know, personal cyber security and cyber hygiene and whatnot. And I like, I think society can learn to deal with a lot of those situations as long as we're imagining the system staying around the human level. I think if you get to vastly superhuman that there's a whole different set of challenges there because we have learned to deal with, you know, bad people in society and you know, nowhere has zero crime but. And states make different trade offs on crime versus liberty. But, you know, the remedies for a government run by AI henchmen are many fewer and might only be revolt at that point. And so I think we really don't want to get to that point. And you know, the majesty of the American legal system is coming up with a system on which, you know, to quote some of the like famous words of our founding, you know, the law is king in America. America. And everyone is supposed to. Everyone in the government has a duty to uphold the law and follow the law. We see law following AI as a way of kind of evolving that vision for a world in which most of the law is being, or most of the government's functions are being carried out by AI agents rather than humans and the different dynamics that it entails. And it will make it easier in some ways to make sure that government agents obey the law. Because it's not ethically acceptable to do the sort of brain surgery on humans that would be required to make them perfectly obedient to the law. And no one thinks that would be a good idea. But it will also be harder because, conversely, very few humans are perfectly obedient to their principles, and they have their own innate sense of morality. And it's really an open design question whether AI will have that same respect for the law.
- [38:51]
Kevin Frazier
Yeah, and this is just such a timely paper. And, folks, there is going to be a version of this paper on lawfare soon in an essay format and then a sort of extra credit assignment, as Renee mentioned, if you want to read this beautiful, lengthy manuscript. This couldn't be more timely. With the recent OMB MMO M2521 basically setting up what I've referred to as government by AI, we're already seeing the barriers to integration of AI come down. And as soon as AI agents are available, we can see a pretty rapid uptake by the government in some critical functions. And so having this conversation now is critically important. And before we let you go, start working on your next paper, which I expect in approximately eight weeks, so. So get on it. What else have we missed? Have we missed anything? Do you want any other big takeaways from your paper before we. We send you on your merry way?
- [39:45]
Colin O'Keefe
Like I said, the main thing that we're trying to do here is start a conversation and grow a field of people interested in this question. You know, I think under my projections about the future, I do think there will be a time where that government is very tempted to automate large fractions of it. And I think that this is a fascinating legal set of legal questions that any aspiring law professors should seriously consider working on. And so if that sounds like you, if you're interested in answering this question about what does it mean for an AI to obey the law? Like, which set of laws should we obey? You know, that's like a whole separate question. It probably shouldn't be literally every law, how rigorously should they obey the law? Again, like, you know, people, we don't think it's a great scandal if someone, you know, jaywalks. And similarly, there's probably some degree of, like, legal risk that AIs are allowed to take. So we have a list of these questions in the long form article and also a shorter list in the short form article. So if you're interested in these questions, I hope you'll reach out and join what we're hoping to build, which is a research, community and field. Really interested in preparing the world for the economy and government of AI agents.
- [40:52]
Kevin Frazier
Well, always love some homework with some additional questions for all those listeners out there. Thanks to Renee. Thank you to Colin for joining and we'll have to leave it there.
- [41:01]
Renee Direzda
Thank you so much.
- [41:02]
Colin O'Keefe
Thanks so much, Kevin. Thanks so much, Renee.
- [41:07]
Kevin Frazier
The Lawfare Podcast is produced in cooperation with the Brookings Institution. You can get ad free versions of this and other Lawfare podcasts by becoming a a Lawfare material supporter at our website lawfairmedia.org support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you get your podcasts. Look for our other podcasts including Rational Security, Allies, the Aftermath and Escalation. Our latest Lawfare Presents podcast series about the one in Ukraine. Check out our written work@lawfaremedia.org the podcast is edited by Jen Pacha. Our theme song is from Alibi Music. As always, thank you for listening.
- [42:00]
Renee Direzda
My husband and I recently realized that neither of us were getting the sleep we deserved. So we sat down and talked about our ideal beds. For him, soft as feathers for me, firm as a plank. This would be a huge issue if it weren't for the Sleep Number Smart Bed. Thankfully, with our new Sleep Number Smart Bed, we can each dial in our desired Sleep Number settings to our ideal comfort and finally get the sleep we deserve. Plus, the Klymit series feature makes sure our bed stays nice and cool through the warm summer months. Why choose a Sleep Number Smart Bed? So you can choose your ideal comfort on either side. And now it's the Sleep Number Everything Smart Bed Sale. Every Smart Bed and base are on sale during our Memorial Day event. Up to 50% off limited time, exclusively at a Sleep Number store near you. C store or sleepnumber.com for details.