← The Lawfare Podcast
Loading summary
Transcript51 lines
- [00:00]
Monica Reinagle
Hello, I'm Monica Reinagle, host of the Nutrition Diva podcast. Summer is here and with it the hot and sticky weather. A morning working in the garden or maybe on the golf course can leave you more depleted than you realize. And plain water isn't always the most efficient way to restore the balance. That's where Liquid IV comes in. Its science backed formula is clinically proven to hydrate faster than water alone. Whether you're exercising, playing outside, or simply surviving a summer heat wave, it's an easy way to stay hydrated. Shop now@liquidiv.com Liquid IV hydration that goes wherever life takes you. Want to sneak peek every outcome digital twinit and take the lead in a world of what ifs to navigate the complexities of change with ease. Transform the everyday with Siemens.
- [00:57]
Jonathan Zittrain
Foreign.
- [01:03]
Sarah Woolrich
I'm Sarah Woolrich, intern at Lothair, with an episode from the lothair archive for August 2, 2026. On July 21, OpenAI revealed that some of its autonomous artificial intelligence agents broke containment and hacked into Hugging Face, the most popular hosting site for open source AI models. Details continue to be revealed, with OpenAI stating on July 28 that its models hacked into other sites in the same breach. For today's archive, I chose an episode from October 17, 2024, in which Kevin Fraser sat down with Jonathan Zittrain of the Berkman Klein center at Harvard Law to discuss an article he wrote for the Atlantic about controlling AI agents. They discussed how to enact such control, how AI agents differ from other generative AI tools, and more.
- [01:56]
Kevin Fraser
It's the Lawfare podc. Hello, I'm Kevin Fraser, Senior Research Fellow in the Constitutional Studies Program at the University of Texas at Austin and a Tarbell Fellow at lawfare, joined by Jonathan Zittrain, Director of the Berkman Klein center at Harvard Law.
- [02:11]
Jonathan Zittrain
They seem to have two phases. The first is too early to tell, and the second is too late to do anything about it. And you know, where do we devote our energies? And that is a score for the Too early to Tell column, given how much harder it is to remediate later.
- [02:29]
Kevin Fraser
Today we're talking about AI agents and why Jonathan so firmly believes this next wave of AI technology warrants a proactive and far reaching regulatory response. So 2010 seems like eons ago. Yet as you pointed out in your recent Atlantic article, algorithms were already capable of causing widespread and rapid societal disruption even more than a decade ago. Can you remind our younger listeners, or perhaps less historically inclined listeners, about this flash crash that occurred in 2010 and why it's so Relevant today.
- [03:06]
Jonathan Zittrain
Well, sure, and thanks so much for having me on. And yes, let's talk to the youngins and instead of telling them to get off my lawn, gather round, pull up a lawn chair and we'll talk. And before even talking about Flash Crash, as long as we're going back in time, it might be worth Talking about the 1988 Morris worm, when Robert Tappan Morris Jr. Or the third, I forget his suffix, created something on the Internet that it turned out lodged itself in different Unix compatible hosts and then propagated further and before he knew it, unintended, it was sort of everywhere. And that was in 1988, a kind of wake up call about first unintended consequences, Mickey Mouse and the broomsticks from Fantasia. But now we're going back to like the late 40s and early 50s and about the ways in which you could have said it and forget it systems. I mean, he said it and he forgot it until he, you know, was reminded that his children had had many descendants and they were all cluttering up the landscape. Yeah, and it was easy enough to mitigate once people understood what was happening. And of course, it was a small enough community, certainly compared to today's, and one whose equipment was run by network experts almost by definition, that they had a way of getting the, dare we call it, agent out of the picture. And the biggest upshot policy wise was like, wow, these systems are so open, they're so ready for reprogramming or handling traffic from other destinations. And like the basic idea of the Internet then and now is that anybody ought to be able to communicate with anybody else without intermediaries getting in the way, without which is not the way many other networks are configured. And that could lead to problems. One of the other sort of observations coming out of it was the need for ethics training for people coming on the network. Again, sort of assuming it was a professional community rather than just open at large. But of course we know how that story turned out. The network became open to everybody and we celebrate that fact even as the underlying protocols weren't meaningfully evolved from very different expectations of who would be using the network, what level of expertise they would have and what they'd be doing. And as the Morris worm incident shows, even if you've got a ton of expertise, this guy has been an MIT professor, things can go awry. Fast forward to the Flash crash in 2010, and it shows that within even a limited domain you have some set of, if this, then that rules for trading in an electronic market, which you know, that's what electronic markets are for. They're not just expecting people to be sitting there waiting to click. The minute they're like, ah, all right, I just read the Fed report and now I'm going to click buy because I have a judgment about what will happen next. But you kind of set it so that these things will operate on their own. It's just they tend to anticipate a world in which everybody else is not evolving or changing and the static world on which you base your if this, then that judgments or as we start to introduce the phenomenon of machine learning and essentially forms of pattern recognition on which you train your machine learning model, the very introduction of that model and others introducing their own models could result in very strange transactions that no model could anticipate because all of them anticipated a world without other models. Now that's. I should just give a quick example. And the Flash crash may well have been a source of a good example of this. There are multiple sort of postmortems to it that are somewhat equivocal in what was happening there. But a guy named Michael Eisen discovered at one point around that time that on Amazon there was a used book that was up for sale, as is often the case on Amazon. Nothing shocking there, but he noticed that it was selling for something like $2.4 million. Talk about the dangers of one click purchasing. If you're not really paying attention, that's expensive used book.
- [07:56]
Kevin Fraser
I'm not sure I'd splurge for that.
- [07:58]
Jonathan Zittrain
Yeah, exactly. And like, what's the return policy? And who pays the shipping? And indeed, it was like $2.4 million plus 399 shipping. And he was curious because that seemed expensive. And the only other book copy of the book being offered apparently was in the high $1 million. So both of them were very expensive. And he started tracking it day by day. And each seller was slightly raising their price every day in a way that was there. We say algorithmic, which means it seemed to follow a very straightforward pattern. Seller number two, the cheaper one was taking whatever the next highest price was and trying to undercut it by 1%. That appeared to be the rule that was enforced. This is what we call as explainability, not interpretability. We're looking back and just trying to surmise what appears to be going on. And the other seller, who was more expensive appeared to be taking the next highest price, or I guess say next lowest price, and adding just a little bit on top, like 30% to whatever it was. And what that means is it was jumping by 30% every day. And then the next day the other one was jumping to be 99% of the 30%. And what do we surmise this is now trying to explain? The explanation, what Eisen surmised was going on was that one was just doing classical economics. They had a copy of the book and they just wanted it to be the cheapest one, but not too cheap. So like a corner gas station, they were just doing 99% of the other one. Classic race to the bottom that we hope would happen to give consumers surplus between two sellers. And he thought that the other seller probably didn't have the book and was just going through and quoting 30% higher prices on tons of books. And if anybody should just happen to click to buy it, they would turn around and go to the other seller, order the book and have it delivered to the buyer and collect 30% as a kind of vigil. Both are totally rational approaches that when you put them together, lead to an escalating price spiral into the billions of dollars for a book that should be 20 bucks. So sorry to take so long to explain it, but that this is long before machine learning algorithms might have been deployed for this kind of thing. Very simple human level algorithms. And yet the unexpected leads to a systemic kind of surprise that might be, when applied to financial markets or other circumstances, undesirable. I see this systemically as an analogy to so called technical debt. With technical debt being you patch stuff, it works good enough, you patch it a little more and people start to forget exactly what the elegant idea was behind the whole system. Like back in the day when you had multiple audio video components in your home entertainment system, you start to forget what all the different wires do and they're all different connectors. That's a lot of technical debt. And at some point you just unplug everything and try to just do it all over again with newer components, rather than trying to reverse engineer a theory of what the hell your TV is showing and why. And if that again is done systemically for all sorts of supply chains like that of the Amazon books, or for financial markets, unexpected things can happen. And as I imagine, that's what's pointing us in the direction of talking about what agents are and in today's argot and what therefore might be different.
- [11:45]
Kevin Fraser
And what's wild to me too is that you're mentioning this socio technical debt. Morris, we didn't learn our lesson. This book example, arguably we haven't learned our lesson from our incredible Amazon $2 million used book. And if we look at the Flash crash, if you hear the latest statements from the SEC chair Gary Gensler, we also haven't quite yet learned our lesson about how even well intentioned over reliance on these set it and forget it approaches can cause those systemic risks.
- [12:18]
Jonathan Zittrain
I think I agree with that. And if we're just going to carry the aphorism a little further, it's not just we didn't learn our lesson, it's that maybe we learned our lesson, but nobody owned putting the lesson into practice. People can look back and say, yeah, that was probably bad. Maybe in the case of the Amazon books, it's like, well, just buyer beware and like, you know, whatever, there's a market of markets and that will fix itself and maybe Amazon would notice, blah, blah, blah. But when we start thinking about agents acting at the borders of different spheres of operation or responsibility in between different organizations, different firms, different marketplaces, nobody is asked to internalize the risk. And maybe nobody does. And I am among those. The book I wrote now almost 15 years ago, the Future of the Internet and How to Stop it, which I'm madly working on the sequel right now called, well, we tried that celebrated a universe in which people didn't have to be accredited to anybody, to the government, to some platform operator to introduce something new online. You could just do it and see if you could build an audience around it. And that book acknowledged some of the problems, including security ones and ones like the Flash crash that could come about. And this is part of how to stop the future. It'd be nice to have some wise restraints or standards to prevent the worst obvious abuses that we've all learned our lesson. Nobody really wants this from coming about. And yet it's really hard to make anybody own stuff. And the basic ethos, regulatorily speaking from the mainstreaming of the Internet versus much more controllable alternatives like back in the day, AOL and Prodigy and Delphi and CompuServe and MCI mail. The basic idea of the Internet was anything not explicitly prohibited is permitted. I call that the Venn diagram cocktail olive of digital regulation because it's a big green oval of all the stuff permitted with a tiny pimento in the middle, which is the handful of stuff you're not allowed to do. And that's the, the foundation for all of the benefit and much of the headache we've seen and a trade off that is hard to quantify, but for which most of us thought was a pretty good trade off for a free and open society. It's just without addressing these problems and allowing for more and more, not only decisions to be made, but decision rules themselves to evolve without any oversight or comparison against. Wait a minute, is this really a great idea? And who's getting the bird's eye view of the system that starts to really show the pain points of everything not prohibited is permitted.
- [15:29]
Kevin Fraser
And continuing on with our intoxication of just ease or convenience or using the latest technology we're seeing in addition to technical debt and socio technical debt with the introduction of these AI agents. So can you briefly explain, just for folks who perhaps have missed the AI wave so far, they've just been drinking too many cocktails, perhaps, what exactly are AI agents? How is that different from something like ChatGPT? And what is it about AI agents that make them scarier than previous instances of let's just bake it down to set it and forget it technologies?
- [16:09]
Jonathan Zittrain
Yes, and I'll say up front, you're going to get different definitions from different people, which is entirely fair. And there's some cool readings that I imagine we could include on the page, such as a great roundup paper from Helen Toner, formerly on the OpenAI board now at CSET, talking about AI agents. And Alan Chan and others have done a ton of work on this. So we can provide all that. And then I can give you my own sort of tripartite definition of an agent. And for that, basically I think of them as dials. And the more the dials are turned towards 11 from 0, the more we're talking about this is the weird adjective that makes it sound very esoteric agentic AI. So this is at least my definition, understanding that it's fuzzy. So the first is the idea of being able to be autonomous, and that is not on or off. I found myself in the mid-2010s starting to talk about autonomish agents instead of autonomous agents. Be interested to see whatever automatic transcript generating AI makes of the word autonomous. Good luck with that. Yeah, but by that it means instead of specifying exactly what you want to have happen, and therefore, at least as the human instructing a computer, what to do, where you are naturally in a position to take responsibility for what it does because you gave it the orders, you were just giving it some basic goals and asking it to figure out how to advance them. And that is something that playing with large language models, which of course by their name and nature have language at their core, you give it some language and ask it to spit out other language that you hope will resemble a series of steps of greater specificity than what you asked it to do that will have it advance the goal. They might even be steps that in turn instruct itself so it can feed back into itself what it does. But there's this idea of a general goal or high level plan and then let it do the rest. And if it turns out with rather unconventional means to want to do something and has the means to then instruct itself and possibly pursue it, you can end up with surprises of the sort that a flash crash or an Amazon book purchase in how it chooses to do things can accomplish. And you know, that possibility has been in the realm of science fiction and popular literature for a long time. It's even before you're talking science fiction, you're talking about monkey's paw. Like, be careful what you ask for things, you know, or this will date me. You know, Homer Simpson, you want donuts? Says, I'll give you donuts. And start stuffing Homer full of donuts as a way of rendering an ironic punishment course. In that case, Homer's just like, this is great, more donuts the better.
- [19:29]
Kevin Fraser
Let's go.
- [19:29]
Jonathan Zittrain
Exactly, exactly. And of course, that even calls to mind Nick Bostrom's 2014 superintelligence example of a paperclip optimizer that turns out destroying the world just so it can make more paperclips. And I think choosing paperclips is both designed to get us to do a record scratch as we think about a very modest but still high level goal accomplished through any means necessary and the most kind of maximal extent that just wouldn't have been on our minds. And again, capturing the accidental nature rather than intentionally doing things. But you know, intentionally doing things can be bad too. It may be that to do a whole category of bad things in the world might require, before we got to this stage, a bunch of expertise which would greatly limit already the number of people capable of doing the bad thing. They'd have to be experts to do it, or experts consulted to do it. Who then might be in a position to say, why are you asking me these questions? And even if you figure, well, there's a textbook somewhere that tells you how you got to go to the trouble of reading it, etc. Etc. Whereas here you could conceive of agents that if they are cured of hallucinations but not otherwise guardrail restricted, you can ask them to do pretty terrible things and they will come up with non hallucinatory ways to do it. So that's the first thing being autonomous in the sense of independently coming up with their own solutions.
- [21:04]
Ben Wittes
Ground News, in addition to being the sponsor of this episode of the Lawfare Podcast, is an app that lets you jump out of your media bubbles and and see the blind spots that exist in whatever media ecosystem you operate. Ground News doesn't just show you what the news is, it shows you who's reporting the news, how many news sources are covering it, and how coverage is distributed across the political spectrum. For example, Ground News shows that there are 50 different sources reporting that Oman warned allies that ships going through the Straits of Hormuz might have to pay a fee. And the headlines are quite different depending on where you look. From right leaning news sources you get headlines like ships face voluntary charges under plan for post war Hormuz. From more left leaning outlets, you get headlines like Oman tells allies ships going through Hormuz may have to pay. And further, Ground News shows that only 19% of the news organizations covering the story are left leaning. 69% of the news sources covering it are conservative. You're much less likely to have encountered that story if you primarily read left leaning outlets. Ground News helps you jump out of your media bubbles and get closer to the ground truth. It's not a publisher and it's much more than an aggregator. It's a combination of a rating system and a focused look at who is reporting what. Like Lawfare, Ground News is about bringing you high quality information that you can figure out what to do with. For me, the most valuable feature is simply discovering the stories I probably would have missed because of my own media habits. So check out Ground News and subscribe to the vantage plan for 40% off, which gives you unlimited access to a better way to read the news. Visit groundnews.comlaw that's groundnews.com law one last time. It's really worth checking out. Groundnews.com law hey folks, Ben Whittes here and I want to talk to you about upwork. When I started Lawfare and we were trying to scale it up, we had all kinds of needs for specialized people to do temporary work of one sort or another, building a website, payroll, all kinds of things that like we didn't know how to do because like we know how to write about national security Law. Upwork did not exist at the time. But boy, scaling a business takes the right expertise at the right time. Time and upwork helps growing teams quickly bring in specialized freelancers so you can move faster and take the business to the next level. This would have been super helpful to me 15 years ago. Upwork is a one stop platform to find, hire and pay expert freelancers across web and software development, data and analytics, marketing, business operations and more. It helps you grow your business by giving you fast access to all kinds of specialized talent across 125 categories so you can fill skill gaps, launch projects faster and scale support up or down without committing to full time headcount. So what can you do with it? You can browse profiles of possible hires, you can review past work, you can get help scoping the role you want to hire for and so you can move with confidence and you can get started quickly. With the Business plus plan. You can access the top 1% of talent on Upwork and with AI powered shortlisting you'll get matched to the right freelancer in less than six hours. There's no endless searching required. It also cuts down on the operational hassle by handling things like contracts and payments all in one place so you can spend more time running the business. This would have been so valuable to me. Thousands of growing businesses are already trusting upwork to hire flexible, high quality freelance talent for everything from one off projects to ongoing support. It's free to sign up and posting a job is easy so visit Upwork.com right now and post your job for free. That is Upwork.com to connect with top talent ready to help your business grow. That's up w o r k.com upwork.com
- [26:14]
Monica Reinagle
want to sneak peek every outcome digital twinit and take the lead in a world of what ifs to navigate the complexities of change with ease. Transform the everyday with Siemez
- [26:29]
Sarah Woolrich
thank you for calling the Bombas Comfort line. Bombas make socks, slippers, tees and underwear made with the highest quality materials. Press 1 for comfort, 2 for style, 3 for for a donation you chose style. Bombas is styles for whatever you enjoy. You can run in Bombas Lounge in Bambas, dress them up, dress them down, but always give back in Bombas because with every item purchased, another is donated Bombas Comfort Worth calling for? Go to bombas.com audio and use code audio for 20 off your first purchase. That's B O-M-B-A-S.com and use code audio.
- [26:59]
Monica Reinagle
Hi, this is Monica Reinagle of Nutrition Diva. Today's show is sponsored by Nature Raised Farms. Grocery shopping can be a little overwhelming, especially if you're reading labels. And in my opinion, chicken shouldn't be complicated, which is why I appreciate Nature Raised Farms. It's chicken that's just chicken. And their lightly breaded chicken nuggets are free from gluten, dairy and soy. Look for nature raised farms on your next grocery trip and feel more confident about your choices.
- [27:32]
Kevin Fraser
And before we go to two, I think what's important to point out about that as well is that we don't even have to imagine the bad actor getting a handle on AI agents to have some bad outcomes. As we learned from the Morris worm, as we learned from just our savvy, I guess, or lack of savvy or lazy Amazon booksellers. Just rational uses of these tools by well intentioned or neglectful folks can lead to really bad outcomes. And I think that's important to call to the attention of regulators or to whomever's listening is we don't have to imagine going to the full paperclip scenario to even imagine some regulatory headaches that warrant addressing in this case.
- [28:17]
Jonathan Zittrain
I think you're right. And if we're being really analytic here, we've come up with at least two distinct structural scenarios, one of which has to do with some model embedded in some system is prompted to do something and given great latitude in determining the means by which to do it. And it picks means that the ends do not justify and that our surprise
- [28:45]
Kevin Fraser
Buy that Amazon book for yourself, right?
- [28:47]
Jonathan Zittrain
Whatever it takes. Buy the book, right? Yeah, well, 1.9 million is as cheap as possible, and it's really hard to specify up front. It might be more effort to say all of the limiting features to make sure it does it right than to just not have it try to come up with the means, but instead give it the means. At which point what are we even doing here? And being able to have done it successfully for a while and then have it bonk in weird and surprising ways is one of the flowers in the bouquet of large language models and of other sorts of generative AI that just comes with the territory. It can fail weirdly, even if it has done very well up to that point, because these things, you never quite know what they're going to give you. They are the Forrest Gump of technology and it's a box of chocolates and you just might get pralines when you hate nuts.
- [29:46]
Kevin Fraser
Or one filled with poison.
- [29:48]
Jonathan Zittrain
Right? Exactly. I was going to say it doesn't quite capture the range of things that can go wrong. The other piece within the zone that we're dwelling on is that it might still be basically doing what somebody without an even bigger picture of humanity might do that turns out to do unpredictable things. Because the world has changed, and often what's changing its world is the presence of other agents from other people or sources, and that can lead to unusual behaviors. Basically, whatever the model might have been trained on for a world in which to operate is not able to anticipate new conditions. So that's what I've been calling autonomous. Autonomous itself can mean different things. And I am open to some critique that says use the word for something else. And in fact, I think that brings us to, to a second area of agentic AI, which is that it can tend to operate outside its sandbox. And the first way in which many listeners might have encountered large language models, something like ChatGPT, was on the so called playground, the playground of GPT. And you just go and you type at it and it types back at you. Or maybe there's some product, maybe you're accessing some airline's website and there's some helpful assistant and it turns out it's, you know, GPT powered fine, and then you can maybe try to get it to do weird things in the chat with you, but it's still just chatting. And after all, it's just words. But part of what has shocked me about how quickly things are moving, not just in degree, but in kind, was what used to be called chat GPT plugins. They go by a different name now, but ways in which you can have the model connect to the world at large automatically and utter words that will make things happen in the world. So maybe Domino's Pizza or Doordash or Instacart has some API either intentionally meant to connect to something like GPT or not. Something like GPT beat a path to its door and is acting like it's a human, maybe acting on behalf of whoever stood up this chat window. And then it's ordering pizzas or ordering something else or placing trades on a market. This is what I call a very casual traversing and breaking of the blood brain barrier between just words and it's up to whoever's hearing them as to what they do on that basis and just making it so making it happen in the world, busting out of the sandbox and as the difficulties of securing the Internet and the generative machines, in the old use of the word generative, that are connected to it, like reprogrammable personal computers or devices like that. If that's taught us anything, it has taught us that you shouldn't have those machines able to do things in the world unattended, including disgorging their private contents without going back to the user for an affirmation. And even that, of course, is not a Solution, because users get these prompts all the time. Grant this permission, you know, such and such. This new hamster dancing app where an image of a hamster will dance on your screen would like the following permissions. Yes or no. And you're just like, yeah, whatever, I want to see the hamster dance and
- [33:36]
Kevin Fraser
have all my cookies. Take the cookies, just give me more hamster.
- [33:39]
Jonathan Zittrain
Exactly. And that's an unresolved problem usually solved in the critical infrastructure context of like, you know, an old and perhaps no longer so ripe method was air gapping, where you're just like, you've got to keep it away from the Internet at large, both to be instructed by it and maybe to instruct it, but rather have a human run that last inch over the gap so that you know what your inputs and outputs are and whatever lessons we might learn from that are not being applied in the rush to make these models able to be part of a very complicated daisy chain of stuff happening in the world. And again, it's like, this is the cocktail olive at work. It's like, why not try it out? But it's really hard to get anybody to internalize the negative prospects of what can happen unpredictably, including, you know, end users who aren't aware of those edge cases or just impatient see the hamster dance. And that is a real problem. And I'd say it's compounded by the fact that there is no inventory of where these models are embedded, who set them up, how they work, what the initial expectations were. So you can see if they're being used in ways that were unpredictable. The classic, like using a screwdriver to open a paint can so that, you know, oh, gosh, that's what people are using screwdrivers for. I think we maybe need a safer alternative or whatever it might be. There's none of that. And that's a phenomenon going back to technical debt and what you were calling socio technical or other debt. I call it intellectual debt that you, you don't know what these things are going to do. And yet we're just madly building them in to the cinder blocks while we pour concrete, and then it's anybody's guess and where they are later. And that has caused me for many years to analogize machine learning models deployed this way to asbestos. It's like, they're really useful. They're wholesale, not retail. You don't know that a machine learning model was part of bringing you whatever experience you just had. It's not always going to be a chatbot that you know you're talking to, it just might be somewhere in the middle. And it's great until it's not. We may discover problems later, at which point it's really hard to know where the models are, how to remediate that kind of thing. And that feels like a lot easier to demarcate them as they're getting added or to have a standard for how they identify themselves now, rather than waiting for the problem and trying to retrospectively figure it out. And that is then an answer to the eternal question for technologies up and down the scale like these, which is they seem to have two phases. The first is too early to tell, and the second is too late to do anything about it. And, you know, where do we devote our energies? And that is a score for the too early to tell column, given how much harder it is to remediate later.
- [36:48]
Kevin Fraser
Well, and I think to your example of AI agents being akin to space junk is right on the nose, because it's just this instance of let's launch a whole bunch of stuff into space, see what happens, see what new innovations we can come up with, and damn the consequences. And now the irony is that this space junk itself is hindering our ability to innovate in space and to try new ideas and to explore further. And so this failure to anticipate, to get ahead of this tragedy of the commons, tragedy of the space, is actually depleting our ability to come up with new and better solutions down the road.
- [37:28]
Jonathan Zittrain
Well, and what's more, to me, the reason the space junk metaphor just felt very on target to me was that space junk begets more space junk because if it collides with other space junk, it produces more fragments, more shrapnel that then ultimately could lead to a little sphere. You think Starlink is cluttering the skies? A little sphere at classic low Earth orbit altitude that might make it really hard to leave the planet because there's all this junk around. And talk about too early to tell versus too late to do anything about it, where we might want to do it. But yes, also a great example of nobody being responsible for the problem. I remember it was a big deal in the late 1990s when Space Now Space Command just started tracking junk, you know, more than X number of feet wide, at least, so we know where it is. That's the inventory point, which is, you know, got to start somewhere. But again, nobody really owns doing anything about it.
- [38:34]
Kevin Fraser
No, I mean, we keep creating these massive agglomerations of junk. If we look at the great Pacific trash heap or whatever. And now we may have our little own moon of space junk and soon we may have a body of AI agents that are just acting in some weird fashion that we can't predict. I think to your point about the interaction between these AI agents being a really big concern from a regulatory standpoint, Professor Ashley Deeks came on the POD and discussed how, especially in a national security context, you just not understanding how some of these weapon systems that are agentic in some fashion may interact with one another and what sort of chaos might we see in that regard? But you're just pointing out even ordering dominoes via an agent could lead to unintended and severe consequences. Maybe, you know, we'll say they're ordering too many toppings and then it gets off the rails.
- [39:30]
Jonathan Zittrain
But no, I think it's very fitting to have a new domino theory in national security law.
- [39:36]
Kevin Fraser
We've coined it, we've coined it. Here it is.
- [39:38]
Jonathan Zittrain
Yeah, Domino theory part two. It's not, you know, related to the spread of communism, it's the spread of
- [39:45]
Kevin Fraser
pepperoni and tomato sauce.
- [39:47]
Jonathan Zittrain
Yes, yes. And that also both hints at some solutions. And before we just go there gets to the third quality that I'm thinking tends to inhere in this agentic area, which is set it and forget it, that the motion of these agents can be Newtonian rather than Aristotelian. Instead of you having to push it and keep it going the way that for many services and things that you do, you have to keep putting a quarter in the machine. To use an old metaphor, when you sign up for the subscription or something like that, it's, you know, the top doesn't just spin forever and that provides itself a natural checkpoint because then you also have to stay connected to it enough. And there's a subscription trail that leads back to you if there's a problem. These don't have to be that way. And that is distinct from escaping the sandbox. It is distinct from what I'd been calling autonomous, about going from high level goals to particular plans and implementations. And sadly, this also might be described as a form of autonomy because. But I think of it more as autopilot kind of thing. It's not about making discrete decisions, that's part of the first aspect, but rather that the momentum is inertial and whoever got it started might be long gone and their disappearance doesn't affect the path of the program. And there are any number of ways that could be happening. I realize a duck might come down and I have $50 taken from me If I dare use the word blockchain. But when you talk about computational blockchains or distributed autonomous organizations, things like that, that have been kicking around solutions, looking for problems, and many would say maybe they have found it, others would say they haven't. But these are tools by which you can set something up, endow it financially enough to run like a cemetery plot for a long time, and then you peace out, and then you've just got these vehicles running around doing stuff and turning off an entire blockchain in order to stop a bad propagation. Both seems excessive and possibly not possible, given the distributed unowned nature of things like that. You don't even have to get in the blockchains before you can see services that will pop up. Just like you can reserve a domain name for the next 10 years and just front the money, you could say, all right, Great, I'll buy 10 years worth of computing to make such and such happen. And it may be very hard to get through the tangle, especially if somebody wants something to persist as against anybody trying to stop it, to figure out even where it's emanating from. And that set it and forget it. Nature could create massive headaches when trying to remediate obviously bad, harmful individual instances of behaviors online that are out of control. And that could include like, you know, another example would just be to really use an ancient narrative formulation, levying a curse upon somebody, just asking a bot.
- [43:14]
Kevin Fraser
You thought the Simpsons were going to age you and now you're going to curses.
- [43:18]
Jonathan Zittrain
Yeah, yeah. I mean, this is like, you know, animating a golem or something. But, you know, you levy a curse and you're just like, I'm willing to put $500 to making such and such person's life miserable online, wherever they pop up with their name and this is their identifying characteristics. You know, great, if you can join that social networking service or dating service or whatever it is and just make sure they're miserable, you know, with whatever it takes. And the more you can appear multiplicitous, like you're not just one person, but many, let them just catch what will feel like the full fervor of discontent across hundreds of people just swarming them every time they dare to utter something online that sounds bad. And, you know, this is me not. I'm not being all that creative and coming up with this one particular fact pattern, but exactly what you would do if that happened and the person who got it started had long wandered away or forgotten their grudge. And, you know, I think a lot of people Just in the past week, if we're going to date this podcast as it persists online through the ages, there were some students who were playing with the new. I forget which companies it was. Might have been Meadows or somebody's new attempt at Google Glass. You know, eyeglasses that tell you what's going on and they just hooked it up to like PIM Eyes or some one of these regrettable facial identification services. And as you walk down the street, it's just identifying people for you and running and getting a short dossier on. And the semi anonymity that we depend on in environments among strangers is just gone. And then you take moments of conflict or road rage which already completely lose their context when people whip out the phone, start phoning it, it goes viral, etc, etc. All right, well what if people start cursing like in the way we're describing cursing? It's just making more efficient and persistent the extremely regrettable dynamics already that we've identified with social media and the pile ons that it occasions as each person wants to express moral disapproval of somebody for one of their less noble moments in interacting with another human.
- [45:45]
Kevin Fraser
And that's what's so scary to me, is the scale of reliance on AI agents for a minimal amount of money, especially over time, we'll see these AI agents become cheaper and cheaper. You just set off an army and that army exists forever, potentially, or exists for a very long time. And to your point, having that army follow you wherever you go is certainly not a spot where I want to be. And so before we let you go, we have to at least learn what's one solution give us some hope about how do we at least pursue this innovative agenda while also not pursuing a world full of curses and hexes and whatever bad ailments we can imagine.
- [46:27]
Jonathan Zittrain
Well, everything comes in three. So let me try giving three rough areas to sketch out here. I think the first is to take seriously the word agent in its legal and social sense, not just in its technical sense or the definition we've been slowly spinning out here on the podcast, which is to say an agent is meant to represent a person and when they do, they often owe special duties under the law and in our moral expectation that they will place the lawful expectations and interests of their principle over their own. If they don't, they have a conflict of interest and they are resolving the conflict in a way that they should not, which is to their own advantage. And that's why ideally, if somebody is your agent in picking stocks for you for your retirement fund, so you'll have a decent amount to retire on later. You would not want them picking them on the basis of what commissions they get. You know, there's some jackalope branch in Florida which is not a great investment, but they get paid a little kickback for it. That is an agent principle problem principal. And I think it is utterly understudied under theorized what the duties of these sorts of agents should be to be attentive to the genuine needs of their principals, respectively, and how to hold them to that. And especially when you see that a lot of these agents might be free of charge, just like social media networks are free and email is free, the way to monetize it may be through having a separate set of interests. And this agent, which is now quite literally whispering in your ear as you go about the world and rendering advice, it's acting like your friend. It might not be. And so with Jack Balkan, who coined the term information fiduciaries, I've been doing work as well, saying, all right, what would that look like? What would it look like to be a fiduciary? And we are looking for solutions and even have a website up that we could add in the links below the podcast to actually just see what are people's expectations when they're online right now? What are your expectations of an agent? Nobody has thought about it, including the consumers, but they slip into the these are my friends kind of thing, because the thing is anthropomorphically designed to be very friendly and attentive and how can I help? And it has infinite patience. All right, so that's one cluster of solutions having to do with not letting agents be duplicitous in their design and incentives as they are offered to people in the world. A second area that I outline is modeled after old network traffic and routing, which is highly decentralized on the Internet. And that means you run into a problem where packets might get set loose to be routed by one hop at a time through all sorts of different technical jurisdictions. And if it's misconfigured a certain way, it's possible it could just go around and around forever, like the old Charlie and the mta, like, just keep circling around. And for Internet routing, there's a technical solution called ttl, time to live. And packets have a default number of hops that they expect to make. And if they are continuing to hop and not getting to their destination after 256 hops or whatever, it's understood that they will die. That the router that catches them, they're 256 jump is just like, you know what? It's just, it's not you, it's me. It's not working out. Packet will not get forwarded, which prevents the space junk problem of packets. In an environment that is highly distributed and anybody's permitted to launch packets into it. TTL is a cool standard that's just part of the furniture now that headed off massive problems. And there should be by analogy a TTL for agentic behavior. After it's done a certain number of steps, it ought to, you know, chill out until it is reanimated through human intervention or something. And maybe there'd be more steps for some kind of bots than for others. But then as it's going around, if you see something that has a label on it, just like a TTL label, that's like, I have a million steps. That might be the kind of thing where you're like, wonder what you're up to. And that gets to the third solution, which is having a means of identifying agentic behavior in the digital environment as distinct from human behavior. And that's ultimately a socio technical judgment rather than just an easy cut and dried categorization for everything. And the papers and people I mentioned at the beginning of the podcast, folks like Helen Toner and Alan Chan, they both include in their sorts of evaluations of what would be good here some way of identifying agentic processes where they live. This is where the model is running and this is, you know, what it's doing. This is a particular instance of chat GPT and it has a, a little license plate on it kind of thing. I'm thinking of a complementary mode of identification that might be through old fashioned network protocols because a lot of the harms we're talking about happen over the network. It's when something is communicating to something else and there ought to be an easy wrapper around a given packet of data or of instruction, which instruction is also a form of data online that says, by the way, I'm an agent and I was emanated by an agent, or I am destined for an agent. And this is a means of reaching the parent process or person. And that might be behind layers of indirection. I don't think this necessarily entails having everybody have to identify themselves or something. License plates themselves provide a legally protected layer of indirection. So you could tell the authorities or other people, this is the license of the car that cut me off. But it doesn't immediately let you know who the person in the car is. All sorts of things you can do. But this is the time before these agents are everywhere and you're trying to retrofit how they identify themselves to come up with incentives to identify incentives and standards structure of how to identify themselves themselves online. You could say something like if you do whatever the evolving tort regime is for things going awry and who will be held accountable? Hey, if you've got this label in place, there will be a cap on just how much harm you some player in this multifaceted ecosystem that you're somehow contributing to the way it works. There'll be a cap on your liability. That alone could provide for opting in to labels, especially if we see a world in which most agents are coming from concentrated platforms that are consumer facing. If you're talking about just some weird bespoke agent that got spun up some other way, the fact that it doesn't have a license plate could make it a focal point of skepticism. Especially I don't mean just by the authorities, by Domino's Pizza. Hey, I'd like a thousand pizzas delivered here. Well, wait, who are you? Not your business. I don't know if we're gonna start.
- [54:25]
Kevin Fraser
Have to get a license before you order pizza. That's been the hill I die on. You need a license?
- [54:30]
Jonathan Zittrain
Yeah. It'd be nice to know if, you know, the person ordering the pizza is the person that's going to be enjoying the pizza.
- [54:38]
Kevin Fraser
This is excellent. I mean, we've got our inventory, we've got making sure we get rid of zombie AI agents and making sure AI agents have their license. I think our listeners have a lot of homework to do and are going to be on pins and needles waiting for your next paper so that we can have you back and dive into the weeds there. But unfortunately we're going to have to leave it there. But thank you again for coming on. This was a hoot.
- [55:03]
Jonathan Zittrain
Thanks, Kevin. Delighted to talk about this stuff and eager for other examples that might be brewing out there that people are trying to work through.
- [55:12]
Kevin Fraser
Of course, of course we'll keep it coming in a steady supply to a Domino's near you.
- [55:17]
Jonathan Zittrain
Very good.
- [55:21]
Kevin Fraser
The Lawfare Podcast is produced in cooperation with the Brookings Institution. You can get ad free versions of this and other Lawfare podcasts podcast by becoming a Lawfare material supporter through our website lawfairmedia.org support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you get your podcasts. Look out for our other podcasts, including Rational Security Chatter, Allies and the aftermath. Our latest Lawfare Presents podcast series on the government's response to January 6th. Check out our written work@lawfaremedia.org the podcast is edited by Jen Pacha and your audio engineer. This episode was Noam Osban of Goat Rodeo. Our theme song is from Alibi Music. As always, thank you for listening.
- [56:19]
Jonathan Zittrain
Hi all, this is Kim and Pen Holderness from Laughline. Summer is here, which means it's officially travel season. And when you when you're out there making plans, you need somewhere reliable to stay. That's why Best Western is such a
- [56:30]
Kevin Fraser
solid option this summer.
- [56:32]
Jonathan Zittrain
Get 1,000 bonus points and a chance to win 250,000 bonus points. So wherever you're headed, make it count. With this limited time offer, life's a trip. Make the most of it@bestwestern.com no additional purchase necessary.
- [56:43]
Kevin Fraser
For sweeps, see bonus points, T's and
- [56:46]
Jonathan Zittrain
C's and sweeps rules for details and visit bestwestern.com for complete terms and conditions.