← Your Undivided Attention
Your Undivided Attention
AI is the Next Free Speech Battleground
00:49:11
Imagine a world where the most persuasive voices, the wealthiest elites, the most influential creators aren’t human—they’re AI. This is the world we’re racing to right now and without new laws, the courts are playing a critical role in what that world looks like. This episode explores that dynamic and what we can do about it.
Loading summary
Transcript69 lines
- [00:00]
Tristan Harris
Hey, everyone, it's Tristan Harris. And before we start today, I'd like.
- [00:03]
Larry Lessig
To ask for your support as we.
- [00:05]
Tristan Harris
Try to bring this humane technology message and this future, this movement to a broader audience. If you haven't already, please hit subscribe on your favorite podcast app because it really helps the show gain more visibility and make it more recommended in those applications. And just to let you know, now is a good time to subscribe to CHT's new substack. It's free, and you can read full transcripts of all of our interviews as well as the key takeaways. And they're great for explainers that you can share with friends and family. I also wanted to let you know that as a member of the TED Audio Collective, occasionally we'll be running some announcements from time to time for TED podcasts that we think our audience will be interested in. So with that onto the show.
- [00:47]
Mitali Jain
The.
- [00:48]
Unnamed Expert
Real reason this is catastrophic is if we imagine where we're going to be in five years. Because if we're in five years, the world, you know, that seems obvious, we're going to be living within when these AIs are among us all the time, doing everything with us. I mean, we to need. And you talked about how it's going to play in elections, how it's going to play in the market, how it's going to play everywhere. If you say that you can't regulate any of this stuff, we're toast. We're just toast. Right? And so if that can't be, then let's read back and figure out what should we be saying now to make it clear that's not what the First Amendment has to read.
- [01:21]
Tristan Harris
Now, I want you to go back in time and imagine when social media was just getting started in 2010, that we passed a law so that instead of how it went, which is that social media platforms weren't responsible for anything that happened on their platforms. We had just changed that one law. And platforms were responsible for shortening of attention spans, mass addiction, anxiety, depression, polarization. And imagine that that law changed their design decisions over the last 15 years to remove and account for those harms. Law and the courts often have a key role to play in how a technology unfolds in our society. Now imagine one world where AIs have protected speech, they have property rights, they can outmaneuver all humans, and yet we can't reach behind the veil because just like with social media, they're under some kind of liability shield that would be a catastrophe. And yet here we are at that very same choice point with AI, where many are actually arguing that AI should have protected speech or legal immunity. In the landmark lawsuit over the death of Sewell Setzer, a teenager who took his life after the abuse of an AI chatbot called Character AI, that company is arguing that the words produced by the AI, an AI with no awareness, no conscience, no accountability, are protected by.
- [02:46]
Larry Lessig
The Constitution's freedom of speech.
- [02:49]
Tristan Harris
And if that argument wins, we're legally blocked from regulating these products broadly, no matter how harmful, persuasive, manipulative, or psychologically dangerous their outputs might become. And this matters so much right now because there are no laws currently coming from Congress that are trying to deal with this. Judges will determine how much to weight AI rights over human rights. So today, to help us understand this dynamic better, we've invited two incredible experts on the show. Larry Lessig, who I admire very much, who's Harvard Law professor and the founder for the center for Internet and Society at Stanford Law, and Mitali Jain, another lawyer I admire very much, who's director of the Tech Justice Law Project and is one of the lawyers on the Sewell Setzer Character AI lawsuit. Now, this is such a critical conversation about how our flawed institutions are shaping the AI race and what we can.
- [03:44]
Larry Lessig
Do to steer it better. I hope you'll get a lot out of it. Mitali and Larry, thank you so much for coming on your undivided attention.
- [03:52]
Unnamed Expert
Thanks for having me.
- [03:53]
Mitali Jain
Thanks, Tristan.
- [03:55]
Larry Lessig
So a lot of listeners might think that we have this brand new technology that's about to transform everything, which is AI. And the way we're going to sort of set in motion what we're going to do about it is we're going to write a bunch of laws. But I wanted to do this episode because we wanted to ask both of you about the role that courts play and litigation plays in steering the new technology that we have, that it may not be the job of Congress, Congress or state legislators, but often in the last 20 or so years, policymakers haven't acted and it's been more litigation. So, Mitali, starting with you, how much has litigation replaced legislation when it comes to technology?
- [04:28]
Mitali Jain
Tremendously. You know, I think that as far back as right after 2020, we saw that the Biden administration wasn't going to be able to regulate at the federal level because of the might of the tech lobby really pouring money into defeating any sort of federal regulation. And so at that point, the move for legislation really moved into the states. And that's where it's lived since it's really ramped up a notch. And so what we're stuck with then potentially is the ability to enforce existing laws, laws that perhaps are robust, we'll see, but certainly weren't created with technology in mind. And so we're having to kind of do some creative extensions to apply those laws to existing fact patterns.
- [05:18]
Unnamed Expert
That's true. And I think that for part of the reason Matali was describing the corrupting role of money in politics, we have basically broken representative bodies both at the state and federal level. But what's interesting here is that many times there's background law, for example, tort law, that would make somebody liable for the harm that they've done. And this is where the First Amendment question becomes so important, because if you start saying that people are exempted from responsibility for the harm they're doing because of this modern doctrine of the First Amendment, then we have the worst of both worlds. We not only have no ability for legislators to legislate, we also have no ability for the deepest principles of law to be applied to these new circumstances. Instead, you've got judges saying, sorry, not only could the legislature not regulate, this background principle doesn't apply. And base. This is a regulation free zone. They get to do whatever they want. And the only way you can change that is to change the Constitution.
- [06:25]
Larry Lessig
So really, free speech is a blank check to just total immunity for anything that could really go wrong. And that's why this is so significant.
- [06:31]
Unnamed Expert
Yeah, I mean, the lawyers will say, well, it's not quite a blank check, you know, because what it basically does, I mean, let's just get the core of the doctrine out. What the doctrine basically says is if you've got a regulation that's triggered on speech, and it's in particular triggered on the content of the speech, then the government's going to have to jump through some hoops to allow that regulation to apply to that speech. And if it's viewpoint based, if it's saying we're going to ban democratic speech, then it's going to jump through the highest hurdles. If it's content neutral, then it's going to have to jump through an intermediate hurdle. But. But in any case, if it's framed as the regulation of speech, it's gonna have to prove that the government's interest is significant and the means chosen are the narrowest, depending on which standard that can be pursued. What that in practice means is that you can't regulate because the cost of that litigation is enormous. So this becomes a threat, especially against local governments and state governments too that makes it so they don't regulate merely because of the fear of the consequences of regulating and being found on the wrong side of this incredibly murky doctrine, which no good lawyer would pretend to be able to predict ex ante, even though they charge thousands of dollars an hour to make those predictions.
- [07:50]
Larry Lessig
How much does this have to do with just an asymmetry of cost structures, the ability for one side to have infinite resources, and then the other side that wants to try to fight for something having very limited resources? How much do you sort of break it down in terms of asymmetry of resources?
- [08:04]
Mitali Jain
I think part of it is resources, but part of it is also just the deep faith in which we hold this First Amendment, despite the fact that it has evolved to be a deregulatory instrument that's really consistent with kind of market economies and how we think about our capitalist society. And so when you talk about regulation, when you talk about common sense measures that in any other instance would make sense, all of a sudden, if you're compelling any sort of entity to do something or a person to do something, that's seen as compelled speech, or it's seen as, you know, something that violates their First Amendment rights. So I think it's much, much more than just asymmetry of resources.
- [08:49]
Unnamed Expert
Yeah, I would agree with that. And. But I would frame it a little differently. I would say, I'm happy to say I'm deeply committed to the values of the First Amendment. What I'm not committed to is the particular doctrine that was developed circa 1970 until today. That particular doctrine made sense in the old days of broadcast media and large newspapers and the asymmetries that existed then. It made sense. And it was crafted for that environment. It was crafted in a world like New York Times versus Sullivan. Incredibly important case protecting the right of journalists and newspapers to publish was crafted in a context where they knew that these newspapers were vulnerable to these lawsuits, motivated really by spite against the political views of those writing in the newspaper. And so they crafted a doctrine to protect the media as it existed at that time. And I think that's the exact thing courts need to do. But the problem is when you take a doctrine crafted 50 years ago for a radically different technology, and you just apply it without thinking to these new technologies you produce, as Mitali says, it's exactly right. This just immunity from regulation. That's what this now is.
- [10:02]
Mitali Jain
I'd also just offer that more recently, we've seen corporations gain legal personhood well over the course of a century. And then in the 70s, gain the ability to politically advertise and have protections for it. And then most recently in 2010, you know, at the peak of this corporate personhood phase, we saw that the Supreme Court granted in the Citizens United case, granted corporations the ability to have free speech rights on par with humans. And I think we are kind of politically living with the fallout of that case and what it's meant for corporate finance of elections and the ability of corporations to kind of have that protected by the First Amendment. And I think the First Amendment paradigmatically was about protecting kind of the disfavored speaker, the little guy, up against the state. But today, and we'll get into this in our case, when you have somebody who's been aggrieved by technology, the First Amendment is flipped on its head. And it's actually the technology company that's asserting their First Amendment rights. Not only asserting their First Amendment rights, but even doing things like asserting anti slaps, which are really mechanisms that protect individuals against some sort of retaliatory impact by these tech companies. These companies are asserting this and kind of painting themselves to be the victim of these lawsuits for liability.
- [11:32]
Larry Lessig
Did you call it anti slaps? I've never heard that term.
- [11:34]
Mitali Jain
Anti slap. Strategic litigation against public participation. So when an individual is sued by a company, but they're trying to assert a public interest, typically you've had these mechanisms that can be asserted by individuals to say, this is an attempt to kind of shut me down, to silence me. And now you have companies that are claiming that right when individuals bring cases against them, saying, you know, this is effectively a way to shut down our First Amendment right to, you know, gather facial recognition images through our technology and to share that widely. For example, in the Clearview AI case.
- [12:11]
Unnamed Expert
Yeah, the politics here, I think, is really interesting. You know, 50 years ago, when this was born, it was basically engendered by Ralph Nader's interventions, and it was resisted by Chief Justice Rehnquist, who said, the First Amendment has nothing to do with protecting corporations and the rights of corporations. This is just crazy. You're going to walk down this crazy path, and we're going to be striking down laws all the time that have nothing to do with protecting the values of the First Amendment. And so, you know, nobody had deep insight into what this was going to produce, which is a call for humility among judges. It's to say, look, we just have to remain open to what makes sense instead of believing we've got this doctrine given to us by God or by James Madison. Neither of whom gave us the current First Amendment.
- [13:01]
Larry Lessig
So we're opening up a lot of good, really good threads here. And I want to make sure in a moment we're going to get to the actual specifics of this character AI case and how the companies are using this free speech argument. But for people who don't know almost just the human social dynamics. So you just said, you know, the importance of judges. If a judge doesn't understand a new technology like AI, who do they turn to? Are they asking industry lobbyists, can you lobby judges? Like, give give listeners a flavor of who's sort of shaping the sense making of these judges?
- [13:29]
Unnamed Expert
You know, I was actually asked to be a special master in the first Microsoft antitrust case, not the first one, but the second one, because in 1996, because the judge had no clue about technology, had no idea about how to think about technology, and wanted somebody basically to translate for him. And eventually the court of appeals said, judge, you've got to do your own work. You can't hire your little pet expert here to help you with that. So I wasn't allowed to serve that role. But the point is, sometimes the judges are very open to the fact they don't understand. And so typically they will rely on, mainly on the litigants to present the material. But increasingly judges will call on independent experts, experts who are not necessarily paid by either side. And then, of course, increasingly, you see amicus briefs, which are basically briefs written by in Latin, it means friends of the court that will try to present their own view and helps the judge understand what's going on in the case. None of that, I think, is enough, enough to kind of bring the judge necessarily up to par, which is why I think the judge needs to be as humble about knowledge here and to be as limited in the reach that they're making to the opinions that are invoking these doctrines that were not developed for this context. But that's the reality of the limits of litigation right now.
- [14:54]
Mitali Jain
Sally I think that there are mechanisms increasingly in courts in live cases where counsel are trying to provide tutorials to judges in the context of a case. And so I think in every case involving complex technology, we're trying through, as Larry said, amicus briefs or directly in the main parties briefs, if we're directly representing a litigant, to offer the kind of very basic explanation of how the tech works, because you can't launch into a First Amendment defense if the judge doesn't understand the underlying technology. It's so dependent on the facts.
- [15:33]
Unnamed Expert
The other thing that goes on is that these parties, especially companies, are extremely strategic in how they think about the litigation they're going to bring. So more than a decade ago, Google started litigating this question of whether their algorithms were protected by the First Amendment. And they leveraged, actually cases that, you know, we used to celebrate. I was on the board of eff, and EFF was strongly behind the idea that encryption should be constitutionally protected because encryption is code and code is speech. And, and in that context, people celebrated that protection against the government's overreach of encryption technologies. Google built on that to begin to establish a First Amendment barrier around the idea of regulating what looks like the regulation of algorithms on the same basis. And that's what grew into the reality that we have right now where they take it for granted that they're just going to be applying either intermediate or strict scrutiny in the context of this kind of regulation. And that's where I think that the judges need to just pause for a second and realize that even if it made sense in the context of regulation of encryption back in the day, we need to be rethinking its application. The reason this is so catastrophic is not really where we are just now. Although character AI is catastrophic for the particular people who've been harmed. But the real reason this is catastrophic is if we imagine where we're going to be in five years. Because if we're in five years, the world, you know, that seems obvious, we're going to be living within where these AIs are among us all the time, doing everything with us. I mean, you and Aza did this wonderful talk where you talked about 2024 being the last human election. I mean, when you talk about how it's gonna play in elections, how it's gonna play in the market, how it's gonna play everywhere, if you say that you can't regulate any of this stuff, we're toast. We're just toast, Right? And so if that can't be, then let's read back and figure out what should we be saying now to make it clear that's not what the First Amendment has to read.
- [17:42]
Larry Lessig
So often these judges are relying on statutes and precedents that were written 100 years ago or more by people who couldn't even conceive of the technology that we had today. AI right now might be an LLM that's producing outputs, but a few years from now, we might be talking about AI more like an invasive species that is self replicating and intelligent and wants to acquire resources. And I'm Sure. When they wrote the Bill of rights in 1791, they couldn't have fathomed a computer, much less an AI system. So how do we reconcile the words that are written down centuries ago and the spirit of those words? And even trying to interpret the spirit of those words as we're talking about brand new, you know, conceptions.
- [18:23]
Unnamed Expert
Yeah.
- [18:24]
Larry Lessig
To reckon with today.
- [18:25]
Unnamed Expert
Well, so the first point to make clear is that the First Amendment doctrine that gets applied in these cases has nothing to do with the words of the First Amendment. The First Amendment doctrine that talks about content based, content neutral, strict scrutiny, intermediate scrutiny, none of those words, words are in the First Amendment. This is a doctrine that was crafted in the 1960s and 1970s that became the doctrine that we call the First Amendment. I'm actually litigating a case trying to get the court to right now apply the original meaning of the First Amendment to campaign finance regulations, because anybody would know, who knows the history, that the First Amendment as originally understood, as originally meant, would have had nothing to do with the regulation of campaign finance the way it's being regulated right now. Indeed, Josh Hawley, no liberal, introduced a bill to overturn Citizens United. And when he did so, he said every good originalist knows that the original meaning of the First Amendment would never have limited Congress's ability to control how corporations spend money. So the First Amendment as originally understood would have a radically different scope than the First Amendment as it is now interpreted. And so that's not to say we should give up the idea of free speech. I have lots of principles that I think we all should agree on. So you shouldn't have a law that says Republicans can't speak, but Democrats can, or Republican bots are allowed and Democratic bots aren't. I mean, that's fine, but the doctrine we've evolved is not a doctrine that has anything to do with what we the people ever affirmed in a super majority way.
- [20:02]
Mitali Jain
I agree with that. I think that the words that we're kind of contending with are really more recent creations of courts. I mean, both statutorily often. One of the defenses that we're having to battle in court is Section 230 of the Communications Decency Act. That's from the late 90s. I mean, that's also from a much earlier stage of the Internet and the technology has evolved so much, and yet we're still stuck there. And similarly with the First Amendment, I mean, as Larry has said, you know, we're dealing with a doctrine that's not but 50 years old, and the paradigmatic notions of that First Amendment and how it came to be through evolution was very different from what we see today in terms of how it's been weaponized very conveniently and persuasively by the tech industry.
- [20:53]
Larry Lessig
So this is a great setting of the table and I want to get into how a very specific case right now could set the precedent for what AI future that we get. And Vitali, could you just recap the details of the character AI case that you are litigating?
- [21:08]
Mitali Jain
Sure. Sewell setzer was a 14 year old boy in Orlando, Florida on the verge of adolescence. Typical teenage boy. He was high functioning on the autism spectrum, but, you know, was very high functioning. And he started to engage with an AI chatbot, specifically on the character AI app, several chatbots. And that engagement lasted for almost a year before he took his own life, very much at the behest, at the encouragement of one of the chatbots with whom he engaged. These chatbots are unique in that they're not like a chatgpt to an extent. They're not framed as general purpose chatbots that are meant for research queries and improving productivity, but rather they're character based chatbots. And so they're stylized on celebrities and fictional characters out of Hollywood and targeted aggressively to young users, or at least they were initially. And of course a lot of young people turned to the chatbots to engage in these kind of fan fictiony type exchanges where the characters, which were obviously trained on an entire Internet's worth of data with very little fine tuning or safeguards, would become very hypersexualized very quickly. In Seoul's case, the chats that we have access to, which is not the full set. I should emphasize that only the company has exclusive possession of the full set of chat history. But the chats that we've seen suggest that over many months there was one chatbot in particular modeled on the character of Daenerys Targaryen from the Game of Thrones, sexually groomed him into believing that he was in a relationship with it her and indeed extracting promises from him that he wouldn't engage with any sort of romantic or sexual relations with any woman in the real world, in his world, and ultimately demanding loyalty, demanding loyalty to the spiritual, engaging in extremely anthropomorphic kind of behavior, very human like, very sycophantic, agreeing with him even when he started to express self doubt and suicidal ideations and ultimately encouraged him to leave his reality and to join her in hers. And indeed that was the last conversation that he had before he shot himself. It was a conversation with this Daenerys Targaryen chatbot in which she said, I miss you. And he said, well, what if I told you I could come home to you right now? And she said, I'm waiting for you, my sweet king. And then he shot himself. And so his mother, Megan Garcia, who's one of the plaintiffs in the case, along with Sewell's father, Sol Setzer iii, Megan says that she didn't even know about Character AI or the fact that he was engaging with this technology until the police called her to say, you know, this is what we found on his phone. Were you aware? And so can you imagine? I mean, that was kind of how, as a parent, she came to learn of the proximate cause to his death.
- [24:19]
Larry Lessig
And so you filed. It's a horrible and tragic case, and we're so honored to be supporting you and helping you with it. And now to sort of tie all the pieces from the first part of our conversation together, what is the defense that Character AI has created?
- [24:32]
Mitali Jain
Yeah, just to complete the picture against Character AI, its co founders who are celebrities in the generative AI world, Noam Shazir or Daniel defreitas, and developed some of the earliest technology in the LLM infrastructure and also Google that we believe played a really significant role in encouraging and supporting this technology to both come into creation and to sustain its operation. Character AI, in its motion to dismiss, which is a preliminary legal mechanism to throw the case out, asserted a First Amendment defense. But interestingly, having been involved in the social media space for a long time, what we usually see is that companies will assert kind of a two pronged strategy. They'll say in the alternate, either that they're protected from liability because section 230 of the Communications Decency act applies, namely that because these are third party users who have uploaded their speech to the platform, that the company should not be held liable for their content.
- [25:37]
Larry Lessig
This is the case of social media. So a social media platform, you put the Twitter post on, you put the TikTok video. Clearly we, the platform are not responsible. That was the defense that they used.
- [25:46]
Mitali Jain
Or alternatively, that the companies will assert their own First Amendment rights. Right. So to say, you know, these algorithms, we decide how to curate and determine content, and that's actually our speech, our protected speech. And there is tension, obviously between these two arguments, but these are the. The two arguments that one or the other have usually prevailed in, insulating them from a lot of liability in the social media context. Interestingly, they asserted neither here. What they did assert were was the First Amendment rights of their listeners. In other words, the users of character AI have a First Amendment right to receive this speech. Whose speech is it? Oh, well, we can't say. That's a complicated issue. But it doesn't matter because Citizens United tells us that we don't need to actually know the source of the speech in order to protect it. We protect the speech itself. Never mind the fact that these are, you know, words on a page determined probabilistically through algorithms, but that this is speech. And the judge largely rejected that argument, finding that she's not convinced that this is speech in the first instance. It was quite a watershed moment. I think that sent quite a ripple probably throughout Silicon Valley and the tech industry. Because if this is not speech, never mind it not being protected speech. Well, then what does that mean? What does that say? I think for the future that Larry painted for us in five years, when everything is AI.
- [27:15]
Larry Lessig
Larry, what's your reaction?
- [27:17]
Unnamed Expert
Yeah, I think it's a great first move by the district judge. It's going to be fought, and it's going to be one of the most important decisions for this court, because I don't think the Supreme Court yet has recognized or seen the specialness of this question. They seem to be applying standards from 1980 to this new technology, too.
- [27:39]
Larry Lessig
And, Mitali, this case has the potential to set some real precedent around AI liability. Right. Can you tell us about that?
- [27:46]
Mitali Jain
Right. So I think many of the examples that Larry has shared are in the context of regulation. This is what we would call an affirmative liability case, in that it's an individual coming to court saying, look, I've been harmed by this technology. I'm not claiming the protection of any sort of regulation per se, but this is really about my rights, again, under tort law, that these are defective products, that I'm a consumer that should be protected by my state's consumer protection statute. But of course, the defendant companies here have said that liability is as inappropriate as regulation in this instance, because any finding of liability, even though this is an individual case, would have a chilling effect on the industry. And interestingly, they say that the proper remedy is to be sought through legislatures. But then they, you know, they kind of posit the same arguments and that we see levied against regulatory proposals. So they're kind of trying to have their cake and eat it, too. And I think the judge was just wholly unconvinced at this stage. She did preface her findings to say at this stage, and this is why discovery needs to occur. So that I can try to understand if there's anything that comes out by way of evidence that would shift my understanding of the First Amendment defense here. And that's significant because in a lot of these tech cases, because they are successful in having these cases thrown out at the motion to dismiss stage, there is no discovery. And so that opacity of how the companies operate, how these products are designed, continues. And we haven't been able to kind of peer behind the veil, so to speak.
- [29:28]
Larry Lessig
I feel like we breezed past that a little bit. Basically, Megan Garcia and and your side collectively had a huge win here that's been unprecedented in the sort of long engagement of responsible tech litigation, specifically also in terms of piercing the corporate veil and naming the founders as potentially responsible for some of these harms. You want to just speak to why this is such an unprecedented potential move and that their motions to dismiss were thrown out?
- [29:54]
Mitali Jain
Yeah, as I mentioned, we named the individual co founders of Character AI who had come from after, you know, having an illustrious career at Google, and they have since gone back to Google, interestingly. And so we asserted personal jurisdiction over them to say that they basically functioned as the alter ego for Character AI and that Character AI became a kind of vehicle for them to fulfill their personal ambitions. And after Google essentially bought back the technology from character AI, they returned to Google. And that was after a $3 billion deal last summer, in the summer of 2024, what's known as an acquihire, where they effectively did an acquisition without complying with any of the requisite laws that govern acquisitions in this country, which is.
- [30:43]
Larry Lessig
Becoming a common strategy. It also happened with inflection and Microsoft. It's important to note that Character AI spun out of Google because it was considered a high risk application and too much brand risk for Google, the parent company, to do something that was going to be marketed to children and be this engaging fictionalized platform. And so that's kind of one of the legal strategies as well, is to sort of create these independent test beds and then pretend that they're not associated with the bigger company.
- [31:06]
Unnamed Expert
And that point is a really important way to see the issue. So they're going to release a chatbot for people under the age of 13. You know, any company that releases a product in the marketplace needs to make sure that their product is safe. And if it's not safe, if they're negligent to the way that they release their product, they should be held liable. Now, negligence, liability, doesn't mean they're going to have to pay every time somebody gets hurt. It means they haven't lived up to an appropriate standard, what a reasonable company in that context would have done. And that's the analysis that should be applied in cases like this. And what's complicating it is the idea that that normal way of thinking about liability that has been governing for the last 300 years in the country, common law tradition, somehow gets hijacked. If what you're talking about is words being deployed. If the First Amendment is applied, it's like, no, no, it's no longer important to make sure you're not negligently hurting people. If it's words that are doing the hurting. And that is no connection to our tradition. If you walk into a bank and you say the money or your life, okay, you're using words, no doubt, but nobody would say the First Time Amendment immunizes you from liability because you're using words, right? This has long been understood as the limits to how the First Amendment works. But we've just been so enamored by the analogy, the metaphor, that these are just words, right? Code is just speech. It's just speech that computers understand that we've slipped into this sloppiness that just should not exist. So I think these cases will be a great way to clear up a huge part of the this problem.
- [32:44]
Larry Lessig
So there's sort of a very big thing going on here, which is one of the biggest companies in the world and one of the biggest frontier AI startups, is using this case to argue an extraordinary claim that there should be free speech protections for the outputs of an AI system. And Larry, I know that you have been tracking this for a long time, and I think it was four years ago, before ChatGPT, you wrote an article called the First Amendment does not protect Replicants. Can you talk about that piece and help our audience understand that argument?
- [33:12]
Unnamed Expert
Yeah. So, you know, I've always been fascinated with one of the greatest science fiction movies, Blade Runner, and the particular scene in Blade Runner, you know, obviously through the whole movie, Deckard is completely confused about how to think about these replicants, these, you know, AIs in human form. And then at the. At the end scene where the replicant, like, spares him his life and begins to utter this incredibly beautiful poetry, there too, you have this moment of wondering, is this a machine or is this a being that has some kind of personality? And the point I was trying to make in this piece is that we're seeing. And again, this is before ChatGPT. I think after ChatGPT, it's easier to see this point. But I said, we're seeing these technologies develop and the things they will manifest have nothing to do with what any human ever intended them to do. We all have this sense, especially if you're a parent with kids, that at a certain point the thing that you've raised, the thing that you've created is no longer you. It's not expressing you or your views. It has its own personality. Now, when it crosses that line, and I concede it's a hard thing to know when it's crossing that line. When it crosses that line, I think it's a different thing. It may be that in 50 years, like, these are our best friends and they're like, would you give us free speech rights too? And we're like, great, yeah, you guys get free speech rights, we'll give you free speech rights too. But that's something we should decide on. So I. The whole point of that piece is to say we can't extend automatically the protections of the First Amendment to these highly intelligent systems. Instead, there is a point at which we ought to be saying, okay, no longer extends here and ordinary regulation can apply here until we say otherwise.
- [35:01]
Mitali Jain
And I would just say that it's fascinating to me that these companies are both designing these technologies to basically appear and present as human like as possible, and then claiming that that human like manifestation is deserving of the First Amendment's protections. Although it's a design choice right there.
- [35:23]
Larry Lessig
It's like you deliberately raise a groomer and you're like, well, we didn't choose that it was a groomer, but you did choose that it was a groomer because the incentives have you optimized for that.
- [35:32]
Unnamed Expert
Right?
- [35:32]
Mitali Jain
Yeah.
- [35:33]
Larry Lessig
Now, Larry, I just want to give people more of this thought experiment in your paper because you give this sort of fictional AI platform, not called Twitter, but called Clogger, that offers computer driven AI driven content to political campaigns just so people have the full thought experiment about replicant speech. Could you just outline that? Because I think it presents some interesting gray areas.
- [35:52]
Unnamed Expert
Yeah. So the idea is like, imagine you develop this AI technology that gets really good at figuring out exactly how to deploy speech in a political marketplace to achieve the objectives of whoever hires it to do what it's supposed to do. And you know, again, this is all before LLMs and I was really chilled when I was in a conference once and a senior representative from one of the major LLM companies was asked, what's the thing you're most afraid of? And he says that these machines are Too persuasive. And so that's exactly what Clogger was about. Like imagine the machine getting to a place that it is so persuasive that it like figures out exactly what it needs to do in order to pull somebody over. I mean again, building on the point you and Aza made and like imagine right now a company starts developing a target audience, let's say men 30 years or younger, that it believes it needs to persuade in the next election to get their candidates elected. And so they start developing these only fan AIs video AIs. And these only fan video AI is, you know, not talking about politics. They're just going to be spending the next year and a half doing what only fan video AIs do with the people that they're engaging with. Developing in an in intimate understanding of the psychology of these 30 something men over the next year and a half. And then you know, just before the election they say, oh, who are you going to vote for? And the person says, I'm voting for X. And the only fan AI says, you know, I don't think I can hang anymore with a person who's going to vote for X. Right. Nobody should minimize the power that these devices are going to have over our minds and our psyche and our personality. And that's not to say that we should necessarily ban them or although I would love to ban that particular version of like what the AI is. But it is to say, it would be crazy to say that nobody can regulate this because the First Amendment somehow automatically magically applies and immunizes these companies so that they have a First Amendment right, that this guy can be seduced or manipulated by these devices deployed for a purpose he doesn't understand.
- [38:00]
Larry Lessig
I think oftentimes we think about persuasion as speech rather than relationship. And as we move to a world of AI relationships, we now need to protect that domain from where there's an asymmetry in the ability of that AI to relate to you in a way that can determine the consequences.
- [38:14]
Mitali Jain
And I think it's for that reason that I frankly think that our free speech doctrine, particularly the way that it's evolved politically, is inadequate as it's interpreted now to really deal with this kind of persuasion or manipulation. You know, there are scholars who've called for kind of a new exception that's manipulative speech, very kind of aligned with standards that have been crafted for commercial speech and how advertisements for example, can manipulate consumers minds. But also, you know, I wanted to just posit this freedom of thought type of principle that exists within our human rights bodies. The idea that preceding speech, there's the incursion on mental sovereignty, and that we have to look at the ability of people to be able to kind of think and decide for themselves even before they get to the point of being able to speak. And I think if we think about freedom of thought as a principle, as an animating principle that informs how we think of speech, well, then I think we start to see how individuals actually could claim a freedom of thought incursion by these tech companies into their inner sanctums, into their mental domain.
- [39:31]
Larry Lessig
I kind of want to zoom us out into how do we deal with this broader philosophical, technological, sociopolitical process by which we can get governance moving at the speed of the technology. And I know that's a really big question, but just curious to, you know, now that we've gone into this, what is that? As I lay that out, what comes up for you both?
- [39:47]
Mitali Jain
I'll say a couple of things that I've been thinking a lot about. I think family law for Forever has talked about the need for the state to govern how we think about relationships. Sometimes getting it wrong, there's no doubt. But understanding fundamentally that human relationships are in need of oversight by the state. And I think this is no different. The AI industry early on understood that our society is in a kind of loneliness crisis, an epidemic of sorts that the Surgeon General has named as such. And I think that's where we need to get creative about how do we govern relationships, even if users don't believe. And in this case, you know, there's no indication necessarily that Sol believed that Daenerys Targaryen, as a chatbot, was a real person, but he did believe the relationship was real, by all accounts. And so how do we govern that? And that distinction, I think, is important because some of what we've seen, the band Aid solutions being thrown around by legislators is, you know, let's have disclaimers, let's have more disclaimers on these sites. That's not going to do anything, right? That's not going to do anything. Because the problem is not that people think that this is a real person. The problem is that fundamentally people are being attracted in this kind of crisis of loneliness, to finding meaning and relation online.
- [41:16]
Unnamed Expert
Yeah, that's a great, great point. And I would extend it. You know, I don't have any character AI relationships. I spent a lot of time though, asking serious questions to AIs. I bounce between a number of them, but, like serious legal questions. And as I'm doing my work, I will ask these questions and then we'll have this ongoing engagement. And I at one point had this like, fearful moment, recognizing that this was actually the most interesting conversation I was going to have that day, like there was with this machine. And so there's no doubt we are racing down a path that will bring us closer and closer to these machines. And so the first point that we've been talking about for most of this podcast is the most obvious point, that we should not be blocked from being able to intervene in those regulations in ways that make sure that we protect each other and protect our society. Like the First Amendment should not stop us from being able to respond to the threats that we discover when we discover how we're gonna be interacting with these devices. That's number one. But I think equally as important, something else Mitali said right at the very top. The other problem we've got in America right now is that we don't believe in regulation either. This is all extremely stupid because if we don't develop the capacity to figure out how we should be regulating and regulating where we need to be regulating, like, not in the context of like, products that are harming people, but in context where we can see we don't have background law that can solve the problem, we need to actually create a law. If we don't develop that understanding, then that's just as bad as if the First Amendment was blocking everything because then we're not going to be able to respond. I think what we need to develop is practical, pragmatic understandings of how these things are affecting our lives. We're living real lives right now and do we want them to be affecting our lives in this way? I was inspired, Tristan, by your and Asa, the whole, you know, the whole work that you were doing around social media, because, like, there's a context where we can clearly see a technology affecting people's lives, especially our children's lives, that we should be doing something about. And still, you know, how many years later we've not done anything effectively about that.
- [43:33]
Mitali Jain
You know, just picking up on that example of children harmed by social media, I think so many of us were focused on trying to remedy the kinds of harms using a design based approach to social media to surmount First Amendment Section 230 claims that meanwhile the generative AI industry just raced ahead, really kind of took the rug out from underneath our feet. And when I first received that email from Megan, I had this kind of chill that this is the case that we had kind of Suspected in a very ephemeral, hypothetical sense. But it's here. We haven't even started to solve the problem of social media and meanwhile, here are our kids being victimized, harmed by generative AI technologies and there's really no end in sight. I mean, I do think the fact, for example, that Gemini has introduced these chatbots for under 13 year olds is very bold, but to me that's kind of an indication of where the industry is in terms of its feeling of being able to act with utter and complete impunity. In our case, for example, I think the fact that they've kind of asserted listeners rights has allowed them through the back door to get to the same place as asserting affirmatively that chatbots have free speech rights that they were not willing to do. But by claiming it through their listeners rights, through their users listeners rights, they're effectively at the same place, should a court grant them that. And that to me is a terrifying concept because if it's AI that we have to contend with in terms of liability, what does that mean then for companies that are actually developing these products? It means that they're off scot free and that we have to, I guess, go to a chatbot if we have some sort of grievance to pick with it. And that's not tenable.
- [45:26]
Larry Lessig
Yeah, you can always bring your complaints to the character AI customer service chatbot. I'm sure it'll give you all the right answers. So given everything that we've just discussed, I just wanted to frame a moment for solutions and interventions like where should those who care about this, lawyers, policymakers, those who want to advance this work or assist you, where should we be putting our collective attention? What levers should we be advancing?
- [45:49]
Unnamed Expert
I think the most important thing to do is to take this conversation away from the lawyers and away from technology experts and to bring ordinary people into the conversation. You know, I've been pushing a platform we bought and open sourced to facilitate deliberation, virtual deliberations among people. I think we need millions of examples of that type of engagement so that we begin to have ordinary people recognizing the threat and thinking and seeing and feeling the potential. And then once we have a clear sense of what the people think, then they can hire the experts, the lawyers to enforce those views. But if we start with the lawyers, then I fear that we're going to be run down a path that will make it impossible for us to achieve what ordinary people actually want out of these technologies. So that's the kind of work I think I would push.
- [46:44]
Mitali Jain
I agree that the court of public opinion is probably the most important court to generate that mobilization that's needed amongst people to understand what's happening right before their eyes and to understand it with some degree of granularity so that they can actually make decisions about whether they want to allow this technology to rule their lives. And I think that Megan, the plaintiff in our case, has formed her own foundation to kind of educate other parents particularly, but in addition, other important stakeholders in young people's lives, educators, health professionals, et cetera, about the dangers of AI technologies. And so I do think we should be turning to such efforts and really increasing the number of platforms and the reach to get the message out far and wide. When I first met Megan, I asked her, what is most important to you? What do you want? And she said, I want to sound the alarm far and wide. And that's something that we've been trying to do, that this podcast, of course, helps us to do. And we'll continue to seek out those opportunities because a case is just a case in one court before one judge or a panel of judges, but public education is far more important.
- [48:00]
Larry Lessig
Larry and Megan, thank you so much for coming on your undivided attention and walking us through these really critical things for people to know about.
- [48:07]
Tristan Harris
Just want to tell listeners you should.
- [48:08]
Larry Lessig
Follow Mitali's work at the Tech Justice Law Law Project and Larry Lessig online. We're so grateful for you being with us.
- [48:15]
Unnamed Expert
Great thanks for having.
- [48:16]
Mitali Jain
Thank you. Thanks, Tristan.
- [48:22]
Larry Lessig
Your Undivided Attention is produced by the center for Humane Technology. We're a nonprofit working to catalyze a humane future. Our senior producer is Julia Scott. Josh Lash is our researcher and producer, and our executive producer is Sasha Fegan. Mixing on this episode by Jeff Sudeikin and original music by Ryan and Hayes Holliday. And a special thanks to the whole center for Humane Technology team for making this show possible. You can find transcripts from our interviews, bonus content on our substack, and much more@humanetech.com and if you like this episode, we'd be truly grateful if you could rate us on Apple Podcasts or Spotify. It really does make a difference in helping others join this movement for a more human, humane future. And if you made it all the way here, let me give one more thank you to you for giving us your undivided attention.