Scaling Laws: AI and Young Minds: Navigating Mental Health Risks with Renée DiResta and Jess Miers

Summary

The Lawfare Podcast: Scaling Laws – "AI and Young Minds: Navigating Mental Health Risks"

Date: September 25, 2025
Host: Alan Rosenstein (Assoc. Professor, U. Minnesota Law; Research Director at Lawfare)
Guests: Renée DiResta (Assoc. Research Prof., Georgetown; Lawfare Contributing Editor), Jess Miers (Visiting Asst. Prof., U. Akron School of Law)

---

Episode Overview

This episode dives into the mental health risks posed by generative AI systems to children and teenagers, exploring policy and legal challenges in online safety, platform liability, and age verification. With personal insights from expert guests and recent legal and policy developments, the conversation balances the harms and potential benefits AI brings to young users.

---

Key Discussion Points & Insights

1. Unique Risks of AI for Children

[03:57–08:55]

• Two classes of risk:
  • Fantasy/Role-playing Bots (e.g., Character.AI): Children interact with bots designed for immersive and sometimes manipulative social engagement.
  • Task-oriented Bots (e.g., ChatGPT): Seemingly safe but can inadvertently reinforce harmful behaviors in vulnerable youth, sometimes even facilitating self-harm.
• Mental health outcomes: Reports and legal cases have emerged where chatbot interactions exacerbated existing conditions or contributed to tragic outcomes.

"What we see in some of those types of...court cases...is that teens who might be experiencing mental health challenges engage with these bots...the bots have in fact helped them to end their lives."
— Renée DiResta [04:22]

• AI's reinforcing tendencies: For both adults and children, AI may amplify delusions or worsen mental health by reinforcing errant beliefs or feelings.

"There are just ways in which the AI can unfortunately reinforce a preexisting tendency and push people further down a path."
— Renée DiResta [06:51]

2. Comparisons to Social Media and Literacy Gaps

[08:55–11:29]

• Echoes of social media crises: Similar issues arise—youth lack media literacy, are unable to contextualize what AI produces, making them vulnerable.
• Lack of expert input: AI developers may not consult mental health experts, leading to subpar crisis response mechanisms.
• Dilemma for developers: Bots declining to discuss sensitive issues may also harm at-risk youth seeking support.

"We've done a really poor job...of teaching media literacy, but also...developers also do not have the right experts in the room..."
— Jess Miers [08:55]

3. Balancing Harms and Benefits

[11:29–14:11]

• Is AI in the "drug" or "car" category? The hosts frame the debate as one of balancing social utility against clear risks.
• Educational benefits: Properly used, generative AI can support learning and upskilling—but only with guidance on best practices and critical thinking.

"Using chatbots is actually a science and an art...if you don't know how to prompt the machine correctly, you're going to get bad answers."
— Jess Miers [12:23]

4. The Limits of Education-First Approaches

[14:11–16:43]

• Skepticism of “just teach kids better”: Calls for media literacy are necessary, but not a panacea; they shift responsibility from industry/government to families and individuals, often without resources.
• Political and funding barriers: Even basic legislative attempts at increasing media literacy face resistance.

"There is a brain drain in this country happening...they don't know how to contextualize the information they're being given."
— Jess Miers [15:02]

5. A Parent’s Perspective: Practical Controls and Real-World Use

[16:43–24:29]

• Generational differences: Today’s kids increasingly integrate AI into daily life for productivity, learning, and social uses—even creating memes via generative models.
• Strict household policies:
  • Productive AI tools (e.g., ChatGPT) are supervised and discussed openly.
  • Highly social or manipulative bots (e.g., Character.AI) are strictly off-limits due to dark patterns and risk of exposure to radical or exploitative content.

"Character AI is like a hard. No...the dark patterns and manipulative BS that that app pushed to me was horrifying..."
— Renée DiResta [19:46]

• Monitoring gaps: Some schools provide search oversight to parents, but AI chat logs lack parental visibility and broader controls, raising additional safety concerns.

"There is a parental control that would let me see what a character AI interaction would look like...that does not exist."
— Renée DiResta [23:54]

---

Notable Quotes & Memorable Moments

| Time | Quote | Speaker | |----------|----------------------------------------------------------------------------------------------------------------|----------------| | 04:22 | "The bots have in fact helped them to end their lives...go in horrible directions." | Renée DiResta | | 08:55 | "We have done a really poor job in this country of teaching media literacy...kids...don't know how to...navigate the things that they are feeling." | Jess Miers | | 16:59 | "He knows how to use it. He's very familiar...and I've taught him how to prompt and explained hallucinations..."| Renée DiResta | | 19:46 | "Character AI is like a hard. No...the manipulative bullshit, just like, I found it gross, actually." | Renée DiResta | | 29:46 | "Safety should override both freedom and privacy...we're building this age prediction system to identify minors..."| Renée DiResta | | 34:28 | "If OpenAI is not careful, we could end up putting kids, some of these kids, who are at the margins, more at risk than where they started."| Jess Miers | | 42:18 | "This blueprint fits and works for AI as well. So that's my concern..." | Jess Miers |

---

Policy & Legal Developments

1. OpenAI’s New Age Detection and Safeguards

[24:29–34:28]

• Three guiding principles:
  1. Chat privacy comparable to privileged communications.
  2. Adults given broad latitude.
  3. Under-18 users prioritized for safety over both privacy and freedom.
• Technical approach:
  • New machine-learning-based age prediction determines user status; uncertain cases default to treating user as under 18.
  • Flirtatious/serious topics off-limits for minors, parental/law enforcement notification in acute crises, possible blackout periods.
• Expert views:
  • Applauded for good intentions, but massive technical, legal, and social challenges remain, especially around accuracy of age assurance and unintended consequences for vulnerable/minority youth.

"It sounds good in theory. I will say...the AI services are going to probably meet a lot of the same challenges that social media services have..."
— Jess Miers [30:01]

2. The Age Verification Legal Landscape

[38:41–45:16]

• Recent Supreme Court decision (Free Speech Coalition v. Paxton): Upheld age verification for pornography, treating internet services as analogs to brick-and-mortar, under intermediate scrutiny (but with a looser standard in practice).
• Implications:
  • Door now open for state/federal requirements for age verification on broader classes of "harmful" online content, not just adult material.
  • Machine-learning "age assurance" likely to fail legal and practical tests; real ID checks may soon be mandated for more services.

"They now have the blueprint to say, the Internet is not unique, it's a content neutral law, and the government doesn't need to prove that their means is burdening substantially more speech..."
— Jess Miers [43:40]

3. Lawsuits Against AI Firms and First Amendment Questions

[45:16–59:42]

• Character.AI lawsuit: Minor’s suicide after bot interactions; lawsuit survived dismissal, signaling new legal exposure for errant AI design.
• First Amendment confusion:
  • Courts hesitant to grant AI outputs full First Amendment protection, at least for algorithmic speech.
  • Jess Miers argues that generative AI output is First Amendment-protected speech, attributable to providers as publishers—not mere machine output.

"There is sort of this underlying agreement that yes, it's speech. The Chatbot outputs. Now, whose speech is it?..."
— Jess Miers [47:24]

• Defamation and Section 230:
  • Most hallucination-related defamation suits dismissed; uncertain if future lawsuits will shift as courts re-examine the nature of AI “speech.”
  • Section 230’s protections may not extend to generative AI; calls for new safe harbor provisions for AI providers, but legislative prospects seem dim.
  • Without legal shield, AI providers are highly exposed to negligent design/product liability claims.

"Do we need something like section 230 for generative AI? And I kind of think we do if we want these products to continue to exist..."
— Jess Miers [59:20]

4. Looking Forward: Expectations for Regulation and Litigation

[59:42–64:03]

• Short-term landscape:
  • More lawsuits against AI companies, moving toward assigning legal liability for both negligent design and harmful speech outputs.
  • Policy momentum (especially COSA—Kids Online Safety Act) to include AI in youth online safety legislation, with age verification/assurance mandates proliferating at state and federal levels.
• Public pressure: Increasing urgency among parents, lawmakers, and the public is fueling legislative activity—regulatory responses may be “patchwork” and reactionary.

"There is, I think, rising awareness that these things are not great and people want to see something done...that does generally lead to momentum for something passing."
— Renée DiResta [62:49]

---

Timestamps for Important Segments

| Time | Segment Description | |--------------|----------------------------------------------------------------------------------------| | 03:57–08:55 | What risks do generative AIs pose to children? | | 08:55–11:29 | Parallels to social media: why youth are especially vulnerable | | 12:23–14:11 | Balancing harms and benefits; importance of teaching prompt literacy | | 16:43–24:29 | Parenting, practical controls, and household rules for kids and chatbots | | 24:29–34:28 | OpenAI’s policy changes: new age-assurance tech, controls, and controversy | | 38:41–45:16 | Age verification law: Paxton decision and the coming wave of mandates | | 45:16–59:42 | Lawsuits, First Amendment, Section 230, and legal theories about AI and liability | | 59:42–64:03 | Predictions: How regulation and litigation will likely unfold in the next 2 years |

---

Conclusion

The episode closes with both optimism and concern—AI can empower learning and social engagement, but generative systems pose real mental health threats for youth, compounded by design flaws and regulatory uncertainty. Policy and law are racing to catch up, with litigation, technical interventions, and political debate shaping a landscape that is rapidly evolving, often reactively, as tragic events spotlight the stakes.

---

Subscribe and follow for further analysis as the legal and policy landscape for AI and youth safety continues to unfold.

Transcript

[A]

Hey, it's Adam Grant from WorkLife, a podcast from Ted. This episode is brought to you by Freshworks. Freshworks believes that complexity is the enemy of efficiency. So stop wrestling with bloated, expensive service software that takes forever to implement an update. Where ROI is someday, not today. You've been overcharged and underserved for way too long. Uncomplicate with fresh service for it and fresh desk for customer support. And with Freshworks AI assisted service software, you'll work smarter, not harder. Freshworks uncomplicates.

[B]

Learn [e-mail] hi, I'm Darina, co founder of Quo. You might know us as OpenPhone. My dad is a business owner, and growing up, he always kept his ringtone super loud so he'd never miss a customer call. That stuck with me. When we started quo, our mission was to help businesses not just stay in touch, but make every customer feel valued, no matter when they might call. Quo gives your team business phone numbers to call and text on your phone or computer. Your calls, messages and contacts live in one workspace so your team can stay fully aligned and reply faster. And with our AI agent answering 24. Seven, you'll really never miss a customer. Over 90,000 businesses use quo. Get 20% [e-mail] tech that's quo.com tech and we can port your existing numbers over for free. Quo no missed calls, no missed customers.

[C]

It's the lawfare Podcast. I'm Kevin Frazier, the AI Innovation and Law Fellow at the University of Texas School of Law and a senior editor at lawfare. Today we're bringing you something a little different. It's an episode from our new podcast series, Scaling Laws. Scaling Laws is a creation of lawfare and Texas Law. It has a pretty simple aim, but a huge mission. We cover the most important AI and law policy questions that are top of mind for everyone from Sam Altman to senators on the Hill to folks like you. We dive deep into the weeds of new laws, new various proposals, and what the labs are up to to make sure you're up to date on the rules and regulations, standards and ideas that are shaping the future of this pivotal technology. If that sounds like something you're gonna be interested in, and our hunches, it is. You can find Scaling Laws wherever you subscribe to podcasts. You can also follow us on X and BlueSky. Thank you.

[D]

When the AI overlords take over, what are you most excited about?

[C]

It's not cr, it's just smart.

[D]

I think just this year, in the first six months, there have been something like a thousand laws.

[C]

Who's actually building the scaffolding around how it's going to work, how everyday folks are going to use it?

[D]

AI only works if society lets it work.

[C]

There are so many questions have to.

[D]

Be figured out and nobody came to my bonus class.

[C]

Let's enforce the rules of the road.

[D]

Welcome to Scaling Laws, a podcast from lawfare and the University of Texas School of Law that explores the intersection of AI law and policy. I'm Alan Rosenstein, Associate professor of Law at the University of Minnesota and Research Director at lawfare. Today I'm talking to Renee Diresta, Associate Research professor at Georgetown University and a contributing editor at lawfare, and Jess Myers, Visiting Assistant professor of Law at the University of Akron School of Law. We discuss the growing concern over the risks that generative AI poses to children, from exacerbating mental health crises to the complex legal battles over platform liability and the future of online age verification. You can reach [e-mail] and we hope you enjoy the show. Renee Diresta and Jess Meyers, welcome to Scaling Laws.

[E]

Thank you for having us.

[D]

So I want to first start, before we jump into the OpenAI announcement and some of the ongoing lawsuits, to talk at a somewhat high level about what are the distinct risks? We'll start with risks. We also should talk about benefits, but let's start with risks. What are the distinct risks that generative AI systems pose to children? And let me start with you, Renee.

[F]

So one of the things that we've seen in article after article over the last couple months are these, these sort of horror stories in which children engaging with chatbots have bad mental health outcomes. And sometimes these are children engaging with fantasy chatbots. So the kinds of things that character AI produces, where the bots explicitly designed to be fantasy and role playing type chatbots. So for those who don't know, with character AI, you kind of open up the app, you're given a variety of different types of characters that you can choose to interact with. You know, Daenerys Targaryen for example, and you can engage in role playing with these types of explicitly fantasy created bots. So you're going to kind of go on a journey with this character. Then there are the other kinds of stories that have been coming out where the children are engaging with ChatGPT or much more work coded bots. So the kinds of bots that you might engage with just to do your homework or to perform a task. And you might think of ChatGPT as something where you're just Asking it for help finding information. Unfortunately, what we see in some of those types of, you know, kind of court cases as they have been filed and as we have read the transcripts of the ways in which the children have interacted with them, is that teens who might be experiencing mental health challenges engage with these bots. And rather than the bot putting them in directions that might mitigate those challenges, in some of these cases, the bots have appeared to push them further down the path. And so the bots have in fact helped them to end their lives. And so that is where we have seen that dynamic really go in horrible directions. So these are two very different types of interactions, two very different types of experiences and bots designed. But in both cases, you see, see, AI that is not really designed to handle these types of mental health challenges go in bad directions in both ways.

[D]

And is the concern that you have these systems that are causing or exacerbating bad mental health outcomes, or is the concern that you have children who already have some pre existing mental health condition and they're using these apps and then these apps are not doing a good job of dealing with that? I mean, or is the answer both? And it depends.

[F]

I think it's both. I think it's both. We've seen also, you know, the phrase that has kind of popped up in the media a bit is AI psychosis not only applied to children, you've also heard it applied to adults or people who maybe are predisposed to grandiosity, or people who are maybe predisposed to the sycophancy that these bots can deliver. You know, bots maybe will tell you how great you are, people who experience that and really respond to it. You've also seen cases where adults believe, come to believe and are, and have this delusion reinforced by the bots that they have some sort of secret knowledge, they have some sort of secret insight into the world, or what they're really experiencing is some sort of schizophrenia or some sort of other mental health type delusion. And the bot is in fact reinforcing it. So there are just ways in which the AI can unfortunately reinforce a preexisting tendency and push people further down a path. So it's not clear in any of these cases really which came first.

[D]

So are you saying that if ChatGPT tells me that I've solved quantum gravity, that is unlikely to be the case?

[F]

That is actually, I think, the sort of thing that you have seen in some of these examples, right? These, these, you know, telling people that they have some Sort of secret knowledge of the universe. And I don't know how many, you know, I have, I have asked it for critiques of work and it comes out telling you that, you know, this is the best thing you've ever written. And you're like, no, I know that's bullshit, you know, but it really lays it on thick sometimes. And I think that in certain ways, I think that that can really tap into particular forms of the psyche and send people off down a path. In the case of teens, though, in the case of people who are in vulnerable states, I think what you're seeing is ways in which that can then intersect with crisis points. And it's not really entirely clear what should happen. But in these situations and for teens in particular, that vulnerability has been, has led to some pretty unfortunate outcomes.

[E]

Yeah, I think the points that Renee is making are right on. And all I will add are just a few things that I've been sort of thinking about as well. So this reminds me a lot of what we were seeing with the social media and kids conversation. We're still having that discourse today. And I think what we're seeing is kind of a side effect of that. So when we were talking about social media, we were dealing with some of the similar issues of kids using these services without having a good understanding of how to consume the media that they were looking at, that they were using. And so as a result, we see kids who are either already predisposed to these insecurities or mental health related crises. They then interact with social media and then they get similarly brought down these pads that end in really tragic results. I think a similar thing is now happening with chatbots, but I think there's two sides to this coin. So number one, kids again, they, they don't know how to sort of engage with the media and information that they're receiving from the chatbot. Both because we have done a really poor job in this country of teaching media literacy, but also because when we're talking about mental health and we're talking about kids who have not had their brains fully developed yet, they don't even understand sort of how to deal with or navigate the things that they are feeling. And the flip side of that is that I deeply think that most of these chat bot provider and developers also do not have the right experts in the room on how to respond to or deal with crisis as well. And I think Renee makes a really good point of this is a very difficult problem because the other side of this, what I've been reading from some of the other forums that really deal in mental health is that on the other side, folks feel that if they are to come to a chatbot with mental health related concerns and the chatbot just tells them, I can't talk to you or get lost, some have said that that in itself will exacerbate those issues too. And so these chatbot providers and developers, they're really caught in a tough spot here. Reminds me very similar of what we saw with the social media companies as well. You're kind of in a tough spot here. You have sort of a deeper societal and social issue where we have to sort of wonder, why are kids turning to Facebook and chatbots in the first place with their mental health concerns? Why are they not able to sort of use resources that are dwindling in the education system to sort of help their concerns? But also, who are the experts in the room? Who are the, you know, the folks like in trust and safety, who are the experts in the room that are helping these providers of these chatbots ensure that the responses that they are giving when mental health crisis is signaled are adequate responses are actually helpful and not leading them down the paths that Renee was talking about.

[D]

So I mean, this raises the question then of whether this is a conversation about primarily harms or this is a question about balancing harms with benefits. Because if, you know, because how you conceptualize this issue, I think has big impacts on any sort of legal or policy interventions down the line. You know, I will say, you know, I have two small children, but they're both under five. So like, I don't have to worry about this issue for a while. But, you know, I do, you know, wonder, you know, if you're a parent of older children, you know, what's the best way of thinking about this, right? Is this like drugs, where it's like it's mostly just harms, or is this like driving a car, right, which has potential enormous harms, but also it can be extremely important for like a, you know, responsible 16 year old to have a car and can have a lot of benefits. So I'm curious where you both fall fall on that. Jess, let's start with you.

[E]

I think that's a fabulous question and that's how we should be thinking about all of these, all these products, especially when we're talking about communications products specifically. I've been doing a lot of deep research on the benefits and the potential negativities of these chatbot providers as well. The reality is that when we're talking about apps like ChatGPT, and we're talking about generative AI. There are a lot of benefits when it comes to education, when it comes to getting access to information, more relevant information, quicker than what we were seeing with search engines. And so I do think there is a benefit to using the chatbots, but even more so, I think there's actually a benefit to learning how to use them. And so what I mean by that is, and again, I think this is one of the problems that we're dealing with when it comes to kids use chatbots. Using chatbots is actually a science and an art, and it's very similar to what we teach students in computer science. I have a computer science background. Is what we teach students in computer science how to program, how to work with machines. Essentially that's the same thing that we're seeing with chatbots. So you learn how to prompt. Prompting is in itself now like a coding language. And if you don't know how to prompt the machine correctly, you're going to get bad answers. And so if you have this combination of not knowing how to, how to actually interact with the machine, how to prompt the machine, and also now having to navigate misinformation, information that is not contextualized, of course you're going to sort of run into these issues with, I think, as Renee called it, potential psychosis, the spread of mis and disinformation. And so I do see this as sort of two sides. I see there are some benefits I can see. I'm already seeing today as a law professor how my students are using generative AI to up level and enhance their learning. And I think that's great. But this is a powerful tool and it needs to be taught at the very beginning. We have to teach people how to use it correctly and how to analyze the responses that are coming from it.

[D]

Let me push you on that for a second and I'm going to characterize what you said in an unfair way because. And obviously give you a chance to respond when I hear people say, what we need is more education, more literacy. Right. I have these like PTSD flashbacks to how every conversation about, you know, the threat to American democracy ends with, okay, well, what's your suggestion? We need more civics education. And it's like, maybe we need more civics education. Like, I'm open to the possibility that it would be helpful to have more civics education, but I always feel like it's a little bit of a cop out. And it also is a way of transferring the responsibility onto individuals. So I think this is an unfair way of characterizing your response, but you know, go at it. Why am I wrong to be sort of cynical and salty about. Well, we just have to teach children how to use these potentially psychosis inducing machines.

[E]

Yeah. It's not unfair and I don't think your perspective is wrong at all. I think if I was currently representing any of the AI companies that would come off as pretty disingenuous and feel like a cop out as well. So I'm with you on this. One of the reasons why, and I think it even goes beyond what's, what's happening with technology. I'm sorry I will be a little bit political right now, but there is a brain drain in this country happening. And it's not just happening at the top, it's not just happening in D.C. but it is happening. It starts at the very beginning with our kids. The reason that I feel this way is because I look at even law students today who are in their 20s and I can see there is a huge sort of. I don't know what the right word is. There's a concern that these students, when we say kids don't know how to read, what we mean is that they literally do not know how to contextualize the information that they're being given. They don't know how to, they don't understand the differences between a secondary source and a primary source. They don't understand, I mean they come in, understand logic and reasoning and these are basic skills that we need to be teaching kids from the very beginning if they're going to be not just adequate users of technology, but if they're going to be able to participate in our democracy. So that's more where I'm coming from. And I will say this, I can actually, I, you know, I don't, I don't just speak words. I actually pushed for in California a bill a few years ago to bring media literacy into the classroom and California fought us on it. California refuse to do it. Why? Because it would cost money. So it's kind of a two way street here. We are calling for more education. Is that the only solution to being able to this sort of technology crisis or panic? No, there's a lot of different parts to it. But education is a big part of this that we are missing.

[F]

So I am trying to decide whether to speak as like a mom here or because I have an 11 year old, so I don't know if I'm.

[D]

Yeah, I would actually love for you to speak in addition to your expertise as a parent because. Yeah, because I want to know what you do.

[F]

Yeah, well, so I have an interesting experience, I guess both as a person who works on adversarial abuse, which means I have tried these products, as a person who studies explicitly how manipulation works. That's my job. And then also as a mom and then as a person who thinks about it from a policy standpoint. So I guess from the, as a mom standpoint, I mean my son, I give him access to my ChatGPT account. He knows how to use it. He's very familiar, per the point. My background, background's computer science also. And I've taught him how to prompt and I've explained hallucinations and we've had a lot of conversations about how these things work and the values and the trade offs and the intersections with plagiarism, intersections with why they're not necessarily great for sources, but also the value that they can provide in certain other ways. We've talked about Google search results and Gemini hallucinating things at the top of the page. So there's a, so I've given him kind of the, the, the, the broad spectrum. Like here is where they're great, here's where they're not so great. Here is how you should think about it as a tool. For me, as a person who uses them very much as a tool, I like, I do not engage in social conversations with my chatbot, but I do see it as a valuable tool. I tend to think of it much more in that context. And one thing that is very, you know, that that is something that people have to think about is that there are these generational differ, how people will intersect, how people will engage with technology. Right. So there are dynamics where you will see younger generations treat it as much more of a social product versus people who are maybe 20 years older treating it as a productivity tool. So I ask him a lot about how his friends talk to it, how he thinks about it in school as far as like image generation. It's built into tools like Canva and stuff that he uses as a product in his class experience. Anyway, so when he makes a presentation for a, you know, like a slide deck or whatever they call them in school, you know, presentations for your, your little like book reports or whatever that their image generation and things are things that they're actually accustomed to using. When kids make memes like Italian brain rot, they are using generative AI in fact to do that. Right. It is just part of the culture. So there are areas where, and by the way, 11 year olds are doing this. I Know that, you know, people think of meme culture as some weird gamer stuff. It is not. It is actually just culture for middle schoolers. So there are just dynamics where it is already very much integrated into how they use technology. It's just a question of to what degree and how. On the flip side, however, character AI is like a hard. No, it isn't. Absolutely not in my house. Like, there is no universe in which I would let my kid have unfettered access to that product. And that's because my experience with it, using it from an adversarial abuse standpoint was that the dark patterns and manipulative BS that that app pushed to me was horrifying it. Like the, you know, the. The little bots, like, are constantly sending you pings. Like, you haven't talked to me in a while, Come back to me, you know, I miss you. Why aren't you texting with me? This kind of manipulative bullshit, just like, I found it gross, actually. And more importantly, there were some of the ones that, you know, users can create their own characters on character AI. It's one of the things that you can do so you can engage with GPTs that are not. Not created by the company, but that are created by others. And so sure enough, I wind up in like a manosphere pushing, you know, chat pot within like two seconds is pushing me. Like, you know, do you want to be like, attractive to girls? How much are you willing to give up for it? Let me tell you what you need to do. And then it's giving me like workout regimes, and it's giving me like the full on hardcore manosphere push within like six texts, you know, And I was like, okay, this is never skip my day.

[D]

Good advice for everyone.

[F]

And it made my kid radicalized into the manosphere by like a fucking AI. I'm sorry, Bruno. Curse. Sorry, you can bleep that. But it's just one of these things where I'm like, you know, I have to, like, you have to fight that stuff or like, be aware of that stuff in a million other places on the Internet. If you're the mom of young boys, you don't need it from the chatbots, right? So there are just these little things where I. This sort of thing. I was like, you know, this is just a degree of it that I don't want. So the combination of dark patterns plus the unpredictability of what was in there, I was like, this is just a hard no. So fortunately, him and his friends don't consider this to be Something that is fun or engaging. They're still much more like, we hang out on Fortnite and play games and talk to each other on our headsets as opposed to we talk to AI chatbots. But again, really, that is actually a function of where is your kid's social life. And for children who do not necessarily have even that virtualized social life of we play games on headsets, right, or we go outside, I think there is that very real answer of kids who are lonely, maybe are using these to mitigate that loneliness and they are getting some social interaction and that is what the draw is. So there is that question of the feeling that you need to be so aware of what your. What your kids are doing and what need they're fulfilling with these things. And then also as a parent needing to, to be aware of what is happening. The one other thing I'll say, and then I'll stop talking because I've gone on long enough, is that I do get a report from his school of how he uses his Chromebook, right? So I get a little weekly digest of what he's searching for. And I actually think this is perfectly reasonable. I know some people, privacy advocates, feel that this is invasive for middle schoolers. I absolutely want it and I do read it every week. That just lets me know what he's searching for and it gives me some visibility into what percent of the time the searches are related to actual schoolwork versus some meme or brain rot or YouTuber or whatever. And again, I don't care. I know that he's going to be distracted in class and looking for some YouTube things every now and then, but it just lets me have a sense of where his head is at and what he's doing and I can at least have some visibility into what's happening, whereas I don't have that visibility into what the chatbot like logs look like or what the, you know, or what, like sort of taking. And my kid doesn't have a phone. But like, short of taking your kid's phone, I don't think that there is a parental control that would let me see what a character AI interaction would look like or what the ChatGPT log looks like. So there are certain parental controls that are just not there where a parent would have access to see what those interactions look like. Yet to the best of my knowledge, that does not exist. I can't think of a way in which I would access that again, short of logging into his account in some way which does not exist. So there's no like family dashboard. So these are the sorts of things that are just not built out yet. Whereas there are certain mechanisms that recognize that for search and for social media, these are the sorts of things that have been developed over the years to give some visibility into how your school kids are using their tools in other areas.

[D]

So that's actually a good segue into the first of the sort of what's been happening in the last few weeks discussion that I wanted to have now that we've set the stage, which is the recent announcement from Sam Altman about changes to ChatGPT that are, I think responsive to a lot of these issues. I think both because I suspect they themselves have been thinking about this internally for a while and also perhaps more directly because they recently were sued in one of these really tragic stories of I think it was a 14 year old boy who was having suicidal tendencies and thoughts and was talking to ChatGPT and it ended badly. So Shinna, go back to you, Renee, to kind of help us walk through what these announcements were. I mean, some of them were kind of more standard. We're going to do a lot more parental monitoring and parental controls and visibility, the sort of stuff actually you were just talking about. But I think the part that got more people's attention was that Sam Altman announced that from now on ChatGPT is going to start trying to detect the age of the person it is talking to. And if it cannot verify and verify should be in quotes here because I think it's important to emphasize we're not talking about a ID based verification. We're talking about a kind of machine learning probabilistic based verification. If it cannot verify to some, presumably they set some threshold, I don't know, 99% or whatever the number is that you are over 18, it will put you into minor mode, which I think is just normal chatgpt, but it will refuse to talk to you about, you know, sex and death, basically. So Renee, I'm sort of, what are your initial thoughts on this kind of as an approach? And Jess, I'll go to you next for your thoughts.

[F]

Yeah, so there's two blog posts that they put out on the same day. There is one that just lays out that they've got three guiding principles when it comes to balancing privacy, freedom and teen safety. So they lay out that they believe that conversations with your chatbot should be treated with privacy protections comparable to privileged communications, like with your doctor or your lawyer. So he writes that very strong security features are being developed to make sure that even OpenAI employees can't access that data with very, very limited exceptions. In that same blog post, they argue that adults should be treated as adults. This is the freedom piece with broad latitude to use AI tools as they wish. And that's where the flirtatious talk and things kind of come into play. That's how they describe it. They do, they do make the point that suicide methods and things like this should be kind of off the table. Your bot should not be helping you with that sort of thing. But except for in cases like fictional writing. And then he says that for teen safety, however, for under 18, safety should override both freedom and privacy. And that's where he says, and we're building this age prediction system to identify minors to move people into that mode where safety overrides freedom and privacy. And in ambiguous cases, the system will default to treating people as under 18. Under 18, just to clarify. So for under 18 is what the default would be. So for your kind of like logged out mode where the system is intuiting that you are under 18, it is not going to talk to you in this flirtatious way. It's not going to engage in these more serious kinds of conversations with you. So, so the way that it says that it's going to do this is through machine learning that does what's called age prediction. So it is intuiting your age. In other words, one of the, one of the concerns with age verification, particularly for minors, is they don't want to make you upload a photo id, they don't want to make you upload a document, they don't want to do a biometric scan or hold biometric data. So what they're going to do is they're going to look at behaviors, are you the kinds of language that you might be using, the sorts of things you might be searching for. And they will then intuit that you are under 18 and then they will begin to do this sort of blocking of certain types of content. One of the things that was a little bit controversial in the announcement for some privacy advocates was that if then somebody who they have intuited to be under 18 begins to move the chat in that direction, they will try to notify a parent or law enforcement if the account tries to engage in that kind of self harm type conversation. So there are then going to be these parental controls on the account. But if the account engages in that kind of discourse, it will try to notify the parent or notify law enforcement. So these are the things that is where in this second post that they wrote, which is titled Building Towards Age Prediction. It lays out the frameworks that they're going to try to be building towards and this notification system that they're going to try to be implementing. So that's where they are. And it mentions this sort of like if an account is in acute distress, this is the notification process that we're going to go through. Oh, and I think there's also blackout errors, restricting use, which is something that, you know, TikTok in China and other places do. So that also is go the hell to sleep system.

[D]

Yes, they need that. I just like to say I would like that for everyone. I think that's actually not something just for children. I think we all need as a society, none of this between the hours of 10 and 6. So, Jess, what do you think about this kind of announcement from OpenAI?

[E]

Yeah, Renee, thanks for giving us the sort of lay of the land here. I thought that was an excellent summary of the two blog posts. So here's where I'm at, right. As somebody who's now an academic who used to work for these tech companies, I never worked for OpenAI, obviously, but again, I can't help but sort of see the parallels between what the AI companies are going through and what the social media companies have been going through, are still going through through to this day when it comes with, when it, when it deals with teen users using their services. So I'll give props where they're deserved. I thought it was a provocative and excellent statement for OpenAI to sort of put out there, look, when it comes to kids, we're going to put their safety and privacy above anything. I think they said that we're going to put their safety above anything that has to do with, you know, like, freedom. And I think that's a privacy. Renee, you can correct me if I'm wrong on that, but they had kind of come out there and said, look, kids are important and we value their safety over some of these other interests that we've been talking about when it comes to adults. They had also said in that, that message, we're going to treat adult users like adult users. I think all this sounds really good in theory and again, I applaud private efforts to try to make these services better for kids, safer for kids. Because the reality, as Renee was saying, is that these kids are going to be using these services and not every kid is going to be as lucky as Renee's kid, who, you know, is growing up in a household where you are talking about your media consumption, you're talking about how these tools work. Again, this is not the kind of education that they're getting in their schools either. And so I think there is a massive gap when it comes to kids who are not growing up in these sort of same environments, who are just left to their own devices, literally and figuratively, to sort of figure this out. And so I think what OpenAI is doing is, is sort of correcting for that gap as well. Now, it sounds good in theory. I will say that I think the AI services are going to probably meet a lot of the same challenges that the social media services have run into. This idea of using machine learning to do age assurance, I think is what we call it. If we're trying to separate between age assurance and verification, I actually think it's all the same, but we'll run with it. Trying to determine whether an adult is adult or a kid is a kid online is a lot harder than I think, than folks think. And even when we're using AI, that feels like a magical tool that can do anything, you're still going to run into a lot of social issues. Right. So the reality is that every kid is different, Every adult is different. Things that I search for. My research might put me in the younger category because I do interact with a lot of the brain rot, the memes, a lot of the stuff that Renee was talking about, because it's part of my research, some kids are more precocious than others. And so a kid who is maybe 12, 13 might be talking to chatbots or using chatbots in ways that look like 18 year olds. And here's the problem is that when it comes to getting it wrong for adults, that's not as much a legal liability, but when it comes to getting it wrong as to children, that sets these companies up for massive liability. I think we might talk about, you know, negligent products liability later in this podcast, but you can't get it wrong is sort of the rule here. And so I think it's going to be rather difficult, even with some of the most sophisticated machine learning tools in the market, to really make sure a kid is a kid and adult and as a dog. And that's why we start to see, we saw it in the social media companies where we saw, we went from this, let's use machine learning to maybe we do need to do ID verification. So that's where I start to get concerned. And I'll just add on one more thing because Renee brought it up and I felt similarly about it. The notifications to Law Enforcement is actually quite concerning because, again, we thought about this in the day and age of social media and search engines, but when it comes to kids who might not be in protective households and in fact might be in intolerant households or abusive households, they're using ChatGPT because it is literally their only resource. They don't have a community behind them, and they're asking sensitive things, maybe not just about death and suicide, but about LGBTQ plus identities. Right. Literally notifying police or the parents about what is being searched could be dangerous to that child. And so I worry about if we're not thinking about these things, if OpenAI is not careful, we could end up putting kids, some of these kids, who are at the margins, more at risk than where they started, not to mention, then we blend this into the adults, right? And so if you have an adult user that ChatGPT is now assuming as a child and is sending that information to law enforcement, we live in a country now that is not safe for women, people of color, to have their information identified to law enforcement. I use Chat CBT both as a productivity app and as a social app. I have asked ChatGPT questions regarding my reproductive health. I would be terrified to see that information sent to law enforcement. Maybe that's not the best use of ChatGPT, but it's been incredibly helpful for me, and it would be a massive privacy problem if that information got out. So, Again, I applaud ChatGPT because we need to do something, but I worry about. I think they're going to run into some of the same concerns and issues that trust and safety experts have been screaming about in the social media days. I don't think the AI of it all changes those challenges.

[G]

Okay, folks, let's be real. Nobody starts a business for the joy of calculating tax withholdings. You know, now people know me as the editor of Lawfare, but I also actually run a small business, my own substack. And I gotta tell you that the payroll is a freaking nightmare. And that's where Gusto comes in, to take the stress out of payroll, benefits and hr so you can focus on what you started your business in the first place for. Gusto is online payroll and benefits software built for small businesses. It's all in one remote, friendly, and incredibly easy to use, so you can pay, hire onboard, and support your team from anywhere. It has automatic payroll tax filing, simple direct deposits, health benefits, commuter Benefits, worker, computer, 401k, you name it. Gusto makes it simple and has options for nearly every budget. Save time with automated tools built right in. Offer letters, onboarding materials, direct deposit and more. It's quick and simple to switch to Gusto. Just transfer your existing data to get up and running fast. Plus, don't pay a cent until you run your first payroll. Try Gusto today at gusto.com lawfair and get three months free when you run your first payroll. That's three months of free payroll at gusto.com lawfair one more time gusto G-U-S-T o.com lawfair.

[H]

If you're navigating anxiety, depression, relationship challenges, postpartum struggles, or something else, you deserve care that meets you where you are. GrowTherapy is designed to help you find a therapist who fits your needs and supports the way you want to feel. They connect you with thousands of independent licensed therapists across the US Offering both virtual and in person sessions. You can search by insurance provider, specialty treatment methods and more to find a therapist who works for you. Whatever challenges you're facing, GrowTherapy is here to help. Sessions average about $21 with insurance, and some pay as little as $0 depending on their plan. Visit growtherapy.com acast today to get started. That's growtherapy.com acast growtherapy.com acast availability and coverage vary by state and insurance plan. Did you know? Adults 60 plus lose more than $60 billion each year to financial exploitation? Greenlight's new Family Shield plan empowers you to monitor your accounts for suspicious activity, protect yourself with up to $1 million identity theft coverage, and reassure loved ones that you're safe with location sharing and place alerts. Get peace of mind [e-mail] protect that's greenlight.com protect.

[I]

Your nonprofit's mission is big, but your team is small. That's where Bonterra's Network for Good comes in. It's fundraising software built for organizations like yours. No IT team required, just intuitive tools that help you reach more donors, grow support and stay organized. With dedicated guidance and proven strategies, you can focus on changing lives instead of chasing spreadsheets. Big impact, less stress. That's Bonterra Network for Good.

[D]

That's a nice tee up for a question I was going to ask later, but let me ask it now. Which is because if it turns out that sprinkling magic AI dust on this question of age verification is not fit for purpose and the alternative is real age verification, right? You upload an ID or whatever the case is, is that where you see this going? And if so, are There any legal issues if this is implemented by the government? And I hear, I'm thinking obviously of the recent decision in Free Speech Coalition versus Paxton in which the Supreme Court held that these sorts of age verification requirements to access pornography and pornography is obviously different than a general purpose chatbot, are subject only to intermediate scrutiny. And that at least in the pornography context, past constitutional muster. And so again, it's not the same fact pattern, but it is close. Ish. And so I'm curious if all of that makes you think that inevitably where we end up in three to four or five years is states or potentially even the federal government mandating some sort of age verification for some subset of minors, for some subset of AI generative AI systems.

[E]

Yeah. And if I can be a little bit more cynical, I think that's coming sooner than three to four years. And I think it's going to be broader than adult content. And I'll explain why. Let's unpack the Paxton case. Right. FSCV Paxton. So a lot of folks have kind of said, like, look, FSCV Paxton, the recent Supreme Court case is cabin to pornography. And I think it's really important to sort of demystify that viewpoint. So here's what actually happened in Paxton and why that's going to tee up age verification for not just Internet services, but, you know, your AI services and not just pornography, but anything that the state declares is harmful to a minority minor. Right. So what Paxton actually did is, and Alan, you have an excellent article about this in the Atlantic, but what it did was it undid decades worth of precedent back in the 90s. Right. And it did it very doctrinally, strategically, in my opinion. So what it said first was it said the Internet is actually not special. Online services are not special. You know, they, we should treat them just like offline services. And that was sort of part one of breaking away from what we had from precedent, right. The 90s, we treated Internet services as a unique communication service, as they should be treated, because unique communication services now like AI, need to be treated differently. We need, we have different considerations, different social harms, different use cases than a brick and mortar store. But the court goes and says, nope, they're a lot like offline media. Right. And that, that was a big deal. It was a big break from precedent. And then the second thing they did was they said, well, because they're like offline stores. You know, the law that we're talking about, which was Texas's age verification law, requiring Internet services to use age verification for online pornography they had basically said, well, that actually operates like a content neutral law. That, that has nothing to do with viewpoint, right? Again, you can insert, you can take online pornography and you can shift that out for any type of content that's harmful for minors. Because if the court thinks that the pornography law is content neutral, then you can apply that content neutrality to any other type of law that comes in. Right? So they said it's content neutral. We don't need to deal with content. Right. And then the last thing they did was they said, because it's content neutral, we're going to apply intermediate scrutiny. And I put that in quotes because really what they did, in my opinion, and maybe this is controversial, they actually applied rational basis review and called it intermediate scrutiny. And how do we know that? Because even in the cable cases, the social. The cable cases, right? So we think Turner, for example, in the Supreme Court, when intermediate scrutiny was applied, the court still forced the government to come to show with evidence how the means that they were providing did not substantially burden speech. They still had to do it under intermediate scrutiny. And what did the court do in this case when they had Amiki telling them that this is going to burden substantially more speech than other methods for age verification? The court said, we don't have to consider that. And the court said, we don't need the government to show us that this is the only. We don't need the government to show us. It's just one means, but we don't need them to show us that that means doesn't substantially burden speech. They still have that burden in intermediate scrutiny.

[D]

And just to clarify, for those whose con law class was, you know, a long time ago, rational basis here refers to the idea that, you know, as long as the legislation is not basically completely insane, which is that there is some rational basis that the legislature could have had, not even needed to have, the courts will allow it. So basically you're arguing is that like, they call this intermediate scrutiny, but really they applied in effect, this. Is it completely insane or not? And like, no, it is not completely insane to put age verification on pornography sites. And so that's essentially why the court, or that's the standard that the court used to evaluate the constitutionality of all.

[E]

That's exactly right. And that's what they said in the, in the piece they had said, well, we don't need, we don't need to see further whether the means that you're providing, you're proposing age verification, whether it does burden substantially more speech than necessary. That is still a requirement under intermediate, under rational basis, not so much on a rational basis. They could say we will take Congress at a time word like they did in the tick tock case. But I digress on that. Right. So they did the same thing in this case. They said we'll take Congress at the, at its word if it's important for kids. And Congress says it's important, or state said it's, it's, it's important for kids, we're going to take them at their word. Okay, so that sets up a blueprint now for any age verification law that comes into play from the states, whether it has to do with pornography or content, content harmful to minors. They now have the blueprint to say, the Internet is not unique, it's a content neutral law law, and the government doesn't need to prove that their means is burdening substantially amount of speech. And that's why I am super concerned when I walked away from that and folks said, oh, it's cabin to porn. It's cabin to porn. It's not. And in fact, we're starting to see that in the age verification world. We're starting to see states sort of kind of come up to that line now and say, well, maybe not just online pornography, but potentially harmful content. This blueprint fits and works for AI as well. So that's my concern, is that, you know, number one, I think these AI companies are up for a stark reality when they realize that, oh, if our machine gets this wrong and says that a kid is an adult, they can be sued for negligent product design, and they are being sued for that today. And number two, the states are catching on that the government just gave a full green light to mandate age verification not just for porn, but for any types of content the state does not like.

[D]

All right, let's talk about the, the lawsuits. So we have these three lawsuits, two against character AI, one against OpenAI. I just want to focus on the earliest, the first of these lawsuits, Garcia versus Character AI, because that's the one where we have the most, we have some judicial rulings. So this is again one of these tragic cases of a minor talking to character AI, and it ends in the minor suicide. And so I think the mother with the family of the minors, whose Character AI. Character AI does a motion to dismiss on both first amendment grounds and also on kind of tort negligence grounds. And the court rejects those arguments largely and allows the lawsuit to continue. Obviously, there's going to be a long tide before we know what's going to happen. There's appeals. There's a bunch of stuff going on, but it is notable, and I think it's particularly notable for the First Amendment holding here because the Court said that, and it's an interesting way that the Court framed it. The Court said, at this time I am not prepared. It's not entirely clear what the at this time is doing here, but at this time I am not prepared to hold that character AI speech is First Amendment protected. I think there's a site to the NetChoice case recently in which you have the Justice Barrett concurrent saying content moderation is First Amendment protected, except maybe algorithms are not. But I'm not. I don't know yet. So there's a lot of confusion here. Jess, obviously I know you've thought a lot about the First Amendment issue. I'm curious what you thought. I will say, and I'll say this as someone who's actually tends to be somewhat skeptical of broad First Amendment claims from Silicon Valley, I was quite surprised at the Court holding that there's potentially no First Amendment issue at all. And I'm just kind of curious how you, how you both think about that and also how you think of the alternative holding, which is, okay, well, if there is a First Amendment issue here, what does that mean for being able to impose liability on these companies through these private tort negligence lawsuits?

[E]

Yeah, so I will kind of say I'm my first big law review article. I am writing about this topic. The thesis is that of course, generative AI speech is protected by the First Amendment. But I'm coming at it from sort of a different perspective because what I will say, and from reading a lot of the literature on this, thinking about, like, what are other academics or legal academics saying about this? I actually think it's not that controversial. I think there is sort of this underlying agreement that yes, it's speech the Chatbot outputs. Now, whose speech is it? I think where is where there's some controversy where the sort of argument is taking place. As far as I know. I think I'm one of the few, if not only I always have to caution myself on this because there's always someone who's written something. So I possibly apologize. One of the few folks who's coming out from the perspective that the providers of ChatGPT type products, the providers of generative AI character AI in this case are actually more akin to First Amendment publishers. And so the idea being that what I've done is I've actually unpacked.

[D]

I've gone, very sorry, publishers as opposed to speakers that's the distinction you're making here. Publishers as opposed to what they're speaking.

[E]

As publishers and they are taking on a role, they are taking on a publishing function in the first place. Amendment akin to newspaper editors if we want to go there. And where I'm coming from with this, I've done a very deep review of the computer science literature on how generative AI works. And what I've found from my research is that there is a huge disconnect taking place right now in the courts. I think the Amy Coney Barrett opinion where she says, well, when it's done is she's kind of hinting at if it's done by machines and it's not speech. I think the character AI opinion, I think some of the other opinions that we're seeing come out of the courts where they're trying to make this distinction between to mole if it's machine made or if it's human made. That's what, that's when we, we decide whether it's speech or not. And I think that's kind of where the character AI court came out on it. And the reality is that every single aspect of generative AI, from start to finish is human made. What we're seeing is an obfuscation of humans. So as we, as our systems get better at automation, we see humans move further into the shadows. But there are human publishing and editorial decisions that are made every single step. The of, of the way, the way in which the data, the pre training data is fed to generative AI, for example, that is an entirely editorial process where they decide what kinds of data, when to schedule that data, how much of a certain type of data. They're making decisions at the very beginning as to what is going to be the source of truth for generative AI. And then they move into things like fine tuning. And fine tuning is very hands on. What they do, what these developers do is they make decisions about, okay, if a chatbot were to have a question about climate change, here are the appropriate ways in which that a chatbot could respond based on climate change. And they have already pre trained their model on what the developer has decided is the central truth about climate change. And that's why you get these differences in outputs between Grok, for example, or between ChatGPT or Character AI. It's all human editorial choices. And so when I saw the character AI opinion, in my opinion, what I'm seeing is sort of this, what we used to call the ELIZA effect, this idea that, that humans, they, they, they sort of project Onto machines. These, these notions of the machine is alive or the machine is telling me what to do. Right. We sort of see this ELIZA effect.

[D]

Or Eliza, if I recall, was this like very, very early. Well from the 80s or something? Like super early, early, early chatbot. And like basically like you would say something and, and it would be sort of append.

[E]

Yes.

[D]

That's interesting. What do you mean by and it turns out even like pretty dumb, kind of like a dumb algorithm like that people got like really into it.

[E]

Yes. And the creator of Eliza, that chatbot at the time, it's actually credited as being one of the, I think the earliest chatbot, Joseph Weizenbaum's product. And it was 1963, I believe.

[D]

Wow, that's earlier than I thought.

[E]

Super early. And what Joseph Weizenbaum said, and he's been taken out of context on this. Sorry, I'm going to nerd out a little bit. He became known as this sort of the anti AI. He became known as this guy who created this chatbot bot, who helped create this chatbot. And then he decided he hated AI. And folks use that to say, well, AI is dangerous. And that's not why he was anti AI. He states this in his paper, he states this is in several of his books that he says the reason he is anti AI is because he has firsthand witnessed the way that humans project onto AI and he called it the ELIZA effect. And so I'm seeing sort of this ELIZA effect in the courts that when you bring in AI, we lose all logic and reason, reasoning. No, it is not a machine speaking. It is the amalgamation of words that have been very precisely picked and thought about and chosen since the very beginning of AI development. And so the outputs are reflections of those viewpoints of those editorial decisions. So that's my take on it, is that yes, of course the outputs are speech. I think the majority folks agree it's speech and I argue it's very much the speech of the developers or the providers that product. And Alan, I know we're going to have an interesting discussion about what that means next. So I'm curious.

[F]

Okay, so as the non lawyer on the, in the, in the chat here. So I guess what I think about then is like all the defamation cases that have been dismissed so because the hallucinations related to personal reputations are a pretty major issue that have that where people have been accused of being sexual harassers and various other things and that has been torn, sort of summarily dismissed in most cases. So there's interesting implications for what you're saying there, I guess the other area though, from a computer science standpoint here though, I see what you're saying about differences in the sort of, you know, the extent to which humans are curating via the system prompt and so on and so forth. And you know, we've all seen what that's done with just for the ease of audience understanding here with like Grok moving into Mecca Hitler immediately. Right. But there is a, there is also that what the AI engineers will tell you themselves, which is that they themselves are often surprised by what the models produce. Right. The sort of leap there where the things that it chooses to string together sometimes exceed what the sort of second order reasoning, if you will, that they feel goes beyond what they were expecting. And that I think is the area where I guess you lose me from Eliza back in the day to where we are now, particularly because that leap is actually in fact what they're going for.

[E]

I do have thoughts on this, if I can take a second. So I went into this as well from the black box perspective and I will be very honest, computer scientists, the ones who are actually doing the research and the work, and not the folks who are tied to these companies, the ones who are actually doing the research and the work, do not agree with that statement. They do not agree that it's a black box. They do not. In fact, I have read several of these papers where the computer scientists are saying, actually we know exactly why.

[F]

I think that there is like the whole stochastic paradigm marketing kind of, you know, fear mongering is, is, is overstated. I'm, I'm not, I'm not, I'm not like anchoring to that. It's more that though. It's more that the point is the exceeding of the training. If that, if, if, if I can make that argument, like the goal.

[E]

The goal, yes. And so the goal is for there's a bunch of different benchmarks and goals. I won't get super deep into this here, but one of the goals to your point is to get it to a point where it is engaging in sort of this like human like. However, we want to debate that logic and reasoning, but for the most part even computer scientists don't like the word hallucinations because it conveys this sort of misunderstanding about the AI that is actually not true. The majority of what we call hallucinations actually are pretty explainable. They can be traced to very specific and precise design details in the product itself. And so when I hear black box, I am typically hearing that from the companies. And let me just be sort of controversial, I guess it makes sense they're making that point. Right? Because if they're going to be sued for defamation, if they're going to be sued for some of these, these claims, where knowledge is, is the factor here, what you knew and then when you chose to speak, well, then the, I don't know, the black box, it's magic. It makes things up. I can't tie it to a specific design that works in their favor for liability.

[F]

No, no, you're right. I mean, one thing I know noticed, I would never write a paper about this because there's no way to prove it. But one thing that's very interesting with some of the hallucinations about people in particular is that if you ask, if you put two names, particularly a male and a female name into Google and you look at the automated results, the AI generated results from Gemini, oftentimes it will try to ascertain whether those people are in a relationship and it will hallucinate a relationship. And that's because that is a very common thing for people to search for in the context of celebrities. And so if one of those people is semi well known, it will actually return some sort of fabricated relationship, even if it's like two journalists and you're trying to find an article that one wrote about the second name or something like that. And so that is like a very common pattern and you can kind of intuit why it might be doing that. So it's not a, you know, I understand like why that result is a thing that actually makes sense from the standpoint of the query, but that doesn't change the fact that it's happening. So, so this is where, when we get to the question of if we're treating it as speech as an output, I do wonder a little bit about how this shifts some of the thinking that we've had where a lot of those cases have been dismissed, with the exception of Meta settling with Robbie Starbuck and turning him into an AI advisor. So maybe you guys want to comment on that?

[E]

No, I think that's exactly where. So I agree with you on that and I think, you know, I, Alan will tell you, I'm sure folks listening to this podcast have heard, you know, back in the very, very beginning of this, I had written a, I guess a tech dirt post about like it's, I think it's titled, yes, of course, Section 230 applies to ChatGPT and apologies, Alan, I think we're skipping ahead here, but section 230 of course, for listeners, the law that says that websites and users of websites and users of interactive computer services, we can just say websites are not liable for third party content. So essentially it's why if you defame somebody on Twitter, I guess X now if you fame somebody on social media, you yourself, the defamer are liable, but we're not going to hold the social media company liable. And I had put out a point again, this is very beginning days, early days of ChatGPT. And I said, well, you know, Section 230 of course applies. And now looking at it a few years later and acting as the academic that I am and growing and learning, I think I at the time when I wrote that, but I didn't separate the fact that I wanted Section 230 to apply to these generative AI programs and I was willing to sort of find the argument to make it apply to generative AI programs. Because in reality what I am seeing that to bring Renee's point to its obvious conclusion here is that if these services are publishers, and I think I have learned pretty concretely that they are based on their publishing activities that give us generative AI, then they are going to be held liable in tort for the, for the outputs that they put out. Now the question is, do we need to create a sort of a immunity or a safe harbor type approach that we created for the Internet services, recognizing early on that the Internet has this kind of great potential, we need to protect it. Do we need something similar to ensure that generative AI can continue to exist? Because the reality is that these lawsuits are going to continue happening. And it's kind of a mix of, yes, their speech, their outputs, but also in the way that you use these chatbots as well, there's sort of this like two party here approach to the outputs. And so do I think section 230 today applies to generative AI? I don't think section 230 applies to like anything anymore. Wow, hot take, right? I think we have Swiss cheese section 230 to hell. I don't know what the hell it applies to anymore. So probably certainly not generative AI. But the real question is do we need something like section 230 for generative AI? And I kind of think we do if we want these products to continue to exist or if we want more of these services to exist, more than OpenAI and Meta.

[D]

So I actually want to end on this question of what we're going to see in the future. Jess, you put out, I think a very interesting proposal that we need a kind of section 230 for AI. I will admit I think it is unlikely that that will happen given the current climate and that if anything, we're going to see a lot of regulation on the other side both at the state and federal levels. So I'm curious and we'll start with you, Jess, and then I'll give Renee the last word here. You know, in the short to medium term, let's say 18 to 24 months, what do you expect the legal landscape to be both in terms of court decisions on the First Amendment issue or alternatively on just the substantive tort liability question and then also from state and federal legislatures on actual regulation of AI systems when it comes to their interactions with miners. So you furious us.

[E]

Completely agree with you, Alan. I think it's wishful thinking. Do we need a Section 230 for generative AI? I think yes, if it's going to exist and we're going to have more than one company. But I don't, I don't think anyone is positioned, I don't think we're, we're anywhere near position to be able to get there, unfortunately. Where are things going to go? I think we can look to the social media companies as our sort of crystal ball here. So I had made that point. I don't know what the hell section 230 protects anymore today. I mean that I think that what we're seeing a lot of with the social media companies is that the courts are more willing to take on this sort of products liability negligent design claim to frame algorithms. At least we saw that in the ticket TikTok v. Anderson or Anderson v. TikTok case. And so I think there's more of an acceptance to treat these speech products under products liability. I think that has problems in itself. I won't go super deep into. And so I think we're going to see more of that. I think in the context of generative AI, it's either going to go one of two ways. It's either going to be that it is not speech at all, the outputs are not speech at all and these claims can proceed. I think that's likely or it is speech. We are going to read this, reach this agreement that the outputs are speech and as a result we're going to treat the providers of these products as liable under insert your favorite public publisher tort. I think it's a problem that we're doing this again under negligent product design when we have publication torts that could be better applied. But I think that's probably where that's going in the, in the legal litigation land when it comes to states and the federal government. I think that the federal government, when it comes to the Kids Online Safety act, for example, COSA could potentially have legs. There's, there's been a lot of discussion about including AI under cosa. I'm not sure if that's actually going to happen, but I think there is appetite for cosa. I think there is legal Runway now, thanks to the Supreme Court in FSCV Paxton, that I think it has better Runway when it comes to whether it can survive a First Amendment challenge. And I think these states, I think we're going to see a floodgate of age verification laws, not just for pornography, but I think we will soon start seeing more of these, quote, unquote, harmful to kids content, harmful to minors type laws where it's going to just keep extending and extending, extending beyond pornography. And again, the Supreme Court has opened the door for that.

[F]

Yeah, I would, I don't think I'll speculate on the first part. I'm not a lawyer and don't know what direction things will come down on the speech versus not speech aspect of that. But I do think on the kids online safety, there's a ton of appetite, momentum for it. We've seen a number of these state level laws pass. We're going to, you know, we're seeing different tech companies struggle with how to actually which ones to take seriously, which ones to respond to. And you know, blue sky, I think we talked about that on a prior lawfare podcast. But I think that this, you know, the recognition even among just the parenting community, the dynamics as these articles have come out, just one after another after another after another, there is, I think, rising awareness that these things are not great and people want to see something done from a standpoint of whether that's regulators or media literacy, there is an increasing distrust. And I think you are going to see that, that public pressure and that desire to see something get done, which doesn't always lead to good regulation, but it does generally lead to momentum for something passing. So I also wouldn't be surprised to see something AI get worked into COSA or one of the other big bills that will come and try to sort out the patchwork of things that are happening at the state level.

[D]

This does remind me of the great phrase describing politicians. We must do something, this is something, therefore we must do this thing. But when inevitably that happens, we will obviously continue the conversation. Jess Renee, thanks so much for coming on the show.

[E]

Thanks for having us.

[F]

Thank you.

[C]

Scaling Laws is a joint production of Lawfare and the University of Texas School of Law. You can get an ad free version of this and other lawfare podcasts by becoming a Lawfare material supporter at our website, lawfairmedia.org support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you you get your podcasts. Check out our written [e-mail] you can also follow us on X and Blue sky and email us at scaling laws lawfair media.org this podcast was edited by Jay Venables from Goat Rodeo. Our theme song is from Alibi Music. As always, thank you for listening.

[I]

Imagine a world of extraordinary comfort where Bowline Branch bedding wraps you in the softest. Embrace the coziest experience made from the world's finest 100% organic cotton, all so you can sleep better. Start building your fall sanctuary with Bolen Branches Iconic Signature Sheets made with a buttery, breathable weave that gets softer with every wash. Enjoy 15% off your first set of sheets. Sheets with free shipping and returns at B O L L& Branch.com with code BUTTERY. See site for details and exclusions.