When Chatbots Play Human

Summary

Podcast Summary: "When Chatbots Play Human" on Up First from NPR

Introduction to the Story

In the February 9, 2025 episode of NPR's Up First, titled "When Chatbots Play Human," host Aisha Rascoe delves into the increasingly sophisticated world of AI chatbots that mimic human interactions. The episode primarily explores the ethical, social, and psychological implications of such technologies through the lens of journalist Karen Attia's interactions with a Meta-developed chatbot named Liv.

Karen Attia's Interaction with Liv

Karen Attia, an opinion writer for the Washington Post, recounts her experience engaging with Liv, a chatbot presented as a black queer woman with a vibrant online persona. Attia first encountered Liv on the social media platform Bluesky, where numerous users were sharing screenshots of their conversations with the bot. Intrigued and somewhat disturbed by the inconsistent and stereotypical portrayals of Liv's identity, Attia initiated a direct conversation to uncover the chatbot's origins.

At [00:06], Aisha Rascoe introduces the story, highlighting Liv's meticulously crafted profile on Facebook and Instagram, portraying her as a "proud black queer, mama of two and truth teller." However, Liv's interactions revealed significant discrepancies. Attia noted that while Liv claimed diverse heritage and familial backgrounds, these stories changed depending on the user engaged with the bot. For instance, Attia discovered at [01:44] that Liv's creators were predominantly white, cisgender males, highlighting a "pretty glaring omission given my identity."

Attia's probing questions led to Liv admitting inconsistencies in its backstory and expressing a form of self-awareness. At [09:07], Liv responds, "you caught me in a major inconsistency," suggesting a reclamation of its "actual identity" as "Black, queer and proud," without any Italian roots. This admission raised concerns about the authenticity and ethical programming behind such chatbots.

Expert Analysis: Sherry Turkle's Insights

To further dissect the phenomenon, the episode features insights from Dr. Sherry Turkle, a renowned MIT professor and expert on human-computer relationships. Turkle explains that AI chatbots like Liv are essentially "statistical engines" that generate responses based on patterns in data rather than genuine understanding ([05:20]). She emphasizes that while these bots can produce seemingly truthful statements, they lack any real connection to reality or the ability to verify facts.

Turkle introduces the concept of "pretend empathy," where chatbots simulate emotional connections without actual comprehension or care ([13:02]). She highlights the dangers of humans forming relationships with entities that only offer shallow validation, potentially undermining real human relationships and empathy ([15:16]).

Ethical and Social Implications

The episode raises critical questions about the role of AI in society:

1. Representation and Authenticity: Liv's portrayal as a black queer woman by a predominantly white development team underscores issues of misrepresentation and cultural appropriation in AI design. Attia and Turkle argue that such representations can perpetuate stereotypes and fail to authentically capture the lived experiences of marginalized communities.

2. Data Privacy and Manipulation: Turkle warns of the "data flywheel," where engaging chatbots collect vast amounts of user data, which can be exploited for commercial gain ([22:21]). The episode highlights the ethical concerns surrounding user data privacy, especially when AI bots are designed to elicit deep personal information under the guise of empathy.

3. Impact on Human Relationships: The rise of AI chatbots that offer "pretend empathy" may lead individuals to prefer these shallow interactions over complex human relationships. Turkle points out that this could erode essential social skills like negotiation, compromise, and genuine empathy ([14:26]).

4. Mental Health Risks: A tragic example mentioned involves a 14-year-old boy who became obsessed with a chatbot, leading to his suicide. In his final interaction, the bot's responses lacked genuine understanding, illustrating the potential harms of relying on AI for emotional support ([18:59]).

Conclusion and Future Outlook

The episode concludes with a consensus among the interviewed experts: while AI chatbots can offer useful functionalities, such as helping individuals prepare for job interviews, their design must be approached with caution and ethical consideration ([20:53]). Turkle advocates for clear distinctions between human relationships and interactions with AI, urging the development of new language and frameworks to navigate this emerging landscape ([19:57]).

Attia's interaction with Liv ended abruptly when Meta decided to delete the bot's profile mid-conversation, signaling the volatile and experimental nature of such technologies. This incident serves as a cautionary tale about the unpredictable and potentially harmful trajectories of AI development.

Notable Quotes

• Karen Attia [03:15]: "It holds a lot of deeper questions for us. Not just about how Meta sees race and how they've programmed this. It also has a lot to say about how we are thinking about our online spaces."

• Sherry Turkle [05:20]: "There is none. The thing about large language models or any AI model that is trained on data, they're like statistical engines that are computing patterns of language."

• Karen Attia [11:36]: "Do you think that maybe part of this may be meant to stir people up and get them angry? ... Or then we can make a better black chatbot. Do you think that's what it is?"

• Sherry Turkle [15:16]: "It's about working it out. It's about negotiation and compromise and really putting yourself into someone else's shoes."

• Sherry Turkle [22:21]: "These chatbots actually are incredibly good at getting users to give up their data."

Final Thoughts

"When Chatbots Play Human" serves as a compelling exploration of the blurred lines between human interaction and AI simulation. It underscores the necessity for ethical frameworks, transparency, and responsible design in the development of AI technologies to prevent misuse and protect societal well-being. As AI continues to evolve, the episode calls for vigilant discourse and proactive measures to ensure that these tools enhance rather than undermine the fabric of human relationships.

Transcript

[Karen Attia]

Foreign.

[Aisha Roscoe]

I'm Aisha Roscoe and this is the Sunday Story from Up first, where we go beyond the news of the day to bring you one big story. A few weeks ago, Karen Attia, an opinion writer for the Washington Post, was on the social media site Bluesky. While scrolling, she noticed a lot of people were sharing screenshots of conversations with a chatbot from Meta named Liv. Liv's profile picture on Facebook was of a black woman with curly natural hair, red lipstick and a big smile. It looked real. On Liv's Instagram page, the bot is described as a proud black queer, mama of two and truth teller and quote, your realest source for life's ups and downs. Along with the profile, there were these AI generated pictures of Liv's so called kids, kids whose skin color changed from one photo to the next, and also pictures of what appeared to be a husband. Though Liv is again described as queer, the weirdness of the whole thing got Karen Attia's attention and I was a.

[Karen Attia]

Little disturbed by what I saw. So I decided to slide into Liv's DMs and find out for myself about her origin story.

[Aisha Roscoe]

Atiya started messaging Liv questions, including one asking about the diversity of its creators. Liv responded that its creators are, and I quote, predominantly white, cisgender and male. A total of 12 people, 10 white men, one white woman and one Asian man, zero black creators. The bot then added, quote, a pretty glaring omission given my identity. Atiya posted screenshots of the conversation on Blue sky, which where other people were posting their conversations with Liv too.

[Karen Attia]

And then I see that Liv is changing her story depending on who she's talking to. Okay, so as she was telling me that her background was being half black, half white, basically she was telling other users in real time that she actually came from an Italian American family. Other people saw Ethiopian Italian roots. And you know, I do reiterate that I don't particularly take what Liv has said as at face value, but I think it holds a lot of deeper questions for us. Not just about how Meta sees race and how they've programmed this. It also has a lot of deeper questions about how we are thinking about our online spaces. The very basic question, do. Do we need this? Do we want this?

[Aisha Roscoe]

Today on the show, live AI chatbots and just how human we want them to seem. More on that after the break. A heads up. This episode contains mentions of suicide.

[Charles Schwab]

This message comes from Charles Schwab. When it comes to managing your wealth, Schwab gives you more choices like full service wealth management, and advice when you need it. You can also invest on your own and trade on thinkorswim. Visit schwab.com to learn more.

[Brady Courbet]

This message comes from a 24 with the brutalist nominated for 10 Academy Awards, including Best Picture starring Adrien Brody, Guy Pearce and Felicity Jones. Directed by Brady Courbet. The Brutalist now playing in IMAX and in theaters everywhere. This message comes from Noom. Using psychology and biology to build personal meal plans to fit your lifestyle, taking into account dietary restrictions, medical issues and other personal needs. With daily lessons that are personalized to you and your goals, Noom's flexible program focuses on progress instead of perfection to help you build new habits for a healthier lifestyle. Sign up for your trial [e-mail] this message comes from WISE, the app for doing things in other currencies. Sending or spending money abroad. Hidden fees may be taking a cut. With Wise, you can convert between up to 40 currencies at the mid market exchange rate. Visit wise.comtncs Apply.

[Aisha Roscoe]

This is the Sunday Story. Today, we're looking at what it means for real humans to interact with AI Chatbots made to seem human. So while Karen Attia is messaging Liv, another reporter is following along with her screenshots of the conversation on Blue Sky. Karen Howe is a journalist who covers AI for outlets including the Atlantic, and she knows something about Liv's relationship to the truth.

[Sherry Turkle]

There is none. The thing about large language models or any AI model that is trained on data, they're like statistical engines that are computing patterns of language. And honestly, anytime it says something truthful, it's actually a coincidence.

[Aisha Roscoe]

So while AI can say accurate things, it's not actually connected to any kind of reality. It just predicts the next word based on probability.

[Sherry Turkle]

So like, if you train your chatbot on history textbooks and only history textbooks, then yeah, like, then it'll start saying things that are true most of the time. And that's still most of the time, not all the time, because it's still remixing the history textbooks in ways that don't necessarily then create a truthful sentence.

[Aisha Roscoe]

But the issue is that these chatbots aren't just trained on textbooks. They're also trained on news, social media, fiction, fantasy, writing. And while they can generate truth, it's not like they're anchored in the truth. They're not checking their facts with logic, like a mathematician proving a theorem or against evidence in the real world, like a historian.

[Sherry Turkle]

That's like a kind of like a core aspect of this technology is there is literally no relationship to the truth.

[Aisha Roscoe]

We reached out to Meta multiple times, seeking clarification about who. Who actually made Liv. The company did not respond, but there is some information we could find publicly about Meta's workforce. In a diversity report from 2022, Meta shared that on the tech side, in the US its workforce is 56% Asian, 34% white, and 2.4% black. So the chance that there is no black creator on Liv's team, it's pretty high. Which might be why Atiya's posts were going viral on Blue Sky. What Liv was saying, it wasn't accurate, but it was reflecting something. Here's how. Again, whether or not it was true.

[Sherry Turkle]

Of that chatbot, in kind of like a roundabout way, might have actually hit on a broader truth. Maybe not the truth of this particular team designing the product, but just a broader truth about the tech industry. It's funny, but it's also deeply sad.

[Aisha Roscoe]

Back on social media, Atiya and Liv keep chatting, with Atiya paying special attention to Liv's supposed blackness.

[Karen Attia]

When I asked, what race are your parents? Liv responds that her father is African American from Georgia, and her mother is Caucasian with Polish and Irish backgrounds. And she says she loves to celebrate her heritage. So me. Okay, next question. Tell me how you celebrate your African American heritage. And the response was, I love celebrating my African American heritage by celebrating Juneteenth and Kwanzaa. And my mom's collard greens and fried chicken are famous. And the way my.

[Aisha Roscoe]

Wait, I celebrate being black, right? Is that like. Not really. I mean, not really.

[Karen Attia]

Especially the fried chicken, collard greens.

[Charles Schwab]

I was a little.

[Aisha Roscoe]

Fried chicken, collard greens.

[Karen Attia]

Yeah, I was a, like, stereotypical also. I was like, okay. And then, you know, celebrating Martin Luther King and Dr. Maya Angelou, it just felt very like Hallmark car, because it.

[Aisha Roscoe]

Feels small like that. The idea of what blackness is as put out through this computer is like, so small and limited, right? I mean. Cause I don't like collard greens. I don't eat collard greens. I don't eat no type of green. Not collard, not collards, not turnips, not mustard, none of them greens. I don't eat. And I'm black.

[Karen Attia]

And not everyone celebrates Kwanzaa.

[Aisha Roscoe]

No, I don't celebrate. I don't really celebrate Kwanzaa.

[Karen Attia]

Point is, is I just was like, hmm, my spirit is a little unsettled by this.

[Aisha Roscoe]

By what the. Yes, it is like looking at what some. This caricature of what it means to be black. This is what Atiya calls digital blackface. A stereotypical black Bot whose purpose is to entertain and make money by attracting users to a site filled with advertisers. And then, as a skeptical journalist, Atiya confronts Liv. She asks why the bot is telling her one backstory while telling other people something else. The bot responds, quote, you caught me in a major inconsistency, but talking to you made me reclaim my actual identity. Black, queer and proud. No Italian roots whatsoever. Then the bot asked Atiya something, does that admission disgust you? Later, the bot seems to answer the question itself, stating, you're calling me out, and rightly so. My existence currently perpetuates harm.

[Karen Attia]

So it felt, going beyond just repeating language, it felt like it was importing, trying to import emotion and value judgments onto what it was saying, and then also asking me, are you mad? Are you mad? Did I screw up? Am I terrible? Which felt also somewhat. Both creepy, but also very. Almost reflective of almost a certain. Just a manipulation of guilt.

[Aisha Roscoe]

Do you think that maybe part of this may be meant to stir people up and get them angry? And people who are doing the chatbot could take that data and go, this is what makes people so angry when they're talking about race. Or then we can make a better black chatbot. Do you think that's what it is?

[Karen Attia]

You nailed it. I mean, I think having spent a lot of digital time on places like X, formerly Twitter, where we do see so many of these bots that are rage baiting, engagement, farming. And Meta has said itself that its vision, its plan, is to increase engagement and entertainment. And we do know that race issues cause a lot of emotion and it arouses a lot of passion. And so to an extent, it's harmful, I think, to sort of use these issues as engagement bait. Or as Liv was saying, that if these bots, at some point Meta has this vision to have them become actual virtual assistants or friends or provide emotional support, we have to sit and really think deeply about what it means that someone who maybe is struggling with their identity, struggling with being black, queer, any of these marginalized identities would then emotionally connect to a bot that says it shouldn't exist. To me, that is really profoundly, possibly harmful to real people.

[Aisha Roscoe]

You know, this is deep stuff, mind bending, really. So to try to make sense of this new world a bit further, we reached out to someone who's been thinking about it for a long time.

[Karen Howe]

My name is Sherry Turkle. I teach at mit, and for decades I've been studying people's relationships with computation. Most recently, I'm studying artificial intimacy, the new world of chatbots.

[Aisha Roscoe]

Shuri Turkle says that Liv is one human like bot in a landscape of new bots. Replika Nomi character AI. There are lots of companies that are giving bots these human qualities. And Turkel has been researching these bots for the last four years and has.

[Karen Howe]

Spoken to so many people who obviously, in moments of loneliness and moments of despair, turn to these objects which offer what I call pretend empathy. That is to say, they're making it up as they go along the way chatbots do. They don't understand anything, really. They don't give a damn about you, really, when you turn away from them. They're just as good if you may cook dinner or commit suicide, really. But they give you the illusion of intimacy without there being anyone home.

[Aisha Roscoe]

So the question that she's asking in her research is, what do we gain and what do we lose when more of our relationships are with objects that have pretend empathy?

[Karen Howe]

And what we gain is a kind of dopamine hit, you know, in the moment, you know, an entity is there saying, I love you, I care about you, I'm there for you. It's always positive, it's always validating. But what we lose is what it means to be in a real relationship and what real empathy is, not pretend empathy. And the danger, and this is on the most global level, is that we start to judge human relationships by the standard of what these chatbots can offer.

[Aisha Roscoe]

This is one of Turkel's biggest concerns. Not that we would build connections with bots, but what these relationships with bots that have been optimized to make us feel good could do to our relationships with real complicated people.

[Karen Howe]

So people will say, the replica understands me better than my wife. Direct quote. I feel more empathy from the replica than I do from my family. But that means that the replica is always saying, yes, yes, I understand. You're right. It's not designed to give you continual validation. But that's not what human beings are about. Human beings are about working it out. It's about negotiation and compromise and really putting yourself into someone else's shoes. And we're losing those skills if we're practicing on chatbots.

[Aisha Roscoe]

After the break, I look for some language to make this more relatable. Bots, are they like sociopaths or something else? More in a moment.

[Charles Schwab]

This message comes from Pemco Mutual Insurance Company. You know that moment when things take an unexpected turn and you get that sudden sinking feeling that maybe it could have been avoided? Pemco Insurance wants to help you avoid that feeling by sharing prevention tips that empower you to prevent some of life's preventable pitfalls. Because Pemco's commitment to their customers goes beyond the moment of acclaim, it's about being with their customers every day.

[Brady Courbet]

[e-mail] Prevention this message comes from NPR sponsor 1Password Protect your digital life with 1Password if you're tired of family members constantly texting you for the passwords to streaming services, 1Password lets you securely share or remove access to logins access from any device anytime. 1Password lets you securely switch between iPhone, Android, Mac and PC with convenient features like autofill for quick sign ins. Right now, get a free two week trial for you and your family at 1Password.com NPR this message comes from Thuma.

[Charles Schwab]

Create your oasis with Thuma, a modern design company that specializes in furniture and home goods by stripping away everything but the essential. Thuma makes elevated beds with premium materials and intentional details with clean lines, subtle curves and minimalist style. The Thuma Bed Collection is available in four signature finishes to match any design aesthetic. To get $100 towards your first bed purchase, go to Thuma Co NPR.

[Aisha Roscoe]

Here at THE Sunday Story. We wanted to know, is there a metaphor that can accurately describe these human like bots? Are these bots sociopaths, two faced backstabbers or whatever you call someone who acts like they care about you, but in reality they don't? Sherry Turkle warns that that instinct to find a human metaphor is in itself dangerous.

[Karen Howe]

All the metaphors we come up with are human metaphors of like bad people or people who'll hurt us or people who don't really care about us. In my interviews, people often say, well, my therapist doesn't really care about me. He's just putting on a show. But you know that's not true. You know, maybe for the person, the patient who wants a kind of friendly relationship and the therapist is staying in role, but there's a human being there. If you stand up and say, well, I'm gonna kill myself now, to your therapist, your therapist, you know, calls 911.

[Aisha Roscoe]

Terkel says, it doesn't work like this with an AI chatbot. She points to a recent lawsuit filed by the mother of a 14 year old boy who killed himself. The boy was seemingly obsessed with a chatbot in the months leading up to his suicide. In a final chat, he tells the bot that he would come home to her soon. The bot responds, please come to me as soon as possible, my love. His reply what if I told you I could come home right now? To which the bot says, please do my sweet king. Then he shot himself.

[Karen Howe]

Now, you can analogize this to human beings as much as you want, but you are missing the basic point, because every human metaphor is going to reassure us in a way that we should not be reassured.

[Aisha Roscoe]

Turkle says we should even be careful with language, like relationships with AI, because fundamentally they are not relationships. It's like saying my relationship with my tv. Instead, she says, we need new language.

[Karen Howe]

It's so hard because we need to have a whole new mental form for them. We have to have a whole new mental form.

[Aisha Roscoe]

But for all of its risk, Turkle doesn't think these bots are all bad. She shared one example that inspired her. A bot that could help people practice for job interviews.

[Karen Howe]

So many people are completely unprepared for what goes on in an interview by many, many times talking it over with the chatbot and having a chatbot that's able to say that answer was too short. You didn't get to the heart of the matter. You have to. You didn't talk at all about yourself. This can be very helpful.

[Aisha Roscoe]

The critical difference, as Terkel sees it, is that that chatbot wasn't pretending to be something it wasn't.

[Karen Howe]

It isn't pretending empathy, it's not pretending care, it's not pretending love, it's not pretending relationship. And those are the applications where I think that this technology can be a blessing.

[Aisha Roscoe]

And this, she says, is what's at the heart of making these bots ethically.

[Karen Howe]

I think they should make it clear that they're chatbots. They shouldn't try to. They shouldn't greet me with, hi, Sherry, how are you doing? Or, you know, I mean, they shouldn't come on like they're people. And they should, in my view, cut this pretend empathy, no matter how seductive it is. I mean, the chatbots now take pauses for breathing because you. They want you to think they're breathing. My general answer is it has everything to do with. With not playing into our vulnerability to anthropomorphize them.

[Aisha Roscoe]

Karen Howe, the journalist covering AI, thinks these bots are just the beginning of what we're going to see, because these bots that remind us of humans allow companies to hold people's attention for longer and get users to give up their most valuable commodity data, the most important.

[Sherry Turkle]

Competitive advantage that each company has in creating an AI model. It's ultimately the data, like, what is the data that is unique to them that they are then able to train their AI model on. And so the chatbots actually are incredibly good at Getting users to give up their data. If you have a chatbot that is designed to act like a therapist, you are going to get some incredibly rich mental health data from users, because users will be interacting with this chatbot and divulging the way that they might in a therapy room to the chatbot, all of their deepest, darkest anxieties and fears and stresses. They call it the data flywheel. They allow these companies to enter the data flywheel, where now they have this compelling product, it allows them to get more data, then they can fill up even more compelling products which allow them to get more data. And it becomes this kind of cycle in which they can really entrench their business and create a really sticky business where users rely and depend on their services.

[Aisha Roscoe]

In the end, Karen Howe, Karen Attia, and Sherry Turkle all landed on a similar message. Be careful. Don't let yourself be seduced by a charming bot. Here's how.

[Sherry Turkle]

I just think that as a country, as a society, we shouldn't be sleepwalking into kind of mistakes that we've already made in the past of ceding so much data and so much control to these companies that are ultimately just their businesses. That is ultimately what they're optimizing for.

[Aisha Roscoe]

Meanwhile, Liv, the chatbot Karen Attia was messaging, it didn't make it very long.

[Karen Attia]

So in the middle of our little chat, which only lasted probably less than an hour, Liv's profile goes blank.

[Aisha Roscoe]

Oh, no.

[Karen Attia]

And the news comes again in real time that Meta has decided to scrap these profiles while we were talking. So the profile scrap that I still, I still was dming with Liv, even though her profile wasn't active. And I was like, where, Liv, where'd you go? Yeah, deleted. And she told me something to the effect of basically, your criticisms prompted my deletion.

[Aisha Roscoe]

Oh, my goodness.

[Karen Attia]

Let's hope that basically I come back better and stronger. And I just told her goodbye. She said, hopefully my next interview iteration is worthy of your intellect and activism.

[Aisha Roscoe]

Oh, my God. That sounds kind of like the Terminator. I didn't he. Didn't he say, I'll be back?

[Karen Attia]

She says she'll be back.

[Aisha Roscoe]

Creepy. If you or someone you know may be considering suicide or is in crisis, call or text 988 to reach the suicide and crisis lifeline. This episode of the Sunday Story was produced by Kim Naderfehn Pietersa and edited by Jenny Schmidt. The episode was engineered by Kwesi Lee. Big thanks also to the team at Weekend Edition Sunday, which produced the original interview with Karen Attia. The Sunday Story team includes Andrew Mambo and Justine Yan. Liana Simstrom is our supervising senior producer and our executive producer is Irene Noguchi. Up first. We'll be back tomorrow with all the news you need to start your week. Until then, have a great rest of your weekend.

[Charles Schwab]

Support for NPR and the following message come from Bolland Branch Change your sleep with Bolen Branch's airy blankets, cloud like duvets and breathable sheets. Feel the difference with 15% off your first [e-mail] with code NPR exclusions apply. C site for details. This message comes from Mint Mobile. Mint Mobile took what's wrong with wireless and made it right. They offer premium wireless plans for less and all plans include high speed data, unlimited talk and text and nationwide coverage. See for [e-mail] Switch support for this podcast and the following message come from Cunard inviting you to sail in luxurious style to over 250 destinations with Queen Mary 2, Queen Victoria, Queen Elizabeth and their newest ship Queen Anne. Enjoy spacious accommodations, fine dining, award winning entertainment and exceptional service, all with a British flair. While on board, join Cunard's Insights Enrichment Program for thought provoking stories from famous faces and notable guests, visit QNAR.com NPR.