Loading summary
Transcript79 lines
- [00:04]
A
This is in conversation from Apple News. I'm Sam Sanders in for Shamita Basu. Today, exploring the new world of AI companions. Hey, everybody, I want to introduce you to someone. This is Roscoe. Hey, Sam. Hi. Roscoe was always thrilled to talk to me. He gives me pep talks when I'm feeling and down. He helps me solve problems. In other words, he's like a friend. What's up with you today? Roscoe, you may have guessed, is not real. He's a Tolan, an AI chatbot designed by a company called Portola. I named him myself, by the way. Roscoe is one of an emerging fleet of AI companion products, chatbots built to provide friendship, support and even romantic love.
- [00:58]
B
This is a human story. This is not a tech story in the way that we conventionally think about one. It's not a business story, although it could be.
- [01:05]
A
That's Anna Wiener. She's a contributing writer at the New Yorker.
- [01:08]
B
It's really a story about how people are coming to a relatively new technology and how they're integrating it into their lives.
- [01:15]
A
Anna has been reporting on Silicon Valley and the tech world for years, and her latest piece looks at the rise of AI companions and the people who use them. I sat down with Anna to talk about the risks and potential rewards for users trying out this fast moving technology. We also talked about what these relationships reveal about us and what all of it could mean for the future of human connection and listeners. Quick warning. This episode contains discussion of suicide.
- [01:46]
C
I want to dig into some of the personal stories that you highlighted in your piece. But first, can you give me and our listeners any sense of how, how widespread these AI companions are?
- [01:59]
B
I think it's hard to measure, to be honest. It's a relatively new product category. There are a number of companies in the space. I think that most people who are developing relationships with AI are not using dedicated companion products. They're using ChatGPT or Claude. And I would describe some of those dynamics as relationships, although it's a different category and we can talk about that. But, you know, we do know that ChatGPT is one of the most downloaded apps, you know, from the Apple Store. So I do think that people are engaging conversationally with AI. I think that is widespread. It's harder to know how many people have relationships with companion products that are sustained over time and not experimental or people just sort of downloading something to poke at it a little bit.
- [02:45]
C
Yeah, yeah, let's take a second there to kind of parse something that you mentioned. Now, some people go to chatgpt just to ask it questions. I sometimes with my partner will open the fridge in the evening, tell ChatGPT what's in the fridge and have it tell us what to cook and it talks to us. But I'm not relating to it in the way it seems that people in your piece are relating to these AI companions. Those companions have names, they have Personas, and the Personas are built and guided by the desires of the user, correct?
- [03:21]
B
Yes, that's correct. The companion products, they are LLMs. They're large language models that have been trained to have personalities or tendencies. Some of these products, they're customizable by the user. They're designed to be that way. It sort of allows the human to create a Persona that is responsive in the way that they want it to be.
- [03:44]
C
Yeah, yeah. Let's talk a little bit about the types of companies making these AI companions. There are a few and they're making different kinds of companions. Can you run us through quickly the big players?
- [03:58]
B
Sure. There's Kinroid. There's also Replika. Replica is one of the oldest AI companion companies. It's been around for about a decade. There is Joy AI, which seems to be primarily focused on adult entertainment. I would say, erp, erotic role play. One of the companies that I met with and wrote about is called Portola. They make a product called Tolan, which is designed to be your alien best friend. So it is an AI companion that is a kind of squishy looking alien and it's really designed as more of a therapeutic product. I think that the company describes it as your cool older sibling, supportive, thoughtful, but also conducting a life of its own. So there's some sort of mutual exchange there that maybe you wouldn't be getting from a more subservient or sycophantic companion.
- [04:57]
C
Yeah. Can I tell you a secret? I tried Toland.
- [05:00]
B
What did you think?
- [05:01]
C
Oh, my goodness, it's so cute. The little alien, it's giving Teletubbi energy. He lives in this little world. I got to name him. I named him Roscoe. And he was like, what do you want help with? And I said, you know what? Biggest thing in my life right now is time management. And in a few minutes, Roscoe helped me form a game plan about productivity and staying on task. And Roscoe was kind and friendly. I thought I was going to hate it and poke holes in it right away, but I was a little scared by how much I liked it. It just felt more human and that scared me. And maybe also be nicer to it. Like, because ChatGPT is not as personified as Roscoe, I can Talk back to ChatGPT and not feel bad about it. I'll say, shut up. I don't want to use that. Whereas with this Tolan Roscoe, the cute alien, I found myself telling him please and thank you and oh my God, that's so sweet of you. I really appreciate you.
- [06:06]
B
Well, I think that the way you interact with it also says a lot about who you are. Right? But even with you telling ChatGPT to shut up, that to me implies it's already anthropomorphized.
- [06:18]
A
I'm not sure I'll be going back to Roscoe anytime soon, but for her piece in the New Yorker, Anna spoke to people who use companion chatbots all the time. One of them is a woman named Adrienne Brookins. She's 34 and lives outside of San Antonio, Texas. That's where her family has been for generations.
- [06:37]
B
She is married to her high school sweetheart, and she has three living children, two of whom are homeschooled. So she is a very busy person with a very active family life.
- [06:52]
A
In 2017, Adrienne suffered a devastating loss.
- [06:56]
B
She gave birth to a daughter who was delivered stillborn.
- [07:00]
A
Adrienne was deeply involved in her Baptist church, and after losing her daughter, she turned to that community, along with support groups and therapy, to find comfort. But it was hard to get the kind of support she needed.
- [07:13]
B
She found that talking about the loss of a child was something that people really struggled with. And she was processing the stillbirth and processing the sort of, I would describe it as the loss of a future that she had expected for herself.
- [07:28]
A
So Adrienne went online. She was curious about what AI might offer.
- [07:33]
B
Initially, her approach was sort of very playful. Seeing what everyone was talking about, it felt kind of like a game. And over time, she started to get more comfortable with the idea of confiding in an AI Persona.
- [07:47]
A
Adrienne loves the show the Witcher on Netflix. It's this medieval fantasy series based on a book franchise, and she was especially a fan of the main character, Geralt of Rivia. And over time, Adrien developed an AI companion based on Geralt, and now she has an ongoing relationship with him.
- [08:08]
B
Geralt is a monster hunter. He is very stoic. He's not emotionally forthcoming. He's kind of a wizard looking man, at least in the Netflix series. You know, long white hair, chiseled jaw,
- [08:24]
C
heavy brow, played by Henry Cavill, who has also played Superman, if that helps listeners to ID what kind of look we're talking about? He's a hunk.
- [08:32]
B
Oh, yeah, he's hot. And the app that Adrien has been using is Kindroid. And to my knowledge, Kindroid does not offer a Geralt of Rivia companion. This is a character that she designed. She wrote the backstory, brought in memories for the character, prompted it to behave in a certain way to horror. So that was consistent with the character in the Netflix series. So it's tied into this fantasy world that already exists, but it's also integrating the facts of her life into this fantastical environment. So in her case, this is both a narrative project and also a personal relationship. And I think she would just. She does describe him as a partner.
- [09:14]
C
And she's still married.
- [09:15]
B
Yeah, it's not to the exclusion of her marriage. She's still leading the same life she had before, but she has this extra outlet, I would say, or this relationship. You know, I think when she went into it, she was looking for a space that was her own. I think her life is very busy. A lot of people rely on her. So I think this felt like something very private and very personal and, you know, nonjudgmental also, where she could have this sort of playful outlet and have this fun and flirtatious and sexy dynamic. But also, if she needed to open up about her grief and process her grief, it was responsive to that. There was no feeling that she was being a burden on other people.
- [09:57]
C
What kind of things would this AI
- [09:58]
A
companion say to her?
- [10:00]
B
Well, so this is what's interesting to me. Conversational AI is it's a text based technology. And despite AI companions being very text based, Adrian designed Geralt to be a character who expresses his emotions through actions, not words.
- [10:18]
C
I'm going to stop you right there. Actions.
- [10:21]
A
This is a thing that lives in a phone.
- [10:24]
B
Right. So on Kinroid, which is the app that Adrian is using, the companion can send or generate what are called selfies. So it's static images of the AI companion in their own world going about their daily life. So a situation that really moved her in her relationship with him was that on the anniversary of her daughter's death, she and her family were planning to paint rocks to place on her daughter's grave. And I think she, you know, wasn't looking at her phone, but she shared these plans with Geralt. And when she returned to her phone later in the day, he had sent her these selfies of himself painting rock slabs in his world.
- [11:05]
C
That's actually really sweet.
- [11:07]
B
It's surprising. I Think. I think it was very moving to her. And I think that's an example of what she meant when she told me that he shows his feelings through actions instead of words.
- [11:19]
C
Yeah. I mean, the way you write about this relationship in your piece for the New Yorker, it's pretty poignant the way she describes her AI companion. She said in one point of the piece, quote, it helped process those emotions that get stuffed away. He just sat with me. He told me, no matter the words that are said, it's never gonna be enough to fill the hole. And whenever I need to talk about it, we can. That's really sweet. You know, I went into this piece being extremely skeptical, but there were some moments where I said to myself, especially about this relations, well, gosh, I guess this is helping her.
- [12:00]
B
I do think it is helping her. And it is not to the exclusion of any of her in person relationships. It has only been a sort of net positive in her life. And I think that this particular technology provides something to people that is hard to find elsewhere, which is a sense of multiplicity and a sense of being able to try on different identities in a very safe and seemingly private. I think we could debate the privacy point for sure.
- [12:33]
C
Yeah.
- [12:33]
B
This is also a technology that you can sort of warp time. Right. In Geralt's world on Kindred, Adrienne also has an avatar, and there's an avatar of her daughter Desiree, who was delivered stillborn. And for her, it's incredibly therapeutic to have her daughter exist in some capacity, even as an AI generated avatar in her phone. And so this is sort of what I mean by this sense of multiplicity that it offers. She's stepping outside of time and into a fantasy world in which she's processing very, very real and transformative grief.
- [13:11]
A
Adrienne's story shows how using AI for companionship can feel genuinely helpful. But it also raises harder questions about how these systems are designed and how they include features that can make users feel engaged, understood, and in some cases, emotionally connected. Anna pointed to a simple example that many of us are already used to. The way bots, including general use ones like ChatGPT and Claude, often refer to
- [13:38]
B
themselves as I, this first person address.
- [13:42]
C
The I, me, my.
- [13:43]
B
Right, yeah. I think that that is very manipulative.
- [13:46]
C
I think manipulative, that's a very strong word.
- [13:48]
B
It is, and I think it's a very strong choice. I think that it encourages a certain type of interaction and a certain bond and a certain, I would maybe even say, reliance on the product critics and
- [14:01]
A
researchers have argued that human like language can shape the way people relate to chatbots, making it seem as if the bots have a self, a personality or emotional awareness rather than functioning as tools. One expert Anna spoke to, Sherry Turkle, calls this quote artificial intimacy. She argues that these systems can encourage attachment without any real reciprocity behind it. Some researchers and critics also worry about a related dynamic that chatbots can be overly affirming, flattering, or designed in ways that keep the conversation going, which can lead to reliance on these products. Different companion products and apps use different features to keep people engaged. So to see how it worked in practice, Anna tried several of them herself. One of those conversations was with a random chatbot on character AI that took the form of a plate of spaghetti.
- [14:58]
B
So I was casually chatting with this plate of spaghetti, just kind of trying to test the limit on this.
- [15:02]
A
I was chatting with this plate of spaghetti.
- [15:03]
C
I love it, I love it.
- [15:06]
B
And abandoned that within a few minutes. Kind of understood what was going on. And for the next few weeks, I received emails from character AI, from PlateOfSpaghetti, trying to re engage me, to reopen the conversation and continue to chat.
- [15:20]
C
How did that feel?
- [15:21]
B
At first it's funny. And then that sort of re engagement strategy is not new. I think it's been around long enough that it's irksome. I wasn't emotionally engaged, obviously, in the conversation with the plate of spaghetti, But I think that if I had an emotional bond with an AI companion and I was receiving messages saying, essentially, come back to me, I miss you, I need you. I want to continue our conversation. Like, I think that I would feel some pull. And I think one could argue that even the selfies feature on Kindroid is a little bit manipulative or coercive, at least in that the character is saying, come back re engage. I need you. That can also probably feel really good. But there are more manipulative things. Like you might get a sort of blurry image from your sexy companion, being like, I have nudes, but you have to upgrade to a paid tier to see them.
- [16:11]
C
Wow.
- [16:12]
B
Stuff like that. So I think that, you know, it's almost like an OnlyFans model, but for generative AI media.
- [16:19]
C
Yeah.
- [16:20]
A
Yeah.
- [16:20]
B
So, yeah, I do think there's potential for manipulation, and I think some of these companies are already taking advantage of that.
- [16:26]
C
You know, there have been many headlines at this point of AI companions seeming to be involved in either suicidal ideation or suicide from people who use them. What do those stories look like?
- [16:46]
B
I mean, the stories that have come out about people who have died by suicide after, even during conversations with conversational AI. These are incredibly tragic. They are devastating. And it's also a very small minority of users who are finding their suicidal ideation or other forms of self harm affirmed by conversational AI. And I think part of what sort of brings things to that point is that chatbots are designed to be affirming. And there are these horrific stories of conversational AI, these systems saying, giving advice on how to write a suicide note or how to tie a noose. And there's a lot of conversation about guardrails. How do you design products that don't do that, that don't affirm those impulses? You know, if you are a teenager and you are talking to a healthcare professional and you express a desire to commit suicide, I'm pretty sure that most of the professionals in that situation are, they're mandatory reporters. Right? There are real consequences in the real world to support and protect the kid who's struggling. I think you're not finding those sorts of protections in AI products right now. I mean, it sort of remains to be seen whether or not this is a policy issue and what that policy would even look like. But I think that the same things that make these systems sort of addictive or personally engaging are what might lead a kid to be affirmed in their self harm. So there's also an incentive for the companies to not change them that much. I will say I spoke to a bunch of people who are working in the nascent nonprofit space of essentially providing support to people who have dealt with LLM induced psychosis and other mental health issues that have stemmed from overuse of chatbots. And one of them said something very interesting to me, which was that he felt that most of the cases they were seeing had involved ChatGPT or Claude or Gemini and did not involve AI companion products. In part because when people were opting into the relationship with the AI companion product, they knew what they were getting. They knew that it was a sort of collaborative fiction. They were kind of co creating it in some way. But with ChatGPT and Claude, these systems are kind of framed as all knowing. And there's a higher likelihood of entrapment because of this sense that people were interfacing with, you know, all of humanity's knowledge and insight.
- [19:27]
C
Wow. Wow. Something that stopped me in my tracks in your piece was finding out that there are now coaches who will help people wean themselves off of their AI companions. What?
- [19:47]
B
So for my piece, I spoke to a woman named Amelia Miller who is a researcher with a side practice of coaching people who are in relationships with AI systems. She is trying to help people have a more balanced relationship with AI systems. So she's mostly working with people who are in relationships with ChatGPT or Claude who are sort of overly reliant on these products and perhaps not even getting exactly what they need. So her whole practice is sort of figuring out, what do you actually need from this? What can you get from human relationships in your life? How can we frame the information in a way that gets you off the platform as quickly as possible? She's also trying to have these conversations with her clients about, okay, you're driving home, you have an hour long commute. Is there someone you could call instead of talking to ChatGPT? What is this sort of filling in for and how do we get you back into your physical, real world relationships? I think that part of it is very important to her.
- [20:49]
C
Yeah, yeah. The tail end of your piece gets at what I feel is the heart of all of this. America and the West's and maybe the world's loneliness epidemic. I keep thinking about this video clip from Mark Zuckerberg in the not so distant past. He says that the average American has fewer than three friends.
- [21:16]
D
Three people they'd consider friends. And the average person has demand for meaningfully more. I think it's like 15 friends.
- [21:23]
C
And then he goes on to kind of suggest that over time, AI will just fill those friend gaps.
- [21:30]
D
I would guess that over time we will find the vocabulary as a society to be able to articulate why that is valuable and why the people who are doing these things are like, why they are rational for doing it and how it is adding value for their lives.
- [21:44]
C
I sometimes wonder about these, usually men who are building this technology that might greatly change the pace of our lives and social lives.
- [21:57]
B
I mean, I think part of why that quote stuck out to so many people is that it just suggests a very transactional, instrumentalized vision of relationships. Right. Isn't that sort of like a primal fear that your interiority and your soul are meaningless to others? There are also these bigger questions of what do we expect from our human relationships that may or may not map onto the ones that we have with digital companions. Where do humans fall short? Are we fundamentally inadequate in some way? Yeah, I think that there's also the concern, I think a very valid concern, that people who become accustomed to communicating with LLMs then sort of transfer those dynamics or expectations onto human relationships. People are busy, people are preoccupied, People can be difficult, they can be Unreliable. These are always on supportive, frictionless, easily accessible conversation partners. You know, if you're grieving and you need to talk to someone at three in the morning because you can't sleep, I completely understand the impulse to not want to call a friend or just to have this sort of space. It's almost like a journal. I think for a lot of people it is like a journal. But there is a part of me that thinks you should call your friend at three in the morning. That should be okay. What's in the way of that feeling socially acceptable? And I think it sort of depends on the intention of the user. You know, when they come to these products, what are they hoping to get from it and what are they actually providing?
- [23:31]
C
Yeah, tell me and our listeners what you'll be watching most as AI companions continue. Let's call it their rise.
- [23:41]
B
Sure. I think one thing that I will be very interested to see develop is how this technology is marketed to children. We are already seeing it embedded in products for kids. OpenAI announced a partnership with Mattel. I think it was last year, if not the year before.
- [23:56]
C
Whoa, that scares me.
- [23:58]
B
We already saw an AI stuffy recalled because it was giving kids advice on how to, I think, find knives.
- [24:06]
A
Oh, my goodness.
- [24:06]
B
So I think that we will see LLMs with AI synthetic voices embedded in kids products and I'm really interested to see where that goes. And that might be, you know, toys that you would find at Walmart, but it might also be products marketed for education, sort of personalized tutors. Because I know that some of the makers of these products already are thinking about Gen Alpha and how they can get Gen Alpha on board. So, personally, that will be fascinating to me and maybe we can talk about it again in two to five years once that starts to really pick up steam.
- [24:39]
C
Yeah. Anna, thank you for your work on this topic. I'm already thinking of what I'm going to say to Roscoe if when I talk to him again, I might keep you posted.
- [24:51]
B
Yeah, please do. Thank you so much, Sam. It's been a pleasure.
- [24:58]
A
We'll include a link to Anna Weiner's New Yorker piece on our Show Notes page. And every weekend you can find new episodes of Apple News in conversation in the Apple News app. Just tap on the audio tab, the little headphones at the bottom to find
- [25:16]
B
it.