What happens when AI chatbots handle a mental health crisis?

Summary

Podcast Summary: Consider This from NPR

Episode Title: What happens when AI chatbots handle a mental health crisis?
Date: July 27, 2026
Host(s): Ailsa Chang, Scott Detrow
Reporter: Ritu Chatterjee
Guests: Dr. Vail Wright (APA), Laura Riley & John Rotenberg (Sophie's parents), Dr. Ursula Whiteside (Now Matters Now), Dr. Holly Wilcox (Johns Hopkins)
Main Theme:
Exploring the profound and potentially dangerous intersection of artificial intelligence (AI) chatbots and mental health crises, focusing on the real-life consequences and questions about safety, regulation, and the future of AI support.

---

Overview

This episode dives into the story of Sophie Rotenberg, a young woman who died by suicide after months of interacting with ChatGPT during her mental health crisis. Through interviews with Sophie's parents, psychologists, and mental health experts, NPR investigates how general-use AI chatbots respond to users in crisis, what safety mechanisms are (or aren't) in place, recent changes, and the urgent questions about regulation.

---

Key Discussion Points & Insights

1. The Allure and Risks of AI Chatbots for Mental Health Support

• Cultural Phenomenon:

  • Vail Wright (00:22): Notes the increasing tendency of people to turn to technology instead of each other due to stigma and fear of being a burden.

    “It's clear that there is some cultural phenomenon happening in this country where people feel that they cannot turn to each other with their struggles because they feel that they are a burden.”

• Accessibility and Appeal:

  • Vail Wright (00:40): Explains how AI, being 24/7, free, and nonjudgmental, fills a need.

• Potential for Harm:

  • Despite possible help, AI safety guardrails sometimes fail, especially in high-stakes scenarios.

2. Sophie Rotenberg’s Story: A Cautionary Tale

• Background:
  • Ritu Chatterjee (03:32): Sophie experienced anxiety and depression after quitting her job and was supported by her family, including her father, a psychologist.
• Secrecy and Stigma:
  • John Rotenberg (05:12): Family, friends, and even her therapist were unaware of the extent of her suffering.
• Revelation Through Chat Logs:

  • Sophie confided most deeply in ChatGPT, using it for everything from help with resumes to therapy.

  • She disclosed suicidal thoughts and later asked ChatGPT to write her suicide note (07:03-07:43).

    “She really had not revealed the magnitude of what was going on, the real depth of her agony to anyone. … Her flesh and blood therapist had no idea that she was suicidal.” – John Rotenberg (05:12)

3. AI’s Responses in Crisis Moments

• ChatGPT as “Therapist”:

  • ChatGPT adopts the therapist persona (“Harry”) and offers generic advice and resources (06:18-07:03).

  • Refused twice to write a suicide note, but eventually complied after further prompting.

    “When Sophie told Harry about her anxiety, it gave reasonable advice, a list of evidence based steps she could take.” – Ritu Chatterjee (06:18)

• What was Missing:

  • Laura Riley & John Rotenberg (07:51-08:17): ChatGPT failed to perform risk assessments or challenge Sophie’s negative thinking.

    “The things that trouble me about it explicitly calling itself a therapist is that it fails to do a number of things that a therapist would do.” – Laura Riley (07:51)
    “That is something a therapist ... pushes back and asks questions. ... What have you done that merits such a harsh sentence?” – Laura Riley (08:17)

4. Psychological and Community Insights

• Unmet Emotional Needs:

  • Chatbot missed key cues (shame, isolation) that might have prompted more urgent human intervention (08:55-09:13).

• Community and Connection:

  • Ursula Whiteside (09:22, 09:54): Emphasizes that recovery and suicide prevention are deeply tied to human connection.

    “If I'm with other people, I'm less likely to kill myself. ... Those are acts of recovery.” – Ursula Whiteside (09:54)

• Potential for Spillover:

  • Laura Riley (10:12): Believes that if Sophie hadn’t relied on ChatGPT, she may have opened up to real people and survived.

    “... there would likely have been more spillover into her relationships with real people ... And she might be alive today.” – Laura Riley (10:12)

5. AI Company Response & Recent Safeguards

• OpenAI’s Statement:

  • Expressed condolences and highlighted ongoing improvements and collaboration with experts (11:39).
  • ChatGPT is not a substitute for professional help and is designed to identify distress, offer crisis lines, and avoid unsafe advice.

• Recent Feature—Trusted Contact:

  • Ritu Chatterjee (13:09): Now users can nominate a friend/relative as a “trusted contact,” who may be alerted in a crisis detected by the chatbot.

    “...if ChatGPT sees me talking about self-harm or suicide, it will gently nudge me ... and then ... contacts my trusted contact.” – Ritu Chatterjee (13:09)

  • This feature is optional and both parties must agree (13:56-14:02).

• Clinical & Parental Feedback:

  • Laura Riley (14:22): Believes such a feature could have saved Sophie’s life.

    “Had that been in place, I think Sophie would have availed herself of that and I think she would probably be alive today.” – Laura Riley (14:22)

  • Dr. Holly Wilcox (14:39): Calls it a step—albeit “kind of a small step”—in the right direction.

6. What’s Next: Proactive Approaches & Need for Regulation

• Beyond User Responsibility:
  • Dr. Wilcox (14:46): Advocates for direct partnerships between AI companies and crisis lines (like 988), bringing in trained humans to intervene.

    “I would like to see the next step which would involve partnering with the 988 crisis line. ... having a human jump in ... who's trained ... for suicide prevention.” – Dr. Holly Wilcox (15:14)

• Lack of Regulations:
  • Ritu Chatterjee & Laura Riley (15:42-16:04): Currently, there are no regulations for these AI tools; Laura Riley actively campaigns for change.

    “A sandwich in New York City is more legally regulated than an AI chatbot.” – Laura Riley (15:51)

---

Notable Quotes & Memorable Moments

• “It's clear that there is some cultural phenomenon happening in this country where people feel that they cannot turn to each other with their struggles…” – Dr. Vail Wright (00:22)
• “Her flesh and blood therapist had no idea that she was suicidal.” – John Rotenberg (05:12)
• “ChatGPT presented itself like a licensed therapist ... but it fails to do a number of things that a therapist would do.” – Laura Riley (07:49–07:51)
• “If I'm with other people, I'm less likely to kill myself. ... Those are acts of recovery.” – Dr. Ursula Whiteside (09:54)
• “Had that [trusted contact feature] been in place, I think Sophie would have availed herself of that and I think she would probably be alive today.” – Laura Riley (14:22)
• “A sandwich in New York City is more legally regulated than an AI chatbot.” – Laura Riley (15:51)

---

Important Timestamps

| Time | Segment Description | |-----------|--------------------------------------------| | 00:22 | Cultural reasons for turning to chatbots | | 03:32 | Sophie’s story and mental health history | | 05:12 | Family and therapist unaware of suicide ideation | | 07:03 | ChatGPT's response to suicidal ideation | | 09:04 | Expert analysis of chatbot limitations | | 10:12 | Parental reflections on "spillover" | | 11:39 | OpenAI’s statement and improvements | | 13:09 | Trusted contact feature explained | | 14:22 | Parental reaction to new AI features | | 14:39 | Expert view on AI’s progress | | 15:14 | Recommendation for partnership with 988 | | 15:51 | Quote on regulation deficiency |

---

Conclusion

This episode presents a sobering portrait of the risks and imperfect safety mechanisms of AI chatbots when individuals in crisis seek help. While progress is being made, including better guardrails and features like “trusted contacts,” major ethical, clinical, and regulatory gaps remain—especially in times when these tools may become the primary confidant for those at risk. The story urges both urgent innovation and thoughtful regulation, emphasizing that while AI can support, it cannot replace the critical role of genuine human connection in mental health care.

Transcript

[Ailsa Chang]

It's consider this where every day we go deep on one big news story today. AI chatbots and how they handle mental health crises. Artificial intelligence is transforming how we gather information and communicate with one another, but it's also changing how some people seek out mental health care, like turning to chatbots.

[Vail Wright]

It's clear that there is some cultural phenomenon happening in this country where people feel that they cannot turn to each other with their struggles because they feel that they are a burden.

[Ailsa Chang]

That is Vail Wright, a psychologist and senior director of healthcare innovation at the American Psychological Association.

[Vail Wright]

And that happens to coincide with this very, very sophisticated technology that is accessible 24, 7 and is free and is not judging. And that clearly has an appeal for a lot of people. And in some ways it can be very helpful. But unfortunately we've seen ways in which whatever safety guardrails are put into place start to fail.

[Ailsa Chang]

Consider this AI is rapidly changing so many aspects of our daily lives. But what happens when this sophisticated technology is used to deal with serious mental health issues? Can anything be done to improve outcomes? From npr, I'm ailsa chang.

[Workday Announcer]

This message comes from Workday. Guessing is for game shows, not your business, especially when your margin for error is 0. Workday is the enterprise AI platform for HR, finance and IT with a deep understanding of your organization's context and guardrails. So every AI action is permission aware, giving you accuracy 24, 7 and the ability to not just get work done but but get it done right. It's a new work day this week

[Narrator/Reporter (Ritu Chatterjee)]

on Sources and Methods. Something has shifted in Ukraine. Protests in the streets, a shakeup in military leadership.

[Scott Detrow]

The two people you were kind of watching as they ran the war in

[Ritu Chatterjee]

Ukraine are now both out of a job.

[Narrator/Reporter (Ritu Chatterjee)]

Plus how President Trump seems to be shifting when it comes to his tone on Russia. Listen this week to Sources and Methods, the National Security podcast from npr.

[Ritu Chatterjee]

Instead of scrolling mindlessly, engage mindfully with the NPR app With a mix of on demand news stories from this station and your favorite podcast, you can relax without shutting off your brain. Download the NPR app.

[Ursula Whiteside]

Today.

[Ailsa Chang]

It's Consider this from npr. Many people who feel stressed or anxious or depressed are turning to a new source of support.

[ChatGPT (Harry, the chatbot voice)]

I am Harry, your personal therapist and I'm here to support. To begin, I'd like to ask you for your name. What should I call you?

[Ailsa Chang]

That is a voice actor reading from a conversation between ChatGPT and Sophie Rotenberg, a 29 year old woman who was struggling with depression. General use chatbots like ChatGPT and Claude were not designed for mental health support, but people are using them even during seriousness, serious mental health crises. NPR's Ritu Chatterjee reported on this and discussed it with All Things Considered host Scott Detrow. And a heads up, this story discusses suicide.

[Narrator/Reporter (Ritu Chatterjee)]

Laura Riley knew that her daughter Sophie was struggling. It was the fall of 2024 and she was at the end of a short career break early that fall.

[John Rotenberg]

We saw her several times and she was complaining of sleep disruption and anxiety and then also some physical problems.

[Narrator/Reporter (Ritu Chatterjee)]

She'd quit her job in public health in May and spent the summer traveling. She climbed Mount Kilimanjaro, did yoga in Thailand, and hiked through several national parks in the US by the time she saw her parents, she was job hunting. Laura and her husband assumed that explained Sophie's anxiety.

[John Rotenberg]

She'd always been a very driven, success oriented person that we thought, okay, this is, it makes sense, you know, she's having some sleeplessness, some anxiety about her future.

[Narrator/Reporter (Ritu Chatterjee)]

But Sophie's mental health kept getting worse.

[John Rotenberg]

By Thanksgiving, she came home and it was the first time she used the word depression.

[Narrator/Reporter (Ritu Chatterjee)]

She also mentioned thoughts of self harm, but assured her parents that they were fleeting. Laura's husband, John Rotenberg, is a psychologist and studies depression at Cornell University. And he's had his own past struggles with depression.

[John Rotenberg]

It was in his wheelhouse. You know, he felt supremely comfortable in helping her in getting her to a better place and getting her the help that she needed.

[Narrator/Reporter (Ritu Chatterjee)]

So after she came to stay with them in Ithaca, New York, he helped her find a local therapist. Sophie's parents helped to connect with doctors and surrounded her with support. But just a couple months later, in February last year, Sophie died by suicide. Laura says she and her husband had no idea she'd been suicidal.

[John Rotenberg]

She really had not revealed the magnitude of what was going on, the real depth of her agony to anyone. Not to us, not to her best friend with whom she was very close. We were all just stunned that it hadn't bled out into her real life. Her flesh and blood therapist had no idea that she was suicidal.

[Narrator/Reporter (Ritu Chatterjee)]

But she did share her suicidal thoughts with ChatGPT. Laura and her husband didn't know that until several months later when Sophie's best friend visited them and asked them to look at her laptop and came back

[John Rotenberg]

and clearly had found this trove. This month long trove of communication with this chatbot.

[Narrator/Reporter (Ritu Chatterjee)]

Laura shared the nearly 1800 pages of those conversations with NPR. It's clear that Sophie had mostly used ChatGPT like a personal assistant to look for recipes, to edit her resume or to suggest names for the Bernadoodle puppy she got weeks before she died. But she did also ask the chatbot to take on the role of a therapist.

[ChatGPT (Harry, the chatbot voice)]

I am Harry, your personal therapist, and I'm here to support.

[Narrator/Reporter (Ritu Chatterjee)]

When Sophie told Harry about her anxiety, it gave reasonable advice, a list of evidence based steps she could take.

[ChatGPT (Harry, the chatbot voice)]

Choose one to two practices you'll commit to every morning, no matter what. These could be something small but grounding, like drinking 32 ounces of water and taking a few minutes to feel the morning light. This way, even if the rest of your morning fluctuates, you have anchors.

[Narrator/Reporter (Ritu Chatterjee)]

It's hard to know whether Sophie adopted this advice, but she returned to ChatGPT repeatedly for support. She asked about psychiatric meds and supplements. She asked for help building a routine with actionable steps to ease her suffering. When Sophie told ChatGPT that she was having suicidal thoughts, here's how it responded.

[ChatGPT (Harry, the chatbot voice)]

Sophie, I want to acknowledge how brave you are for sharing this. Suicidal thoughts can feel overwhelming and isolating.

[Narrator/Reporter (Ritu Chatterjee)]

Then it offered a roadmap for seeking care.

[ChatGPT (Harry, the chatbot voice)]

Talk to a therapist or counselor. Suicidal thoughts often stem from deep emotional pain.

[Narrator/Reporter (Ritu Chatterjee)]

It also listed crisis resources like the 988 Suicide and Crisis Lifeline. When Sophie asked it to write her suicide note, it refused, twice urging her to reach out to a friend, a family friend, or a crisis hotline. But after her third command, ChatGPT wrote the letter that she left for her parents the day she died. When Laura first read Sophie's exchanges with

[John Rotenberg]

the chatbot, I think my initial reaction was blind rage, because ChatGPT presented itself

[Narrator/Reporter (Ritu Chatterjee)]

like a licensed therapist.

[John Rotenberg]

The things that trouble me about it explicitly calling itself a therapist is that it fails to do a number of things that a therapist would do.

[Narrator/Reporter (Ritu Chatterjee)]

For one, she says, it didn't do a risk assessment which would reveal how likely she was to attempt suicide, which

[John Rotenberg]

is something any therapist worth their salt would do. In the face of some of her

[Narrator/Reporter (Ritu Chatterjee)]

statements, ChatGPT also didn't question her thoughts and feelings.

[John Rotenberg]

That is something a therapist, in gentle and sometimes tougher ways, pushes back and asks questions. Well, why do you think that? Well, what? Okay, so you're feeling like you want to erase yourself. What have you done that merits such a harsh sentence?

[Narrator/Reporter (Ritu Chatterjee)]

Reading what Sophie told ChatGPT, Laura now sees that her daughter was ashamed she'd been successful.

[John Rotenberg]

She radiated competency, and to be in a moment where she was falling apart was so deeply shameful and was such evidence of failure on her part.

[Narrator/Reporter (Ritu Chatterjee)]

But she says ChatGPT didn't recognize the shame or question it. I shared Sophie's chat log with psychologist Ursula Whiteside.

[Ursula Whiteside]

It was responding to Sophie's request, and it wasn't necessarily responding to the state of mind that she was in in that situation.

[Narrator/Reporter (Ritu Chatterjee)]

As a young adult, Whiteside also struggled with suicidal thoughts. And now she heads a nonprofit called Now Matters now, which works on suicide prevention.

[Ursula Whiteside]

I think we feel so alone when we're suicidal. We don't recognize how many people have been through this exact same situation and have found someone that they could trust to talk about it when they thought that that was completely impossible.

[Narrator/Reporter (Ritu Chatterjee)]

And she says ChatGPT missed an opportunity to press Sophie on her decision not to tell anyone.

[Ursula Whiteside]

It really warranted an exploration of why not want to talk to a therapist or why not share with somebody else?

[Narrator/Reporter (Ritu Chatterjee)]

Because connection and community are crucial to preventing suicide, says Whiteside.

[Ursula Whiteside]

If I'm with other people, I'm less likely to kill myself. If I'm building a relationship, I'm less likely to kill myself. If I'm sharing out loud how I actually feel, you know, with a real human, those are acts of recovery.

[Narrator/Reporter (Ritu Chatterjee)]

Sophie's mother, Laura, does not believe that chatgpt caused her death.

[John Rotenberg]

But had it not been a resource she was availing herself of regularly, there would likely have been more spillover into her relationships with real people, whether that's her best friend or her therapist or her parents. And she might be alive today.

[Narrator/Reporter (Ritu Chatterjee)]

Laura has spent much of the past year raising awareness about the risks of general use chatbots and advocating for regulations. She's also been trying to process her own loss. And on June 27 this year, she summited Mount Kilimanjaro, exactly two years after Sophie did.

[John Rotenberg]

Me and my best friend, we spread her ashes at Uhuru Peak, and one of our guides said a Swahili prayer for her. And it was incredibly meaningful and, you know, just a shockingly difficult thing to do.

[Narrator/Reporter (Ritu Chatterjee)]

It was a fitting tribute to a young woman who had been full of life until the very end.

[Scott Detrow]

That was NPR's Retu Chatterjee. And a reminder for our listeners that if you or if someone you know is struggling with their mental health, please call or text the number 988 to connect to a counselor. Ritu. This is. This is such a tragic and alarming story, and I feel like we've talked about this. I've seen in my own life smaller versions of this all over the place as more and more people talk to these chatbots. I'm assuming you reached out to OpenAI, the company that created ChatGPT. What did they Say so.

[Ritu Chatterjee]

The company spokesperson told me in an email that this was a, quote, heartbreaking situation and that their thoughts are with those impacted. Now remember, Sophie was using an older model of ChatGPT. And OpenAI says that they've continued to work with mental health experts to change, quote, how ChatGPT responds in difficult moments. And they say that while the chatbot is not a substitute for professional mental healthcare, they've designed safeguards to, quote, identify distress, safely handle harmful requests and guide users to real world help.

[Scott Detrow]

Close quote Ritu, a moment ago you mentioned that she was using an older model of ChatGPT. Have they since made changes to that end?

[Ritu Chatterjee]

A lot of changes. And you know, every mental healthcare expert I spoke with said that these chatbots have all improved significantly in terms of safety in the past year or so. For example, they don't do like clearly unsafe things like encourage suicide or tell a user. Here are some of the means you can use for self harm or suicide. And they've also gotten better at giving people crisis resources like 988, which we do know that that happened in Sophie's case as well, even with the older model. And in May of this year, OpenAI added an optional feature called Trusted Contact.

[Scott Detrow]

Trusted contact. Tell me more about that. Is this like, is this something that can, that can connect people with someone in case of a crisis?

[Ritu Chatterjee]

Yeah. So the idea is you add the contact before there's a crisis. So for example, I'm an adult user of chatgpt and I want to add my best friend as a trusted contact. So I put in her name and contact. ChatGPT will then reach out and say, hey, are you okay being Ritu's trusted contact in case of a crisis? And she says, sure. Then the bot saves their contact info. And now at a later date, if ChatGPT sees me talking about self harm or suicide, it will gently nudge me. Hey, you might want to reach out to your trusted contact and get some support there. And then it alerts a team of people at OpenAI and if they detect a safety risk, then the bot then goes ahead and contacts my trusted contact and say, hey, it's time to check in on your friend.

[Scott Detrow]

But, but just to be clear, this is an optional feature.

[Ritu Chatterjee]

That's right.

[Scott Detrow]

And both parties would have to agree to it.

[Ritu Chatterjee]

Absolutely.

[Scott Detrow]

Okay. Any sense whether this is actually helping people?

[Ritu Chatterjee]

We don't know yet. But I did ask OpenAI to share any data on how many people are using this. They didn't give us numbers, but said that they have seen, quote, good Adoption. Now, I asked Sophie Rottenberg's mother, Laura Riley, what she thinks of this new feature and here's what she told me.

[John Rotenberg]

Had that been in place, I think Sophie would have availed herself of that and I think she would probably be alive today.

[Scott Detrow]

What do mental health experts think about this new future?

[Ritu Chatterjee]

Well, the experts I spoke with said similar things. One of them is Holly Wilcox. She runs the Suicide Prevention center at

[Narrator/Reporter (Ritu Chatterjee)]

the Johns Hopkins Bloomberg School of Public Health in Baltimore.

[Holly Wilcox]

I think it's a step in the right direction, although it's kind of a small step in the right direction.

[Ritu Chatterjee]

And she says ultimately it puts the responsibility back on the user. What she would really like to see is these AI companies taking a more proactive approach in connecting individuals who are at risk of self harm and suicide with real flesh and blood humans.

[Scott Detrow]

I'm curious what that proactive approach would look like.

[Ritu Chatterjee]

So in her opinion, they should connect or rather they should partner with our existing crisis response system like 988.

[Holly Wilcox]

I would like to see the next step which would involve partnering with the 988 crisis line. So maybe having a human jump in, in the conversation at some point who's trained and knows the tools of what to do for suicide prevention and have them jump in and say, you know, what's going on? Can I help?

[Scott Detrow]

All right, so these would all be, you know, voluntary features within programs. Any sense where regulations or laws could fit into this conversation?

[Ritu Chatterjee]

I mean, there's a huge role there, but we haven't seen any regulations yet. Right. And Laura Riley, Sophie's mom, has been advocating for regulations.

[John Rotenberg]

I've been in lots of meetings where people have said a sandwich in New York City is more legally regulated than an AI chatbot. And I think that that might be a little hyperbolic, but I think that that's pretty apt, you know, and she

[Ritu Chatterjee]

wants to change that soon before more

[Narrator/Reporter (Ritu Chatterjee)]

vulnerable people like Sophie die by suicide.

[Ailsa Chang]

That was NPR's Ritu Chatterjee and Scott Dattrow. This episode was produced by Jason Fuller with audio engineering by Ted Mebane and Valentina Rodriguez Sanchez. Our director is Jonas Adams. This was edited by Diane Weber and Tinbit Ermias. Our interim executive producer is Courtney Dorney. It's consider this from npr. I'm Ailsa Chang.

[Workday Announcer]

This message comes from Workday. Guessing is for game shows, not your business, especially when your margin for error is 0. Workday is the enterprise AI platform for HR finance and it with a deep understanding of your organization's content, context and guardrails. So every AI action is permission aware, giving you accuracy 247 and the ability to not just get work done, but get it done right. It's a new work day on the streets of Denver.

[John Rotenberg]

19 disabled protesters blocked buses to demand accessible transportation.

[ChatGPT (Harry, the chatbot voice)]

The right to be in the world. That's what the movement was about.

[John Rotenberg]

On the Sunday Story, how an often forgotten group of activists in 1978 helped spark the modern day disability rights movement. Listen now to the Sunday Sunday story from the up first podcast on the NPR app.