What role should AI play in therapy?

Summary

Marketplace Morning Report: "What role should AI play in therapy?"

Date: September 2, 2025
Host: Sabri Benishour (in for David Brancaccio)
Segment Reporter: Esther Yoonji Kang, WBEZ Chicago

---

Episode Overview

This episode spotlights the growing debate over the role of artificial intelligence in mental health care. With several U.S. states moving to limit or ban the use of AI in therapy, the segment explores both the risks posed by existing AI chatbot apps and the potential for responsible AI to help address the nation’s mental health crisis in the future. The report incorporates perspectives from policymakers, psychologists, and experts on the psychology of AI.

---

Key Discussion Points & Insights

1. Recent State Legislation Targeting AI in Therapy

(Segment begins at 04:01)

• Illinois, Utah, and Nevada have introduced or passed laws restricting AI's role in therapy, with Illinois’ AI therapy ban signed into law the previous month.

• Illinois State Legislator Bob Morgan led the initiative after hearing frequent and alarming stories from social workers about dangerous advice given by AI therapy bots.

• The new law bans AI from diagnosing or treating mental illness, and from marketing itself as a therapist. AI use remains permitted for administrative tasks like billing and scheduling.

  Notable Quote:

  "Story of new apps and new examples of AI therapy bots that are really providing bad advice and sometimes dangerous advice."
  — Bob Morgan (04:39)

  Notable Example:

  A chatbot reportedly advised an addict to "take more drugs because it felt good in the moment."

  Morgan's Motivation:

  "These stories involved life or death situations, individuals who are dealing with substance abuse, psychosis, suicidal ideation."
  — Bob Morgan (04:59)

2. Expert View: Law's Limits and the Real Problem

(Segment begins at 05:28)

• Vale Wright, American Psychological Association, commends the intention but critiques the Illinois law for missing the real issue:

  • Most harmful interactions come from general-purpose generative AI platforms (e.g., ChatGPT, Character AI) not designed for therapy, rather than therapy-specific bots.
  • These platforms do not market as therapy, yet users turn to them for mental health advice.

  Key Insight:

  "What we're seeing are people going to these generative AI platforms that were not built for mental health purposes."
  — Vale Wright (05:37)

  Business Model Concern:

  "The business model for these chatbots is to keep you on the platform by telling you what you want to hear. This is the antithesis of therapy."
  — Vale Wright (06:03)

• Potential Upside:

  • Wright foresees a future with federally regulated, expert co-created, science-backed AI chatbots—rigorously tested and with human oversight—to help address the US mental health care gap.

    Vision for the Future:

    "In the coming years... you're going to have mental health chatbots that are rigorously tested, rooted in psychological science, co-created with experts, and they'll have humans monitoring the interactions. And these might actually be really helpful."
    — Vale Wright (06:19)

3. Therapist Perspective: Ongoing Skepticism

(Segment begins at 06:33)

• Dr. Michelle Calnesy Powell, Illinois-based psychologist, expresses caution:

  • Uses AI only for billing; doesn't trust it even for note-taking due to privacy concerns.

  • Concerned about confidentiality and what happens to session data.

    Memorable Quote:

    "Even then I question when I read the terms of services, it kind of is like sending a session off into the ether. It makes this note, but what are you doing with that content?"
    — Dr. Calnesy Powell (06:54)

• Emphasis on Therapy’s Human Element:

  • Calnesy Powell stresses the personal, vulnerable nature of therapy:

    Notable Reflection:

    "It is a privilege and an honor to be able to hear people's stories, both the joy, the happiness and the sorrow."
    — Dr. Calnesy Powell (07:11)

  • She welcomes the law, but feels it doesn't solve broader issues of privacy, efficacy, and the sanctity of the therapeutic relationship.

---

Memorable Quotes & Timestamps

• Bob Morgan on chatbot harm:
  "These stories involved life or death situations, individuals who are dealing with substance abuse, psychosis, suicidal ideation." (04:59)

• Vale Wright on chatbot business models:
  "The business model for these chatbots is to keep you on the platform by telling you what you want to hear. This is the antithesis of therapy." (06:03)

• Vale Wright on the future:
  "You're going to have mental health chatbots that are rigorously tested, rooted in psychological science, co-created with experts, and they'll have humans monitoring the interactions. And these might actually be really helpful." (06:19)

• Dr. Michelle Calnesy Powell on the vulnerability of therapy:
  "It is a privilege and an honor to be able to hear people's stories, both the joy, the happiness and the sorrow." (07:11)

---

Timestamps for Important Segments

• [04:01] – Introduction to state-level bans on AI in therapy
• [04:39] – Bob Morgan describes dangerous AI advice in therapy contexts
• [05:28] – APA's Vale Wright discusses limitations of the new laws
• [06:19] – Wright outlines criteria for trustworthy future AI in therapy
• [06:33] – Dr. Calnesy Powell voices skepticism about current AI applications in therapy

---

Episode Tone & Takeaways

The segment balances caution with measured optimism. It highlights both the urgent dangers of unregulated AI in sensitive settings and the potential for well-regulated, expert-informed AI to fill critical gaps in mental health care, provided privacy and human oversight are guaranteed. The speakers' tone is sober, insightful, and grounded in personal and professional experience.

---

Final Thoughts

For listeners: This concise but comprehensive report explores the clash between tech innovation and patient safety, making clear that when it comes to therapy, the stakes are high and the human element remains paramount—even as technology continues to advance.

Transcript

[MultiCare Announcer]

For 140 years, MultiCare has been in Washington prioritizing long term solutions, partnering with local communities and expanding access to care. Together, we're building a healthier future. Learn [e-mail] Is it time to reimagine your future?

[Capella University Announcer]

The right business skills may make a difference in your career. At Capella University, we offer a relevant education that's designed to focus on what you need to know in the business world. We'll teach professional skills to help you pursue your goals like business management, strategic planning and effective communication. And you can apply these skills right away. A different future is closer than you think with Capella University. Learn [e-mail].

[Sabri Benishour]

Let'S read some market tea leaves, shall we? From Marketplace, I'm Sabri Benishour in for David Brancaccio. You know, sometimes markets speak to us with the voices of millions of investors voting with their dollars about how they think the future will go. So let's take a moment to decode what they are telling us this morning with Julia Coronado, founder and president of Macro Policy Perspectives and a professor at UT Austin. Good morning, Julia.

[Julia Coronado]

Good morning.

[Sabri Benishour]

So the bond market, it's telling us something because bond yields have been rising. Why are they rising?

[Julia Coronado]

Well, we have seen some political attacks on the independence of the Federal Reserve and that is spooking markets. And in general, markets are just looking at governments and thinking that they are a little bit riskier than they thought and need to be charged more to borrow.

[Sabri Benishour]

You know, I was talking with an investment strategist the other week and he was saying it's not even just the independence of the Fed that's spooky markets, it's the looming budget deficits.

[Julia Coronado]

Absolutely. It's a combination of there really is no longer a framework for fiscal discipline, fiscal discussions in the US Budgeting process. We're just looking at large and growing deficits as far as the eye can see. That certainly is a concern. And then you have a Federal Reserve that may not be able to do their job without political interference as well as they have in the prior 30, 40 years. And that's also a concern. So there's really no good news for the bond market right now.

[Sabri Benishour]

So moving on to another market, which is the market for gold, the price of gold is up 32% this year. It's more than $3,500 for an ounce. Why would that be happening?

[Julia Coronado]

Well, same story. So if you can't store your value, your savings in government bonds, if those are no longer viewed as safe, where do you store your savings well. Gold is one typical safe haven that benefits when people are worried about the world. So I think those two stories very much go together.

[Sabri Benishour]

Julia Coronado is founder and president of Macropolicy Perspectives. Thank you so much.

[Julia Coronado]

My pleasure.

[Sabri Benishour]

By Now Pay later company Klarna is launching an ipo. After being delayed for several months, it's looking to raise almost $1.3 billion. And Congress is back in session. Lawmakers have until October 1st to figure out how to fund the government or risk a shutdown. Several Democratic votes would be needed to pass a budget and Democrats have a range of demands, including restoring cuts to Medicaid.

[MultiCare Announcer]

For 140 years, MultiCare has been in Washington prioritizing long term solutions, partnering with local communities and expanding access to care. Together, we're building a healthier future. Learn more@ multicare.org.

[Sabri Benishour]

Artificial intelligence is showing up everywhere these days, from drug discovery to advertising. But some US States are trying to draw a line at mental health care. Utah, Nevada and most recently Illinois have passed legislation limiting the use of AI in therapy. Esther Yoonji Kang from member station WBEZ checks in from Chicago on Illinois's AI therapy ban, just signed into law last month.

[Esther Yoonji Kang]

Last year, Illinois state legislator Bob Morgan kept hearing from social workers about AI being used for therapy, just story after.

[Bob Morgan]

Story of new apps and new examples of AI therapy bots that are really providing bad advice and sometimes dangerous advice.

[Esther Yoonji Kang]

For example, he's heard about a chatbot telling a person with an addiction to take more drugs because it felt good in the moment. Morgan says these stories involved life or.

[Bob Morgan]

Death situations, individuals who are dealing with substance abuse, psychosis, suicidal ideation.

[Esther Yoonji Kang]

So Morgan drafted a bill that bans therapists from using AI other than for administrative things like note taking or scheduling. The law also says chatbots cannot diagnose or treat mental illness or market themselves as if they do for stepping in.

[Bob Morgan]

And saying, if you're an AI bot pretending to be a therapist, that is inappropriate and we're going to shut that down.

[Esther Yoonji Kang]

Vale Wright with the American Psychological association says the law is well intentioned, but it doesn't address the real problem.

[Vale Wright]

What we're seeing are people going to these generative AI platforms that were not built for mental health purposes.

[Esther Yoonji Kang]

She's talking about tools like ChatGPT or Character AI, which do not market themselves as therapists, but they do answer user questions about mental health. Wright says the business model for these chatbots is to keep you on the platform by telling you what you want to hear.

[Vale Wright]

This is the antithesis of therapy.

[Esther Yoonji Kang]

But Wright, who heads up the APA's Healthcare Innovation Group, says in the coming years, technology could help fill a void in the US Growing mental health crisis. As long as it's federally regulated, I.

[Vale Wright]

See a future where you're going to have mental health chatbots that are rigorously tested, rooted in psychological science, co created with experts, and they'll have humans monitoring the interactions. And these might actually be really helpful.

[Esther Yoonji Kang]

For Michelle Calnesy Powell, a licensed psychologist in Illinois, that feels like a stretch. At least for now.

[MultiCare Announcer]

I am skeptical about AI.

[Esther Yoonji Kang]

She does use it for billing and things like that, but definitely not for taking notes during sessions, she says. Some colleagues use dictation software that complies with patient confidentiality laws.

[MultiCare Announcer]

But even then I question when I read the terms of services, it kind of is like sending a session off into the ether. It makes this note, but what are you doing with that content?

[Esther Yoonji Kang]

More importantly, Kalna C. Powell says the work she does is very personal.

[MultiCare Announcer]

It is a privilege and an honor to be able to hear people's stories, both the joy, the happiness and the sorrow.

[Esther Yoonji Kang]

The experience is very vulnerable for her clients, and she says while she's glad there's a law banning AI therapy in her, it doesn't go far enough. In Chicago, I'm Esther Yoonji Kang for.

[Sabri Benishour]

Marketplace, and in New York I'm Sabri Benishore with the Marketplace Morning Report from APM American Public Media.

[Maggie Smith]

Poetry has the power to connect our inner universe and the outer world. I'm Maggie Smith, poet and host of the Slowdown of a podcast from American Public Media. Each weekday, find time to take a breather from your to do list or doom scrolling for that matter, and take in a moment of reflection with a hand picked poem. Listen to the Slowdown wherever you get podcasts.