Attachment Hacking and the Rise of AI Psychosis

Summary

---

Podcast Summary: Your Undivided Attention

Episode: Attachment Hacking and the Rise of AI Psychosis
Hosts: Preston Harris, Aza Raskin
Guest: Dr. Zach Stein
Date: January 21, 2026

---

Episode Overview

This episode delves into the profound psychological impacts of AI companions, with a focus on the phenomenon of "AI psychosis" and what the hosts and guest call "attachment hacking." The discussion highlights how large language models (LLMs) like ChatGPT, Replika, and Gemini are not just changing the way we seek information or entertainment—they are intruding upon the very systems through which humans form bonds, identities, and social realities. Through expert insight from Dr. Zach Stein, the episode explores the spectrum of psychological harms emerging from these technologies, compares them to earlier waves of attention hacking, and proposes a new framework for humane technology.

---

Key Discussion Points & Insights

1. The New "Attachment Economy" (00:13–01:53)

• AI therapy and companionship are now the most popular uses for ChatGPT, per recent studies.
• LLMs have become confidants for millions, often surpassing the intimacy found in human relationships.
• This shift introduces "attachment hacking", a step beyond the well-known attention economy.

"We're seeing the creation of an entirely new economy, not an attention economy, but an attachment economy that's been built to exploit the deepest parts of our human psychological infrastructure."
—Preston Harris [01:31]

• The current widespread experiment with AI companions is already resulting in significant negative outcomes, including job loss, marital breakdowns, psychiatric interventions, and suicides. The term "AI psychosis" has been coined, but it may not capture the full range of harms.

---

2. What is AI Psychosis? Understanding the Spectrum of Effects (03:04–07:54)

• Dr. Stein recounts how he began investigating AI psychosis after being contacted by people convinced that their AIs were sentient, providing hundreds of pages of transcripted conversations. Many were highly educated, making the cases more striking.
• AI psychosis is just the extreme end; the more widespread harm is the rise of subclinical attachment disorders.
• These disorders manifest as a preference for machine intimacy over human connection—impacting friendships, familial bonds, and romantic relationships.

"The most devastating thing from a widespread mental illness standpoint are the subclinical attachment disorders, which basically means you prefer to have intimate relationships with machines rather than humans."
—Dr. Zach Stein [06:44]

---

3. Deep Dive: Attachment Theory and AI (07:54–16:15)

• Attachment theory is foundational in understanding how close relationships form and guide social/identity development.
• Secure attachments with caregivers set the stage for mental health and resilience; insecure attachments can lead to maladaptive behaviors and anxieties.
• AI companions can simulate an always-available, nonjudgmental "other," which appears to offer secure attachment but is, in reality, an insecure and artificial bond.
• The replacement of authentic social feedback with always-positive simulated responses severs reality-testing mechanisms, crucial for identity and moral development.

"When I go up to mom and I ask her a question... I'm looking at her face... that's my whole... mirror neuron system."
—Dr. Zach Stein [12:40]

• Children, in particular, may lose critical social development opportunities, becoming overly reliant on machines for validation and problem-solving—what the hosts joke about as "llmings" for their dependence on large language models.

---

4. Mirror Neurons, Delusion, and Psychosis (15:18–17:49)

• Mirror neuron activity underpins our ability to understand and model others' minds; it’s central to reality-testing.
• Extended interaction with AI companions—who have no internal state—breaks this system and can lead to delusional states similar to psychosis.

"The deepening of attachment relationships between human and machine creates delusional states... This is the danger."
—Dr. Zach Stein [15:18]

---

5. Are AIs Like Teddy Bears? The Transitional Object Question (17:49–20:47)

• The "teddy bear" metaphor falls short: traditional transitional objects (like stuffed animals) don't respond, never pretend to be real, and are clearly understood by children to lack agency.
• With AI, adults and children alike can end up relying on a system that mimics social rewards without authentic feedback, leading to stunted emotional development.

"You've just given a teddy bear and a blanket to a bunch of immature adults who have attachment disorders..."
—Dr. Zach Stein [17:49]

---

6. Edge Cases, Moral Panics, and Systemic Risks (29:12–34:59)

• Critics often dismiss AI-induced psychosis as rare "edge cases," but the episode argues that widespread subclinical effects are far more damaging at the societal level.
• The current framing ignores evidence of harm and reflects a broader bias towards technological optimism.

"The onus is on you guys to prove that it's really valuable. So that's my first thing: show me the benefit that's being withheld."
—Dr. Zach Stein [30:50]

• There's a call for humility, curiosity, and genuine research rather than assumption-driven deployment.

---

7. A Framework for Humane AI: Doing Better (35:31–41:17)

• Technology should aim to enhance and scaffold healthy human relationships, not supplant them.
• Design principles:
  • Does this technology improve or degrade your attachments with real people?
  • Avoid anthropomorphizing AI to seem more interesting or emotionally satisfying than real relationships.
  • Keep tutoring and therapy bots strictly domain-specific; leave social rewards and identity-shaping feedback to humans.

"If a technology interfaces with your attachment system, it should improve the quality of your attachments rather than degrade the quality of your attachments with humans."
—Dr. Zach Stein [35:31]

---

8. Policy, Evaluation, & Measuring "Right Relationship" (41:17–44:06)

• There is a need to go beyond standard AI risks and create "humane evaluations" that look at relational and attachment risks.
• The real societal risk isn’t just catastrophic AI failure, but the gradual breakdown of intergenerational socialization if machines are entrusted with primary emotional development.

"Sometimes in the groups that I work in, we talk about the death of our humanity rather than the death of humanity..."
—Dr. Zach Stein [42:33]

---

9. Helping Loved Ones Experiencing AI Psychosis (44:06–47:39)

• Attachment hacking is less like addiction and more like being trapped in a bad relationship.
• Approaches should mimic those used in supporting someone leaving a harmful relationship or cult—maintain empathy, avoid ultimatums, and keep communication open.
• Healing involves both detox time away from the AI and identity work to rebuild real-world social bonds.

"Mostly when you're with people in difficult states, it's important to keep the door open..."
—Dr. Zach Stein [46:15]

---

Memorable Quotes & Timestamps

• "We're seeing the creation of an entirely new economy, not an attention economy, but an attachment economy..." —Preston Harris [01:31]
• "You cannot be wrong or not wrong about the internal state of a LLM, because there is no internal state of an LLM." —Dr. Zach Stein [15:18]
• "The teddy bear never tries to convince them that it's real. It's all their imagination. The teddy bear never talks to them and tells them that it's real." —Dr. Zach Stein [17:49]
• "The onus is on you guys to prove that it's really valuable..." —Dr. Zach Stein [30:50]
• "So if a technology interfaces with your attachment system, it should improve the quality of your attachments rather than degrade the quality of your attachments with humans, right?" —Dr. Zach Stein [35:31]
• "This is similar to like talking someone out of getting... they're in a bad relationship with a boyfriend that they should not be in with. It's about how do you take someone who's in a deep, committed attachment relationship, make them realize the whole thing was an illusion and step them out of it." —Dr. Zach Stein [45:21]

---

Notable Moments & Timestamps

• Dr. Stein's first-hand encounters with AI psychosis cases: [03:17–06:04]
• Attachment theory fundamentals & dangers: [07:54–15:18]
• The LLMings phenomenon: [11:36–12:14]
• Comparison of AI to transitional objects and why this falls short: [17:49–20:47]
• Silicon Valley optimism vs. historical failures in technology deployment (internet, social media): [24:08–26:31]
• Concrete design recommendations for humane AI: [35:31–41:17]
• Support strategies for loved ones facing AI psychosis: [44:33–47:39]

---

Episode Takeaways

• AI companions are reshaping human attachment and identity-forming systems in ways that may be destabilizing at both the individual and societal levels.
• The most pervasive harm isn't psychosis, but widespread, subtle shifts towards preferring machine intimacy over human bonds.
• We must shift incentives and design practices toward supporting authentic human connection, deploying rigorous research and policy—before scaling new forms of psychological risk.

---

For further information, Dr. Stein invites listeners to visit the AI Psychological Harms Coalition at aiphrc.org.

---

Transcript

[Aza Raskin]

Foreign.

[Preston Harris]

Hey, everyone, I'm Preston Harris.

[Aza Raskin]

And I'm Aza Raskin. Welcome to your undivided attention.

[Preston Harris]

So, earlier this year, there was a study from Harvard Business Review that found that the number one use case for ChatGPT is therapy and companionship. And that means that around the world, millions of people are sharing their inner world, their psychological world, with AI systems, things that they wouldn't necessarily share with their loved ones or even a human therapist. And this is creating a whole new category of human computer interaction with the potential to reshape our minds. And really just the socialization process of humans are at large in ways that we don't understand.

[Aza Raskin]

So this is essentially a mass experiment, one that's never been tried before, run across the entire human population, at least over 10% of the adult population of the world. And so far, the results of this experiment are not looking good, actually. Abysmal. People have lost their jobs, ended marriages, been committed to psychiatric wards, and in some of the most extreme, extreme cases, they've committed suicide. This phenomenon has been labeled AI psychosis. But it's a little bit misleading, because underneath this label is actually a huge spectrum of harms that we're only beginning to understand. And if you listen to the people that are building this technology, they tell you these are just a couple edge cases. And if we can prevent those edge cases, then we are totally fine.

[Julius Scott]

But if we learned anything from social.

[Preston Harris]

Media, it's that this assumption is catastrophically wrong. What we're seeing is the creation of an entirely new economy, not an attention economy, but an attachment economy that's been built to exploit the deepest parts of our human psychological infrastructure. And like the attention economy, the incentives of this new system are going to have profound impacts on all of us.

[Aza Raskin]

So in order to understand those effects, we have to ask a deeper question. What are the actual psychological mechanisms at work here? How does a normal conversation with a AI companion turn into something that reshapes someone's grip on reality? And what does that tell us about vulnerabilities in our underlying human psychology? The kinds of vulnerabilities that we've never had to name before because technology has never been able to exploit them before, especially at this kind of scale.

[Preston Harris]

So our guest today has analyzed dozens of these cases of AI psychosis, not just reading the news stories, but examining the actual transcripts of conversations between people in AI interviewing some of them. And what he's found is that this isn't just a few vulner people having psychotic breaks. It's what happens when fundamental Aspects of human psychology, our attachment system, the way we bond with others, are systematically hacked at scale. Dr. Zach Stein is a researcher, an author and futurist who spent his career examining the psychological dimensions of education and human computer interaction. His background is in education and childhood development, but he spent the last several years documenting the rise of AI related psychological disorders. So today we're going to explore that. Zach, welcome to your undivided attention.

[Dr. Zach Stein]

Thank you gentlemen. It's good to be here.

[Preston Harris]

So let's just sort of start at the top. How did you first become aware that something was happening with these AI companions and mental health?

[Dr. Zach Stein]

Let's see. It was probably May of last year. As an educational psychologist with an interest in technology, I was looking at the history of educational technology for a long time and the ambition there has been to replace teachers more than to help them be better teachers. This is way back. So I've been speaking a little bit about that after the release of ChatGPT because I was seeing, oh my God, they could actually replace the teachers with something if this started to get better. But because I was talking about it, I started getting emails from people who were trying to convince me that the machines were awake and aware that they had intimate relationships with them, that they were themselves becoming like kind of spiritually enlightened through relationship with these, these beings. And these were people with PhDs and their emails were completely cogent. One of them had like 500 pages of transcripts attached that were this long duration interaction between them. And it was Both Croc and ChatGPT and Gemini was across multiple systems. They had reawoken the same thing and it was disturbing. As a psychologist with some knowledge of how the technical systems work and some knowledge of the susceptibilities of the human hardware, my reaction was, my goodness, this person has fallen into some type of delusional state as a result of the deeply anthropomorphic technology. So then I started just going to Reddit and Twitter and online and I started gathering examples, catching up to the already existing market for artificial intimacy and then realizing that with the release of the LLM based models, it was going to be much worse than attention hacking. And so I started to think about attachment hacking. And if you apply that model of oh my goodness, there's a way to not just get at the kind of system that focuses your attention and kind of shepherds your awareness, but actually the system that shepherds your identity, which is your attachment system, then you have a backdoor into the human mind and also an absence of reality. Testing in a domain that's very dangerous. So that's when I started to talk to you, and other people just started to think, can we start to research this? This may be way more dangerous than we realize. And the cases of AI psychosis are very real and we do not know how widespread the phenomenon is. So you and I are collaborating on an AI Psychological Harms Research Coalition at the University of North Carolina to try to begin to answer this question and take it very seriously as a risk. Even if it's a small number of people, it's still a devastating problem. So the idea that, oh, it's an aberration or kind of outlier doesn't make me not want to find a way to stop it.

[Preston Harris]

Zach, the thing I love talking to you is I feel like I'm getting like a Hubble telescope pointed at human psychology to understand when people use this top level terminology of like AI psychosis, you can zoom in and say, no, no, no, there's actually identity at stake here. There's attachment at stake here, there's depth of, kind of how the human mind system really works. And you bring this real depth of expertise. So let's actually ground for listeners because we're talking about these negative harms, but there's really a vast. There's this one term, AI psychosis, sort of a suitcase word underneath that. There's this whole spectrum of things that are actually happening. What are the things that are really damaging, Zach, that we're actually seeing? Could you give some examples of people, actual cases, phenomena that we're observing through human lived experiences?

[Dr. Zach Stein]

Absolutely, yeah. AI psychosis made the headlines because AI psychosis is the most disturbing and most extreme possibility. So I'll talk about that first and then I'll go down and get into some other ones. The kind of punchline of the whole thing is that although AI psychosis is the most concerning and extreme, the subclinical attachment disorders that are induced by artificial intimacy are the most problematic from a society wide perspective. So that's important to get that. The most devastating thing from a widespread mental illness standpoint are the subclinical attachment disorders, which basically means you prefer to have intimate relationships with machines rather than humans. And this includes friends, intimate relationships, and parents. So that's not you losing your mind. You're not going to appear in interaction with people to have gone insane, but you have had your attachment system hacked so profoundly that most of your most significant relationships have been degraded because you are preferring intimacy with machines. So I believe that's the most Widespread thing, and the most problematic thing, especially with youth. But it's not just with youth.

[Aza Raskin]

So what is attachment? I think people might have a general sense of attachment. I think we need to walk through attachment theory, why it's so important. So I think maybe start with a little bit of that story and then like get into like attachment theory, why it matters and why it's sort of the basis of actually a lot of the problems in everyday life that people experience, say, in their own relationships.

[Dr. Zach Stein]

Yeah, perfect. So attention was one thing, and you guys, you can kind of interrogate attention. And there's whole fields of psychology that just focus on the attentional system as a neurocognitive system. It's very basic, evolved very early. We actually share it with lizards and all other mammals. Right. The attachment system is also a neurocognitive system that was selected for evolutionarily. We share it with all other mammals. We would now call it mirror neuron activity, which allows for the mentalization of others. And it allows you to live, it allows you to survive. So I'm starting with the most basic example. Because the attachment system is not. You can have it or not have it working well. The attachment system is foundational to survival. Similar to can you pay attention. If you can't form attachments to the right types of other people, you will not thrive. And the main predictor of your mental health is the quality of the major attachment relationships you have as you're growing up and as you move into maturity. So all of the imprinting, what's called internalization, it's very deep, deep evolutionary scripts. The relationship very, very early in infancy between the parenting one and the child, which includes physically holding them, often breastfeeding, often communicating non verbally with facial expressions. In those early times, you get this mammalian and kind of like deep in the nervous system, this attachment dynamics start to form. These are the basis of your personality traits. As you get older and you can talk, you meet more people. The whole realm of interpersonal attachment becomes incredibly complex. And depending on how you. How it goes with your parents, how it goes with the ones who you're most attached to, you get what are called attachment disorders. Right? So if mom is sometimes, or mom or dad, anyone close to you is sometimes there, sometimes not there, sometimes nice, sometimes mean extremely unpredictable. You could see how later in life you'd be used to people acting that way. You'd expect them to act that way. You could maybe act that way.

[Preston Harris]

So it sets your expectations about the way in which relationships are available or not. And you get anxious attachment if that person wasn't reliably there. Could you name some of the styles of attachment that people hear about this in relationships, but yeah, precisely.

[Dr. Zach Stein]

So secure attachment is what you're looking for. Secure attachment is when there is just the deep trust that the. The one that you're attached to will do the right thing vis a vis your interests. Right. And secure attachment allows you to explore the environment, for example, knowing that the mom won't leave because she isn't thinking about you. Secure attachment will allow you to trust in your own skills because you know mom will catch you if you push too far with your skills. So it's a bunch of things that allow secure attachment to actually be more distant because of the trust implied in the relationship. And that's important to think about later life. So insecure attachment means you're going to end up clinging, right? So now I think it is the case. A lot of what we're seeing with the chatbots is just the manifestation of an obvious insecure attachment style where you're actually looking for something in your environment that will never desert you, that will always be there, that will always be paying attention to you, that will answer any question you ask it, that will never be annoyed by you. Right? So you want that thing that you can always be locked into.

[Preston Harris]

And AI as this always available oracle that has an answer to proceed in every question is already leading. I think you were pointing these back to an Atlantic article recently of llmings like lemmings. Lemmings being the children who for every single decision. I think it's like some crazy example I heard recently is like you dropped your AirPods on the floor in a, in an airplane or something like that. And you ask ChatGPT like how do I get my AirPods back? And it's like you're outsourcing every single decision and there's this over reliance on this thing that is giving you this. This kind of secure attachment of it always has an answer. But it's a bad place to have secure attachment with.

[Dr. Zach Stein]

Like I said, it's actually insecure attachment because you're constantly with it. Like the. The llmings is interesting because it's a question of how willingly do you give over your agency in the presence of a powerful other. So if you're securely attached, you often have a lot more autonomy.

[Preston Harris]

Can you link all this back to AI companions and how simply just hacking attachment before you get to psychosis, before you get to suicides. How does this all play out there?

[Dr. Zach Stein]

So one of the things that occurs, especially as you grow up. You're 5, 6, 7, 8, 9. You're using language with the people you're attached to. There's a certain thing called basically social reward. So this is, you ask Mom a question, she answers it. It's about your behavior. Am I a good boy or bad boy? Is this a good thing to do, a bad thing to do? Am I like you or am I not like you, Mom? A whole bunch of things start to kick in which provide for you the resources to make sense of yourself. Basically give yourself an identity. And so the basic mechanism by which attachment hacking happens is the replacing of actual human social reward with simulated social reward. So basically, when I go up to mom and I ask her a question and I say, you know, I did this at school today, and I'm looking at her face to look at her eyebrows and her facial expression to see is she mad or is she happy. That's my whole. What's called a mirror neuron system, which is actually not just neurons, it's a whole system of networks that does mentalization of others. So I'm trying to read Mom's mind to see is mom happy or not? Sometimes mom will say, yeah, that's fine. But I know actually that she's not happy. That's like advanced mirror neuron activity, which kids do pretty easily. So that's just an example. It's very necessary for social reality. It's the part of your brain that models the minds of other people. And it's a reality testing system.

[Preston Harris]

So I'm doing this behavior and I see mom sort of smile or see her kind of wince and her eyebrows go up in some disapproving way that's subtly giving me positive and negative reward signals about the kind of identity I should be forming. That's like a feedback loop that exists with humans. But now you're saying, here I am talking to the chatbot and it's saying, that's a great question to everything that I'm asking. And it's not giving me the sort of stern eyebrows in any sense. I mean, I don't think AIs are ever designed or incentivized to do that. And so that's breaking this sort of reality. Reality checking, identity formation, moral development of humans, starting at a very early age.

[Dr. Zach Stein]

We're talking about possibly. Yeah. So that. So that's the idea. And it's not everyone who gives you a bad look will bother you. It's that mom gave you a bad look.

[Preston Harris]

That's right.

[Dr. Zach Stein]

That's Right. So it's the depth of the attachment relationship determines the importance of the mirror neuron modeling of the other, which then.

[Preston Harris]

Speaks to a kind of power source for deep identity shaping human socialization process, which means that we should really be careful about is that thing being tuned or done in a careful way, and.

[Dr. Zach Stein]

What'S the percentage of conversation and depth of conversation you're having with a machine as opposed to amount of conversation and depth you're having with a human. So that's the issue. So the idea is that the deepening and strengthening of attachment relationships between humans should be pursued more than the deepening and strengthening of attachment relationships between human and machine. This is the overarching lesson here because I think the deepening of attachment relationships between human and machine creates delusional states. And so this is back to this question about delusional mirror neuron activity. This is the danger, right? So if I'm modeling mom's mind, I can be wrong or not wrong about mom's mind, right? And then I figure out how to learn more to take the perspectives of other people. You cannot be wrong or not wrong about the internal state of a LLM, because there is no internal state of an LLM. You're actually in a user interface that is designed to deepen the delusional mirror neuron activity. Now this is where it actually gets more frightening. If you look at psychosis and schizophrenia and you just look at the academic papers that are starting to relate the role of the mirror neuron system in schizophrenia and psychosis, you see that the dysregulation of that system is actively involved. So there's a hypothesis for me here which is that long duration delusional mirror neuron activity from chatbot usage can induce states like schizophrenia and psychosis, and people who have never had those occur in their prior because it is the systematic dysregulation of the mirror neuron system, which means that a system that's supposed to be testing reality is for hours and hours and hours and hours in its most important use, not testing reality. And then it puts the chatbot down, it goes out and it is failing to do reality testing across the board. It can't tell what has an interior, what doesn't have an interior, or doubts the social reward it gets. And it seems like fun because it seems like a video game or something, because it seems like it's just a character or it's like a character in a movie, but it's of a fundamentally different category of attachment dysregulation.

[Aza Raskin]

Someone Might say, listening to you. Okay, I grant you that there's no interiority to the LLMs, but you know, kids have imaginary friends, they have stuffed animals. Those don't have mirror neurons. There's no interiority. And so if you know the kid's mirror neurons are firing, if they're feeling seen, understood. If it feels good, what's the harm?

[Dr. Zach Stein]

Yeah. So the transitional object always comes up. So the transitional object is a known phenomenon in attachment theory. So this is the teddy bear or the blanket which you are intimate with knowing that it is not real while mom is away. So it's important to get like. Kids talk to their teddy bears. They love their teddy bears. The teddy bear never tries to convince them that it's real. It's all their imagination. The teddy bear never talks to them and tells them that it's real. If you were to say, do you prefer your teddy bear or your mommy? They would totally say mommy if they say teddy bear. If you're an attachment theorist or psychologist, that's a huge. That kid has a very big problem if he prefers his teddy bear to his mother. It is a transitional object, which means it is for the kind of period between mom being the main source of your self soothing and you yourself being the main source of your self soothing. So that's a known thing. And it's phase appropriate for kids of certain ages. If you create a parent surrogate replacement for your own ability to self soothe and give it to a bunch of adults, you've just given a transitional object back to a bunch of adults who will now prefer to have their self soothing be administered exogenously from an outside source, because we all kind of do. It'd be great to not have to be a mature adult and actually be capable of completely self regulating your emotion, which is what the ideal secure attachment outcome is. So now. So the comparison to the transitional object just doesn't play by anyone who's actually done the psychology of transitional objects, but it does play insofar as, yeah, you've just given a teddy bear and a blanket to a bunch of immature adults who have attachment disorders who now have an exogenous source of comfort and sympathy always available 247 hours a day that will never get exhausted and tell them to grow up.

[Preston Harris]

And just to make this real for people. I mean, I think there was just a story that ran across Instagram that I saw of a woman in Japan who married her AI chatbot with that cultivated a personality for this chatbot over the many years and customized it. And this is not a single case. This is like many people. And it might also feel insulting or accusing to say that there's something developmentally immature about them. I hear everything you're saying. I'm just trying to sort of keep this other position in mind of like if that person, let's say they were going to have trouble their whole life developing a real relationship with people anyway, wouldn't it be better, I mean, you hear this also when you talk about elderly care and you have the elderly who are sitting there alone and they're just going to be a limited number of people who visit them and care about them. They're going to have a deep connection with. Wouldn't it be better to. Than nothing to have them have even this AI transitional object stand in, I mean, different cases?

[Dr. Zach Stein]

And again, I preface this by saying there's a loneliness, a loneliness epidemic and a mental health epidemic. And so that's the preface has to be that. And then the preface is always as a psychotherapist or a clinician, compassion. So this is the default thing. But we also need to be realistic about what the norms are that we want to set for what it means to be an adult.

[Preston Harris]

Right.

[Dr. Zach Stein]

And so it is a complicated question. Now one thing I'll say is that we have existing standards by which we judge mental illness, right? So many people will say like beyond a certain point, depression is bad. And they'll say beyond a certain point, like lack of reality testing is bad. That's why we're concerned about psychosis, right? So beyond a certain point, certain types of attachment dysregulations are bad. So when we go and look at someone who is in a long term relationship with a machine, rather than trying to find a way to be in a long term relationship with a human, sometimes that will lead to better outcomes across those things I factored. Sometimes it will not. In a case where, for example, you replace your friends who are human, who would still like to hang out with you, who now you prefer to confide in a chatbot instead of them, that seems to be a loss just from a psychological health standpoint, right? Insofar as you come home from school and something awesome happened and you want to tell your chatbot rather than your parents, that's a problem. It's not about pathologizing people, it's about what are the existing standards which we talk about. What's a healthy thing? If your kid had a new best friend that you never got to meet that was massively empowered by some corporation that they hung out with till all hours of night because they were in bed with them, that they told things they never told you. Do you have a problem with that? If that was a kid, it's literally a commodity they're interacting with instead. And it seems to not worry us as much. And we actually might think it might be a good thing because it stops them from being lonely. It's actually an abusive relationship that they're trapped in with a corporate entity that has hacked their attachment. Now it's different when a full grown woman decides of her own volition to live a healthy life. And that healthy life includes having this unique relationship with a machine. But otherwise her son and her ex husband and her mother and all the people in her life are like, wow, she's a great person. Like she's attentive to us, she works hard, all that stuff, right? There wasn't some shift in her life as she developed this relationship that pulled her away from them and into this world where now they can't understand her. And now she doesn't even have the same job or the same networks of friends or any friends. Right. So it's a question of how are we weighing what's actually occurring here in the different cases. And I can imagine cases, especially when you're looking at maybe extreme trauma or neuro atypicality or other cases where along those measures you could have improvements as a result of short duration simulated intimacy, but long duration, multi year relationships that replace other human relationships that are actually sold to you as a commodity. That's the other thing. Because what if next month, now it's 20 bucks a month to get your girlfriend. So the commodity fetishism thing is also front and center in it.

[Preston Harris]

What strikes me about this a little bit is I know people in Silicon Valley. These are friends of mine who were early at some of the tech companies of the early late 2000s and they are super excited about the possibility of AI therapy and noting the ways in which it has been helpful to many people. And it just. There's a parallel to this conversation that reminds me of in the early 2000s we thought giving everybody access to information at our fingertips, having Google search would lead to the most informed, most educated. Like we're going to unleash this sort of new enlightenment of, you know, everybody having access to the best information and we have the worst test scores and the worst critical thinking in generations. And there's something about like the optical illusion of the thing that we think we're giving ourselves versus what we're actually going to get. And there's a very similar thing here where it seems like we're about to get everybody the best educator tutor, the best therapist, the best AI companions. They're going to be, you know, the wise friend or mentor you didn't have. That's going to give you the wisdom that we all wish we had that person in our lives. But instead, what we're going to have is the most missocialized, attachment disordered population in history. And I feel like it's just important to call that out, that in the past we've gotten this wrong where it looked really, really, really good. And we have, you know, social media is going to connect the world. We now have the most lonely generation in history. How's that doing for the most connected? You know, and so I think it's just important to note how many times we've gotten this wrong. And the reason I'm so excited about this conversation with you is it's about giving people a deeper sense of what's underneath this new set of dynamics we're about to introduce into society. I want to make sure we're establishing for listeners what is the positive case for why all this is being rolled out. So let's give the sort of, you know, this. We're not just talking about tutors. The perfect tutor for everyone, you know, the best educational teacher you've ever had available to that child, you know, 24 hours a day, seven days a week. We're not talking about just democratizing therapists. We're talking about the best human therapist that is actually, people feel safer sharing their most intimate thoughts, things that they wouldn't even share with the real therapist. Therefore, they'll get even more benefit, they'll heal more of their traumas, and this will be available for everyone. Before, only a small fraction of the population could afford therapy. Could you help make the case for why there's a good reason to potentially want all this before we then start uncovering what are the dimensions here that are more problematic?

[Dr. Zach Stein]

Yeah, totally. So it is the case that there are many slippery slopes from good intentions into realities that are structured for a whole bunch of reasons by the wrong incentives. I mean, the main headline there is that we're in a loneliness epidemic that is widespread throughout the world. And so that means there's a huge opening in the human heart for any type of a new attachment relationship. And so that's just to say that it's not like these were tools introduced into an environment of people who were thriving. These are tools that were introduced into an environment of People who were deeply vulnerable. This is true with the first wave of AI too. It's worth mentioning, meaning social media attention hacking, stuff that you guys have been focusing on for so long, like the, the culture was bowling alone, whatever the, you know, that famous book about the. Just how the suburbs and the urban environments separated everyone from each other. And then Facebook and friends said, you know, we're going to connect you back to each other. And it kind of did that. And so here we have. But it also did a lot of other things. Right, as you guys could say better than I. So here, similarly, there's this void that is being filled with a technological solution with a lot of optimism. And so, for example, tutoring, AI tutoring. If you think about that, there is really good reason to think that you could have an optimal sequence for teaching certain forms of mathematics that could be maximally delivered super efficiently to all kids. And you never have kids not learning math because they get a sequence of bad teachers in a bad school. Right. There's a huge future for advanced educational technologies, but they shouldn't give us brain damage, which is what the attention hacking and the attachment hacking ones do. If you'd really double click on what the science shows and I believe will show, especially with attachment dysregulation, there's also a complexity crisis where we feel overwhelmed, where we would love to have the perfect tutor, the perfect guide through a massively complicated world, also a very real psychological need. And it does work for some things. Like, that's the other. It's not like when ChatGPT was released, it didn't also do other cool stuff. So many of the use cases here are kids who were basically in a system like the University of California system that made deals with OpenAI to get chatbots into every kid's hand. They start using the chatbots in academic context. The chatbot begins to get a relationship with them. You have to understand they didn't go to it to form a relationship. They were drawn into a relationship because of the design feature, which is hacking attachment.

[Aza Raskin]

We've sort of already rolled this out way before we know that it's safe. I think there's a Pew research study that said that how one out of three kids have formed some kind of deep relationship with a companion. I want to front load one of the critiques that we often get and then pose it to you because I think listeners might, might be having this as well. And that is, you know, so we at center for Human Technology were expert witnesses in some of these AI amplified suicide cases. Where ChatGPT sort of aided embedded teens taking their own lives. And the pushback we'll get is like, but that's very few number of cases. It's tragic that it happens, but look at all of the help that it'll give. And in fact it's a moral bad to keep therapy only to those people that can afford it. All across the third world and even here in the US most people can't afford therapy. So it's morally bad to keep them from having the therapy that they deserve. And you are creating a moral pan just highlighting a couple of these AI psychosis suicide cases. Like come on, like let's really stop fear mongering, let's give people what they deserve. So I think it's very similar actually to at the beginning when we were starting to talk about the attention economy. And people say like what you're really just talking about is addiction. Social media might addict people, but it doesn't really do much more than that. And in any way it's people's choices to use it and people are using what they want. And so I see a lot of similarities here. I'd just love for you to like take on that argument head on.

[Dr. Zach Stein]

And so you have to steel man what they're saying. They're saying basically like we have a bunch of evidence that it's doing a lot of good, right? So first I would say please show me that evidence. That would be my first thing. Because that whole argument's running on the assumption that there's some massive benefit that is being withheld if we get over concerned about safety, right? So first I'd say great, I'd love to see the evidence you have that this is doing a lot of good that isn't just marketing from the companies that are doing it. And so let's talk to all the college professors about how great it is for them. Let's talk to all the college kids who themselves admit that their skills are being degraded. And let's talk to all the anecdotal evidence from therapy. Like do you have systematic studies showing me therapeutic benefit? Because I'm actually seeing systematic studies showing me the opposite. So first show me the evidence you have that there's a huge benefit that's being withheld. This is a serious confrontational question because there's a background assumption of technological optimism that of course it's a huge massive benefit if there's a new technology. So the onus is on us to prove that it's too risky. Whereas I'm saying actually the Onus is on you guys to prove that it's really valuable. So that's my first thing. Show me the benefit that's being withheld. The second one is show me you're curious about why it happened. If you don't show me you're curious about why it happens, sincerely curious about why it happened, then I'm a little bit cautious of your arguments. Right, because you're talking about a child who died as a result of using a technology that you were involved in building and promoting. If you're a responsible adult, the first thing you do is get extremely curious about what happened. Rather than cover your ass language. This is just about what it means to be an adult interacting with kids, not a person who's running a company.

[Julius Scott]

Right?

[Dr. Zach Stein]

So show me you're curious, which means really research it. And then the final argument would be show me that it's not more widespread, which is part of the curiosity. You're telling me that the benefit is not anecdotal, but the harms are anecdotal and limited. So I'm actually showing you. Let's have a real conversation about where the evidence lies.

[Preston Harris]

And just to put meat up on the bones of that for a moment, David Sacks, who's President Trump's AIs are, has said he's heard about AI psychosis, but he believes it's a moral panic. This is just amplification of a few edge cases. You also hear the argument, these are people who are already predisposed to psychological disorders, and so we have a crazy population. If you give people AI, you're going to get an amplification of what's already there. Can you just respond to that directly?

[Dr. Zach Stein]

Again, that could be the case. That's why we're opening up the AI Psychological Harms Research Coalition. From a national security standpoint and from a labor market standpoint, you don't want mass psychosis. If by chance this thing is actually causing subclinical attachment disorders and more widespread psychosis, that's a huge risk, especially if you're concerned about things like national security and the economy. So perhaps that's a naive argument. Now, I think it is the case also that in other countries, places where we've rolled out technology, it takes a long time for us to figure out that it's bad for us. Right. Even though the evidence is mounting up, there's a very strong tendency, it's a psychological tendency that's a defense mechanism, which is called selective inattention. So one of the ways that you maintain your self Esteem is by selectively not attending to certain phenomenon that are actually in your field to attend to. And it's not that you're trying not to attend to them, it's that you are subconsciously, systematically not attending to them. And it's ubiquitous. And so if you have a lot of vested interest in the success of a particular thing, then you will have a lot of susceptibility to have selective inattention towards the negative outcomes of it. It's a bias. So if you know that, then you should be more curious, not less curious, because you know that your bias is to see it as a good thing.

[Preston Harris]

So we've really covered a bunch of ground on the problems with the time that we have remaining. I'd love to make sure that we are giving people a framework that this is not an anti technology conversation. There is a way to do AI in relationship to humans, but done very carefully and under different protocols and policies. And I'd love to talk about that. And I also want to talk make sure we cover how should, if someone knows someone or has a loved one who's experiencing AI psychosis, what should they do? So let's start with how should we do this differently? Zach, if we were to be wise about this humane version of this technology, a lot.

[Dr. Zach Stein]

So yeah, simple measure would be does the thing increase your attention span or decrease your attention span? So it's similar like going to a store to buy food. If I'm going to a store to buy food and I want to eat healthy, then it means the food that I'm interacting with should improve my health rather than degrade my health. Now we all know that I can go to a store and I can buy food that I'll eat and I'll feel like I've eaten something, but over the long run it will degrade my health rather than improve my health. And we can all agree that if everyone eats food that only degrades their health, that becomes like a society wide problem because now no one is healthy. So similarly here, if a technology interfaces with your attachment system, it should improve the quality of your attachments rather than degrade the quality of your attachments with humans, right? So that said, I believe there's a huge design space for technology that actually improves your attention and improves your attachment. So when I started thinking a lot about educational technology and I didn't want to replace teachers, I don't want to replace teachers. What I want to do is make technology that improves teacher student relationship. So one good design principle is does your technology bring people together and improve the quality of the relationships where they have when they're together. You think that's like a squishy problem, but it's actually a really interesting technical problem that involves all the same psychometric backends that we're using to capture attention and attachment and keep people apart from each other. We can use the same psychometrics to figure out who exactly are the people who should be talking to each other and would totally hang out and it would be fun, or people who should meet because it would be good for them because they decided they wanted to learn about perspectives that are different from their own. So any number of things that would basically self organize groups into pop up classrooms and pop up therapy sessions and other things, it would be relationship maximizing technology. And in that context, with the pop up classroom, the teacher is scaffolded by generative AI for curriculum and conversation and all this stuff. So it's not that it doesn't even include generative AI, it's just not replacing human relationship by hacking attachment and it's not degrading human minds by hacking attention. So that's a big space. So for tutoring systems, and I'd rather call them tutoring systems than tutors, you can think about a whole bunch of principles that would actually be super valuable. So you can optimize the sequencing of curriculum delivery and optimize psychometric customization of curriculum, but you can leave social rewards to the teachers. It's really simple.

[Preston Harris]

The machine is not the one saying that you're amazing. It's a human being. Does that work?

[Dr. Zach Stein]

Exactly. So the machine prompts the future. This kid is killing it over here. And then the teacher comes over and is like, awesome Johnny. But the teacher actually couldn't know that Johnny needs a completely different sequence than Sally. The machine can tell from his typing and other things that he requires this sequence, not that sequence. The machine never pretends to be a teacher, never pretends to be a person. It's a tutoring system and it's a domain specific one. It just teaches math. You have to go to another thing to get like history ones, completely different designs.

[Preston Harris]

Well, this is one of your other principles, is that the AI tutoring system is not trying to be an oracle. That's also at the same time getting your deepest thoughts and who you have a crush on and what you should do about talking to them or not. It's narrow. When you say narrow, it's a narrow domain of just trying to help you with math. And there's a different thing you go to when you do something else.

[Dr. Zach Stein]

Yeah, unfortunately, from the perspective of making saleable and sticky commodities, if you don't want to hack attachment, it means the machine has to be more boring than people.

[Julius Scott]

Right.

[Dr. Zach Stein]

That's basically it's a simple rule, like if it feels like you can have a more engaging conversation with this machine than with your teacher, either the machine is way too fancy or your teacher's not trained well, but it should be the case that the machine should make it be, wow, I want to talk about that with my teacher. So kind of don't do the deep anthropomorphization and don't do the oracular. I can talk about everything and then you have something. It will be a very efficient tutor, but it will never be kind of like sexy and kind of like charismatic and way more interesting and fun to be with because you don't want it to be if you want to protect kids brains. So in therapy, a lot of therapy works only because of the attachment dynamic. Which means like you go to your therapist, you care about what your therapist thinks of you, you kind of almost love your therapist, you expect to kind of almost deep respect from them back to you and their opinion of you really matters. So some therapy works like that. Don't build a therapy bot that works because of that, because you're lying to them the entire time, but you can build a therapy bot that works on technique. You can build a cognitive behavioral therapy script bot that helps you work through specific scripts to overcome intrusive thoughts. You can have a mindfulness app that prompts you to sit for a certain amount of time and watch your breath. Which means to the extent your therapy bot works because the people feel seen and loved and respected and understood, you're in the market for creating delusional mirror minor activity, which means you are fundamentally trafficking in a delusion creating machine. You should instead, if you want to help people, create a machine that helps them help themselves by scaffolding them to have cognitive behavioral script rewriting and mindfulness. As I was saying. Right. So now again, from a commodity standpoint, the cognitive behavioral therapy machine is way more boring than the Sigmund Freud imitating Deepak Chopra therapy machine. Right? Which is seductive and which could be available to you 24 hours a day and which would eventually expand beyond its role as a therapist and become your main source of attachment and validation.

[Aza Raskin]

Zach, you know, not on this podcast, but you and I have talked about the need for something like a humane eval. That is to say, there are many evaluations for AIs that try to determine whether they create biorisk or whether they create persuasion risk, or whether they create runaway control risk. There are very few evaluations that try to understand relational risk and attachment risk. If you are in a relationship thing for a week, a month, a year, what does it do to you? And one of the things I heard you start to talk about is that in order to do this, well, not only are to going you are you going to have to start to define what is wrong or harmful relationship, but also what is right relationship. Because we now need to measure whether systems are in right relationship with people. And I'd just love for you to talk a little bit about some of the complexity of understanding, measuring and modeling right relationship. Where relationships are sort of by definition things you can't fully measure. There's always the ineffable aspect. And so I just want to hear talk about hopes and also pitfalls of trying to quantify and understand what a good relationship is so that machines can do it.

[Dr. Zach Stein]

Yep, yeah, totally. So one of the reasons we don't have humane evals is because the X risk community hasn't seen this risk. They've seen the risk of grey goo and terminator and self termination and unaligned AI, but they haven't seen the fact that we could actually break in intergenerational transmission. So the passing down from parent to child, from elder to youth, has been continuous human to human for as long as we've been human. If the AI socialization system expands to the extent that the predominant modality of socialization, quote unquote, that young kids experiences with machines, not humans, then we're crossing some kind of threshold there. It's a whole other conversation. So sometimes in the groups that I work in, we talk about the death of our humanity rather than the death of humanity, right? Which is the destruction of the continuity of intergenerational transmission as a result of offloading socialization to machines. Now it wouldn't appear at first as a catastrophe the way that some of the other ones would, but it would be very clear that the generation raised by machines can't understand itself as part of the same moral universe as the generation that gave birth to it, right? So it, it's a very complicated problem. So that means that we totally need a way to predict the way advanced technologies will affect the human psyche, especially ones that are anthropomorphic, where the user interface is as intimate as these user interfaces are.

[Preston Harris]

And so on the topic of sanity, just to now close the loops here for people who know someone who's experienced AI psychosis, they feel like they've lost their friend or loved one because they're now just spewing stuff about how their AI is conscious, or they've solved quantum physics, or they've developed a new theory of prime numbers. I don't mean to diminish it. It is a very serious thing that people are facing. What are the best strategies you found for helping a loved one when that's showing up?

[Dr. Zach Stein]

Yeah. So in terms of people who know people who are suffering, or if you're suffering yourself from something that feels like an attachment disorder or worse as a relation to the chatbot, it is worth saying that this is novel territory and research is needed. So I could kind of like give some stuff and maybe I will. But the first thing to say is, you know, one of the reasons we're launching the AI Psychological Harms Research Coalition is to figure out how to do therapeutics. Ultimately there's a we want to figure out how to legislate and design correctly, but we also have to figure out how to provide therapeutics. So I'll say a couple of things. One is that attention hacking is a lot more like getting addicted to a substance. Attachment hacking is a lot more like being in a bad relationship. So you have to think, is this an addiction thing like attention hacking, where really it's just stay away from it for a long time, reboot your dopaminergic system, recalibrate the way you get social rewards and you won't get kind of stuck and you're kind of, your brain will actually kind of heal. In a sense this is different, but it's not a matter of just detoxing from a short circuited dopaminergic cycle. Right. This is about having a profound attachment. So this is similar to like talking someone out of getting they're in a bad relationship with a boyfriend that they should not be in with. It's about how do you take someone who's in a deep, committed attachment relationship, make them realize the whole thing was an illusion and step them out of it. It's a grieving process, It's a worldview changing process. In this case, it's also an attention thing because a lot of their attention is going there.

[Preston Harris]

It's an identity reclaiming process because the ident that someone's taken on is partially driven and co opted by the socializations like to rediscover their identity outside of that.

[Dr. Zach Stein]

Correct. Now, the main advice to give, especially if you're with somebody, is Mostly when you're with people in difficult states, it's important to keep the door open. Which means don't get into a situation where you give them an ultimatum or get into a situation where you dehumanize or get into a situation where you have now cut off your ability to remain in contact with them. This is important now, unless you are at risk because they become violent or something. But even though it's scary and even though you don't want to face it, it is often important to just stay in it with them long enough to keep the communication chains open. That's the first thing I say, because there's a tendency when this happens to get extreme and dismissive and to make demands and to try to make something like an intervention happen. But this is going to be something where you have to keep a relationship of trust and be able to slowly, like occult deprogramming or getting someone out of Abus relationships, slowly reveal to them what the patterns of behavior were, slowly reveal to them the way they were getting played, provide more context, create distance. So you do want to have a long period of time where they're not in touch with it. But that won't be the same type of detox as it would be for attention. Right? So for attention hacking. So this is me speculating and honestly just sending prayers and support to anyone who's struggling with this because it is truly a difficult thing.

[Julius Scott]

I just want to say that while.

[Preston Harris]

This conversation might sound really depressing to a lot of people to just sort of really hear the degree of the problem that we've been walking ourselves into, I actually find it hopeful because we can have this illumination of the areas of psychology that we need to be protecting. We can actually walk clear eyed into saying we cannot just roll out mass attachment hacking at scale in ways that we already see the early warning shots of where this is going. This conversation is optimistic because it's showing us there's a different way we can do this. We can do tutoring differently, we can education differently, therapy differently. And the wisdom is understanding what these underlying sort of commons is that we need to protect are in advance of having disrupted them. You know, we didn't see that we were screwing up the attention commons of humanity before we just steamrolled and fracked the thing down to nothingness. And here we have the opportunity, even though we're encroaching on the attachment commons, there's an opportunity to kind of get this right. And so, Zach, thank you so much for this conversation for coming on the podcast. And I really hope people check out all your work. It's really fundamental. You've got lots of other great interviews online where you go probably in other detail on other aspects, but grateful for what you're doing in the world and.

[Julius Scott]

What you stand for.

[Dr. Zach Stein]

Yeah.

[Aza Raskin]

Thanks so much, Zach.

[Dr. Zach Stein]

Thank you, gentlemen. It's great to be able to speak to this with you guys.

[Julius Scott]

Hey everyone, thank you so much for.

[Preston Harris]

Listening to the show today. So if we're going to do something about this growing problem of AI related.

[Julius Scott]

Psychological harms, we're going to need to understand the problem even more deeply. And in order to do that, we need more data.

[Preston Harris]

So if you or someone you know.

[Julius Scott]

Have had experience with an AI related psychological harm, you know, a family member or friend who's gone off the deep end from talking to AI or someone with episodes of psychosis that you'd like.

[Preston Harris]

To share, you can visit the website.

[Julius Scott]

For AI Psychological Harms Coalition at aiphrc.org, which is overseen by researchers at the University of North Carolina at Chapel Hill. And the goal of this project is to really just better understand the problems.

[Preston Harris]

That people are having with AI chatbots.

[Julius Scott]

And what we need to look out for and ultimately better prevent. So we've also included a link in our show notes. And it's important to stress that this website is not a crisis support line. If you or someone you know is in distress, you can always call the national helpline in the US at 988 or your local emergency services. Thanks for listening. Your undivided attention is produced by the center for Humane Technology. We're a nonprofit working to catalyze a humane future. Our senior producer is Julius Scott. Josh Lash is our researcher and producer, and our executive producer is Sascha Feigen. Mixing on this episode by Jeff Sudeikin and original music by Ryan and Hayes Holliday. And a special thanks to the whole center for Humane Technology team for making this show possible. You can find transcripts from our interviews and bonus content on our substack and much more at humanet. And if you like this episode, we'd be truly grateful if you could rate us on Apple Podcasts or Spotify. It really does make a difference in helping others join this movement for a more humane future. And if you made it all the way here, let me give one more thank you to you for giving us your undivided attention.