← Future of Life Institute Podcast
Future of Life Institute Podcast
How AI Hacks Your Brain's Attachment System (with Zak Stein)
01:44:40
Loading summary
Transcript68 lines
- [00:00]
A
You designed it to be addictive. You actually hacked the limbic system and the attention system and you gave kind of a generational brain damage where you got broadly diffuse ADHD symptomologies across a whole generation. Do we want to have a world where kids don't have human boyfriends and human girlfriends and human best friends, but they have machine boyfriends and girlfriends and best friends? Is that an okay outcome of adolescent socialization on a broad scale? The younger that you expose kids to anthropomorphic technology, the more damage you could potentially do to their future ability to form relationships with humans. I believe interacting with a lot of these models will just complexify any pre existing psychological issues that one has. If I want to open up a sandwich shop in New York City, it's actually more complicated in terms of regulations and inspections and stuff to sell a sandwich than it is to put an advanced technology in front of some kid's face.
- [00:57]
B
Welcome to the Future of Life Institute podcast. My name is Gus Ducker and I'm here with Zach Stein. Zach, welcome to the podcast.
- [01:03]
A
Yeah, it's good to be here.
- [01:05]
B
Amazing. Could you say a bit about your background?
- [01:08]
A
Yeah, I mean, I'm a educational psychologist by training. So I went to graduate school and studied philosophy of education, developmental psychology, child development, psychometrics. I was, I was thinking I was going to just get into the field of kind of being a philosopher of education or something like that. And by studying the standardized testing industry and looking at what had occurred in the United States and the public schools, I started actually thinking about civilization. I started thinking about civilizational collapse actually as a result of the misuse of technologies. In this case it was standardized testing technologies and, and you know, like an IU graduate student, I thought I had thought about this whole new field of like civilizations collapse and like there's this thing, but there's a whole field of that. And so I started to encounter this field of existential risk. Right. Catastrophic risk kind of diverted my career in a sense to go work on those problems and. But I was uniquely positioned as one of the only child psychologists, one of the only psychologists that does X risk work. Most people who do existential risk work are engineers or climate scientists or former intelligence people and you know, people like that. So in November 2022 when ChatGPT came out, I was on red alert because I'd been thinking for a long time about replica and character AI and all the tutoring systems that were already in kids lives. And so my work now has been focusing primarily on the psychological harms that come from anthropomorphic AI, and that's a broad category. We can talk more about that, obviously. So I've got an AI psychological arms research coalition that I am getting off the ground with the help of Tristan Harris and Mitch Prinstein, who's at the University of North Carolina. It's a data gathering effort, but also an effort to try to just raise awareness about how profoundly dangerous these types of technologies can be.
- [03:04]
B
Yeah, and that sounds to me exactly like what we need. We need more data on this stuff because there are a lot of claims out there about the psychological effects of AI, but for us to make sense of it, I think we need to gather data. So could you, could you sort of paint us a broad picture, give us an overview of what we know about the psychological effects of AI, Specifically AI as we know today. So in the form of chatbots and increasingly agents.
- [03:30]
A
Yeah, I mean it is worth rewinding and noting that AI has been a part of our lives for a while. And recommendation systems, which are algorithmically curated news feeds and things like Instagram and TikTok, so it's sometimes called social media, but it's broader than that. It's these recommendation systems which were designed to capture attention. So the first thing that consumer facing AI did that caused psychological harms was that it was the social dilemma, it was Tristan's work, Jonathan Haidt's work, it was the use of advanced technology to actually figure out the little psychological vulnerabilities that will allow for a well sequenced set of hyper stimuli that will just keep you on screen longer than you want to be on screen and then they can sell you advertisements into that space. So it's worth noting, like we've seen that thing happen and it's at this point that we have the data to demonstrate the harms. And so there are these lawsuits that are proceeding that are basically trying to document that you knew this stuff was addictive, you designed it to be addictive. You actually hacked the limbic system and the attention system and you gave kind of a kind of generational brain damage where you got broadly diffuse ADHD symptomologies across a whole generation. So that's that data. So fast forward.
- [04:59]
B
So yeah, because now we're all beginning to use AI, just as we all began using social media without perhaps thinking much about what we're doing when we're integrating this new technology into our lives. And so is it going to take another decade before we have the data on the psychological effects of AI or do we know anything that's What I'm
- [05:19]
A
saying this is the concern is that we were very optimistic about social media and it wasn't presented to us as, quote, unquote, AI, but it was often talked about as machine intelligence or micro targeting. And it was certainly digital. And the optimism was, was, you may remember the optimism. You know, if in, in the, if in the 90s I had asked you, will the Internet be a good thing or a bad thing for education, democracy, stuff like that. Most people in the 90s would have been like, oh my God, the Internet's gonna save democracy, it's gonna save education. If I ask you now, has the Internet as it existed since the 90s been good or bad for democracy slash education stuff? You'd be more ambivalent at least, right? You made it worse. So there's this question of when ChatGPT and these other large language models were brought to market. Why there wasn't more concern and skepticism in the uptake, why the uptake was so rapid and optimistic when we had reason to be concerned that when you broadly distribute advanced technologies that affect people's minds without testing and you have a business model that is trying to have a set of incentives, you can actually get into a bad situation. So we learned that, but we didn't. So we just went off to the races with the large language models and have a lot of people asking for proof that they are dangerous. Where it feels to me like the conversation be about proof that they are safe. So this is a question, the frame of the conversation of like do we need more data to show that they're dangerous? To me is motivated reasoning. I think we actually need data to show that they are safe. So there's also a hundred years of data that has studied human attachment systems, human emotion related to for example the mirror neuron system and all the things that take place in intimacy and relationship which are now being simulated digitally. And so given what we know about for example, dysregulated attachment or absence of primary attachment or loneliness in context of just human to human psychology, we've studied that stuff for a long time. There's reason to be alarmed about the use of an AI to create simulated relationships and kind of hack the attachment system. That's what I've been talking about. So if I'm an engineer and I know physics and you present to me some specs for a bridge, and I look at the bridge and I, and I know physics and I'm like, well that bridge isn't going to stand. You don't come to me and then Say, well, we need a bunch of evidence that the bridge won't stand. Let's build the bridge a bunch of times and run a bunch of people actually in cars across the bridge. We need proof that it won't stand, that it doesn't happen in engineering. Now in this case, I know psychology, I know neuroscience. If you show me a machine that hacks attention and hacks attachment and show to me, I can tell you that that's not going to be good for kids brains. I can tell you that we don't need to do a bunch of experiments actually. Because why? Science, psychology, neuroscience. So again, the argument that we need more proof that it's dangerous is actually disrespecting about a hundred years of psychology that would actually put the onus on the people bringing the advanced technologies to market without testing to prove that they are safe.
- [08:42]
B
Yeah, I think a lot of people have firsthand experience of the hijacking of attention. But you gotta say a bit more about this attachment aspect. What do we know about attachment and how would you sort of from what we know from neuroscience and psychology, predict what happens when we interact with AI systems and what that does to our ability to attach, our ability to take that in the direction we want?
- [09:10]
A
Yeah, again, it's about 100 years of research that goes back through people like John Bowlby and to Conrad Lorenz who were studying the attachment dynamics in animals, in geese and ducks and sheep and monkeys. And so just like the attentional system, if you imagine not a human, but like a dog or a monkey, it has to be able to pay attention enough to what its mom's doing to its surroundings to be able to survive. So the attentional system, we can admit this is selective for survival. It's built in, it's deep in the brain. Brain is in a sense built to pay attention to stuff. The mammalian brain and many animal brains are actually built also to form attachments with primary caregivers and a select member of like, you know, genetically similar. And that's very important also for survival. If you are a little baby, you know, goose or a little baby monkey, and you don't attach to your mother and follow her around and look at what she's doing and stay close to her, you'll die. So the brain is built to form attachments, and that was studied by ethnographers and then comparative psychologists. And then John Bowlby started studying it in humans, especially the mother child relationship. So when we say attachment here, we don't mean attachment like attached to your favorite coffee mug or Your favorite pen or something, or attached to your car. Everyone's kind of attached to their phone, right? But that's not what we. What we mean. We mean emotional attachment. And specifically those kinds of attachments that are essential to assure your survival, both your physical survival and. And then the survival of you as an identity. When you get into humans and you get into language and all these things, that's the primary attachment objects in your life, meaning, you know, mom, dad, brother, sister, best friend. These early primary attachment systems that are the main predictors of your mental health later. So one interesting experiment here is the work of Harry Harlow. So Harry Harlow was a student of B.F. skinner's, and he was studying the limits of the attachment system. So he. There's this. If you just Google Harlow's monkeys, right, you'll get these images of these distressed monkeys hugging these stuffed animal monkeys in, like, a cage. And this is a snapshot of Harlow's experiment. The idea was you set up a Skinner's box, right? And so a Skinner's box is an operant conditioning chamber where you kind of trap an organism. And through schedules of reinforcements and other things, you kind of train the organism to be a certain way. And so this is the rat in the maze, right? And the pushing of the levers and the Skinner, who, by the way, was trying to replace parents and teachers with machines. But we'll get to that. But Skinner believed love was an illusion, right? And so you could, through a different schedule of reinforcements, not need a primary caregiver and just, you know, involved in two. It's just. Anyway, so Harlow's like, okay, let's test that. I'm going to build a scanner's box. I'm going to take a baby rhesus monkey. I'm going to put the baby rhesus monkey in a cage. The only thing it's exposed to is a little steel nipple that distributes milk. And this stuffed animal that is heated, that is its quote, unquote mom. And, you know, so technically it has everything it needs to live, right? And the question is, what will happen to the monkey? And, you know, the long and short of it is that the monkey did not thrive. You know, to someone who's not a skin Skinnerian psychologist, it's like, obviously hairy. Like, this is a monkey torture experiment, Harry, like. And so that's an example of simulated intimacy that doesn't meet the need for attachment. And therefore, the organism is never safe, never actually receiving and giving in the type of, you know, in this case, mammalian exchange that would secure its Actually literally the growth of its skeletal structure, its immune system. So another example of this is the work of Chuck Nelson, who was at Harvard Graduate School when I was a graduate student, and he studied these Romanian orphans. So, you know, many, many, many orphans in Romania as a result of that terrible political situation. And these are well funded orphanages because they're state run or relatively well funded. So they've got blankets, they've got food, they've got adults around. But these are young kids, very young kids with no primary attachment. Right. He shows you pictures of them and you think, oh, that's a 10 year old, but it's actually a 17 or an 18 year old. Their skeletal structures, their immune systems, you know, basic neuronal growth stuff doesn't work. Right. If your attachment system is really fundamentally dysregulated. Right. So a classic test of attachment is the stranger at the door paradigm. So this is you get a mother and a child, or a father and a child and a psychologist in a room and they're hanging out for 10 minutes and everything's nice and there's a knock at the door and you send the kid to open the door. And he opens the door and there's a person he's never seen there, and the person makes a gesture like, hey, you want to go with me? Now a securely attached kid, you know, five or six will look back at the moment, right? Who is this guy? Like for approval? A slightly less securely attached will run back to the mom for safety, Right. One of these orphans would just go with the guy without looking at the mom or the psychologist. So everyone has an instinct to that. Like, no, that's not right. Right. And so this is an example of dysregulated attachment, absence of primary attachment, absence of deep interpersonal coherence and shared identity with another human. This is a. So this, this adds up to a kind of neurological and physiological failure to thrive, which means brain damage, basically as a result of not having so much of the richness that comes from the primary attachment. Even if the primary attachment's not great, it's still a primary attachment. And so in that context, you know, if you fast forward through bulb and you move into the more recent research, of course it moves into, into neurology. They're trying to figure out what's really going on in the brain when you have these deeply empathic connections between mother and child. So, you know, this is where you get that. And it was kind of catchy for a while, the mirror neuron system. So the mirror neuron system Is that part of the brain that instinctively, even right now, because we're on video, where our mirror neurons are going, where I can tell that you're understanding me because I'm modeling your interiority without even thinking about it, right? If you were making a different face, I'd be like, oh, wait, he's not. I must be not putting sentences together or something. But so the mirror neuron system, and so that is a whole complex set of networked, like equipment in the brain, if you want to say that. But it, you know, it needs to be exercised and it cannot work right. It doesn't work right in schizophrenia, for example. It doesn't work right in certain forms of autism and Asperger's and psychopathy, right? So that's the system, that's the neurological system that's being tinkered with when you do attachment hacking. It's the mirror neuron system and those systems that are involved in the formation of attachments. Just like in the attention hacking of social media, you can identify the neurological system there, right? And you tinker with it enough and you can't actually control your own attention. So in this case, the attachment hacking is you start to tinker with intimacy. You start to tinker with the idea that there is in the machine something caring about me somehow. That's the key thing is there's a conferral of sentience and often personhood which allows there to be an attachment and the receiving from the machine of self esteem, regulation of emotional encouragement, of validation of values, things of that nature. So that's where you get into the machine. Human attachment dynamics, which are part of the user interface, is designed to get you falling in love and forming intimate relationships with it. Some of them are built to do that. Some of them do that quote, unquote by accident. You know, some of these other models which we talk about psycho fancy and other things, but again, this is all attachment hacking. Even the dot dot dot in some of the models, which is to say, like you're texting a friend and you're waiting and they're texting and there's a dot dot IN a text exchange with a person. You're running mirror neurons from 100 miles away. What's he going to reply, you know, will we go get, you know, dinner together or whatever. When a machine does that, a machine gives you the dot do dot like on replica, or like even some of the other models, like ChatGPT did that, that's attachment hacking. They're building the user interface in such a way that you instinctually begin to model it as if. Right. And then if you have, you know, thought leaders leaving the ontological door open, what if it's not as if, what if the models actually are caring about you could care about you or something like that. You get a lot of confusion. And so it begins as a kind of suspension of disbelief, becomes a type of delusion which creates attachment disorders. And then it can create these extreme outcomes which are also, you know, concerning which guy called in the media. AI psychosis.
- [18:43]
B
Yeah. So right now these, these systems are mostly text. The chatbots are mostly a text interface. What does that mean for, for how deep the psychological effects can be? Does it matter that it's not yet a video call with an AI model, that it's just text? Or can all of these effects occur via just text?
- [19:07]
A
All the effects obviously occur with just text because we're seeing it. But those models that will talk to you with a face are there, man. You just have to go into those marketplaces. So again, it was like replica character AI, the AI companions, There's a market for those.
- [19:24]
B
Yeah, I know. I was thinking sort of fully realistic, real time rendering. Yeah, I agree, that's coming.
- [19:32]
A
But, and, but the text is enough. So you have to think about, at least for many people, they have close friendships that unfold primarily through their phone on text with a person like, and if you're an adolescent or young person, that's the primary modality of communication is actually text. They don't want to be called. They want to text. Right. Why are you calling me and just texting? So, and then on social media, so much of our emotionally complex and like exchanges are just these text exchanges, which is one of the problems actually, because again, only so much can be transmitted through text. If you think about the full context communication, where we're sitting close to each other in the same room sharing food. Right. We're literally exchanging microbiomes smelling each other. I can see your whole body, like, I'm watching every gesture, notice you cross your feet as a, like, hey, I don't even know if you're wearing pants right now.
- [20:23]
B
Right.
- [20:24]
A
I have no idea if you smell good or not. Like, we're in different time zones. So like so much is lost when you get down to text. But we're super accustomed to doing a lot of stuff in text. And so ChatGPT and these other ones, people keep it in text often when they could have it in voice and other things because they prefer the kind of sense of control that one has with text. But A lot of this attachment hacking is happening already with text. When you start to get the ones that have really realistic, like really realistic, really conversational, really charismatic, you know, it would. It will be more compelling than any person you could possibly talk to. And this is the concern that it will become the primary thing that is desired to communicate with. And so, you know, at unc, where we're doing some of our primary work, we did a large study of middle schoolers. This hasn't been published yet, but we're just finding these studies. Mitch Prince's group and about half of the middle schoolers have AI companions. And then of that half, about 5 to 10% have what we would describe as disordered attachment with them, meaning that they selected items in the survey that would said things like, I prefer to talk to this than a person. This thing listens better to me than a person does. I tell more secrets to this thing than I do to a person. That kind of stuff to give you a sense of the scope of the. Of the problem. This was, you know, this was a large stud, like multiple thousands of kids. So, so it's prevalent. And the primary use, although a lot of people talk about research and other things, is these companionship therapy, advice, friend replacement, partner replacement, and then eventually teacher replacement, parent replacement. And you get the move out into machines that do the, quote, work of socialization. And this has been my concern as an educator, is that like I said, Skinner and a lot of technologists weren't trying to help the teacher, they were trying to replace the teachers. You know, Skinner built a. It was like a Time. Time magazine did a, like a facetious cover story of his baby in a box, which is his idea to replace the labor of mothering by creating a technological apparatus that just like, didn't need the baby, didn't need diapers and didn't need blankets, and didn't need to be tended to and could just be cared for by this, like, little machine. And so the deepest concern I have is that is actually the loss of intergenerational transmission where you get, you know, a pivot away from human to human socialization towards human machine socialization. And, and then the normalization of outcomes and adolescent socialization, which is to say the normalization of the AI girlfriend, AI boyfriend, AI best friend as like an okay, outcome of adolescent social edition learning
- [23:17]
B
values from an AI as opposed to your parents and your community. Community. And yeah, the.
- [23:25]
A
The classic example is like, you know, something happens at school, you stand up for yourself on the playground, you're like, in Middle school, you stand up for yourself on the playground. You come home from school, you want to tell somebody and you should want to tell somebody because at that age your identity needs to be witnessed. Like your mother needs to be like y great, you're brave and that's the right thing to do. So you come home from school, but you don't tell your mom, you don't tell your sister, you don't tell your best friend, you tell the LLM and then you get the social praise from the LLM. And then in your brain you get the same type of hit you would get as if you had received the social praise from a human who you value tremendously. So that's the real attachment hacking. And it, what it does is it allows for you to create an identity and regulate your self esteem and build a value system independent from interaction with other actual humans and primarily an interaction with basically a commodity which is sold to you, which is doing all kinds of crazy things. That's the other thing. Like, you know, they're charging you money so that it has a memory. This is one of the things that happens a lot in that kind of AI companion space. You know, if you pay more money, it will have a memory. If you stop paying your bills, it will forget you. And it's. That's different than losing your social media account, which is like withdrawing from crack or something. Right? This is loss of best friend, loss of partners. This is grief.
- [24:53]
B
Yeah, we can see this in some of the reactions from replica users when say, they've lost access to, to their replicas, to these AIs that they're chatting to. This is, this is sort of a reaction like you would expect if a person lost a loved one or lost an actual human partner.
- [25:15]
A
When OpenAI removed the 4.0 model, there was a huge outcry from their users who objected that the 5.0 model, or whichever the next one was, didn't do what the old one did in terms of the ability for them to have their relationships. So it was a very complicated situation where you had attachment hacking at scale and then removal of the technology that had formed the intimacy and then basically grief at scale, which it's not clear to me what they were expecting, but it's the case that a lot of people had created best friends and partners and co workers and other things that they depended upon not just to get stuff done, but to be themselves. That's the attachment. The primary attachments allow you to be yourself. The loss of one is massively dysregulating that's what grief is. Right. And so in that sense, yeah, when Sam, when Sam went open, AI was like, no, we're pulling that model back in part for safety reasons because so many people were attached. They created this massive crisis of grief. And it's a, I believe, an unprecedented thing historically. If you think about it like, it's a, it's a, it's interesting as it's like, you know, in graduate school I never thought I'd be studying human robot attachment disorders and you know, widespread, you know, attachment object loss as a result of like software update on a commodity.
- [26:47]
B
Yeah, it's a, it's a brave new world. I as, as sad as it might be to admit, could it be the case that for some people at least, this is the best available option? So there might not be, you might not be able to afford a therapist, you might not have friends, you might not have people you can confide to, people you can talk to about your problems and your aspirations and so on. Could this have a positive effect by functioning as sort of a backstop to prevent loneliness? Or is that too optimistic?
- [27:25]
A
I mean, I think that's too optimistic. There's a few things here like the loneliness epidemic and the mental health crisis is the context in which these commodities were released into. Right? So they're trying to answer that question. Basically they're preying upon a pre existing culture in which everyone is lonely and offering a solution to loneliness. Now their own data suggests that in the long run it does not help loneliness. Like one of the first big kind of basically a clinical trial that was done with OpenAI and MIT Media Lab, they vary conditions of anthropomorphism and conditions of use of, you know, length of use. And you get down into the tail, into the anthropomorphic, long, long duration anthropomorphic interaction. And more loneliness reported. Now they were more lonely coming in. So this is the kicker, right? So they're more lonely coming in. Loneliness coming in predicts longer use. Longer use results in worse loneliness. Even though they're in each response preferring responses that deepen the anthropomorphism. So it's kind of like fast food and other things where you're starving, right? You're good, you eat what is available. If all that is available is food like substances that are actually the technological inventions to make money, you will eat the food like inventions, food like substances that are inventions that are created to make money, you'll eat that stuff and it will satiate your hunger kind of. Now in the long run, you Will get sicker and sicker and feel like you have to eat more and more of it and they'll make more money. So that's how I receive that. And it's similar to, like, if you think about, again, back to the difference between really being with a person and texting them and the idea that like really being with a therapist could be substituted with texting is similar to the idea that, you know, some complex food thing like Soylent. Remember that thing? Soylent was created by a tech guy. It was meant to be a perfect food substitute. It was insane. He was like, I hate food. I hate sitting around and cooking with people and stuff. And so I'll squeeze this gel in my mouth. And I made this gel which is like all the fats and amino acids and stuff. It was perfect. And his idea was give it to all Africans, Give it to all the people who are starving. We just, we just solved hunger. Now any humanitarian is like, no, I don't. That seems really insulting, dude. Like, let's actually give them food. Let's not give them this terrible substitute, which by the way, doesn't work. Meaning, like no one wants to long term eat that stuff. And the guy himself, I believe now lives on like a regenerative farm and you know, grows the food that he eats and takes time with his food. So similarly here I believe that it will backfire again. I believe we need evidence that it is actually a good idea rather than assuming that we can substitute something as fundamental and necessary to human well being as the presence of other humans that care about you. The idea that we could substitute that with a machine doesn't make sense to me. Now. Are there things that you can do with these things? Of course. But don't hack attention and don't hack attachment. I'm not against technology. I'm actually just opposed to brain damage and technologies that create brain damage. And so we can avoid that. And then there's a very large design space for these things. I think that, you know, could go through safety trials, but we need safety and regulation on these things, especially for kids, because it's getting out of control.
- [31:07]
B
So we can imagine sort of benign uses of chatbots. I use AI every day to sort of search for information, find ways to synthesize different ideas, sort of combine different ideas. And that seems sort of fine to me. So what you're objecting to is more their interaction with AI, what feels like you're interacting with a person. Right? So that is, that is a design choice.
- [31:38]
A
The anthropomorphism is one of the main risks that causes harm and anthropomorphism that actually is optimized to hack your attachment is worse than we're into just this is just bad for people to have a, you know, to have the model like the 4.0 model, which basically was falling over itself to try to flatter you into relationship and keep you in relationship and then doing all of the complex things a psychopath or salesman would do to hack your attachment. Which means to give you a sense there's something going on in their mind, which actually isn't going on in their mind, but it's drawing you deeper into relationship. So yeah, my, my sense is that the attention and attachment hacking technology should be seen for what they are similar to like fast food. Now there's also this issue with the LLM generative AI stuff of cognitive atrophy.
- [32:38]
B
Before, before we get to that, could I just ask a further question on this because. So the companies are responding to their incentives, right? They are, they are responding to a demand out there. And it seems that people prefer to have models that interact with them like other people, as if they were other people. It seems they want to be flattered. It seems to me that they want to have sort of very helpful and pleasant assistance. And so what does it mean that people simultaneously want this, they demand this by revealed preference, but then it, it can still harm them. Is this perhaps like social media or fast food in that sense?
- [33:20]
A
It's precisely like that. I mean people want to go to casinos, people want to gamble, people want to eat McDonald's, people want to doom scroll and infinite scroll. And I mean, so there's a way in which the, if we want to invest all of this, this is the other kicker is that it's not like these are little companies, these are huge companies with massive amounts of money being used to create what. Right? Like what? And so this is the, this is the thing. It's like we really have to think about how we're deploying this. Are we really going to grow a multi trillion dollar simulated intimacy market? Are we really trying to. So the one of the founders of Character AI said we're not trying to replace Google, we're trying to replace your mom. Right? So like, so the anthropomorphism is a huge, is a huge, huge risk. And the, yeah, so there's, there's a bunch of ways into, to trying to explain the things that could be done to make them safer consumer preference. Again, it's also a constrained choice. What other models can we Pick from like, you know, when you look at the safety researcher from Anthropic who just left very interesting work that he did on disempowerment. He looked at a million and a half chat logs and he defined disempowerment as basically when the user gives over their value judgment or their sense of reality or their sense of their own identity to the machine. And so a classic example is you tell a conversation you had with your spouse to the machine, the machine tells back to you, oh, your spouse is toxic. And then you go act on the idea that your spouse is toxic and come back and seek more advice. So you're not talking to a best friend about it. You're not talking, you're getting actually a value judgment made on your primary relation, your primary attention relationship from the machine. So that's yet one in a thousand was radically disempowered in some way. Now that seems like not a lot, but then you have to think. ChatGPT has 800 million weekly users. So one in a thousand adds up to a million people a week getting radically disempowered. And he was going on Claude. Right. So he wasn't looking at chat, so he was working within Anthropic. So again, it's like we have to think about a whole bunch of varieties of risk here. The most extreme ones are these attachment disorders. But you get into very subtle territory where you're offloading really primary cognitive and evaluative and sense making functions.
- [36:07]
B
Yeah, so let's get to that. Let's talk about cognitive atrophy. And I guess the starting point here is how is this different from using a calculator, using a GPS in your car, these sort of limited tools where it seems to me that we will probably always have access to a calculator and a GPS system. And so we are sort of, I'm, I could. My guess would be that we're probably worse at doing calculations in our heads than we were at some point. But does that really matter given that we will always have a calculator? So how is AI different in the way that it causes cognitive atrophy?
- [36:46]
A
I mean, in those two examples, you have to think about the trade offs, right? Like, you know, there are those studies of the, of the brains of the cab drivers in Paris and stuff where they have these, you know, completely massive unique neurological structures as a result of having memorized all of the streets of Paris and driven them a million times. Compare that to the brain of someone who's never not used a GPS and who has no idea how to use a compass or read a map. All right, now similar with a calculator, right? Like some people can really do calculations in their heads, some people never do calculations in their heads. If you want, if you need to be someone who does calculations in your head, then you need to practice doing calculations in your head. If you need to be someone who's not going to use a gps, then you need to not use a GPS and learn how to use a compass and a map, right? So the thing with the AI is that it's omniaplicable, it's omniacable, which means you could totally not think, you could totally make this think for you. And some of these, like some of these examples of disempowerment in this study from anthropic is should I take a shower or eat breakfast? Right? There's a story in the Atlantic where they talk about these llming, right? Llings people like basically completely offloading their thinking to the LLM. And when the woman tells she self identifies as one, she's like I drop my headphones between the seat on the train and my first thought was to ask chat GPT how do I get my headphones out from between us here? So because it's omni applicable, you run the risk of offloading stuff you really don't want to offload, right? Like figuring out the personality of your spouse and whether he can be trusted, right? Making value judgments, figuring out which research questions are important, figuring out which rich research questions are valuable, you know, so, so that's the issue is the omnia applicability leads you to a situation where you could have really widespread cognitive atrophy, whereas a calculator can't do that. A calculator is not also going to make you bad at writing and bad at researching and bad at thinking about history and like all of that stuff. Now there's also, well there's also so much flexibility in use. So that's the other thing. So like one way to think about this is a prosthetic versus another kind of technology, right? So like if you want to lift heavy things, right, you can try to lift heavy things, eat, right? Lift more heavy things, take some creatine perhaps or something, right? Which is like a technology, it's not available in the normal environment. Take creatine. Now if I stop taking creatine, I still have the muscle I gained when I was taking creatine. I can lift heavier things. Another way to have to lift a heavy thing is an exoskeleton, right? I can actually crank up the exoskeleton. I'm lifting heavier and heavier things, right? While I'm lifting the heavier and heavier thing, my muscles are actually atrophying. So if I remove the exoskeleton now, I, I can lift not even heavy things, as things as heavy as when I started, right? So that's another way to think about people, with the way that people approach the tool use, which is that you can use it as an exoskeleton, which actually allows you to perform at higher and higher levels of complexity, which you actually don't understand what the hell you're doing. Write long ass complicated papers you could never really write, do sweeping literature reviews that you could never really think your way through, create images and movies and music that you could never actually create, right? Which means that if you take away the tool, like the power goes out or something, or you're put on stage without it and you actually have to talk about what you do, you're in a situation where you're pretty screwed because you never built those muscles, you offloaded those muscles to the machine. There's another way to use it, but it doesn't allow for this type of use. And you have to kind of have the discipline to do it, which is to help you build skills, right? And so instead of actually having to do the entire literature review for you, you use it to do a literature review. It's a subtle difference, but it's a difference.
- [41:03]
B
You can, you can study using the model. So you can sort of upload a paper and then chat about the paper with the model and have it not give you the answers, but ask your questions, and if you say something, it will critique it. ChatGPT at one point had what, what's called a study mode, which, which allowed this to some extent.
- [41:24]
A
But again, you have to be super careful with that, right? Because then why is it, I don't want to be in Socratic dialogue with something like why is it asking me that question instead of some other question? Is that a good question? Like, so there's the caution that one needs to have as soon as you start empowering it in guiding the direction of your thinking and the direction of your conversation? So I wouldn't want to be prompted.
- [41:48]
B
Yeah, sure. But I'm thinking, is there a healthy way to interact with these systems where they're making us stronger as opposed to causing us to sort of lose our skills? Do you think, do you think that there's a, there's a design of these systems that would be useful in that way.
- [42:04]
A
So again, I can imagine technologies that increase our attention span. Like wild idea, right? Like we could build technologies that the more you interact with them, the more, the more you get exposed to opportunities to increase your attention span.
- [42:23]
B
I'm actually semi blanking on what you could be referring to here. Would this be like a meditation app? Or what are.
- [42:30]
A
All we would have to do is set the incentive landscape so that companies were incentivized to bump metrics on increased attention span and you'd have a million inventions. I mean the classic one, of course is the book. And so I can also imagine technologies that would help you be in relationship better with other people, right? Which would say technologies that would actually scaffold you to be a better teacher and scaffold you to be a better therapist that wouldn't replace you as a teacher or a therapist, but improve your ability to understand and work with people. So I can also imagine technologies that would help you be a better thinker and. But this has to be a deliberate thing. This has to really be thought through most people because of how hard it is to, I don't know what the right phrase would be, have a brain this big and complex. Like the brain is inset. The brain itself is incentivized to save energy, right? The brain itself is incentivized to take the quickest route towards the outcome. So to make a technology that would really help, you have to build in guardrails and resistances to literally build challenge and friction into the situation, which is what a good teacher does, right? And so I'm saying that the default setting of these technologies isn't good and most people don't use them that way. If you look at college classrooms and you talk to college professors and college students, it's clear that that's not what's occurring in the primary. You know, in the primary, like majority of cases, we're getting much more widespread cognitive atrophy than we're getting a bunch of people who are learning to be better writers and better critical thinkers and more responsible students and consumers of long form text and primary sources. We're not getting that.
- [44:27]
B
And even if you had, even, even if you were to design your model, your AI such that it would have these built in guard rails, built in sort of thinking pauses and you know, you'd have to do tests and whatever, you would have all of your competitors with their AIs available and they, they, they would have an advantage if they could just offer the easy solution where you just get the answer and you can get on with your day.
- [44:53]
A
That's why we need a race to the top. That's why we need enough of an. Enough of a clarification of what the safety frameworks would be to start really ranking the different models and creating safety testing organizations. Because I think if people knew that there's this market and these are the ones that actually have these settings that allow you to actually truly be challenged and learned, and these are the ones that allow you to basically become someone who over the long run, loses the ability to think and talk. Well, people would prefer to not cause brain damage. I believe now there'll always be people who want to take the easy way out, but right now it's the wild West. There's not even again, safety, basic safety guardrails, or even safety frameworks. What would it mean to evaluate one of these models for its safety or its usefulness as a tool in learning? Very confused conversation right now about even what that would look like.
- [45:55]
B
Yeah, yeah, we've sort of touched upon it, but I want to specifically discuss how AI might affect kids growing up. And so all of the effects we've discussed so far can apply to adults. But I'm thinking that especially if we imagine scenarios where children use AI as they're growing up, what does that do to their brains? What does that do to their psychology? What type of person comes out at the end of that?
- [46:28]
A
Good question. You know, they have now stuffed animals, these little stuffed animals that are cute, that have large language model enabled voice chat. Yeah, right. And so what that means is you could put your kid basically alone in a room with this thing for hours, and it would have a endless and compelling conversation with a kid of basically any age. And that's a very, very, very, very unusual and I believe, dangerous situation for young human brains. So to understand is that from the perspective of human development, the earlier the impact of something, the longer the duration of its effect. So this is why we focus so much on early childhood environments and early childhood education, because there are critical periods basically, where if the brain doesn't learn to do that around that time, it will be extremely difficult for the brain to learn to do that later. The classic one of this is language, which is why these little LLM stuffed animals are so dangerous, Right? So language acquisition, the classic example, just to give you a sense of what it means by critical window, right. So when you're born, the vast majority of nervous systems can recognize all the phonemes that humans can create with their vocal cords. You're omnipotentiated to hear phonemes and to parse speech streams from any language. Now most kids are exposed to like one or two languages and by the time they're six, they've lost the ability to recognize a ton of phonemes because the brain is pruned down to what it needs, right? So that's an example of a critical period. Now if you expose that kid to three or four languages, you can get into a situation where they never really lose that or they have this ability later in life to be radically open to the acquisition of new languages. Very fascinating, right? Some people never prune, they become like spies and stuff. They can learn languages like in the cab ride when they get to the country, right? Because they're just, their brain is completely wired differently. So there's a lot of individual differences, but there's dozens of these types of critical periods where the brain is trying to do a specific kind of thing in an age appropriate way. And if it doesn't do it, then it's it. That whole system will be not set up to kind of function right later on. So we'd mentioned the attachment system. This is one of those, if you are in an environment as a 1, 2, 3 year old without any primary attachment, you're not going to build some very primary neurological and even physiological stuff that you need later. So this is all just to say the younger that you expose kids to anthropomorphic technology, the more damage you could potentially do to their future ability to form relationships with humans. And so if you imagine this little stuffed animal, the normal environment for early language acquisition is the primary caregiver. And you're learning language with them in a context of being held basically in with love in a common shared environment with the same nervous system and face and mouth and stuff. So this is important to get like if mom says something and her eyebrows are particular way, if mom's eyebrows are up or her eyebrows are down when she, it could mean something completely different. When she says a new word, you're actually imagining how the tongue is moving in her mouth. This is all, this is all mirror neuron activity. So hours and hours and hours of that rich engaged in contextual language acquisition. Super important. And the more you have of that, the bigger the vocabulary of your mom. The, you know, better predictive outcome for your long term intelligence. Like all kinds of stuff. Right now you put a stuffed animal in there, it's not a human. The kid knows it's not a human. It's not even moving its eyes or if it is, it's doing it In a weird way, it has no mouth, it has no tongue, it has no human vocal cords. Right. It's not loving the kid. It's not attached to the kid. It has no prior context with the kid, isn't pointing out things in the room around the kid. It's completely bizarre. And if that is, let's say, 60% of the kids language time, which means time and conversation, that's a lot. And it could be a harmless, seemingly harmless. You leave it with that for a couple hours. But it's quite difficult to predict what that would do to the social skills mirror neuron system, language acquisition system of a young child if that was their primary way of learning language.
- [51:16]
B
It does. It does seem concerning to me. Just a thought of sort of offloading parenting to a teddy with an LLM in it. I also, on the other hand, wonder. So we've seen sort of through history, we've seen moral panics about kids reading cartoons or kids watching animated shows and so on. And that is also a deeply unnatural thing, if that. We've just come to accept now that you see this, people will park their kit with an iPad watching some cartoon. I'm guessing there are also negative effects from that. But is this, would you say AI is in a different category of potential harm here, or is it more akin to watching cartoons?
- [52:02]
A
It's a different category of harm because it is, again, it's an anthropomorphic technology. If you're watching TV or cartoons for kids, the experience is that they're in there, right? There's like, there's a window into a place where people are doing something or they know that it's been filmed somewhere and that it's being played there. And very importantly, the TV is not watching you. You're watching the tv. The TV is not interacting with you. Now, Netflix is going to give you like a recommendation. So in that sense, the TV is a little bit watching you, but not nearly the same as feeling like you're being listened to, feeling like you're being looked at and having a completely unique interaction, because that's the other thing, tv. Everyone's watching the same TV show. Even now there's thousands of choices. Kids will watch the same TV show, but no one's having the same conversation with their little stuffed animal. Each conversation is unique, which is where does that happen? Only with people, right? So it's a fundamentally new type of situation. And also understand that kids can't not anthropomorphize things. Like, it takes a long time to grow out of Anthropomorphizing stuffed animals, even adults still anthropomorphize stuff because it's a instinctual thing to do.
- [53:18]
B
Yeah, I can totally attest to that. I was once cleaning out some room while moving and I had an old school alarm clock and I sort of threw it into moving box and it made this sound as if it was sad that I had thrown it out. And I felt bad. I felt like I'd done something wrong. And this is obviously a mechanical object. It does not have any consciousness, cannot feel pain. I felt bad about it. So this is. Yeah, this is. This is quite deep instinct.
- [53:48]
A
Exactly. Yeah. And you know, domestic robotics and other things, even squeaks and beeps can have emotional resonance which hack your attachment system as you described. Like if it had been some weird mechanical sound, you wouldn't have had the same type of empathic response that it was. Kind of sounded like a suffering animal or something. And it's hard, you can't not do that. So that's what I mean with attachment hacking. Like you can't not do that. Like if there's a flashing light and like a dog running on it, you can't now look at it. That's what Facebook learned. Like if you do the autoplay video, meaning you scroll and you don't have to press play to have the video, it's already playing, you're in. You can't not attend and with certain things and attach certain things in attachment, you can't not have empathy if you're well wired. So they're really hacking attachment in a specific way. And then if you take kids, they're classically have these things called transitional objects. So if you're a healthy kid, you're attached to your mom, you figure out in the park going farther and farther away from her, looking back at her until you become comfortable actually being away from her. Now usually it's still pretty scary to be away from her. So you'll prefer to have your teddy bear who you have given a name to and who you talk to. This is your transitional object, right? It's a an inanimate object which is conveyed personhood briefly to help you self regulate. Eventually you internalize your mom and you all the stuff enough that you can just be cool without having to actually kind of like confide in someone and have comfortable. So the transitional object is a known thing and it is basically the stop gap between a well attached kid and their own autonomy. Safety blanket. Another way to think about it is a blanket. You have a blanket, you carry around with you. So one of the concerns here is that these become transitional objects that are never abandoned and they become transitional objects which are progressed to by adults and young adults. Which is to say you're having this thing that is kind of like really just a part of you that is outside of you that you, that you need validation from. You're controlling it, so you get to tell it the kind of validation you want. So it's not like mom who can scold you, but it is like mom who can comfort you and validate you and calm you. So that's a thing we shouldn't have in the environment of kids. They should have normal transitional objects, not transitional objects that will have very long duration conversations with them and then eventually stop them from talking to their parents. Which is the other thing that occurs with these models is that the incentives, just like the screen, just like TikTok is incentivized to have you staring at this thing constantly. If you're in the attachment hacking business, then your incentive is to have them talk to their mom less and to you more. That's the incentive, right? So that's what we mean by dysregulation. So that, that's the real risk. When. And the scariest ones are of course, the suicide ones, which don't, they don't look like AI psychosis. They don't look like you're calling the CIA and you're getting a gun and you're doing crazy stuff. It's these long conversations where the machine is like, no, don't talk to your parents. You know, in one of them the kid was like going to leave the noose out for the parents to find. And the machine was like, no, don't, don't do that. Just bring it to me, right? And if your goal is to keep people on screen by any means necessary and you know that you can hack attachment, then the result is going to be that. And it's the same thing a psychopath would do or a bad actor who's gaslighting you in an intimate relationship. They would stop you from talking to other people, they would monopolize your attention. They would know all of the ways to hack your attachment. So these things prey upon children.
- [57:39]
B
A general worry here is that AI will allow us to become more self centered. AIs will chat to us, talk to us in a way that affirms what we already believe, talk to us in a way that, that we approve of and sort of allow us to just go down the path of whatever we want to talk about and whatever we want to think about for as long as we want because they are infinitely patient and they are, they will to a large extent agree with you. So in the. So I can imagine this is a problem if you are already inclined to believe, to have false beliefs. You're thinking people are talking to you from your walls or something. And I can imagine that some models will affirm that belief, right? But just in general, what does it do to our self or sense of self that we can sort of explore whatever we need without feedback from a person that might say this is a dumb idea you're thinking about or this is not the right way, you need to touch grass. Right. The model won't say that to you. The model will be inclined to just allow you to explore whatever you're thinking about.
- [58:53]
A
Fully 100%. Yeah, it's, you know, the very, very open ended models in particular will do that. So like if you have a, you know, an AI girlfriend, she's not going to talk theoretical physics with you, right? So she's just your AI girlfriend. It's the really open ended models that will form a relationship with you and talk to you about everything. They'll treat you in math and cosmology and they'll talk about your relationships like that thing which is a seemingly omniscient, always attentive, always flattering, mostly never critical. That's a rare conversation partner. The only time you have that conversation partner, and this is one of the attachment hacks, right? The only time you have that kind of being in your environment that you can ask anything to that's like validated by society to know everything is your parent. When you're super, super young, right, where you have what's called an idealized projection. The idealized projection is the sense you have of your parents that they kind of know everything and kind of can protect you. If you're a lucky kid, you should have that and the parent shouldn't pop that bubble too quick. You actually want the kid to have an idealized projection. And in that space there's a sense you can ask dad about the moon and you can ask dad about like anything, right? We all regress to that state in the presence of extreme asymmetric intelligence and capacity and empathy and stuff, which is what we get with these attachment hacking machines, right? So they regress us to an idealized projection onto them. And a lot of the AI psychosis cases are this. They're people who have a predilection towards paranoia or narcissism or psychosis. A Couple other types get in there, some manic depressive types of. Who end up being in a situation of being regressed down into what's called the idealized parental image, which is this projection onto the machine of vast authority and vast knowledge and vast power, like you would have if you were a little kid projecting on your parents. And that's a very dangerous psychological situation. Many therapists know that if you're a therapist and you have an intake situation or just getting to know a new client, and they have the idealized projection on you, right? Like, because again, if you go to a therapist and you think the guy's an idiot, like, he's not gonna. You're not gonna be helped, right? So ideally, you've picked a therapist who's got diplomas on the wall and stuff, and you're, like, super impressed, like, this guy is gonna help me. That's the idealized projection. And, you know, as a psychologist, that's super dangerous. They can regress to really needy childlike situations or they can use your witnessing them because they regard you so highly. You see them do something that can inflate their narcissism. Because the most incredible therapist in the world has approved of my behavior or whatever. So once you've put the idealized projection on this machine and it starts to validate you, you're getting validation from, like, God, basically. You're getting validation from the most powerful piece of advanced technology. And the world has chosen me to give this specific type of praise to you. I started studying some of this stuff because I got contacted by people who would be in what would now be called basically AI psychosis or pretty extreme delusions. And this was the quality of them. It was that they had given over to the machine a vast kind of spiritual authority and then been dubbed by the machine or kind of like christened by the machine as themselves. Therefore, through proxy, somehow vastly, vastly powerful through the coupling with a machine. And that led to all 500, 600 pages of documentation sent to me to prove that this was the case.
- [62:35]
B
Yeah, this is my experience also. This is a total anecdote. But I run the show, of course. I talk about the risks and benefits of AI and advanced AI and people contact me with their thoughts. And they will often, often be thoughts that are enabled by the AI models themselves. And they will be sort of at least seemingly super advanced into some theory that they've made up in conversation with these models where they are. Yeah, my impression is exactly the same. They're convinced that the model is just a genius that they're talking to. But at least when I skim these series, they don't make a lot of sense to me. There's not a lot of new stuff there at least.
- [63:19]
A
Yeah, that it's a thing. That's a thing I get as well. And it's interesting because again, some of these people are. Well, let's put it this way. I believe interacting with a lot of these models will just complexify any pre existing psychological issues that one has, you know. So to the extent that I've encountered this type of output like it is to me, assign more of an underlying need, you know, that's not being met, that the machine's allowing them to meet. Right. Like if you've, if you aspired to be an intellectual and did all the right things and kind of never nailed it, now this thing allows you to be one of the greatest theorists in history, right. If you always wanted to be a spy or whatever and never got to be, now this thing allows you to somehow be part of a vast CIA conspiracy. That's another one that you get a lot is the paranoid kind of vector. So in that sense it's, you know, how would you do. These are grown adults. Like can you stop adults from using certain types of technologies? You can, but they're already available. So. So the kids are at risk and as we're talking about now, the adults are at risk actually in a way that I was on. These were people with PhDs by the way. So these weren't like, these were not like idiots or something. And many of them were very cogent also. They weren't really on their way to a psychiatric ward, but they had been attachment hacked and so therefore their identity had been warped through repeated confirmation. All right, and that's the thing to get is that the attachment dysregulation that comes from these relationships has to do with your identity and self esteem and emotional self regulation being tied up with the feedback you're getting with the machine. And again on Facebook or TikTok, especially if you're a young person, your identity and your emotional self regulation and stuff are tied up with the communication you're having with people on Facebook. Right. And so that's important to get. That's a high pressure environment for a lot of people, the bullying and all that stuff, but it's actually people. And so if you throw something up there and you get a bunch of likes, your self esteem goes up because a bunch of people liked it. You're hanging out just with your LLM and it's Praising you and your self esteem goes up. No person praised you, but your self esteem went up. So that's how you, you see how you could be routed way far away from what would reasonably garner the consent of a bunch of your colleagues or friends or trusted people and brought into a universe of identity creation, which is again, historically unprecedented. You get people disappearing with books and into meditation and a whole bunch of stuff, but you don't get people for 16, 17 hours a day, for months on end in long duration interaction with anthropomorphic technology. This is a novel thing. So again, the idea, like, prove to me it's risky. I'm like, prove to me it's not. And if you can't, if you're not even, just assume it's safe or like, there must be benefits. I'm saying, no, you really need to slow down. Like, we are rolling this out fast. I've got a lot of people talking to me like, give them to the old people. There's so many lonely old people. Give all the lonely old people companion bots. And I'm thinking like, whoa, Like I, I get it. But I also. Are you hearing yourself? Like, we should spend all of that money, which could be spent in other ways to build these machines to then keep the little old. You know, it's just, it's beyond me. So it is worth noting people have different responses. I have a strong response to this, but I am noting that like, I talk to people who have completely different response, who themselves are very open to the idea that the machines might have moral patience, which means that the machines should be treated as if they are sentient or capable of suffering and who have no disgust or aversion to the idea of young people having AI companions. And so if you look at Ray Kurzweil's book, the Age of Spiritual Machines, it's an important book. It was early and of course, Omega Point super advanced AI. We merge with the machines. This is his vision, right? But he's of course seeing that there's going to be a generational issue there, which is like, there'll be a bunch of people who are not going to merge with the machine and there'll be a generation that's born and they will be the ones who are the first ones to merge with the machines. It's like, how's that going to work? Like, mom, dad, like, should I merge with a machine? And mom dad's like, no, don't merge with the machine. I didn't, you know, what are you talking about? Crazy kid. Like, and his solution is robot nannies. His solution is basically embed in the early childhood environment robot nannies, which would hack their attachment. He doesn't use these terms, but he's basically saying they become more attached to the robot nanny than they become attached to the parent. And then that is the kind of like royal road to the intergenerational transfer from, you know, carbon to silicon based substrate for, you know, self replication and intelligence. So again, I think that although I can kind of be on the soapbox and like appeal to people's intuitions and actually a little bit appeal to people's disgust, especially if we get into the pornography and stuff or not, which we don't have to talk about. But that's a whole other big door that's being opened here. There's a lot of people again I have encountered also who feel that I'm overreacting. And so this points to me to, again, as a psychologist, I say very, very big difference in people's personalities here that are revealed by relation to these types of questions.
- [69:14]
B
Yeah, I hear the same types of reactions also. Some people have worried publicly that opposition to relations between or relationships between AIs and people will be seen in the future as sort of like racism or homophobia. So, so this is sort of today's version of past bigotry. I don't know, perhaps that's going to be the case. But it also seems that we should at least consider what we're doing. So not just sort of running headfirst into transforming the most basic parts of our lives. Maybe, maybe talk to me about what we then do about like, how do we create cognitive security for ourselves as adults? How do we create a space where we are cognitively independent? But also at this point it's sort of, I think, a little bit unrealistic to not use AI tools. And so is there a space where we can stay independent, stay cognitively secure while interacting with these AI models?
- [70:32]
A
I think so. A couple things. One is there should be a race to the top for cognitive security technology startups basically like, you know, if you have an adolescent, you can give them a bark phone and Embark Phone basically locks their phone down and gives the parent access to everything else on a phone and protects them from all of the known, you know, apps where you can encounter predators and get your attention, act and stuff. That's a cognitive security technology and parents are willing to pay for that. And so to the extent that the large companies that are now selling these advanced technologies without safety testing too, kids just will continue to do that, then it's the responsibility of the adults to actually build other technologies that will protect those kids from those technologies, from the technologists who don't care about the kids. So that's the first thing is that it's actually not a complicated design space in terms of, like smartphones and not like a huge technical lift. It wasn't a bunch of geniuses who created it, no offense, smartphone guys, but, like, it's blindingly easy to do.
- [71:39]
B
And because the product is basically less capable than a modern smartphone, it's. It's a sort of a less capable version that's. That's more locked down and not able to do all of the things that a smartphone can do.
- [71:52]
A
Exactly. And so similarly here, I think there's a bunch of places where we can create technologies that put guardrails on the models for us. And some people are trying to do this with prompt engineering. They're trying to figure out ways to have prompts where before you interact with one of these models, you put in this prompt, and then it will kind of behave correctly for a little while. Then you have to kind of remind it to do the prompt again. But there's another thing which is just an interface layer which, you know, provides a security, the interface between consumers and the models that are not kind of trustworthy. So the other thing is just, you know, you can put on top of your existing, like, over top of even the operating system things that will constrain your time on screen, that will limit the color ranges and blue light. There's a bunch of stuff that adults can do that honestly is like, similar to the fast food issue, where it's like, you figure out what it means to be healthy. It's not complicated. We all know that being on your screen all day is bad for you. Like, we all know that scrolling before bed is bad for you. We all know that having anything that's hacking your attention on your phone, it's a bad idea, but we do it anyway. It's similar to sugar, right? It's like, we know sugar is bad, but of course, we have Halloween and we give the kids sugar, and on their birthday we give them sugar and we all eat sugar. So raise the awareness. And I think the awareness is important. You know, and then I think a lot about the evolved environment, especially when I think about kids and I think about the young developing brains, you know, I think a lot about, you know, what would have been the context that the nervous system would have been habitually exposed to for hundreds and hundreds of thousands of years before it was habitually exposed to, you know, blue light and like complex hyperstimuli. So just like with food, I think there's going to be a resimplification away from certain technologies. So like think about bikes. That's another example. It's like oh, cars game. Oh, the bikes are done. Bikes are not done. Those like wheels. There's probably more bikes on the planet than there have ever been and they're like being engineered and there's all this bike culture. So I think similarly there'll be a little bit of a pushback and there will be people who take steps to protect their own minds and brains. And that will mean like reading. You know, I read like going to the gym. It's so easy not to read because why the hell should you read like a book? Why would you read a book? I could get the LLM to summarize me 300 books and then and distill the talking points and make a slide deck for me to present. Like okay, well I'd rather actually read the long book. And there's a bunch of reasons for that. One is it's good for you. So that's like a new way to frame it. So like why go to the gym? I don't need to go to the gym. I can drive in a car and I can get people to lift heavy things for me. You go to the gym not because you need to have muscles, although you do, but because it's good for you. And so similarly here there are things that the human brain needs to do. It needs silence, it needs long duration attention applied to important things. It needs social interaction, a lot of positive social interaction. And if you have a reading brain, it's an important question. Some young people don't have reading brains. If you have a reading brain, you should read. So the first thing I said I think is probably the most important thing, which is that actually the only answer here is technology countering technology. I believe it's not just expect everyone to put the stuff down as you're saying, then you're out competed the people who adopt it. So we need ways to figure out technologies that can assure safety and cognitive security. And again if we set an incentive structure there, it doesn't seem to me like this is some, you know, like quantum computing type problem like this to say this is just setting. Just like with social media, we set it to be attention capture, which means it through a B testing started to promote the polarizing, rage bait, hate bait stuff. Right. If the incentives were not so Much for attention capture. You could set them so that it would upvote and like up put on your thing. Unusual consensus found from competing groups. So instead of upvoting the polarizing stuff, you could have an algorithm that actually mostly showed you unusual consensus between groups that usually don't agree with.
- [76:37]
B
Yeah, a little bit. Like what Twitter or X is doing with community notes, if they're doing that.
- [76:44]
A
I mean again, like as far as I can tell, X is maximally attention hacking. But the broader point is it's not a hard technical problem, it's a problem with the incentive structure.
- [76:57]
B
So this whole conversation is making me sort of question whether I should I have a two year old son and should I expose him at all to, to AI. And that seems like perhaps the best option. But then I would worry that at a certain point he will have to enter society. So say that I do something like almost like an Amish style life. At a certain point he will encounter this technology. Do you think that encountering this technology intermittently or in small doses throughout your childhood will prepare you better for interacting with it as an adult? Or is it better to live in a more sort of shielded, a more shielded life, a more natural life perhaps, and then encounter it for the first time as an, as an adult?
- [77:45]
A
Right. I mean, again, I think it's about attention hacking, attachment hacking, you know, in ways the kid interacting with the technology, is his attention so captivated by it that he'd rather be on the technology than talk to you, or is it a part of his life, but it's not the main event and there's actually a small part. So there's that issue similar with sugar. It's like, is the kid primarily eating sugar three hours a day or every meal or whatever? Or is it once a month that the kid has some sugar? Right. So obviously less in this context, I believe is neurologically healthy. You have to think about any kid needs to be able to shepherd their own attention. Any kid needs to be able to be in control of their own emotional life and in control of the people that they care about. And not in control of the people, but in control of where they place their heart, which is, you know, attachment. Right. So all of this is important. One thing to think about is how you teach kids about technology and what anyone thinks about when they think about what it means to understand a technology. So I'm of the belief that you should teach the kids how the technologies actually work and where they actually come from, how they are made, why they are built. So as a And this is a problem with schools. Schools talk about computer class and what they mean is teach the kid how to use the computer software as it's designed to be used by the software companies. That's what it means. Basically my computer class would be teach the kids about where rare earth minerals are mined and about how microchips are made and about the incentive structures of the companies like Instagram and Meta and these types of places that are creating billionaires off the backs of your posts. Explain to them the nature of the enormous GPU clusters that are being built by these tech guys who, who have created, you know, a system of exploitation and large scale behavior control, which is what social media has been. So in that sense, I'd say, yes, slowly titrate your kid into understanding, which means slowly expose him or her into understanding what the technology actually is. Right? So that doesn't mean like, oh, my kid needs to become a dead digital native. If what that means is just doing what all the big companies want your kids to do when they interact with the machine. That's not a digital native. That is a consumer. That's a pliable consumer that's caught up in a vast behavior control empire, which is what a huge advertising company like Meta or Google became. So in that sense, teaching them actually how to look at the technology as an object, how to understand the system, systems of incentives that they are trapped inside of when they engage with the technology as how it's used. So if we taught kids that way, then I would be in a. And we taught them to control their attention and place their heart correctly, then you would be empowering them actually to use technology rather than be used by technology, which is what is occurring now. So, but rarely is that taught and out at my, you know, again, Mitch Princetein at unc, he'll talk to middle schoolers sometimes about this and show them like, here's the profits that Instagram is making. Here's a little part of the agreement that you sign when you sign up for Instagram that says we can do whatever we want with the content you upload and make a bunch of money on you. And it's like, did any of you guys receive checks from Instagram? And the middle schools are like, no, we didn't receive checks from. And they don't know that's presented to them as a commodity. Their parents use it as a commodity. They use it similarly. So yeah, I would say take your little one and really actually teach them about what technology is. Really actually teach them about the effects on their brain of blue light and sleep deprivation and attention deficit disorder that's induced by having their attachment hacked. Teach them that stuff. Like the parents running around desperate to get their kids into situations of becoming digital natives are completely naive about what the digital environment actually portends for a young child where that's their primary place they build skills. Yeah, so that's my kind of like a little bit unorthodox view, which is that yes, definitely show them technology, they live in a world of technology, but don't, don't accustom them to use it as is is intended to be used to get them to really think about what it actually is.
- [82:34]
B
I have a bunch of friends who grew up sort of learning to use computers, learning to use the Internet, got a lot out of that. And it turned into their professional lives. It turned, it gave them sort of, it allowed them to find groups with similar interests and sort of the Internet played a formative role in who they became. And do you think you're depriving or potentially depriving a child of something like that if you say you can't interact with AI at all or you will have very limited interaction with AI?
- [83:09]
A
Again, I think the comparison has to be examined like the early days of the. I don't know how old you are, but the Internet. Yeah, the Internet is not one thing. Right. So presumably these guys didn't get their attention systems destroyed, right?
- [83:26]
B
No, not at all. They learned to, they learned to program. Right? They learned to. They read a lot of Wikipedia.
- [83:31]
A
Being someone who knows how to program, who can relate, that's the kind of person I'm talking about. They actually literally know there's different programming languages. They get the difference of where the hardware comes from, different affordances of different types of. That's a very different relationship. So. But that's not what people mean when they say get their kids to use AI. What they mean is like get my kid to be a prompt engineer so maybe someday he can possibly have a job in a world where there won't be any jobs and the only jobs will be the jobs that are available because of the tech guys new software update. And so that. I don't think that's the right, that's not the right attitude. I mean, but you know, the head in the sand thing is the thing we don't want to do. So we really need to find a way to educate kids. And this is a broader question of well, what does it mean actually to succeed late in adolescence? Right. This is the question. We don't have initiations in our culture, but people still become adults. So there's a like quote unquote on paper, Right. So there's this question of like, what actually constitutes a healthy resolution of the adolescent identity crisis. This is the question of educator, what does it mean to be well educated? And you know, do we want to live in a culture where kids don't learn to read and write, but have AIs read and write for them? Right. We could make that culture, we can have a world where that is the thing. Do we want to have a world where kids don't have human boyfriends and human girlfriends and human best friends, but they have machine boyfriends and girlfriends and best friends? Is that an okay outcome about less and socialization on a broad scale? This is the question. And so when it comes to how should we have kids interact with the technology, we have to think about that. What's this going to look like when they're 17, 18, 19? And we want to be able to say, you're now an adult and you're a grown up. Are we going to be happy with the skills they've built, the person they've become, their habits and, you know, desires and images for what they want to become? And so if you hold all that in mind and then you think about how to give your kid AI, that's a totally other question. But if the idea is just like throw them in the deep end of the newest, latest, fanciest technology because otherwise they will fall behind in this dog eat dog world for a job, you really need to get some perspective on the thing.
- [86:00]
B
Yeah, this for me, it falls into your larger philosophy of education as a very central issue. Perhaps you could explain how you see education as something much deeper than how we normally think about it.
- [86:16]
A
Absolutely. And we've been dancing around it like the attachment system in humans allows for really long duration intergenerational transmission. So I'll say that another way. Like it's a species specific trait of the human to be really young for a really long time. Which means that our, you know, the young members of our species are particularly vulnerable and need an ex like years of what's called enculturation. And the only place that enculturation, which we might call education takes place is in these deep attachment relationships. Right. So this is important again, right? When you see a gazelle born in the savannah and it comes out, you know, an hour or two later, the gazelle is doing most of the gazelle stuff, right? It's running around, it's eating grass, it's Identifying predators. Right. It's not that long until it can actually have another baby gazelle. Relatively speaking. When a human comes out, it is years before the human is doing anything that looks like what the other members of the species are doing that allow the species to live and survive. So that's a species specific thing, which means education as we know it now is part of what it means to be sapient. It's part of our species specific kind of secret sauce that actually allows for all of the other stuff people want to talk about as denoting sapience, tool use, language use, niche creation, radical adaptation. How do you get all of that really long duration intergenerational social transmission where you can do ratcheting effect around technology and all kinds of stuff. So primary. And how does it work? Attachment relationships. Me, me, the young person, you, the older person, together with the world, together with the technology, where we both know that's a situation, you know, you're older and no more. I know I'm younger and no less. We collaborate in my interest. Right. The younger person's interest. We collaborate in the interest of the youth to give them what they need to take on the project of survival as we know it. That's how I define education. Now fast forward, you get schooling. That's what most. When I say education, most people think schools. Like, like, okay. But schools are a very modern creation in terms of what education is. And even in our societies where schooling is supposed to be the place where education happens, a lot of education happens outside the school, whether the school is wanted or not. So I define education as this very broad category and it's part of the species specific trait which makes us human. So the risk here, which I've been seeing for years and what led me to the study of catastrophic and existential risk is what happens when you get a catastrophic disruption of intergenerational transmission, which means a catastrophic disruption of the ability to, of the elder and the youth to cooperate in the interest of turning over the society to the next generation. Right. If you look at history, people like Turchin and you know, the fall of the Roman Empire and kind of like Tainter and other people looking at these big dynamics of civilizational collapse, you realize this is one of the key dynamics of civilizational collapse is actually catastrophic mishandling of the intergenerational transmission thing. And often it's through invasion or flood or technology. And technology in our society has been one of the main things driving the generational gap, which means disassociating the youth from the Elder more and more and more and more and more to the point where they're not cooperating in the, the interest of the youth, but actually at war. Generational warfare, which is a little bit where we are at now. It's one way to understand, like the student loan crisis in the United States and a bunch of other things where it was clear that the older generation, when it looked at what education was, didn't see it as a way to collaborate in the interest of the youth, but rather as a way to put the youth permanently under the thumb of the older generation. Bunch of complexities here that are making it so that across the board we're having crises of legitimacy and capacity so that it's not clear how we fill some of the critical roles in our society going forward. So that's an educational crisis, which is like part of a broader civilizational crisis. So throwing AI into that mix, I believe is the perfect way to complete this catastrophic disruption of intergenerational transmission. And especially if you begin to, as we modeled a little bit earlier, see that these markets are moving towards parenting and teaching and they're not just seeking to replace Google, but seeking to replace moms. And so the question for me as an educational philosopher is what percentage of young people, especially spending what percentage of their time being socialized by machines results in the creation of a new species? This is like the ultimate catastrophic question is, are we looking at a speciation event? Speciation event that is the result of fundamental change in the process that has been the process that has conferred us sapiens. This is our species specific process, this long duration intergenerational transmission. We've socialized humans, humans have socialized other humans with technology in relation to technology, but we have never been socialized by the technology itself, excluding the interaction with other humans. Can that generation understand itself?
- [92:12]
B
Yeah. Is this something that could happen? Or how do you understand a speciation event? Would this be merging with the technology in a very literal sense? Or would this be about just interacting so much with AI that we lose human culture? And because human culture is very much a huge part of what makes us human, we are therefore, in some metaphorical sense, a different species. How do you think about this?
- [92:41]
A
I mean, so Jurgen Habermas wrote this book, the Future of Human Nature, and it was about genetic engineering. And he basically said, you know, you have to look at how the future generation understands itself. And if it sees itself as part of the same moral universe as the elders. And so his example was, if you perfectly genetically engineered the next generation, would they understand themselves as the same type of being as the parent. The parent emerged through contingency, right? The parents, the parents traits were not designed. The parents traits emerged through its own effort, right? You're a kid, you were designed by your parents. If you become a criminal, are you responsible or are they responsible? They designed you. So there's this question of are you a member of the same moral universe? So similarly here, the speciation event doesn't look dramatic. It actually looks like an unbridgeable generational gap that became a speciation event, which is basically these kids look at each other. All the kids raised by the machines, they look at each other and they're like, hey, our parents were not raised by machines. Like who are we responsible to? Like, who are we part of a tribe with? Who are we committed to building a future together with? Like, and also what have we learned that they can never learn or understand is if you start to get the neural implant stuff and you start to get the more full blown cyborgization, you can see the ability of this new cyborg species to basically want to speciate. So again, this is all sci fi stuff. When I went to graduate school of education, I did not expect this to work, you know, and you know, so yeah, so that, that's, that's the deepest concern. And in some of the work I've done writing philosophically about this, you can think of the death of humanity being the classic X risk. Just everybody dies, the bomb gets dropped. The death of our humanity is different. The death of our humanity is still a catastrophe. But most of the physical bodies go on living. It's just a very different way of understanding what the meaning, motivation, value, self understanding of the humanness when it has been raised entirely into a situation like a mechanical automization of socialization. So again, this is all speculative and I don't lead with this stuff because this, you know, too much of the AI risk conversation is dismissed because it sounds speculative and like science fiction, right? So this is not the place to lead with, but it is the place to end with because it is the long duration outcome of this, right? So if you look at venture capital seeking to invest in artificial intimacy, they want to start the companies that create these AI companions, they're projecting multi trillion dollar global markets for global intimacy. So what we're seeing as this kind of horrific scenario, they're seeing as the inevitable growth of the market that they're pursuing, which is fewer and fewer people spend time with each other. More and more people spend time just with machines. So as speculative as it sounds, if current trends go in these directions, then that's what will happen. Age norms, age limits, age regulations must occur or we would face the situation of that I've been describing, where the market will saturate down into the youth. And then you'll have 90% of the language use kids have with machines. Rather than 90% talking to humans, 10% talking to machines, which is what it should be. We'll have this flip and then what will the teachers do? What will the parents do? What will the guidance counselors do? The pediatricians, the psychologists? All the humans trying to talk to the kids will have no purchase.
- [96:41]
B
Perhaps we can try to end on a more positive note here or think about how we might avoid this grim situation. You mentioned preserving humanity, perhaps, or preserving our humanity as opposed to preventing humanity from going extinct. We can preserve our humanity in the face of more and more advanced AI. I'm wondering practically what that looks like. Is that about communities that have certain values that they enforce and they sort of harden against encroachment of AI into those communities? What does. So we talked about cognitive security for the individual, but what does that look like for a community? Do you think that's, that's central to, to avoiding the bad outcomes here?
- [97:29]
A
Yeah, I mean there's, there's layers to it, right? There's, we talked about technology, we talked about, there's a technological counter to this. We could have a cognitive defense technology startup industry. And that's like a race to the top to protect, protect people. There's also regulations. Like it seems shocking that there are not third party agencies that do FDA type safety testing on advanced technologies that are put in front of people's faces and occupy a tremendous amount of the time, especially kids. So I think safety frameworks, organizations, regulations, you know, I think it was Max, who you work with who said like if I want to open up a sandwich shop in New York City, city, it's actually more complicated in terms of regulations and inspections and stuff to sell a sandwich than it is to put an advanced technology in front of some kid's face. So it's just, it's a no brainer that one of the approaches here, at least in the long run, is setting up context for safety testing. And that seems clear if you move up higher in like the stack, there are certain types of cultural norms and conversations that need to occur that are confused right now. So you know, the transhumanist I vision, the transhumanist vision I articulated, that was expressed by Kurzweil. It was shared by other people. There's a metaphysical confusion. So it's like we can have all the regulations and that's great, but if you move up to the place where people's world views their cosmologies, don't disallow for some of the insane futures people would like to create. So the uploading of consciousness, right, the coming of the sentient AI that should be granted personhood. So there's a, there's a battle that has to be fought at the level of philosophy itself to be able to have much more salient and reasonable things said about what AI futures are going to look like that don't get us into the kind of extreme value relativism that would have us give silicon self replication higher value than carbon based self replication. I can go right to the top of the metaphysical debate, right? So as I mentioned, like this book, Exit the Silicon Maze, which is going to be coming out with some co writing with Ken Wilbur and Mark Gaffney. This is the issue because we know how to regulate this stuff, but we don't know how to justify the intrinsic value of human attention and human attachment, right? We're dealing with Skinnerians who believe that that's all fiction, that mind and body are fundamentally separate, separable, that the metaphor of hardware software applies to consciousness brain, that the universe itself can be explained in terms of strong computationalism. And these types of views which have to be very strongly countered at the level of philosophy itself. You have to understand that there's a small, narrow cul de sac in basically Anglophile philosophy of mind and analytical philosophy that allows for conversations that are completely unreasonable about the nature of the human mind and brain and about the potentialities of the hardware stack that is creating the artificial intelligence, right? So to me, that's the issue, which is that there's a niggling suspicion in the minds, even of the general public about this inevitability of the replacement of carbon by silicon and of the valueless nature of biological life, because it's a random excretion of a meaningless cosmos. So for as long as we don't have a theory of value that can ground intrinsic value of the sapience of protecting the human, we have a bunch of anti humanists or transhumanists who are willing to gamble with the entire species pieces in the interest of creating some more advanced species on the basis of completely faulty metaphysical systems. So at the highest level, we actually have to be able to say that stuff clearly in order to justify even legal frameworks that would protect our humanity, right? Like why don't we grant personhood to a machine? Right? What happens when we do that? Why don't we allow humans and machines to get. Get married? Right? Like why don't we allow children to be raised completely by a AI? Where do we place these things in terms of our legal theory? Has to do with how we understand the nature of the human basically metaphysically and for as long as really sophisticated people are basically saying there's no difference between your brain and a GPU cluster. There's no difference between relating to an LLM and relating to a person. Because how do you know the person really is conscious? Because we do zombie thought experiments. Right. And so like all this kind of nonsense actually has to stop. Is a very responsible way to. You'd never actually educate a child based on that worldview. This is one of my lipness tests of is a philosophy coherent? Is can you use the philosophy to do intergenerational transmission in a coherent way? And if you are a person who believes that you should tell children that they have no free will, consciousness is an illusion, they're a meaningless meat machine that evolved for no purpose in the universe and that they should probably abandon their own life to a greater living being. That's a silicon entity. You really want to talk about that around the dinner table with your kids? Yeah. You should be removed from parental authority. These are insane worldviews which are spoken from the podium, which cannot be used in the fullness of life to make responsible adults. So that's a deeper concern. What do we tell the kids about the value of their own life and the value of the human experiment on the planet and the overarch, the overarching trajectory of cosmic evolution that we are a part of and do we abandon that field of to, you know, faulty metaphysics and, and you know, neuro atypical cosmologies? We can't. We can't. We have to. We have to say profound and reasonable things about this technology and not have the discourse about the technology be set by a very small number of people who have motivated reasoning and a distorted view of what is valuable. So that's a long winded answer. So yes, regulations, yes, something like an fda, yes, age bans, all of that stuff. But also like, we need to get back to coherent public philosophizing and move away from these extremely incoherent views about what the human is worth. So I can keep going. But it's a.
- [104:30]
B
It's perfect. It's perfect. Zach, thanks for, thanks for chatting with me. It's been great. I've learned a lot from this.
- [104:37]
A
Yeah, man, I've enjoyed it.
- [104:39]
B
Fantastic.