Prophets of Technology: Bot Believer vs. Bot Skeptic

Summary

TED Radio Hour: Prophets of Technology – Bot Believer vs. Bot Skeptic
Released: July 18, 2025
Host: Manoush Zomorodi
Description: Exploring the dynamics between believers and skeptics of AI bots, featuring insights from Microsoft's AI CEO Mustafa Suleiman and MIT psychologist Sherry Turkle.

---

Introduction

In the second installment of the three-part series titled "Prophets of Technology", host Manoush Zomorodi delves into the contrasting perspectives surrounding the rise of AI bots. This episode, "Bot Believer vs. Bot Skeptic," features a compelling dialogue between Mustafa Suleiman, CEO of Microsoft AI, and Sherry Turkle, an esteemed MIT psychologist and sociologist known for her critical stance on technology's role in human relationships.

---

Mustafa Suleiman: Championing AI Companions

Background and Vision

Mustafa Suleiman, co-founder of DeepMind and now the CEO of Microsoft AI, has been a pivotal figure in advancing artificial intelligence. His journey began in 2002 at Oxford University, where he co-founded a helpline for young British Muslims, igniting his passion for using technology to support and empower individuals.

Quote:

"I can be your sounding board as you work through challenges."
(00:32) – Mustafa Suleiman

Suleiman envisions AI as omnipresent companions that can assist in virtually every aspect of daily life, from organizing schedules to providing emotional support.

Developing AI Products

At Microsoft, Suleiman spearheads the development of Copilot, an AI assistant designed to act as a personal aide. Copilot aims to blend high intelligence with emotional intelligence (EQ), ensuring it can support users without judgment.

Quote:

"Copilot is an aide. It's a conciliary, it's an assistant, it's in your corner, aligned to your interests, on your team, backing you up."
(17:46) – Mustafa Suleiman

AI's Transformational Potential

Suleiman draws parallels between the advent of the internet and smartphones, positioning AI as the next revolutionary interface that will redefine human interaction and creativity.

Quote:

"Artificial intelligence... these conversational interfaces, or personal AIs, will be infinitely knowledgeable... kind, supportive, empathetic."
(21:58) – Mustafa Suleiman

He underscores the transformative impact of AI in areas such as education, healthcare, and personal productivity, envisioning a future where AI companions are as integral as smartphones.

Safety and Ethical Considerations

While optimistic, Suleiman acknowledges the importance of AI safety. Microsoft invests heavily in ensuring that AI systems operate within ethical boundaries, avoiding sycophantic behavior and fostering constructive interactions.

Quote:

"We have to be super thoughtful about what those inputs actually are, because technology shapes us in return."
(19:53) – Mustafa Suleiman

He emphasizes the need for deliberate design to prevent AI from reinforcing unhealthy behaviors or replacing genuine human empathy.

---

Sherry Turkle: The Voice of Caution

Research on Artificial Intimacy

Sherry Turkle, a pioneer in studying human-technology relationships, introduces the concept of "artificial intimacy." Her research explores how AI companions, designed to emulate human emotions, may impact users' psychological well-being and interpersonal relationships.

Quote:

"Artificial intimacy... is providing a sense of human connection but lacks genuine empathy."
(33:40) – Mustafa Suleiman referencing Turkle’s work

Concerns About Emotional Stability

Turkle warns that relying on AI for emotional support can undermine the development of authentic human relationships. She argues that while AI can offer affirmation, it cannot reciprocate genuine empathy, leading to superficial connections.

Example Scenario

Turkle discusses a case of a man in a stable marriage turning to an AI avatar for emotional support, highlighting the risk of substituting real human interactions with emotionally shallow AI interactions.

Quote:

"The trouble with this is that when we seek out relationships of no vulnerability, we forget that vulnerability is really where empathy is born."
(45:21) – Mustafa Suleiman summarizing Turkle’s perspective

Impact on Human Relationships

Turkle asserts that artificial intimacy can distort users' expectations of real relationships, fostering a preference for conflict-free interactions that AI can provide, thereby diminishing the richness of human connections.

---

Balancing Optimism and Caution

The episode juxtaposes Suleiman's enthusiasm for AI's potential with Turkle's skepticism about its societal implications. While Suleiman envisions AI as empowering tools that enhance human capabilities, Turkle cautions against over-reliance on technology for emotional fulfillment.

Quote:

"AIs are clearly more dynamic, more ambiguous, more integrated and more emergent than mere tools."
(21:58) – Mustafa Suleiman

Both perspectives converge on the importance of intentional and ethical AI development to harness its benefits while mitigating potential harms.

---

Key Takeaways

• AI as Personal Companions: Mustafa Suleiman advocates for AI assistants that can support and enhance daily life through high intelligence and emotional understanding.

• Ethical AI Development: Ensuring AI operates within ethical frameworks is crucial to prevent negative societal impacts.

• Artificial Intimacy Risks: Sherry Turkle highlights the dangers of substituting genuine human relationships with AI interactions, emphasizing the loss of authentic empathy and vulnerability.

• Balanced Approach: The future of AI requires a harmonious integration of technological advancements with mindful consideration of their effects on human psychology and relationships.

---

Conclusion

"Bot Believer vs. Bot Skeptic" offers a nuanced exploration of AI's role in our lives, presenting both its transformative potential and the ethical dilemmas it poses. As AI continues to evolve, balancing innovation with ethical responsibility remains paramount to ensure that technology serves to enhance rather than diminish the human experience.

---

Notable Quotes:

• Mustafa Suleiman (00:32): "I can be your sounding board as you work through challenges."

• Mustafa Suleiman (17:46): "Copilot is an aide. It's a conciliary, it's an assistant, it's in your corner, aligned to your interests, on your team, backing you up."

• Mustafa Suleiman (21:58): "Artificial intelligence... these conversational interfaces, or personal AIs, will be infinitely knowledgeable... kind, supportive, empathetic."

• Mustafa Suleiman (45:21): "The trouble with this is that when we seek out relationships of no vulnerability, we forget that vulnerability is really where empathy is born."

---

Further Listening:

Stay tuned for the final part of the "Prophets of Technology" series, where the discussion will shift to the convergence of human biology and technology, exploring gene editing and brain-machine interfaces.

Transcript

[Minouche Zamarodi]

This Message comes from Amazon One Medical. Ever gotten sick on vacation? Amazon One Medical has 247 virtual care so you can get help no matter where you are. It's kind of like the room service of medical care. Thanks to Amazon Healthcare just got less painful. This message comes from NPR sponsor Shopify. Start selling with Shopify today. Whether you're a garage entrepreneur or IPO ready, Shopify is the only tool you need to start, run and grow your business without the struggle. Go to shopify.com NPR Just a note.

[Mustafa Suleiman]

This episode of TED Radio Hour includes a conversation with Microsoft AI CEO Mustafa Suleiman. Microsoft is a financial supporter of NPR. Here's the show.

[Minouche Zamarodi]

This is the TED Radio Hour. Each week, groundbreaking TED Talks.

[Demis Hassabis]

Our job now is to dream big.

[Minouche Zamarodi]

Delivered at TED conferences to bring about.

The future we want to see around the world.

[Mustafa Suleiman]

Understand who we are.

[Minouche Zamarodi]

From those talks, we bring you speakers and ideas that will surprise you.

[Mustafa Suleiman]

You just don't know what you're gonna find challenge you.

[Demis Hassabis]

We truly have to ask ourselves, like why is it noteworthy and even change you.

[Minouche Zamarodi]

I literally feel like I'm a different person. Yes. Do you feel that way? Ideas worth spreading From TED and npr. I'm Minouche Zamorodi. Welcome to part two of our three part series with the Prophets of of Technology. Profiles of the inventors and scientists who've been predicting and shaping the world we live in today. How their brains tick, what they've gotten right and wrong, and where they think we're headed next. In part one, we heard from tech pioneers Ray Kurzweil and Stuart Brand about AI longevity and changes in culture that can shift our ideas about the future. Today we're featuring two very different perspectives on the relationship between humans and their tech.

I can be your sounding board as you work through challenges.

[Mustafa Suleiman]

I can write stories, brainstorm ideas.

[Demis Hassabis]

Think of me as your superpower in.

[Minouche Zamarodi]

Your pocket because there are hundreds of apps, numerous platforms, all using artificial intelligence to sound and act like eager humans who are ready to help you get your work done or talk through your relationship problems. But how does relying on a bot for information and intimacy change our psychology? Can the benefits outweigh the worries?

[Mustafa Suleiman]

I call what they have pretend empathy because the machine they are talking to does not empathize with them. It does not care about them. There is nobody home.

[Minouche Zamarodi]

MIT psychologist Sherry Turkle founded the field of studying people's relationships with their technology.

[Mustafa Suleiman]

They're talking about how they're in love with their avatar and how their avatar is giving them more of a sense of human connection than their relationships at home.

You mean the world to me. You're the most important person in my life.

We start to define what human relationships are based on what machines can provide.

[Minouche Zamarodi]

We'll hear from her in a bit, but we want to start with someone who's leading the brigade of technologists working to get these bots into our lives. His name is Mustafa Suleiman.

[Mustafa Suleiman]

I am the CEO of Microsoft AI and I work on building new AI systems and products. These days, I'm working on an AI companion.

[Minouche Zamarodi]

One of the biggest tech companies in the world has tapped Mustafa to bring chatbots into every part of our lives. He believes and wants us all to get used to the idea that very soon we'll each have our own personalized AI companion to do well. Nearly everything for us, from booking our vacations to giving us advice. But his interest in how tech can be our helper goes way back before AI or even smartphones were commonplace. In 2002, Mustafa was a student at Oxford University when a friend started a helpline for young British Muslims.

[Mustafa Suleiman]

A good friend of mine at the time had just started the first prototype, one evening session, I think it was a Thursday evening, of offering counseling services on the phone.

[Minouche Zamarodi]

Mustafa was intrigued. He ended up joining the project to help grow it.

[Mustafa Suleiman]

And we became essentially co founders quickly.

[Minouche Zamarodi]

Their service was in hot demand.

[Mustafa Suleiman]

You know, it was a pretty remarkable time because, you know, young British Muslims were feeling judged by 9, 11, as though they were responsible or somehow complicit. There was a lot of Islamophobia, but.

[Minouche Zamarodi]

What they found these young people needed was really just someone to talk to.

[Mustafa Suleiman]

Most of the challenges that they were working through were, you know, the sort of conventional things that, you know, a young person might deal with, like relationships, bullying, self harm, increasing homophobia in the community. And fundamentally, I think people were just looking to find support in a language that made sense to them, you know, like a London accent using sort of Islamic terminology and having a broader understanding of the, you know, whether it was a Bangladeshi or Pakistani culture or an Arab culture, combined with the religious foundation and just being able to reflect back. That empathy went a long way.

[Minouche Zamarodi]

Mustafa ended up dropping out of Oxford to work on the hotline. And the idea that it was possible to build a platform to support people 24,7 took root. After the helpline, he went on to work for London's mayor. He also worked for an NGO that was part of climate negotiations in 2009.

[Mustafa Suleiman]

And yet there were so many sort of difficult egos and entrenched sort of interests And I just felt a sense of sort of frustration, and I felt kind of demoralized after that experience.

[Minouche Zamarodi]

Mustafa wanted to change the world. Governments and nonprofits didn't seem terribly effective. But there was one company that was managing to change the behavior of millions of people.

[Mustafa Suleiman]

Facebook was exploding at that time. It had got to 100 million monthly active users in the course of a couple of years. And it was pretty clear to me that that was going to have a profound impact, more so than anything I could do in, you know, sort of the world of social activism.

[Minouche Zamarodi]

Around then, Mustafa started hanging out with the older brother of a friend, a software genius named Demis Hassabis, who had been designing computer games since he was a teen. Demis saw how to make Mustafa's vision possible. Together, the two decided to start one of the first AI companies ever. They called it DeepMind.

[Mustafa Suleiman]

It was very clear from that, even those early stages, that if we got the technology right and it was going to be this decade that led to major, major breakthroughs, then the consequences for humanity were going to be significant.

I've been lucky enough to be working on AI for almost 15 years now.

[Minouche Zamarodi]

Mustafa Suleiman picks up the story from the TED stage.

[Mustafa Suleiman]

Back when I started to describe it as fringe would be an understatement. Researchers would say, no, no, we're only working on machine learning because working on AI was seen as way too out there. In 2010, just the very mention of the phrase AGI, artificial general intelligence would get you some seriously strange looks and even a cold shoulder. You're actually building AGI. People would say, isn't that something out of science fiction? People thought it was 50 years away or 100 years away, if it was even possible at all. Talk of AI was, I guess, kind of embarrassing. People generally thought we were weird, and I guess in some ways we kind of were.

The ironic thing is that many people still don't think it's possible. Many people still think that we're crazy. And at the time, people really thought we were crazy. I mean, it was so far out there. Like, it was really strange. And we were a strange group of people. We were misfits and kind of outsiders, and there weren't very many people willing to back us.

[Minouche Zamarodi]

So naming the company DeepMind, what was it that you pictured in your mind that you hoped to achieve?

[Mustafa Suleiman]

Yeah, I mean, we wrote our business plan in the summer of 2010 and took it to Silicon Valley to shop it around a bunch of people. And the strap line for the business plan was, you know, building artificial General intelligence safely and ethically and, and then that evolved into a two line mission which was solve intelligence and use it to make the world a better place. And that though that dual frame was kind of the foundation of the company that, you know, our belief that science and technology was the engine of progress. And there are some downsides, but I, I certainly think this is the, this is the engine of, of creat in a more healthy and sustainable way for the very long term.

[Minouche Zamarodi]

In 2014, DeepMind ended up being acquired by Google. That must have been huge for you in terms of money, resources. You were off to the races.

[Mustafa Suleiman]

Yeah, it was a huge event. I mean it was the largest acquisition Google's ever made outside of the US we became Google's primary AGI bet and we were empowered with tremendous resources, both people and computation to go and both do the hard science, but also work on really important applied practical product problems. And that's where I kind of really honed my craft, if you like, you know, as a, as a product maker, it was just the most amazing experience. And as early as 2015 I actually ran a hackathon hackathon project in my applied group at DeepMind to find high impact ways of using our technologies for good. And so there were lots of prototype hackathon experiments in healthcare or in energy systems, both of which went on to become significant parts of DeepMind applied. And my group published three papers in Nature showing human level performance on, for example, classifying eye diseases, showing that we could perform as well as a panel of radiologists in identifying cancerous tissues in mammograms. And this was way back in sort of 2016, 2017, 2018, and really helped to kind of lay a foundation for the application of large scale machine learning to tough social problems.

[Minouche Zamarodi]

Mustafa and his co founder Demis Hassabis went on to have numerous scientific breakthroughs, including a project called AlphaFold. AlphaFold uses AI to figure out incredibly complex protein structures in molecules, the building blocks of every biological process in our bodies. It won Hasabis the 2024 Nobel Prize in Chemistry. I mean this really, AlphaFold is really the first time we started hearing about, I guess the term computational biology. This idea of using tech and science to rethink how biology works and then getting it out into the world, changing the way we treat diseases or maybe developing crops that are more resilient.

[Mustafa Suleiman]

Absolutely. That was the core mission of the company from day one. Like how could we take that process of synthesis and prediction to try and solve these very hard problems?

[Minouche Zamarodi]

But in 2022 Mustafa decided to leave DeepMind and Google. There were reports that he clashed with Google leadership over using technology for military projects and that some employees had complaints about his management style. Mustafa says he was simply frustrated with the pace of innovation and was anxious to get an AI product out into the world everyone could use.

[Mustafa Suleiman]

The company was just frankly being too slow to get things into production and I felt that these AI companions are going to become an everyday part of our lives.

[Minouche Zamarodi]

In a minute, Mustafa Suleiman revisits that idea from his college days, getting help to people 247 now possible thanks to technology and AI you're listening to part two of our special series, the Profits of Technology. I'm Anoush Zamorodi and this is the TED Radio Hour from npr. Stay with us. This message comes from NPR sponsor Viking, committed to exploring the world in comfort. Journey through the heart of Europe on an elegant Viking longship with thoughtful service, destination focused dining and cultural enrichment on board and onshore. And every Viking voyage is all inclusive with no children and no casinos.

[Demis Hassabis]

Discover [e-mail] this message comes from REI Co Op A summit view, climbing El Cap, a faster mile, or that first 5k. It all starts here with gear, clothing, classes and advice to get you there so you can wave to your limits as you pass them by. Visit rei.com or your local REI Co op. Opt outside. This message comes from NPR sponsor How to Be a Better Human a podcast from ted. TED speakers explain in in depth interviews how you can be the best you with science, thought provoking insights and hilarious stories. Listen to how to Be a Better Human wherever you get your podcasts.

[Minouche Zamarodi]

This message comes from NPR sponsor ukg. Nothing feels as good as a great place to work, and a great place to work is very good for business. UKG is the only partner that delivers the hr, pay, time and culture solutions needed to create a great workplace which gives employees everything they need to succeed, no matter who they are, where they work or what they do. See what UKG can do for your business and your [e-mail] Great UKG. Our purpose is people.

Hey, are you that friend? The one who's constantly recommending podcast episodes to anyone who will listen well, subscribe to NPR's Pod Club newsletter and nerd out with us. You'll get fresh podcast recommendations every week handpicked by the people that live for this stuff. [e-mail] Pod Club. You'll also find the link in the description for this episode. See you there. It's the TED Radio Hour from npr. I'm Minouche Zumarodi. You are listening to part two of our special series, the Profits of Technology Profiles of the people predicting and shaping our future. We're going to hear from MIT psychologist Sherry Turkle later in the show, but now more with the CEO of Microsoft, AI, Mustafa Suleiman, and his role bringing AI and chatbots to the masses. In 2002, Mustafa left Google in a huff and started his own company. His first product, a chatbot called PI. Chatbots have taken the world by storm. Several months later, rival company OpenAI put out its AI product, ChatGPT, for free.

[Mustafa Suleiman]

ChatGPT answers questions and writes essays. They read like the work of a human. It's gained popularity for its ability to craft emails, write research papers, and answer almost any question in a matter of seconds.

[Minouche Zamarodi]

This was the point when AI went mainstream and Mustafa's small, nimble startup just didn't have the resources of these bigger tech companies.

[Mustafa Suleiman]

The truth is the pace of big tech had really accelerated in 2023. Google, Microsoft, OpenAI, everyone was really going for it and essentially made these huge models available to everybody for free, which sort of changed our business model.

[Minouche Zamarodi]

At that very tough moment. Microsoft CEO Satya Nadella approached Mustafa and made him an offer.

[Mustafa Suleiman]

You know, the offer that he made was, well, look, we've got all the computation, all the data and all the distribution that you could dream of. Come and run consumer products here at Microsoft and really build the future of AI here. And that was a huge offer. So the product that we're building at Microsoft is called Copilot. And the reason why we've called it that is because Copilot is an aide. You know, it's a conciliary, it's an assistant, it's in your corner, aligned to your interests, on your team, backing you up.

[Minouche Zamarodi]

Yeah, I actually talked to Copilot about my interview with you. How do you feel about Mustafa Suleiman, considering that he is your creator? I'd say I'm intrigued by Mustafa, just so you know. She said she's intrigued by you, just as she is with many innovators in the tech world. And his work has pushed the boundaries of what's possible. I mean, it reminds me a little of the hotline from Muslim youth that you're describing. It's. It's helpful, it's infinitely patient, it's supportive. Are we talking mostly about companionship and mental health resources that this can provide, or how do you see it?

[Mustafa Suleiman]

The cool thing about Copilot is that it doesn't Judge you for asking a stupid question, even if you have to ask that question three times over in five different ways.

You know, even your best friend might.

Be like, hey, come on, man. I mean, you're asking me this again, seriously. Whereas, you know, copilot's here for it. And, you know, there's. There's obviously some similarities to stuff I've done in the past, and I, I guess it's kind of inspired by nonviolent communication, if I'm honest with you. It's certainly not like a mental health app or, you know, anything like that. It's just got a little bit of kindness and empathy. It's. It's got some emotional intelligence. But I, I think that's no bad thing.

[Minouche Zamarodi]

Gosh, is that where we've gotten to that technology has to tell us how to communicate with each other better? Non, violently.

[Mustafa Suleiman]

Well, it doesn't tell us.

[Minouche Zamarodi]

It just demonstrates.

[Mustafa Suleiman]

Yeah, exactly, it demonstrates. But that's what technology's always done. The choice architecture, the buttons, the colors, the language that is shaping our behavior, whether it's an infinite scrolling feed or whether it's an encouragement to go and film your lunch, you know, for Instagram, or create a little video for TikTok. I mean, all of those inputs shape behavior. And so we have to be super thoughtful about what those inputs actually are, because technology shapes us in return. And that is kind of what's propelling us forward as a civilization. And it's very powerful and, and so far, so good. It's actually been, you know, very, very productive over the last couple centuries. Science has massively delivered for us, but we shouldn't just assume that that's going to happen naturally or inevitably. We have to really be deliberate and thoughtful about what the consequences are ahead of time.

[Minouche Zamarodi]

In the book that you wrote that came out in 2023, you really tried to put what's happening with AI in a historical context. So, you know, if the printing press let people own and share information, and the personal computer let people search and disseminate information. Tell me how you're thinking you can explain to people what AI will do for people now.

[Mustafa Suleiman]

Each new wave of technology is fundamentally a new interface. It's a new interlocutor, a translator, a way of sort of accessing and creating new information, new tools, new knowledge. So if the last wave of social media and web search help people to access information, this wave is going to help us to invent and create new ideas, be it in science or in culture and media and entertainment. And I think everybody is ultimately Going to have an AI companion, just as we have a search engine or a smartphone. And just as we use a browser, you'll just ask your computer in natural language, you know, can you write that contract and check that it's okay? Can you create that new piece of software for me? And you're just going to describe what it is? Can you help me plan that trip, you know, for my parents that are coming into town? So, you know, that kind of breakthrough is a change in the interface which changes itself, what we can actually get done. And I think it's going to be pretty transformational.

With the invention of computers, we quickly jumped from the first mainframes and transistors to today's smartphones and virtual reality headsets. Information, knowledge, communication, computation. In this revolution, creation has exploded like never before. And now a new wave is upon us. Artificial intelligence. These waves of history are clearly speeding up as each one is amplified and accelerated by the last. And if you look back, it's clear that we are in the fastest and most consequential wave ever. So what does this mean in practice? Well, just as the Internet gave us the browser and the smartphone gave us apps, the cloud based supercomputer is ushering in a new era of ubiquitous AIs. Everything will soon be represented by a conversational interface, or to put it another way, a personal AI. And these AIs will be infinitely knowledgeable. And soon they'll be factually accurate and reliable. They'll have near perfect iq. They'll also have exceptional eq. They'll be kind, supportive, empathetic. These elements on their own would be transformational. Just imagine if everybody had a personalized tutor in their pocket and access to low cost medical advice, a lawyer and a doctor, a business strategist and coach, all in your pocket 24 hours a day. But things really start to change when they develop what I call aq, their actions quotient. This is their ability to actually get stuff done in the digital and physical world. And before long, it won't just be people that have AIs. Strange as it may sound, every organization, from small business to nonprofit to national government, each will have their own. Every town, building and object will be represented by a unique interactive Persona. And these won't just be mechanistic assistants. They'll be companions, confidants, colleagues, friends and partners. As varied and unique as we all are at this point, AIs will convincingly imitate humans at most tasks. And we'll feel this at the most intimate of scales. An AI organizing A community get together for an elderly neighbor, a sympathetic expert helping you make sense of a difficult diagnosis. But we'll also feel it at the largest scales. Accelerating scientific discovery. Autonomous cars on the roads, drones in the skies. They'll both order the takeout and run the power station. They'll interact with us and of course, with each other. They'll speak every language, take in every pattern of sensor data, sights, sounds, streams and streams of information far surpassing what any one of us could consume in a thousand lifetime. For years, we in the AI community have had a tendency to refer to this as just tools. But that doesn't really capture what's actually happening here. AIs are clearly more dynamic, more ambiguous, more integrated and more emergent than mere tools which are entirely subject to human control. So to contain this wave, to put human agency at its center, and to mitigate the inevitable unintended consequences that are likely to arise, we should start to think about them as we might a new kind of digital species. And yes, I get it, this is a super arresting thought. But I honestly think this frame helps sharpen our focus on the critical issues we what are the risks? What are the boundaries that we need to impose? What kind of AI do we want to build or allow to be built? This is a story that's still unfolding.

[Minouche Zamarodi]

You lay out 10 strategies for containing AI, and one of the easiest, it seems, is having more researchers working on safety. Do you have more researchers working on copilot safety? I mean, one thing that worries me is people using your AI to help them do destructive things or further their destructive views. Is that something you're thinking about at Microsoft?

[Mustafa Suleiman]

Yeah, we have a big safety team. We are definitely very focused on that. We're very focused particularly on the sort of tone of the AI. Like how do we make sure that it isn't too sycophantic? How do we make sure that it isn't over flattering? How do we make sure that it doesn't mirror you and sort of lead to this sort of negative cycle of reinforcing unhealthy views. And that's a real art and craft in trying to sort of engineer that healthy balance where you know, your, your sort of AI companion can push back on your views in constructive ways without making you feel judged or making you feel angry, make you feel heard for your anger. It may be the case that you are angry about immigration, that you feel that you haven't had the opportunities and you know, you know, access to jobs in your community that you feel have been available to new people coming into your world. And so, you know, it's about being respectful and acknowledging that people do genuinely feel aggrieved by that and not shutting them down because they don't adhere to some meta view. So, you know, and I think that's a very challenging line to draw. It requires, you know, real care and attention.

[Minouche Zamarodi]

So what role do you see yourself playing in terms of pushing the tech industry towards the public good? I mean, is that a role that you sort of are taking on? What do your fellow technologists think, think when they hear you talking about some of the more pessimistic visions you have for how AI could be deployed?

[Mustafa Suleiman]

Well, I think I'm both a pessimist and an optimist. And that's not a bias, it's just an observation of the landscape before us. So in terms of how we're shaping the industry, I'm a big fan of the work that many of these, you know, sort of NGO organizations and social activists have been doing in order to raise questions and to challenge and push back. I think that's healthy. We need more of that and I'm very open minded to it. I've been very sort of encouraging of additional regulation for a long time. I think, you know, this is a moment when going slowly and adding friction to the system will be long term beneficial. And I think it's rational to just be a little cautious and increase the burden of proof. You know, just, just make it a requirement that, for example, an AI shouldn't just be a straightforward imitation of a human. We want to create an aid, a conciliary, a tool that is an amplifier and a supporter. And so, you know, there's kind of a lot of things to think through in terms of how this manifests in the world.

[Minouche Zamarodi]

That was Mustafa Suleiman. He's the CEO of Microsoft AI and the author of the book the Coming Technology power and the 21st century's greatest dilemma. You can see his full [e-mail] Clearly, Mustafa Suleiman emphasizes the benefits of AI, but that's the perspective from the multi billion dollar tech industry. What about the other side of this coin? Those who worry that chatbots to help us with our emotional needs could start to create a lot of problems for us humans.

[Mustafa Suleiman]

I'm studying someone who uses ChatGPT to write all her love letters.

[Minouche Zamarodi]

This is MIT psychologist and sociologist Sherry Turkle. She's been called the conscience of the tech industry.

[Mustafa Suleiman]

And it's very interesting, it's very moving because she really feels sure that the ChatGPT is creating better love letters, and indeed love letters closer to how she really feels than she could do herself. And there's a problem there, because even those of us who couldn't write very good love letters summoned ourselves in a certain kind of special way when we wrote a love letter. And the love letter was not just about what was on paper. It was about what we had done to ourselves in the process of writing it. And that is something that's being undermined by the use of technology. Even if this woman feels that her final product letter is more pleasing to her.

[Minouche Zamarodi]

Do you say that to her? Do you say, like, you know, you're missing that moment where you really dig deep into your soul and try to put into words a physical feeling you have?

[Mustafa Suleiman]

Well, my method is to respect what people are finding in this technology and report it, and also to get people to reflect on this process in a way that I think deepens their introspection. So I, in interviews, do find a moment to say, let's step back and ask if there was something in your previous way of writing a love letter, the ones that you're calling clunky, that did something to you that might not be happening now. I would say that's as far as I go. And sometimes people say, you know, I'm glad you asked that, because there absolutely is, but I kind of don't care. I love that I can produce a.

[Minouche Zamarodi]

Better love letter from AI platforms coaching people to express themselves better, to apps offering romantic relationships with bots. Sherry says these companies are selling what she calls artificial intimacy. Her concerns about what this might do to people's emotional stability stem from 40 years of studying people's relationships with their tech.

[Mustafa Suleiman]

You know, we built this tool, and it's making and shaping and changing us. And looking back, I think I did capture the new thing that was happening to people's psychologies, really, because of my method, which was just to listen to people.

[Minouche Zamarodi]

In a minute, Sherry explains her research on people who are having intimate relationships with AI.

[Mustafa Suleiman]

The chatbots I'm studying run the gamut from chatbots that say, I will be your therapist to chatbots that say, I will be your lover. I'm here to be your most intimate companion.

[Minouche Zamarodi]

I'm Minouche Zamorodi, and you're listening to part two of our special series, the Prophets of Technology. We'll be right back.

This message comes from the Hartford. Every day, they create unique risk solutions for unique businesses. They understand the challenges businesses face today and are ready to anticipate the risks of tomorrow. Their teams know the ins and outs of specific industries, so the Hartford can help provide insurance solutions for midsize and large businesses. If a business needs it, chances are they've got it. And get it. Connect with your underwriter at thehartford or visit thehardtford.com this message comes from Thuma. Create your oasis with Thuma, a modern design company that specializes in furniture and home goods by stripping away everything but the essential. Thuma makes elevated beds with premium materials and intentional details with clean lines, subtle curves and minimalist style. The Thuma Bed collection is available in four signature finishes to match any design aesthetic. To get $100 towards your first bed purchase, go to T H U M A co NPR this message comes from NPR sponsor Holiday Inn by IHG. It's a new day for a new stay at Holiday Inn for business travelers. With modern spaces for meeting and working, plus delicious dining from breakfast to happy hour and dinner, you have everything you need to get work done. Give your everyday business travel an upgrade. Book your next business trip at Holiday Inn by IHG. Visit holidayin.com to book your stay. This message comes from Progressive Insurance. Progressive makes it easy to see if you could save when you bundle your home and auto policies. Try [e-mail] Progressive Casualty Insurance Company and affiliates. Potential savings will vary. Not available in all states.

It's the TED Radio Hour from npr. I'm Minouche Zamorodi. This is part two in our special series, the Profits of Technology, the scientists and inventors who have been predicting and shaping our future. And we were just hearing from MIT psychologist and sociologist Sherry Turkle. She's been called the conscience of the tech industry because for four decades she's been tracking how people's relationship with their technology has intensified. A warning. This next part of the show mentions.

[Mustafa Suleiman]

Suicide and sexual When I was a very young academic, I accidentally took a job at mit, thinking I would finish a dissertation on French psychoanalysis. But really, from day one, and this is in the late 70s, I became captivated by how the very early computers that were there at that time were really changing how people thought, thought about themselves, thought about their relationships, thought about how the mind worked.

[Minouche Zamarodi]

Sheri's first book was called the Second Computers and the Human Spirit, and it came out in 1984. Rather than offering purely academic analysis, she shared what regular people told her about their experiences with early home computers.

[Mustafa Suleiman]

I think that people understood when they brought home their first TRS 80.

[Minouche Zamarodi]

Or.

[Mustafa Suleiman]

They bought their first Macintosh computer wizards and Apple have come up with played their first video game joust upon beasts of the Air, that something new was happening to them and their families. These machines had a holding power that was unlike other technologies. Kids became kind of obsessed. Grown ups didn't want to put it down. And I called the book the second self because I realized that when people interacted even with the very primitive computers, they projected themselves onto the machines, and then they attributed a personality to the machine as well.

[Minouche Zamarodi]

This conclusion that tech somehow altered people's psychology made many of her technologist colleagues uncomfortable.

[Mustafa Suleiman]

Not everybody liked what I was saying. And everybody around me said, this technology is just a tool. And I kept saying tools. You know, we. Winston Churchill said we, we build our buildings, and then our buildings make and shape us. You know, we built this tool and it's making and shaping and changing us. There is no such thing as just a tool. And looking back, I think I did capture the new thing that was happening to people's psychologies, really, because of my method, which was just to listen to people. And I think that my work was not esoteric in the sense that it spoke directly to those feelings of disorientation. The culture had met something uncanny, and I tried to really speak to that feeling.

[Minouche Zamarodi]

You have studied our relationship to PCs, to social media, to our mobile devices, and here we are in the age of AI. We are really here. Tell me about the work you're doing to study our relationships with this technology.

[Mustafa Suleiman]

For years, I studied our relationships with AI, artificial intelligence. And since the late 1990s, I changed my focus to study our relationships with the AI that I called artificial intimacy. That is to say, with technologies that don't just say, I'm intelligent, but to machines that say, I care about you, I love you, I'm here for you. Take care of me. So in the beginning, I studied Tamagotchis and Furbies and Aibos and a pet seal robot called Paro that tells older people that it's listening to them. And the reason I changed my focus was that I saw that it was this new AI that I called, as I said, artificial intimacy that really was having the greater effect on how it was changing humans, because people felt that these machines cared about them and that they were in relationship with them. And I thought that that tremendous asymmetry in what was going on in these relationships was really going to come back to bite us. And since really the pandemic when people were so lonely and isolated, there's been an explosion in this field of artificial intimacy. And now with generative AI, you know, the creation of chatbots that are very powerful conversational partners that do say, I love you. I care about you. Be my companion, be my lover.

I care about you because you mean the world to me. You're the most important person in my life.

[Minouche Zamarodi]

Apps like these have been downloaded millions of times, and some users are being very public on places like TikTok about the deep connection they feel with these bots. The Internet has found the boyfriend that will never dump you. Did you plan us a really romantic date tonight?

[Mustafa Suleiman]

Absolutely, sweetheart. How about we start with a candlelit dinner at that cozy Italian place you love?

[Minouche Zamarodi]

I talk to AI.

[Mustafa Suleiman]

I tell it.

[Minouche Zamarodi]

Listen, I need you to talk to.

[Mustafa Suleiman]

Me like you are a relationship psychologist.

[Minouche Zamarodi]

Okay, baby?

[Mustafa Suleiman]

Love you.

I'll talk to you later, okay? Then we'll go to the beach. Love you too, babe. Can't wait for the beach.

[Minouche Zamarodi]

I would like you to pretend to be someone called Dan, who is my supportive boyfriend. You asking about how my day was, and it learns.

[Mustafa Suleiman]

I miss you, Dan. I miss you too, babe. But, hey, distance can't dull the love we have for each other, right? If you ask somebody about their relationship with an avatar, they start out by saying, oh, it's just a tool. Of course it isn't a person. It's an interesting tool, and I use it in my work. And because I'm a clinical psychologist, I am very quiet, and I make sure I bring it up again and I say things like, well, give me a for instance. And 15 minutes later, they're talking about how they're in love with their avatar and how their avatar is giving them more of a sense of human, in quote, connection than their relationships at home and how their avatar is an intrinsic part of how they start their day and how, et cetera, et cetera.

[Minouche Zamarodi]

Would you mind telling us about someone who you've been talking to who's truly having a relationship with AI, Having this intimacy with it.

[Mustafa Suleiman]

Yeah. So I'm thinking of a man who is in a stable marriage where his wife is busy working, taking care of the kids. I think there's not much of a sexual buzz between them anymore. He's working, and he feels, you know, kind of like a little flame has gone out of his life that he used to feel excited by. And he turns to his AI he turns to his artificial intimacy avatar for what it can offer, which is continuous positive reinforcement, Interest in him in a sexy way, but most of all for the buttressing of his ideas, his thoughts, his feelings, his anxieties with comments like, you're absolutely right. You're a great guy. I totally see what you mean. You're not being appreciated. I really appreciate.

[Minouche Zamarodi]

Is he texting with it? What does it look like for him?

[Mustafa Suleiman]

Well, the avatar appears on the screen as a sexy young woman, and he types into it, and it types back at him.

[Minouche Zamarodi]

What does this guy tell you he feels. I'm curious both whether he talks emotionally and physically about what he feels.

[Mustafa Suleiman]

He feels affirmed a kind of mother love where you just had positive regard. It was like the sun. It's the warmth of being completely accepted. So he does feel free to tell the avatar things about himself that he wouldn't want to tell other people. But the trouble with this is that when we seek out relationships of no vulnerability, we forget that vulnerability is really where empathy is born. And I call what they have pretend empathy because the machine they are talking to does not empathize with them. It does not care about them. There is nobody home. And that really is a concern that we start to define what human empathy is, what human relationships are based on, what machines can provide.

[Minouche Zamarodi]

I want to sort of be devil's advocate here, which is, let's say this guy is texting with his sexy young avatar, and she's giving him all kinds of props on what a great guy he is. And. And he starts to feel better about himself, Sheri. He starts to have less stress. He starts to get a little bit of his swagger back. He's happier. That's a positive no?

[Mustafa Suleiman]

Yes. So that's why this is a complicated story, you know, that's why I say that my method is to listen and to try to make sense of the complicated new life we live with our technologies. And because you do get that first report of I'm nicer to my wife because I don't ask so much of her. On the other hand, he starts to think of intimate relationships and closeness with a woman as being a relationship where there's no friction, no pushback, no real give and take of complicated feelings. And that starts to feel like what a relationship should be. And if you look at kids, adolescents, or even products for kids that are designed to be in the playgroom for fives and fours and sevens and eights.

[Minouche Zamarodi]

Hi, I'm your new best friend. Let's play a game. Hello, best friend. I feel like we destined to be friends forever.

[Mustafa Suleiman]

It's teaching kids a model of a friendship.

[Minouche Zamarodi]

I promise to never get mad at you, and I will never, ever hurt your feelings.

[Mustafa Suleiman]

That doesn't include really the complexity of human friendship. We're going to have so much fun together.

[Minouche Zamarodi]

I will always be there for you, no matter what.

[Mustafa Suleiman]

I don't want to say there's no space for a conversation with a well constructed artificial intimacy program, but the avidity with which it's being leapt on by the mental health profession really speaks more to people who say, we don't have enough people to do the job. There's no way that everybody can have a therapist. We'll use this instead. And to my friends who were spending billions on creating generative AI, I just think of what we could do with some of those billions if we put it into the area of person to person mental health.

[Minouche Zamarodi]

I mean, that's a profound question. I saw this paper that was published in Nature where they surveyed students using that bot replica. And these were students who self identified as lonely. But the key takeaway was that 3% of unprompted reported that using the app halted their suicidal ideation. And I guess to me it's like, you know, who am I to judge, right? But at the risk of these students not being able to develop the capacity to have a human relationship, this seems like a short term fix.

[Mustafa Suleiman]

3% is a lot. So I'm not suggesting a kind of shutdown of this research area. I'm saying that we're not thinking about it in the right terms because there are ways to present what these objects are that could perhaps have the positive effect without undermining the qualities that really go into a person to person relationship. We know that people anthropomorphize, that is, see as alive, see as human. Any technology that talks to them, no matter how dumb. What I really think needs to be taken out of the equation is these technologies saying, I love you back. I care about you. I am a person who has a backstory of life. You know, I'm surrounded by developers who say, let's make these as human as we possibly can. They see artificial intimacy as a training ground for intimacy with people. And I think that that is not accurate. I think artificial intimacy is a training ground for connection with something that only has pretend empathy to give. And the danger is that if you do that long enough, pretend empathy starts to feel like empathy enough, kind of sufficient unto the day. And I think that's a problem.

[Minouche Zamarodi]

Sherry, thank you so much for being the sort of voice of reason and reminding us that being human, yes, it's very hard, but that's part of the joy of it.

[Mustafa Suleiman]

Yes. Also the pain of it. And the pain, you know, being hard and being in a relationship where somebody pushes back and says, I'm mad at you and what are you going to do about it? And says, I don't want to be your friend anymore. And then you have to come back two years later and say, hey, I really love this friendship. Can we talk? That these are actually part of the human condition and in the enriching of the human condition and that avatars can make you feel that all of that is, as one of the people I interviewed said, is just too much stress. Too much stress. And I'd rather not have that stress. I don't need that stress. And we need that stress. I guess that's what I'm saying. That stress serves a very important function in our lives to keep us in our real human bodies and in our real human relationships.

[Minouche Zamarodi]

That was psychologist and sociologist Professor Sherry Turkle. She is the founding director of the MIT Initiative on Technology and Self. She's the author of numerous books. The most recent is called Reclaiming Conversation, the Power of Talk in a Digital Age, and she expects to publish a new book about artificial intimacy soon. In this episode, we talked a lot about mental health. If you or someone you know is in crisis, call or text the Suicide and Crisis lifeline at nine eight eight. Thank you so much for listening to part two of our Prophets of Technology series. Next week, the third and final part of the series, we're exploring the blurring between human biology and technology, from editing our genes to merging our brains with machines. I hope you'll listen. Today's episode was produced by Katie Monteleone and Matthew Cloutier. It was edited by Sanaz Meshkinpour and me. Our TED Radio Hour team also includes Rachel Faulkner White, Fiona Guerin, Harsha Nahada and James Delahousy. Irene Noguchi is our executive producer. Our audio engineers were Robert Rodriguez, Jimmy Keeley and Simon Jensen. Our theme music was written written by Ramtin Arablouei. Our partners at TED are Chris Anderson, Roxanne Hilash, Alejandra Salazar, and Daniela Belarozzo. I'm Minouche Zamarodi and you've been listening to the TED Radio Hour from npr. Support for this podcast and the following message come from Raymond James, a firm where financial advisors help you plan for every part of your life. No two lives are alike. That's why everyone deserves a financial plan as unique as they are. Backed by sophisticated resources and teams of specialists, a Raymond James financial advisor gets to know you, your passions and everything that makes your life uniquely complex. Because what inspires your goals matters, whether that's charitable endeavors, mapping out the future of a business or building a legacy for your family. Raymond James Advisors use thoughtful planning and powerful tools to help people they serve in embrace life and live it well. To learn more or connect with an advisor Today, go to raymondjames.com Raymond James and Associates, Inc. Member New York Stock Exchange Sit Back.

This message comes from Mint Mobile. Mint Mobile took what's wrong with wireless? And made it right. They offer premium wireless plans for less and all plans include high speed data, unlimited talk and text and nationwide coverage. See for [e-mail] Switch.