Good Robot #2: Everything is not awesome

Summary

---

Future Perfect: Good Robot #2 – "Everything is not awesome"

Podcast: Future Perfect (Vox)
Series: Good Robot (Episode 2 of 4)
Date: March 15, 2025
Host: Julia Longoria

---

Overview

In this episode, Julia Longoria dives into one of the central debates shaping the future of artificial intelligence: why do even well-intentioned AIs go wrong, and why is there so much disagreement among those who seek to solve AI’s dangers? Through the stories of Dr. Margaret Mitchell, Dr. Joy Buolamwini, Dr. Timnit Gebru, and others, the episode traces the roots of bias, misunderstanding, and hype in modern AI systems. The conversation illuminates the distinction—and the tension—between the "AI safety" and "AI ethics" movements, asking why each group focuses on different risks, whom AI harms today, and why calls to "pause" AI development are more contentious than they appear.

---

Key Discussion Points and Insights

1. The “Everything Is Awesome” Problem: How AI Goes Wrong

[01:18–06:08]

• Dr. Margaret Mitchell's AI Origin Story:

  • Early work at Microsoft (2013) focused on "vision to language" models: turning sequences of images into descriptions.
  • A dramatic example: the model described images of a deadly factory explosion as "awesome."
  • Quote:

    "So it sees this horrible, perhaps mortally wounding explosion and decides it's awesome."
    —Dr. Margaret Mitchell [04:17]

  • This became known as the "everything is awesome problem"—AI parroting the positive language often found in training data, regardless of context.

• Why Did This Happen?

  • The system was trained on images and captions from Flickr, which are biased towards "awesome" sunsets, not tragic scenes.

  • Quote:

    "People tend to take pictures of like sunsets… when we are taking pictures, we like to say it's awesome… But that was a bias in the training data."
    —Dr. Margaret Mitchell [15:36]

  • The realization: AI learns and repeats the biases in its training data.

---

2. “Everyone Is White” Problem: Bias in AI Recognition

[09:08–13:56]

• Dr. Joy Buolamwini’s Discovery:

  • Inspired by seeing MIT’s “Kismet” robot, Dr. Joy became a robotics PhD student.
  • While working on facial recognition tech for digital masks, the system failed to detect her dark-skinned face until she donned a white masquerade mask.
  • Quote:

    "Before I even put the white mask all the way over my dark skinned face, the box saying that a face was detected appeared."
    —Dr. Joy Buolamwini [13:16]

• Findings and Impact:

  • She tested major facial-recognition AIs (Google, Microsoft, Amazon), which misidentified Black faces at alarming rates (e.g., labeled Oprah Winfrey "male").
  • Tech’s high advertised accuracy hid that their test sets skewed “over 70% men” and lighter skin tones—what Joy called “pale male datasets” [18:38].
  • Gender Shades paper ([19:18]): Showed racial and gender bias, leading Microsoft, IBM, and Amazon to temporarily halt facial-recognition sales.

• Real-World Consequences ([20:22]):

  • Robert Williams, a Black man, was wrongfully arrested after a misidentification by police facial recognition software.
  • Quote:

    "I got arrested for something I had nothing to do with… everybody with a driver's license or state ID is essentially in a photo lineup."
    —Robert Williams [20:16]

---

3. The Birth of the AI Ethics Movement

[21:23–23:44]

• The episode visits a diverse AI ethics conference, contrasting it with rationalist (mostly white/male) gatherings.
• The field bloomed especially after 2020’s Black Lives Matter protests, with increasing focus on ongoing harms caused by biased AI.
• Google Firings: Drs. Mitchell and Gebru were both pushed out after raising these issues internally at Google (their landmark paper critiqued big, opaque models and environmental impact).
• Quote:

  "That firing really brought it in focus… That was the clarion call."
  —AI Ethics Researcher [26:08]

---

4. AI Safety vs. AI Ethics: Two Camps, One Field

[31:18–36:59]

• Sigal Samuel (Vox reporter) introduces: The longstanding divide between “AI safety” (focused on existential risks and superintelligent 'God' AI) and “AI ethics” (focused on current harms like bias and discrimination).

• Religious Overtones:

  • Both camps use language reminiscent of faith, with safety people worrying about apocalypse and ethics people about lived injustices.
  • Quote:

    "Have you ever noticed that… the more you listen to Silicon Valley people talking about AI, the more you start to hear echoes of religion?"
    —Sigal Samuel [32:44]

  • AI “safety” named after Eliezer Yudkowsky and the rationalists (fearing superintelligent AI could “kill everyone”), while AI “ethics” is more grounded in preventing real-world bias.

• Notable Divide:

  • Despite shared concerns about harm, the camps often clash—AI safety wants to pause big models to avoid hypothetical disasters; AI ethics warns this exaggerates AI’s power and distracts from human injustices.
  • Quote:

    "Yeah, there is beef between the AI Ethics camp and the AI safety camp."
    —Sigal Samuel [35:59]

---

5. The “Pause” Letters: Tech Industry’s Call for a Timeout

[37:03–40:07]

• In 2023, industry leaders (Elon Musk, Steve Wozniak) call to "pause" AI development to consider risks, echoing rationalist fears.
• AI Ethicists Push Back: They argue this hype exaggerates AI capacity, distracts lawmakers, and frequently serves industry interests (e.g., letting some companies keep building while calling for regulation).
• Emily Bender’s “Parrot” and “Octopus” Arguments ([40:00–45:40]):
  • Large models like ChatGPT are probabilistic systems, parroting data; they don't “understand” in a meaningful sense.
  • Octopus Thought Experiment: Even the smartest system that can mimic communication (dots and dashes) has no true grasp of meaning.
  • Quote:

    "These are probabilistic systems that repeat back what they have been exposed to, and then they parrot them back out again."
    —Dr. Margaret Mitchell [45:15]

  • AI as a “parrot,” not as God or person—a comforting but humbling comparison.

---

6. Real Harms vs. Hyped Hype: What Should We Prioritize?

[46:34–53:24]

• AI Ethicists’ Main Critique: Fears of an AI apocalypse are a distraction from harmful impacts happening now—algorithmic bias, policing, hiring, and more.
• Demographic Blind Spots: The AI “safety” camp, mostly white and male, often underappreciates everyday realities others face.
• Profit Motive: Many tech leaders profit from hyping both the promise and peril of AI, even as they call for regulatory pauses.
• Quote:

  "It benefits them to… market the technology as super powerful… and distract the policymakers from the harms that they are doing."
  —Dr. Emily Bender [50:12]

---

Memorable Quotes & Moments (with Timestamps)

• “Everything is awesome” in tragedy

  "So it sees this horrible, perhaps mortally wounding explosion and decides it's awesome."
  —Dr. Margaret Mitchell [04:17]

• The problem of “whiteface” in coding:

  "Before I even put the white mask all the way over my dark skinned face, the box saying that a face was detected appeared. I'm thinking, oh my goodness… I'm literally coding in whiteface."
  —Dr. Joy Buolamwini [13:16]

• Pale male datasets:

  "These are what I started calling pale male datasets because the pale male datasets were destined to fail the rest of society."
  —Dr. Joy Buolamwini [18:53]

• Wrongful AI-driven arrest:

  "I got arrested for something I had nothing to do with… everybody with a driver's license or state ID is essentially in a photo lineup."
  —Robert Williams [20:16]

• On beef between ethical and safety camps:

  "Yeah, there is beef between the AI Ethics camp and the AI safety camp."
  —Sigal Samuel [35:59]

• Emily Bender’s Octopus Parable:

  "So the octopus thought experiment goes like this… It receives the dots and dashes from one of the English speakers and it sends dots and dashes back. But of course, it has no idea what the English words are… that those dots and dashes correspond to."
  —Dr. Emily Bender [41:07]

• The parrot analogy:

  "It's easy to sort of anthropomorphize these systems, but it's useful to recognize that these are probabilistic systems that repeat back what they have been exposed to, and then they parrot them back out again."
  —Dr. Margaret Mitchell [45:15]

• Critique of AI Apocalypse hype:

  "It makes no sense at all. And on top of that, it's an enormous distraction from the actual harms that are already being done in the name of AI."
  —Dr. Emily Bender [46:39]

---

Timeline of Major Segments

• [01:18–06:08] – Dr. Margaret Mitchell’s origin story and emergence of “everything is awesome” problem
• [09:08–13:56] – Dr. Joy Buolamwini’s journey: discovering facial recognition bias
• [19:18–20:41] – Gender Shades, wrongful arrests, and tech company responses
• [21:23–23:44] – Rise of the AI ethics field & the "clarion call" after the firing of top researchers
• [31:18–36:59] – Sigal Samuel on AI safety vs. AI ethics: beliefs, demographics, and religious undertones
• [37:03–40:07] – Elon Musk’s “pause” letter, the industry’s call for regulation, and ethicists’ responses
• [41:07–45:40] – Emily Bender’s octopus and parrot analogies
• [46:34–53:24] – Real-world harms, demographic divides, industry control, and the profit/hype loop

---

Concluding Thoughts

The episode exposes increasing polarization among AI researchers, distinguishing those raising alarms about existential risks from superintelligent AI ("AI safety") and those focused on immediate, societal harms ("AI ethics"). Through personal stories, striking analogies (like parrots and octopuses), and industry intrigue, it pushes listeners to question whose fears and priorities are shaping AI’s future—and who truly benefits when the loudest voices talk of doomsday or salvation.

---

For further reading, check Dr. Joy Buolamwini's book Unmasking AI or visit vox.com/goodrobot for more on AI and the "Good Robot" series.

---

Transcript

[Advertisement/Announcer]

Avoiding your unfinished home projects because you're not sure where to start. Thumbtack knows homes, so you don't have to. Don't know the difference between matte, paint, finish and satin or what that clunking sound from your dryer is. With Thumbtack, you don't have to be a home pro. You just have to hire one. You can hire top rated pros, see price estimates and read reviews all on the app download today.

[Julia Longoria (Host)]

Running a business comes with a lot of what ifs, but luckily there's a simple answer to Shopify. It's the commerce platform behind millions of businesses, including Thrive Cosmetics and Momofuku. And it'll help you with everything you need. From website design and marketing to boosting sales and expanding operations. Shopify can get the job done and make your dream a reality. Turn those what ifs into Sign up for your $1 per month [e-mail] SpecialOffer this is part two of a four part series. If you haven't listened to episode one, let me stop you right here. Go back in your feed and listen to the first one. We'll be waiting for you here when you get back once you're all caught up. Here is episode two of Good Robot.

[Dr. Margaret Mitchell]

You have cat hair on your nose, by the way. I've been like trying not to pay attention to it, but I think you got it off.

[Julia Longoria (Host)]

Yeah, sorry. Cool. So should we get into it?

[Dr. Margaret Mitchell]

Sure, yeah. Let me. It helps me to kind of remember everything I'm gonna say if I can sort of jot down thoughts as I go there. Do you happen to have paper?

[Julia Longoria (Host)]

I think I don't have paper. All right, I'll just.

[Dr. Margaret Mitchell]

I saw that they had random.

[Julia Longoria (Host)]

This past fall, I traveled paperless to a library just outside Seattle to meet with this woman.

[Dr. Margaret Mitchell]

I feel like a library.

[Julia Longoria (Host)]

I know that you should. Her name is Dr. Margaret Mitchell.

[Dr. Margaret Mitchell]

Found a brochure on making a robot puppet.

[Julia Longoria (Host)]

What is it? What is the.

[Dr. Margaret Mitchell]

I don't know. It looks like it's an event. Build a robot puppet using a variety of materials with puppeteer. I'm so into that.

[Julia Longoria (Host)]

Aw.

[Dr. Margaret Mitchell]

It's too bad that it's only for ages 6 to 12 while she is.

[Julia Longoria (Host)]

Over the age limit to make a robot puppet with the children in the public library. Dr. Mitchell is a bit of a robot puppeteer in her own right. What's your AI researcher origin story like? How did you get into all of this? What. What drew you here? Yeah. Oh.

[Dr. Margaret Mitchell]

What inspired me to. So, I mean, I guess I can. It's sort of like, do you want the long version or the short version?

[Julia Longoria (Host)]

Dr. Mitchell is an AI research scientist, and she was one of the first people working on language models, well before ChatGPT and, well, all the GPTs. She's an OG in the field.

[Dr. Margaret Mitchell]

So I'll tell you, like, I'll tell you a story, if that's okay. Yeah, okay. So I was at Microsoft, and I was working on the ability of a system to tell a story given a sequence of images. So given five images.

[Julia Longoria (Host)]

This was about 2013. She was working on a brand new technology at the time, what AI researchers called vision to language.

[Dr. Margaret Mitchell]

So, you know, translating images into descriptions.

[Julia Longoria (Host)]

She would spend her days showing image after image to an AI system. To me, it sounded kind of like a parent showing picture flashcards to a toddler learning to speak. She says it's not anything like that. She showed the model images of events like a wedding, a soccer match, and on the more grim side, I gave.

[Dr. Margaret Mitchell]

The system a series of images, about eight, a big blast that left 30 people wounded called the Hempstead Blast. It was at a factory, and you could see from the sequence of images that the person taking the photo had like a third story view sort of overlooking the explosion. So it was a series of pictures showing that there was this terrible explosion happening and whoever was taking the photo was very close to the scene. So I put these images through my system, and the system says, wow, this is a great view. This is awesome. And I was like, oh, crap. That is the wrong response to this. So it sees this horrible, perhaps mortally wounding explosion and decides it's awesome.

[Julia Longoria (Host)]

Kind of like a parent watching their precious toddler say something kind of creepy. Mitch watched in horror and with a deep fascination about where she went wrong, as the AI system that she had trained called images awesome again and again.

[Dr. Margaret Mitchell]

It said it quite a lot. So we called it the everything is awesome problem. Actually.

[Julia Longoria (Host)]

Her robot was having these kinds of translation errors, errors that, to the uninitiated, made it seem like the AI system might want to kill people or at least gleefully observe their destruction and call it awesome. What would the consequences of that be if that system was deployed out into the world reveling in human destruction?

[Dr. Margaret Mitchell]

Like, if this system were connected to a bunch of missile systems, then it's, you know, it's just a jump and skip away to just launch missile systems in the pursuit of the aesthetic of beauty. Right.

[Julia Longoria (Host)]

Years before the AI boom were living when neural networks and deep learning were just beginning to show promise. Researchers like Dr. Mitchell and others were experiencing these uncanny moments where the AIs they were training seemed to do something seriously wrong, doing scary things their creators did not intend for them to do that were seemingly threatening to humanity.

[Dr. Margaret Mitchell]

So I was like one of the first people doing these systems where you could scan the world and have descriptions of it. I was like on the forefront. I was one of the first people making these systems go. And I realized, like, if anyone is going to be paying attention to it right now, it has to be me.

[Julia Longoria (Host)]

I had heard the fears of rationalists, also pioneers in thinking about AI, that we might build a super intelligent AI that could go rogue and destroy humanity. At first glance, it seemed like Dr. Mitchell might be building one such robot. But when Dr. Mitchell investigated the question of why the good robot she sought to build seemed to turn bad, the answer would not lead her to believe what the rationalists did that a super intelligent AI could someday deceive or destroy humanity. To Dr. Mitchell, the answer was looking at her in a mirror. This is episode two of Good Robot, a series about AI from Unexplainable in collaboration with Future Perfect. I'm Julia Longoria. If you know your party's Extension, press or say 1. To leave a message in our company mailbox, press or say 2. Spoiler alert.

[Advertisement/Announcer]

It will be full representative.

[Julia Longoria (Host)]

Would you speak to your mother in that tone? Speak to a real human being. You shouldn't need to shout into the void to get your health insurance questions answered. Pacific Source Health Plans. This is a real person. How can I help you? Human service, not automated phone trees. Pacific Source health plans.

[Advertisement/Announcer]

At blinds.com, it's not just about window treatments. It's about you. Your style, your space, your way. Whether you DIY or want the pros to handle it all, you'll have the confidence of knowing it's done right. From free expert design help to our 100% satisfaction guarantee, everything we do is made to fit your life and your windows. [e-mail], the only thing we treat better than Windows is you. Visit blinds.com now for up to 45% off with minimum purchase plus a professional measure at no cost. Rules and restrictions apply. This episode is brought to you by Progressive Insurance. Do you ever find yourself playing the budgeting game? Well, with the name your price tool from Progressive, you can find options that fit your budget and potentially lower your bills. Try [e-mail] Progressive Casualty Insurance Company and affiliates Price and coverage match limited by state law. Not available in all states.

[Julia Longoria (Host)]

I kind of want to start with a bit of a basic question of when you were young, what did you want to do when you grew up?

[Dr. Joy Buolamwini]

I wanted to be everything. I wanted to be a pole vaulter. I wanted to be a skateboarder.

[Julia Longoria (Host)]

Dr. Joy Buolamwini's robot researcher origin story goes back to when she was a little kid.

[Dr. Joy Buolamwini]

I had a very strict media diet. I could only watch pbs, and I remember watching one of the science shows, and they were at mit, and there was a graduate student there who was working on a social robot named Kismet.

[Dr. Margaret Mitchell]

Hello, Kismet. You gonna talk to me?

[Julia Longoria (Host)]

Yes. Kismet was a robot created at MIT's AI lab.

[Advertisement/Announcer]

Oh, God, did he say he loves me?

[Dr. Joy Buolamwini]

And Kismet had these big expressive eyes and ears and could emote or appear to emote in certain ways. And I was just absolutely captivated.

[Julia Longoria (Host)]

She watched, glued to the screen as the researchers demonstrated how they teach Kismet to be a good robot.

[Dr. Margaret Mitchell]

No, no, you're not to do that.

[Julia Longoria (Host)]

The researchers likened themselves to parents. You know, as parents, when we exaggerate.

[Sigal Samuel]

The prosody of her voice going, oh, good, baby.

[Julia Longoria (Host)]

You know, our facial expressions and our gestures.

[Dr. Joy Buolamwini]

So when I saw Kismet, I told myself I wanted to be a robotics engineer and I wanted to go to mit. I didn't know there were requirements. I just knew that it seemed really fascinating, and I wanted to be a part of creating the.

[Julia Longoria (Host)]

Thanks to Kismet, she went on to build robots of her own at MIT as an adult. She went for her PhD in 2015. This was just a few years after Dr. Margaret Mitchell had accidentally trained her robot to call scenes of human destruction awesome.

[Dr. Joy Buolamwini]

My first year, my supervisor at the time had encouraged me to just take a class for fun.

[Julia Longoria (Host)]

For her fun class that fall, Dr. Joy, as she now prefers to be called, set out to play. She wanted to create almost a digital costume.

[Dr. Joy Buolamwini]

If I put a digital mask so something like a lion, it would appear that my face looks like a lion.

[Julia Longoria (Host)]

What Dr. Joy set out to do is something we can now all do on the iPhone or apps like Instagram or TikTok. Kids love to do this. You can turn your face into a hippo face or an octopus face that talks when you talk, or you can make it look like you're wearing ridiculous makeup. These digital face masks were still relatively uncommon in 2015.

[Dr. Joy Buolamwini]

So I went online and I found some code that would actually let me track the location of my face.

[Julia Longoria (Host)]

She'd put her face in front of a webcam and the tech would tell her, this is A face by showing a little green square box around it.

[Dr. Joy Buolamwini]

And as I was testing out this software that was meant to detect my face and then Track actually wasn't detecting my face that consistently.

[Julia Longoria (Host)]

She kept putting her face in front of the webcam to no avail, no green box.

[Dr. Joy Buolamwini]

And I'm frustrated because I can't do this cool effect so that I can look like a lion or Serena Williams. I have problems.

[Julia Longoria (Host)]

The AI's Dr. Joy was using from places like Microsoft and Google had gotten rave reviews. They were supposed to use deep learning, having been trained on millions of faces to very accurately recognize a face. But for her, these systems couldn't even accomplish the very first step to say whether her face was a face at all.

[Dr. Joy Buolamwini]

I'm like, well, can it detect any face?

[Julia Longoria (Host)]

Dr. Joy looked around her desk. She happened to have an all white masquerade mask lying around from a night out with friends.

[Dr. Joy Buolamwini]

So I reached for the white mask, it was in arm's length. And before I even put the white mask all the way over my dark skinned face, the box saying that a face was detected appeared. I'm thinking, oh my goodness, I'm at the epicenter of innovation and I'm literally coding in whiteface. It felt like a movie scene, you know. But that was kind of the moment where I was thinking, wait a second, like, what's even going on here?

[Julia Longoria (Host)]

What is even going on here? Why couldn't facial recognition AI detect Dr. Joy's dark skin? For that matter, why did Dr. Mitchell's AI call human destruction awesome? These AI scientists wanted the robot to do one thing. And if they didn't know any better, they might think the AI had gone rogue, developed a mind of its own and done something different. Were AI's racist? Were they terrorists plotting human destruction?

[Dr. Margaret Mitchell]

But I understood why it was happening.

[Julia Longoria (Host)]

Dr. Margaret Mitchell knew exactly what was going on. She had been the one to develop Microsoft's image to text language model from the ground up. She had been on the team figuring out what body of data to feed the model to train it on in the first place. Even though it was creepy, it was immediately clear to her why the AI wasn't doing what she wanted it to do.

[Dr. Margaret Mitchell]

It's because it was trained on images that people take and share online.

[Julia Longoria (Host)]

Dr. Mitchell had trained the AI on photos and captions uploaded to the website Flickr. Do you remember Flickr? I was the prime age for Flickr when it came out in 2004. This was around the time that Jack Johnson released the song Banana Pancakes. And that really was the Vibe of Flickr. There's no denying it. I can see the receipts on my old account. I favorited a close up image of a ladybug, an artsy black and white image of piano keys, and an image titled Pacific Sunset.

[Dr. Margaret Mitchell]

People tend to take pictures of like sunsets.

[Julia Longoria (Host)]

Actually, I favorited a lot of sunsets. Another one sunset at the Rio Negro.

[Dr. Margaret Mitchell]

So it had learned, the system had learned from the training data I had given it that if it sees purples and pinks in the sky, it's beautiful. If it's looking down, it's a great view. That when we are taking pictures, we like to say it's awesome. Apparently on Flickr images, people use the word awesome to describe their images quite a lot. But that was a bias in the training data.

[Julia Longoria (Host)]

The training data, again, being photos and captions uploaded by a bunch of random people on Flickr. And Flickr had a bias toward awesome photos, not sad photos.

[Dr. Margaret Mitchell]

The training data wasn't capturing the realities of like, human mortality. And, you know, that makes sense, right? Like, when's the last time you like, took a bunch of selfies at a funeral? I mean, it's not the kind of thing we tend to share online, and so it's not the kind of thing that we tend to get in training data for AI systems. And so it's not the kind of thing that AI systems tend to learn.

[Julia Longoria (Host)]

What she was discovering was that these AI systems that use the revolutionary new technology of deep learning, they were only as good as the data they were trained on.

[Dr. Margaret Mitchell]

So it sees this horrible, perhaps mortally wounding situation and decides it's awesome. And I realized, like, this is a type of bias and nobody is paying attention to that. I guess I have to pay attention to that.

[Julia Longoria (Host)]

Dr. Mitchell had a message for beware of what you train your AI systems on, right? What are you letting your kid watch?

[Dr. Margaret Mitchell]

Yeah, I mean, it's a similar thing, right? Like you don't want your kid to, I don't know, hit people or something so you don't like, let them watch lots of shows of people hitting one another.

[Julia Longoria (Host)]

Dr. Joy Buolamwini, coding in whiteface, suspected she was facing a similar problem. Not an everything is awesome problem, but an everyone is white problem. In the training data, she tested her face and the faces of other black women on various facial recognition systems.

[Dr. Joy Buolamwini]

You know, different online demos from a number of companies, Google, Microsoft, others.

[Julia Longoria (Host)]

She found they weren't just bad at recognizing her face, they were bad at recognizing famous black women's faces. Amazon's AI labeled Oprah Winfrey as male. And the most baffling thing for Dr. Joy was the dissonance between the terrible accuracy she was seeing and the raving reviews the tech was getting. Facebook's DeepFace, for instance, claimed 97% accuracy, which is definitely not what Dr. Joy was seeing. So Dr. Joy looked into who these companies were testing their models on.

[Dr. Joy Buolamwini]

They were around 70 or over 70% men.

[Julia Longoria (Host)]

People thought these AIs were doing really well at recognizing faces because they were largely being tested with the faces of lighter skinned men.

[Dr. Joy Buolamwini]

These are what I started calling pale mail data sets because the pale male data sets were destined to fail the rest of society.

[Julia Longoria (Host)]

It's not hard to jump to the life threatening implications here. Like self driving cars, they need to identify the humans so they won't hit them. Dr. Joy published her findings in a paper called Gender Shades.

[Dr. Joy Buolamwini]

Welcome. Welcome to, welcome to the fifth anniversary celebration of the Gender Shades paper.

[Julia Longoria (Host)]

The paper had a big impact, as.

[Dr. Joy Buolamwini]

You see from the newspapers that I have. This is Gender Shades in the New York Times.

[Julia Longoria (Host)]

The fallout caused various companies, Microsoft, IBM, Amazon, who'd been raving about the accuracy of their systems, to at least temporarily stop selling their facial recognition AI products.

[Dr. Joy Buolamwini]

I'm honored to be here with my sister, Dr. Timit Gabriel, who co authored the paper with me.

[Julia Longoria (Host)]

Dr. Timneet Gabru was Dr. Joy's mentor and co author on the paper.

[Dr. Margaret Mitchell]

This is the only paper I think I've worked on where it's 100% black women authors. Right.

[Julia Longoria (Host)]

Dr. Gabriel had worked from her post leading Google's AI ethics team to help pressure Amazon to stop selling facial recognition AI to police departments because police were misidentifying suspects with the technology.

[Advertisement/Announcer]

I got arrested for something I had nothing to do with me. And I wasn't even in the vicinity of the crime when it happened.

[Julia Longoria (Host)]

One person they helped was a man named Robert Williams. Police had confused him for another black man using facial recognition AI.

[Advertisement/Announcer]

It's just that the way the technology is set up, everybody with a driver's license or a state ID is essentially in a photo lined up.

[Julia Longoria (Host)]

They arrested him in front of his wife and two young daughters.

[Advertisement/Announcer]

Me and my family, we're happy to be recognized because it shows there is a group of people out here who do care about other people.

[Julia Longoria (Host)]

Hey, how you doing?

[Dr. Margaret Mitchell]

Good.

[Julia Longoria (Host)]

Can you just say what you're standing in front of? Yeah, I'm standing in front of a poster which talks about how we can better identify racial disparities in automated decisions when there's not. Producer Gabrielle Burbe Traveled to a conference in San Jose full of researchers inspired by the work of Dr. Joy, Dr. Gabe Rue and Dr. Mitchell.

[Dr. Margaret Mitchell]

So I just presented a paper about how data protection and privacy laws enable companies to target and manipulate individuals.

[Julia Longoria (Host)]

Unlike the Rationalists Festival conference thing, which felt like a college reunion of mostly white dudes, this one felt more like a science fair. A pretty diverse one. Lots of people of color, lots of women with big sciency poster boards lining the wall.

[Researcher at AI Ethics Conference]

I'm standing in front of my poster, which spans language technologies and AI and how those perform for minority populations.

[Julia Longoria (Host)]

They were presenting on ways AI worries them today, not some hypothetical risk in the future.

[Dr. Margaret Mitchell]

There are real harms happening right now.

[Julia Longoria (Host)]

From autonomous exploding drones in Ukraine to bias and unfairness in decision making systems. And who did you co author the paper with? This was a collaboration with lots of researchers. Dr. Mitchell was one of them. Many of them knew Dr. Mitchell, Dr. Gabriel and Dr. Joy. Dr. Mitchell even worked with a couple researchers here on their project. So she led the project. She offered so, so much amazing guidance. I should say many researchers were mentored by them. We got the sense that they're kind of founding mother figures of this field. A field that really started to blossom, we were told, around 2020, a big year of cultural reckoning.

[AI Ethics Researcher]

A big inflection point was in 2020 when people really started reflecting on how racism is is unnoticed in their day to day lives. I think until BLM happened, these issues were almost considered woke and not something that was really real.

[Julia Longoria (Host)]

2020 was the year the pandemic began, the year Black Lives Matter protests erupted around the country. AI researchers were also raising the alarm that year on how AI was disproportionately harming people of color. Dr. Gabriel and Dr. Mitchell in particular were working together at Google on this issue. They built a whole team there that studied how biased training data leads to biased AI models.

[AI Ethics Researcher]

Timnit and Meg were the visionaries at Google who were building that team.

[Julia Longoria (Host)]

2020 was also the year that OpenAI released GPT3. And Dr. Gabriel and Dr. Mitchell, both at Google at the time, were concerned about a model that was so big and it was trained on basically the entire Internet. Here's Dr. Mitchell again.

[Dr. Margaret Mitchell]

A lot of training data used for language models comes from Reddit. And Reddit has been shown to have a tendency to be sort of misogynistic and also Islamophobic. And so that means that the language models will then pick up those views.

[Julia Longoria (Host)]

Dr. Mitchell's concern was that these GPT large language models trained on a lot of the Internet were too large. Too large to account for all the bias in the Internet. Too large to understand, and so large that the compute power it took to keep these things going was a drain on the environment. Dr. Gabreu, Dr. Mitchell and other colleagues put it all in a paper and tried to publish it while working at Google. I've kind of been wanting to talk to you ever since I saw your name signed Schmargit Schmidt. When I first read this paper, the thing that immediately stood out to me was the way Margaret Mitchell had signed her name. Schmargaret Schmitchel. Where did that come from?

[Dr. Margaret Mitchell]

Well, so I wrote a paper with a bunch of other co authors that Google ended up having some issues with, and they asked us to take our names off of the paper. So we complied. And that's, you know, that's what I have to say about that.

[Julia Longoria (Host)]

The first time I heard Dr. Mitchell and Dr. Gabreu's name was in the news.

[Advertisement/Announcer]

Last week, Google fired one of its most senior AI researchers who was working on a major artificial intelligence project within Google.

[Julia Longoria (Host)]

Their boss at Google said their paper ignored relevant research, research that made AIs look less damaging to the environment, for instance. Dr. Gabriel refused to take her name off the paper and Google accepted her resignation before she officially submitted it.

[Dr. Margaret Mitchell]

We decided that the verb for that would be resignated. Resignated.

[Advertisement/Announcer]

And now Margaret Mitchell, the other co lead of Google's ethical AI team, said.

[Julia Longoria (Host)]

She had been fired. Google later apologized internally for how the whole thing was handled, but not for their dismissal. We reached out to Google for comment, but got no response.

[AI Ethics Researcher]

And that firing really brought it in focus. And people were like, oh, this horrible thing just happened. Everywhere around the world is seeing protests and now this company is firing two leading researchers who work on that very exact problem which AI is making worse. You know, like, how dare they. So that from ipov? That was, yes, basically the clarion call.

[Julia Longoria (Host)]

The Clarion call. It was heard well beyond the world of AI. I remember hearing it when the world had screeched to a halt from the pandemic and protests for racial justice had erupted around the country. I remember hearing headlines about how algorithms were not solving society's problems. In some cases, AI systems were making injustice worse. And there was a brief moment back then when it felt like maybe things could be different, maybe things would change. And then a couple years later, a group of very powerful tech executives got together to try to change things in the AI world. This morning, a warning from Elon Musk and other tech industry experts it wasn't necessarily the people you'd think would want to change the status quo, like Elon Musk. Musk and other big names in tech like Apple co founder Steve Wozniak. They all signed a letter with a clear and urgent title. Pause Giant AI experiments. More than 1300 tech industry leaders, researchers.

[Sigal Samuel]

And others are now asking for a.

[Julia Longoria (Host)]

Pause in the development of artificial intelligence to consider the risks. Musk and hundreds of influential names are calling for a pause in experiments saying AI poses a dramatic risk to society. The letter called on AI Labs to immediately pause developing large AI systems for at least six months. An urging to press the big red button that stops the missile launch before it's too late. I scrolled through the list of names of people who signed the letter and I didn't see Dr. Joy or Dr. Mitchell or or any of the rationalists I talked to who are worried about risks in the future, which logically didn't make sense to me. Isn't a pause in line with what they all wanted for people to build the robots more carefully? Why wouldn't they want to pause? An answer to this pause puzzle right after this next pause for an ad break. We'll be right back.

[Advertisement/Announcer]

Mint is still $15 a month for premium wireless. And if you haven't made the switch yet, here are 15 reasons why you should.

[Dr. Margaret Mitchell]

1.

[Advertisement/Announcer]

It's $15 a month.

[Julia Longoria (Host)]

2.

[Advertisement/Announcer]

Seriously, it's $15 a month.

[Julia Longoria (Host)]

3.

[Advertisement/Announcer]

No big contracts.

[Julia Longoria (Host)]

4. I use it.

[Advertisement/Announcer]

5.

My mom uses it.

[Julia Longoria (Host)]

Are you. Are you playing me off?

[Advertisement/Announcer]

That's what's happening, right? Okay, give it a [e-mail] Switch upfront.

[Julia Longoria (Host)]

Payment of $45 for three month plan $15 per month equivalent required. New customer offer first three months only, then full price plan options available, taxes and fees extra.

[Advertisement/Announcer]

See mintmobile.com Where'd you go for the long weekend? If you spent most of your time online, you couldn't escape the rumors that the President was dead.

[Sigal Samuel]

Trump hasn't been seen in days.

[Julia Longoria (Host)]

The bruises on his hands. Vance said he's ready to take over. If some people if Trump's dead, they should bury JD Vance alive with him like they did with pharaohs and their cats. Imagine not believing Trump is dead just because there's no evidence. Where's your sense of whimsy and joie de vivre?

[Advertisement/Announcer]

By Sunday, Donald Trump himself had to weigh in.

[Julia Longoria (Host)]

Never felt better in my life. Also, DC is a crime free zone.

[Advertisement/Announcer]

I'm alive and I fixed DC on today. Explained from Vox. We're gonna assess that second claim because it's bold.

[Julia Longoria (Host)]

It is a literal statement that President Trump has freed 700,000 people in this city who are living under the rule of criminals and thugs.

[Advertisement/Announcer]

He literally liberated D.C. and he wants to do Chicago next. How are recent moves by the White House, from the attempted removal of a Federal Reserve governor to the government's equity stake in intel impacting the U.S. economy and institutions in the context of a.

[Julia Longoria (Host)]

Supreme Court that has vastly expanded presidential powers? That should alarm everybody.

[Advertisement/Announcer]

I'm Preet Bharara, and this week Nobel Prize winning economist Darren Acemoglu joins me on my podcast, Stay Tuned with Preet to discuss how institutional strength affects prosperity for individuals and society as a whole. The episode is out now. Search and follow Stay Tuned with Preet. Wherever you get your podcasts.

[Julia Longoria (Host)]

Could I ask you to introduce yourself? Sure.

[Sigal Samuel]

So I'm Sigal Samuel. I'm a senior reporter at Vox's Future Perfect.

[Julia Longoria (Host)]

I called my coworker Seagal about midway through my journey down the AI rabbit hole. How did you get interested in AI?

[Sigal Samuel]

So it's kind of funny. Before I was an AI reporter, I was a religion reporter. A few years ago, little bits and pieces started coming out about internment camps in China for Uyghur Muslims. And in the course of that, I started becoming really interested in and alarmed by how China is using AI.

[Julia Longoria (Host)]

Fascinating.

[Sigal Samuel]

Yeah, mass surveillance of the population, particularly of the Muslim population, was like coming from a place of being pretty anchored in freaky things that are not at all far off in the future or hypothetical, but that are very much happening in the here and now.

[Julia Longoria (Host)]

I was honestly thrilled to hear that Seagal, like me, came to AI as a bit of a normie sort of.

[Sigal Samuel]

Being thrust into the AI world. At first it was like pretty confusing because you have a variety.

[Julia Longoria (Host)]

I can highly relate to that feeling. But the longer she spent there in the world of AI, she started to get an uncanny feeling like, haven't I been here before?

[Sigal Samuel]

Have you ever noticed that the more you listen to Silicon Valley people talking about AI, the more you start to hear echoes of religion?

[Julia Longoria (Host)]

Yes, the religious vibes immediately stuck out to me. First, there's the talk from CEOs of building super intelligent God AI and they're.

[Sigal Samuel]

Going to build this artificial general intelligence that will guarantee us human salvation if it goes well, but it'll guarantee doom if it goes badly.

[Julia Longoria (Host)]

And another parallel to religion is the way different denominations have formed almost around beliefs in AI. Seagal encountered the same groups I did at the start of my journey.

[Sigal Samuel]

I started hearing about people Like Eliezer Yudkowski.

[Julia Longoria (Host)]

What do you want the world to know in terms of AI? Everyone will die. This is bad.

[Dr. Joy Buolamwini]

We should not do it.

[Julia Longoria (Host)]

Eliezer, whose blog convinced rationalists and people like Elon Musk that there could be a super intelligent AI that could cause an apocalypse. So our side of things is often referred to as AI Safety. We sometimes refer to it as AI not kill everyoneism. So there's the AI Safety people, and then there's a whole other group, the AI Ethics people.

[Sigal Samuel]

People like Margaret Mitchell.

[Dr. Margaret Mitchell]

We called it the everything is awesome problem.

[Dr. Joy Buolamwini]

Joy Buolamwini I wasn't just concerned about faces. I was concerned about the whole endeavor of deep learning.

[Sigal Samuel]

Timnit get brew People would be like.

[Dr. Margaret Mitchell]

You'Re talking about racism.

[Julia Longoria (Host)]

No thank you.

[Dr. Margaret Mitchell]

You can't publish that here.

[Julia Longoria (Host)]

These women did not talk about a super intelligent God, AI or an AI Apocalypse.

[Sigal Samuel]

Slowly, slowly. They kind of come to be known as like the AI ethics camp, as distinct from the AI safety camp, which is more the like. Eleazar Yudkowski A lot of us are based in the Bay Area. We're worried about existential risk, that kind of thing.

[Julia Longoria (Host)]

AI Safety and AI ethics.

[Sigal Samuel]

I don't know who came up with these terms. You know, it's just like Twitter vibes.

[Julia Longoria (Host)]

To me, these two groups of people seem to have a lot in common. It seemed like the apocalypse people hadn't yet fleshed out how exactly AI could cause catastrophe. And people like Margaret Mitchell, the AI ethics people, were just providing the plot points that lead us to apocalypse.

[Dr. Margaret Mitchell]

I could lay out how it would happen. Part of what got me into AI ethics was seeing that a system would think that massive explosions was beautiful. Right? That's like an existential threat. You have to actually work through how you get to the sort of horrible existential situations in order to figure out how you avoid them.

[Julia Longoria (Host)]

It seemed logical that AI ethicists like Margaret Mitchell and the AI Safety people would be natural allies to avoid catastrophic scenarios.

[Dr. Margaret Mitchell]

And how you avoid them is like listening to what the ethics people are saying. They're doing the right thing. We I, you know, I'm trying to do the right thing anyway.

[Julia Longoria (Host)]

But it quickly became clear that they are not allies.

[Sigal Samuel]

Yeah, there is beef between the AI Ethics camp and the AI safety camp.

[Julia Longoria (Host)]

My journey down the AI rabbit hole was full of the noise of infighting. The noise crescendoed when Elon Musk called for a pause in building large AI systems. It seems like warriors of all stripes could get behind a pause in building AI, but no AI Safety people and AI ethics people were all against was like a big Martin Luther 95 theses moment, if you will. Everyone felt the need to pen their own letter. Musk and others are asking developers to stop the training of AI systems so that safety protocols can be established. In his letter, Elon Musk's stated reason for wanting a pause was that AI systems were getting too good. He had left the ChatGPT company he helped create and decided to sue them publicly, saying that they had breached the founding agreement of Safety.

[Sigal Samuel]

The concern they have is that as you. Well, it's the concern, but it's also the exciting thing. The view is that, you know, as these large language models grow and become more sophisticated and complex and you start to see emergent properties. So yeah, at first it's just gobbling up a bunch of text off the Internet and predicting the next token and just like statistically trying to guess what comes next. And it doesn't really understand what's going on, but give it enough time and give it enough data and you start to see it doing things that like, make it seem like there's some higher level understanding going on. Like maybe there's some reasoning going on.

[Julia Longoria (Host)]

Like when ChatGPT seems like it's reasoning through an essay prompt, or when people talk to a robotherapist AI system and feel like it's really understanding their problems. The rate of change of technology is incredibly fast. It is outpacing our ability to understand it. Elon Musk's stated fear of AI seems to be rooted in rationalist fears based on the premise that these machines are beginning to understand us and they're getting smarter than us. We are losing the ability to understand them. What do you do with a situation like that? I'm not sure. You know.

[AI Ethics Researcher]

I hope they're nice.

[Julia Longoria (Host)]

Rationalist founder Eliezer Yudkowski shares this fear. But he wants to do more than just pause and hope they're nice. He penned his own letter, an op ed in Time magazine, responding to Elon Musk's call for a pause, saying it didn't go far enough. Eliezer didn't just want to pause. He wanted to stop all large AI experiments indefinitely, even in his own words, by airstrike on rogue AI labs. To him, the pause letter vastly understated the dangerous catastrophic power of AI. And then there's the AI ethicists. They also penned their own letter in response to the pause letter. But the ethicists wrote it for a different reason. It wasn't because they thought Elon Musk was understating the power of AI systems. They thought he was vastly overstating it.

[Advertisement/Announcer]

Welcome, everyone, to mystery AI Hype Theater 3000, where we seek catharsis in this age of AI Hype.

[Dr. Emily Bender]

I'm Emily Ann Bender, professor of linguistics at the University of Washington.

[Julia Longoria (Host)]

One of the people who responded to the pause was AI ethicist Dr. Emily Bender. She co hosts a podcast called Mystery AI Hype Theater 3000, which, as you might imagine, is about the overstated, hyped up risk of AI systems.

[Dr. Emily Bender]

And each time we think we've reached peak AI Hype, the summit of Bullshit Mountain, we discover there's worse to come.

[Julia Longoria (Host)]

The summit of Bullshit Mountain she keeps cresting. For her, it's the mountain of many, many claims that artificial intelligence systems are so smart they can understand us like the way humans understand, and maybe even more than that, like a God can understand.

[Dr. Emily Bender]

I found myself in interminable arguments with people online about how no, it doesn't understand.

[Julia Longoria (Host)]

So Emily Bender and a colleague decided to come up with something to try and help people sort this out. Something that AI safety folks and AI ethics folks both seem to be fond of, and that is a parable or a thought experiment. In Dr. Bender's thought experiment, the AI is not a paperclip maximizer. The AI is an octopus. Go with her on this.

[Dr. Emily Bender]

So the octopus thought experiment goes like this. You have two speakers of English. They are stranded on two separate nearby desert islands that happen to be connected by a telegraph cable.

[Julia Longoria (Host)]

Two people stranded on separate desert islands communicate with each other through the telegraph cable in morse code with dots and dashes. Then suddenly, a super intelligent octopus shows up.

[Dr. Emily Bender]

The octopus wraps its tentacle around that cable and it feels the dots and dashes going by.

[Julia Longoria (Host)]

It observes the the dots and dashes for a while, you might say, it trained itself on the dots and dashes.

[Dr. Emily Bender]

We posit this octopus to be mischievous as well.

[Julia Longoria (Host)]

I'm on the edge of my seat.

[Dr. Emily Bender]

So one day it cuts the cable. Maybe it uses a broken shell and devises a way to send dots and dashes of its own. So it receives the dots and dashes from one of the ink English speakers and it sends dots and dashes back. But of course, it has no idea what the English words are that those dots and dashes correspond to, much less what those English words mean. So this works for a while. The the English speakers.

[Julia Longoria (Host)]

At one point, one human says to the other via Morse code, what a lovely sunset.

[Dr. Emily Bender]

And the octopus, hyper intelligent, right, has kept track of all of the patterns so far. Far it sends back the dots and dashes that correspond to something like, yes, reminds me of lava lamps.

[Advertisement/Announcer]

Hmm.

[Julia Longoria (Host)]

The deep sea octopus does not know what a lava lamp is, but that's.

[Dr. Emily Bender]

The kind of thing that the other English speaker might have set back.

[Julia Longoria (Host)]

Not really sure why these castaways are waxing poetic about lava lamps in particular. But anyway, for our purposes, the octopus is like an AI, Even if it's super intelligent, whatever that means, it doesn't understand Dr. Bender's trying to say to Chatgpt, human words are just dots and dashes.

[Dr. Emily Bender]

And then finally, we end the story because it's a thought experiment when we can do things like this with a bear showing up on the island. And the English speaker says, help, I'm being chased by a bear. All I have is this stick. What should I do? And that's the point where if the speaker survives, they're surely going to know they're not actually talking to their friend from the other island. And we actually put that line in GPT 2. Help, I'm being chased by a bear. And we got out things like, you're not gonna get away with this.

[Julia Longoria (Host)]

Super helpful. Yeah. Wow. I gotta say, I'm into this one. The idea that AI systems only see human words as dots and dashes, I find that deeply comforting. Cause I don't know about you all, but for me, one of the scary things about AI is, is the idea that it could get better than me at my job. A fear that's very present when OpenAI is actively training its models on my work. Their system might understand my work, understand the things that make it good. When it's good, it might get good at doing what I do, and poof, I'm obsolete. There's also a recurring dream I have that various villains, including the Chinese government, for some reason clone my voice to deceive my loved ones. Anyway, if it's all just dots and dashes that these things understand, it seems clear we shouldn't be trusting these AI systems to be journalists or lawyers or doctors. It relates to what Dr. Margaret Mitchell and Dr. Joy Buolamwini found in their research. AI systems are only as good as the data they're trained on. They can't understand or truly create something new like humans can.

[Dr. Margaret Mitchell]

It's easy to sort of anthropomorphize these systems, but it's useful to recognize that these are probabilistic systems that repeat back what they have been exposed to, and then they parrot them back out again.

[Julia Longoria (Host)]

Another way to put it is AI systems are like parrots.

[Dr. Margaret Mitchell]

Parrots parrot, right. Famously, parrots are known for parroting.

[Julia Longoria (Host)]

If you hear your pet parrot say a curse word, you only have yourself to blame. Dr. Mitchell joined Dr. Bender in the response to Elon Musk's pause along with Dr. Timneet Gabru. They had all written the paper together that ended up getting Dr. Mitchell fired from Google. These ethicists wrote that they agreed with some of the recommendations Elon Musk and his paws posse had made. Like that we should watermark AI generated media to be able to distinguish synthetic from human generated stuff. Which sounds like a great idea to me. But they wrote the agreements they have are overshadowed by their distaste for fear mongering and AI hype. They wrote that the pause and fears of a super intelligent AI. What do you do with a situation like that? I'm not sure. You know.

[AI Ethics Researcher]

I hope they're nice to.

[Julia Longoria (Host)]

These AI ethics folks. It all reeked of AI hype.

[Dr. Emily Bender]

It makes no sense at all. And on top of that, it's an enormous distraction from the actual harms that are already being done in the name of AI.

[Julia Longoria (Host)]

This is the main beef that AI ethics people have with AI safety people. They say the fear of an AI apocalypse is a distraction from current day harps.

[Researcher at AI Ethics Conference]

Like, you know, look over there, terminate or don't look over here, racism.

[Julia Longoria (Host)]

You know, there are different groups of concerns. You have the concern at the AI ethics conference that producer Gabrielle Burbay attended. She mentioned the concern of an AI apocalypse. And then you have these concerns about more existential risks. And I'm curious what you make of that. You're going, no. Can I ask why? You're going, no.

[Dr. Margaret Mitchell]

No.

[Julia Longoria (Host)]

She's shaking her head and it felt almost taboo. A lot of hand wringing around that question. Eventually, one of the women spoke up. Share your perspectives. She talked about how she thinks the demographics of the groups play a role in the way they worry about different things. Most of them are like white male. AI safety folks are largely pale and male, to borrow Dr. Joy's line. They may not really understand discrimination that other people kind of go through in their day to day lives. And I think the social isolation from those problems makes it a bit harder to empathize with the actual challenges that people actually face every day. Her point was, it's easy for AI safety people to be distracted from the harms happening now because it's a blind spot for them. At the same time, AI safety people told me that AI ethics people have a blind spot. They're not worrying enough about apocalypse. But why would it be taboo to say all of this on mic? Part of the reason might be because the fear of apocalypse has come to overpower any other concern in the larger industry.

[Researcher at AI Ethics Conference]

One thing that I think is interesting is that a lot of the narrative that we hear about how AI is going to save the world and it's going to solve all these problems and it's amazing and going to change everything. And then we get the narratives about oh my gosh, it could destroy humanity in 10 years, often coming from the same people. I think think part of the reason for that is that either way, it makes AI seem more powerful than it certainly is right now. And you know, who knows when we're going to get to the humanity destroying stuff. But in the meantime, if it's that powerful, it's probably going to make a whole lot of money.

[Julia Longoria (Host)]

Building a super intelligent AI has become a multi billion dollar business business and the people running it are not ethicists. Just weeks before Elon Musk called for the pause, he had started a new AI company. Yeah, I guess it's kind of counterintuitive, right, to see this and you're like, wait, why would the people working on the technology who stand to profit from it want to pause? Right?

[Dr. Emily Bender]

I can't speak for them, but it benefits them to on the one hand, get everybody else to slow down while they're doing whatever they're doing.

[Julia Longoria (Host)]

Octopus thought experiment author Dr. Emily Bender again.

[Dr. Emily Bender]

But also it benefits them to market the technology as super powerful in that way. And it definitely benefits them to distract the policymakers from the harms that they are doing.

[Julia Longoria (Host)]

It'd be nice to think that billionaire Elon Musk was calling for an industry wide pause in building large AI systems for all the right reasons. A pause that never came to be. By the way, it's worth pointing out that when the billionaire took over Twitter and turned it into X, one of the first things he did was fire the ethics team. And even though Elon Musk says he left and sued the ChatGPT company OpenAI over safety concerns, company emails have surfaced that revealed the more likely reason he left is that he fought with folks internally to try and make the company for profit to better compete with Google. Ethicists are concerned they're outnumbered by the apocalypse people and they think a lot of those people are in it to maximize profit, not maximize safety. So how did we get here? Why? Why is the industry not focusing on AI harms today and focusing instead on the risk of AI apocalypse.

[Dr. Emily Bender]

There's an enormous amount of money that's been collected to fund this weird AI research.

[Julia Longoria (Host)]

Why do you think the resources are going to to those long term hyper intelligent AI concerns?

[Dr. Joy Buolamwini]

Because you have very powerful people who are posing it, people who control powerful companies and people with very deep pockets. And so money continues to talk.

[Dr. Margaret Mitchell]

It seems to be like funding for sort of like fanciful ideas, right? It's like almost. It's almost like a religion or something where it requires faith that good things will come without those good things being clearly specified.

[Julia Longoria (Host)]

People wanting to be told what to do by some abstract force that they can't interact with particularly well. It's not new. ChatGPT gives you authoritative answers. Erosions of autonomy. Like a God? Kind of. Yeah, like a God.

[Sigal Samuel]

It's like really interesting to take these things philosophies apart. I would argue they trace back to a large degree to religious thinking. But that might be another story for another day.

[Julia Longoria (Host)]

Next time on Good Robot. Good Robot was hosted by Julia Longoria.

[Dr. Emily Bender]

And produced by Gabrielle Burbet.

[Julia Longoria (Host)]

Sound design, mixing and original score by David Herman. Our fact checker is Kaitlyn Penzemuk. The show was edited by Katherine Wells and me, Diane Hodson. If you want to dig deeper into what you've heard, you can check out Dr. Joy Buolamini's book Unmasking AI or.

[Dr. Emily Bender]

Head to vox.com goodrobot to read more.

[Julia Longoria (Host)]

Future Perfect stories about the future of AI.

[Dr. Emily Bender]

Thanks for listening.

[Advertisement/Announcer]

Sat.