Dr. Joy Buolamwini on Unmasking AI: My Mission to Protect What Is Human in a World of Machines

Summary

Dare to Lead with Brené Brown

Episode: Dr. Joy Buolamwini on Unmasking AI: My Mission to Protect What Is Human in a World of Machines

Date: May 8, 2024
Host: Brené Brown | Guest: Dr. Joy Buolamwini

---

Overview

In this rich and illuminating episode, Brené Brown sits down with Dr. Joy Buolamwini—groundbreaking AI researcher, artist, poet, MIT graduate, and founder of the Algorithmic Justice League—to unravel the themes of Dr. Joy’s book, Unmasking AI: My Mission to Protect What Is Human in a World of Machines. The conversation explores how personal journey, art, activism, and rigorous research converge in the fight for algorithmic justice and human dignity in an era shaped by artificial intelligence. Dr. Joy’s blend of technical expertise, creativity, and advocacy provides both a warning and a blueprint for engaging ethically and empathetically with rapidly evolving technologies.

---

Major Themes & Structure

1. Dr. Joy’s Origin Story and Identity [04:05–22:33]
2. Discovering Bias in AI and Genesis of Her Research [22:40–31:52]
3. Translating Hard Science into Art and Advocacy [32:52–46:30]
4. The Evocative Audit & Poetry as Counter-Narrative [46:30–54:38]
5. Building Broader Tables in AI: Diversity, Lived Experience, and Ethics [35:32–54:38]
6. Creative Plagiarism and Regurgitative AI Debate [63:19–71:42]
7. Rapid Fire Personal and Leadership Insights [71:52–end]

---

Detailed Summary & Key Insights

1. Dr. Joy’s Origin Story and Identity

• Childhood and Education: Dr. Joy describes her cosmopolitan upbringing—born in Edmonton, Alberta while her father pursued a PhD; moving to Ghana to live with her grandmother; then relocating to Mississippi and Memphis [04:13]. Her family tree is rooted in academia and art, which shaped her as “a daughter of an artist and a scientist.”
• Adolescence & Georgia Tech: As a teenager, she was equal parts daredevil, athlete (notably a pole vaulter), coder, and Knowledge Bowl competitor:

  "I was just full of energy. Sometimes I would skip basketball practice to be at Knowledge Bowl sessions." [07:51]

• Entrepreneurial Spirit: From bartering web design for track uniforms to founding a hair care tech company after college, Dr. Joy never followed a straight path, always harnessing her technical skills for practical, personal, or social good [09:45].

2. Discovering Bias in AI and Genesis of Her Research

• Early Career and MIT: The journey took her from Georgia Tech to a Fulbright in Zambia, then to Oxford as a Rhodes Scholar, where she pivoted through African Studies, Learning & Technology, and eventually to MIT Media Lab [14:01–22:33]. MIT was always the dream (“the Future Factory”), and she reveled in the creative freedom there [20:27].
• The White Mask Moment: While building an art project using facial tracking, she discovered that the algorithm wouldn’t recognize her dark-skinned face, but it would recognize a face drawn on her hand or covered by a white mask.

  “…it wasn’t actually following my face… I held up my hand and drew a smiley face… and it detected not my dark-skinned face, but the face I had drawn on my palm. So after that I was like, yo, anything’s up for grabs." [25:24]
  This catalyzed her research into bias in commercial facial recognition systems, notably the "Gender Shades" study, which showed major tech products worked much better for lighter-skinned, male-coded faces [25:23–31:09].

• Corporate Responses: Mixed reactions from companies—IBM was proactive, Microsoft responsive, Amazon combative (“they actually had a corporate vice president attempt to discredit the research” [31:10]).

3. Translating Hard Science into Art and Advocacy

• Art, Science, and Identity: Dr. Joy speaks to the internal and external pressure to “orphan the art” within oneself to be taken seriously in tech [35:32–47:36]. She overcame this by bringing poetry and visual storytelling into her research communications:

  “For a really long time, I felt that to be taken seriously as a computer scientist, I had to hide the art part of myself. And so I struggled… to embody this notion of being a poet of code.” [35:32]

• Ain’t I a Woman?: Inspired by Sojourner Truth, Dr. Joy’s poem "AI Ain’t I a Woman?" exemplified her method of evocative audit—using art and powerful imagery to demonstrate the human cost of biased algorithms.
  Notable reading: [38:20–41:25]

  “Can machines ever see my queens as I view them? Can machines ever see our grandmothers as we knew them?” [39:20]

• Historic Roots: She describes how Black women scholars like Dr. Latanya Sweeney and Dr. Safiya Noble (Algorithms of Oppression) also rooted technical discoveries in lived experience, and how their legacy informed her research methods [43:30–46:30].

4. The Evocative Audit & Poetry as Counter-Narrative

• Evocative Audit: Dr. Joy developed the concept of the “evocative audit”—using art and poetry to make algorithmic bias felt, not just known. This strategy opened doors to rooms of power, sparked empathy, and changed minds in ways technical metrics alone could not [46:30–50:04].

  “Way when you were reciting your poetry, I didn't understand the bias. I felt the bias.” – Brené [46:30]

• Reach of Art: Her work resonated globally (“I presented AI Ain’t I a Woman… to EU defense ministers ahead of a conversation on lethal autonomous weapons” [49:12]) and helped demystify AI harms for activists and communities, such as Brooklyn tenants fighting facial recognition in their building [50:06].

5. Building Broader Tables in AI: Diversity, Lived Experience, and Ethics

• Who’s at the Table: Dr. Joy and Brené discuss the vital need for multidisciplinary teams in AI. Technical communities must welcome artists, ethicists, and people with lived experience:

  “The idea that... we can only have computer scientists and computational mathematicians at the table and not humanists and people with lived experiences... that's gotta end now.” [34:35]

• Ethics of Care: She argues for Black feminist thought and ethics of care as critical frameworks for meaningful AI research and advocacy [44:48].

6. Creative Plagiarism and Regurgitative AI (AI and the Theft of Art) [63:19–71:42]

• On AI and Intellectual Property: Dr. Joy addresses the controversial use of authors’ creative works to “train” generative AI models without consent:

  “We are really looking at what you could view as a heist. You didn’t know that your books were included in data sets used to train powerful generative AI systems.” [63:34]

• The Four Cs of Creative Rights: She campaigns for consent, compensation, control, and credit for artists and writers whose works are used to build AI [70:44].
• Regurgitative AI: She reframes “generative AI” as “regurgitative AI,” arguing these systems’ most impressive feats rely on uncredited, unpaid creative labor.

7. Personal & Leadership Insights – Rapid Fire [71:52–end]

• On Vulnerability:

  “Vulnerability is unmasking truth.” [71:57]

• On Leadership:

  “Leadership as the distribution of loss at a pace which people can handle… but it didn’t sit well with me... I find it useful in that it’s thought provoking.” [72:14]

• Poetry & Plumbing: Brené shares a quote: “Leadership is poetry and plumbing. You have to be able to cast a vision that inspires and builds systems that deliver.” [73:51]
• How Dr. Joy deals with fear: “I pray. I pray.” [74:11]
• Pop Culture Favorites: Binge-watching Bridgerton (with discussion on AI and representation), Sound of Music, a Nico concert (“Jericho” as her unofficial theme song for Unmasking AI) [74:20–78:28].
• Family & Gratitude: She shares a moving reflection on the legacy of her grandmother and a recent trip to Ghana after 30 years as a moment of gratitude and connection [80:35]:

  “I’m so deeply grateful for my grandmother… to see the legacy of everything that has come come through her in her children and in her grandchildren." [80:35]

---

Memorable Quotes

• On Art in Science:

  “I tell stories that make daughters of diasporas dream and sons of privilege pause.” – Dr. Joy [22:40]

• On AI’s Limitations:

  “For commercially sold products... they would overall work better on male labeled faces than female labeled faces. They would overall work better on lighter skinned faces than darker skinned faces.” – Dr. Joy [28:25]

• On Poetry as Audit:

  “I wanted to go from performance metrics to performance arts because that’s when I knew it would actually touch people so they could see what it means.” – Dr. Joy [43:48]

• On Creative Theft:

  “You don’t need to have any kind of tech background to know it’s not fair for companies to get billions of dollars of investment to then sell products that are made using your work and argue fair use.” – Dr. Joy [63:34]

• On the Role of Lived Experience:

  “Lived experience truly matters. And then I map it with the research that they did and then how my personal experience of coding in the white mask builds on that.” – Dr. Joy [44:10]

---

Timestamps for Key Segments

• Dr. Joy’s formative years and academic journey: [04:05–22:33]
• The “White Mask” discovery & origin of her research: [22:40–29:31]
• Corporate reactions to Gender Shades research: [30:08–31:52]
• On intersectionality, poetry, and “AI Ain’t I a Woman?”: [35:32–41:25]
• Discussion of evocative audits and institutional power: [46:30–54:38]
• AI copyright/creative theft and creative rights debate: [63:19–71:42]
• Rapid Fire (personal favorites, leadership wisdom, gratitude): [71:52–end]

---

Resources & References Mentioned

• Unmasking AI (Dr. Joy’s book)
• Gender Shades research and video
• AI Ain’t I a Woman? (poem and performance)
• Coded Bias (documentary)
• Algorithms of Oppression by Dr. Safiya Noble
• Weapons of Math Destruction by Dr. Cathy O’Neil
• Bridge to Algorithmic Justice League: AJL.org/writers for the Four Cs campaign

---

Final Reflections

This episode stands as an essential primer on the intertwined futures of technology, ethics, art, and humanity, delivered in Dr. Joy Buolamwini’s singular voice—a blend of incisive analysis, creative spirit, and unwavering advocacy. She invites listeners to join the fight for algorithmic justice, asserting:

"If you have a face, you have a place in that conversation." [82:07]

Transcript

[A]

Hi everyone, I'm Brene Brown and this is Dare to Lead. If it sounds like I've got a big grin on my face, it's because I have a huge smile on my face. If you do not know Dr. Joy Buolamwini, you are missing out. Let me just say that I'm just going to draw a line. She's one of the most amazing guests I've ever had the privilege to talk to. We are going to dig into her book Unmasking AI My mission to protect what is Human in a world of machines. She's got every tech bonafide you can imagine. Let me tell you a little bit about her. She is the founder of the Algorithmic Justice League, a groundbreaking MIT researcher and an artist. She is the author of the national bestseller Unmasking AI. She advises world leaders on preventing AI harms. Her research on facial recognition technologies transform the field of AI auditing and has been covered in over 40 countries. Her Gender Shades paper is one of the most cited peer reviewed AI ethics publication in the world. She's got an amazing TED Talk. We'll link to it on the page on brenebrown.com she's a Rhodes Scholar, a Fulbright Fellow, a recipient of the Technological Innovation Award from Martin Luther King Jr. Center. She's the 2024 winner of the NAACP Archewell Foundation Digital Civil Rights Award. She has a PhD from MIT, an honorary doctor of Fine Arts from Knox Colle. She's a poet, she's a coder, she is light, she is magic. And I cannot wait for y' all to hear this interview. Let's jump in. This message is brought to you by Apple Card. Each Apple product, like the iPhone, is thoughtfully designed by skilled designers. The Titanium Apple Card is no different. It's laser etched, has no numbers, and it earns you daily cash on everything you buy, including 3% back on everything at Apple. Apply for Apple Card on your iPhone in minutes, subject to credit approval. Apple Card is issued by Goldman Sachs Bank USA Salt Lake City Branch terms and more at applecard.com avoiding your unfinished home projects because you're not sure where to start. Thumbtack knows homes so you don't have to don't know the difference between matte paint, finish and satin or what that clunking sound from your dryer is with thumbtack. You don't have to be a home pro, you just have to hire one. You can hire top rated pros, see price estimates and read reviews all on the app. Download today. Welcome to dare to lead Dr. Joy.

[B]

Thank you so much for having me.

[A]

I totally stalked you in your DMs.

[B]

You did? DMs have been working out. That's how the COVID of the book got made. I actually reached out to the illustrator, sent her an Instagram DM and she just happened to have time. She'd seen the documentary Coded Bias, so she knew a little bit about the work, but it was a last minute request. So Instagram DMs for the win.

[A]

I wasn't even shy either. I'm like, hey, I love your work.

[B]

I was suspect. I was highly suspect. I said, all right, let's double check everything.

[A]

So before we get started, we always start with the same question. But I had to start with telling you, like how absolutely obsessed I am with your work. I think that I can get out of bed in the morning and face AI because there are people like you in the world asking really hard questions, holding people accountable. I'm just so grateful for you and your work.

[B]

Well, I'm grateful to be here and to have the opportunity to share more about it.

[A]

So let's start with your story. And I mean from Baby Joy.

[B]

Yes. So Baby Joy actually popped up in Canada, of all places. I was born in Edmonton, Alberta. I was there because of education. My dad was getting his PhD at the University of Alberta. I was born the year he defended, you know, so I really had no choice but to go down this academic path. And so I'm the daughter of a scientist and I'm also the daughter of an artist. And funny enough, my mom's dad, Dr. Danielle Juma Bedou, so I'm third generation when it comes to this academic tree, actually taught my father. Right. So there's this whole lineage. And a few weeks ago I visited Ghana for the first time in 30 years and. And I went to the university that he was dean of the School of Pharmacy for. And so it was very nice, full circle, because my mother would tell me all of these stories growing up and the places she described I finally got to see in person. And so I'm the daughter of an artist and a scientist. I was born in Canada. When I was about two years old, I moved to Kumasi, Ghana. And so that's the heart of the Ashanti region. And those are where my earliest memories are. I lived with my maternal grandmother. And then when I was about four years old, I was reunited with my parents. And I didn't know who these people were forcing me to eat vegetables, but I was told we were related and then we moved To Oxford, Mississippi. So my dad was teaching at Ole Miss. You can tell we were immigrants because they were calling it Ole Miss. So it was clearly not a. There are some outsiders at Ole Miss. Yeah, right. At Ole Miss, I learned their British English would not fly at school quite quickly. So definitely living in between cultures. Then when I was around 10, we moved to Memphis, Tennessee. So keep not Southern. And then went to undergraduate. I studied computer science at Georgia Tech in Atlanta, Georgia. So just despite the very cold start in Edmonton, Alberta, kept it pretty warm.

[A]

Wow. So Georgia Tech.

[B]

Yes. Yellow jacket. Rambling yellowjacket.

[A]

Yeah. Yes. We've done some dare to lead work at Georgia Tech. A couple of my favorite people are there. It's a really interesting school.

[B]

Absolutely.

[A]

What were you like and what did you love as a teenager?

[B]

As a teenager, what was I like? I was the kid who was skateboarding, who had my electric guitar and would make up songs and turn up my amp really loudly. And then I was also very athletic. So I was on all the sports teams, cross country, basketball. I did track and field. And my favorite event was pole vaulting. The risk taker in me was all for that. I used to skateboard and so that just the loud. Actually, no, the reason I got into pole vaulting was I was at track practice one day and we were doing push ups. The coach comes around, it's like, man, those are some really nice push ups. I'm like, yeah, I got some good push ups. But he was talking about the girl behind me, and I was like, what, you think her push ups are good? And so she was telling this girl that she should go try pole vaulting. So I was like, if you think she can pole vault, then I can definitely pole vault. The thing with pole vaulting is it's so much more than strength. You have to be a little bit of a daredevil. She didn't have that daredevil side. I definitely had that daredevil side. So they say it takes the speed of a sprinter, the strength of a thrower, and the body awareness of a gymnast. I had two out of three, so that was my thing. So I was just full of energy. Sometimes I would skip basketball practice to be at Knowledge bowl sessions. And the coach didn't know what to do with me because. Because it's like. It's Quiz bowl, you know, so it's nerdy, educational. So the coach, the Knowledge bowl coach and the basketball coach would be trying to say, you know, split time. I think I was probably more of an asset to the Knowledge bowl team. Than the basketball team, to be honest. Vertically challenged here, which. So I like pole vaulting. So yeah, me as a teenager, full of energy, exploring all kinds of things, building websites so that I wouldn't have to pay for my track uniform and my basketball stuff. So I was like, oh yes, I make some cash, I can barter with these tech skills. So a little entrepreneurial, just testing and experimenting and trying many things, showing up with all my AP bucks. I remember this. We would go to basketball games and in the locker room I'd have like AP Physics, AP Chemistry, Chemistry or whatever else. I would be doing that homework during halftime so I would have time to get the work done. So yeah, I think that's probably a snapshot of teenage me all over the place having too much fun.

[A]

Wow. I mean, you were engaged. Left brain, right brain, full body contact, you were all in.

[B]

Yes, full brain, full human.

[A]

Love it. Okay, Georgia Tech, what do you learn about yourself at Georgia Tech?

[B]

Oh, Georgia Tech was such a fascinating experience for me. I came in international affairs, biomedical engineering, then switched to computer science quite quickly, then switched to some other field, computational media, then switched back to computer science, and then along the way started a hair care technology company. I was doing all of these entrepreneurial projects and so I think the biggest lesson I got from Georgia Tech was finding ways to use my official schoolwork to explore what I wanted to do regardless. For example, my senior project ended up being experiment really with the Carter Center. And so for my Capstone project I was actually working with the Carter Center. We went to Ethiopia and we were exploring ways of transforming their paper based methods of assessing the effectiveness of their campaigns. So monitoring and evaluation. At that time, Android tablets were coming on the scene and because they were open source, I could actually program them. And so I was working with them to program and transform their paper based way of testing their campaigns for various neglected tropical diseases. So in this case trachoma, which is an infectious disease that can lead to blindness, but it was actually something there was medication for. So Pfizer had this drug called Zithromax that they were distributing, but they weren't actually so clear on how effective it was. So they had this Maltra program, malaria and trachoma, where they would go to different villages, distribute it and so forth. The problem with the paper based way of trying to assess it is there's a lot of data entry and that data transcription is not exact. And also when it came to the locations, trying to put in the GPS coordinates, not exact at all. And so being able to use The GPS on the Android tablets helped to make it a bit more precise. So for me, this was so exciting because I was motivated to get into computer science to try to use tech for good. And so at Georgia Tech I found that it was possible to translate these technical skills towards big problems in the world and make a meaningful contribution. So I think that was one of the biggest takeaways for me. Go, yellow jackets, go.

[A]

Yeah, I mean, I guess I'm in a state of shock because this seems like the progress of a 50 year old at the Gates foundation or something. So I'm so amazed that you're programming these new. And I'm doing the math in my head. So the Android tablets were relatively new?

[B]

Yes. This was around 2011, 2012, yes.

[A]

Okay, where do you go after Georgia Tech?

[B]

So after Georgia Tech, I take a detour and I do a hair care technology company send us strands of your hair. We analyze it, give you unique product recommendations. I'm having a good old time. I get my mom saying, wait, have you thought about applying for the Rhodes Scholarship? And like, Oxford sounds cold. How about the Fulbright fellowship? So I applied for the Fulbright fellowship and Fulbright fellowship, just like pole vaulting. I was hanging out with some friends one day and, and they were talking about what they were planning to do for the next year or so and one's like, oh yeah, I'm applying for the Fulbright. Oh, she applied. I could probably apply, right? I applied. I ended up getting it. She didn't, unfortunately, but she inspired me to apply. So shout out. And then as I was going to do that, my mom was still adamant about applying for the Rhodes Scholarship and I was still, it's cold, mom. But anyways, I applied for it and. And thankfully both came through. So I went to. It sounds like a AI generated story, but this is truth.

[A]

It does sound like an AI generated story.

[B]

Sound real. But that's what happened. So I was in Lusaka, Zambia, and inspired by some of the work I had done with the Carter center and in Ethiopia, I was saying, okay, I wanted to really think through the part that I was playing in trying to use tech for good, because in the situation of working with the Carter Center, I was literally the Westerner parachuting in a tech project. When I did that, I found out that the assumptions I made when I was programming this in my childhood bedroom in Memphis, Tennessee, didn't actually hold up to the real world conditions. And I was also wondering, well, why aren't the local people developing these systems? Why am I coming in with my Fulbright Fellowship in Lusaka, Zambia. The focus really was equipping Zambian youth to make mobile apps for things that they cared about. So we partnered with Ischool Zambia and we worked on these zegipads, which were Android tablets that had the entire Zambian school curriculum on there. And I got to train youth who. It was funny, they'd walk into class, they wouldn't ever think I was the instructor. So I would learn a lot before class started. Always undercover. And then I mentioned I used to skateboard. There were Zambian skateboarders there. They would get their parts from South Africa. And then we also found this poetry scene. So I was hanging out with the poet. It was a great time. Meanwhile, I was teaching people how to code. And so then when the Peace Corps folks would come in and they would see photos of my projects, they would always be confused because it's usually like some white guy and a bunch of black people out with the fishery. My photos were all of these dark skinned people programming apps. So they were like on skateboards. Right? Skateboarding. Having a good time. I have to send you some of the skateboard photos and videos. We had ways. Oh my God, we made music videos. It was a great time. So then after around seven months, it was time to go to Oxford. The second Oxford. Oxford, not Oxford, Mississippi. Yeah, Time for Oxford, England. And so I went there to start the adventure of being a Rhodes scholar. And I made the mistake of enrolling into African studies. After the first lecture, I knew I was in the wrong place. In the wrong place. It was just such a disconnect from the work I had done in Ethiopia, the work that I was doing in Lusaka. And they were taking more of an anthropology perspective, which is fine, just not where I was at the time. So I quickly switched to another discipline. So I switched to the education department, Learning and Technology, which built more on the work I had been doing teaching youth how to make mobile apps that they cared about. And so yes, just like when I went to Georgia Tech, now I'm like, okay, international relations, probably not me. Oxford, African studies, probably not me. So I got through the first year, it was cool. Then with the Rhodes scholarship, two years are funded and the program I did was a one year program. So I had applied to a master's, I think, in public health. Global health. Right. Because the work in Ethiopia, I had a store. It made sense. Yeah, right. Somewhere along the line I realized I really didn't want to do global health. They didn't like my jokes. It was like a very somber kind of culture. And I'm not that person. So I was like, maybe I should do something else. So I had an idea. So I proposed a year of service where one such scholar, hoping to use their year to make a meaningful impact and fight the world's fight, might, instead of doing a course, pursue some other idea. So I pretty much wrote a 40 page plea or proposal to describe this year of service, how I would do it, why it was impactful. And then basically I said I would do it one way or the other, so they might as well get the credit for it. And it's been 100 years, we haven't changed our way. They told me not to get my hopes up, but long story short, they allowed me to do it. And so my second year at Oxford was actually teaching Oxford youth how to code apps they cared about. And I used a very similar curriculum. Yeah, we actually did a first responders app focused on campus sexual violence. The limestone at Oxford was just about as impenetrable as anywhere else. So the low connectivity issues also transferred. There was a lot going on there, but so I had an opportunity to do that. And along the way I get a call from my dad. I mentioned the PhDs earlier. He's like, so what about the terminal degree? Think about graduate school. So I applied to just one place. One. If I. Look, I applied to one place, I've done what my dad wants me to, I've applied to graduate school. If I make it, great. If I don't, I keep on my entrepreneurial adventures. So I applied to MIT, I get in and then I'm off to MIT.

[A]

Dr. Joy's got this whole thing where it's like, hey, if those are your push ups, I can go to mit. If those are your push ups, I can be a Rhodes scholar, a Fulbright.

[B]

It wasn't on my radar, but I think I got this, though. I mean, to be fair, when I was little, MIT was the dream tech school because all of these documentaries I would watch, it was always the backdrop, such and such happening at mit. I didn't know there were requirements, but I was inspired.

[A]

She just thought it was going to be a great narrative documentary experience. But it turns out there's like prerequisites. That's terrible, right?

[B]

Things of that nature.

[A]

So you're in mit?

[B]

Yes.

[A]

And you love it.

[B]

Oh, my goodness. So I am finally at the Future Factory and I'm there when they're celebrating the 30th anniversary of the Media Lab. Martha Stewart is there, you have the magicians. They're talking about magic and mischief. And I was so excited. I mean, nerd and me could not have been more thrilled. And one of my professors was a Japanese pop star, the design one. There's this other lady making synthetic estrogen, and she had this project, Sad Bad Chickens. It was just like, I can't even make up. I can't make it up. I mean, I can't make it up. I can't make it up. So I'm thrilled. And also one of the friends who'd also been a Rhodes scholar with me, she was from Zimbabwe. She and her husband were at Harvard as resident tutors. And so they needed scholarship tutors. And having done a bunch of scholarships, I was like, oh, I could probably do that. So thankfully they accepted me as a resident tutor. So I was a resident tutor at Adam's house. So I was living in Harvard Square and then going to school at mit. So I had these two communities. So that was how I entered the Cambridge, Massachusetts experience. And now that I look back on it, I can appreciate and can be very grateful for the amount of privilege it was to be in both spaces simultaneously. At the time, I was like, yay, free housing and free toilet paper. Yo. Harvard's making these kids soft. What is going on? Right? Wow. They edited out the free toilet paper from the book. But I think the world should know. I really do. It might have changed since my time. This was in 2013, so things could have changed.

[A]

How would you describe where you are, what you do today, and how did you get here?

[B]

So I am a poet of code. I tell stories that make daughters of diasporas dream and sons of privilege. Pause. I am able to do AI research that shows discrimination in products as a way of actually preventing harmful deployment of AI systems. And the way I got to doing this work that really allows me to embrace the artist within while also having my research hat was a bit accidentally. When I finally made it to my dream school at the Future Factory MIT Media Lab, I took my first course. One of them was science fabrication. You read science fiction, you try to build something you might otherwise not do, as long as you can do it in six weeks. So I wanted a shapeshift, but the six week part made it a little bit challenging. So I was like, all right, not going to be able to change the laws of physics, but maybe I could change my reflection in a mirror. And so I learned about this special kind of material called half silvered glass. And when you put a black background behind it, it reflects just like a regular mirror. But if you shine Light through the light will go. And so I was like, oh, if I put something completely black and then have a mask on it, it can look like a filter. But instead of a filter on a camera, like you would have with Snapchat where you have the dog ear, I could actually make that effect on the mirror because of the material. So when I figured that out, I was like, oh, that's how I can shape shift in less than six weeks. So I started experimenting with that and I got it to work and then I thought, okay, right now it's like a fun. It's kind of like when you go to an amusement park and they have the cutout and you have to put your face in the right place. So that's fun. But I was like, ooh, what if it could actually follow my face in the mirror? So that's when I thought, oh, well, we have technology that can actually detect faces and track faces. So let me add some of that kind of technology into my project. So I went and I found the kind of software that's meant to do it and I integrated it into the project I was doing. I know how to a program. So I was getting the right code into my project. And then I was excited, I wanted to try it, but it wasn't actually following my face. So the cool effect I wanted to have wasn't happening. And so, you know, scientists I start to debug. So I was like, all right, let me see if it can get any face. So I held up my hand and I drew like basically a smiley face, right? Two eyes, a nose, and a mouth.

[A]

Just on the palm of your hand?

[B]

On the palm of my hand. I held that to the camera and it detected not my dark skinned face, but the face I had drawn on my palm. So after that I was like, yo, anything's up for grabs. So I look around my office and I happen to have a white mask because it was Halloween time and a friend had wanted to do a girls night out. So for the girls night out, they said bring masks. They actually meant beauty mask. And she had a bunch of Korean beauty masks. So I didn't need the mask anyways. But because of that, that's why I had a mask in my office. So thank you, Cindy. Right. So I have the mask. I put the. Actually before I even got the white mask all the way over my face, it was already being detected. So it was that moment of putting on a white mask over my dark skinned face to have it detected by a machine that I was like, oh, my Goodness, what's going on here? So that's what led me to start asking questions like, like how do machines see us in the first place? How are we even training machines to detect people? And also our AI systems as neutral as I assume they would be because it's math after all. So that's what really got me started. Down this long rabbit hole art project trying to see if I could shapeshift, seeing that the face tracking software wasn't detecting me until I put on a white mask and then really thinking about, oh my goodness, what does this mean? Maybe this is what I should research during my master's time at mit. And so that's what I did. So I had that white mask experience and I shared it in a TED Talk, TEDx Talk, and it got a lot of views. So I thought, you know what, people might question my claims. Let me check myself. So I took my TED profile image and I ran that headshot through online AI demos from a number of different tech companies. And some didn't detect my face at all, just kind of like the coating in the white mask. And the other ones that detected my face labeled me male. Like, wait, phenomenal woman here. What's happening? Why am I being labeled a male? And so that's what actually led me to focus my research on gender classification. How are machines even reading gender in the first place? And does it matter the skin type of the person behind the photo? So those were the types of explorations that led to my research. And so long story short, with my research, I tested AI systems from giants we know, right? So IBM, Microsoft, later on Amazon and for commercially sold products that they had, AI powered products, they were getting it wrong when it came to facial analysis, guessing the gender of a face. And so they would overall work better on male labeled faces than female labeled faces. They would overall work better on lighter skinned faces than darker skinned faces. And then when you did a deeper analysis and, and looked at it by skin type and gender, on some of them, there would be flawless performance. Like Microsoft was 100% accurate on lighter males, whereas they were closer to 80% on darker females. And those were the good numbers. Right. So then when we go to other companies and we pull it apart, I saw that in some cases the accuracy was around 68% for women with a darker skin. And mind you, at this point we've broken gender into a binary for just for the study. So you had a 50, 50 shot of just guessing it. Right?

[A]

The toss of a coin.

[B]

Exactly. And if we broke it down even further, by specific skin Type for Type 6 skin. Right. It was close to coin toss in terms of the accuracy. So it was. I was actually curious. I was like, I wonder if the companies know. So I sent them the results once the paper had been officially accepted to conference. Then I shared pre results with all of the companies and I shared all of the results, but I blacked out their competitors names.

[A]

Oh, interesting.

[B]

Right. And so I had different responses from the various companies over the two studies that we did. So. So one response was no response, another kind of.

[A]

Which is a response, yes.

[B]

And then the other, IBM was actually proactive. They said, oh, come, come to our office, show us what's happening. And part of that is because I had been at this Aspen Institute event and the head of AI research was there. And so the day I was turning in my MIT master's, I was walking from Harvard Square over to MIT and he was there with his two daughters and he introduced me to his daughters and I said, oh, there are findings in here you're going to want to know. And so he kind of already knew that this work was going on and I had met him. So I think that was part of why IBM was a bit more responsive. And they've played this game a lot longer than let's say Amazon.

[A]

For sure.

[B]

Amazon was combative to say the least. So they actually had a corporate vice president attempt to discredit the research and it got so bad we had a whole petition was put together by over, I think 70 researchers, including the former principal AI researcher of Amazon, saying that if you're attacking this research, it's actually going to impede important work in the field of AI because we have to know our limitations in order to reach our aspirations. Oh.

[A]

Go all in on fall with Abercrombie kids.

[B]

Their newest drop of on trend outfits are ready for everything from the bus stop to family bonfires.

[A]

And it wouldn't be fall without football.

[B]

Gear up.

[A]

The kids with officially licensed NFL team.

[B]

Tees and sweatshirts shop Abercrombie Kids this season in the app, online and in store.

[A]

This episode is brought to you by. Swiped a Hulu original from 20th Century Studios.

[B]

Meet the woman who made the first move.

[A]

Starring Lily James as Whitney Wolf, the visionary founder of Bumble. Through extraordinary grit and ingenuity, Whitney breaks into the male dominated tech industry and launches an innovative, globally lauded dating app. Forever changing dating culture. A Hulu original swiped streaming September 19th on Hulu and Hulu on Disney for bundle subscribers.

[B]

Terms apply.

[A]

When I watch the gender shades and we'll Link to this. If you're listening, you're going to want to see this. It's just, I mean the way you made it so accessible for me to understand. Now I will say there's a part after Gender Shades that follows the colon as all good academics have a snappy title followed by a really shitty serious subtitle. That's our training, right? So the full thing is Ginger Shades, which I love. But what comes after that in the actual title?

[B]

Intersectional Analysis of Commercial Gender Classifiers.

[A]

Hey, if she can do that kind of push up, she can call anything she wants.

[B]

It was actually funny because I went back and forth with my committee about the title. So that was my compromise. I just wanted it to be Gender shades, like for the book. I went back and forth with the publishers. I just wanted it to be Unmasking AI. They're like, that's not how it's done. Right.

[A]

We are a simple title in a complex subtitle world. I never want a subtitle either. But they're not having it it.

[B]

Exactly. So that was our compromise and there. Are you sure? Gender. I'm positive. Gender shades. This is so good. Can remember.

[A]

Yes.

[B]

The subtitle. No one's going to remember that.

[A]

So anyways, when I started my shame research, I was just like, can I just call it not good Enough? And they're like, we need a subtitle. And I was like, Variables Mitigating Self Conscious Affect. They're like, now you're talking. And I'm like, no, Now I'm losing 90% of the people who related to that feeling of not being good enough. So. Damn it.

[B]

Exactly.

[A]

Yeah.

[B]

But now you get to write books the way you want. So there you go.

[A]

I actually, I get to a little bit sometimes. Let me ask you this. I don't know if you know S. Craig Watkins, who works at UT on AI but also has spent some time at MIT and we had him on and he's been in AI for 20 years. And one of the things that he talked about that I learned and I'm curious on your thoughts about this too as you tell us about what surprised you the most is he just said the idea that when we're creating AI and building algorithms that we can only have computer scientists and computational mathematicians at the table and not humanists and people with lived experiences and liberal arts people and ethicists. He's like, that's gotta end now. And so tell me one what surprised you the most and do you agree about building longer, wider tables when we're developing AI?

[B]

Yes. I think what surprised me the most also answers that question. So for a really long time, I felt that to be taken seriously as a computer scientist, I had to hide the art part of myself. And so I struggled for a really long time to embody this notion of being a poet of code. So by the time I was doing my PhD, I asked myself, what would a poetic PhD look like? And I wasn't sure if I was willing to take that risk. But because I had done other academic degrees, I knew I could go through that process, right? So it was saying, how can I push myself further this time around? And when it came to the Gender Shades paper, which we've talked about, that research, you know, showing gender bias, showing skin type bias, and all of these major tech companies with lots of money and resources, I was struck by the fact that all of the companies failed so miserably on women like me, women with dark skin. And so I wanted to take a look into that part a bit more. And this is what allowed me to start exploring, using poetry as a way of sharing what was going on with AI systems. And so after Gender Shades came this poem that I now called AI Ain't I a Woman? Inspired by Sojourner Truth's 19th century speech in Akron, Ohio, really talking about for the need for women's rights to be intersectional and think through the lived experiences of other women, not just those who had the podium or the platform at the time. And so I remember waking up with these words in my head. My heart smiles as I bask in their legacies, knowing their lives have altered many destinies in her eyes I see my mother's poise in her face I glimpse my auntie's grace in this case of deja vu. A 19th century question comes into view in a time when Sojourner Truth asked, ain't I a woman? Today we pose this question to new powers, making bets on artificial intelligence hope towers. The Amazonians peek through windows blocking deep blues as faces increment scars, old burns, new urns, collecting data, chronicling our past, often forgetting to deal with gender, race and class. Again, I ask, ain't I a woman? Face by face, the answers seem uncertain. Young and old, proud icons are dismissed. Can machines ever see my queens as I view them? Can machines ever see our grandmothers as we knew them? And then as this was coming, I also started testing images of iconic women, historic women, women from contemporary times. So Ida B. Wells, data science pioneer, hanging facts, stacking stats on the lynching of humanity, teaching Truths hidden in data, each entry and omission a person worthy of respect. Shirley Chisholm, unbought and embossed, the first black congresswoman, but not the first to be misunderstood by machines well versed in data driven mistakes. Michelle Obama, unabashed and unafraid to wear her crown of history. Yet her crown seems a mystery to systems. Unsure of her hair, a wig, a bouffant, a toupee? Maybe not. Are there no words for our braids and our locks? Does sunny skin and relaxed hair make Oprah the First lady even for her face well known? Some algorithms fault her echoing sentiments that strong women are men. We laugh, celebrating the successes of our sisters with Serena Smiles. No label is worthy of our beauty. And so that was really my first way of saying, okay, let me let the artists take the stage. And in letting the artist take the stage, those words are also accompanied by visuals. So when I'm talking about Serena Smiles, you see IBM labeling the goat and her sister Venus right as men. When I'm talking about Ida B. Wells, data science pioneer, you see Microsoft describing her as a small boy wearing a hat. And so you're seeing these images. So for me, these started to become counter demos or counter narratives to that master narrative of the superiority of tech and tech companies. And it was this way that I realized I could allow people to bear witness to AI failure. So instead of the typical tech demo, which is about, oh, look at all of the things it can do, this kind of demo, which I then ended up calling an evocative audit, which was part of what my PhD became, was to say, how do we actually allow people to see what's at stake when you have a study like Gender Shade? So I wanted to go from performance metrics to performance arts because that's when I knew it would actually touch people so they could see what it means. When I'm asking, can machines ever see my queens as I view them, can machines ever see our grandmothers as we knew them? So to that question of who is and who isn't at the table and who's bringing their experiences or not, what I learned was the scholarship that came before me was from black women who were bringing in their lived experience. One was Dr. Latonya Sweeney. She was one of my PhD committee members and actually the first woman to get a computer science PhD from MIT. And some of the work she did, it came through accidentally, almost like my white mask experience. She was being interviewed by a reporter and she was trying to look up one of her papers. So she searched it in, I want to say it was Google. And what came up wasn't just her search for the paper, but also advertisements indicating she might have been arrested. And so the reporter was saying, wait, wait, wait, hold on, hold on. Your paper was interesting, but what about the supposed arrest record? And she was saying, wait, what? I've, that's, I've never been arrested, so what's going on here? And then the reporter was saying, it's because you have one of those black sounding names, Latonya. And she's like, look, I'm a computer scientist. This seems far fetched. So she set up this experiment trying to prove this guy wrong, where she had a list of names that were more likely to be assigned to black babies than to white babies. And they did the comparison. And in fact it was the case, right, that. But the search engine with the ads that would come up would more likely put up a false arrest record for people with black sounding names. And that work came before my work, but that was showing bias in search engines. And then I was looking at the work of Sophia Noble, also great researcher, Dr. Noble, author of Algorithms of Oppression, MacArthur genius award winner. So much respect for her work and her work. She was looking for activities for her, her daughter to do. I think there was a sleepover happening. Kids were around middle school, so she searched black girls. What she pulled up instead of activities for black girls were pornographic images. More often than not, similar thing if you put in Asian girls, white girls, a little more innocent content. And so that also sent her down this exploration that then led to the book Algorithms of oppression, the MacArthur Fellowship and all of that. And so I say this to really point to the legacy of black women in the tech space who have used their own lived experience, combined with their academic training, combined with their technical expertise to, to open up new schools of scholarship. And so in my PhD, I make this argument about the importance of black feminist thought and then ethics of care and going beyond positivism when it comes to the way in which we do research and saying lived experience truly matters. And then I map it with the research that they did and then how my personal experience of coding in the white mask builds on that. Then I go back to Sojourner Truth and I go back to Frederick Douglass and I show how they were using daguerreotypes and the imaging, the advanced technology of their era. So now how we have aio, everyone's excited there. To have a photo of somebody was incredible, but those kinds of images were being used to support scientific racism. To say look at the shape of this head. Clearly an inferior specimen of a human, this sort of thing. So they actually took that same technology and they had dignified portraits of themselves. So in AI Anti Woman, I intentionally use one of sirjourner Truth's images taken like that. She said the shadow supports the substance because she would sell these images. And that's the image that I show Google labeling her as a clean shaven old gentleman. So there are all of these historic connections. And so as somebody who's a researcher and an artist, I do oftentimes, and I did, especially earlier in my career, feel like I wouldn't be respected as much if I talked about my lived experience or if I let the art aspect come through, it would be viewed as too subjective and thus disqualify the data that I had gathered. And so in conceptualizing this notion of the evocative audit, this was actually saying, no, this is also a valid form of knowledge. And in fact, it builds on black feminist epistemology, you know. And so this is all within my MIT PhD alongside the algorithmic audits of Amazon and others. So I was saying, you can put the algorithmic audit with the evocative audit, and you can actually reach more people that way.

[A]

Way when you were reciting your poetry, I didn't understand the bias. I felt the bias. It takes me on a trail from Frederick Douglass to someone we both know, Sarah Lewis at Harvard, to the power of art and imagery, aesthetic force.

[B]

Absolutely.

[A]

And when you think of the evocative audit, I've mentored so many female PhD students, Latina, black, white, and the compelling need to orphan the art in them in themselves, to be seen to feel like a place is earned at the table when that is their power. Do you think that's changed at all, Dr. Joy? I mean, do you think. Or do you think there's a season for it in a career? Like, what do you make of that? Because I still see it with the PhD students.

[B]

I think for me, there's a reason I started with the tech, right? So by the time I'm doing AI, ain't I a woman? I have three degrees, technical degrees, learning and technology. Like, in some ways, I felt that I had, quote, unquote, paid my dues. And I think that's not necessarily a good thing. That was the armor I felt I needed to have to let the Turtle artists come out. And what I found so fascinating with AI Anti Woman was how far it traveled. I think I was most surprised at some point. I had, through one of my mentors Been recommended to be on the EU Global Tech panel. So president of Microsoft, others. And I remember being in Belgium and showing AI anti woman for the first time on this EU Global Tech panel. And just the response to it because it's so visceral. And the only other dark faces in that room were just the faces in the video and mine. And even before showing it, I almost didn't make it into that room because the human gate checkers were asking me what business that I have, you know, at this high level convening. And I mean, I. I'm five two, I barely look 28 at the time, you know, and so it was the thinking about how the physical gatekeepers become algorithmic gatekeepers.

[A]

Yes.

[B]

And then looking at who has again and a seat at the table. So the head of the World Economic Forum was there and he saw it. And then later I was invited to Davos. And so I presented AI entire woman in that space. And later it was shown to EU defense ministers ahead of a conversation on lethal autonomous weapons. And again, this is what I mean by how far it could travel. Because at the end of the day, the storytelling connects all of us. And that's what the poetry is doing. That's what the evocative audit is doing. Alongside these photos, you can't unsee, these are the biggest, most powerful tech companies, right. And here are iconic faces of women that I respect so much. And I mean, if you're getting Oprah's face. When I showed Amazon misgendering Oprah.

[A]

Yeah, I mean, yeah, it's not like.

[B]

She doesn't have photos available, you know, kind of thing. So the other thing I found with the evocative audit was being able to show and not just tell helped us get to the conversation. So instead of like, oh, I'm not sure if this is really a problem or this, that or the other. It's almost the mic drop, white mass mic drop, anti woman, mic drop, gender shades. Okay, let's get to the conversation. This is real. This exists. You're not going to gaslight me into saying it's not a thing. But also, more importantly, it invited other people into the conversation. So I spent so much time trying to make the gender shades results as accessible as possible. So taking the time to do the video, the interactive website. So later on, and you see this in the film, the documentary Coded Bias, when I go to the Brooklyn tenants and I'm working with people there, mainly women of color who've had the landlord install a facial recognition entry system. And so they're saying it feels like Fort Knox just to get into their house. They were saying thank you for the website because it made the information so clear where you don't need to have a PhD from MIT to know that this is wrong, right? And so that was a thing that I kept struggling with even when I was thinking about do I really want to finish the PhD and so forth. The work that I was doing, the what I delivered as a poet that was already there without the MIT degree. And so it's not what enabled the poetry, but I was also very aware of the use of institutional power and the use of narrative power. And so I write all of this in my PhD. The ways in which I'm using credentials for institutional power to get into the halls of congress, to get into eu, Global Tech Panel, Davos, Other spaces. But alongside that, I'm bringing the narrative of power right to say these are the marginalized, these are the X coded, and they matter and they must be centered in these conversations.

[A]

Adjective used to describe an individual whose spirit is unyielding, unconstrained, one who navigates life on their own terms, effortlessly. They do not always show up on time, but when they arrive, you notice an individual confident in their contradictions. They know the rules but behave as if they do not exist. The new fragrance by Miu Miu defined.

[B]

By you Lowe's knows how to help make your home holiday ready for less. Get select style selections vinyl flooring for just a doll per square foot and have it installed before the festivities begin. Our team can help you every step of the way. See a Lowe's Red Vest Associate or visit lowe's.comholidayinstall to get started. Lowe's we help you save basic install only date restrictions applied subject to availability installed by independent contractors. See Associate for details. Contiguous us only.

[A]

I just can't aptly tell you how grateful I am for your work and your ability to not just do the work, but name the process of how the work is coming to you and getting to me. I know quantitative hard, like what I would call not nicely, but hardcore quantoids would claim you as their own. But I'm like, no, she's one of us. She's a qualitative person. She's an ethnographer. Qualitative person for sure. And then I was talking to a friend.

[B]

She's like.

[A]

Like Dr. Joy. She's quantitative to the bone. And what I think is so interesting is that the answer is yes. You are both qualitative and quantitative. You are institutional and narrative. You Are poetry and code, daughter of art and science. A daughter of art and science and a pole vaulter.

[B]

Only to show I'm stronger.

[A]

I mean. But I think your pole vaulting is not just physical pole vaulting. I think you take a really hard, fast run at things and fly right over them and bring us along. I could work the pole vaulting metaphor for your work until you were in tears and begging me to stop.

[B]

I mean, I tried. In the book, I talked about the difference between bar gazing and stargazing. And. And I did snowboarding. The athlete in me said, just a little bit here. Just a little bit.

[A]

I picked it up. I picked up the crumbs you were laying down as an athlete. Yeah, I get it.

[B]

I try to leave something in there for everybody. For sure.

[A]

Let me ask you this. How do you envision the future of AI in our daily lives with respect to human dignity and freedom?

[B]

I have a poem about this, if you permit me to share it.

[A]

Oh, my God, yes. I would love it. It would mean everything. Yes.

[B]

Okay, so it's in the section about the Brooklyn tenets. Let me see. So, intrepid poet, I think it's safe now that I have all this academic tech cred armor. You're in the door. So, like, all right, let's.

[A]

You are the door.

[B]

So then I had the opportunity to work with the Brooklyn tenants, and it was at a time during my research where after research had been attacked, and I felt that the people who were closest to me academically didn't stand up for the research in the way I had hoped at that time. I was just like, is this really where I want to be? And when I got the opportunity to work with the Brooklyn tenants who were resisting the installation of a facial recognition entry system as the way to access their homes, I was so inspired by them, and I was like, oh, I'm not doing this for my committee or for my supervisor or for the institution. I'm doing it for the X coded. I'm doing it for people like the Brooklyn tenants. And so this is the poem I wrote for them. To the Brooklyn tenants, resisting and revealing the lies that we must accept. The surrender of our faces, the harvesting of our data, the plunder of our traces. We celebrate your courage. No silence, no consent. You show the path to algorithmic justice requires a league, a sisterhood, a neighborhood hallway gathering, Sharpies and posters, coalitions, petitions, testimonies, legacy research and podcast dancing and music. Everyone playing a role to orchestrate change to the Brooklyn tenants and freedom fighters around the world persisting and prevailing against Algorithms of Oppression. Automating Inequality through Weapons of Math Destruction. We stand with you in gratitude. You demonstrate the people have a voice and a choice. When defiant melodies harmonize to elevate human life, dignity and rights, the victory is ours.

[A]

God, oh God, it's so good.

[B]

And that's truly how I felt when I saw. Oh, my goodness, these aren't people with PhDs from MIT. These are people who saw problems, educated themselves about it, reached out and resisted and resisted successfully. And for me, that was such an inspiring model to understand that the research I was doing could equip people on the front lines. And that also people seeing themselves identified in me in various ways. People know how, whether you're the outsider because you're a woman or you're an immigrant, or you're more of the artsy person in a very technical whatever it might be. And seeing that inspiring others, that gave me the motivation I needed to continue.

[A]

Weapons of Math Destruction.

[B]

Yes. Book by Kathy o'. Neill. So that whole part is a bibliography. Can I nerd out a little bit?

[A]

Yes. I mean, yes, I know.

[B]

Go, go, go. Algorithms of oppression. Already mentioned Dr. Sophia Noble, OG algorithms of inequality. Right. Automating Inequality. That's a book by Virginia Eubanks. Through Weapons of Math Destruction, that's a book by Dr. Cathy O'. Neill. So that whole thing is a bibliography.

[A]

Yeah, I mean, it is literally a bibliography. It is literally a shout out.

[B]

Yes. For those who know, you know, so all these gems dropped out throughout for sure. And then also going back to Weapons of Math Destruction when I was, I wasn't sure I wanted to do this research in the first place because, you know, calling out big tech could be dangerous. Right.

[A]

Yeah.

[B]

I was on the fence and I had a mentor of mine tell me that Kathy o' Neill was coming to do a book talk and she was doing a book talk literally on the same block I live on. So I went to that book talk. It was at Harvard Bookstore. And I saw this, this blue haired, blue eyed mathematician, you know, talking about how machines can be harmful and problematic. And until then I was, I felt I was a little alone in these observations. And so I remember introducing myself and telling her about computer vision. She's kind of following what I was saying. But for me, I was like, oh, I'm not alone, right. If I'm going to be a gadfly, I might have an ally. I was like, okay, all right, we can do this. Weapons of Math Destruction was a really inspirational book for me. And now Kathy o' Neill. Later in the book, I talk about how we team up to do an algorithmic audit for Olay. She knows nothing about skincare. As a Olay ambassador, however, I am now better. Better versed on skincare. That super serum is great. Clinically proven, you know, but we had so much fun together. And then we're both musicians. Kathy's an amazing musician. She plays all the things, but she also plays fiddle. So from time to time, I'll go. Her mom's also an amazing mathematician, so she'll invite me over. Her mom will be solving the hardest math puzzles you've ever seen, correcting Kathy on the solution. I just love this. Love. I love everything about this, you know? And then she wrote this data science book from way back in the day. O' Neill and o', Neill, because both of her parents were mathematicians, and then she was a computer scientist. And she's. I guess they call us data scientists now. She's just the best. So you have these incredibly brilliant women. We're spending. I have the opportunity to spend time with them, eat with them, chill with her sons, and then we make music together. So that's from that first time of seeing her at Harvard Bookstore, being so inspired to now being able to call her a friend and drop her references in some of my poetry.

[A]

It's beautiful. It's just. It's beautiful. You must be in the minds of big tech, the scariest poet on the planet.

[B]

I'm just a poet. What could a poet do?

[A]

Oh, really? Do we. Do we want to get into the wars, the heartbreak, the murders caused by poets throughout time? I mean, I never underestimate the power of a poet. Come on.

[B]

No. That's why they burn us.

[A]

Okay, I have one question before we get to our rapid fire. I get a lot of texts and emails from friends of mine who are writers, fiction and nonfiction that our books have been fed into the machines. I don't know exactly how that works, but our. But I know my books have been fed. We just kind of kick back. And that's good. Or what is our thinking about that? It doesn't feel good, but I don't know. I'm asking for both the mathematician, entrepreneur and poet in you to braid an answer together. That's heart commerce and IP protection. And let me know what you think.

[B]

Sure. As a new member of the Authors Guild and also the national association for Voice Actors, this topic came close, very close to home. Once I became a published author and I started looking at the ways in which the work of so many authors had been to train generative AI systems. And this is why I call them regurgitative AI systems, because oftentimes the best of what we see in AI that's impressive, right, is because it's been fine tuned on some of the best of what's been made by human creators, be they visual artists, authors, filmmakers and so forth. And so for me, we are really looking at what you could view as a heist. You didn't know that your books were included in data sets used to train powerful generative AI systems. And you don't need to have any kind of tech background to know it's not fair for companies to get billions of dollars of investment to then sell products that are made, made using your work and argue fair use when it's copyrighted, and also when some of those outputs can almost be whole, entire paragraphs or samplings of that work. And furthermore, now you have the ability for some of these systems to say, write this in the style of, let's say, yes, let's make it a brown style, or let's make it a Bullowini style, or let's do it in the style of, of an artist. And so you're seeing right now lawsuit after lawsuit, right, because of the ways in which AI systems that are called generative are truly regurgitative systems. And they're regurgitating the hard work and the IP of artists from all around the world. And we also have evidence of this. And so the books, data sets that have been used to train some of these systems, we have the titles, we have the authors, so we can say, yes, actually this author's work was used to train this particular system. And no, it's not fair. And so this is part of why we launched this campaign with the algorithmic Justice League. So that's my day job. I like poetry, I like writing and things like that, almost the president and artist in chief of the algorithmic Justice League. And we really fight for the X coded people who have been excluded or exploited by AI systems or AI companies. And in this case, when we're looking at the future of the creative class, we have to push back and demand creative rights. And so what do creative rights look like? Well, first of all, consent. Hello, Ask basics. We talking basic, you know, so it shouldn't be after the fact that you find out, oh, my book was used to train, or my books in your case, right, was used to train these systems compensation. And so it's not just acknowledging that the books were there, it's making sure that there is fair and adequate compensation for those whose work has been used in this way. I think that would be a form of redress. And then there's also this notion of control. And this part I find really fascinating because this is what I was grappling with when I recorded the audiobook for unmasking AI. I'm busy. You're busy. They said it'd be four days in the studio. Actually, the same studio I'm in. I was like, maybe this is an opportunity for AI voice clone, Right?

[A]

Yeah.

[B]

And I thought that might be an option because I didn't understand that voice acting is an art. And it wasn't until I was working with the director who's won, like, the Oscar version for voice acting and all of that, that she really helped me appreciate it. I thought I was going to be reading aloud. No, I was performing. And by the time I would get home every day at around 5, I was out. By 6, I was done. And it was that process that just made me see how easy it is to underestimate the art forms that you don't call your own.

[A]

Oh, my God. I mean, do you know that I can only do three and a half to four hours a day of that? It is so difficult to do that.

[B]

Well, absolutely. And then also, the voice can only go so far. I made the mistake of having. I think. I don't. I had dairy. And because of that, my voice was cracking. So I was also learning different things. Just about being a biological being. Yeah, right. That made it. I also had an attitude, you know? No, I had an attitude. So there would be. There are some parts of the book where I'm talking about insults. I think I was reading through the TED insults. So I was struggling each time I was reading it. Right. And I. Director was so patient with me. She's like, can you. I'm gonna repeat it. Just give me a second. Oh, wait, no. I wish I would have been in.

[A]

The recording thing next to you. Cause we could have had. Yeah. Cause I mean, I literally would start talking, and then they'd be like, you have to do that again. I'm like, mfer. That's the hardest story in my adult life. I am not saying that again. And they're like, no, you gotta go again. I'm like, mother trucker.

[B]

Right. Or you'd get close, but you'd, like, transpose one word. I also learned I can't correctly pronounce many things, but I will do the English as a second language card.

[A]

Okay, okay. Oxford.

[B]

Oxford times two out of excuses. Here no, but that was a whole fascinating experiment for me. I actually wrote this whole op ed. It's not yet published, but it was just about how easy it is to underestimate the art forms that aren't your own.

[A]

That's it.

[B]

And so the control piece. Right. If I. I have. Now that I know voice acting is a lot more than I thought it was, I would never actually want to use an AI voice clone for that type of work. But there might be other type of work where you decide you have the agency to say, hey, I want to experiment with AI in this way. Or maybe I'm using AI to do a thematic analysis of everything I've written, written on this particular topic. And so using it as a tool to know, boost collaboration or something in that nature. You should have the control to do that. So that's that third C, and that fourth C is credit. Again, this is why I keep calling it regurgitative AI. Let's credit the artists who make what seems breathtaking from AI possible.

[A]

I'm just writing down these Cs because they're very important to me.

[B]

AJL.org writers, you'll see the four Cs now there we have the four C campaign. So AJL.org writers, I'll put that up.

[A]

Y' all on the brenebrown.com so you can find it. But I think consent, compensation, control, and credit is the entire way I've been brought up about work, period. As an academic, this is really important work. We'll link everything to it. I'm going to sign up or whatever you do over there because I support this movement. It's so easy to underestimate the amount of work and art that you don't create. And that's like when you're walking through, like, the Louvre, people ahead of you are like, oh, I could have done that. I'm like, dude, then where's your show? It. Show me your art. I'd like to see it too.

[B]

You can regurgitate it when you hear the line, when you see the final creation, but you cannot reproduce the process of creation as it went through the lived experience of the artist who gifted you that work.

[A]

And there we end, and we go right into rapid fire. Because that was beautiful. Okay, are you ready?

[B]

I'm ready.

[A]

Fill in the blank for me. Vulnerability is.

[B]

Unmasking truth.

[A]

I can't cry after each of these. Get your shit together, Brene. Okay, One piece of leadership advice that you've been given that it's so great. You need to tell us about it are so shitty. You need to warn.

[B]

This one is probably on the warning side. I was in an executive education course, and one of the professors described leadership as the distribution of loss at a pace which people can handle.

[A]

Oh, my God.

[B]

And I could see the perspective coming into a new organization as an outsider trying to bring in change. So they were talking about how often it happens when somebody is brought and they try to change things too quickly without really understanding the stakeholders or the motivations or what people will perceive as loss with that new thing they're bringing. Right. So I got it from that standpoint, but it didn't sit well with me from. You know, I'm a storyteller, Right.

[A]

Yeah.

[B]

Ooh. Do we start with disincentives or do we paint a compelling vision? Right. So that one, I still. I find it useful in that it's thought provoking. Okay. What's driving that particular perspective?

[A]

Yes.

[B]

And in which context does it make sense? And then I also found that it reaffirmed my approach, which is more joyful, as you might imagine. Yeah.

[A]

You'll appreciate this. One of my favorite leadership quotes, and I think you'll appreciate it because it's the paradoxical piece. It's from John. John March, who was at Stanford. He said, leadership is poetry and plumbing. You have to be able to cast a vision that inspires and builds systems that deliver. Ooh, isn't that good?

[B]

Poetry and plumbing.

[A]

Yeah, I kind of liked it.

[B]

Yeah, I'm gonna have to write that one down.

[A]

We'll send it to you.

[B]

Thank you.

[A]

Okay, you ready? You, Dr. Joy, are called to be very brave, but your fear is real. You can feel it in the back of your throat. You can taste the fear. What's the very first thing you do?

[B]

I pray. I pray.

[A]

Last TV show that you binged and loved.

[B]

Ooh, Bridgerton. Oh, and also, I got to meet aa, who is. I was at the NAACP Awards. I was there for my situation, and people were like, Dr. Jordan. I was like, look, it's my fangirl moment. I got. I gotta go meet some queens over here. And so season three is coming out, but they were there for Queen Charlotte. Oh, my gosh. So I binged Bridgerton. Then, of course, I had to watch all of Queen Charlotte as well. So, yeah, definitely excited for the new season to come out. I also watch a bunch of nature shows, but I don't binge them the way that Bridgerton. I gotta know what's next.

[A]

Queen. She. Charlotte.

[B]

Also really interesting with Bridgerton. When Gemini came out with Google's Gemini AI prompt system. So text to image. I was describing it like Bridgerton, because you would put in something like the Founding Fathers and they were showing black men as founding fathers in the US to which there was a public outcry. This is not an accurate representation that they put in Pope and there was a Native American woman as Pope. What is going on with this kind of representation? I was like, okay, so this is almost Hamilton Bridgerton.

[A]

I mean, Shonda Rhimes got a hold of Gemini in that way.

[B]

But what I found fascinating was people who were used to being centered were marginalized. And so the experience of this kind of reverse symbolic annihilation. And then also how quickly Google shut it down, they did. They took it offline and people were calling it Woke AI These were actually evocative audits. These were examples of real world evocative audits. But when I started reading and seeing the examples, or for example, they did Nazis who were, I think. I don't know if they had them as Asians in their representation. I'll need to double check. Check. Okay, so these were not historically accurate. So it led to another kind of conversation about representation, particularly when you're talking about the past. And then it made me think, because so often AI is described as being a mirror. It's like, okay, it's biased, but we're also biased. But oftentimes what I see is AI is a kaleidoscope of distortion. And that's a bit of what we are seeing with Gemini as well. But I go on and on.

[A]

I love it. Okay, so I might have to send you a DM when we're in the middle of the new Bridgerton series, because I'm into it. I'm such a fan. Okay. Favorite movie?

[B]

Favorite movie. This one is hard. I'll say the Sound of Music because that's partially how I learned English. I would watch it almost every day when I was four and it was my on ramp to the English language. So Sound of Music and my brother would come home. Sound of Music again.

[A]

I do love the Sound of Music. A concert that you'll never forget.

[B]

Ooh, I recently went to a Nico concert. She does this song called Jericho, which has become kind of my theme song because she has this part. I'm your future, past and present. I'm a fine line. I'm the missing link of this illusion. But she's talking about artificial intelligence in many different sorts of ways. Ways. It's just an incredible, ethereal sounding song. And I first heard it Acapella was this TikTok viral, a sensation. And then I heard the full piece and I thought, oh, my goodness, this is literally the theme song for unmasking AI. So when she came to Cambridge, I got the meet and greet. I signed a book for her just to let her know how excited I was about the kind of music she's making. So that was an amazing, amazing concert to attend and then to share that experience with friends as well. So at the Sinclair.

[A]

I love that. And we'll link to the song. Favorite meal.

[B]

Favorite meal. Ooh.

[A]

Wow.

[B]

I just went to Ghana for the first time in 30 years and I was well fed by so many different family members. But my mother's older sister, sister, she makes stews just the way my mom makes stews. And I hadn't tasted her cooking since I was maybe three. And so what I appreciated about that experience was how familiar it was. And then when I was in Ghana, I have so many cousins and so many people who look like me and my brother. And we're just. Because growing up in the U.S. we were the four of us, you know, our little unit. And I would always come kind of feel a bit of a distance when people talk about all their cousins. They're like, oh, I wish I had cousins close by. Or aunties and uncles. And I do. And they're there, you know, so we had an amazing time together. But Mills just. Or sitting next to my grandmother. Okay. So when my grandmother and I did the thing where you hold the unmasking AI book, her facial features match this so perfectly. It's uncanny. I was like, oh, my goodness. And also the way her hands were. You could. You saw her hands were like mask. Yeah. It was like. So it was this. Her hands were there and then it's her face and it's this perfect match of the face on the book. And I'm thinking, wow, the powerful is there. Yes. Favorite ville.

[A]

Then eating stew that tastes like home. I mean.

[B]

Yes. Yes.

[A]

Yeah. What is a snapshot of an ordinary moment in your life that gives you real joy?

[B]

Probably sitting on my couch, playing my guitar, you know, making up some song. I love words. And playing around. So making up some random song about something that's happened. Yeah.

[A]

Beautiful.

[B]

Pull it out.

[A]

And then one thing, last question. One thing you're deeply grateful for right now and you're life.

[B]

I'm so deeply grateful for my grandmother. Again, I mentioned this trip that was three decades, you know, in the making and to see the family she's created. My grandfather actually died before I was Born and I think she lost him when they were both in their 50s. Right. And then to see the legacy of everything that has come come through her in her children and in her grandchildren. I'm just so grateful that I was able to see her. Looking forward to seeing her again. I call her Nana and so I've been calling her Nana AI and Nana Robo. So feeling really grateful for my grandmother in particular and for my lovely extended family and also for these genes because I met my dad's sisters who used to look after me when I was little and I felt like, saw my future. And the future is bright. The future is bright. The skin care is tight. So I'm like, okay. Life is. I'm grateful for that. Grateful for these jeans. Thank you, mom and dad.

[A]

Yeah, they delivered. Unmasking AI is just one of the best books I've read. It helped me so much. I felt walked through it by you. I didn't feel alone when I was reading it. I felt like I had had a guide and I felt not just your mind, but your heart. And that makes a big difference.

[B]

I'm so glad to hear that because that was my hope. I wanted people to not feel like, oh, AI tech, this is a topic I can't engage in. I really believe if you have a face, you have a place in that conversation. And I wanted to be that friend to invite you into the conversation. So your nerd cousin that went to tech school.

[A]

Yeah.

[B]

But can still speak to people. I wanted to be that person for others.

[A]

You felt like that person in there for me. I never, I never lost sight of your heart the whole time.

[B]

Thank you. Thank you.

[A]

And I really appreciate you being on the podcast.

[B]

Thank you so much for having me here.

[A]

I told y', all, I told y' all that this was going to be an amazing conversation. I mean, I learned so much not just about technology and AI and racial bias, but about myself and about what it means to engage with your whole brain, your right side, your left side. To be an artist and a coder, to be, be a poet and a mathematician. Dr. Joy is just a beautiful person doing, I think, life saving work. Given what we're looking at. This series has been just one of the most interesting and kind of hardest and scariest things that I've ever talked about and done because I think AI is not coming. It's here. And the more we know and the more we can talk about about it, the more we can make decisions informed by our values and our ethics. You'll find everything you need to find about Dr. Joy on brene brown.com under her podcast, her Instagram all of her talks. I'm going to do one final episode where I just talk about what I've learned and how it's changed how I work and how I lead our organization. I really appreciate you being with me on this really weird walk through AI technology, social media, all the things that are living beyond human scale. Y' all stay awkward. Ravenkind Dare to Lead is produced by Brene Brown Education and Research Group. Music is by the sufferers. Get new episodes as soon as they're published by following Dear to the Lead on your favorite podcast app. We are part of the Vox Media Podcast Network. Discover more award winning [e-mail] I just gotta get out most days you see I like walking around it's good for me could you just for me where we could go eat? Take me to the good times I just gotta get out those days you see I like walking around it's good for me could you tell me where we could go eat? Take me to the good town.