← Assembly Required with Stacey Abrams
Assembly Required with Stacey Abrams
Do You See What I See? Building AI for All of Us
00:46:28
In the face of unbridled AI development and incoming President Trump’s close advisors who happen to be big investors in AI, it’s more important than ever to raise the alarm about areas of concern. Stacey Abrams speaks to Joy Buolamwini, the AI researcher and artist who brought to national attention the way bias is coded into artificial intelligence, particularly in facial recognition technology – what Buolamwini coined the “coded gaze.” They discuss what we should know about the pitfalls and potentials of AI today, and Buolamwini invites listeners to join the ongoing mission of the Algorithmic Justice League to raise awareness about the impact of AI and how we can all contribute to a more equitable use of the technology.
Loading summary
Transcript46 lines
- [00:02]
Advertiser
Now AT T Mobile get four 5G phones on us and four lines for $25 a line per month when you switch with eligible trade ins. All on America's largest 5G network. Minimum of 4 lines for 25 per line per month with auto pay discount using debit or bank account, $5 more per line without autopay plus taxes and fees and $10 device connection charge phones via 24 monthly bill credits for well qualified customers. Contact us before canceling entire account to continue bill credits or credit stop and balance on a required finance agreement too. Bill credits end if you pay off devices early. CT mobile.com.
- [00:39]
Stacey Abrams
Welcome to Assembly Required with Stacey Abrams from Crooked Media. I'm your host Stacey Abrams. If you're a regular listener, you've heard me mention artificial intelligence a few times since One of our goals on the show is to understand the challenges we face and the tools at our disposal. AI ranks near the top of both. And for all of the promise that AI holds, it's crucial that at this early stage in its development and deployment, we create ground rules to ensure a truly world altering technology does not grow unregulated and unchecked. And don't just take my word for it. Tonight, a stark warning that artificial intelligence could lead to the extinction of humanity. It comes from dozens of industry leaders, including the CEO of Chad, GPT Creator OpenAI. The expert signed the statement which says mitigating the risk of extinction from AI should be a global priority alongside other societal scale risks such as pandemics and nuclear war. That letter was signed by 350 industry leaders in 2023. And yet here we are entering 2025, still waiting for meaningful steps to be taken by our government on AI regulation to add to our necessary worries. The incoming Trump administration doesn't seem like it's full of that threat. Seriously, Trump recently announced that his new crypto and aizar, a brand new title would be Investor David Sachs, a longtime critic of what Silicon Valley tends to see as government meddling in tech. Sachs has also invested in Elon Musk's AI company xai. So two of the most influential people in the President elect's ear about the topic of AI are also the two people who profit the most from unbridled, unregulated AI development. Shocking. Now we could go on forever listing the conflicts of interest and the devastating intentions, but today we're going to play an interview we recorded just before the election. Its focus is on one facet of AI, which is, sadly more relevant than ever, given who's coming to Washington for the reasons I just mentioned.
- [02:53]
Dr. Joy Buolamwini
Hi, camera. I've got a face. Can you see my face? No glasses. Face. You can see her face. What about my face?
- [03:05]
Stacey Abrams
That is Dr. Joy Buolamwini, a poet, researcher, and computer scientist, in her 2016 TED talk. In it, she shows the audience how a technology for identification functionally cannot see her. Buolamwini is black, and it's only when she puts on a white mask, something you can buy at a Halloween store, that the technology detects her sitting in front of the camera. In her research at MIT, Dr. Buolamwini demonstrated that the algorithms behind facial recognition technology literally couldn't detect her because they'd been developed by people who don't look like her. It's one of the many areas of AI that need a deeper look. The prospect for racial and gender bias. She gave this phenomenon a name, the coded gaze. The coded gaze refers to the priorities, preferences, and sometimes prejudices of those who have the power to shape technology. In addition to the lack of AI regulation, we must also consider the overt hostility that Trump Musk and others have shown to the potential harm that AI can create for communities already under siege. This is particularly alarming given Project 2025's visceral hatred for diversity and its stated intention to strip protections from marginalized communities. As we prepare for the global AI funding boom to continue unabated, for the Trump administration to take a hands off, if not aggressively hostile approach, and for the technology to inch closer to ubiquity, we need to understand how to hold AI and the companies, agencies and organizations that use it accountable. So please take a listen to this fascinating conversation with the incredible Dr. Joy Buolamwini. Joy, thank you so much for joining me here today on Assembly Required.
- [05:14]
Dr. Joy Buolamwini
Thank you for having me. Could not be more excited.
- [05:17]
Stacey Abrams
Well, I appreciate that. What I love about the work that you do is part of the reason for this podcast. So we have this formula on Assembly Required. What is the problem? Why is it a problem? And then how can we address it? And one of the reasons I wanted to have you as a guest is because there's this AI conversation happening that for some is absolutely a problem. And I don't think that's quite right. You've talked about data being destiny. You said data is destiny. Can you explain what you mean in the context of defining the moment we're in when it comes to AI?
- [05:54]
Dr. Joy Buolamwini
Absolutely. So people are having all kinds of conversations about AI. What is AI? And right now, when we're looking at AI, is really giving computers abilities we've associated with humans in the past, ability to communicate, ability to create things. The question is how? How are these abilities being given to machines? And the how is this approach called machine learning, where machines are learning patterns from humans and that pattern comes from data sets. So let's say you want to learn the pattern of a face instead of trying to code each individual way any face might look like. The alternative is to say, here's a data set of faces. Or think of it as your diet, really. Right. In some cases, you have a very bland diet. When we're talking about data being destiny and we're talking about artificial intelligence that's using machine learning right now, machines are learning from the data. And oftentimes the past dwells within our data. Past discrimination, past exclusion. And so depending on whatever data we're feeding the machines, this becomes the diet. And so when I say data is destiny, it's saying that if we have data that isn't reflective of the world, or if we have data that's actually entrenching inequalities, those are the patterns AI systems are destined to learn and then to reproduce and then to amplify. So that's what I mean by the phrase of data is destiny. And we don't want it to be that data is destinying us to discrimination. But that will happen if we're not actually intentional about our data diet.
- [07:40]
Stacey Abrams
That's perfect way to lead into the next part of the conversation, because I've known about you for a while, and I have really respected the work you've done. But I had a good chance to dive into your fuller story this summer when I was writing my new Avery Keen novel, and it's about AI in healthcare. And in fact, your book Unmasking AI became a foundational text for how I understood my characters and their motivations. So can you tell your listeners about your first encounter with what you term the coded gaze? What is it? How does this connect to data being destiny? And why was it so important for you to tell this story?
- [08:22]
Dr. Joy Buolamwini
Yes. So some of our listeners might have heard of the male gaze, might have heard of the white gaze, maybe the post colonial gaze. Okay, to that lexicon, I'm adding the coded gaze. And it's really about power. Who has the power to decide the priorities, right? The preferences of the machines that are being created, of the AI systems that are being deployed, and also at times, whose prejudices get embedded into these machines. And so my own visceral first encounter with the coded gaze that I truly remember was back when I was a graduate student and I was taking a class where we read science fiction. And then from that science fiction, you made something you probably wouldn't make otherwise if not for that class and your six week assignment, right? So I thought, oh my goodness, what would be a fun thing to create? And I wanted to shape shift, change my body in some way. Notice we had six weeks, right? So I was like, that might be a little bit difficult to alter physics, biology, but what if I could change myself in a reflection? And so I started looking at different materials and I found something called a half silvered mirror, which is think of a one way mirror. And it had a really interesting property where if you had something black behind the mirror, it would look just like a regular mirror, but if light shone through, that light would pass. So using that I realized, oh, I can create like a filter you would see on your phone, but instead of it being on a video feed, it could actually be in the mirror. So I mean, I was at MIT Media Lab. It's just the kind of. I wasn't doing this for anything other.
- [10:10]
Stacey Abrams
Than this is a Thursday.
- [10:12]
Dr. Joy Buolamwini
Why not? Let's see if it can work, right? And so I actually got the effect to work. And I was using Serena Williams face because go goat check. That's who I want to be, right? So I have Serena Williams face. I have it. I think there was a shot of her from Platon, the photographer. And it has this beautiful black background and just her core feature. So that was exactly what I needed. And I'm having her eyes line up with mine and her nose so it's looking good. But now it's kind of like just having a cutout where you have to put your head right where the face is to get the effect. So I thought, all right, engineering chops, let's go one step further. So this time I said, all right, if I can add a webcam and have some software that can track the location of my face, then when I move in the mirror, Serena's face moves with me. So I get to stay the goat, right? I get to continue having that effect. That's what I was playing with. Now what happened was once I started playing with the software to track the location of my face, it actually wasn't finding my face. I was frustrated. So first I actually drew basically a smiley face on my palm and I held that up to the webcam and it detected the smiley face on my palm as a face, but not my dark skinned face. So at this point I'M thinking, okay, anything's up for grabs, right? And in my office, I happen to have a white mask because it was around Halloween time. And I grabbed this white mask before I even put it all the way over my dark face. The mask has been detected as the human face. And so it was literally that moment of coating in a white mask. Coating in white face. At what's supposed to be this epicenter of innovation at mit, I thought I'd arrived at tech mecca Woohoo. And here I am wondering, well, why is it that it can detect the face? I drew on my palm this white mask that is very much not a human, but it has enough of the features, but not my actual face. And that's when I started asking, are machines as neutral as I had hoped them to be? And that's what started the research.
- [12:44]
Stacey Abrams
Well, you followed up on that initial seminal finding by building research projects and you looked at other examples, including what I found to be fascinating research on politicians, especially women. What did your research teach you about how to understand this current political moment?
- [13:03]
Dr. Joy Buolamwini
That's a great question. When we're talking about data is destiny. Going back to the start of the conversation, I was curious why my face wasn't being detected. But I also knew that I couldn't just focus on my face alone. I started collecting many different face data sets and I started collecting face data sets of women in power and men in power, right, Based on the representation of women in parliaments around the world. So when it came to the top 10 in the world, Rwanda was actually number one. Policies make a difference. When you say parity is required in the constitution, lo and behold, you see a difference. You also had Nordic countries moving towards egalitarian ideals. The US was no, we were not top 10, top 20. We were not even in the conversation. And so in going through that, it was actually a reflection of what I called power shadows within the data set. Because I was looking at these data sets and the majority of them were men and the majority of them were individuals with lighter skin. And the question is, well, why? And it goes back to our actual methods of creating these data sets that then train the systems that then have the data is destiny situation. And so when you look at how we got the data sets, oftentimes for face data sets, at that time, it was generally scraping images of public officials, elected officials. And so there what do we get? We get that power shadow of the patriarchy stark and clear. We're still getting that power shadow of the patriarchy in terms of who is expected to be a leader who looks like the pattern we've seen in the past. And what happens when somebody breaks that pattern, oftentimes it breaks the system in certain ways. And that's what my research was showing. So I noticed that, okay, we have these skewed data sets. So I collected something a bit more inclusive. And because I had a more inclusive data set, I could actually start testing AI models from some of the leading tech companies in ways they hadn't been publicly tested before. Because this is what was happening. You had data sets that were largely male and I would also say lighter skin, so pale and male data sets. And people were patting themselves on the back because they had done well on that data set. Right. But let's say that Data set was 90% either light skinned individuals and or men. Right. Then when you had a test of the real world, suddenly the results don't look so great because the actual measures for success were misleading. I think thinking about this political moment, we've also had a situation where the measures of what successful leadership look like have been misleading because we're saying success means you've been elected versus success means being you've changed society for the better.
- [16:21]
Advertiser
After investing billions to light up our network, T Mobile is America's largest 5G network. Plus right now you can switch keep your phone and we'll pay it off up to $800. See how you can save on every plan versus Verizon and AT&T. @t mobile.com keepandswitch up to four lines via virtual prepaid card left 50, 15 days qualifying unlock device credit service ported 90 plus days with device and eligible carrier and timely redemption required. Card has no cash access and expires in six months.
- [16:50]
Stacey Abrams
Assembly required with Stacey Abrams is brought to you by Quince. I have five siblings and finding the perfect gift that isn't exactly what you got someone else is harder than winning the lottery. So it's great when you can pick a gift they wouldn't necessarily get for themselves. That little bit of luxury that they didn't know they were missing. For quality gifts at an affordable price, there's Quince. Quince lets you treat your loved ones and yourself to everyday luxury at an affordable price. All Quince items are priced at 50 to 80% less than similar brands. So whether it's a handbag for one of my sisters or a sweater for my brother that looks nothing like the other sweater for my other brother, Quint's is the place to go. Gift luxury this holiday season without the luxury price tag, go to quince.comstacy for 365 day returns plus free shipping on your order. That's Q-U-I-N-C-E.comstacy to get free shipping and 365 day returns. Quince.comstacy if you're ready to elevate your.
- [17:58]
Advertiser
Driving experience, the first ever Kia K4 is worth a closer look. It combines style and performance with a sleek, futuristic design featuring star map LED headlights and an available panoramic display for unforgettable looks. SiriusXM comes standard in every Kia K4, bringing you closer to what you love. Plus, with an available 190 horsepower turbocharged engine, the Kia K4 delivers everything driving enthusiasts crave. Learn more@kia.com K4.
- [18:30]
Stacey Abrams
You are an incredibly gifted wordsmith. So we've got the coded gaze, we've got power shadows, and as I said earlier, I've known about you for a while, mainly because of your Algorithmic Justice League. Another another fantastic turn of phrase. It appeals both to my inner nerd and my outer activist. And to what you were just saying. When you founded the Algorithmic Justice League, you took on Amazon, IBM, Microsoft, and you specifically called out the bias in their facial recognition technology and you forced not only a conversation, but real change. So I want to talk to you a little bit about that. So first, can you talk about how you decided to launch the AJL and some of your early wins?
- [19:17]
Dr. Joy Buolamwini
Oh, sure. So Algorithmic Justice League, as I was starting to see these issues within data sets and within AI systems, I'm thinking we need to do something, right? And we need to do something that's not just going to be about researchers. We're going to need the storytellers, right? We're going to need the artists, we're going to need the activists, we're going to need the academics, the authors, all of us, just anyone. I say if you have a face, you have a place in the conversation about AI, right? And so for me, Algorithmic Justice League was this way of putting an umbrella that there needs to be some sort of movement for algorithmic justice and people who are focusing on these issues. Because when I started it in 2016 and I talked about it on a talk that ended up on the TED platform, it was very much this concept that AI is coming and we are overconfident, as my friend Cathy O'Neill likes to say, overconfident and underprepared. And so my thought was we need to get prepared and we need to get ready. So let's form an algorithmic Justice League and also have some fun while doing it. So we definitely lean into the levity. But alongside the levity there is accountability. And so what I understood as a student at the time, and particularly being a black woman, was if I just said we have these issues with tech, I did not believe my single story alone would be enough. I understood I needed to bring receipts. And those receipts were called algorithmic audits. Algorithmic audits being we've tested the systems, you can test for yourself. These are the issues that we're seeing. The research that I'm probably most known for from MIT was called gender shades. In that research, I created that more inclusive data set and started testing AI systems from IBM, Microsoft face billion dollar tech company in China at the time. Later, Amazon and I wanted to know how accurate were they when it came to guessing the gender of a face. Right. And long story short, not as accurate for some faces versus others.
- [21:42]
Stacey Abrams
Shocking.
- [21:43]
Dr. Joy Buolamwini
Shocking. What was shocking to me was that I was doing this research as a graduate student, right, showing some of the largest gaps in accuracy in commercially sold AI products. These are arguably tests that the companies could have run internally. And so I was really curious as to why it hadn't been made a priority. Later when the research came out, I heard from senior scholars. Again, I was still a grad student at this point. They're like, oh, we've known about that issue. My question was, well, if this is an open secret, why aren't people talking about it? Right? And also, if you're talking about it but you don't have the empirical evidence, it's easy to say, oh, that's just the one off. And so the research that I did as a graduate student at MIT was to say, here's the evidence to back it up and here's the evidence on some of the latest AI models that are coming out from some of the largest tech companies. So you also can't say, oh, it's just, it's just in the research arena or they're using old methods and things of that nature. And so it was really important to me to say that this is where we are in a moment where there's so much excitement about what AI can be. And here you have Amazon labeling Oprah's face male. Here you have IBM describing the Williams sisters with all of these tropes in terms of mis gendering. You had IBM describing Ida B. Wells as a man in a coonskin cap. All kinds of tropes being propagated by these AI systems. And so this is why I did the study, I called it the counter demo and I called these counter demos, something that's part of a larger exploration of an evocative audit. So we're just talking about audits, testing AI systems, right? So the test that I did showing IBM, Microsoft, later on Amazon, we have gender disparities, we have skin type disparities, and disparities at the intersection, right? So to be more specific, you might have error rates of maybe 0 to 1% when it came to lighter skinned males, but you would have error rates over 40% in some cases when it came to darker skinned women in commercially sold products. So this is what I mean by disparities, to put numbers behind it. And so I realized those numbers were important, but the numbers without stories didn't quite connect. So we had these performance metrics, but I wanted to go from performance metrics to performance arts. And so that's why I started things like AI Ain't Tie A Woman, which is a spoken word poem. And these evocative audits I was doing, they were counter narratives because they were counter demonstrations. Because if Amazon can't get Oprah, maybe we don't want them selling facial recognition technologies to police.
- [24:59]
Stacey Abrams
Well, that is a perfect segue because you were in grad school when you began this engagement, this counter demonstration. And for a lot of folks who are in this political moment, who are in any of these social justice moments, there's a fear, especially someone who's coming out of mit, there's a worry that if you are so insistent here, these are potential future employers, or at the very least, these are massive companies that could block you from future success. Yet, like the women you named, you persisted. Why? How did you reconcile your personal fears of loss with your sense of responsibility?
- [25:40]
Dr. Joy Buolamwini
In some ways, my aspiration to be a poet, literally to be a poet of code, helped because my goal when I finished MIT was not necessarily to get a job at a tech company, nor was it to have a research lab that would be funded by one of these tech giants. But the there is a lot of corporate capture. Many of the research labs that are out there are funded by tech dollars. So I was in some ways really fortunate to be in a situation where the funding for my lab allowed me to ask hard questions. And because I had that privilege, I remember as a grad student taking a class over at Harvard Graduate School of Education with Karen Brennan, and Karen was asking me, what will you do with your privilege? And I had this privilege of having the megaphone that was being at the MIT Media lab At the time, these technical skills honed over time, and also this poetic ability to communicate. But once I started seeing the real world implications, you have police departments adopting facial recognition. At the time, we are assuming people would be falsely arrested. That happened. Now, think drones, guns, facial recognition, bad if it works and bad if it doesn't. Maybe now you have the drones targeting people you want eliminated. Right. So it's not even questions about accuracy, because we know accurate systems can be abused. But I want to be very honest, and I get into this. In the book Unmasking AI, I struggled with speaking up about these issues and also being one of few in a space, there's this question of, man, we just got in here, and it took a lot to get to this space, and there's still so much more you want to do. And I was warned, I was told, if you do this sort of research, you really risk being canceled or being pigeonholed. And then I was shown kind of the bones of graduate students who tried this sort of. I was not encouraged. Yeah, I was not encouraged, and I was discouraged. But also, many of their warnings turned out to be true.
- [28:14]
Stacey Abrams
And you still do this work. So when people are thinking local police are using facial recognition technology, or I'm walking through the airport and TSA wants to take my picture, we know that there is a tendency in human behavior to comply. We don't want to stand out. We don't want to risk. We don't want to get in trouble. And so part of what you are saying is people have to strike this balance between compliance but also risk, and that there's a risk that they won't be seen as they are, that they will be unfairly targeted. So I want to talk to you about two pieces to that. One is, can you tell us where this technology is being used, where we may not know it? And as a part of that, how do you move through the world knowing what the risks really are and knowing how you had to push through that instinct towards compliance to move to resistance?
- [29:12]
Dr. Joy Buolamwini
In terms of areas, AI is being used that you might not see. Because one of the things with the initial work of the algorithmic Justice League, we start with your face. We start with the white mask. It's in your face, literally. But then there are the algorithms, algorithmic gatekeepers. You don't see the algorithms that determine if you get a kidney or not, a kidney transplant or not. Right. You have the algorithms determining who's hired or fired, algorithms filtering out resumes. Amazon had to shut down an internal recruiting algorithm because they kept seeing that resumes with women's colleges were being downgraded. But maybe if you played lacrosse, that was a bonus. Because why Data is destiny. They trained on the data of who had performed well and had other signals that weren't just about your ability to perform the job, but how much you fit those who had had the role before. And so those are some of. I think even it can be a bit more insidious because you can't see it at work. And so that's why it's even more important to resist these systems that are harmful where you can see it. So one place you can see it is at the airport, at the TSA checkpoints. So for domestic flights, this is supposed to be a pilot. But to your point, when a TSA officer says step up, you step up generally, right? And then not only do you have that power dynamic going on, you have people behind you, social pressure. I don't know what time you got to the airport, but you might have some time pressure as well, financial pressure. These tickets aren't cheap, you know. So by the time you're going through all of this, right, is this your moment to resist or do you just comply so you can try to fly? And it can be difficult to make that choice. And sometimes people ask, well, if I said if I haven't complied in the past, is it too late? I absolutely think every time you say no to having your face scanned by tsa, it's actually a vote for biometric consent. And why I say this is so many people don't even know you have a right to opt out because of the way it's been implemented. Step up next. Step up next. And it's requiring you to go against the typical process. Right. And so it's making it even less likely. And I also understand, I just recently went through tsa. I had this viral moment where my hair was being inspected down to the scalp. It wasn't just, we're going to pat you down, we're going to become an esthetician and see the. If you have flakes in your scalp or that kind of thing. And so I personally experience what it is to be singled out and feel harassed in these sorts of situations. So I completely understand when people say, for where I am right now, it feels too precarious, like there's no judgment on that. But I also feel others who have that sense that this is something I can do, maybe you are in a position of more privileged. And in some ways, because of the work that I do with the Algorithmic Justice League. That's why I got a direct apology from TSA. Sorry, Dr. Balamwini, about your recent experiences. Right. Had I been a general traveler, would that have happened?
- [32:41]
Stacey Abrams
Right.
- [32:42]
Dr. Joy Buolamwini
So in some ways, I do think if you feel you're in a more privileged position, right. It's even more of an imperative, right, to opt out for those who have been robbed of that choice, or even your former self robbed of that choice. And then the government doesn't have the best track record for biometric data. There have been leaks. Right? There have been stolen. So even if you're like, solidarity is nice, but just for your own peace of mind, you don't actually know where all of that data is going. And they might say, we've deleted the photo. This gets interesting. This is where I put on my tech hat. You've deleted the photo, but you might have other information you got from the photo before it was deleted. So just like you have a fingerprint, you can also have a face print. They haven't been clear about what they're doing with the face prints. Right. So they can be. They can truthfully say they've deleted the image, but it doesn't mean they've deleted all of the data associated with it as well. And that data can be compromised, which.
- [33:46]
Stacey Abrams
Is perfect segue, because in addition to. As you pointed out, this is an issue of privacy. This use of technology will feed data sets and train algorithms based on who's using the data and what they're trying to learn from it. Talk a little bit about how you navigate being a scientist who believes in this technology, who believes in. And I want to give you a second to talk about that. What's the benefit of this technology? And then where do you draw the line of, as you said, how do you decide consent? And how do you decide how we train things to be better if we don't give them information?
- [34:29]
Dr. Joy Buolamwini
It is very contingent on what we mean when we say AI. If we're talking about AI, it can mean so many different things. When I say, broadly, I support the development of AI, this is to say, yes, I support or developing technologies where we can explore our capacities as humanity or find ways to address ongoing problems. When you have AI being used for things like AlphaFold, let's predict these protein structures so that we can actually do better drug development. This is literally what my dad does. He's a professor of medicinal chemistry and pharmaceutical sciences. When I was a little girl, going to his lab and seeing the protein folding structures on his computer, this is something that now AI systems are doing that can help him do his work better and then help us be healthier. So I'm not against those uses of AI. And I also think there are so many data disparities. Like, let's think about women's heart health. Less than a quarter of research participants for clinical trials are women. But cardiovascular disease, heart attacks and so forth, they're the number one killer of women. And so I support an organization, Bloomer Tech, where they're actually, they've developed these bras that can monitor women's heart health. And so now you're getting this very intimate data that's filling a vital data gap right when it comes to health. But here is what's so important. Agency you decided to put on the bra, right? Where we're not having agency is where you have surveillance uses of AI and you don't have a real choice. And so I think it is possible to opt into a society where we're using AI to solve real and present problems while we're opting out of surveillance. We don't have to have discrimination or surveillance or the surrender of our data as the entry ticket into innovation that just we can do so much better. Right? And so when I hear it's like, oh, we want to solve all of these problems with AI and we're going to collect your creative data without consent or compensation, it's a disconnect for me because the mechanism you're using doesn't actually support the outcome you claim. Let's be clear about the outcomes we're looking for or what our aspirations are, and then make sure we have ethical AI pipelines and pathways to get there, which we have to push for.
- [37:07]
Advertiser
After investing billions to light up our network. T Mobile is America's largest 5G network. Plus right now you can switch, keep your phone and we'll pay it off up to $800. See how you can save on every plan versus Verizon and at t. @t mobile.com keepandswitch up to 4 lines via virtual prepaid card. Allow 15 days qualifying unlock device, credit credit service ported, 90 days with device ineligible carrier and timely redemption required. Card has no cash access and expires in six months.
- [37:36]
If you're ready to elevate your driving experience, the first ever Kia K4 is worth a closer look. It combines style and performance with a sleek, futuristic design featuring StarMap LED headlights and an available panoramic display for unforgettable looks. SiriusXM comes standard in every Kia K4, bringing you closer to what you love. Plus, with an available 190 horsepower turbocharged engine, the Kia K4 delivers every everything driving enthusiasts crave. Learn more at kia.com K4NetCredit is here.
- [38:08]
To say yes to a personal loan or line of credit when other lenders say no, apply in minutes and get.
- [38:12]
Stacey Abrams
A decision as soon as the same day.
- [38:14]
Dr. Joy Buolamwini
Loans offered by NetCredit or lending partner.
- [38:16]
Advertiser
Banks and serviced by NetCredit applications subject to review and approval.
- [38:19]
Stacey Abrams
Learn more at netcredit.com partners. NetCredit Credit to the people the AJL has put out this framework for understanding and improving what AI could be that calls for equitable and accountable AI. So I want to do a little bit of unpacking. So, number one, what is inclusive AI?
- [38:39]
Dr. Joy Buolamwini
So inclusive AI might sound enticing, right? It's like, let's make sure. Let's get all the data, let's make sure everybody is part of AI. Broadly speaking, and at face value, it probably sounds good, right? But how are you being included? And included into what? So if you're being included into surveillance, if you're being included into exploitation, you might want to say no. And if you're being included without your consent, I didn't want to stand in this line, right? I didn't want those socks you're selling, you know. And so for me, the limitation of inclusive AI goes back to that question of agency, particularly because some of the ways in which you might quote, unquote, make AI inclusive can mean violating people's privacy or disregarding consent. One of the examples I write about in the book, shortly after my research gender shades came out and people are seeing, oh, we have some power shadows in these data sets. You had companies like Google saying, we will make more inclusive data sets. So let's hire a subcontractor. And guess what subcontractor, they're on the streets of Atlanta getting face data, face photos from homeless people. So this is why inclusive AI isn't the banner that we go under. But we're actually asking, well, what does it mean to be accountable and what does it mean to be equitable? And part of what that means is agency, right? People have a voice and a choice. There's affirmative consent. If I don't know what's going to happen with this data, or you're just telling me to go through airport security without and tell me to scan this QR code to go find more. That is not informed consent, nor can you really affirm your consent in a coercive sort of situation. And so that's that's why we shifted from this language of inclusion and also even this language of ethical. Right. You mentioned that I like to play with words. And what we were seeing with the term ethical AI was a lot of what some would call ethics. Washing, yes, we're ethical, but what are your practices? So this leads to accountability because you can say ethical all you want, but we're going to say, okay, how did you collect the data? Did you get consent? Was there meaningful transparency? And the biggest thing for accountability that we seldom see is redress. Oops, made a problem. It might not be a small loops, it might be somebody's in jail.
- [41:16]
Stacey Abrams
Right?
- [41:17]
Dr. Joy Buolamwini
Mistake, Right. And so being accountable means not just saying we did our best, good luck. Right? It means if something goes wrong, you have to address and affirm that it's gone wrong. So part of accountability is saying, yes, we messed up, but you have to also take a step further and look for redress as well.
- [41:42]
Stacey Abrams
So you've got some countries like Kenya that have set very hard limits on how their national information can be used. And you've got the Wild west, that is the United States, who is doing it kind of right.
- [41:54]
Dr. Joy Buolamwini
I would kind of say the European Union, with the passage of the EU AI act is the most comprehensive AI governance legislation that we've seen that actually puts in guardrails and also a risk based framework for determining what kind of AI systems can and cannot be used. And I think there's a lot to learn from Europeans in that regard. I also think even within the US we do have frameworks, they have not yet been put into law, but they are reference points. We have the blueprint for an AI Bill of Rights that talks about the need for safe and effective AI systems, the need for notice and explanation, and most importantly, meaningful alternatives and fallbacks. We did a campaign around the government's use of facial recognition for access to government services. It meant some veterans weren't able to access their services. White men in Colorado, slow Internet connection, not the best webcam, not able to get something that they actually should be getting. And so I think it's so important that as people are excited about the possibilities of tech, we don't lose the reality of infrastructure. Right? It's not being accessed by everybody in the same way. And so part of what a government, a federal government, local government, state government should also have in place, are these meaningful alternatives? For sure.
- [43:31]
Stacey Abrams
Okay, so you have been a perfect guest. You helped me set up the problem, explain why it's a problem, and now we gotta solve it. So what are three things that a listener can do today or in the next few months? What can they do to address the concerns of inequitable and unaccountable AI in their daily lives? Short of getting into a fight with the tsa, you can join me on that.
- [43:59]
Dr. Joy Buolamwini
I will say writing Unmasking AI and even the documentary Coded Bias on Netflix Flicks showed me the power of sharing your story. So I share that story of an AI failure. I'm literally coding in a white mask. And I think sometimes we underestimate the importance of sharing what we've experienced. But when you share what you've experienced, other people say, oh, me too, right? I'm not alone in that. And you also build an evidentiary record so they can't gaslight us and say, oh, what you think is happening is not happening when you have all of these stories. So maybe as we get these kinds of stories submitted to AJL all the time. Report.ajl.org Right my my child was flagged as cheating with AI, but it turns out that English is a second language, right? Not actually cheating. Those stories are important because it makes it easier to hold companies accountable. So the first thing I would say is please, please don't underestimate the power of your lived experience. If you are encountering AI bias, some people will say, look, I tried to generate an image using generative AI and I got racist stereotypes. I tried to use a filter to professionalize my photo and my skin was whitened or lightened. All of those examples are so important. So share your story. Capture that counter demo. And I think the other thing that I have found to be really helpful is educating your community. Because so often I've seen this time and time again. If you're not a tech bro, you don't need to go to MIT and have a PhD in AI to be part of this conversation. And when you have more voices saying, no, we actually want it a different way, that's how you build power. And so building power by sharing these stories with the communities you're part of. I saw this in such a strong way. When we were working with the Brooklyn tenants, their landlord installed a facial recognition entry system. They didn't want it and some of them started organizing and they found some of my research and I had taken a lot of time to try to make it as accessible as possible. Explainer videos, walkthroughs, all of this. And they were using it to speak to their elders about what was happening so they could also be educated about it. So I would not underestimate the power of sharing what you're learning with the communities you care about.
- [46:44]
Stacey Abrams
This was such a pleasure. Dr. Joy Buolamwini, thank you so much for spending time with us here on Assembly Required.
- [46:52]
Dr. Joy Buolamwini
Thank you so much for having me.
- [46:53]
Stacey Abrams
Be each week we want to leave the audience with a new way to act against what can feel inevitable, an opportunity to make a difference, and a way to get involved or just to get started on working on a solution. In a segment we like to call our Toolkit, at Assembly Required, we encourage the audience to be curious, solve problems, and do good. Now Dr. Joy gave us a great primer on Biometric agency Number one tell your story. Number two capture your images and number three share with your community. I would say watch the Netflix documentary Coded Bias. It is a fantastic entry point into understanding the ins and outs of artificial intelligence and the challenges of the Coded Gaze. Prepare for all the awkward holiday conversations you'll be having by boning up on AI right now and you can wow your audience. Number two check out Dr. Joy Buolamwini's book Unmasking AI My Mission to Protect what Is Human in a World of Machines. It is out in paperback November 19th. I love this book. Pick it up now for solving problems. At ajl.org you can find the Take Action tab, which has multiple ways to participate in the movement for equitable and accountable AI. AI is with us, so let's make sure it works for us. You can share your story of when you confronted the Coded Gaze, you can sign up for their newsletter, you can make a donation, and you can help the organization's work in educating more of us on the future of AI. To check out their most recent campaign, look up the algorithmic Justice League's hashtag Freedom Flyers. There you'll find information on knowing your rights with facial recognition at the airport, as well as opportunities to share your experience with it and to do your bit of good. You can spread the word about what you've learned on social media by tagging AJL United. And tell your friends to join us here on Assembly Required, where they too can satisfy their inner nerd and their outer activist. If you want to tell us what you'd like to learn more about or hear about from us, send us an email atassemblyrequired@crooked.com or leave us a voicemail and you and your questions and comments might be featured on the pod. Our number is 213-293-9509. Thank you all so much for joining me on this journey to create Assembly Required with Stacey Abrams however, with the holidays upon us, we're not going to be meeting next week. Instead, we will see you again on January 9th. Assembly required with Stacey Abrams is a Crooked Media production. Our lead show producer is Ilona Minkowski and our Associate producer is Paulina Velasco. Kirill Palaviv is our video producer. This episode was recorded and mixed by Evan Sutton. Our theme song is by Vasilius Fotopoulos. Thank you to Matt De Groat, Kyle Seglin, Tyler B and Samantha Slosberg for production support. Our executive producers are Katie Long, Madeline Herringer and me, Stacey Abrams.
- [50:24]
Advertiser
If you're ready to elevate your driving experience, the first ever Kia K4 is worth a closer look. It combines style and performance with a sleek, futuristic design featuring starmap LED headlights and an available panoramic display for unforgettable looks. SiriusXM comes standard in every Kia K4, bringing you closer to what you love. Plus, with an available 190 horsepower turbocharged engine, the Kia K4 delivers everything driving enthusiasts crave. Learn more at kia.com k4.