Should We Stop Catastrophising About AI? With Eleanor Drage and Carl Miller

Summary

Podcast Summary: Intelligence Squared — Should We Stop Catastrophising About AI? With Eleanor Drage and Carl Miller (July 25, 2026)

---

Episode Overview

This episode of Intelligence Squared, hosted by Carl Miller, features Dr. Eleanor Drage—senior research fellow at Cambridge’s Leverhulme Centre for the Future of Intelligence and author of What if We Got AI Right? How to Stop Catastrophizing and Build an Ethical Future. Together, they challenge the dramatic narratives surrounding AI—catastrophe versus utopia—and argue that urgent real-world issues are masked by such extreme stories. The conversation dissects who benefits from “AI doom,” why both hype and dystopia come from the same Silicon Valley circle, and how everyday people and professionals can pursue a more ethical, grounded, and participatory vision for AI.

---

Key Discussion Points & Insights

Imagining a "Got-It-Right" AI Future (02:31–06:01)

• Redistribution and Decentralization: Drage emphasizes the need to decentralize compute power and build open-license technologies to redistribute wealth and opportunity globally, referencing accessible innovations like DIY 3D-printed microscopes. (02:53)
• Day-to-Day Agency: Individuals should actively question the AI tools at work, demand transparency about their impact on productivity, and push back against discriminatory or unproven systems (e.g., in policing or HR).
• Challenging AI Images: Drage calls out the persistent use of metaphors like “Terminator” or the “Vitruvian Man” as misleading. She advocates for imagery that reflects the real, material nature of AI—like silicon monocrystals or quartz mines—making AI’s physical roots and environmental costs visible. (05:20)

“People don’t know what AI is or looks like... Data labelers, show them, show their faces.” — Eleanor Drage (04:55)

Utopia vs. Apocalypse—the Dueling Myths (06:01–12:39)

• Utopia Is Incremental: Drage draws on her PhD in utopian studies, arguing genuine progress is small, continual actions—“the act of brushing your teeth, getting up every morning, committing yourself to a better world”—not distant, perfect endpoints. (06:44)
• Silicon Valley’s Twin Narratives: Both the utopian (“We’ll overcome death!”) and doomsday (“AI will destroy us all!”) tropes originate with Silicon Valley figures, often as marketing strategies to push their products and personal mythos (10:43).
• Instrumentalization of Doom: The apocalyptic narrative serves to amplify hype and, critically, to drum up investment and attention—even from academics and media. Drage calls out overt product launches (e.g., Anthropic’s Mythos model) that leverage “AI too dangerous to release” hype for marketing.

“‘Doom’ is a form of hype.” — Eleanor Drage (12:39)

Diversifying Sci-Fi Fantasies (12:39–14:48)

• Feminist & Diverse Futures: Drage urges the field to embrace broader, more imaginative science fiction—highlighting women, Caribbean writers (e.g., Nalo Hopkinson), and alternate AI-human relationships, like “ancestor spirit Alexa,” as more nuanced possibilities.

“Why don’t we fantasize about other things too? ... There are loads of science fiction writers who write about cool futures ... complicated, interesting, and ethical...” — Eleanor Drage (13:32)

Hype, Hysteria—and the Role of Everyday People (20:18–24:39)

• Ordinary People’s Concerns: Miller and Drage discuss prevalent anxieties among teachers, parents, and professionals—about jobs, child safety, and deepfakes—feeding into both justified concern and exaggerated doom.
• Empowering Communities: Drage spotlights how teachers and workers often demonstrate practical, collective wisdom, actively shaping how AI impacts their institutions (e.g., resisting unfair grading algorithms or recruitment tools).

“Teachers knew what was going on pretty quickly... went on daytime television to shout at Gavin Williamson with zero understanding... But they really knew what was going on.” — Eleanor Drage (22:48)

Existential Risk vs. Real-World Harm (24:39–32:15)

• Two Schools of AI Risk: Drage explains the divide between those focused on existential risks (low-probability, apocalyptic scenarios like “paperclip maximizers”) and those prioritizing tangible, contemporary harms (bias, policing, surveillance, workplace coercion).
• Ethics Is Contextual: Instead of searching for a single most important AI harm, Drage insists that ethical interventions should address local, context-sensitive issues—what matters to you or your community, from recruitment discrimination to accessibility for disabled individuals.
• Ethical “Whack-a-Mole”: Fixes in AI ethics often have trade-offs; for example, making facial recognition better for everyone can also aid surveillance of already targeted communities.

“It has to all happen at the same time... Ethics is a game of whack-a-mole—you think, ‘Oh, I fixed that’, and then some other side effect pops up...” — Eleanor Drage (32:10)

Ethics, Power, and Reclaiming Technology (33:01–36:17)

• Beyond Abstract Ethics: Drage reframes ‘ethics’ as a question of shifting power and agency—locally created, open-source tools for real needs (e.g., pollution monitoring in the Niger Delta, open-hardware scientific devices).
• Cultural Translatability: Not every language has a term for “bias,” but every society understands “power” and its distribution. Drage advocates redefining ethics this way for more global, grounded, and actionable frameworks.

“It’s time to rethink ethics as shifting power... every language has a word for power.” — Eleanor Drage (35:21)

Literature and Imagination—Reclaiming Our Futures (36:17–41:42)

• The Limits of Silicon Valley’s Sci-Fi: Drage critiques the dominance of traditional sci-fi tropes and reading lists (Asimov, etc.) as fundamentally limiting our ability to imagine better futures (37:03).
• Encouraging Concrete Utopias: Rather than abstract, disconnected “good” tech, ask: What technologies actually work—and are flexible, transparent, and hackable—for you in your real life?
• Practical Advice: Drage encourages listeners to compare the AI tools they’re presented with to the trustworthy, useful, adaptable technologies they already use and understand. This forms the basis for resisting opaque or hostile tech in daily life and at work.

“The first step is just pinpointing what the issue is and how these technologies relate to stuff that we do enjoy on a daily basis...” — Eleanor Drage (41:14)

---

Notable Quotes & Memorable Moments

• On Popular Misconceptions:
  “We need to be going into work and saying to our boss, ‘What is this thing you’re making me use? Is it really proven to improve my productivity or will it just give me more work?’” — Eleanor Drage (03:39)
• On Silicon Valley Myths:
  “The problem with [AI apocalypse narratives] is that they’re a white apocalypse... where the Terminator lands first in San Francisco... For the rest of us, the planet is what we need to focus on and reconstruct.” — Eleanor Drage (08:12)
• On Market Incentives and Academic Buy-In:
  “It’s an incredible marketing campaign for them to say we need to press pause… but actually we also need to press go... Think about the benefits to humankind if we channel energy from stars. I don't understand why more people don't find that ridiculous.” — Eleanor Drage (10:46)
• On Ethical Action:
  “Mother Teresa wasn't perfect, but I really like what she said about helping one person at a time, starting from the person closest to you.” — Eleanor Drage (28:58)
• On Diversity of Imagination:
  “Elon Musk would stop publishing his reading lists. I'm so bored of this crap... I just wish we would draw from more of [the rich feminist and international sci-fi tradition].” — Eleanor Drage (38:01)

---

Timestamps for Major Segments

• 02:31: Envisioning a world where “we got AI right”
• 06:37: Utopia, apocalypse, and Silicon Valley’s narratives
• 12:39: Doom as a form of hype; marketing the apocalypse
• 20:18: Practical worries and everyday experiences with AI
• 24:39: Existential risk vs. real-world harm, and the politics behind them
• 32:15: The practical limits and contradictions of “AI ethics”
• 33:01: Real-world ethical innovation and shifting the power dynamic
• 36:17: The role of science fiction in imagining possible AI futures
• 40:08: How listeners can become more discerning, critical participants in shaping AI deployments

---

Final Takeaways

This episode urges listeners to move beyond paralyzing doom or fantastical optimism about AI—both often engineered by those already in power—and instead ground their concerns and hopes in incremental, communal, and pragmatic action. Whether it’s choosing open-source tools, demanding transparency at work, or reimagining whose stories define the future, Drage and Miller advocate for ethics understood as empowerment, not abstraction.

Transcript

[Progressive Insurance Narrator]

Insurance isn't one size fits all, and shopping for it shouldn't feel like squeezing into something that just doesn't fit. That's why drivers have enjoyed Progressive's name your price tool for years. With the name your price tool, you tell them what you want to pay and they show you options that fit your budget enough. Hunting for discounts, trying to calculate rates, and tinkering with coverages. Maybe you're picking out your very first policy, or maybe you're just looking for something that works better for you and your family. Either way, they make it simple to see your options. No guesswork, no surprises. Ready to see how easy and fun shopping for car insurance can be? Visit progressive.com and give the name your price tool a try. Take the stress out of shopping and find coverage that fits your life on your terms. Progressive Casualty Insurance Company and Affiliates Price and coverage match Limited by state law

[Google Chrome Advertiser]

this episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome? That's new. It can help you with with practically anything on the web, like restoring a vintage motorcycle from a 50 page restoration block. Or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it, ready to make anything online make sense. There's no place like Chrome. Check responses set up required compatibility and availability varies.

[Mia Sorrenti]

18 welcome to intelligence Squared, where great minds meet. I'm senior producer Mia Sorrenti. We are deeply confused about artificial intelligence, what it is, who it serves, and whether it leads us towards utopia or catastrophe. But according to AI ethicist Eleanor Drage, these dramatic narratives often distract us from the urgent problems already embedded in the technology. On today's episode, Drage joins researcher and author Carl Miller to explore the culture and ideas driving the AI arms race. From billionaires seeking to overcome death to biased policing, algorithms, and the growing influence of big tech over governments, they examine the forces shaping artificial intelligence and ask who stands to benefit. Let's join Carl now with more.

[Carl Miller]

Well, welcome everyone to Intelligence Squared. I'm Carl Miller and our guest today is Eleanor Drage. She's a senior research fellow at the Leverhulme Centre for the Future of Intelligence at Cambridge, host of the Good Robot podcast, author of several books. And now another one, the one we're talking about today. What if we got AI Right? How to Stop Catastrophizing and Build an Ethical Future. Very warm welcome to mnl.

[Eleanor Drage]

Thanks so much for having me.

[Carl Miller]

All right, so it's 2030. Eleanor, everyone in the entire world has read your book. Everyone has Bought into the vision and we have got AI right. What does that actually look like for someone listening to this? Like what interactions do they have with AI? How are they governed? How are they protected? How are they liberated?

[Eleanor Drage]

Do you know what, we could even say 2027 and why not? Let's be ambitious, infinitely doable. So on a bigger scale we need to be decentralizing compute power. We need to be making open license technologies that encourage the redistribution of wealth. And I work with incredible people who are making microscopes out of 3D printed bits and off the shelf computer part that anyone can make anywhere in the world and make money from. We need to be tackling big tech monopolies and incendiary foreign policy. Good luck to us. But also we need to be doing things that you and I can do day to day. We need to be going into work and saying to our boss, what is this thing that you're making me use? Is it really proven to improve my productivity or will it just give me more work? Can you show me the data behind this? Can you explain to me how it works? We need to be preventing the police from using weird AI tools to track and monitor protests that are proven to discriminate against people of color and left wing protests. We need to be asking whether consent mechanisms that we have to use to hand over our data really are consent mechanisms or are just a total conversation. So these are things that we can be doing short term. We can also be doing things like changing the ideologies around AI and changing the images we use to depict it so that it's not just Terminators dropping off bridges in San Francisco and the Vitruvian man that's hanging in front of the desk at Google DeepMind that symbolizes human perfectionism, this future where we will become perfect because of AI and instead use symbols of AI that are realistic and feasible, like silicon cubes. Does anyone know what that looks like they look like? Or quartz mines? Has anyone seen one of those? People don't know what AI is or looks like. Data labelers show them, show their faces. And I think that we can incrementally come to a place where we get a better understanding of what AI is and what actually we want from it.

[Carl Miller]

What does a silicon cube look like? I mean, is it a white cube or is it.

[Eleanor Drage]

Well, silicon monocrystals are. They're really beautiful. And I think when I saw a picture by Katherine Breslin, who's a photographer and did like a real close up of one of these cubes, it was like glistening and silvery and shiny and tactile and. And I thought, well, what a great image. And why don't we have that in the FT or the Times or whatever people read. That gives you a really good sense of oh, AI is actually a material thing and it's not just a brain floating in a vat and other science fiction fantasies. And from there we can ask questions about the environment and labor that we're unable to ask otherwise.

[Carl Miller]

Okay, so it's 2027. There's more glistening cubes, there's less Vitruvian men. We've got rid of incendiary foreign policy, which I think is particularly laudatory for us to have done in one year, but to do so. Eleanor, I think you argue that there's two different bodies of thought and thinking about AI that we kind of have to puncture in equal way, isn't there? You've got the kind of doom mongers on the one side and you've got the kind of hype mongers on the other hand. Tell us a bit about how these two different discourses kind of poison or like shroud our way forwards.

[Eleanor Drage]

Well, first of all, I want to talk about Utopia, which has to do with your question.

[Carl Miller]

Yeah. Which is why I asked you.

[Eleanor Drage]

Yeah. So my PhD was in utopian studies and science fiction written by women writers, which my parents thought was completely useless. And they're probably right. But what I learned from that is that Utopia is not some Thomas More island in the 15th century floating in a vacuum, detached from space and time. It's not a future where we've got everything right. And it's normal that you ask that question because that's the way that utopia is perceived in popular culture. But Utopia is, and ask anyone who works for a charity or some, you know, feminist organization, it's really not pure and easy, Kumbaya, you know, a future where no one works, it's incremental, hard, day by day change. It's the act of brushing your teeth, getting up every morning, committing yourself to a better world. And so I don't, you know, of course I want things solved and I want solutions. And you know, I'm not closed minded in that way, but learning to work together and ask the right questions to me really is Utopia. And that's why it's feasible. That's why we can actually do this. We just need to learn to disagree better and have high octane conversations at work about the politics of AI because it does have a politics and it doesn't matter whether you're working at McKinsey or Lidl, there is always something there that you can unpick. How is this thing made, et cetera. So it is really feasible. That's what I want to say about that. And then, yes, we are stranded between an apocalypse defined by these guys in Silicon Valley that say, hey, apocalypse is not actually what's happening in Gaza or genocide or whatever. And Nick Bostrom has said in one of his essays that I read, sadly, that genocide is not important. It's a drop in the ocean, in the great sea of life, because people remain. So for him, apocalypse is when everyone dies. And the problem with it is just that human intelligence and capability are eradicated. So this is a white apocalypse. This is an apocalypse that affects San Francisco, the places where the terminator lands first. And then, on the other hand, you have utopia, defined also by these same people as a world where we don't work anymore, we can live forever, and we freeze ourselves like the cryogenesis and wake up one day in some perfect form. And, you know, obviously that's horrific for the planet because we have an overpopulation issue anyway, which is why these guys want to go to Mars. But. But I say, marg, let them go to Mars, but for the rest of us, the planet is what we need to focus on and reconstruct, and that is utopia. That hard work of rebuilding really is the thing that we need to focus on.

[Carl Miller]

Well, you touched on something there, which you also pointed out in the book, which is obvious when you say it, but yet I haven't quite realised, and I think it's so interesting, which is that you've got this kind of utopian narrative, often shaped by and expressed most vocally by engineers in Silicon Valley. And you've also got the doom mill vocalized by and expressed by most vocally engineers in Silicon Valley. And it's extraordinary that the same diametrically opposed series of ideas are coming from the same people. Why on earth is this the case? Why does Silicon Valley and San Francisco have this outsized intellectual footprint as well as technical footprint in this debate?

[Eleanor Drage]

Because it's an incredible marketing campaign for them to say that we're developing digital minds, we need to press pause or have a moratorium, which is something that was said by a whole host of Silicon Valley entrepreneurs and also quite sensible philosophers who got kind of sucked into this way of thinking a while ago in an open letter written by the Future of Life Institute. And they are this paradox of saying we need to press pause, but actually we also need to Press go. Because we need star power. And think about the benefits to humankind if we channel energy from stars. I don't understand why more people don't find that ridiculous. And you get really sensible people. Like, a few years ago, I was interviewed on Moral Maze by some fantastic BBC4 people who insisted on quoting Musk quote after quote after quote, saying, but the world could be amazing, but we could all die. And not seeing how ridiculous that whole formulation is and how dependent their version of utopia is on their version of apocalypse. Nobody wants that. And yet we're. We've kind of been sucked into it because it's great for selling their products. You know, they tell us they're developing conscious systems, and there's no evidence to suggest that these systems are digital minds or have any version of consciousness. And I work with the best people in Theory of Mind and Consciousness at the University of Cambridge and Broadchurch, but they all disagree with those statements. So, you know, I'm glad that you're mentioning it. And we really need to just show it for what it is, which is just a series of horrible lies.

[Carl Miller]

So are you saying essentially that doom is a form of hype?

[Eleanor Drage]

Yes, it is. And I saw this for the first time in 2016 at a science fiction conference in Helsinki, where we were packed into a room. There was about 1,000 people in this room in a huge conference center bursting at the seams. People were frothing at the mouth about the topic of conversation that day, which was the Singularity, an intelligence explosion. And on the panel were four science fiction writers. And literature is very important, and this is why literature degrees are great. But what was going on here, you know, people were so excited. There was this erotic energy. People were talking about hard science and hard science fiction. It's very masculine energy as well. We see this obsession over apocalypse, a fetishization of apocalypse that is exciting, erotically charged, et cetera. And it's very. Makes a lot of sense to engineers who. My mom is an engineer. She loves her tools. She is obsessed with the things that she makes. And. And that relationship is very strong. But I think for the rest of us, we need to step back and say, why don't we fantasize about other things too? And all I'm calling for is diversification of fantasy, because there are loads of science fiction writers who write about cool futures where women give birth to mind ships past that are both ancestor, spirit and machine, and also animal, and in ways that are interesting and ethical, but also complicated. There's no sort of perfect utopia where Are those dreams of women writers or Caribbean writers like Nalo Hopkinson, who talks about AI devices that are grannies, the household granny nanny. That is a kind of ancestor Spirit Alexa. Just a way more fun, interesting version of the future and I'd really like to see these stories come to life.

[Carl Miller]

I love the idea of an ancestor Spirit Alexa, especially if it isn't actually connected to Amazon. That would be particularly useful. But we are going to get onto science fiction and literature in a moment. But just one more thought around this kind of the kind of instrumentalization of Doom as actually a marketing ploy, which I think is such an important thing. So you would see Mythos and for those listening to this that might not know what I'm talking about, Mythos was a new mod that was kind of released, not released. It's a great fanfare by Anthropic who basically said, look, we've got a model that can find all of these cybersecurity gaps in the world and it's too powerful and essentially too apocalyptic for us to actually make available to everyone. You would just see that essentially as being doom harnessed onto a product launch in order to get people talking about it.

[Eleanor Drage]

Oh, now I feel like, I'm mean, I don't know how to work at anthropology. I feel like I've got friends who work at Anthropic and I think for the most part they're really good at what they do. But the fact is that everybody who works in a silo is a bit clueless about the impact of what they're doing. And I think we have a very overinflated sense of our own importance. And it's easy to have that when Trump is trying to get involved with Anthropic's business. And suddenly the founder and all these engineers who just signed up to make stuff are having to really think about their implication in war zones and in US politics and all this stuff. And the fact is that they're not well enough equipped to make those decisions. We cannot have a world where we have engineers come up to me, as happens all the time, and say, look, I made this thing for Google Gemini, and people are upset about it, but I'm not really sure and are upset and confused. Like, I feel bad for these guys. You know, they need to be creating stuff in an environment where they can go and chat to people about the implications of what they're doing and that will improve their own well being. But it will also create a situation where we get a sensible narrative around these tools by Anthropic or whoever because at the moment it's, it's hyped because they're afraid and they actually just don't really know. And so they're saying just random stuff to the press. That's not sensible.

[Quince Advertiser]

Finding quality summer clothes can be tricky. You want them to be lightweight and breathable, but also well made so it lasts more than a single season. For me, it's all about looking for specific fabrics. Linen has become my personal favorite in warmer months. And Quince has tons of 100% European linen pieces that you will be reaching for all summer long, starting at just $32. They also have great cotton, silk and denim pieces. I have this cropped cotton cardigan that works really well as an evening layer or if I'm going into a restaurant with a lot of air conditioning. The buttons have this cool pattern and the fit of the sweater works well with a ton of summer outfits. Quince also has jewelry. They've got beautiful 14 karat gold hoops or delicate chains that can really make the most effortless outfit feel intentional. And everything at Quint is priced 50 to 80% less than similar brands because they work directly with ethical factories and cut out the middlemen. So you're paying for exceptional quality and not brand markup. And beyond their clothing and jewelry, Quints also offers elevated essentials for your home, from bedding and bath to kitchen essentials and even furniture. So make your summer wardrobe feel easier. Go to quince.com intelligence for free shipping on your order and 365 day return returns. Now available in Canada too. That's Q-U-I-N-C-E.com intelligence for free shipping and 365 day returns. Quince.com intelligence

[Starbucks Advertiser]

this episode is brought to you by Starbucks.

[Eleanor Drage]

That is fire. Whoa, that's good.

[Carl Miller]

This might be the drink of the summer.

[Eleanor Drage]

Okay, I like this one too. I'm rocking with it.

[Starbucks Advertiser]

Okay, try it for yourself. Starbucks refreshers concentrates are coming home. Find them in the coffee aisle and make it yours.

[Ryan Reynolds]

Hey, it's Ryan Reynolds here from Mint Mobile. Now I was looking for fun ways to tell you that Mint's offer of unlimited Premium Wireless for $15 a month is back. So I thought it would be fun if we made $15 bills, but it turns out that's very illegal. So there goes my big idea for the commercial. Give it a [e-mail] upfront payment of

[Starbucks Advertiser]

$45 for three months, $90 for six months, or $180 for a 12 month plan required $15 per month equivalent taxes and fees. Extra initial plan term only greater than 50 gigabytes. Me slow when network is busy.

[Wayfair Advertiser]

See terms that refresh you've been putting off until the right deal came along. It's here. Wayfair's Black Friday in July sale is happening now, so you can finally get the style you've been waiting for for less. Get up to 80% off area rugs and up to 60% off outdoor and bedroom furniture. Shop Wayfair's huge selection of styles and find the piece to fit your style, budget and space, plus free shipping. Black Friday in July ends July 27th. Shop [e-mail] Wayfair Every style, every home.

[Carl Miller]

Okay, so those are the engineers. But. But what about the normal people? So, you know, you speak a lot of AI events and also do I. You know, and I'm sure you've seen the same thing as me, which is that a lot of people are unbelievably concerned about what's happening, you know, and, you know, they always, you know, it's, it's. They genuinely don't know what jobs are going to exist in the future. You know, they genuinely don't know how to equip their kids to navigate an AI age. Then we might have a dead Internet at some point. Like deepfakes are a problem that we haven't got a technical solution to. So how do people kind of. Who would agree with you? Okay, there is this kind of marketing kind of tactic engine that's using doom. How would they kind of navigate around that and not catastrophise yet still kind of genuinely, kind of experience and try and deal with a whole bunch of questions that are so important and that essentially I don't feel anyway, that we actually have much in the way of meaningful response to at the moment.

[Eleanor Drage]

Yeah. So part of what's really nice about my job and the centre I work for at Cambridge is that we are encouraged to go out into the world and talk to people, which is relatively unusual.

[Carl Miller]

It's a useful thing to do.

[Eleanor Drage]

I know, right? I mean, the fact is, you know, most of my friends don't give a crap about Cambridge. And, you know, so you're forced to explain things to people in a way that makes sense to them. But also I can learn things from teachers who tell me very sensibly. You know, I go in to give a talk at a school. I give loads of school talks and talks to parents and I always expect to have to explain a lot, but often it's the teachers who are saying to me, look, this is what we're going to do because we know our students very well, we understand what learning is. In fact, we've sat down to say what do we mean by learning? And how does our definition of learning incorporate AI or not? And that conversation changes from discipline to discipline. And teachers always disagree about whether going to ask ChatGPT a question about how to answer some essay question is good or bad. But that's an ongoing conversation that teachers are used to having. They're used to sitting down every quarter and saying what's going on now in the education system and what does this mean for us? And because they've got those systems and habits in place, they're really good at reacting to what's going on. So we saw this in the A level algorithm crisis where Gavin Williamson, the Education Secretary during COVID decided to commission this AI tool that was going to give grades to students that were unable to sit exams. And it was predicting that students who were in deprived areas in state schools were likely to get grades that were two or three grades below the predicted grades of their teachers. Right. And the teachers knew what was going on pretty quickly and then went on daytime television to shout at Gavin Williamson with zero understanding really of the algorithms that were behind them. But they really knew what was going on. And Gavin had no idea. We kept parroting some nonsense about maths being neutral, right. Which just shows how bad our maths education is in the uk. So we know, I think, more than we think recruiters know that some weird snake oil AI tool is going to not do what they do when it claims to be able to read your personality from looking at your face. They know that that's not de biasing AI because turning us into neutral data points isn't going to eradicate racism or sexism or discrimination against the disabled in an organization. It just parrots this logic of liberal colorblindness where we can just close our eyes and everything will be okay because we're all human beneath it all. And I'm so encouraged by these sensible people who are good at what they do, making really great choices, and in turn that teaches the market what there is an appetite for and what people actually don't want.

[Carl Miller]

Well, you're bringing us into this kind of landscape now, which is where I wanted to go anyway. So thank you, Eleanor. Around the kind of debates, around the live, active debates, around the real harms of AI, you touched on one and perhaps the nastiest pressure point in the whole ethical debate. I Think at the beginning when you spoke about the kind of existential risk versus contemporary risk sort of debate. Why don't we begin there? Tell us a bit about that conflict, because that really is the one that seems to be the most systemic, doesn't it, in terms of where the ethical chits are going to fall.

[Eleanor Drage]

So existential risk for people who don't know is worrying about very important things. So low probability, high impact events and how. Well, hang on, so how you define that is very political. So the centers around the world that look at this in ways I don't think are useful or ethical look only at AI turning us into paperclips and they get a lot of funding from that. I'd be remiss as an academic not to point this out. People love investing in speculative problems.

[Carl Miller]

Why?

[Eleanor Drage]

Because they're not actually sad. Things that are sad are like poverty and cancer and destitution. Those things, they're not erotic in the same way as we've talked about with the apocalypse fantasies. But the centre that I work for at Cambridge deals with mostly short term current prescient issues and that's quite political, but it can also be not just to do with the police or hiring or whatever, but other pro justice ways of building technology. But we have a sister center downstairs from us called the center for Existential Risk and they deal with existential risk, these long term low probability events in a really great way. So they also look at biowarfare and geopolitics and other terrible crises. And we have the leading volcanologist who apparently is so famous in the volcano sector that people kind of bombard her with photos at conferences. So we have lots of. I'm not saying that thinking about catastrophe is always terrible. It can be extremely useful, in fact, saves lives of the lives of the people that live near volcanoes all the time. So that's fine. What I'm trying to do is hold hands between those two things and say, look, sensible approaches to low probability events should be consistent with ways of looking at the world and the problems we have now, like structural issues, power discrepancies, feelings of lack of agency of consumers who are forced to use tools that their boss tells them to. Those things have everything to do with thinking about geopolitics and biowarfare and AI disasters. And it must be consistent because if you've got some guy in a room in Silicon Valley saying, right, we're just worried about Star Energy actually killing everyone, and that's got nothing to do with teachers, so shush with your tiny problem there's something that's going really wrong there and it's probably not a useful way of thinking.

[Carl Miller]

And at the risk of forcing you to distill your entry and career, an entire massive global field, into a single aunt selling law. Of the many problems that you've been kind of taking us through and describing kind of climate bias, gender, jobs, inequality, poverty, which are the ones you would point to, people listening to this as the kind of the ones where we've got the most evidence now to really believe that they're kind of the kind of pointiest problems that AIs created or will be creating in the kind of

[Eleanor Drage]

short term, like which is the most important of the short term problems I can possibly say that the whole point I think of ethics is that it shouldn't be focusing just on one or two things, just like the charity sector shouldn't focus on one or two things. What charity do you give to once a month, if you give to a charity at all? Well, it's something probably that's concerned your family or someone that you know, a loved one or that concerns you from your trip to work and seeing hopelessness or we have to do. And you know, Mother Teresa wasn't perfect, but I really like what she said about we need to help one person at a time, starting from the person closest to you. Like focus on what you know and what matters to you, whether it's your child being subject to video, AI tools that you don't think are suitable for them because they have autism, or because they don't want to use a system that is analyzing their personality. And if we do that, we're way more likely to be successful. So I think that it has to all happen at the same time. And sometimes different strategies can be contradictory. For example, there was one approach to making facial recognition good for everybody that the two people who made it thought really would fix facial recognition and bias against women. For example, particularly black women who were often not seen by facial recognition. Their faces weren't observed as faces because there wasn't a lot of data on black women's faces that was going into these systems. And actually that was the case for a software that we had in the UK that HMRC built and then deployed without proper testing to help people upload photos of themselves instead of having to go and take a passport photo. And this project was called Gender Shades and it was great in many ways. And it was two engineers, Tenit Gebru and Joy Buolamwini, who went about trying to do it and they're great girls. I have so much respect for their work. But the criticism around that was, well, are you trying to make facial recognition better at observing the faces of people who are disproportionately likely to be targeted by facial recognition used by police, for example. So essentially you're making the technology better at doing worse stuff. And this is what ethics is. It's these headaches where you have two well intentioned parties working towards the same goal. But some people want short term integration and some people want long term structural upheaval. So it's not even a question of fixing particular things, because ethics is a game of whack a mole. You think like, oh, I've fixed that. And then some other side effect pops up, which is why Utopia has to be day by day and persistent and conversational and kind, because the world is really complicated.

[Carl Miller]

How kind of much of a grip do you think ethics has over AI and its development at the moment? Because there's a long running, fairly cynical criticism, which is that there's all these ethical frameworks and all these debates and lots of roundtables and lots of panels at hay, you know, that we'll always talk about the ethics, but the thing that really is going to shape and determine the world is what technology makes possible. And that's, as we said at the very beginning, essentially being conducted by a very small number of hyper capitalized companies, largely sitting in either Silicon Valley or China at the moment. Do you think that ethics is getting into those places in a meaningful way? Is it meaningfully shaping those technologies as we speak?

[Eleanor Drage]

I never had any background in ethics. I, you know, come from gender studies. We talk about power, we don't talk about ethics. And I think, you know, a lot of my colleagues were really over ethics in the formalized way that it's often used. We didn't even know what it was. You know, ask someone on the street, what is ethics? I think you'd think that you had to be Sophocles for a minute to a terrifyingly enormous and very haughty aim. So I'd been asking people for years on my podcast, what is good technology? Which I think is actually, it's a sort of pithy provocation, but a more sensible way of approaching it. And I love people's varied responses to what good is. And they get at the heart of ethics, but perhaps in ways that, that aren't really expected. And I tend to claim their responses as feminist, even though they maybe never thought about feminism ever. So, like this guy was talking about who developed a open license microscope, which means that anyone in the world can use it and take the license and then recreate it and sell it on. So that literally is the redistribution of, you know, of power, giving people a chance to build their own economies through open license software. And this is a really fantastic way of building and deploying technology, particularly technologies that are coming from academia where you have no expectation of earning money anyway. So it doesn't really matter. This to me is ethics. And we don't have to call it ethics, we don't have to call it good or feminist or whatever, call it whatever you like, whatever's acceptable and approachable to you. But the fact is there are people building open software systems in Latin America that, as you said, are not using Amazon Web Services. They are cultivating these technologies locally and they are making them useful and understandable to people in local areas. There are people in the Niger Delta making systems that can assess the pollution of different areas where they have unacceptable levels of pollution and they need to collect data to give to governments using very simple solar panels and other off the shelf computational parts that are meaningful to people. Would they define their work as ethics? I don't know, probably not. But this is great. So I think it's time to rethink ethics as shifting power, which is what I argue a lot in the book. Because every language has a word for power. Not every language has a word for bias. In fact, when I said to a lot of my French friends, you know, De biasing, and they looked at me as if I was a bit mad a few years ago, what does that mean? But we all know what power is and every. Yeah, every culture has a sense of shifting power and how AI can actually do that.

[Carl Miller]

I said we were going to talk about sci fi and I'd like us to end with that because I think it's such an interesting and often overlooked kind of element of all of this. You painted that scene of that erotic charged panel talking about hard science and hard sci fi at that convention. Is it a problem? Do you think that our kind of imaginative repertoire when it comes to AI is shaped by a kind of literature that's kind of like massively predates AI in many ways and probably imports into it all of these tropes that might not well be, well, descriptors of what AI actually could be, should be today? I'm thinking of Isaac Asimov thinking about AI so many decades ago now.

[Eleanor Drage]

Yeah, absolutely. I was on a panel with Stuart Russell, who's a much listened to computer scientist at The Southback center in London a while ago. And he was telling people that no one can imagine utopia. And this is nonsense. I mean, real nonsense. To me, that is propaganda of the worst kind. There is so much utopian literature out there. It's just not, as I was saying before, this perfect detached utopia where we've got everything right. It's, you know, utopias that are complicated and different and there's so much we can draw from, so much feminist literature. I did my PhD on it. There are, yes, there are like hundreds of books and articles and zines and blogs looking at different science fictional or speculative worlds where we have different configurations of machine and humankind that are not adversarial, they're integrated and they're interesting. And I just wish that we would read more of this stuff and that Elon Musk would stop publishing his reading lists. I'm so bored of this crap. I have nothing personal against Asimov. I think he was really great. I just wish that we would draw from more of this. And in fact, I've had to change my teaching at Cambridge to incorporate more of what students imagine as a positive version of the future as a kind of utopia because they were so miserable over the past few years and I felt people on the verge of tears that I had a moral obligation as a teacher to stop talking to them about apocalypse and start saying, okay, what would a good technology look like for you? Maybe not like a good technology that's good all the time, everywhere, but a specific good technology that is made from materials that are interesting and ethical and sustainable that's good just for you at a specific moment of time. Because it's not that we need to think smaller, but we need to be realistic and upbeat about what we consider to be good. Like, I don't know, my friend who has type 1 diabetes will say that a good technology is her blood glucose monitor, but only when it's working properly and the cap hasn't fallen off. And, you know, it's contingent on all this stuff. And if we can think more concretely, more materially, more personally about, you know, our values and what good technology could be for us, what we use every day we find useful. But also when that technology ceases to be useful and starts to become unpleasant, like our phones, right? This is the great phone crisis. Then we can start working towards stuff that we actually want.

[Carl Miller]

And if people are currently thinking right now as they listen to this podcast of a good piece of technology, what might they do? Eleanor, is it to write about it? Is it to Go somewhere, read a resource. What might people's next step be to try and get that out into the world?

[Eleanor Drage]

I would use it to compare against AI tools that they're being asked to use or exposed to. And then that gives you a really good basis for whether you think that tool is appropriate or not. But also I think it gives us a voice to say, okay, my values are using something that is useful to me, but also that I understand that I can change, I can hack it, I can change the features and compare that against our phones that are designed to be impenetrable. The iPhone is designed not to be able to lift the hood and see what's underneath. And so are many of the technologies that we're being asked to use. And then from there we can go to work and have a better sense of what it is about these technologies that we're exposed to day to day. That feels uncomfortable to us because actually, I think the first step is just pinpointing what the issue is and how these technologies relate to stuff that we do enjoy on a daily basis, whether it's your favorite mug or in my case, my podcasting software or whatever, whatever it is.

[Carl Miller]

Well, hopefully we have done that first step today and pinpointing some of those issues and even foregrounded our favorite muggle or versions of podcasting software. So, Eleanor, thank you so much. That was Eleanor Drage, everyone. She's the author of what if We Got AI how to Stop Catastrophizing and Build an Ethical Future. I've been Carmilla, you've been listening to Intelligence Squared. Thank you so much as ever for joining us.

[Mia Sorrenti]

Thanks for listening to Intelligence Squared. This episode was produced by Conor Boyle and it was edited by Mark Roberts. For ad free episodes and full length recordings, you can become a [e-mail] membership. If you'd like to join us at future live events, you can find our full program or buy [e-mail] forward slash, attend. You've been listening to Intelligence Squared. Thanks for joining us.