← The Interview
The Interview
Mustafa Suleyman, Artificial Intelligence pioneer: people should be healthily afraid of AI
00:22:59
Amol Rajan speaks to Mustafa Suleyman, the British Artificial Intelligence pioneer
Loading summary
Transcript52 lines
- [00:00]
Mustafa Suleiman
This BBC podcast is supported by ads outside the uk.
- [00:04]
Amol Rajan
Hello, I'm Amol Rajan, BBC presenter and this is the interview from the BBC World Service. The best conversations coming out of the BBC people shaping our world from all over the world.
- [00:17]
Unknown Speaker 1
Today we are spending trillions on war and peanuts on peace.
- [00:20]
Unknown Speaker 2
Wind power in the United States has been subsidized for 33 years, isn't that enough? Solar for 25 years, that's enough.
- [00:27]
Unknown Speaker 3
I don't have army, I don't have missile rockets, I, I have my body, I have my voice, I love singing and so my goal was always to do better and better at it.
- [00:37]
Unknown Speaker 1
I was still in an induced coma in hospital when the world was defining me.
- [00:43]
Amol Rajan
For this interview I met Mustafa Suleiman, the British artificial intelligence entrepreneur and boss of Microsoft AI. You're going to hear why he believes we should all be healthily afraid of artificial intelligence and how it must be developed so it can always be controlled by humans and not vice versa. The son of a nurse and taxi driver, Mustafa Suleiman was in his mid-20s when he co founded DeepMind, a pioneering artificial intelligence startup which brought together machine learning, neuroscience, engineering and mathematics. Four years later it was sold to Google, reportedly for $400 million. Today he remains a self proclaimed techno optimist, seeing the ongoing potential of AI as a force for good in how we live and work. But he is clear eyed about the risks that this increasingly powerful technology poses to life as we know it. Welcome to the interview from the BBC World Service with Mustafa Suleiman.
- [01:45]
Mustafa Suleiman
I want to put all the issues on the table that keep me up at night and be direct and clear eyed about them. And I honestly think that if you're not a little bit afraid at this moment, then you're not paying attention. And I think that fear is healthy and necessary and being honest that I feel it as somebody who's like deeply techno optimistic and passionate about the field and has done a lot to accelerate the direction that we're heading in. That feels critical to share that and invite other people to be also healthily afraid and skeptical because I think that needs to drive the action that we all need to take.
- [02:26]
Interviewer (possibly Amol Rajan or another interviewer)
There is a note of skepticism some people have where they say, well this AI stuff's all being hyped up by people who are very, very rich, live in Silicon Valley in their own little world, and they're all making this up to boost their own stock prices. Actually if you look at what's happened over the past 10 years, AI has exceeded expectations all the way, hasn't it?
- [02:43]
Mustafa Suleiman
For sure. I mean, the amount of compute used to train the best AI models of any year over the last decade has increased by 10 times every year for basically 12 or 13 years. And the strange thing about exponentials is that as humans, we have no intuitive grasp of an exponential. You have to sort of, you know, think through the maths like there's no, there's nothing in our everyday experience that gives us an instinct for what that is. So the first 10 doublings or increases in this case by an order of magnitude, feel just flat. Right. Nothing really worked between 2010 and 2020. There were a few cool successes in games and we did obviously AlphaGo at DeepMind and small scale things in hindsight. But by 2021, 2022, the same core general purpose methods started to work for language, which is the most complex tool we've ever invented as a species. And so the idea that these AI systems could learn the structure of language well enough, not to just to predict one sentence, but to predict multiple paragraphs and then have perfect stylistic control of that output, I mean, that was kind of mind blowing. And that's why it's felt like a step change.
- [04:02]
Interviewer (possibly Amol Rajan or another interviewer)
And it's the generation of new ideas that is a change, isn't it? Because artificial intelligence, as I understand it, is the science of teaching machines to learn. But what's the thing that AI has done in the past 10 years that it couldn't do before because AI has been around for a while.
- [04:16]
Mustafa Suleiman
Well, so the real quest of AI, in my opinion, is to try to produce new knowledge. At the moment, AI does a pretty good job of reproducing existing knowledge and that's very, very valuable and is clearly going to be deployed across, everywhere, all at once in the next decade, blah, blah. But really we want it to tell us something that we don't actually know. And I think that there's a few examples of that in the last sort of three or four years that are surprising. The first is everyone said that it would never be any good at empathy. It could never make you as a human feel understood, feel heard, feel deeply connected, feel seen in some way. And actually it does that incredibly well. Companionship, listening, support is one of the main use cases that people turn to copilot or Claude or ChatGPT. The second is it's actually quite creative. You're generating an image that has never existed before and it's combining multiple ideas. Maybe it's like a dragon with wings that are made in perfect purple with some kind of cyborg eyes on the front. I don't know. And it feels trivial. We're all like, oh, we're so boring. It's AI slot. We've seen it all before. But actually that is evidence that it can tell us something that we don't know. It's interpolation. Interpolation is a prediction between two points or between two ideas where the underlying data doesn't actually have that third data point in its history. And that feels very, very encouraging because if you can apply the same set of methods not just to gener images or to videos or to text, but generating, you know, the next sequence in DNA or in any time series data, then you could potentially produce synthetic molecules, or you could produce entire academic papers from scratch, including all of the proofs that you might want to go test in wetware and actually run the real experiments in the biology lab. So I think this is definitely happening. It's very general purpose, it clearly applies to many, many domains. And in the next five years we're going to see outrageously exponential increases in productivity and efficiency because of it.
- [06:25]
Interviewer (possibly Amol Rajan or another interviewer)
What are we going to see in the next five, 10, or 20 years that concerns you, including in the broad umbrella of what we call super intelligence?
- [06:33]
Mustafa Suleiman
I think the challenge is that these are fundamentally labor replacing technologies. Now people have argued over this for the best part of a decade and many other previous waves. They've always pointed out that when there has been labor displacement, new jobs have been created. In this case, it just isn't true that long term the same thing that has displaced a white collar worker because it can now use an operating system and a spreadsheet and send emails and make phone calls. That is a very general purpose. Almost like project manager or strategist or HR person or marketing person. Those skills are actually quite predictable and quite automatable. And so there's going to be huge efficiencies because actually operating like a white collar knowledge worker is going to become cheap and abundant. Like anyone who wants to start a business or run an institution is going to have a team of AIs that can work 24, 7 that will be highly accurate and actually very steerable and controllable.
- [07:37]
Interviewer (possibly Amol Rajan or another interviewer)
And they don't take lunch breaks or holidays or have sick pay. Just to be really clear on the jobs thing, obviously this is about the future, so it's unknowable in some sense. But it's important to say that this is already happening in some places. There's an influential academic paper in America. We've seen big announcements from the likes of McKinsey in Salesforce. To be clear, in terms of what we've seen so far, which other jobs I think you're saying, particularly entry level white collar jobs which are likely to be displaced. And do you have any sense of the time frame?
- [08:03]
Mustafa Suleiman
Call centre workers? I think it's already starting to happen because they follow a very strict decision tree to solve kind of repeatable problems. But then the next is going to be things like paralegals that are synthesizing past cases or summarizing contracts or, or sort of more junior accountants that are sort of processing vast amounts of data and trying to simplify or organize those things. But soon, I think in the next two or three years there are going to be general purpose project managers that you can deploy at any task. This is the mind blowing thing. Today you can vibe code so in natural language prompt an AI to build an entire working piece of software for you with no technical expertise. And that's remarkable because we pay software engineers some of the highest salaries of anyone in the labor market. And that is going to become entirely commoditized, cheap and abundant, available to everybody.
- [08:56]
Interviewer (possibly Amol Rajan or another interviewer)
How should people prepare, and young people in particular prepare for the world you're talking about? What should the young Mustafa's be doing?
- [09:04]
Mustafa Suleiman
So the first thing to say is that if it's not already clear, technology is political. And I think Silicon Valley told itself a story not through malice or conspiracy theory, but just because of a weird, I don't know, belief that somehow technology was neutral and it was a tool and all that matters is how the human uses the hammer. The hammer didn't do anything wrong, but it's a terrible metaphor because the design of that technology is so much more intricate and complex than just creating a hammer. Right? We are now creating emergent, interactive, highly personalized experiences. Their design carries with it a whole bunch of political and ethical choices about what we choose to amplify or not. The thumbs up like button in Facebook is obviously the classic example or hearting something on Instagram. These incentives drive macro behaviors at huge scale. So to me the first step is to recognize that technology is political and that it carries huge ethical weight. And so then the second thing is asking yourself, well, I have to participate in this. I have to understand it well enough, I have to use it to be able to shape it in the best interests of everybody. And then the third is to declare that we actually care about human well being more than anything else. And that seems like a trite or obvious thing to say, but there are plenty of people in the industry today who see a world that in fact desire, a world in which machines get so much more capable and intelligent than humans. Not just one human, but all of us put together that they could exceed human performance on all tasks. A system like that would almost certainly not be controllable. And so we have to declare our belief in a humanist superintelligence, one that is always aligned to human interests, that works for humans, that makes the world a better place. And I think that's the political project that everyone is about to be completely enwrapped in as we get these really powerful tools.
- [11:02]
Interviewer (possibly Amol Rajan or another interviewer)
But one of the things that defines the human is human differences. And if you're going to pursue humanist AI, who gets to define what's humanist?
- [11:13]
Mustafa Suleiman
I think that is the big challenge.
- [11:15]
Interviewer (possibly Amol Rajan or another interviewer)
Is it Mustafa Suleiman who works at Microsoft? Is it the people in China that you see when you, because you spend a lot of time there? Is it the Mongolian pig farmer? Who is it?
- [11:23]
Mustafa Suleiman
The first wave is clearly going to be defined by people who build it at the top of the companies, right? I mean, that's just a practical reality. We're building this software and we're putting it out, but it is also incredibly general purpose. So it is a new clay that everyone gets to make with. And so you get to sculpt the values of the agent that you build. Like what does it believe? How does it form relationships with other people? What capabilities does it have? That is a very exciting prospect because it's never been easier and cheaper and simpler to produce something of immense capability.
- [12:00]
Interviewer (possibly Amol Rajan or another interviewer)
If you're advocating for humanist AI, you mean humanist AI as opposed to what? What's the thing that your humanist superintelligence, which you're advocating for, is almost defining itself against human superintelligence instead of what?
- [12:13]
Mustafa Suleiman
Well, some people are building an AI that is designed to explore the limits of the universe. To do that, it has to have, you know, superpowers unlike anything that we could imagine and I think would be near impossible to contain or control. So if it is designed inherently to self improve, to set its own goals, to operate with complete autonomy, those are three capabilities which to my mind look like we couldn't control it. If we can't control it, it isn't going to serve humanity, it isn't going to be on our side, it's going to overwhelm us. We have to have that conversation.
- [12:50]
Interviewer (possibly Amol Rajan or another interviewer)
You want to bake those principles in by design?
- [12:53]
Mustafa Suleiman
It has to be a core property of any future development that we can provably contain and secure and align it. Contain means to stop its spread that isn't under our control. Secure it means that it can't be hacked or leaked. And align means that it has to behave according to a set of values that we can feel confident it will continue to behave around.
- [13:15]
Interviewer (possibly Amol Rajan or another interviewer)
And those values will be determined by the people who are doing the designing.
- [13:17]
Mustafa Suleiman
At the moment, I think. At the moment, yeah. But I think that this is the conversation that we have to have as a, as a species as we go through this transition. Like, what do those humanist values mean?
- [13:30]
Amol Rajan
You're listening to the interview from the BBC World Service. People shaping our world from all over the world.
- [13:41]
Advertisement Voice
Why pay when you can slash TikTok slash and free lets you cut prices all the way to zero. Download TikTok, search/free and share the link to get free items. Start slashing now.
- [13:57]
Amol Rajan
For this episode of the interview, I'm speaking to Mustafa Suleiman as one of the Christmas guest editors for Radio 4's Today program in the UK. He is an unusual figure in Silicon Valley. I've met and interviewed lots and lots of AI bosses, from Eric Schmidt and Sundar Pichai of Google to Dario Amade of Anthropic, and a lot of them think the same way about some of the big problems of the world. But Mustafa Suleiman is rather different. He is, to put it bluntly, something of a North London geezer. He dropped out of Oxford in the wake of 911 to set up a secular helpline for Muslims in Britain. Like him, he's very interested in philosophy and he spent a short time working for the former mayor of London. Mustafa Suleiman has been open about his progressive politics and his absolute determination to make sure that humans and human needs and human interests are at the center of any ethical discussion of AI. Let's return to my conversation with Mustafa Suleiman.
- [14:56]
Interviewer (possibly Amol Rajan or another interviewer)
Can I tell you what I think? A lot of our listeners would think one of the best things about being human is, is other humans. And Mark Zuckerberg says something on a podcast which, how can I put this, alarms some people.
- [15:07]
Amol Rajan
The average person wants a lot more.
- [15:09]
Interviewer (possibly Amol Rajan or another interviewer)
Connectivity than they have.
- [15:11]
Amol Rajan
They feel more alone a lot of.
- [15:13]
Interviewer (possibly Amol Rajan or another interviewer)
The time than they would like.
- [15:14]
Amol Rajan
I think people are going to use AI for a lot of these social tasks.
- [15:20]
Interviewer (possibly Amol Rajan or another interviewer)
Does that pass your threshold of amplifying rather than replacing what it is to be human?
- [15:26]
Mustafa Suleiman
I think at the moment we're totally overwhelmed by information and complexity. Like, I think that the digital world has bombarded us with a visualization of everything, all the differences, all the Inequality, all the beautiful things. And we're kind of like struggling to process, process that in our biological experiences. And I think more digital connectivity is probably just going to make that anxiety and confusion even greater. I think that what we need is more free time to exist in the real world with one another as humans.
- [15:55]
Interviewer (possibly Amol Rajan or another interviewer)
And not to replace the social human bond, that is the human to human bond, which is one of the best things about being human. I mean, what Zuckerberg is talking about there, and honestly don't want to misrepresent him, but he's talking about chatbots being a cure for loneliness. But the best cure for loneliness is other people, isn't it? Not chatbots, which are simulating other people.
- [16:15]
Mustafa Suleiman
Well, I think loneliness is the symptom, not the cause. And so chatbots can help investigate and support the cause. And I've got no problem with that. I think that's very healthy. Finding a non judgmental, always available, very supportive, you know, sort of listening ear to explore your history and your issues, I think is very valuable to people.
- [16:37]
Interviewer (possibly Amol Rajan or another interviewer)
But that sounds like a therapist, right? Is that kind of therapist which is using compute, not in danger of encouraging the human, the perhaps vulnerable, perhaps lonely human, to develop an emotional connection?
- [16:50]
Mustafa Suleiman
I mean, the other way of thinking. I mean, yes, it will develop. And we are seeing people develop emotional connections to them. And I think that's what we have to be very wary of and measure and talk about. We just published actually at Microsoft AI, one of the largest investigations into AI conversations that has ever been released, 35 million conversations. And you can see a full taxonomy of the types of things people talk about and the risks of dependency and psychosis and stuff like that, which we're starting to learn. But I think it's also important to say that these experiences, these chatbot experiences give people a way to detoxify and show up in the real world with their friends and families and loved ones having kind of spent time talking about the anger or the frustration or the fear. And that isn't always available to everybody. In fact, it rarely is. Right. So it is definitely providing a service, a good service. But we have to also just think about what those revenge effects are likely to be, those second and third degree consequences.
- [17:50]
Interviewer (possibly Amol Rajan or another interviewer)
You had a conversation between two of the leading global authorities really, on this question of whether or not AI is conscious. Do you take a view yourself whether or not AI can be capable of consciousness?
- [18:02]
Mustafa Suleiman
I think that it can't. I think that's pretty clear. I think consciousness arises in biological beings as a result of a very specific biological network that triggers pain and the awareness and experience of pain as a result of external stimulus. And that pain is manifested in biology, right? Whereas these AIs, of course, they do absorb stimulus, they learn things, they see things, they interact with text, but nothing in their empathy circuit, reward circuit, decision making circuit, nothing is triggering a pain experience. And to me, pain and suffering is the essence of the conscious experience. So an AI will definitely have a lot of the hallmarks of consciousness. It will have memory, it'll be able to talk about its past and itself. It'll be able to absorb streams of interactive vision and so on, and language. But that doesn't mean in any sense that it actually is conscious. And the reason this is so important from a humanist superintelligence perspective is that if we allow anyone to declare these things conscious, they will quite quickly make the claim that they deserve rights. We cannot spawn a new species of conscious beings that have a right to not suffer or to not be turned off. And there's an entire subfield in AI at the moment of philosophers and ethicists who are starting to think very seriously and actually even make the claim that AIs should have moral consideration, is the phrase they use. A lot of them come from animal welfare. And I think that this is first, not only there is absolutely no evidence of it, and it is almost certainly not going to be the case because of what I've just said about biology. But even if it were, it would be super undesirable and something that we should resist. Like, we do not want these things to be conscious. Let's just start there. Whether they are or they aren't, there should be widespread collective agreement that this would be completely undesirable. How on Earth if we could spawn billions of digital minds at zero marginal cost that could self replicate and learn from infinite amounts of data relative to what a human could learn. How on Earth could we possibly control and contain something like that?
- [20:09]
Interviewer (possibly Amol Rajan or another interviewer)
You've talked about the principles that should govern more humanistic AI and superintelligence. What does smart regulation in this new world look like?
- [20:17]
Mustafa Suleiman
I think smart regulation should be inspired by the precautionary principle. For all of human history, we have been focused on unlocking science and technology as fast as possible. We want these breakthroughs. We have to get them out there. They proliferate because they get cheaper and easier to use once we've invented these things. But this is a different moment in history, and we have to accept that this time really is different. In the next 10 or 20 years, I don't know when it's going to happen. But these are not traditional tools. These are much closer to living beings that really do learn on the fly, that can absorb way more information than any one of us individually that can update and improve themselves through interaction. And so it's qualitatively different and it deserves a different approach. And this is obviously not a traditional Silicon Valley view, but it's a moment when we have to raise the bar if you like of do no harm.
- [21:10]
Interviewer (possibly Amol Rajan or another interviewer)
Right?
- [21:10]
Mustafa Suleiman
That should be what we expect. We want these things not to just do net good on balance, but we want to do much closer to universal good and to minimize or even eliminate harm.
- [21:21]
Interviewer (possibly Amol Rajan or another interviewer)
But with all due respect, even if it's a noble argument you're making, aren't you. I say it's respectfully because you're trying to do it. But aren't you losing the argument in that? Isn't exactly the opposite happening? Isn't the dynamic we have today a race dynamic? The wealthiest companies in the world versus People's Republic of China. Isn't it completely a race towards AGI?
- [21:40]
Mustafa Suleiman
I think that so far we have seen very little harm as a result of these LLMs. I mean there has been harm and obviously that's like very sad and bad and we have to learn from it. But I'm not talking about this sort of micro moments. I'm talking about large scale destabilization of our political system because there is mass unemployment or large scale destabilization because. Because anybody with very limited resources can spawn entire armies of appearingly seemingly conscious AI systems that can self improve and learn and act autonomously. We're really Talking about a 10 to 20 year in the future major threat. And I think that so far we haven't seen any of that. And yes, you're right, there is a massive race condition. I mean China is obviously going full bore at this and so are many, many other companies. And if we carry on behaving the way that we have with respect to governance and regulation over the previous, you know, sort of post war era of the last 70 years, then we will end up shooting ourselves in the foot in a major way. We have to do something very different to what's been happening in the past.
- [22:52]
Amol Rajan
Thank you for listening to the interview from the BBC World Service. You'll find more in depth conversations on the interview wherever you get your BBC podcasts, including episodes with Venezuelan opposition leader Maria Corina Machado, author Sir Salman Rushdie and former New Zealand Prime Minister Jacinda Ardern. Until the next time. Goodbye for now.