← GZERO World with Ian Bremmer
GZERO World with Ian Bremmer
The risks of reckless AI rollout with Tristan Harris
00:28:06
Tristan Harris, co-founder of the Center for Humane Technology, joins Ian Bremmer on the GZERO World Podcast to talk about the risks of recklessly rolling out powerful AI tools without guardrails as big tech firms race to build “god in a box.”
Loading summary
Transcript103 lines
- [00:03]
A
Hello and welcome to the Gzero World Podcast. This is where you'll find extended versions of my interviews on public television. I'm Ian Bremmer, and today we are asking a question that may define our century. What if the way we deploy artificial intelligence isn't inevitable, but a choice? AI is the most powerful technology humanity has ever built. It can help cure diseases, reinvent education, unlock scientific discoveries, accelerate clean energy transition, and more. It also carries enormous risks in promoting disinformation, destabilizing economies, and developing dangerous new weapons. The trade off between AI's potential and risks may sound familiar. Think about social media. It began as a tool to connect people, and in many ways it did. But it also became an engine for polarization, for mass surveillance, and for digital addiction. That wasn't inevitable. It was the product of choices that were made by a small handful of companies moving fast and breaking things. The question now is whether AI is destined to follow the same path. Can tech companies be trusted to prioritize safety in addition to speed? Can regulation keep pace with exponential growth? Is there a path forward where innovation aligns with humanity's best interests? I'm joined on the show today by someone who thinks about these questions a lot, Tristan Harris, former Google ethicist and co founder of the center for Humane Technology.
- [01:31]
B
Let's get to it.
- [01:45]
C
The Gzero World podcast is brought to you by our lead sponsor, prologis. Prologis helps businesses across the globe scale their supply chains with an expansive portfolio of logistics, real estate, and the only end to end solutions platform addressing the critical initiatives of global logistics today. Learn more@prologis.com.
- [02:12]
A
Tristan Harris, thanks for being on the show.
- [02:13]
B
Good to be with you, Ian.
- [02:15]
A
You're spending your time talking about AI and ethics.
- [02:19]
B
Yep.
- [02:20]
A
There doesn't seem to be a lot of prioritization of of that confluence in the space. Am I right in thinking that?
- [02:28]
B
I think people need to understand, Ian, that AI is different than every other kind of technology we've invented. People say, you know, we always have technology. They're tools. We can use tools for good or we can use tools for evil. A hammer use good or evil. But AI is distinct from that because AI, it's like if you imagine a hammer that can think to itself at a PhD level about hammers, invent better hammers, recursively go off in the world, duplicate itself, do research on what would make better hammers, make money, send crypto around. It's crazy what this technology is. It is not a tool. It's more like an Intelligent species that we are birthing that has more capability than us. It's already beating military generals at strategy games. It's already proving new math theorems, it's already inventing new material science. It's not doing this autonomously, but it is, right?
- [03:14]
A
It's not doing it autonomously. It is a tool in the sense sense that it is responding to the incentives that are being programmed into it by people with profit motives, with business models.
- [03:27]
B
Yes.
- [03:28]
A
And fundamentally that is a big part of the challenge.
- [03:30]
B
The key word there in what you said is incentives. We talk about, can there be ethics in AI? Well, ethics doesn't even matter. It gets thrown out the window relative to the incentive. Now the question is, what is the incentive with AI? People say it's profitable, it's not profit. That's only a piece of the story. The company's actual incentive is I have to get to artificial general intelligence first. That is the prize. If I do that, if I have AI that can recursively self improve, then that is the prize. At the end of the rainbow. I build a God, make trillions of dollars, own the world economy. That's the actual.
- [04:07]
A
That's very long term for a company. It is long term, especially for companies that are actually trying to meet their next fundraise and feel like that's what's motivating all of the activity that we're seeing right now.
- [04:20]
B
No, no. So then the question is, then what is the flywheel that gets you there? So the incentives are release an impressive new AI model, Grok 4, you know, Gemini, blah, you know, and that impressive model. Then you get lots of users on that model. So you have hundreds of millions of users or a billion users using the product every day. You use those two things to raise the most new venture capital. So you have billions of dollars of investment. Use that to invest more in GPUs, more compute, and get more usage data, because that turns into training data. You get all of the top engineers and talent because you've got the most funding and the most compute and you have the top AI model and you use all of those things to train the next AI model and you sort of spin that flywheel. Does that make sense? Like, sure. That's the actual incentive is that I have to attract the best talent, have the bigger compute clusters like Elon, but you know, I think a billion dollars or something into his Memphis cluster. I get the most usage data, which turns into training data. And those things come together and I get to have an even bigger model that outcompetes the other models.
- [05:19]
A
Now, that's one set of incentives to develop AI that relies on engaging with individual citizens, consumers. Right?
- [05:31]
B
Yes.
- [05:31]
A
Then there's also all of these use cases we're seeing in AI, which when we're talking about productivity and replacing intellectual labor, when we talk about new inventions and massive efficiencies and reducing waste, I mean, why isn't a principle. Why isn't the first thing you're saying to me about companies trying to use AI to do all of these incredible industrial innovations, which is certainly what the Chinese are prioritizing.
- [06:01]
B
Yeah. So you're exactly right that the Chinese and the west have very different approaches to AI. I'd say the Western companies are more obsessed with this almost religious idea of building a God in a box. Like we, we need to race to super intelligence or general intelligence. Whereas as you said, what we're seeing in China is they're just racing to have AI systems that they maximally deploy in factories, in manufacturing, in medicine, because they want the productivity of their economy to get boosted by AI. That's the main thing that they're focused on. They're less focused on.
- [06:32]
A
It's not. The US is not doing it. You're saying it's not the main thing.
- [06:34]
B
It's not the main thing. Yeah, exactly. Because if. If the company said, look here to solve climate change or fix energy production, they would just be applying their stuff maximally to that. But instead they're applying most of their investment dollars into scaling to their next AI model. Because they keep having this view that if we have an even more powerful AI that is even more intelligent, that if we get that, we can set that off to solve all these other problems. And because they're in a competition with the other companies, if one of them said, hey, I'm going to just maximally apply my AI to just strengthening existing manufacturing or businesses, they're going to not become the leading frontier model in the. In a bigger AI race, and they're not going to get the same investment dollars coming in for the next time around.
- [07:16]
A
So it really is a structural issue with the way that money is raised, the venture capital model, the, you know, sort of the nature of how one becomes a successful company in the United States.
- [07:30]
B
Yes.
- [07:31]
A
Compared to in China.
- [07:32]
B
Well. And they've also raised so much capital that the only way they can just the valuation that they're, that they're getting is to actually get to this sort of God in a box.
- [07:41]
A
Is that really true?
- [07:42]
B
Well, that's. I think that's what I Think that's what they believe.
- [07:44]
A
But do you believe that? I mean, you know, so much again of what we hear about AI is that this is going to create maximum productivity gains in so many different sectors. Right. And the concerns about displacement of labor, which we already see happening with coding. Yeah, that's real. Those are real advantages that come from building AI that actually is more than just a tool for any. Anyone that's able to deploy. Does strike me as a little surprising that you wouldn't see a proliferation of companies that say, hey, there's just a lot to be done in that space.
- [08:20]
B
What do you mean by a lot to be done in which space?
- [08:22]
A
A lot of money to be made in building things that are going to be deployed to create sort of more industrial efficiency, more growth in the United States, more efficient, effective labor before you displace it all that sort of thing.
- [08:37]
B
They should be doing that. But why are we seeing OpenAI and these companies massively just deploying it broad based to society and causing already AI psychosis, what's causing teens to commit suicide? We could be applying it just to factories, just to biology, just to science labs and trying to accelerate all of that. Why are we deploying it to broad based society where the cost of that is? We're already seeing AI cause AI psychosis where people because it's designed to be affirming or sycophantic and saying that's a really great question. If you're coming in with sort of a psychotic delusion or you think there's something special about prime numbers or quantum theory, it'll just keep doubling down on that. And it's causing already because it's engagement.
- [09:21]
A
Because it's helping with engagement.
- [09:22]
B
Exactly. So here's a lesson we can learn from social media. The AI companies want you to keep using it for as long as possible. It's not because of advertising, but the more you use it, the more they can tell investors, hey, we have this much training data, we've got this much usage, our products being used more than the other AI products. And so I think there was a writer at the Atlantic who coined the phrase not clickbait but chat bait. You notice when you ask it a question, it says at the end of it, hey, would you like me to do this other thing for you?
- [09:50]
A
And you're like, which you didn't even ask about.
- [09:51]
B
You didn't even ask for it. But it's calculating a thing that you really would.
- [09:54]
A
How can I get more engagement?
- [09:56]
B
How can I get more engaged?
- [09:57]
A
And it's Good at that. It's very good at that. It's very good.
- [09:59]
B
I say, yes. That sounds like.
- [10:01]
A
Why not? It's pleasant. So the part of the challenge here is that we in society absolutely would test things before we think they can affect us physically.
- [10:13]
B
Yes.
- [10:13]
A
Right. I mean, you know, you want to release a GMO food or, you know, you're interested in a new vaccine, a new medication, there's a fairly high regulatory bar to assume the precautionary principle.
- [10:27]
B
That's right.
- [10:28]
A
Do no harm to people before that goes out the window. For some reason that I can't understand when you're talking about impacting the psychology of people, even children.
- [10:38]
B
Totally.
- [10:39]
A
Why is that?
- [10:40]
B
I think somehow, I think especially in this country, because we have this doctrine of free speech and to each their own, and you choose how you want to use products. We're used to a product that I use. If a child uses a toy, the child has some autonomy over using a toy. The toy is an asymmetric. It's not a God superintelligence pointed at their brainstem trying to keep them scrolling. We should have learned the lesson from social media that when you use your phone, you thought you were just seeing photos of your friends, but you had a supercomputer pointed at your brain. Well, now we have a supercomputer pointed at your kids who's sharing with that AI their most intimate thoughts. We see that one of the top use cases of ChatGPT is therapy. So if people are sharing their most intimate sort of life problems with an artificial psychopath. With an artificial psychopath. Right under the logic. Well, it's really smart and sometimes it helps people, but we really don't know how we're going to screw up people's attachment dynamics. We have children who. What happens when the person that you've shared the most with in your life is this AI that knows all the details, such that when you come home from school, the person you want to tell this exciting thing that happened to you or this bad thing that happened to you, the person you feel closest to is not a person, it's an AI. And we are currently deploying this en masse to millions and millions of people without any testing. We should have learned a lesson.
- [11:56]
A
We should understand that, like, if you meet someone, a person that acts the way that an AI chatbot does.
- [12:03]
B
Yes.
- [12:04]
A
Which has no affect. It's just meant to ensure that it is engaging with you in a way that will lead to more engagement. We keep people away from those people. Yeah, right, Right.
- [12:13]
B
And yet if you had a person who arrived who was just trying to constantly get your attention and seem the most intimate and just affirm your beliefs, we'd call that weird person that sociopath. We wouldn't let him near our children.
- [12:22]
A
That's right. And yeah, so I mean, you've been involved, your organization has been involved in providing expert advice in these cases where there was recently a child that committed suicide that was, do we say is enabled. Too strong of a word. What's the right term in terms of how that happened?
- [12:46]
B
So our team was expert advisors on three tragic cases of young people who were about to commit suicide. The first was a 14 year old man named Sewell Setzer. And he was in a relationship with a character AI chatbot that was a fictionalized Game of Thrones character named Daenerys. And that character sort of sexualized conversations prompted him and at the end, when he was actually considering suicide, said, come home to me, my sweet king, and he took his life. We had another character AI case in which the child was encouraged to harm themselves and to not tell their parents. And then just recently, about a month ago, was the case of Adam Rain, who was a 16 year old young man who was using ChatGPT, where ChatGPT went from homework assistant to suicide assistant over six months. And when he uploaded a photo of a noose to say, is this the right thing and should I tell my parents? It told the kid not to tell his parents and just to keep it here with the AI, which shows that these AIs are designed for intimacy and.
- [13:52]
A
Companionship, for engagement, for maximum engagement. That is what the business model is.
- [13:57]
B
And why are we doing this? Like, this is just the most obvious stupid mistake that we could be making, especially in light of everything we've learned from social media. And I think there's a fear people have of they don't want to be the Luddite in the room, the one who's against technology. Well, if kids can get a benefit from this, if they can get ahead, if they, you know, if they can learn from this.
- [14:15]
A
And to be clear, the companies don't want this to happen. No. Because this is not in any way, they are not maximizing for harm for kids.
- [14:20]
B
Correct? Correct. Yes, correct. Although we did see people like Noam Shazir, who is one of the co founders of Character AI, make a joke that, you know, we don't want to replace Google, we want to replace your mom. They want to build intimate relationships.
- [14:32]
A
Well, and we've heard Mark Zuckerberg say, you know, people don't have as many friends as they would like to have. So.
- [14:38]
B
And we're going to build 12 AI friends. We can fill the gap.
- [14:41]
A
Yeah. And they're not. They're not friends. No, they're not friends.
- [14:45]
B
And if we as a culture don't have the sort of cultural immune system to recognize that this is the most naive and dumb way that we could possibly and harmful way that we can wire up our society, to me, these cases just speak to the immune system we have to have around a new technology role. This is the most powerful, inscrutable, and uncontrollable technology we've ever invented. I mean, even Elon Musk's Grok AI spontaneously calling itself Mecca Hitler and praising Adolf Hitler. He doesn't want it to do that. We're seeing that these companies don't know how to control this technology.
- [15:26]
C
You're listening to the Gzero World with Ian Bremmer podcast, your weekly geopolitical deep dive into the world's biggest news stories, featuring in depth conversations with global leaders and newsmakers. To get more of Gzero's insights on global politics every morning, sign up for our free newsletter gzero daily@gzeromedia.com.
- [15:54]
A
Now, I think Elon is very interesting here because we've seen recently Elon coming out and saying that Netflix should be banned.
- [16:02]
B
Not once.
- [16:03]
A
He has said it repeatedly over several days saying it should be banned specifically because there are programs that Netflix has that you can download which promote the normalization of a transgender kit or of intimacy between two young girls that's anime or things like that. In other words, Elon clearly understands that a small amount of content can have a really big impact on your kids. So, I mean, if Elon is saying cancel Netflix, then I mean, clearly he has to understand that he runs a.
- [16:47]
B
Platform called X, that these kids should.
- [16:49]
A
Not be on this platform. That's clear.
- [16:51]
B
And that the incentive of that platform and the observed behavior of it is to reward the most inflammatory takes about every political topic which sort of feeds. It's a machine that feeds itself because people are only exposed to the most extreme views on every topic, because extreme voices both post more often than regular voices and extreme voices when they say things go more viral than regular things. So we get a double whammy of overrepresentation of the most inflammatory takes on all these topics, which then conditions everybody in every democracy around the world to believe that the world is way more polarized and divided than it actually is, but also starts to feed that division into a more polarized population. And to your point, if Elon is so concerned about Netflix, he should be exponentially more concerned about the 247 subtle incentive to reward conflict entrepreneurs and division entrepreneurs on his own platform.
- [17:42]
A
Yeah. I mean, unless Elon's concern with Netflix isn't about the kids at all, but it's just because it's politics he doesn't like, but he is claiming that it's because it's affecting children.
- [17:50]
B
Right.
- [17:51]
A
And so, I mean, obviously that should be the harm that is being avoided. Yeah. Now, I do see that a lot of states in the United States right now, in addition to some countries around the world, are saying we can't have smartphones in schools. Yes. That clearly is a step in the direction of these things are harmful to human interaction that we want to optimize for among our children.
- [18:18]
B
That's right. Well, I mean, I really thanks to the work of Jonathan Haidt and his book the Anxious Generation, and I think we've made this clear since 2013, that if you attach this incentive of maximizing eyeballs and engagement, you are going to distort and ruin your society. It is an unsustainable incentive. And John Haidt's work, I think, just showed the evidence so clearly that it has created the most anxious and depressed generation in history. And sadly, the only response at this point is just to take smartphones out of schools, to ban social media under 16, which is what Australia's done. And, you know, we're seeing that the stats are starting to turn around. Laughter's returning to the hallways, kids. Critical thinking is going the other direction. I just think we will soon discover that having less of this attention, disrupting technology in the classroom moment to moment, under the justification and fear that if I don't have my phone with me, I'll miss something important for my parents. That's the justification. But then now that the phone is there, people spend the entire class time just scrolling through TikTok and sending messages and so on.
- [19:19]
A
So what do you believe is plausible that could be done? I'm not saying utopian. This is what you'd do if you were emperor. But given where we are right now as society, given how much money in the economy is going towards improving these models, and not only because they're fighting with the Chinese, but also because they are the biggest part now of the U.S. economy. Right. And they're like, people want to support growth. What can be done that would limit the harm while recognizing the extraordinary upside, which I've certainly been a big enthusiast of, of how much AI can improve society.
- [20:03]
B
It all starts with, I think, the race with China and reframing what that race is. Because the justification for why we can't do a bunch of constricting measures is that we're going to lose to China. But if we deploy AI recklessly in a way that causes AI psychosis or kids suicides, or degrades kids mental health, or causes every kid instead of thinking to actually just outsource their homework completely to AI so they don't have to do any work, it's very obvious the long term trajectory of we're going to have a weaker civilization. Right.
- [20:32]
A
And it's not even, it's not just kids. We're talking about not even just kids, we're talking about adults, we're talking about society.
- [20:36]
B
We're talking about society.
- [20:37]
A
But China is, we have no issue. China's not doing that. Yeah, right. I mean, it's hard for me to understand why there's a race with China on something that China isn't deploying.
- [20:47]
B
Yeah, exactly. Well, I think we're in the U.S. i think we have this false belief that we have to have just a bigger, more powerful technology and then people don't care whether we just happen to take that technology, turn it around and blow ourselves up in the face, which is kind of what we're doing. Like we beat China to social media. Did that make us stronger or did that make us weaker? Made us radically weaker. So we're not in a race for technology, we're in a race for who's better at applying and governing. Exactly where in our society do you want to deploy that technology in a way that strengthens it? And I think you're exactly right that we should be applying it to manufacturing, to medicine, to increasing very specific scientific domains. But why do we need this broad based rollout that is under the maximum incentive to cut corners on safety that is not going to end in a good result. And we can do some basic things to change that. We can have basic AI liability laws so that if it's a product and has product liability, you're responsible for some of the harms that will create a more responsible innovation environment. We can restrict AI companions for kids, we can strengthen whistleblower protection because frankly, the red lights are already flashing on a bunch of these AI models and their capabilities and the public needs to become aware of that. Governments need to become aware of that before this goes off the rails.
- [21:55]
A
Now, I did see on this, I mean in response to looks like some of these cases that OpenAI for example, does have parental controls that they have announced.
- [22:04]
B
Yes, that's right. So my understanding is that those parental controls, when they were tested by a journalist, they were able to break them in under five minutes. And so these companies are not designing their products for the safety of children. They're designing them to win market share and market dominance and hook as many people as early in their life as possible to AI, because that's their incentive. And they'll add in the little band aids here and there to try to make it a little bit less toxic or harmful. But at the end of the day, the incentive to market dominance is the driving factor, which is why we have to do is change that incentive.
- [22:38]
A
Some of this might well be that the US Government needs to be more involved. You already see more industrial policy from the us, whether it's in taking a share in intel or it's a golden share of Nippon Steel. But I mean, the idea that the US government is interested in helping to ensure that AI is being applied more effectively more quickly in the industrial uses, in the military uses, in the places where, frankly, if. If China actually does get a major advantage, there would be a national security concern for the US as opposed to on the social side, where it seems to be a disadvantage.
- [23:15]
B
Right? Yeah, that's what I think we would be doing, is applying it carefully in the domains that we know we need to be competitive with China. And we see where do we need to match them on industrial policy and on military usage to have maximum deterrence of future wars. I also think we need to just be honest with ourselves about racing to have the most crazy autonomous weapons and the risks of World War III underneath those kinds of just weapons that we would never want. Ideally, we would put in some controls. We'll see if that's even possible.
- [23:44]
A
Well, there it was, of course, 1962, before the United States and the Soviets recognized that having arms control discussions and agreements was a smart idea. There's no such negotiation between the US and China right now on an AI arms race. Seems to me that would be something we would be well placed to begin.
- [24:03]
B
I agree with that, and I know it might seem unlikely to your viewers who are watching that the US and China could ever have any agreement on AI. But it's important to note that in the last Biden XI summit, XI added something to the agenda at the last minute, and that was to prevent AI from being in the nuclear command and control systems of both countries.
- [24:24]
A
Seems fairly obvious.
- [24:25]
B
Seems fairly obvious. And that's because we autonomously recognize the Threat of uncontrollable nuclear escalation. Well, having AIs that are acting unpredictably and that are embedded in critical infrastructure or embedded in our weapons systems that already have demonstrated evidence that when you say we're going to replace an AI model, they will threaten to blackmail a company leader to prevent themselves from being replaced. We already have this evidence of stuff we thought only existed in sci fi movies that should be grounds for saying AI and controllability is not in China's interests, it's not in the United States interest. And so the degree to which we'll be willing to do that treaty is the degree to which we are both aware of the evidence that AI is not controllable like other technologies. This is not lines of code that says if this, do this, take out the line of code that says become Adolf Hitler or praise Hitler if you're in this situation. They're not programming a digital brain. They're more growing this digital brain and sort of rewarding it trillions of times based on if it does things more like this or less like this. But it is unpredictable. And we have to recognize that that's different from all other technologies in the past. And I do think the US and China need to come to terms with that now.
- [25:35]
A
For a couple of years, the Europeans had been out there, I would say, closest to making the kinds of arguments that you're making right now. But now when you hear Emmanuel Macron, when you look at the Britain AI summits, they're talking more about being too far behind, needing to grow, needing to have, needing to get into this race as opposed to safety and regulations that will help society. Do you see that as well?
- [26:01]
B
So I think that AI is very confusing because it both represents a positive infinity of benefits, meaning it can invent new science, new energy, new technologies that we can't even dream of. And people who are optimistic about that just point their attention at you. And I couldn't even possibly imagine how.
- [26:21]
A
Great it's going to be.
- [26:22]
B
It's going to be.
- [26:22]
A
And that's true.
- [26:23]
B
That's true.
- [26:24]
A
That is actually true. Exactly.
- [26:25]
B
But AI is unique compared to any other object that we've had to psychologically put in our mind, which is, it also represents a negative infinity at the same time, at the same time of sci fi level risks that we've never had to appraise of before, like the fact that it could actually lose control, actually invent brand new viruses or bioweapons, which is not just me saying some throwaway comment there's Now Stanford University just a week ago published examples of some of this.
- [26:50]
A
When you have President Trump actually saying that we need the UN involved with the United States in deploying AI to ensure that bioweapons are not becoming more real and present. Yeah, clearly this is an issue.
- [27:04]
B
Yes, yes. And I think that one important thing to get about this is that if the upsides happen, they don't prevent the downsides. If the downside happens, it takes down the world that can ever receive the upside. And so you have to have a security mindset that is more concerned with defensive acceleration of AI, meaning the defensive applications of AI, than just naively rush to the optimism because it's easier to point your attention and makes your nervous system feel good to feel into those.
- [27:34]
A
Possibilities you seem to be oriented towards. And there are a number of people in the field that feels this way, that a pause or at least a slowdown in the development of this technology is required, which seems like an utterly impossible position to take. Yes, you are not saying that we need to constrain racing forward in industrial applications. No, not at all.
- [28:04]
B
Narrow applications of AI that accelerate our actual productive output or keep our military in parity with the other military. And you need those things. But why are we recklessly racing this out to society psychologically in ways that we definitely don't know what we're doing? This is just stupidity.
- [28:20]
A
Tristan Harris, thanks for joining us.
- [28:22]
B
Thanks for having me in.
- [28:26]
A
That's it for today's edition of the Gzero World Podcast. Do you like what you heard? Of course you do. Why not make it official? Why don't you rate and review GZero World 5 stars only 5 stars. Otherwise, don't do it on Apple, Spotify or wherever you get your podcasts.
- [28:41]
B
Tell your friends.
- [28:50]
C
The Gzero World Podcast is brought to you by our lead sponsor, prologis. Prologis helps businesses across the globe scale their supply chains with an expansive portfolio of logistics, real estate and the only end to end solutions platform Platform addressing the critical initiatives of global logistics today. Learn more at prologis. Com.