Demis Hassabis, Google DeepMind and the Battle Over AI Safety

Summary

---

Podcast Summary: On with Kara Swisher

Episode: Demis Hassabis, Google DeepMind and the Battle Over AI Safety
Date: May 28, 2026
Host: Kara Swisher
Guest: Sebastian Mallaby, journalist and author of "The Infinity Machine"

---

Episode Overview

This episode features an in-depth conversation between Kara Swisher and Sebastian Mallaby—acclaimed journalist and author—about his new book, The Infinity Machine, which chronicles the story of Demis Hassabis, co-founder and CEO of Google DeepMind. The discussion dives into Hassabis’s unique position in the world of AI, his influence on the direction and safety of AI, and the broader questions about power, governance, and the race for AGI (Artificial General Intelligence). Mallaby and Swisher analyze Hassabis’s motivations, his legacy, the fractious state of AI safety, and the growing concentration of power among a tiny elite in the AI world.

---

Key Discussion Points and Insights

1. Who Is Demis Hassabis? (03:52–06:11)

• Originality & Pioneering Role:
  • Demis founded DeepMind in 2010, pioneering in the AI field long before it was clear AI would become so central.
  • “He was the first scientist entrepreneur...when AI couldn't even recognize a cat photo. Nothing worked, deep AI winter, and then the others came afterwards as straight derivatives.” (Sebastian Mallaby, 04:32)
• Personal Traits:
  • Hassabis is described as ambitious, competitive, and deeply driven by scientific curiosity—sometimes to a spiritual degree.
  • “When I'm reading a scientific paper...reality is screaming at me, demanding to be discovered...And if I can understand it...then I will understand the intelligent meaning that might have created this. And I'd be closer to God, or at least his understanding of God.” (Sebastian Mallaby, 07:09)
• Differentiation from Other AI Leaders:
  • Unlike figures such as Sam Altman or Elon Musk, Hassabis is less public, more focused on scientific breakthroughs than PR or business antics.
  • Came from a working-class, immigrant background, and was a child chess prodigy.

2. DeepMind’s London Roots and the Sale to Google (12:14–16:56)

• Staying in London:
  • Hassabis cites the UK’s scientific talent as the rationale, but Mallaby suggests patriotism and disillusionment with Silicon Valley’s culture were true drivers.
  • “He simply thinks that the values in Silicon Valley suck and he would rather stay in Britain.” (Sebastian Mallaby, 13:14)
• Google Acquisition & Concessions:
  • Sold DeepMind to Google in 2014 for $650M, with conditions: keeping it in London, banning military uses, and establishing a safety review board.
  • Google agreed, largely to secure Demis himself, rather than just the technology.

3. The Struggle for AI Safety and Oversight (20:07–23:36)

• Singleton Vision Falters:
  • Hassabis hoped for a “Manhattan Project” style collective to develop AI safely. This was quickly undermined by the rise of rivals, especially OpenAI.
  • Mallaby credits Sundar Pichai (Google CEO) with using corporate strategy to block Hassabis from spinning DeepMind out, forcing its continued integration.
• Project Mario:
  • For three years, DeepMind’s leadership fought for outside oversight—even considering spinning out DeepMind with outside investment, efforts that ultimately failed due to exhaustion and corporate resistance.

4. AI Safety: Naiveté, Egos, and Escalation (24:32–29:37)

• Naive Idealism vs. Reality:
  • Hassabis is described as idealistic—believing one organization could safely steer AI. Mallaby challenges this as “naive,” given human nature and corporate competition.
  • “It's still crazy, still naive, still immature to think that when you're confronted with a godlike technology, there won't be lots of acolytes trying to do it, and there'll be sectarian splits.” (Sebastian Mallaby, 25:35)
• Power, Not Just Money:
  • Motives in AI are less about direct profit, more about ego and power: “AI is such a weird technology. It attracts people who are basically doing it for power, for ego. It’s Promethean.” (Sebastian Mallaby, 25:11)
• Declining Power Over Safety:
  • As competition increases, even Hassabis’s influence over safety has waned. Google has begun providing AI for military and open-source applications, despite previous pledges.

5. DeepMind’s Breakthrough: Alphafold and Scientific Impact (29:39–34:02)

• Alphafold’s Nobel-Winning Achievement:
  • DeepMind’s use of AI to solve protein folding led to a Nobel Prize in Chemistry (2024), revolutionizing biology and medicine.
  • Demis insisted on actual solution rather than “declaring victory early,” showing his scientific tenacity.
  • “Demis said to me, well, I was being unreasonable, but I wanted to be reasonable in my unreasonableness.” (Sebastian Mallaby, 33:27)
• Open Sourcing:
  • They released Alphafold’s models freely, enabling scientists worldwide to access protein structures—an act highlighted as genuinely beneficial and not just marketing.

6. The Oppenheimer Parallel and Public Backlash (36:49–40:19)

• Scientific Heroism vs. Exile:
  • Hassabis embraces the ‘warrior monk’/heroic scientist image of Oppenheimer, but doesn’t focus on becoming a political exile.
  • Swisher questions whether AI figures are prepared for the public backlash that tech optimism is facing today—mentioning AI’s popularity is “only better than Trump at this point.”
• Varied AGI Timelines:
  • Anthropic thinks AGI could arrive as soon as 2028, Google/DeepMind projects later (2030+), dependent on definitions (“Can it be Einstein?”).

7. The Corporate and Social Impact of AI (41:35–45:59)

• Enterprise Adoption & Labor Market:
  • Hassabis is shown to “have zero to say” about enterprise adoption, emphasizing the gap between scientific builders and practical deployment.
  • Swisher and Mallaby debate the real impact of AI on job loss: it may be larger than previous tech shocks, but not necessarily catastrophic for all jobs.
  • “I think we should beware of both extremes.” (Sebastian Mallaby, 44:06)

8. The U.S.–China Race and the Case for a Global Safety Pact (45:59–50:34)

• Beating China or Collaborating?:
  • Swisher and Mallaby oppose purely adversarial thinking (“the Xi or me” frame), arguing for global safety pacts akin to the nuclear arms regime.
  • Many U.S. officials claim “China can’t be trusted on safety,” but Mallaby counters: “You think it was easy to trust Khrushchev? ...But in that period is when we created the nonproliferation regime.” (Sebastian Mallaby, 49:03)
• China’s Own Safety Fears:
  • Chinese leaders, on Mallaby’s recent visit, raised safety concerns as much as American or European leaders.

9. Building Public Trust and Managing the Transition (Kent Walker and Adena Friedman Q&A: 51:25–55:00)

• Optimism vs. Doom:
  • Mallaby recommends showing genuine medical breakthroughs, and being proactive with policies such as wage insurance and retraining.
• Political and Corporate Obstacles:
  • Lack of diverse voices in policy (Swisher: “they refuse to have wide ranging points of view”), and tech companies’ resistance to regulation impedes progress.

10. The Centralization of Power and the Need for Regulation (57:09–62:41)

• Worrying Concentration:
  • Despite Hassabis being “a good person,” his individual will is powerless against “race dynamics”; meaningful change requires government action.
  • “The only way to counter the power of this handful of individuals ...you need government to take a serious intervention...Why would an AI model be released without being tested by the government first?” (Sebastian Mallaby, 59:42)
• Regulatory Hope?:
  • The U.S. government’s (under Trump) recent U-turn towards AI intervention is cited as a positive, if reactionary, sign: “They've gone from being laissez faire to maximally interventionist. It shows you how freaked out they are and maybe that's a hopeful sign for action on regulation in the future.” (Sebastian Mallaby, 62:10)

---

Notable Quotes & Memorable Moments

• On Silicon Valley Values:
  • "He simply thinks that the values in Silicon Valley suck and he would rather stay in Britain." (Sebastian Mallaby, 13:14)
• Defining AGI:
  • “If you could train a computer or an AI system on everything that was known in 1911 and then you waited to see if by itself it would discover general relativity and then it did, that would be AGI.” (Sebastian Mallaby, 10:27)
• On Motives in AI:
  • “It's not about money. It's Promethean. It's for power, for ego.” (Sebastian Mallaby, 25:10)
• On Historical Repeats:
  • "You think it was easy to trust Khrushchev? ... But in that period is when we created the nonproliferation regime." (Sebastian Mallaby, 49:03)
• On Regulation:
  • “Why would an AI model be released without being tested by the government first? ... It's crazy, right?” (Sebastian Mallaby, 59:42)
• On Tech and Government:
  • “The moral of the story is that tech just needs to be scary enough to force the government to act. That's really not the way. That's not the government I want. But fine, I'll take it in some ways.” (Kara Swisher, 62:56)

---

Key Timestamps for Important Segments

• Demis’s Uniqueness and Early Days: 03:52–07:32
• DeepMind's Move and Sale to Google: 12:14–16:56
• AI Safety Struggles (Project Mario): 20:07–23:36
• Debate on Singleton AI and Naiveté: 24:32–29:37
• Alphafold and Nobel Prize: 29:39–34:02
• Public AI Backlash and Oppenheimer Parallels: 36:49–40:19
• Enterprise AI, Layoffs, and Societal Impact: 41:35–45:59
• US–China AI Relations & Safety Pacts: 45:59–50:34
• Public Trust and Political Obstacles: 51:25–55:00
• Centralization, Regulation, and Future Outlook: 57:09–62:41
• Closing Joke/Quote: 62:56

---

Tone and Style

The tone is characteristically Kara Swisher—direct, occasionally sardonic, unafraid to challenge her guest but respectful of expertise. Mallaby is nuanced, reflective, and at times philosophical, but anchors his commentary in storytelling and realpolitik. The conversation blends hard-edged realism with a deep appreciation for the technical and ethical stakes of AI.

---

Conclusion

Sebastian Mallaby’s portrait of Demis Hassabis is that of a brilliant yet embattled figure, increasingly outmaneuvered by corporate and national pressures in the AI arms race. While DeepMind’s scientific achievements are transformative, individual virtue or intention is shown to be insufficient against systemic forces. Both Swisher and Mallaby argue for assertive, collective governance of AI—before the disproportionate power of a few morphs from worrying to unmanageable, or catastrophic.

---

Transcript

[Sebastian Mallaby]

I was trying to do this sort of optimistic, you know, ending riff, and you're like, sebastian, you just burst my balloon.

[Kara Swisher]

You know, intelligence has its limitations, but stupidity and greed are infinite. So that's my feeling. I'm sorry.

[Sebastian Mallaby]

It's on.

[Podcast Host/Narrator]

Hi, everyone. From New York Magazine and the Vox Media podcast network, this is on with Kara Swisher. And I'm Kara Swisher. My guest today is journalist and author Sebastian Malaby. He's a longtime chronicler of power and innovation, especially in the world of finance and economics. For his latest book, the Infinity Machine, he turns his attention to tech. His central figure is Demis Hassabis, the CEO and co founder of Google's AI R&D lab, DeepMind, and a Nobel Prize winner in chemistry. Hassabis has dedicated his life to using AI to unlock the mysteries of physics and biology. A former child chess prodigy, he's rabidly competitive and insatiably curious. But Malabi says Demis is also one

[Kara Swisher]

of the very few big names in

[Podcast Host/Narrator]

AI development who genuinely cares about public safety. His struggle to balance his ambition, his personal goals, and the realities of the corporate AI race is, according to Malaby, one of the most defining stories of the era.

[Kara Swisher]

I think it's really important to look

[Podcast Host/Narrator]

at characters like Demis because he was

[Kara Swisher]

a very early person in AI, at least the modern version of AI, because it's been around forever. He's also based in London and kept

[Podcast Host/Narrator]

his company there and away from Silicon Valley. So he has different goals at the same time.

[Kara Swisher]

He's incredibly ambitious and a bit crusty according to lots of people.

[Podcast Host/Narrator]

Very typical of a science researcher type.

[Kara Swisher]

But he has less and less power

[Podcast Host/Narrator]

over what's happening because of the vast amounts of money pouring into the space. People tend to forego safety for profits, if in fact they make profits anytime soon. He is different from other AI developers

[Kara Swisher]

or there's a lot of people who are sort of in his area, but he's certainly one of the leading minds

[Podcast Host/Narrator]

of this age and someone you should know well. All right, let's get to my interview with Sebastian Malabi. We have two expert questions today from NASDAQ CEO Adena Friedman and Kent Walker, president of Global affairs at Google and Alphabet. This interview was recorded live in front of a virtual audience. It's part of the Dean's Summer Book

[Kara Swisher]

series at American University's Kogod School of Business, so stick around.

[Podcast Host/Narrator]

Support for this show comes from the Guardian and their new show Stateside, where journalists Kai Wright and Carter Sherman use the entire independent reporting resources of a Guardian to slow down the news and wrestle with the questions we all have about what is actually happening in the world. Join Kai and Carter three times a week as they utilize all the reporting resources a Guardian has across news, international coverage, climate, culture, wellness and more. And the Guardians not owned by a billionaire, they fearlessly report the facts without interference. Go to theguardian.com stateside to learn more and listen wherever you get your podcast or watch on YouTube. That's theguardian.com stateside.

[Skylar Diggins]

What's up y'?

[Sebastian Mallaby]

All?

[Skylar Diggins]

I'm Skylar Diggins, seven time WNBA All Star, Olympic gold medalist and mom.

[Kara Swisher]

And I'm Cassidy Hubbard, host and reporter for nearly 20 years, covering the biggest names and stories in sports and mom.

[Skylar Diggins]

And this is AM mom, a community for athletes, game changers and moms of all kinds. Dropping May 14th tap in with us.

[Kara Swisher]

It is on Sebastian Mallaby, thanks for coming on on.

[Sebastian Mallaby]

Great to be with you, Kara.

[Kara Swisher]

So for people who don't know who Demis is, he isn't a household name the way other major AO figures have become, like OpenAI, Sam Altman, Anthropic's Dario Emodi, or of course XAI's Elon Musk, even though he's accomplished as much or more in AI. I'll just say I broke the story when they bought his company at Google a long time ago and I met him just briefly during that. He was mad at me because I wrote the story. But nonetheless, let's put him into context because I did understand the importance of him in AI and he definitely he's not in the shadows, but he's certainly not as well known and he doesn't put himself out there. So how is he like them and what sets him apart?

[Sebastian Mallaby]

So the first thing that marks him out is that he was the first. Right? So the first scientist entrepreneur sets up DeepMind in 2010 when AI can't even recognize a cat photo. Nothing worked deep AI winter and then the others came afterwards and they came afterwards as straight derivatives. Right? So OpenAI is set up five years later and Elon and Sam are explicitly trying to do the anti Demis, anti Google DeepMind company they were. And then you go forward Anthropic and dario as a PhD scientist wanting to sort of do it with more of a social conscious and make it safer or like yesafree says, I think also admire Demis and people at Anthropic say sometimes he's the one who is the closest to a model that Dario had in his head. So first point is he's the original Demis. He's the only one with a Nobel Prize. He's different in that because he started early. His approach to AI was not merely to scale an existing technology path, but actually to invent the technology path. And so they brought together agentic systems from reinforcement learning, mixed that up with deep learning, learning from data, and kind of invented the field. And I think there's some DNA left over today from that experience, where if it was to be the case that AI in the next three years went down a novel path, something more than just scaling the transformer architecture, it would be more likely to come from Google, DeepMind and Demis Hassabis than from the Rivals.

[Kara Swisher]

So talk a little bit. Also, by the way, AI has been around for a very long time. He started in England, that was one of the things. Demis started the company there. And of course, Alan Turing. Hello, nice to meet you. But a lot of stuff went on very early in England, actually, which is interesting. But how is he like the others, for better or worse? How do you. If they look at him as a. Or someone that they look up to or are trying to copy in some ways, talk about the attributes and maybe some of the negatives about that?

[Sebastian Mallaby]

He is different, I think, in that his motivation is this intensely, intensely scientific curiosity to the point where he expresses it to me in spiritual language. And there was this moment, you know, fairly early on in my more than 30 hours of conversations with him, where we were sitting in a park in London, and at the next table, there were people having regular chats about their friend who went to hospital or whatever it was. And I was opposite this sort of messianic possessed person who was banging at the table and saying, you know, when I'm reading a scientific paper and it's 2 o' clock in the morning, reality is screaming at me, demanding to be discovered, demanding to be understood. And if I can understand it and get close to the fabric of reality and nature, then I will understand the intelligent meaning that that might have created this. And I'd be closer to God, or at least his understanding of God. So that is something which is very distinct. Right.

[Kara Swisher]

Some of them are getting spiritual. Peter Thiel talks about the Antichrist, which is anyone who's against AI. You know, when you said that when he's doing that, that actually would trouble me, because these people do tend to think of themselves as gods manipulating machines in some fashion. And so a lot of it is indeed spiritual. And at the same time, they're not gods, they're people. Correct.

[Sebastian Mallaby]

Correct. And I mean, this gets to this question of, you know, is the motivation simply the sweetness? This is the quotation from Robert Oppenheimer, the Manhattan Project. The quote was, you know, repeated. And at the time, Jeff Hinton didn't know that he was being overheard by a journalist from the New Yorker, but he was. And what he said was, you know, he repeated Robbenheimer's line, I'm not creating AI for any particular good reason. I'm doing it because when you see a technology that sweet, you just do it and you figure out the consequences afterwards. And as a scientist, I can't resist. And I think Demis has quite a lot of that. And so.

[Kara Swisher]

Except Oppenheimer was a little more thoughtful, wouldn't you say? I mean, I have become death or that kind of thing.

[Sebastian Mallaby]

I don't know. I mean, Demis uses some pretty funky philosophic references too. He talks about Spinoza, he talks about Immanuel Kant. He can riff on anything.

[Kara Swisher]

Yes, comparatively, he's quite like deep. So is Dario Amod in many ways. I mean, Dario Moda can quote from plays which none of these others can do. Elon's mostly sci fi crap. And Sam, nothing at all. Although he's not stupid by any means. But Demis comes from a working class immigrant family. As a kid, he was a chess prodigy. Not a surprise. He was consumed by the idea of AI. Around 17, he said he decided to dedicate his life to this. Talk about what drove that singular focus to be the persons who solves artificial general intelligence and explain what that means, because that's defined differently by different people. I want to hear how you see his definition of it. And this is called AGI. And again you write that its definition remains fuzzy. So talk about his definition.

[Sebastian Mallaby]

Sure. So just on the kind of early motivation, it really was extraordinary. I mean, to. At 17, and we're talking kind of 1993, you know, AI was talked about since Alan Turing in the 1940s, but it went through these rises and falls, as you know, and this was definitely winter at that time. And, you know, the kind of AI that was happening was academics fiddling around with toy experiments that had no relevance to the real world at that time. So to believe not only that you were going to devote your career to AI, but to super powerful AI, artificial general intelligence, which as you say, is defined by different people. But we could say it's the ability for an AI system to be smarter than humans at any screen based task. I think that's probably a sort of average type definition it's not the most extreme or not the most modest. So you know, this is what he wanted to do when he was 17, which is pretty amazing, right?

[Kara Swisher]

And that's how he defines it. Any screen based task of figuring out.

[Sebastian Mallaby]

Well, actually, you know, the funny thing about Demis is that he so loves the process of discovery and of research and the quest. And he's got one Nobel Prize, but he definitely would like another one that he loves to go for a definition of, of AGI these days, which puts it out further into the future so that the joy of the journey can be extended. And so what he says is if you could train a computer or an AI system on everything that was known in 1911 and then you waited to see if by itself it would discover general relativity and then it did, that would be AGI. That's its new definition.

[Podcast Host/Narrator]

Ah, probably not maximalist.

[Kara Swisher]

Probably it couldn't. Yeah, I guess. Wow, that would be something. Sorry, I doubt that would happen. Actually I'm going with Einstein on this one. He co founded DeepMind in 2010. Obviously the name DeepMind is exactly what he's going for. Four years later, as I said, Google bought it for $650 million. He was able to extract big concessions in the sale. I remember writing about it, it would remain in London, the offices and there would be restrictions around the use of their technology. Talk about being in London because most people working on this stuff were in, whether it was Fei Fei Li or Hinton, he was at Google. Fei Fei Li was at Google for a little bit. And then elsewhere, some of the early people were deep in Silicon Valley. So why stay in London during this sort of social media smartphone boom? At the time when I wrote about, everyone was like, what's that? I'm like, no, no, this is a big friggin deal for Google to buy. I thought understood what he was doing there. But talk a little bit about why he sold it. Obviously need the money to grow it, I think because as, as it turned out, this needs a lot of money. Talk about remaining in London and why.

[Sebastian Mallaby]

Yeah, so you know, Demis's official explanation for why he stayed in London was that there was a lot of scientific talent in London and in Europe. And you know, if you were trying to recruit great scientists, you had a competitive advantage of hoovering up that, you know, geography and you know, if you were in Silicon Valley, it'd be super competitive to get the best people and it would be, you know, better to be in London. When you actually look at who he recruited the first PhDs who came, they were sort of from all over the place. They came from Canada, from people who'd studied under Geoff Hinton in Toronto. They came from Switzerland where there was a good PhD program in AI. There was Koray Kavak Choglou who had studied in New York under Yann Lecun. So this notion that it was for

[Kara Swisher]

people who don't know. Jeff Hinton is considered one of the early godfathers of modern AI. Also Fei Fei Li is. And Yann Lecun later ran, until recently met as AI experts and he was

[Sebastian Mallaby]

also an academic kind of pioneer and remains. And the point is that what Demis claimed as being the reason for staying in London was not how it turned out because he was recruiting from all over the world, including from the us. And so I actually think the reason why he stayed in Britain is simple patriotism. He just likes Britain. You know, he's a classic melting pot product of immigrant parents in London, very attached to London, and he simply thinks that the values in Silicon Valley suck and he would rather stay in Britain.

[Kara Swisher]

Yeah, I heard him say that. So the mentality around the company was very different. So it was happening. And especially at that moment when things sort of started to go off the rails, I would say 2010, 2011 is when Silicon Valley started to lose its ever loving mind. So, you know, beyond the move fast and break things nonsense, it really, it became something much different. And that didn't exist in London, obviously. And the safety demands were the biggest concession for Google. DeepMind demanded a ban on military uses. Think about Darya Modi at this point. Demis and his team also insisted on forming an outside safety review board to dilute Google's influence over the technology. Smart. Why did Google ultimately agree to those terms given it was obvious how important AI was to the company's future, especially to their core business search, which this would decimate essentially in its current.

[Sebastian Mallaby]

Yeah, yeah. I mean the point person in the negotiation with Demis in that transaction was actually Larry Page. And that was important because Larry, you know, his father had worked academically on deep learning and that impressed Demis. And Demis felt that if he sold the company to Google specifically, he would be in the hands of somebody who loves science. Larry, you know, he once said to me, you know, you could imagine Larry as a professor at a top college like Stanford. And from Demis, that's the ultimate comment of respect. So whereas Mark Zuckerberg tried to buy Demis and DeepMind and Demis sort of laughed him off, and Elon tried to buy DeepMind. And Demis refused. He was happy and comfortable with Google because of that scientific culture, as they

[Kara Swisher]

would be because these were two PhD candidates themselves and Google was started in a much more scientifically focused way.

[Sebastian Mallaby]

And then came the conditions, as you say, around safety, around the AI sort of safety of the siteboard. And I did speak to the sort of chief lawyer who was on the M and a team at Google who remembered sort of basically being terrified by the idea of diluting Google's right to do what the heck it wanted with this asset for which it was playing

[Kara Swisher]

several hundred million dollars at the time was huge. That was an enormous shock to people at the moment.

[Sebastian Mallaby]

Correct.

[Kara Swisher]

Today it's nothing. It's like some credit card.

[Sebastian Mallaby]

But this lawyer said to me, we have a fiduciary duty as a public corporation to own assets from which we derive value. And if we're told, well, you can't derive value because you have to have this outside bunch of safety overseers who are going to tell us when you can or cannot deploy, forget it. Are we even allowed to sign that kind of deal? And the way he described it is that the reason that Google caved and gave DeepMind what it wanted is that whereas with many startups that Google buys, the notion is they've got some good tech and we'll get rid of the founder if we don't like him. To the contrary, with DeepMind, they wanted Demis. He personally was the reason they were so keen on paying up and by extension they were willing to give him what he wanted on safety.

[Podcast Host/Narrator]

We'll be back in a minute. Support for this show comes from Boll and Branch. If there's even one thing off about your bed, it'll show up in your sleep. Maybe your sheets slip around too much or your pillows not quite right. These issues only get worse over time as materials wear down, which can wear you down alongside of them. With bowl and branch, you can upgrade bed and find everything you need to get better sleep. They make everything from cotton sheets to pillows to blankets and comforters. All are designed to be breathable, incredibly soft and get better over time. Most people start with signature sheets and it's not uncommon to buy two sets of sheets so you can rotate them with Bolen branch. The moment you get into bed, you notice the difference. I have tried out the bedding. It is fantastic. I have a waffle blanket that I

[Kara Swisher]

love so much, especially in the summer where it's cool and warm at the same time.

[Podcast Host/Narrator]

So get 20% off your first order plus free shipping during the Memorial Day [e-mail] Kara with code Kara that's Bole and Branch B O L L A N D Branch B R A N C H.com Kara K A R A code Kara to get 20% off bowlandbranch.com Kara code Kara exclusions apply.

[Skylar Diggins]

Pregnant athletes are not fragile. Yeah, that's right, I said pregnant athletes. I'm rabbinatison VPN head instructor at Peloton and I PR'd my deadlift the week before my son was born. I was also a quote, geriatric type 1 diabetes pregnancy and so I know there can be a lot of fear and uncertainty about what is healthy movement when you're pregnant. That is why I got trained in pre and postnatal fitness and this week on my podcast Project Swagger, I am sharing some key guidelines and the story of how I stayed active during my pregnancies. Listen now at Project Swagger.

[Podcast Host/Narrator]

Support for this show comes from the Guardian Trying to locate news sources that reliably separate fact from fiction can seem like looking for a needle in a haystack these days. Social media feeds, cable news and subscription based news outlets might be able to give you one version of the news, but not in the way that you need. You want news fast, but slow enough to actually absorb the information. Stateside with Kai Wright and Carter Sherman is here to do just that. The podcast tries to slow down the news and wrestle with the questions regarding the world at large at your own pace. Three times a week, hosts Kai Wright and Carter Sherman bring their expertise together to showcase different and engaging perspectives regarding the world. Today they will use their collective knowledge to discuss the news, international affairs, climate, culture, sports, lifestyle, fashion and wellness. Plus, the Guardian wants to emphasize they're not billionaire owned, meaning they're free to report the whole picture without interference. Go to theguardian.com stateside to learn more and listen wherever you get your podcast or watch on YouTube starting May 13.

[Kara Swisher]

Now, he had hoped there'd be one collective effort to develop AI because there wasn't a lot of AI development. As you said, it was a winter of AI at the time, and he had hoped there'd be sort of a Manhattan Project with Google and DeepMind at the center. And one of the early interviews I did with Sam and Musk when they were starting OpenAI was their worry that DeepMind in Google would dominate not just them, but possibly Mark, who they had thought little of at the time. In that regard, it was shattered after the first Ethics and Safety Group meeting in 2015, Musk hosted it and then teamed up with Sam to start OpenAI when they were getting along and it was a rival and it was started. He was testifying honestly for a second when he talked about the original reasons. I happened to write that story when that happened when Opelind was created. So talk a little bit about the fallout of the meeting and how it changed his approach to AI development and the way he ran DeepMind.

[Sebastian Mallaby]

Yeah, I mean, so what happened, as you say, is there was this safety oversight meeting which was supposed to be for DeepMind to get useful feedback. And it turned out the people at the meeting were not giving useful feedback. They were kind of learning how DeepMind was doing stuff and then collecting information. So in that meeting there was both Elon, but also Reid Hoffman, who was one of the funders of OpenAI. And so the reaction was from Google, forget the idea of safety oversight boards. We've tried that once and it was terrible. So we're not going to do that again. Demis reaction was, well, we can't give up on the idea of a safety oversight board. It's essential for this technology and so we're going to have a fight about it. And so now this is 2016 is when this really gets going. And there's this thing called Project Mario, as they called it internally at DeepMind where for three years they retained, you know, armies of lawyers and investment bank strategists and people like that. And there were term sheets running to 50 pages that flew back and forth between the negotiators in Mountain View and the negotiators in London. And I was shown some of these things.

[Kara Swisher]

He tried to spin it out. He tried to spin it out.

[Sebastian Mallaby]

Yeah, I mean he thought of spinning out because that would be the threat that would force Google to agree to the safety oversight board. And Reid Hoffman promised a billion dollars to finance a spin out. They went to see Joe Tsai, the co founder of Alibaba in Hong Kong to try and get some money out of him. It was a real thing. And what happened after three years is that Demis caved. He was so exhausted by this, three years of negotiation. And he'd get another long term sheet from the lawyers and he would hold his head in the hands and say, I don't want this.

[Kara Swisher]

They were never going to let him do it.

[Sebastian Mallaby]

I don't want this. Part of my brain to expand is what Demis wound up legal documents. And so the upshot was, you know, he didn't get what he wanted. Google was never Going to do it. I. I kind of developed a weird respect for Sundar in learning about this process because he was very canny. You know, he knew he didn't want Demis to spin out. He didn't say it quite directly, but he, you know, hid behind his chief counsel and other people who played bat cop on his behalf. And at the end of the day, he got what he wanted, which was to keep Demis inside the tent, not spun out so that he would be

[Kara Swisher]

the secret weapon, because he did understand what was happening in Silicon Valley with the competitors. Right. Google had to be at the forefront of this, and Sundar, who's a very lovely affect, is really good at playing rope. A dope, I always found, you know, in niceness. But let's talk about this idea of what Demes wanted to do here. He says he was naive to believe in a singleton scenario where AI developed collectively, but he also saw himself at the center of the effort. Now, you could say that's egotistical, and it does take a certain amount of narcissism that you should be in charge of all AI development. I have problems with. I am more with the collective, but at the same time, it always devolves into either one person or just a few people developing this thing. But talk about this idea, because they were never gonna do it once they understood the financials here, the amount of money and the amount of power they would. Was that naive of him or egomaniacal? He says he was naive.

[Sebastian Mallaby]

Yeah. I think it was naive of Demis to believe that AI would be developed by one single lab on behalf of all humanity. I mean, humans are just disputatious and tribal and jealous, and they don't do that.

[Kara Swisher]

Greedy, mostly.

[Sebastian Mallaby]

Oh, that too, probably. But, you know, I. I like that

[Kara Swisher]

you said probably there.

[Sebastian Mallaby]

Oh, God, no. I said probably. Because what strikes me, Kara, about this whole field is that the normal cynical explanation that people are doing it for the money is weirdly less true. Because AI is such a weird technology. It attracts people who are basically doing it for power, for ego.

[Kara Swisher]

For power is.

[Sebastian Mallaby]

Yes, it's Promethean.

[Kara Swisher]

You're right.

[Sebastian Mallaby]

It's not about money.

[Kara Swisher]

So in hindsight, why did he not see this at the time? And I don't think he was doing it for the money. He is a true believer of all these people. And, of course, there's the glory. He wants a Nobel Prize. That's pretty great. And he has curiosity. But should he have been less naive or explain the naivete? Because it seems obvious to see that they were not gonna do this.

[Sebastian Mallaby]

Yeah, I mean, I think there's, you know, there's some, you know, mitigating circumstances that you can cite that make it a bit less crazy to have believed in this single thing. That would be simply that when he was beginning, you really could fit all the AI believers in one conference hall. And so there was a sense of a single community at that point. And then, you know, he starts DeepMind, and it's going really, really well. And he does, you know, this Atari model, which is just an astonishing agent that can learn by itself how to pay, you know, dozens of different video games. And that impresses everybody. And then he gets, you know, the Google checkbook behind him, and it feels like he's the only game in town. And if there's a second game in town, it would actually be Google Brain in Mountain View. And so that is the same company. And OpenAI just wasn't a thing then and didn't really become a thing until, let's say, 2019 until recently. Yeah. So I think it wasn't totally crazy at the time to think, well, I can't see anybody else on the horizon who's doing this, so it's going to be me. But it's still crazy, still naive, still immature to think that when you're confronted with a godlike technology, there won't be lots of acolytes trying to do it, and there'll be sectarian splits. And you can continue that metaphor right now.

[Kara Swisher]

We've heard Altman, Musk, and even Mark Zuckerberg claim they care about AI safety. I believe none of them. Only to watch them prioritize growth and their own egos time after time. Is he different? Does he remain different? There's almost no incentive for tech companies like Google to make AI safety their top priority. For example, Google is eager to supply AI to the Pentagon after agreeing not to do so when it bought DeepMind. So while he understands, Demis understands the risks of developing AGI, does he have any more power to contain them in the face of the pressure both national Security, financial, just to win, to be the dominant technology of the next era, et cetera?

[Sebastian Mallaby]

Yeah, I think Demis has gone a long journey right since founding DeepMind 16 years ago. the beginning, he thought there might be a singleton scenario. Then when that started to break down because OpenAI launched a competitor, he still hoped to get oversight for his technology, as we discussed with Project Mario negotiating. And then when that didn't work, he said, well, at least I'll make AI good for humanity by doing it for science. And that's when he did the protein folding system. And then along the way, although interestingly he didn't ever tell me this, I only know it from other sources. He was the one who said to Rishi Sunak, the British Prime Minister, you know, let's have a global safety discussion at Bletchley park in 2023. And that happened. And so I think he had a

[Kara Swisher]

reasonable track explain why Bletchley park, that's where.

[Sebastian Mallaby]

Well, that was where during the Second World War the German code was broken with by Turing with sort of the early kind of precursors of AI.

[Kara Swisher]

It was very symbolic.

[Sebastian Mallaby]

Exactly. So I think up to 2023 and that summit, he had a pretty good track record Demis in walking the walk as well as talking the talk on safety. I think though that once he got into the race, full on to do chatbots in competition with OpenAI and then anthropic, he's. He's done less and less, at least visibly. And it strikes me for example that by his own logic, open source models are dangerous and yet Google releases them. He wanted to deprive the military of AI and yet Google now supplies them. And his sort of rationalization for this is simply that there's a race on Chinese labs are part of the race. If he were to step back, if he were to quit, it would make zero difference to outcomes for humanity because there'd be six other labs doing it anyway.

[Kara Swisher]

Sure. But he wouldn't be part of it, like in some ways. Right?

[Sebastian Mallaby]

Yes. Look, and I think that gets to a super interesting debate about do we care about the gesture? Do we care about the outcome?

[Kara Swisher]

Right. I mean, I think his power had waned in that regard because people caught up, even if they're not as brilliant as he is now. He reminds me a little of Nikola Tesla, ultimately, you know, the greatest mind. And Edison just ran right over him. In 2024 he shared the Nobel Prize in chemistry for his work using AI to predict structures of proteins. As you noted, amazing thing. It could lead. It will lead to the creation of drugs and vaccines to fight diseases. An amazing thing and an astonishing use of AI in service of humanity. I think one of the best examples and not marketing that the rest of them foist upon me on a daily basis. And it could be where AI makes people's lives better. Talk about this because. And what it can mean for medicine. I know a lot about this, I've done a lot of reporting on this. But it really was a moment and it has taken off, whether it's mRNA, all kinds of stuff, drug discovery, the quickening of drug discovery and ideas. Talk a little bit about this.

[Sebastian Mallaby]

Yeah, well, I know your CNN show gets into this, right? And so you probably could talk about it more than I can, but I mean, no, no, certainly in the creation of alphafold, this system. I mean, that in itself is an amazing story where Demis at Cambridge as an undergraduate had been told by a biologist friend, oh, there's this conjecture in structural biology by a Nobel laureate from the 1970s called Christian Damfinsen, that if you stretched out an amino acid chain and you looked at the DNA sequence on the chain, the sequence tells you how that chain will fold itself up like a self executing origami model into an intricate, beautiful shape, which is a protein. And we have proteins in our bodies and there are proteins in plants that get the basic building blocks of nature. And you can tell this intricate structure just by looking at the code. And so it became a sort of grand challenge in biology. Who can create the computational system that will just look at the code and then guess the shape? And from the 1990s, there were teams in different universities competing to do this. And DeepMind decides after winning Go, defeating the Go champion in 2016. Well, that was a.

[Kara Swisher]

That's a game.

[Sebastian Mallaby]

Yeah, the game of Go. Yeah, exactly. That was just a game. But then having solved that problem, which involved this massive combinatorial complexity, because GO has all these different permutations.

[Kara Swisher]

They do, but boring in comparison.

[Sebastian Mallaby]

Boring in comparison. So why don't we move on checkers and leap all the way to predicting the combinatorial complexity of these possible shapes. One strand of amino acid could fold itself up in billions upon billions of different shapes. So in that sense it was similar to go. And having solved Go, they felt that their computer science skills had reached a point where maybe protein folding was crackable. And so they set out to do this in 2016. By 2018, they'd beaten all the universities. They had the best model, but it wasn't good enough that a pharmaceutical research team could simply use the prediction from computation to create a drug. So then the question was, should we push on and try to really make it that accurate? And the view of the leader of the team within DeepMind was, forget it, boss, we can't do this, this is impossible. Don't d send us down a blind alley. We should just declare victory. We're the best team in the world and then we should move on. And Demis was like, no, I want you to actually solve the problem, not just be the best. And so they had a bit of a fight about this, and Demis said to me, well, I was being unreasonable, but I wanted to be reasonable in my unreasonableness. So I listened in to the team's discussions for a while, and when I heard they had lots of ideas that were just flowing naturally, which they hadn't yet tested in the lab or in the computational lab, I figured, well, there's more stuff they could try. And so then he pushed them. He switched out the team leader, put a new person in charge, and then in 2020 they succeeded and they did predict the shapes of proteins. And then the badass move was to open and source it, just like for free. Any scientist in the world can effectively do a Google search and get the shape of the protein that they want to work on.

[Podcast Host/Narrator]

We'll be back in a minute.

[John Finer and Jake Sullivan]

Was the biggest cybersecurity risk in America

[Kara Swisher]

built by software companies Software manufacturers have been allowed to develop and deliver flawed, defective, insecure software because they've prioritized speed to market and convenience all over security.

[John Finer and Jake Sullivan]

I'm John Finer. And I'm Jake Sullivan and we're the hosts of the Long Game, a weekly national security podcast. This week, Jen Easterly, former director of the Cybersecurity and Infrastructure Security Agency, joins us on the podcast. The episode's out now. Search for and follow the Long Game wherever you get your podcasts.

[Podcast Host/Narrator]

Support for this show comes from NetSuite. Every business is looking to see how AI can work for them. But while the possibilities are endless, the risks are as well. And you can't sit it out because your competitors are already making their move. But with NetSuite by Oracle, you can put AI to work. Today, NetSuite is a top rated AI Cloud ERP trusted by over 43,000 businesses. It's a unified suite that brings your financials, inventory, commerce, HR and CRM into a single store source of truth. That connected data is what makes your AI smarter. It knows how to intelligently automate routine tasks, how to deliver actionable insights, and can even help you cut costs and make fast AI powered decisions with confidence, giving you total flexibility. From software and IT services to healthcare, equipment, manufacturing, financial services and many other great American industries, NetSuite delivers a customized solution for your business if your revenues are at least released in the seven figures. Get our free business guide Demystifying AI at netsuite.com on the guide is free to you at netsuite.com on netsuite.com on.

[YourRichBFF]

Buy now, pay later is everywhere. And honestly, we need to talk about it. 60% of Coachella goers put their tickets on Buy now, pay later this year. You can even pay off your doordash order and installments now like your sushi. So this week on Net Worth and Chill, I'm breaking down exactly how these companies are making money off of you, why missing one payment can flip your interest rate from zero to nearly 36% overnight, and the uncomfortable truth about what it really means when you need to split a 200 purchase into four payments. Plus, I'm answering your questions, including what to do if you're already in over your head. Listen wherever you get your podcasts or watch on YouTube.com YourRichBFF

[Kara Swisher]

One of the things that's interesting, the parallel you make in the book and is frequently made about the development AI is to J. Robert Oppenheimer, who we discussed earlier in the creation of the atomic bomb. It's one that Demis and others in the field welcome. But Oppenheimer is also not just because I saw the Chris Nolan movie. I read a lot about Oppenheimer. He saw the bomb as an evil thing. It kicked off the nuclear arms race. He became a scientific exile during the red scare. How does he feel, Demis feel about this? Because right now, as you note correctly, there's a huge backlash to not just data centers in the US But AI in general. The polling with the public is, you know, only Trump has worse polls at this point, but AI is pretty disliked. So talk a little bit about that.

[Sebastian Mallaby]

Yeah, I mean, I think he feels slightly that that's why you have to talk up the optimistic side of the AI story. I think he's a little frustrated with Dario, frankly, for going on TV and saying within five years 50% of entry level jobs will be gone. You know, and to propose to kind of say that kind of thing without proposing the policy remedy. You know, Demis sort of points slightly a finger at Dario on that, but I think, you know, I'm not sure I agree with him, frankly. I think, you know, Dario has a good case to say, hey, we've got to call it like we see it. And that will wake people up and politically maybe there'll be a stronger policy response if we do that. So I'm not sure Demis has any particular solution to this, you know, popular fury.

[Kara Swisher]

Does he see himself as an Oppenheimer? I mean, Oppenheimer went through a pretty rough period before we liked him again, right. Or when he got finally feted at the end of his life. But there Was an exile happening.

[Sebastian Mallaby]

There was exile happening. You know, I think Demis embraces the kind of scientific glory, the leading of the Manhattan Project, the heroic story of, you know, going off into the wilderness and focusing on nothing but the scientific mission. I mean, there's something about Demis which is almost. Somebody called him a warrior monk. And off he goes, you know, in his mind's eye, to the desert in isolation. And that just is a super appealing self image for him.

[Kara Swisher]

Yeah. Nothing narcissistic about that. But go ahead.

[Sebastian Mallaby]

That's the Oppenheimer bit that he wants to identify with. He. He doesn't identify or doesn't talk about the idea that he would be ostracized and there would be kind of a political.

[Kara Swisher]

Yeah, Dario's taking the flag for that at this point, right?

[Sebastian Mallaby]

Yeah. And I think there's an interesting debate about whether if you look at Dario's principal stance on the Pentagon using AI for weapons, it was good in terms of raising public awareness of the issue. It didn't change the outcome. Outcome. So if you're Demis and you look at that, you could say to yourself, well, it didn't change the outcome. So what was the point of that? I'm much smarter. I'm going to go do this behind closed doors. I will talk to politicians as I did with Rishi Sunak, and suggest things. They can take the credit. What's the point of going public if you don't change how the world works?

[Kara Swisher]

Absolutely. Except he didn't change how the world works either. But that's another issue. So we're in the middle of an explosion of growth around AI and you write the estimates of achieving human level AGI, again, much to see disputed by 2030 now appear slightly conservative. Talk about how Demis looks at this. Because many. I get a different answer, different people, and I don't think they even. I think they're just guessing. I feel like at this point they're just making it up. But talk a little bit about. You get very different numbers from different people.

[Sebastian Mallaby]

You do get different numbers. I mean, if you talk to people at Anthropic, they're really saying 2028. And what they say is that by 2028 there'll be recursive self improvement, meaning that the models will be coding the next model. And once they've done coding the next model, the new model will recode the next one after that. And so you'll get this vertical acceleration in the capacity of the systems. And at that point you're done.

[Kara Swisher]

So they go from dolphins to humans.

[Sebastian Mallaby]

At that point, you're done, the race is over. And they say 2028 in written material. I think actually privately they even think it could be next year. So they are super near term in their prediction. Demis would love to kind of nudge his predictions beyond 2030, 2031, 32. And as I was saying, that's partly because its definition is. Can it be Einstein? Which is obviously the most expansive definition. Then there are people in between who say, well, it's about when most economically valuable human tasks that you could do in front of a screen could be taken over by AI. And then people might put a 2030 kind of number on that. So I think that's the range.

[Kara Swisher]

Right, right. And you just of course, are seeing a lot of the layoffs from Meta and many others. So every episode get a question from an outside expert. We have two for you. Here's the first one.

[Adena Friedman]

Hi there. I'm Adena Friedman, the chair and CEO of nasdaq. Based on all of the time that you spent with Demis, what do you see as the role of enterprise adoption in realizing the potential of AI? And if you do see it as being significant, what are the key obstacles that enterprises need to overcome in order to be able to achieve its potential within the enterprise?

[Kara Swisher]

So I just referred to the layoffs, so go ahead.

[Sebastian Mallaby]

Yeah, so I think the revealing thing here is that that the answer to the question of what does Demis have to say about enterprise adoption is zero. Nothing. And that shows us the separation between the builders of the models and the real world users who are going to actually change the way corporations function. All that stuff.

[Kara Swisher]

Although many of the builders are talking about it, Anthropic would be the one who seems to be making the most headway in that area.

[Sebastian Mallaby]

Yeah, I mean, I think Anthropic is very good at sort of going for enterprise tools, so coding first, cybersecurity second, and then having four deployed engineers. This is the kind of Palantir model who help corporations to adopt this stuff. So I agree with you. But certainly in a big company like Google, the people who are doing the Palantir stuff are in some other division of Google. And Demis is really just focused on building the product. And to the extent that you have this separation, and you're right, Kara, that it's not as clear in a smaller company like Anthropic. But I think the separation maybe does tell us something about what could go wrong here. Right. That one set of people are just focused on building something and they're not really thinking about how it's going to be used.

[Kara Swisher]

Yeah. And then another is actually actively thinking. And we're starting to see more companies, for example, justify layoffs by pointing to AI. Sometimes that's not the case, but they still are using it as an excuse. Cisco laid off 4,000 people earlier this month while announcing record revenue and earnings, laid off 8,000. And Goldman Sachs estimates AI Ltd. Around 16,000 net jobs a month over the last year. Now, some of these, as I said, companies are using as a convenient scapegoat for too much hiring they did during COVID which I think most people, especially the tech companies did, talk about mass joblessness. And is that something that Demis thinks about? And if so, what's standing in the way of that? And he said that he thinks that companies who are replacing developers with AI, quote, show a lack of imagination and a lack of understanding. Is he being, again, naive about corporate incentives around tech? And some people are moving too fast forward and it's not going to work, obviously. But talk a little bit about this.

[Sebastian Mallaby]

Yeah, I think there's sort of two extremes which are wrong. One extreme is to say there's nothing to see here. There's not going to be any big jump in unemployment. We've been here with previous kinds of technologies and the labor market always adjusts. And we had the Internet and actually unemployment was very low throughout that period, period. And I think that's naive because AI is not just another technology. It's more powerful, it's more general. It's directly competing with human cognition. And it can be scaled if you have enough data centers up the wazoo so that you replace tons of people. So I think we can't just draw comfort and be complacent just because of the technology history. On the other hand, it's also way too simple to say now that this system can do all this stuff, humans are done because new jobs will emerge to some extent. I mean, it's partly that there are tasks that humans are better at dealing with humans sometimes. So if you think about the sales team, enterprise sales team in a big company where the job is basically to go schmooze the humans who are your customers and forge a bond with them, I think humans have an edge at that. So I think there are a lot of tasks where humans will remain superior. And to the extent that the economy grows because of AI, there'll be more of those tasks. So I think we should beware of both extremes. I mean, one number I like, it's unclear. I Like to cite this statistic, which is that no technology has ever driven economic growth per capita at the frontier, more than 2.5% a year. So people who tell you that AI is going to double the size of the economy in 15 years are smoking something.

[Kara Swisher]

Abundance. You're not an abundance person. Neither am I. The argument, I'm so tired of that. The argument around rushing to develop, obviously AG I think, has largely centered on beating China. But in many interviews with these people, people, I call it the Xi or me argument. Like we gotta do it or China will. And I get it. But in a recent op ed and I really appreciated it, that you wrote in the New York Times, you call for the US to negotiate a safety pact with China because we cannot beat them. And not just falling behind, but there's certain things, as with nuclear energy, as with everything else. If we're falling behind, what incentive does China have in signing a safety pact? I think there's huge incentives for them to do so. And it's not just to keep them from being ahead of us. It's that there are commonalities here. As with, as I said, nuclear energy or nuclear weapons or cloning. There's a lot of stuff that is global. And oddly enough, I ran into Tony Blinken last night and he said I had urged him when he first got his job to do a global pact around safety and with China especially. And he's like, well, I didn't get that done. I'm like, no, you didn't. And then I walked away. I was rather intent on getting them to stop, you know, to pay attention to this. But talk a little bit about this, this sort of Xi or me, and what happens with China, because you are correct, they are doing astonishing work here. And we had been ahead, obviously, the US or democracies. And now I'll put all democracies because Demis is in London. But talk a little bit about this race and also the need for a safety pack, which I think is critical.

[Sebastian Mallaby]

Yeah, yeah. So, you know, China does everything fast. So my book came out in China before it came out in the us US And I went to China in March for the launch and I spent eight days basically going around talking to AI leaders, both in academic labs and in industrial tech companies. And what struck me was how often they brought out the topic of safety. And this was interesting because that is not what you hear from people who put the chip export controls in place in 2022 in the Biden term. So. So those people, several of whom are my friends and I've talked to them a lot about this. They said to themselves, credit to them the scaling laws are real. AI is going to be super powerful. And they said this before ChatGPT came out. So we're going to do something. We're going to prevent the bad guys from getting this AI. But that definition of bad guy was China. And in my view there's a more important set of bad guys who are criminals, who are, who want to do cyber attacks. Terrorists, individuals, rogue states, you know, random bad people.

[Kara Swisher]

Rogues. Just rogues.

[Sebastian Mallaby]

Individual rogues. Exactly. And they missed out that whole category. And by making China the enemy and the rival and saying you cannot get AI and we're going to deny you the chips and say you won't have Nvidia. No Nvidia, no dice. You'll be left behind. You know, the mistake was to make China into the enemy and lose maybe a chance to talk to them about what about non proliferation of this stuff? Maybe we should just say you're a technology superpower. We are a technology superpower. The way we're going to avoid some catastrophic war between us is the same way we did it in the Cold War with the Soviets, which was parity, mutually assured destruction. Parity is actually good for balance and stability. Whereas when you're talking about about random rogues using this stuff for bad ends, the Cold War lesson is that's where you need the IAEA to keep track of the material and the non proliferation treaty. And my friends who are in the Biden administration say, well, yeah, but you can't talk to the Chinese because they don't care about safety. I went there and they did talk to me about safety. They say you can't trust the Chinese. And I go, you think it was easy to trust Khrushchev? I mean this was the Soviet leader who banged his shoe on the table at the UN and put missiles in Cuba. Not an easy guy to talk to. But in that period is when we created the non proliferation regime for.

[Kara Swisher]

That's right. And their interests of their economy being destroyed by a rogue something is just as high as ours.

[Sebastian Mallaby]

Totally.

[Kara Swisher]

It's just really kind of. I was always like not so sure I want either of you. I'd like a global safety group that includes companies, includes legislators, includes China, includes the US So that we have. It's very similar to nuclear. Exactly. We trust Khrushchev. That's a great way to put it. I thought that piece was terrific.

[Sebastian Mallaby]

Thank you.

[Kara Swisher]

Now we have a second expert question for you. This is from Kent Walker, president of Global affairs at Google and Alphabet. And the bad cop, apparently. Let's listen.

[Kent Walker]

Hey, Kara. Hey, Sebastian. In reading the book, I was struck by how, well, Sebastian, you captured Demis's vision of using AI as a way of solving intelligence, to then solve everything else as a scientific tool to help us address that our biggest problems. The challenge is that polling in the US suggests that America is one of the least optimistic countries about AI, much less optimistic than China is. Maybe we've had too many debates between AI accelerationists and AI doomers, but how would you suggest we go at the challenge of creating a grounded optimism, recognizing the challenges, but also encouraging people to put these tools to work to benefit themselves in their everyday lives and to benefit our society? Thanks.

[Kara Swisher]

That is a great question. One Google. You should stop being so aggressive, but go ahead.

[Sebastian Mallaby]

Yeah, so I mean, we referred to this a bit before, but I think that medical breakthroughs that delivered AI generated or AI assisted drug discoveries that really save people's lives. That would be a great thing to change public opinion on whether AI is good or not. I also think that we ought to put in place preemptively before we totally need it. Things like wage insurance, retraining schemes, active labor market schemes to help people get other jobs. Do that early, don't wait around until it's obviously essential, because at that point public opinion will be so mad at you that it won't make any difference.

[Kara Swisher]

There'll be riots, there'll be work riots just as they were before. In other.

[Sebastian Mallaby]

I think a useful statistic here is that in the 12 years from 1999 to 2011, the total number of job losses as a result of China entering the WTO was 2 million. In the US, 2 million is nothing. 2 million is like the amount of labor market churn you get in an average month in the United States. And yet the backlash against China, the perception of the China shock, the fury about globalization politically, was absolutely massive. So imagine if you got an AI shock that was way, way bigger than China. China, the reaction will be, as you say, it'll be people in the streets. I mean, with the industrial revolution, what happened was you got revolution politically, you got Marxism, you got a whole lot of turmoil, and that was a revolution which took place probably 1/10 the speed of the AI revolution and may arguably have been smaller.

[Kara Swisher]

Absolutely. I think they don't even understand the anger, the rage that is building and it's manifesting itself. I think a lot of the AI anger is a manifestation of affordability and nervousness about the Future. And now it has become that. It has become that.

[Sebastian Mallaby]

I don't know what you think, but I think there's some merit to this idea of Trump accounts where you would put shares in AI companies or other stakes. You distribute that democratically to all young Americans.

[Kara Swisher]

Sounds good. No, he doesn't seem interested in doing. He does them, but he does other things more. The stuff that he does. That's populist. I'm not against like the Trump accounts or the. I don't like him calling them Trump accounts or Trump Rx or whatever. He has to put his fucking name on everything. But no, but that idea is correct. You know, ubs, some version of this or, or more like where are the jobs? And do some really serious government studies and where we work with companies and you know, this recent AI advisory board has nobody on it except for business. There's no way they're gonna come to any conclusion but their own. And they could use critics. They could use someone like you, someone like me, someone like an academic, someone like, you know, they just refuse to have wide ranging points of view here. That is probably gonna be the biggest problem. I think.

[Sebastian Mallaby]

I think it's also relevant to Kent's question that, you know, the record in the last couple of years, well, during the Trump period, is that all the energy on legislation has come from the states and the AI companies, the tech companies have lobbied actively to stop that stuff from passing. And they have a decent argument in that a patchwork of state level stuff would be way less effective than a federal thing. But then that's why you need federal intervention. Right.

[Kara Swisher]

Which they are also trying to stop, which is why they're sitting in the front row of the Trump inauguration.

[Sebastian Mallaby]

Yeah, I don't quite. I think that's a bit tough. I know you like to be tough, but I think that, that in the Ben Buchanan period, when Ben Buchanan was the czar in the White House, what he said on the record publicly is that whenever he talked to the labs, they were willing to support what he was trying to do in terms of setting it up, an AI Safety Institute, in terms of then requiring labs to share their models with the Safety Institute before they were released, clearly that regulation should have gone further. But he says that actually the labs were encouraging him to go down that path of regulating. And I think I don't have any reason to disbelieve that. So, you know, I think I disbelieve you disbelieve.

[Kara Swisher]

But look at what the Trump administration just is starting to announce now. The same thing the exact. That they rejected. Correct. Now, they do that with a lot of stuff, but their most recent noises have been exactly what that Biden executive order said.

[Sebastian Mallaby]

I agree. So.

[Kara Swisher]

So Trump, which is crazy.

[Sebastian Mallaby]

The Trump people have done.

[Kara Swisher]

They're like, this is our new idea. And I'm like, well, that was two years ago or three years ago at Bletchley Park.

[Sebastian Mallaby]

100% completely agree with that. But that's a description of the Trump administration's U turn. I think the position of the AI labs, which is what we were talking about earlier, I think they actually have been open to sensible federal action. They just hated the state action.

[Kara Swisher]

Well, they're getting the state action because initially they lobbied against any kind of tech regulation. Right. And so they find them. They should. They should be supporting it. They shouldn't have kneecapped Amy Klobuchar, for example, or they've been kneecapping politicians for a long time. And I think people don't trust them. I think that's where it is. And then when you have any manner of them swatting around, looking like Dottie Warbachs everywhere across the world, it's not great. It's not. You know, Elon hasn't helped. Jeff Bezos certainly has. And neither is Mar. I mean, I think the imagery. You don't have demises there or, you know, Daario is a hero and, like, he's fine, but, like, that's it. Like, that's the problem. There's no heroes. There's a lot of people who look like they're in it for the money.

[Sebastian Mallaby]

So what is your explanation? I'd love to get your view on this. So why is it that Demis is not particularly famous in the U.S. i mean, I've been.

[Kara Swisher]

He doesn't want to be. Yeah, he doesn't want to be. Yeah, he could be. He could be all over the place. He could be Dario emoting everything. Because he deserves that spot. You know, I think he's a true wonk, probably. And so he's. The science is everything. But, you know, history is littered with people who. Sciences weren't as savvy. Nikola Tesla wasn't as savvy as Edison. Edison was a PR person. He did all manner of nefarious tricks against Tesla. Like, you know, I don't think he's that much of a victim the way Tesla certainly was, if you read a lot about what happened there. But I think he likes to be pure. He likes to have that image of himself that's my. I don't know him very well, but that's what I can see. But he certainly deserves it. He certainly deserves the attention. You know, Geoffrey Hinton likes to take up space, that's for sure. But you could see him out there more. But, no, here he is with your book. So obviously he's talking to you. So, last question. Today's AI race is exactly the scenario he had hoped to avoid. But in an excerpt in the Atlantic, you wrote that has, quote, come to see salvation, paradoxically, in his own career advancement and securing personal influence. We have concentrated so much power in the hands of just a few people after years of talking to many of them for the book. What do you make of his conclusion? And what does it mean for the rest of us who are worried about concentrated power? I know I am, of anyone, even if they were the angels, I don't feel like they should have this many small amount of individuals to have this much power over something so important. Of course it's happened in history, but it's never ending. And this is quantumly more powerful than any, like railroads, trains, the telegraph, stuff like that. Are you worried, and should the rest of us be worried about this concentration of power, even if it's in the hands of people that are better than others, but a lot of them aren't?

[Sebastian Mallaby]

Yeah, yeah, yeah, yeah. Look, I think in a way, the story of Demis Hasabis is, you know, of a decent person, a good person who wants to be good, but can't actually have a good impact because he's inside this race dynamic, inside a big corporation. And whatever he may personally think about safety, there are these larger forces that are driving him on. And if he were to quit tomorrow and take a professorship in theoretical physics at Printon, it wouldn't change the race. It wouldn't make the world better. It wouldn't do anything very constructive. I'm not sure I want him to do that because it would just be a gesture and prefer to focus on the outcomes. So what this points to is that we do need government intervention. The only way to counter the power of this handful of individuals who are kind of collectively very powerful, but individually impotent. In terms of the big social questions, you need government to take a serious intervention. And why not? We have drugs in the United States which are approved by a regulator, and if they're dangerous, they're not allowed to be released. Right. Why would an AI model be released without being tested by the government first? Why would we allow open weight models that can be ripped off by anybody, abused by anybody, used for a cyber attack and you wouldn't be able to kill it with a kill switch. It's crazy, right? So I think what we need is far more active government action precisely because AI needs to be right.

[Podcast Host/Narrator]

So where does that leave the rest

[Kara Swisher]

of us if AI goes the way that social media did and the government fails to act? I mean, obviously different governments are doing different things, but there's no coordinated global effort at all. And in this country, forget it so far. And of course we did nothing to slow down the relentless and often poisonous and toxic pace of social media. So what has to happen? What do you see happening? What does demos see happening happening?

[Sebastian Mallaby]

I think, I think he's got to the point where he's trying a little bit to, you know, do politicking behind the scenes and suggest that, you know, more action is needed and it has to be collective and the government needs to lead that. But whether he's putting real energy into it, I can't tell you for sure. And so I think it's just, you know, that history we talked about before of Project Mario and trying to get safety over south side, it's almost like he's burnt out on trying to solve the governance problem and so he's just heads down on the technical problems, which is a regrettable place to end up. But all I can say is that's the most optimistic thing that's happened in the last few months was when Anthropic released Mythos and the Trump administration freaked out and they called in the heads of the banks and said, you guys are going to have hacks that empty your bank accounts unless you can harden your cyber defense senses, because this stuff is for real. And they basically did a 180. And now you know what? Anthropic had this list of 40 kind of responsible tech companies that were going to get the model first before anybody else got it and then they were going to roll it out to others. Those subsequent waves have been frozen because the whole decision making around the release of the models has been requisitioned by the Trump administration. And they're sitting on it it and they're saying nobody's allowed to have this. And so they've gone from being laissez faire to maximally interventionist. It shows you how freaked out they are and maybe that's a hopeful sign for action on regulation in the future.

[Kara Swisher]

Unless they're trying to do it to make more money. Who knows? How can we take all those bank accounts and get them for ourselves?

[Sebastian Mallaby]

I was trying to do this sort of optimistic, you know, ending rift. When you're like Sebastian, you just burst my balloon.

[Kara Swisher]

Oh, I don't know. You know, intelligence has its limitations, but stupidity and greed are infinite. So that's my feeling. I'm sorry, but I think there the moral of the story is that tech just needs to be scary enough to force the government to act. That's really not the way. That's not the government I want. But fine, I'll take it in some ways. Anyway. This is a marvelous book. I really appreciate it and thank you so much for your time.

[Sebastian Mallaby]

Of course. Thank you. I really enjoy it.

[Podcast Host/Narrator]

Today's show was produced by Christian Castro Wisel, Michelle Aloy, Katherine Milsop, Megan Birney and Caitlin Lynch. Nishat Girwa is Vox Media's executive producer of podcasts. Special thanks to Bradley Sylvester, Madeline LaPlante, Dubie, and Julia Sharp Levine. And thanks again to American University's KOGOD School of Business for hosting this event. Our International Our engineers are Fernando Arruda and Rick Kwan, and our theme music is by Trackademics. If you're already following the show, you win a Nobel Prize. If not, you're just in it for the money. Go wherever you listen to podcasts, search for on with Kara Swisher and hit follow. Thanks for listening to on with Kara Swisher from Podium Media, New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Monday with more.