EP 4: Speedrun

Summary

---

The Last Invention – EP 4: Speedrun

Podcast Host: Longview
Date: October 16, 2025
Episode Overview:
This episode explores how the race to build artificial general intelligence (AGI) accelerated due to a paradoxical dynamic: the very people most concerned about AI's risks became the ones competing to build it fastest. Through deep dives into pivotal moments—Elon Musk’s alarms, Google’s DeepMind, OpenAI’s origin, and the development of super-powered models like ChatGPT—the episode traces rivalries, transitions from nonprofit idealism to for-profit pragmatism, and the immense pressures fueling a breakneck "speedrun" in AI development.

---

Main Theme:

The AI Speedrun—How Warnings Became a Race
The episode unpacks the core paradox driving the current AI landscape: the people most anxious about superintelligent AI (Elon Musk, Sam Altman, Demis Hassabis, Dario Amodei, et al.) concluded the only way to prevent catastrophe was to build transformative AI themselves—faster than anyone else.

---

Key Discussion Points & Insights

1. The Musk Paradox: Sounding the Alarm, Starting the Race

• 2017 National Governors Association: Elon Musk warns governors that AI is an "existential risk" and says proactive regulation is necessary—otherwise robots will eventually "do everything better than us" ([02:43]).

  • Quote:
    • "AI is a fundamental risk to the existence of human civilization. ... we need to be proactive in regulation instead of reactive. Because by the time we are reactive ... it's too late." – Elon Musk [02:43]

• Backstory—The DeepMind Meeting:

  • 2012 meeting (organized by Peter Thiel) between Musk and Demis Hassabis (DeepMind). Hassabis claims AI could follow humans to Mars, which unsettles Musk ([06:26]).
    • "My AI will be able to follow you to Mars." – Attributed to Demis Hassabis (as recounted by interviewee) [05:06]
    • Musk invests in DeepMind to "keep tabs" but fails to prevent Google’s acquisition.
    • Musk ramps up public warnings about AI after losing out on DeepMind ([07:32]).
      • “AI is far more dangerous than nukes. ... the single biggest existential crisis that we face.” – Elon Musk [07:32]

• Musk Tries to Rouse Action:

  • Unsuccessful lobbying with President Obama and lawmakers; hosts "AI safety" dinners with tech leaders ([09:12]).
  • At one such dinner, Sam Altman (then Y Combinator president) convinces Musk to create OpenAI as a nonprofit counterweight to Google ([12:15]).

---

2. OpenAI: The Nonprofit Ideals & Internal Fissures

• Founding OpenAI (2015):

  • Born out of existential fear: "If you want to stop a dangerous AI ... we need to make a safe AI before they make a dangerous one." – Sam Altman [12:15]
  • Nonprofit, open source, for humanity ([13:13])
    • "OpenAI is structured as a 501c3 nonprofit to help spread out AI technology so it doesn't get concentrated in the hands of a few." – Elon Musk [13:13]

• Recruiting ‘Believers’:

  • OpenAI’s early team: AGI doomers (AI could kill us) and accelerationists (AGI will bring utopia) ([15:02])
    • "They were both there together in that one lab. ... The doomers and the accelerationists are just two sides of the same coin." – Sam Altman [15:44]

• First Setbacks & DeepMind’s Go Triumph:

  • OpenAI starts with scattered research, no major gains.
  • DeepMind’s AlphaGo defeats Go champion Lee Sedol in 2016—a landmark moment ([17:50]).
    • Move 37’s impact: AlphaGo invents a creative move no human ever played ([24:54])
      • "That move was what got everyone to say ... oh, the computer, like this machine can be creative, it can be intuitive." – Expert/Analyst [26:03]
  • Sparks Musk’s frustration—demands more focus and action.
    • “There obviously needs to be immediate and dramatic action, or else everyone except for Google will be consigned to irrelevance.” – Musk, quoted from internal OpenAI email [28:25]

---

3. Nonprofit to For-profit: The Musk–Altman Rift

• Tensions Over OpenAI’s Direction:
  • To compete with DeepMind/Google, OpenAI needs more computing power—proposes a for-profit arm ([29:25]).
  • Musk wants to be CEO if OpenAI turns for-profit, threatening to leave if not given control ([30:48]).
    • "If you're going to turn that nonprofit into a for profit, then I should be the head of that." – Interviewer/Commentator paraphrasing Musk [30:27]
  • Others resist, citing original mission to "avoid an AGI dictatorship" ([31:16]).
    • "The goal of OpenAI is to make the future good and avoid an AGI dictatorship... So it is a bad idea to create a structure where you could become a dictator if you choose to." – Ilya Sutskever, in email to Musk [31:16]
  • Musk leaves; Altman & team proceed without him ([32:06]).

---

4. New Directions: Language Models & the Power of Scale

• OpenAI’s Breakthroughs Post-Musk:

  • Shift from games to language models marks a turning point.
    • "DeepMind had to copy paste OpenAI." – Andrej Karpathy quoted by Keech Hagee [38:33]
  • Team focuses on scaling (bigger models, more data), leading to the creation of GPT models ([39:12]).

• Dario Amodei’s Influence:

  • Pioneer in "mechanistic interpretability" and evangelist for the ‘scaling’ approach: more data + bigger models → more intelligence ([42:26]).
  • Major Microsoft partnership supplies OpenAI with unprecedented GPU resources ([44:52]).
    • “Guys, what if next time ... we take this model and we crank it up to 10,000 GPUs?” – Recounted from Dario Amodei’s proposal [45:17]

• The Data Grab and Ethical Tensions:

  • To match their compute, OpenAI ingests massive swaths of internet data—copyright and ethical debates arise ([49:01]).
    • “They just started dumping big chunks of the Internet into their AI ... better to ask for forgiveness instead of permission.” – Interviewer/Commentator paraphrasing [49:33]

---

5. The Accelerationist Paradox & the Birth of Anthropic

• Dario Amodei’s Departure:

  • As safety advocates become accelerationists (“to ensure safety, you must build first”), Dario Amodei and his safety team leave OpenAI, founding Anthropic due to both philosophical and practical differences ([54:20]).
    • “If you want to do cutting edge AI safety research on very powerful systems, you need to actually build those powerful systems.” – Narrator/Host [51:47]

• The Race Intensifies:

  • Now, multiple players (OpenAI and Anthropic) seek to bring safe superintelligence to market first.
    • "To make AGI safe, you need to make it fast." – Interviewer/Commentator paraphrasing [53:29]
  • Fear of missing “winner-take-most” rewards and being perceived as second movers compels OpenAI to release ChatGPT preemptively ([57:09]).

---

6. The Unlikely Launch and Mass Adoption of ChatGPT

• ChatGPT Releases Almost by Accident:
  • Initially considered a “low key research preview,” ChatGPT is released to stake claim before Anthropic ([58:38]).
    • “The highest bet was 100,000 [users]. So that’s how many users they provisioned their servers for.” – Sam Altman [58:49]
  • ChatGPT’s viral explosion shocks even its creators ([58:49]–[59:42]).

---

7. A Sobering Reflection

• Celebration and Dread:
  • AI scientists, in the wake of recognition and achievement, reflect on the anxiety that comes with rapid progress.
    • "I thought, I've achieved the greatest prize ... and my career has been so rewarding ... But ... aren't we going to build machines that we don't control and could potentially destroy us?" – Narrator/Host, quoting a leading AI scientist [60:58]

---

Notable Quotes & Memorable Moments

• Elon Musk warning governors:

  • “Until people see robots going down the street killing people, they don't know how to react.” – Elon Musk [02:20]

• On Move 37 (AlphaGo):

  • "The computer can be creative, it can be intuitive, it can sort of like master this thing that I always thought was a human task." – Expert/Analyst [26:03]
  • "Maybe he just can show humans something we never discovered. Maybe it's beautiful." – Interviewer/Commentator [26:16]

• On AI safety:

  • “The only way to stop a bad guy with a powerful AI is a good guy with a powerful AI.” – Narrator/Host [52:41]

---

Timestamps for Important Segments

• Musk’s 2017 public warning: [02:20]–[03:34]
• Musk–Demis DeepMind meeting: [05:06]–[06:26]
• AlphaGo vs. Lee Sedol (the Go match and Move 37): [17:52]–[26:16]
• OpenAI’s internal emails: Musk’s frustration: [28:25]–[31:15]
• Musk leaving OpenAI: [31:47]–[32:06]
• Birth of language model focus at OpenAI: [39:12]–[41:00]
• Massive scaling hypothesis (10,000 GPUs): [45:17]
• OpenAI’s data collection tactics and legal/ethical debate: [49:01]–[50:07]
• Dario Amodei's departure and Anthropic’s founding: [54:20]–[56:04]
• Rumor of Anthropic’s chatbot, ChatGPT pre-emptive launch: [56:42]–[59:05]
• ChatGPT’s viral reception: [59:25]–[59:45]
• AI leaders reflect on progress and anxiety: [60:54]–[62:31]

---

Tone, Language, and Atmosphere

In the voices of the commentators, executives, and AI pioneers, this episode weaves urgency, competition, and barely-contained existential dread. The tone shifts from Musk’s prophetic urgency, through Silicon Valley’s mythmaking and rivalry, to a sobering, often self-aware anxiety as the pace of AI outstrips the capacity of its creators to ensure safe outcomes.

---

Conclusion

This episode of The Last Invention vividly maps the origins of the AI “speedrun”—the drive to build AGI faster and “safely,” led paradoxically by those who most feared its dangers. Rivalries, existential panic, and once-unthinkable technical achievements—all fueled by an arms-race mentality—have made AI’s present and future as chaotic as it is promising. The final reflection is both triumphant and uneasy, as the builders acknowledge how little time may be left to answer the existential questions they themselves set in motion.

---

Transcript

[Narrator/Host]

The progressive development of man is vitally dependent on invention. It is the most important product of his creative brain. Its ultimate purpose is the complete mastery of mind over the material world. The harnessing of the forces of nature to human nature needs. This is the difficult task of the inventor who is often misunderstood and unrewarded. Does anybody know who wrote that passage? Nikola Tesla.

[Gregory Warner]

This is the last invention. I'm Gregory Warner.

[Interviewer/Commentator]

So now for the main event.

[Narrator/Host]

There are some like Tesla, Edison, the Wright brothers, Ford jobs.

[Gregory Warner]

At the 2017 meeting of the National.

[Narrator/Host]

Governors association, rare entrepreneurs who make the impossible possible.

[Gregory Warner]

All these governors from red states and blue came together in a room to find out what they could do to prepare for the future. From a man who seemed to be ushering in the future.

[Narrator/Host]

You know, I'm really thrilled to introduce a man who's arguably the personification of technological innovation. Please join me in welcoming Elon Musk.

[Gregory Warner]

And the governors are eager to ask him about his plans for Tesla, about electric car infrastructure, about how to get ready for autonomous vehicles and even SpaceX flights. They want to know, what does Elon see as the next big tech on the horizon?

[Narrator/Host]

What would you want things to look like in five to 10 years with autonomous vehicles, electric vehicles?

[Elon Musk]

Well, I think things are going to grow exponentially. So there's a big difference between 5 and 10 years.

[Gregory Warner]

But no one seems prepared for where Elon wants to take this conversation.

[Elon Musk]

I have exposure to the most cutting edge AI and I think people should be really concerned about it. I keep sounding the alarm bell, but until people see robots going down the street killing people, they don't know how to react.

[Gregory Warner]

He tells them the best thing that lawmakers can do to prepare for the future is make sure that humanity has a future.

[Elon Musk]

AI is a fundamental risk to the existence of human civilization. AI is a rare case where I think we need to be proactive in regulation instead of reactive. Because I think by the time we are reactive in AI regulation, it's too late. Because what's going to happen is robots will be able to do everything better than us. I mean, all of us, you know? Yeah, not sure exactly what to do about this.

[Gregory Warner]

And at first it seems like the governors maybe think he's pulling their legs, but he just keeps going.

[Elon Musk]

But when I say everything, like the robots will be able to do everything, bar nothing.

[Narrator/Host]

Let's move back to. You're rolling out the Model 3 this year, right? How many orders?

[Gregory Warner]

And after a moment of uncomfortable silence, the lawmakers eagerly move on. They never return to the subject. Now it Would not be long after this very public warning that Elon Musk himself was accelerating to build that very technology he seemed so alarmed about. And he wasn't alone today how some of the very people most concerned about artificial superintelligence came to decide one after another, that the best way to protect the world from this technology is was for them to build it first and build it fast.

[Interviewer/Commentator]

The AI race was started by the people who warned about it. It was started by the exact people. Sam Altman, Elon Musk, Demis Hassabis, Dario Amadei, who said, at least nominally, that they were the most concerned and they wanted to prevent this to happen. They are the exact people who actually brought us into the situation we are in now, and they're still doing it.

[Gregory Warner]

So, Andy, walk me through how we get from Elon Musk warning about AI to him trying to build AI like going from Doomer to accelerationist.

[Interviewer/Commentator]

All right, so this all started with a meeting between Elon Musk and Demis Hassabis.

[Gregory Warner]

The Demis Hassabis of DeepMind, the child prodigy.

[Interviewer/Commentator]

Right, the gamer of gamers. The child genius. Back in 2012, Peter Thiel set up this meeting between these two men. And in the years since, that meeting has become like a Silicon Valley folktale. Like, I heard about it from dozens of people who I spoke to for the series, and some of them were saying that if AI becomes even half as powerful as people think that it's going to, the future will look back at this meeting and see it as some kind of turning point. You know, it was the Demis meeting with. With Elon that we sort of brokered. Peter Thiel himself recently told a version of the story to my old colleague Ross Douthit. The rough conversation was, you know, demis tells Elon, I'm working on the most important project in the world. I'm building a superhuman AI. And Elon responds to Demis, well, I'm working on the most important project in the world. I am turning us into an interplanetary species. As the story goes, Musk says, I'm sending us to Mars so that if anything terrible happens here on planet Earth, you know, nuclear war, some kind of civilization ending pandemic, we've got this escape valve. We can actually travel to other planets. Our species can survive. And then Demis said, you know, my AI will be able to follow you to Mars. And. And then Elon sort of went quiet.

[Sam Altman]

And this is a huge trigger for Musk, where he is like, who is this guy? What is he trying to do. Why is he telling me that he is ultimately doing something that might kill us all?

[Interviewer/Commentator]

Karen Howe, the author of Empire of AI, she says that Musk quickly decides that he wants to keep his eye.

[Sam Altman]

On Demis, and so he invests in DeepMind to keep tabs on the company.

[Interviewer/Commentator]

So his reaction is to be concerned and maybe a little freaked out and then to say, here's some money so I know exactly what you're up to.

[Sam Altman]

Yeah, exactly.

[Interviewer/Commentator]

So Muskie becomes an early investor in DeepMind, and then a few years later, after the impressive Atari demo, the AI.

[Gregory Warner]

That mastered Space invaders, right?

[Interviewer/Commentator]

Yes. Without any training.

[Gregory Warner]

Yes.

[Interviewer/Commentator]

Google quickly jumps in, wanting to acquire DeepMind. Elon actually tries to get in the way of that and buy DeepMind himself, but it doesn't work. And when Google acquires DeepMind, that's the moment where suddenly Elon starts going out in public and really sounding the alarm about what he believes are the existential dangers of AI.

[Elon Musk]

I don't think most people understand just how quickly machine intelligence is advancing. Mark my words, AI is far more dangerous than nukes. I think that's the single biggest existential crisis that we face.

[Interviewer/Commentator]

Keech Hagee from the Wall Street Journal, she says that this is what inspired Elon to start going out and trying to lobby lawmakers and President Barack Obama.

[Keech Hagee]

Elon even has a meeting with Obama, and it was kind of interesting because there was a sense that, yes, Obama understood the risks, and yes, he also understood how important AI was going to be for the economic development of the country.

[Narrator/Host]

It promises to create a vastly more.

[Interviewer/Commentator]

Productive and efficient economy.

[Keech Hagee]

He gave an interview to Wired around this time, you know, saying, if properly.

[Interviewer/Commentator]

Harnessed, can generate enormous prosperity for people, opportunity for people can cure diseases that we haven't seen before, but it could increase inequality, it can suppress wages, and so we're going to have to develop new social constructs in order to embrace fully.

[Keech Hagee]

And yet Elon left that meeting with a sense that Obama wasn't really going to do anything about the existential risk piece.

[Interviewer/Commentator]

Elon, he doesn't just strike out with President Obama. He's also talking to Vanity Fair, he's talking at colleges, he's speaking at conferences like, remember, this is the Tony Stark era. Elon Musk we're talking about here, right?

[Gregory Warner]

This is a time when Elon Musk had pretty broad market appeal to lots of audiences, especially on the subject of technology.

[Interviewer/Commentator]

This is Elon at his peak celebrity pre controversies that would follow. But even for him, he feels like no one's taking him seriously. And so he starts to host these dinners where he would invite other tech leaders, sometimes other billionaires, and they would get together and try to brainstorm a way that they could stop Demis, Hassabis and Google from making some sort of civilization ending AI.

[Gregory Warner]

All right, so first he tries to buy DeepMind, fails at that. Then he goes to the leader of the free world, tries to warn him. That doesn't work. Goes to the press, gives some speeches. Then it's theme dinners, where basically the theme is, how do we save the world from superintelligence?

[Interviewer/Commentator]

That's pretty much the story. Yes. And one of those dinner guests was none other than Sam Altman. It is my belief that in the next few decades, someone will build a software system that is smarter and more capable than humans in every way, and that very quickly it will go from being a little bit more capable than humans to something that is like a million or a billion times more capable than humans. Who was Sam Altman at this time?

[Keech Hagee]

Sam Altman was the president of Y Combinator, which basically meant he was like the king of Silicon Valley.

[Interviewer/Commentator]

Kicagi actually wrote a biography of Sam Altman called the Optimist, which is really good. I recommend people check it out. And she says that by 2015, Altman was already almost this mythical figure in Silicon Valley. He had helped to turn companies like Doordash, Instacart, Airbnb into household names. What's Sam Altman's superpower? How would you sum up what he's so good at?

[Keech Hagee]

Sam Altman is a once in a generation fundraising talent. He's an incredible storyteller. He can convince people that he can see the future. He can sort of summon companies into being just by persuasion. He is also kind of a fixer with lots of relationships all around Silicon Valley, people who owe him favors and sort of make anything happen in Silicon Valley that anyone wanted.

[Interviewer/Commentator]

It turns out that since Altman was young, he had always been enamored with this idea of making a true AI thinking machine. But by the time he's having this meeting with Elon Musk, he had read the book Superintelligence by Nick Bostrom, and he had come to believe that if AI was made irresponsibly, that it could possibly lead to the end of the human race.

[Keech Hagee]

And he even began blogging about this idea that if AI happens, it could be the most consequential thing that ever happened to humanity, but it could also be dangerous.

[Interviewer/Commentator]

So when he goes to this meeting.

[Sam Altman]

With Musk, Altman starts talking to him about this idea that he is also very, very worried about AI potentially going wrong and becoming an existential threat to humanity.

[Interviewer/Commentator]

He pitches him on the idea that if you want to stop a dangerous AI, if you want to stop Demis Hassabis, if you want to stop Google, then what we need to do is make a safe AI before they make a dangerous one.

[Sam Altman]

What do you think about the idea of us creating a lab that counters Google?

[Keech Hagee]

Why don't we make a lab that would create the same technology, the same AGI technology, as a counterweight to Google, except it would be nonprofit, it would be open source, and it would be for the benefit of humanity. And Elon says, great, let's do that. Sam basically convinces Elon to bankroll this thing and by the end of the year they have created OpenAI.

[Interviewer/Commentator]

We started a group called OpenAI. It is a non profit. The goal is to build general super AI for the benefit of humanity.

[Elon Musk]

OpenAI is structured as a 501c3 nonprofit to help spread out AI technology so it doesn't get concentrated in the hands of a few.

[Expert/Analyst]

What is going to be your sort of biggest differentiator then? Like OpenAI versus like the mega corpse?

[Interviewer/Commentator]

I hope that our biggest differentiator is, number one, we do the best research in the world, and number two, we care the most about how it gets deployed.

[Gregory Warner]

Okay, so OpenAI, the company behind ChatGPT, came out of this plan to stop Google by beating Google at their own game, but do it in a totally different way than Google was doing it because it was going to be nonprofit.

[Interviewer/Commentator]

Right. The idea is to create almost like an anti Google. They called it a nonprofit research lab. They didn't even call it a tech company. And at the core of that lab is this mission that not only are they going to make the supermind, the AGI, but that they are going to ensure that this thing is good for the entire planet.

[Gregory Warner]

Right.

[Interviewer/Commentator]

Okay, so when they start OpenAI, what's it like at first?

[Sam Altman]

So the first thing that's quite interesting is in order to pull off what they wanted to do, they needed to recruit talent. So they needed to break up Google's monopoly on AI research talent. And they used their nonprofit ethos and this mission driven idea to very effectively poach a bunch of researchers from Google and then also bring a bunch of new PhD grads into the founding team.

[Interviewer/Commentator]

And now remember, even with Google purchasing DeepMind, most people in technology still don't really buy into the idea that AGI is coming anytime soon.

[Sam Altman]

And so the people that primarily ended up joining OpenAI were self selected, so called AGI believers, people that were there for the crazy quest to try and recreate human intelligence. And they were of one of two camps. There were the people who were AGI believers, but doomers that were really focused on the AI safety orientation of we're ultimately trying to recreate this thing in order to prevent existential risk. And there were the accelerationists who were like, we believe in this thing because we think it's going to bring us to utopia.

[Interviewer/Commentator]

They were both there together in this one lab working on this project.

[Sam Altman]

They were both there together in that one lab. And at the time, they philosophically did not seem that different because they were compared to the rest of the field, which just did not think that this idea of creating AGI was really something that held water. The boomers and the accelerationists are just two sides of the same coin. They both believe in AGI. There were only so many of them, so they were all kind of banded together based on that shared belief and excitement and fear around doing this journey together.

[Interviewer/Commentator]

Yeah, essentially they were sending out this signal to the world of technology. And in response, they end up actually bringing together a really fascinating mix of people. They were able to poach Ilya Suskever, one of the guys behind the imagenet, win with Hinton from his job at Google. They get Greg Brockman to join them from Stripe. They eventually bring in this guy, Dario Amade, who had also worked at Google. And these were people leaving big paying, very stable jobs in the world of technology to come work at this new research lab. Because as they said it, they truly believed in this mission and how important it was.

[Gregory Warner]

Okay, so they walk away from these big paychecks, these stable jobs, and they build. What, like, what do they begin to make at OpenAI?

[Interviewer/Commentator]

Well, at first, Elon is very excited about the idea that they should make an AI that's going to go head to head in some kind of game against Demis. Legendarily, Elon Musk is a gamer, and so he's pushing him down that path. But there's also just this kind of looseness. Right. They're a research lab, so there's this sense of like, let a thousand flowers bloom. Like, what path might lead to AGI? We don't know. Let's try this one, let's try that one. But after months and months of this, without any real meaningful progress, suddenly Demis Hassabis strikes again. Now the wait is almost over. In less than one hour from now.

[Narrator/Host]

Man will face off against machine in.

[Interviewer/Commentator]

An epic game of Go, the competitors are Grandmaster Lee Se do of Korea, and he's taking on the artificial intelligence supercomputer called AlphaGo. In 2016, Dimas and the team at DeepMind, they thunder back onto the public stage again, this time to play the game Go. The game of Go is the holy grail of artificial intelligence. For many years, people have looked at this game and they've thought, wow, this is just too hard. Everything we've ever tried in AI, it just falls over when you try the game of Go. And so that's why it feels like a real litmus test of progress. If we can crack Go, we know we've done something special. Are you familiar with Go?

[Gregory Warner]

I was obsessed with Go as a kid. I remember because I didn't really get into chess, although I got into it briefly. But Go was like, the stones were very beautiful. The pieces or just these black and white stones?

[Interviewer/Commentator]

Mm. What I think's cool about it is that it is an ancient Chinese game, and I was looking it up, and it appears as if, like, we don't even know how old it is. Like, there's records of people playing Go, like, 2,000 years ago.

[Gregory Warner]

It's older than chess, though, right?

[Interviewer/Commentator]

Way older than chess.

[Gregory Warner]

And what's crazy about it, too, is that you can be playing for a while and not even know who's winning. It's that complex.

[Interviewer/Commentator]

Mm. Jasmine sun, who was one of the tech writers that I spoke to about this, she told me you could play Go every day of your life and you would never play the same game twice.

[Expert/Analyst]

The game of Go is like an ancient Chinese game that is known for having more possible board positions than the number of atoms in the universe.

[Interviewer/Commentator]

You're saying you can't even calculate the positions?

[Expert/Analyst]

It's unfathomable. It's literally unfathomable. Right. There's no way, through some sort of brute force search, that you can just search every possible move and compare them all against each other.

[Sponsor/Advertisement Voice]

Right.

[Expert/Analyst]

Like, you can't do that. You can't memorize the strategy.

[Interviewer/Commentator]

This was a game that pretty much no expert system could have hoped to truly master, which is exactly why Demis Hassabis wanted to create an AI that could. So even if you took all the computers in the world and ran them for a million years, that wouldn't be enough compute power to calculate all the possible variations. But another way that it's really different from chess is that Go players, because there is no way for them to really calculate their way to victory. The ones who become masters of this game are often described as having some sort of deep instinct or often they use the word intuition.

[Expert/Analyst]

Champion Go players are known for being really intuitive, I suppose, for having some sort of like deep feel of like strategy in the board that you cannot learn by like memorizing any sort of rule book.

[Interviewer/Commentator]

If you ask a great Go player why they played a particular move, sometimes they'll just tell you it felt right. So the one way you could think of it is that Go is a much more intuitive game, whereas chess is a much more logic based game.

[Expert/Analyst]

A lot of games players call Go a very human game.

[Interviewer/Commentator]

So Dimmis and DeepMind, they create this AI system called AlphaGo. And the way that they train it sounds like sci fi. And it actually plays into one of the big fears that the Doomers have about how AGI might one day turn into ASI and, you know, replace us all. And it starts off like this. So at first they just load it up with a whole bunch of data of human beings playing Go so that it can find its own patterns and see that that's working for this person and that's working for that person.

[Expert/Analyst]

But again, like Go has more possible board positions than the number of atoms in the universe. There are so many moves and positions that no one has ever thought of yet.

[Interviewer/Commentator]

But then what they do is they make a identical copy of the AI system so that the AI can play against itself millions and millions of times, each time learning new strategies and gathering more data. And learning new strategies and gathering more data.

[Expert/Analyst]

The self play is what they call it. This is like the thing that really makes a system not just like quite good, but superhuman at playing Go because it's able to put in so many reps, like an infinite amount of practice that no ordinary player ever could.

[Gregory Warner]

Interesting. So that makes me think of that idea from Malcolm Gladwell, the 10,000 hours thing. Like it takes 10,000 hours to become an expert in something, but this thing can basically log the equivalent of 10,000 hours of practice in like a month or something.

[Interviewer/Commentator]

Well, it's even crazier than that because AlphaGo can play itself so quickly that in the span of a week it could play more than a human could in centuries. Wow, like hundreds of years worth of non stop playing in a week. Hello and welcome to the DeepMind challenge.

[Narrator/Host]

Game one, round one.

[Interviewer/Commentator]

Live from the Four Seasons here in Seoul, Korea. In March of 2016, they set up this showdown against the player Lee Sedol, who is often described as the greatest player of his generation. It garners all this attention, more than 100 million viewers.

[Expert/Analyst]

All these journalists are there. Everyone in Korea is watching. The game is huge there. Everyone here is very excited for the.

[Keech Hagee]

Match of the century.

[Interviewer/Commentator]

Journalists from Asia and around the world, about 350 members of the press are.

[Keech Hagee]

Here to see if artificial intelligence can.

[Expert/Analyst]

Really beat human intelligence.

[Interviewer/Commentator]

Now, I should say just a level set here that pretty much everyone thinks that Lee is going to win. Even Google believes that their system is very likely to lose. Dimas Hassabas himself, he later said that the team was given just a 5% chance of pulling this off, which is.

[Gregory Warner]

Interesting because even though chess had fallen to AI and Jeopardy had fallen to AI, that GO was seen as just too hard or maybe too human.

[Interviewer/Commentator]

Yeah, it's too something. Lee is too good. The game is too complex for an AI and maybe even just like it's not ready yet. Maybe this, like, one day it could happen, but not today. And in the middle of one of the matches, during move 37, as it's called, AlphaGo ended up doing this weird thing that seem to prove the doubters may have been right all along. Interesting. AlphaGo played this move, which I want to hear more about in a second, but Lee has left the room.

[Expert/Analyst]

It makes a move on the board that to all of the spectators, to Lee Sedol, looks almost like a mistake.

[Interviewer/Commentator]

It's a very surprising move. It's a surprising move.

[Elon Musk]

I wasn't expecting that.

[Narrator/Host]

I don't really know if it's a.

[Interviewer/Commentator]

Good or bad move at this point.

[Expert/Analyst]

Like the DeepMind AlphaGo team, they were watching it and even they thought it had made a mistake because it seemed weird, it seemed bad. It was something that no human player would ever do, had ever done in a position like that.

[Interviewer/Commentator]

So while the cameras are rolling, while the world is watching, the DeepMind team is like, damn, did this thing just malfunction?

[Gregory Warner]

Like it glitched?

[Interviewer/Commentator]

Yeah, like a glitch. The professional commentators almost unanimously said that not a single human player would have chosen move 37. I can't believe what I see right now. And yet in the end, I think he resigned. Oh, my gosh. AlphaGo wins. Yeah, Lee has. I'm getting word Lee has resigned. In the battle between man versus machine.

[Narrator/Host]

A computer just came out the picture.

[Interviewer/Commentator]

Meet my. Put its computer program to the test against one of the brightest minds in the world and won. AlphaGo beat a professional player who has.

[Narrator/Host]

18 Go World Championships under his belt.

[Interviewer/Commentator]

And then suddenly people start to go back and re examine that move, move 37. It went beyond its human guide. And it came up with something new and creative and different. And they realized that it was the turning point in that match.

[Expert/Analyst]

That move was what got everyone to say in the Go community and the broader community, even Lee himself, to say, oh, the computer, like this machine can be creative, it can be intuitive, it can sort of like master this thing that I always thought was a human task.

[Interviewer/Commentator]

The more I see this move, I feel something changed. Maybe he just can show humans something we never discovered. Maybe it's beautiful.

[Gregory Warner]

So the AI played a move that no human had ever made in any at least recorded Go game. And that means it discovered its own original strategy.

[Interviewer/Commentator]

Some people would go as far as to say that it had something like an original thought, an original idea.

[Gregory Warner]

And what do we know about how it did that? How did Demis Hasabis and his team at DeepMind explain move 37?

[Interviewer/Commentator]

Well, they really wanted to know. And so they spent time digging through the code and looking inside the guts of the system. They wrote a paper about it. And while they were able to, you know, glean some information, because it's one of those connectionist neural net, AI, toddler styles of AI. Remember the trade off that Yoshua Bengio was talking to us about? To get this kind of impressive performance, this level of intelligence, you just have to accept that you're not going to get satisfying answers to these kinds of questions.

[Gregory Warner]

You have to accept some level of mystery.

[Interviewer/Commentator]

This is the black box. And when I asked Yoshua BENGIO about this AlphaGo moment with AlphaGo, I thought, Ooh, now we're getting close to something important. He said this is when he realized that AI was now entering a whole new era. And he wasn't the only one. All across Silicon Valley, across the world of technology, people were singing Demis's praises. People were abuzz about AlphaGo. And of course, yet again, Elon Musk doesn't like this one bit.

[Keech Hagee]

DeepMind's show of force in AlphaGo freaked Elon out a lot and made this sort of ambling approach that OpenAI had at the beginning of, you know, let a thousand flowers bloom. All the researchers kind of pursue their own different areas. It made him have little tolerance for that.

[Interviewer/Commentator]

We now know in some detail what happened inside of OpenAI during this time, because a number of their internal emails were revealed as a part of a lawsuit. And so you can see in these emails, Elon Musk telling Altman and the leaders at OpenAI just how frustrated he is that they're losing to Demis. Still in One of them, he says, OpenAI is on a path of certain failure relative to Google. There obviously needs to be immediate and dramatic action, or else everyone except for Google will be consigned to irrelevance.

[Gregory Warner]

What does immediate and dramatic action mean?

[Interviewer/Commentator]

Well, according to Keech Hagee, this is when Musk was saying, where is our game player?

[Keech Hagee]

Elon wanted to drive like fighting tit for Tat with DeepMind. And he really wanted to respond to it by showing an even cooler and harder game that AI could beat.

[Interviewer/Commentator]

We need to challenge DeepMind in some kind of public display of our dominance in a game with an AI game player. And Altman and the team at OpenAI, they were saying to Musk, well, we do have a way that we think we could beat DeepMind. We've got this strategy. But for us to implement that strategy, we're going to need way more compute power and we're going to need more money. So they pitch him on this idea that the nonprofit OpenAI could have a for profit arm. And then that way Altman could go out and do the thing that he's best at, right? He can go get investor money that they can use to up their compute power.

[Sam Altman]

They start discussing, how are we going to convert the nonprofit into a for profit. And all of a sudden, Musk and Altman start butting heads because when they go to form the for profit, the question becomes, who will be the official CEO of the for profit? And both of them want to be the CEO and they cannot agree.

[Interviewer/Commentator]

And Musk comes back and basically says, no way. I gave you all this money to start a nonprofit. If you're going to turn that nonprofit into a for profit, then I should be the head of that. He even thought about folding it into Tesla and just making it an arm of a for profit company he was already running.

[Sam Altman]

He wanted to be CEO and have controlling voting power.

[Gregory Warner]

So Elon says, if it's going to be for profit, then he's going to be in charge.

[Interviewer/Commentator]

Yes. Or he's going to walk. And so now OpenAI has a decision to make lose Elon Musk and his celebrity and his money and his tech prowess, or change the structure of their company that they designed specifically not to have one person be the ultimate controller of this technology that they think is going to be so powerful that no one man should wield it. Right?

[Gregory Warner]

Right.

[Interviewer/Commentator]

1. OpenAI co founder, Ilya Suskever. He wrote to Elon in this email and he says, quote, the goal of OpenAI is to make the future good and avoid an AGI dictatorship. You are concerned that Demis could create an AGI dictatorship. So are we. So it is a bad idea to create a structure where you could become a dictator if you choose to.

[Gregory Warner]

And so what does Elon do?

[Interviewer/Commentator]

Well, he responds to this email, guys, I've had enough. This is the final straw.

[Sam Altman]

And then Musk decides in a huff, if this is not going to stay a nonprofit and it's converting to a for profit where I am not in total control, I am leaving.

[Interviewer/Commentator]

And not long after, he quits OpenAI. But within a few months, he starts going around on a very different kind of campaign, essentially telling people that he has also quit trying to sound the alarm about AGI and what it's gonna give rise to. 4, 3, 2, 1. Boom. Thank you. Thanks for doing this, man. Really appreciate it.

[Elon Musk]

Hey, you're welcome.

[Interviewer/Commentator]

Very good to meet you.

[Elon Musk]

Nice to meet you, too.

[Interviewer/Commentator]

And thanks for not lighting this place on fire.

[Elon Musk]

You're welcome.

[Interviewer/Commentator]

And it's at this time that Elon goes and makes his first appearance on the Joe Rogan Experience.

[Gregory Warner]

This is the infamous episode where Musk smoked pot on camera.

[Interviewer/Commentator]

Yes.

[Elon Musk]

I mean, it's legal, right?

[Interviewer/Commentator]

Totally legal.

[Elon Musk]

Okay.

[Interviewer/Commentator]

Tobacco and marijuana in there. How does that work? Do people get upset at you if you do certain things? And as you remember, this became like, a whole big thing.

[Expert/Analyst]

The stock value of electric car manufacturer.

[Interviewer/Commentator]

Tesla tumbled 9% Friday morning. Billionaire Tesla head Elon Musk. What is he up to? Shares in Tesla took a hit shortly after video was posted of CEO Elon Musk apparently smoking pot. But of all the spectacle in this moment, there was this other part of the podcast that even I didn't really notice until I went back recently and listened to it. You scare the shit out of me when you talk about AI. Between you and Sam Harris, I realized, like, oh, well, this is a genie that once it's out of the bottle, you're never getting it back in.

[Elon Musk]

That's true.

[Interviewer/Commentator]

Are you honestly, legitimately concerned about this is like, AI one of your main worries in regards to the future?

[Elon Musk]

It's less of a worry than it used to be, mostly due to taking more of a fatalistic attitude.

[Interviewer/Commentator]

Musk basically tells Rogan, hey, I did my best to warn people.

[Elon Musk]

I tried to convince people to slow down, slow down AI, to regulate AI. This was futile. I tried for years.

[Interviewer/Commentator]

This seems in a movie where the robots are gonna fucking take over and you're freaking me out. Nobody listened.

[Elon Musk]

Nobody listened.

[Interviewer/Commentator]

No one.

[Elon Musk]

You met with Obama just for one reason, because we were talking about AI. Yes, I met with Congress. I was at a meeting of all 50 governors and talked about just AI danger. And I talked to everyone I could. No one seemed to realize where this was going.

[Gregory Warner]

After a short break, OpenAI now without Musk stumbles into a breakthrough that will transform the industry and yet again make the people most concerned about building AI safely decide that they need to build this even faster. Stay with us.

[Sponsor/Advertisement Voice]

The last invention is sponsored by Ground News. Ground News is one of the most helpful tools that I use to avoid the echo chambers and media bias online, especially when it comes to shining a light on our blind spots. So whether you're politically on the left or the right or somewhere in the center, the blind spot feature from Ground News highlights the stories that tend to be disproportionately covered by one side or the other. As an example, take these two stories about President Donald Trump, one which had low coverage among left leaning outlets.

[Interviewer/Commentator]

It's a very important relationship we're going to get along good with.

[Sponsor/Advertisement Voice]

China reported that Trump says US will accept 600,000 Chinese students as part of a trade deal.

[Interviewer/Commentator]

I hear so many stories about we're not going to allow their students, we're going to allow. It's very important. 600,000 students.

[Sponsor/Advertisement Voice]

And another largely uncovered by right leaning outlets.

[Sam Altman]

Trump's social media company is using crypto.com's.

[Sponsor/Advertisement Voice]

Trump family crypto empire expands with crypto.com partnership.

[Elon Musk]

That's our transactional Trump family. Make some money when you can by.

[Sponsor/Advertisement Voice]

Seeing which stories are amplified or ignored depending on the outlet. Ground News helps you step outside the filter bubbles that shape most people's news diets, giving you a fuller picture of what's actually happening. I really think that if you like this podcast, you're going to like their mission. Go to ground news.com invent to get 40% off the same unlimited access Vantage plan that we use. You can even sign up to stay up to date with the biases in your coverage proactively with the weekly Blind spot report delivered directly to your inbox. This is a great way to support them and the work that they do. Because Ground News is a subscriber supported platform, we appreciate what they're up to. We appreciate their support for this podcast. So go check them out and make sure to use our link ground news.com invent so they know we set you.

[Interviewer/Commentator]

Foreign.

[Sponsor/Advertisement Voice]

This episode of the Last Invention is brought to you by fire, the foundation for Individual Rights and Expression. There's a pattern that you can trace throughout history. In ancient Athens, Socrates was put to death for asking tough questions of the powerful. Centuries later, monarchs banned and burned books they considered dangerous. And in the last century, authoritarian governments shut down newspapers, censored broadcasts, even jailed their critics. The struggle was always the same. Who gets to decide what people can know. Today, that struggle is playing out in a new arena, and the risk now is subtler search results that quietly vanish, recommendation engines that steer us towards safe and comfortable answers, and AI filters that can suppress ideas before we ever even see them. That's where fire comes in. Fire has spent decades defending free inquiry on our campuses, in the courts, and. And in our culture. And now, through a $1 million grant program in collaboration with the Cosmos Institute, they are supporting projects that keep free thought alive in the era of AI. Join us [e-mail] TheLastInvention. By supporting FIRE, you're protecting the future of free inquiry in America and ensuring that tomorrow's most important questions can still be asked. Once again, visit thefire.org thelastinvention and thanks.

[Gregory Warner]

Okay, so once Elon Musk walks out, what happens at OpenAI?

[Interviewer/Commentator]

Well, it turns out that even though this was a nightmare for everyone at OpenAI, and they were worried that this might spell the end of the company, it kind of turned out to be a blessing in disguise.

[Keech Hagee]

I talked to Andrej Karpathy, who worked for both Elon at one point and OpenAI, and he said, you know, in the beginning, OpenAI was trying to copy paste DeepMind, and in the end, it turned out that DeepMind had to copy paste OpenAI.

[Interviewer/Commentator]

As Kiche was saying it to me pretty much the whole time that Elon was at OpenAI, he was pushing them towards this AI game player strategy.

[Keech Hagee]

And after Elon left in 2018, over in the corner, a completely different researcher had a breakthrough with a completely different technology, a language model.

[Interviewer/Commentator]

And it was only after he was gone that instead, they focused eventually on language. And that is how, by 2022, they flip everything and have Demis and Google chasing after them instead of the other way around.

[Expert/Analyst]

The next generation of artificial intelligence is here.

[Interviewer/Commentator]

It's called ChatGPT. But before that would happen, there was another split inside this company. And it's a split that some people in Silicon Valley think may end up being far more consequential even than Elon Musk's. And this is the paradox of Dario Amadei. Who is Dario Amadei? Why does he end up at OpenAI? And what is it that he really contributes to the team there?

[Expert/Analyst]

Dario is like Demis, actually Has a neuroscience background, which is one interesting thing. He is also very interested in the brain, which sort of ends up informing a lot of his theories for AI systems and how they should work.

[Narrator/Host]

What he was really known for at OpenAI was his emphasis on safety.

[Interviewer/Commentator]

Jasmine sun and Kevin Roose, they're both working on a book right now about AI, and Dario is one of its central characters.

[Narrator/Host]

He is, by his own admission, kind of a nervous person. And he was really a pioneer, not just in developing AI systems, but in worrying about them and how they might go wrong.

[Expert/Analyst]

He does end up being extremely concerned about AI risk and the potential for systems much smarter than us to develop their own goals, become unaligned with, opposed to, or just sort of not caring about human goals, and then to sort of end up taking over, screwing humans up.

[Gregory Warner]

So Dario was one of the people who came to OpenAI as someone who's motivation was more about stopping a dangerous AI.

[Interviewer/Commentator]

Yeah, he's very worried about superintelligence. He's associated with this group called the Effect of Altruists. And he says that he comes to OpenAI in large part because of this altruistic, safety focused mission that they have. And how do you sum up what it means for someone like Dario to study AI safety? Like, what exactly is AI safety?

[Narrator/Host]

So AI safety is a big field. It contains a bunch of different subfields. One of them that Dario and his colleagues have been very instrumental in is called mechanistic interpretability. That's a very long name. I've told them they should rebrand it to something people can actually pronounce, but they listen to me. But basically, mechanistic interpretability is the science of figuring out how AI models make decisions, why they behave like they do, what is going on inside the guts of the system.

[Interviewer/Commentator]

So he's studying the mysterious AI black box.

[Narrator/Host]

Yes, Making that interpretable to humans. Okay.

[Gregory Warner]

So he's trying to probe the AI to figure out why it's doing what it's doing and is it going to do anything we don't want it to do.

[Interviewer/Commentator]

Yes, that's one part of AI safety. There's other parts of it too. The other thing that Dario is really into is size.

[Narrator/Host]

Early on in his career, Dario Amadei had worked on a project at Baidu, the Chinese Internet conglomerate, that dealt with these so called scaling laws. And this was a theory at that time, it was an unproven theory that basically the key to making an AI system more intelligent was just making it bigger and training it on more data. This was sort of Countercultural. In AI research at the time, lots of people were theorizing that you needed some clever new algorithm or some very different architecture to make these models smarter. But Dario and his colleagues sort of had this idea that you could actually just make them bigger and the systems would get smarter.

[Interviewer/Commentator]

And so the idea here is just that if we take an already promising neural network AI system and we just make it bigger, that maybe like the human brain, which is bigger than the bird brain or the cat brain, is smarter, this thing will also get smarter and smarter, and maybe one day we'll even become a general intelligence.

[Narrator/Host]

Yes. And essentially this is sort of his best guess at how companies like OpenAI are going to get more intelligent systems. It's not by training them on more specialized data. It's not by coming up with clever efficiency hacks. They are just going to make the models bigger, and that is going to take care of a lot of the problems.

[Interviewer/Commentator]

So his theory is that if you just take a promising AI model for OpenAI that became their language model, that is a neural net that looks for patterns in text and language, then what you just need is a massive amount of text and data to pump into it, as well as a massive amount of GPU computer chips, which are, of course, very expensive.

[Gregory Warner]

And this is why they're pretty sad to lose Elon Musk and his money.

[Interviewer/Commentator]

Yes, this is one of the reasons that it was sad to see Elon walk out the door. But not long after he does, Sam Altman goes out. He does the thing that he's so good at. He knows that they need lots of money, lots of GPUs. So he goes out and strikes up a partnership with one of the biggest companies of all time, Microsoft.

[Sam Altman]

So Microsoft ends up fulfilling both of these things. They become the largest investor in OpenAI, and they partner to build the supercomputers that OpenAI needs.

[Interviewer/Commentator]

Greg, this is a detail of the story that's especially wild to me when Dario and his team at OpenAI, when they get access to these Microsoft supercomputers, they decide to take their scaling theory as far as they possibly can.

[Sam Altman]

Dario Amade was really pushing for the idea of, no, we really go big or go home.

[Interviewer/Commentator]

So, for example, at DeepMind, when they did that Atari demo, they were using just one GPU for the richest universities.

[Sam Altman]

In the world, like mit, Stanford. It would be a big deal to have a few dozen chips. And in places like India, you would have grad students, multiple grad students, sharing one computer chip. So they're trying to do their research on fractional amounts of GPUs.

[Interviewer/Commentator]

But with this Microsoft partnership, OpenAI now has access to these supercomputers with thousands and thousands of GPUs. And so, as the story goes, Dario approaches the leadership team at OpenAI and says, Guys, what if next time, what if we take this model and we crank it up to 10,000 GPUs?

[Sam Altman]

So 10,000 computer chips. I mean, no one had ever thought about that before. Like, that's bananas. Actually, like, within OpenAI, this was a contentious decision because some people were like, is that even possible? That just seems improbable. Other people were like, no one has ever done this before. And if we think that AI could go badly, maybe we should more gradually scale it, not just do this dramatic step change. But his philosophy was, we need to accelerate the development of this technology so that we can then retain hold of it and figure out how to perfect it in the lead time that we have over other potentially bad actors getting a hold of it. And Sam Altman also really liked the idea because his entire career has been adding zeros to things. So he was like, let's do it. And Ilya Sutskever, also, philosophically, was always more in the camp of scaling will potentially bring wondrous and potentially terrifying things, but we should not be afraid to go in that direction. And so the main people that were running OpenAI all converged on, yeah, let's give it a go.

[Interviewer/Commentator]

However, if you are going to massively scale up your GPUs, your compute, you also have to massively scale up the data that it is searching for patterns inside of. Because just think of it like, no matter how smart and powerful it is, if it only has access to a limited amount of data, it's never going to truly become like an artificial general intelligence.

[Gregory Warner]

Or it's almost like if Einstein, as smart as he was, if he'd only ever read one book, he would know.

[Interviewer/Commentator]

That book really well.

[Gregory Warner]

Right? But he wouldn't be that smart.

[Interviewer/Commentator]

Yes. And so they're going to need a massive amount of data to match the massive amount of compute. But here's the problem. There's only so many open source free databases on the Internet. And So here's where OpenAI does something that right now has a lot of people, a lot of different corporations suing them, including the New York Times, because it appears, and some former OpenAI employees have leaked some details about this, that they just started dumping big chunks of the Internet into their AI. I feel like this is going to be the pinnacle Scene in the inevitable Hollywood depiction of this story one day where they're just ramping up all the stuff that they're throwing into their system. Like, oh, here's a free database, let's put that in. Oh, look, let's put some Reddit in there.

[Keech Hagee]

Yeah. And then, hey, how about Wikipedia? And then, hey, man, these researchers over at the University of Toronto just scraped all these books off the Internet. I'm sure they wouldn't care if we just took all them copyrighted books and fed that to the LLM. No problem.

[Interviewer/Commentator]

Right, what else we got?

[Keech Hagee]

Yeah, that's pretty much what happened.

[Interviewer/Commentator]

Allegedly they started throwing in scientific journals, news articles, blogs, transcripts from YouTube videos. They just kept going and going.

[Gregory Warner]

And how far does this go? Do they feed it the whole Internet?

[Interviewer/Commentator]

Like I said, there is a lawsuit that's happening right now. So we're going to learn more details, I think, as information comes out from those suits. But it's been reported that basically if a website or if some text online didn't explicitly have a label on it saying, do not use this to train your AI, they adopted a stance of better to ask for forgiveness instead of permission.

[Gregory Warner]

So all we know is that they just dumped a lot of the Internet in the system. We don't know how much or what exactly they put in there.

[Interviewer/Commentator]

Yes. And we know that this is eventually the strategy that would give birth to what we now call ChatGPT. And so Dario is both the guy who is saying, let's scale this thing up further and faster than anyone has before, let's crank this up to 10,000 GPUs. But he's also the safety guy. Isn't there a tension between those two, between, let's crank the knob up to 11 and oh, we really need to make sure this is safe.

[Narrator/Host]

Yeah. This is sort of the classic paradox of Dario Amadei and of AI safety in general, is that they, on one hand, fear the effects and implications of these very large, very powerful models, and they're trying to build them and stay on the cutting edge of AI capabilities. And I've asked Dario about this before and he says in order to be able to study the safety challenges of very powerful AI systems, you have to have very powerful AI systems to use as your testing grounds. You can't sort of learn about safety on a Formula one car by practicing on like a, you know, jalopy of a, you know, 10 year old Honda Civic. It just won't teach you that much about what kinds of risks are going to take place when AI is very powerful.

[Interviewer/Commentator]

And so the argument here is that to make a powerful AI that is safe, that is good for humanity, you are going to need to learn about powerful AI systems and test powerful AI systems. And so therefore, you're going to have to make one.

[Narrator/Host]

Yes. If you want to do cutting edge AI safety research on very powerful systems, you need to actually build those powerful systems.

[Expert/Analyst]

I think the other thing that, like Andari Amade would say is, like, whichever system is the best, it's going to be embedded in every part of society. Like, we're going to use it to make decisions about who to give a loan to. We're going to use it to plan our cities. We're going to use the super intelligence to maybe even figure out our military strategy. And so the only way to have the impact that we want to have in the world, to ensure that we have superintelligence and that the superintelligence does things like curing cancer instead of screwing us all over and self sabotaging ourselves, is by having both the safest and the best model. Because if ours is safest, but it's not actually a very good model, then the unsafe model is going to be the one that's going to be widely deployed, and that's a much worse world.

[Narrator/Host]

And the last one I've heard him make is the only way to stop a bad guy with a powerful AI is a good guy with a powerful AI. Essentially, this is the argument that this technology, it's so powerful and so lucrative that someone is going to build it. And in Dario's mind, that someone could be an authoritarian government. It could be a rival AI company that doesn't care as much about safety. It could be a terrorist group. And so the ethical thing to do in his mind, if you are concerned about the power of these AI systems, is for you to be the one who builds it and keeps it safe and kind of sets the high bar of safety that the rest of the industry will have to follow.

[Interviewer/Commentator]

And it was this mindset, this idea that to make AGI safe, you need to make it fast. This is part of what Drew Dario to OpenAI in the first place. This is a part of the mission that he loved. However, one day late in 2020, he just up and quit, along with several members of the AI safety team. And they went out and pretty much immediately started a rival AI company called Anthropic. All right, so Kevin Dario Amade has yet to accept my interview request, although the people that he works with are very nice. And I had A nice meeting with him and maybe one day he will come on the show. But in the meantime, I know you've spoken to him. What do you understand is the reason that he leaves OpenAI? What does he see there? What is it he doesn't like?

[Narrator/Host]

So the official story of why Dario and his colleagues left OpenAI is that they had philosophical differences about AI safety approaches and priorities, that Dario and his team wanted the company to put more emphasis on safety, and that others at the company were less interested in that. I think the real story is more complicated and involves a lot of not only philosophical differences, but also like real personal differences and beefs. Lots of disagreements I've heard about in reporting about specific decisions they were making, whether they were taking safety seriously enough, whether they were becoming too commercial. I mean, you have to remember that When Dario joined OpenAI, it was a research nonprofit. It was specifically set up not to be and act like a normal AI company. And by the time he left, it had started this for profit subsidiary, it had struck this deal with Microsoft. It was starting to look more and more like a kind of normal tech startup.

[Interviewer/Commentator]

And.

[Narrator/Host]

And I think that made him and his colleagues very uncomfortable. And there are some other juicier stories that I'm going to save for my book.

[Gregory Warner]

So we don't know what Daario saw that scared him?

[Interviewer/Commentator]

No, we don't know. I mean, maybe Kevin knows something and we'll just have to wait for his book. All we know is that he quits. He says that it's connected in some way with AI safety and that he opens a competitor claiming that now he's going to be the one to make AI truly safe.

[Gregory Warner]

And so now we have more competitors in the race.

[Interviewer/Commentator]

Yes. And this pushes everyone to work even faster. And really where you see that most dramatically is around ChatGPT, because OpenAI had already decided that eventually they wanted to release a version of ChatGPT to the public, but they didn't think it was quite ready. It had gone from GPT3 to GPT 3.5, but it was still buggy. It still regularly had these hallucinations that they didn't understand. So they were trying to hit their benchmark of GPT4 before going public with their chatbot.

[Sam Altman]

But suddenly this rumor starts to spread within the company that Anthropic also has a choppa and they might release it soon.

[Interviewer/Commentator]

Allegedly, this rumor starts to go around OpenAI that Dario Amadei and Anthropic are planning to release their own chatbot and to do it before OpenAI can.

[Sam Altman]

And so OpenAI executives make a decision. We are not going to wait for the GPT4 launch because the model's just not ready. But we have the chat interface and we have GPT 3.5.

[Interviewer/Commentator]

And so they're nervous that if Anthropic beats them to market with their chatbot, then OpenAI is going to seem like they're behind the ball, they're going to come off like a copycat.

[Sam Altman]

They're operating, operating under a very Silicon Valley belief of winner takes most. So you need to be the number one. You need to be the one that has the name recognition, the one who invented this kind of chatbot.

[Interviewer/Commentator]

And so they just decide, you know, let's do a low key, you know, no press release, no advertising, no social media blitz. Release of ChatGPT 3.5 and Keech Hagee and Karen Howe, they were telling me that supposedly back at OpenAI, the team did not think that this was going to be a very big deal outside of Silicon Valley. Like they didn't think it was going to make a very big public splash.

[Gregory Warner]

And so why release it if they didn't think it was going to be a hit?

[Interviewer/Commentator]

Well, in some ways it was like insider signaling to just say to the world of technology, we were here first. It doesn't matter if the public uses it or not. It mattered that the world of technology doesn't think that they are just copying off of their rival's anthropic inside the company.

[Keech Hagee]

It was a low key research preview is how they described it. Let's just release this model with this new interface that is just like a chatbot and see what people think.

[Sam Altman]

The night before, they were like making bets on how many people would actually start using the model in I think the first weekend. And the highest bet was 100,000. So that's how many users they provisioned their servers for.

[Interviewer/Commentator]

And so on November 30, Sam Altman goes onto Twitter and he just writes, Today we launched ChatGPT. Try talking with it here. And he pastes a link.

[Expert/Analyst]

The next generation of artificial intelligence is here. The future is now.

[Interviewer/Commentator]

The Internet's going crazy over new artificial intelligence called ChatGPT.

[Expert/Analyst]

A new artificial intelligence chat bot.

[Interviewer/Commentator]

ChatGPT is like a Google.

[Sponsor/Advertisement Voice]

You can ask to do things, they.

[Keech Hagee]

Can answer essay questions, write songs, who.

[Sponsor/Advertisement Voice]

Knows what companies and ethical issues.

[Expert/Analyst]

It already has more than a million.

[Keech Hagee]

Very creepy.

[Expert/Analyst]

A new artificial has gone viral.

[Interviewer/Commentator]

Probably the area that caught the eye.

[Keech Hagee]

Of Wall street this week.

[Interviewer/Commentator]

To some establishment companies.

[Sponsor/Advertisement Voice]

Next time on the last invention, I'd.

[Interviewer/Commentator]

Love if you could just take me back to this time period in your life where after years of being on the fringes, as you've described it, being rejected, you and Hinton and your fellow connectionist AI researchers, your contrarian views are proved right. And then all of these actual AI systems, these promising new technologies are born out of you guys determination to chase after this idea. Despite all the naysaying. How did that feel? Like, I imagine it felt really good. Oh yeah. I mean, it was. It was great.

[Narrator/Host]

It was.

[Interviewer/Commentator]

Let me share something emotional. So shortly after AlphaGo, I don't know, maybe 2018 or something. Oh, I guess that's when I got the Turing Award with Jeff and Jan. I thought, I've achieved the greatest prize that a computer scientist can expect in their life and I've accomplished so much and my career has been so rewarding and successful. What else is there to do? I felt like if I die tomorrow, I'll go with serenity. You did it. But. Wait, but. There's a but. November 22, chatgpt it dawned on me. Yes, but look, this has been a really big step. How far are we from human level? Maybe just a few years, maybe a decade, maybe two. And then what? What's going to happen with this kind of technology? Aren't we going to build machines that we don't control and could potentially destroy us? How do we make sure this doesn't happen? And I didn't have an answer.

[Sponsor/Advertisement Voice]

The last invention is produced by Longview, home to the curious and open minded. We are an independent outlet focused on giving people the backstory to the debates shaping our future. To support our work, click on the link in our show notes or Visit [e-mail] and become a subscriber. And as always, it really helps us if you leave a rating and a review on Apple or Spotify or wherever you listen to your podcasts. One last thing to mention. Audio from the documentary AlphaGo was used in this episode. Thank you for listening and we'll see you soon. This episode is sponsored by Ground News, the app that helps you spot media bias and see a broader picture of the news shaping our world. Get 40% off their vantage plan at Ground News Invent. This episode is sponsored by Fire Defending Free Thought in the Age of AI. You can learn more at thefire.org thelastinvention.