Loading summary
Transcript105 lines
- [00:00]
A
Hey, everyone, it's Bradley. Before we dive into today's episode, we've got some interesting news. Firewall was nominated for its first ever Webby Award. If you're wondering what are the Webby Awards, it's like the Oscars for Internet stuff, and we're up for best Individual episode in the podcast category. And the way they determine the winner is through voting. And we could really use your help because we're up against some really heavy hitters, including Oprah. So if you go to the Show Notes, you can find the link that'll take you right to the ballot. Or if you go to Webby Awards, you can navigate your way there too, and we'd really appreciate the support. Thank you. All right, welcome back to Firewall. I'm your host, Bradley Tusk. I have a really exciting guest today, Sebastian Malaby, who I think a lot of you already know. He's done so many incredible things. He's currently senior Fellow at the Council of Foreign Relations, contributing editor at the ft. He's written multiple incredible books, but he's got a new one out that excited to talk to him about called the Infinity machine demos. Hasabis, DeepMind, and the quest for Superintelligence. And the thing that ultimately led to me asking Sebastian to join us was this incredible op ed that he wrote in the Times entitled, this is what Convinced Me opening Iowa, we're out of money. And that just kind of blew my mind. So, Sebastian, thank you so much for being here.
- [01:23]
B
Really appreciate it. Great to be with you.
- [01:25]
A
Yeah. So, you know, you made this. I guess the first thing is, before we even get into the why, how much shit did you get for what you wrote in the Times?
- [01:34]
B
Not so much shit, actually. I got the opposite. I got people saying, well, you finally said what I thought, and I didn't say it, but you said it.
- [01:40]
A
And you didn't have, like, Chris Lehane running attack ads against you and stuff because you dared to be critical of OpenAI.
- [01:46]
B
Nothing. Not a squeak from Chris Lehane.
- [01:48]
A
All right. Amazing. Amazing that you might be the first one.
- [01:51]
B
I'm still here, right?
- [01:52]
A
You're still. Yeah, yeah. You know, you might want to watch your back. But. But anyway, so let's just start with this, which was, you know, you made the case that OpenAI may run out of cash before it turns a. Prof. And in some ways, maybe I just read it this way, and you mean it, but Sam Altman came off as sort of a great pitch man, but not necessarily a great business person or technologist. Give us your Thesis here, please.
- [02:19]
B
Okay, so I would say that Sam Altman is a great pitch man. He's also just a very smart guy and you can read his stuff, his essays. He's thoughtful. He had vision to have the idea of OpenAI in 2015. Not as early as Demis Hassabis, who was 2010, but still he was pretty early. So credit where credit is due. I think, though, he represents the kind of distilled essence of Silicon Valley, meaning he's incredibly good at the network. He's embedded in the Silicon Network, Silicon Valley network. And that means he can recruit great people because he has a network. He can raise money like nobody else because he has a network. He can tweet or X or whatever you call it. He can put stuff on social media and the echo chamber will magnify what he says. And he's used those three superpowers in terms of recruitment, money and message to propel OpenAI so far that it's over its skis. And it's over its skis in two ways. First is just the valuation. Whatever it is, it's like 800,000,000 billion.
- [03:19]
A
Now it's close to a trillion.
- [03:21]
B
Yeah, it's kind of double anthropic, roughly speaking. And that's just a metric of how when you're that high up in your valuation, you can't have a down round because then everybody's stock options among your employees, they all get wiped out. So you need. It's like a shock. You have to keep on going forwards. And I don't see how you keep on going forwards from this incredibly high valuation. They already have, given the fact that they are bleeding money. And that gets to the second point. The money bleed is not just a normal Silicon Valley. A little bit. We're going to get there soon. This is epic. In terms of the amount of cash bleed, their own Internal numbers suggest 660 billion of cash burn between now and 2030, when they expect to turn a profit. Now those estimates were leaked. They're a little bit old, maybe a couple of months. And I would say that I wrote the Times op ed maybe at peak, like the peak moment to do that. Since then, I would concede that OpenAI has canceled Sora, for example. That was a huge cash burning. Video generation is very expensive. It's mostly slop. So people won't pay you to do this. So it just made no sense in business terms. So that was good that they got rid of that. They've also backed off of shopping on their app. Who knows what's going to happen to this much ballyhooed partnership on hardware with Jony. I've. But I suspect they're just going to let that fade away. They could rationalize what they're trying to do, cut their cash burn and maybe make it. But then you just add the final kicker, which is Iran. Iran has pushed up the path of interest rates. It's made it harder to go public for everybody. And I just think they're in a really hard spot. So I think what I said in the piece in the New York Times, which I still believe really is there's kind of a 50% chance that in the next 18 months they run out of cash and somebody has to buy them.
- [05:19]
A
So the 660 billion, is that mainly for infrastructure of data centers and the energy costs around it, or is it to kind of keep creating new verticals?
- [05:30]
B
My understanding it's. You can't separate out the spend on the data centers and the energy from the fact that you're creating new products and you need to serve those products. But I think a large amount of it is data centers and energy and then some of it is just hiring people. You know, it's a very expensive field. When you're competing with the crazy engineers
- [05:52]
A
getting $100 million, the crazy guy over
- [05:54]
B
at Meta is just bidding out the wages and.
- [05:57]
A
Right. So last I checked, they had something like 15 to 20 billion dollars in true revenue off of a valuation of nearly a trillion dollars. If you were trying to put yourselves in the shoes of someone who invested in the last round at that valuation, which means you have to believe that ultimately revenue is going to increase exponentially to justify it. Where does it come from?
- [06:22]
B
Yeah, I mean, you need to make the app sticky with your customers so that they don't leave when you either serve them ads or you charge them money to use it. Right now there are like 920 million regular users. It's huge, huge user base, but only something like 5% pay them any money. And the reason is that if you tried to charge them money, they would just go use Anthropic or Gemini or something. It's just not sticky. So I think we have to think forward to a time when the systems are more agentic. They've got your credit card, they're doing your shopping, they're paying your utility bills, they know everything about you because they've had conversations with you for five years and now it's kind of like a bank account. It's a pain in the neck to switch. And at that point you can start charging money, right, but that's some way off. And then they need to ramp enterprise. That's probably even bigger than the consumer side, but they need to ramp enterprise. And, you know, so far Anthropic has been more focused on that and better at it. Right.
- [07:18]
A
And most of the examples of other tech companies with sort of valuations that are somewhere near with the letter T, you know, in my experience. So whether it's Google or, you know, the meta products, whatever it is, it's still less about capturing someone and then charging them more and more money, as opposed to a free product with massive amounts of advertising and selling their data and making money, but in other ways. Are there really good examples that you can point to of the, hey, we're gonna get them in the door for free and then ultimately we're gonna charge them so much and they're gonna be so stuck with us that they're gonna have to just keep paying it?
- [07:54]
B
I mean, you know, there's a freemium tradition, right? Spotify, you can use that, but that's
- [07:59]
A
not in the trillion dollar ballpark.
- [08:02]
B
Maybe you have a point. I mean, you know, and maybe does that make us more worried then about OpenAI?
- [08:07]
A
I mean, I. Yes, right. I mean, yes, like me, right, there's subscriptions that we all have, Netflix, Spotify, whatever it is. And you know, whether those are properly. I haven't checked the valuations of anything lately. They may or may not be properly valued by the market, but nonetheless, no one is talking or thinking about them in the way they are open AI, nor are they spending anywhere near the kind of money that OpenAI is. Like, I'm not even really sure. Like, you know, my day job as a venture capitalist, most of my investment to this point involve AI in some way, because what. What wouldn't at this point? But, you know, I, I don't even know that I understand the revenue path to get there, let alone, you know, so. So given that, I mean, you have spent a career kind of analyzing and observing the mindset of investors in different ways you invested that, you know, most recent round, what are you thinking? Why are you doing it? If the path for ever generating the revenue to justify the valuation you bought in at, let alone the 3.4x that you're hoping to achieve.
- [09:13]
B
Yeah. Yes. Yeah. Well, you know, like you, I'm not a venture capitalist, but I do go to conferences with them and I read a book about them. And so, yes, you know, on my trips out to the west coast, when I go for walks with people and Pose this question. The answers are kind of interesting, right? I say, why the heck does anyone invest in this valuation? And the answer is some of the people who looked like they invested, really, they just organized an SPV and it was other people's money. It's basically sovereign money from the Gulf. So that's part of the answer. Then there's all this stuff about warrants which is flagrant. Right. So I think Thrive Capital has a vehicle where the headline valuation of OpenAI at the time they put this together was 500 billion, but people were going in at 375. Because if you put in 100, you got half of that went into warrants which were valued at 250.
- [10:02]
A
Yeah, you got some sort of 25% warrant coverage or whatever it is.
- [10:04]
B
Precisely. So again, that's telling us that the headline valuation of OpenAI, even when it was 500 billion and now it's gone up to 800 something, the people putting money in didn't believe the 500 valuation. So what the heck are they going to believe now? Right. And then, by the way, there was a strong kind of symbiotic partnership between OpenAI and Oracle. There's that point where Oracle was going to do lots of database deals and Oracle's stock price zoomed up and that was the point where Ellison overtook musk as the richest man. Right. Okay. Since then, Oracle has crashed back to where it was before because people just don't believe these contracts on data centers are ever going to materialize.
- [10:52]
A
A circular economy of nonsense in this
- [10:55]
B
time period when Oracle is kind of down 50% or something, OpenAI's paper valuation has been marked up 60%.
- [11:02]
A
Right.
- [11:03]
B
What's up with that?
- [11:04]
A
Right. So I have posited before that whether it's Nvidia, Oracle, OpenAI, whoever they are, disguising short term profiteering, whether it's for the share price or the valuation, as long term planning and thinking. Right. So the market says, oh, you're thinking 20 years down the road we're going to reward you for that. So you're, you know, raising and investing all this money in infrastructure and whatever else of deals that ultimately, like you said, are just very circular in nature. And I'm not sure there's any real there there to it. And it's sort of, they get rewarded short term for it, but there's no fundamentals to really back it up. And do you think Altman just thinks he can be sort of the next, you know, Elon's the only person that I know that can have a company with a valuation around a trillion dollars. I don't know exactly what Tesla is today with. No, with fundamentals that are a fraction of that. Right. Do you think Altman just thinks he can pull that off too?
- [12:05]
B
Yeah, maybe. I mean, you know, it's, it's important to remember that in around 2014, 2015, they were extremely close and Sam pitched Elon on the idea of doing OpenAI together. It was Sam's idea. You can read the emails because they come out in this law case and they apparently at the time were having regular dinners and they were quite close. And definitely Elon is the senior, older, more seasoned person at that point. Sam is just watching him and saying, how can I be that? How can I do that? So I think there's some plausible truth to this, but I also think that what we have to remember is there's a lot of stories in the valley of people who fake it till they make it. And that works. And it's fine because you grow into the valuation and the investors end up being fine. The difference this time is that you've got so much capital that needs to be raised, that this is an experiment not merely in a frontier technology, this is an experiment in how deep are the global capital markets.
- [13:01]
A
Right, right. Is there this much money?
- [13:03]
B
In 2025, they raised 41 billion. And that was bigger than by far, by far bigger than any private fundraise in the history of capitalism. And bigger than the biggest ever IPO in history, which was the Saudi Aramco deal, which raised less than 30 billion for Saudi Aramco. And these guys raised 41 billion in one year.
- [13:22]
A
Right.
- [13:23]
B
And so amazing. Well done, Sam. But you're going to have to do it at time. And if the burn is 660, even if you halve that and say it's really 330, you got to do it again and again and again. And I think the latest raise, which is whatever 100 something billion on paper, you kick the tires on that, there's all these conditional promises the money will come if you go public.
- [13:45]
A
Right. Well, but the problem there is you can kind of keep playing the game, like you said, where you raise and then you kind of grow into the valuation and you have a headline number, the real number, eventually there's gotta be an exit. Right. Or the company goes bankrupt. Right. And if the exit is an ipo, which is the anticipation because no one can buy a trillion dollar company, the capital markets are actually, I find, pretty smart and efficient and you can have whatever bullshit valuation you Want if they don't get with Tesla as sort of this weird outlier. You constantly see highly valued startups eventually go public and then by the time the lockup is expired, you know, the market has cut the valuation by 60, 70%.
- [14:35]
B
Yeah.
- [14:35]
A
So I don't even know how that wouldn't happen.
- [14:38]
B
And you also see things like we work, which had this very high valuation. They put out the prospectus for the ipo. There's too many pictures of the guy with the long hair, and the market says, forget it, we're not buying this. And they have to pull the ipo.
- [14:50]
A
Yeah, yeah, totally. Two more on this and I want to. Want to turn to the. To the book. One is one thing I don't understand. And you know, mentioned Lehane earlier, so maybe his thinking here would be useful if you have insight into it, which is there's all this money raised, pledged to build all of these data centers. Fine. There seemed to be no thought around how are we going to power these things in a way that doesn't, you know, with the exception of Microsoft turned it by Three Mile island or whatever it is, that doesn't put us into PJM and the grid, wherever you're gonna be. And you know, my original background is in politics. And the one thing I can tell you for sure about politicians is they understand what's in their electoral interest and they act in that way at all times. That is one thing I know for sure. And I don't know a single politician that says, I'm gonna tell my voters that when they're upset because their energy bills went up 40% because demand just doubled. And supply, as we know, is really hard to bring new supply online at scale. It's okay, because the most important thing is that Sam Altman become a trillionaire. It didn't seem like anybody thought about this. And now you've got 39 states working on various forms of legislation ranging from flat out moratoriums to prohibitions on anti pass along cost to utility ratepayers. Did this enter from your research or knowledge? Did this enter the thought process at all? And if not, how did it not?
- [16:25]
B
Totally. I mean, I think you're completely right about the political calculus and you said it better than I could. I mean, I've noticed that there are. There's a lot of talk about the midterms being quite AI connected because the groundswell of fury about why are we building this stuff? Right? Why is it good for me?
- [16:43]
A
Right.
- [16:44]
B
It's, as you say, driving my power bills up. And it's going to take away my job.
- [16:48]
A
What? And the data centers don't even create that many jobs.
- [16:51]
B
Right, right.
- [16:52]
A
You can't even argue that.
- [16:53]
B
Yeah, yeah. I mean, exactly. So I think this is politically toxic. I also think that, by the way, the political system is going to have to get around to the bigger safety issues at some point.
- [17:04]
A
Right.
- [17:05]
B
And they're kind of putting it off.
- [17:06]
A
I would argue on that one that if we assume that at least the House goes to the Democrats, which I think feels like a safe assumption, Senate's still less likely, but not this war keeps going. Maybe not impossible. If there's one thing in which there potentially could be in 27, a broad based bipartisan bill that Trump supports too, it would be dealing with AI simply because they're all going to be so terrified of getting thrown out of all of their jobs in 28 that they're out of self protection, going to actually have to act on it.
- [17:37]
B
Right, right. This is coming. I mean, you can't. This is. If this is as big as the Industrial revolution or bigger, which is what I think. You look at the Industrial revolution in the 19th century, that industrial Revolution is followed by political revolution. Right. Karl Marx, 1848, across Europe, different revolutions. I mean, this is super disruptive. And you can't sort of change the way people raise their kids, how they're going to work, how they think of themselves as humans because the rival form of cognition in a machine, you can't do that to them and expect zero political fallout.
- [18:10]
A
No. Look, even I'll give you. I was with my son the other day, 17 years old, Junior high school. He's a rich kid in Manhattan that goes to a fancy private school. This is not a kid that is dealing with lots of problems in the big scheme of things. And I just said to him, hey, what's your biggest societal concern? I thought he would say climate change tyranny, said job opportunity in the world of AI. So this is a kid at an outstanding high school, he's gonna go to a great college, he's gonna have a strong safety net and he's really worried about it. Is that anything everyone else feels?
- [18:39]
B
Yeah. Yeah.
- [18:41]
A
And then given your thoughts around kind of OpenAI's cash flow valuation, then eventual decisions have to make. If they can't pass along their energy costs to consumers, doesn't that change the calculus even worse for them yet again?
- [18:57]
B
Yeah, I think one thing I would say though, on the energy deals is that I know that from friends at Google that they are doing more sensible, innovative stuff around alternative sources of energy. A very clever energy mix. Don't pressure the grid too much, bring in the local community to talk about it. They do more of this stuff than they kind of talk about. Because they don't want to be accused of being woke.
- [19:23]
A
Right.
- [19:24]
B
They don't want to be accused of favoring alternatives, which is the kiss of death in this administration. Right, right. So they just do this quietly.
- [19:30]
A
But then they're also, because Altman has gone so in on MAGA and Trump. You know, I would argue that unless AOC is the nominee, there's a extremely strong chance that the Democrat, whoever it is, wins in 28 simply cuz the pendulum, even if it had just been a normal Republican and not Trump, I think she's gonna keep shifting back and forth with Trump. Whole different ballgame. What's their plan in 29 if it's president? You know, I don't know, Gavin Newsom or Gretchen Whitmer, whoever it is. And they've totally put their lot in with the defeated party.
- [20:03]
B
Right, right.
- [20:04]
A
That just seems like another black mark if you're gonna go public. Right. How are you gonna do that? Last one of this. And then I wanna turn to the book, which is it does feel like even since your op ed came out, anthropic in sort of at least the court of public opinion has really surged past OpenAI, both in terms of. I think they played the whole Pentagon thing brilliantly. But even if you put that aside, and maybe I'm not a good proxy for this, but when I talk to most people, they talk about Claude. Right. They don't talk about OpenAI. It just feels like that's becoming, at least among the cognizanti, the kind of default platform. Has it changed meaningfully since you even wrote that piece and started sort of thinking about all this?
- [20:48]
B
You know, I'd say on the frontier models, you're right about Claude. At the moment they had this agentix system that dropped in January, I think, and that really caught people's attention. But in my experience of watching this quite closely over the last few years, each lab seems to have a moment. At the beginning, OpenAI was way ahead. But in the last, say year or so back in November, Gemini was the leading model, the one from Google. And now it's maybe Claude. It just goes up and down. It's not game over. They're all pretty close.
- [21:23]
A
All right, Demis Asavis, which is the person that so we know, we've met, we've named a bunch of people in the last 20 minutes. That the audience totally knows who is he and why did you choose to focus on him?
- [21:35]
B
So I'd say he's the most fascinating of all of them because he's the guy, he's the guy who really kicked all this off. In 2010 he founds DeepMind, which is this London based lab with a mission explicitly to build artificial general intelligence. His co founder Shane Legg was a computer science PhD and his PhD was in AGI. How you define it? He was talking about this in 2010, remember ImageNet when recognition of cats. That breakthrough was like end of 2012. So this is two years before AI could do anything and they're out saying we're going to do a lab to build AGI. So give him credit for the early vision. Actually my book I trace back to his years in college. He already believed it in the mid-90s that AI was going to be the thing, the biggest invention ever, and that he was going to bring it into the world. So he is an extraordinarily early convert to this vision. He then pursues the vision. He has the guts to start an AI company in London. Right? It's not even in Silicon Valley. Nobody starts companies in London in those times. He did. Of course he has to come to the US to raise the money. He goes to see Peter Thiel, he talks the money out of him, gets the first check. Then he gets Elon to invest a bit later. Then he has Selena Chao, the Li Ka Shing partner investing out of Asia. So he's building this company and everybody else is just like waking up to AI might be a thing. Then just at the moment when they all wake up, he organizes this sort of clever auction where Meta wants to buy him. Actually it was Facebook back in the day. Facebook wants to buy him and Google wants to buy him. And he kind of plays them off against each other. And he sells in 2014 to Google. And so at this point, there he is, he's got Google's checkbook, deep pockets behind him. He's already built a thing called the Atari system, which agentic system that could beat Atari, beat the computer at Atari, exceed human levels. He's on his way to doing the AlphaGo thing where he beats the top human player of Go in South Korea in a big match. That's 2016. So he's doing all of this stuff before anybody's even kind of woken up really. So that's the first thing, he's super early. And the second thing is that he is this mixture of a scientist and we're talking serious scientist. Nobel Prize scientist. He has Nobel Prize. And he's a business guy as well. That's pretty dang.
- [24:01]
A
Yeah, give me a comp. Who else would you say could even fit that category?
- [24:06]
B
Well, it's clearly not Sam Altman because he doesn't even have one degree. He dropped out of Stanford. It's not really Elon, because he's more of an engineer than a pure scientist. Although credit he. I respect his command of rocket physics for sure. I'd say the closest compass, Dario. And Dario again is a proper research scientist who has shown quite a lot of ability to lead a lab. So I think that's the closest computer.
- [24:36]
A
But when he made the. When he kind of had that auction and chose to sell to Google just for the audience, like it wasn't just about the highest number. Right. And this is part of why I think the book's so interesting and why you find it interesting was that he wanted. And actually he actually cared about ethics and he cared about safety. And so like, how do you. Like why do you think that he saw this in a different way and then how much of what he wanted to happen has actually happened Happen.
- [25:06]
B
Yeah, he really, really cared about safety. You're right. In fact, he had bonded with his co founder Shane Legg at a safety lecture in London about AI safety. So this was right in the origin story of the company that they cared about safety. And when he sold to Google, as you say, there were red lines. One of them was, you can't use this AI for military purposes. Another red line was you have to have a sort of ethics oversight board which dilutes the corporate Google board's control of how you deploy AI. It's got to be some outsiders. He was thinking like big famous names at one point. They wanted Barack Obama to come and do it. So they were very ambitious about who they wanted. And that was the terms on which they sold to Google. Now it was a massive fight to actually make any of that real. And one of the things I discovered which kind of annoyed Demis Hassabis, was he actually threatened to quit Google, spin his company back out again. And he had this whole team of lawyers and advisors and bankers between 2016 and 2019, where they were getting Reid Hoffman to pledge a billion dollars to fund DeepMind to spin out and that kind of stuff. So this was all in the name of safety. He really did care about safety. I spent more than 30 hours talking to him. I stress tested his personality. You always have these fights when you're talking to A source that deeply. Yeah, he got me talk to his lawyer. At one point he got mad at me. But basically he was on the scale of these things I've dealt with over the course of six books. He was very reasonable. So he is a good, reasonable, sincere person who actually cares about safety. But just caring about safety doesn't mean you get safety. Today he's locked in this multiple labs in multiple countries racing to put out AGI. And if one lab says, well, I'm going to slow down and do safety, the other guys will just race ahead and beat you. So it's a collective action problem and he's a bit stuck.
- [27:05]
A
Right, so what do you think will happen? What do you do? And also, what would you do if you were him?
- [27:10]
B
So this is a super interesting question. So we saw recently with the Dario Pentagon thing that Anthropic said, you can't use our system for mass surveillance or totally autonomous lethal weapons. And the Pentagon just says, screw you, and they do a deal with OpenAI and they just roll him. Now, that played well for Dario and Anthropic in the public square. So that's fine. But it does show you that one lab cannot really push the government. I mean, it needs to be. The government has to want to do some kind of safety and impose the rules. And so I think Demis, one might say, has a moral dilemma in that Dario has shown the courage to do that, stand up to the Pentagon. Demis has never done something that confrontational. Should he now? I think he would say, no, it didn't produce results. What you should do is you should sort of build coalitions around the idea that government has to come in and lead this. And even if the Trump administration is never going to do that, the Trump guys are not there forever. And so maybe you talk to the British government, you try to get other European governments, you get the Japanese. I just spent eight days in China. There is an elite discussion about AI safety amongst the lab leaders and the research scientists in China. So I think you could kind of.
- [28:32]
A
Do you think the government. Do you think Xi, if you said to President Xi, okay, you could prioritize safety, but then you're going to lose
- [28:41]
B
the race to the US he's not going to lose it. You have to give him something.
- [28:44]
A
Right.
- [28:45]
B
Basically, we have US Export controls on semiconductors to China. I think if you did a deal where you took those away, but they sign up for a kind of a version of the Nuclear Non Proliferation Treaty.
- [28:57]
A
Yeah, but that would require us to. We're not starting in the right place. So one, one area where, and I don't know if you talked to him about this or if he's thought about this, but from a US Perspective, the other approach is to go to the states. Right. So California, New York, with the Raise Act. I mean, the individual states have tried to address some of these issues under the theory of not just Trump, but even Obama. Right. Like there's been no regulation of Internet 2.0. Forget about AI, right. We still haven't regulated social media in any way, shape or form, despite all of the mirror. I mean, social media might end up being more harmful than cigarettes by the end of the, by the time all said and done. And yet we've done nothing on that. Has he thought about or even discussed with him the notion of like, okay, Washington's a fucking mess. It's always going to be a mess. Maybe there will be a moment in 27 where for their own political security they'll have to do something. But overall the path in the US would be at the state based level. And Trump has tried to preempt with his executive order and others, but the reality is state. I used to be the deputy governor of Illinois. We didn't deal with national security. The only real preemption that I think the federal government could get away with would be things on national security grounds. But there's just not that much the states do in that. And so I think ultimately the 10th amendment protects that. Has that been a thought process at all that you're aware of?
- [30:27]
B
Obviously it's a thought process amongst those state leaders who are doing what you're describing. For Demis, he is British, he's rooted in Britain, he's quite patriotically British. His main contribution was at one point he suggested to Rishi Sunak, who was then the Prime Minister of Britain, what about a multinational AI safety conference where we at least get the conversation started and we invite the Chinese. And that happened end of 2023. Bletchley park, where they did the Second World War, you know, cryptographic work. Right. The Enigma project and all that. So there was this sort of tradition of computer science breakthroughs in Bletchley park. And they had this conference there. People came from all over. Kamala Harris came over. So that was a Demis idea kind of behind the scenes. And I think that's how he sees the future. Like trying to get countries to talk to each other because it has to be multinational. Because look, if the US thinks that China's going to race, fundamentally, it's not rational to hold the US Back because you don't make the world safer. And meantime you lose the race against China. So it's got to be US and China both involved. That feels crazy and impossible today, but we've got to remember that in 1962 you had the Cuban Missile Crisis, super extreme moment of nuclear tension. And then six years later, 1968, nuclear non proliferation Treaty. So in geopolitical standoffs, it goes in waves. There's aggression and then there's data.
- [32:03]
A
But do you need. I was talking to someone this about dinner last night where so I would argue perhaps the greatest accomplishment of humanity is that for the last 80 years we've had nuclear weapons and we haven't actually used them.
- [32:15]
B
Right.
- [32:16]
A
And perhaps the reason why in some ways is because it started with what Truman did. And the horrors of that were so apparent that that created a sense of like, yeah, we can't really. No matter how much people were sort of the brinkmanship, ultimately, no one quite yet, knock on wood, has got to that point. In order for the kind of AI safety that you're talking about to be politically feasible, is it going to require something absolutely catastrophic first that will then wake people up?
- [32:52]
B
Right, it might do, but it's the responsibility of anybody who is involved in this conversation to try to wake people up and say, hey, wouldn't it be nice if we had the political responsibility for the catastrophe?
- [33:04]
A
Right, right. But look at climate change. We're not. We're kind of doing that, but not really.
- [33:08]
B
I think I have one thing here which is, I think relevant to this, this, to kind of the flavor and the texture of this whole debate, which is that I thought going into this project three years ago, three and a half years ago, that I wanted to kind of capture the tingling sensation of these scientists who have their hand on the modern version of the nuclear material. You guys are creating something which is dangerous. It could really destroy humanity at the extreme. What's the feel like to. To deal with that every day? How do you sleep? And I thought, okay, I want to get to that. But it's going to be kind of sensitive to raise this with these guys. It's not sensitive. They raise it themselves. I'm talking to Demis one day and I say, what's it like? You open your office in 2010, the first DeepMind office. It's in Russell Square in London, this historic part of London. How did that feel? And I've written lots of books, so I know that usually you say, hey, recapture the sensation of 15 years ago, the subject of the interview is like, yeah, it was cool. And that's what you get. Right. But Demis is such a kind of vivid storyteller that he says, well, you come out of the office, right? I'm at the top, in the attic of this house on Russell Square. You come down the stairs, there's no elevator. Ding, ding, ding, ding, down the stairs, you're in front of Russell Square. And then on the right, there is the London Mathematical Society where Turing invented the origins of computer science. And now we're completing it. So that made sense. But then further on there is the pedestrian crossing where it goes black, white, black, white across the street. And that is where the nuclear scientist, the Hungarian Szilard, thought of the nuclear chain reaction which led to the Manhattan Project. And now we're doing the modern version of the Manhattan Project. So this is just, you know, isn't that a perfect echo, Sebastian?
- [34:51]
A
Whoa. Didn't expect that good of an answer. Right.
- [34:55]
B
And so, you know, the point here is that the nuclear parallel, the Oppenheimer syndrome, is infused in the conversation in these.
- [35:06]
A
Right. So then this kind of perfectly leads to the last topic I wanted to hit with you. And this is both, I think, true of this book, but it just, I think your work across the board informs it, which is goodness versus greatness. Right. So you cover a lot of people who are considered great by some standard, maybe not. And I think a lot of those people, the Musks, the Altmans, the Zuckerbergs, would say, don't be naive in order to be great. You can't expect me to live by sort of normal moral and ethical standards. You know, that's for everyone else, not for us. Is it possible to be both? And second, can you be truly great if you are not good?
- [35:48]
B
Wow. I think in. Okay, so this is very interesting debate. So I'd say that there's a greatness which is kind of political leader, maybe a military leader, that type of greatness, probably you can't be good. And this was the dominant definition of greatness until the 20th century. So whether it's Alexander the Great.
- [36:16]
A
Yeah.
- [36:17]
B
You know, or I don't know what other great. But I mean, there's all these people who were essentially guys who conquered lots of land and killed a lot of people. This was not being a friendly, next door kind of good person in the 20th century. I would say that kind of greatness went out of style. We became more democratic societies. We didn't tolerate tyrants who drove people to go to war. Hitler was the opposite of great. Et cetera. So we had a new definition of greatness, and that was more Einstein, that was more pure scientific invention. And we're happy to call those people great and those people can be good. And so the question is, Demis Hassabis is sort of a mixture.
- [37:06]
A
But even the Einsteins, if you were, and I don't want to demean by saying just. But if you are coming, if you're an academic, you're a scientist and you're coming up with ideas, theories, but ultimately to put them into practice and scale across the world, you're still relying on somebody else. Whereas a CEO of a founder of a massive company or a president, whoever it is, has a much greater impact. So are we almost letting the Einsteins off a little too easy? Because like. Yeah, but their responsibility only went so far.
- [37:44]
B
Fair. And obviously it depends on your definition of greatness. If you want to define greatness as sort of acting in the world and having large scale impact, you're right. Then if that's the definition of greatness, you can't mean Einstein isn't really great.
- [37:59]
A
I mean, obviously he is, but under this approach at least.
- [38:02]
B
Yeah, yeah, yeah. But I think, I think what's super, super interesting about Demis Sabes is that he is both a huge scientist. You know, as I said, he won a Nobel Prize. And by the way, he told me he'd quite like to win a second one. I mean, his ambition is, yes, big.
- [38:17]
A
Yeah.
- [38:20]
B
But at the same time, he's an actor in the world. He's leading this lab that's creating products that are going to completely transform how we live.
- [38:26]
A
Right. That's what he struck me. I kind of was always working on the assumption that it's very hard to be both at that level. Right. And that, you know, and if anything, and bear with me here, but it's sort of a nascent thesis that I'm working through in my head. But in the 1960s, there was a normative shift towards individualism, kind of led by the baby boomers that produced some good things. Civil rights, voting rights, and in some ways just sort of liberalized society, but also very much leaned away from the collective good and towards individual needs. That is in some ways selfishness. Right. And the logical manifestation of all that is Donald Trump, which is a totally zero sum worldview that clearly 51% of America seems to agree with. Because they elected him. We elected him. And it does seem like, putting Demis aside, Musk, Altman, Zuckerberg very much are in line with that zero sum mentality do you see it shifting back or do you think that? Because I just don't see how we ultimately, like, survive. I mean, this is very, very big picture. But, you know, 100, 200,000 years ago, you had multiple forms of human beings. And the reason why Homo sapiens beat the Neanderthals, even though they were not only, to my understanding, bigger and stronger, but even smarter, was they couldn't work together and Homo sapiens could. And in a zero sum world, you don't. Right. That's why Trump can't get any of the allies to support him in the strait of hormones.
- [40:17]
B
Right.
- [40:18]
A
So if that's the road we're going down, is it reversible? And if it's not, is it survivable?
- [40:27]
B
Yeah. These are sort of incredibly interesting things. I mean, let me just sort of pick on one thought, which is, I think if you look at the AI leaders, there are different styles of kind of trying to create coalitions who work for you in a team and bring stuff to fruition. And if you are, I would say, Elon Musk or Sam Altman, it's a lot about, I'm a megastar in terms of my public profile. I can make stuff happen. I can do deals all over the world. I'm a money machine. I raise money like nobody else. I can talk my stock price up. If it's Elon, like nobody else, come work for me because I am momentum incarnate. That's one kind of pitch. Problem is then if your momentum stops, as we were talking about with OpenAI, you could disappear down the hole very fast. Then there's another thought which is like Daria Amide, it's more, I'm the good guy. I really care about safety. I'm going to invest my lab's resources in safety. I'm going to stand up to the Pentagon. I really care about this stuff. And then you recruit people based on a belief that this is the White Hat mission. I think Demis is quite close to Dario in that he's trying to have a coalesced team by kind of projecting a set of values that people buy into, a sort of scientific curiosity. I mean, it's very important to Demis you do AI because you're fundamentally trying to understand how nature works. And there's almost a spiritual dimension for him. As you understand nature better, you get closer to God. I was very surprised when he talked to me about this, but this is how he thinks about it. So there is a sort of spiritual thing. Anyway, the package for Demis is that he is getting people to work for him. I mean, one of the miracles of his career is that he founds this company in 2010. AI isn't going to do anything, but he hires super smart PhDs to come and work for him. They give up their postdocs and their academic positions to come and be with him. Why? Because he's sort of inspiring. He really is inspiring.
- [42:40]
A
Right.
- [42:40]
B
And so I think, which by the
- [42:42]
A
way, I would argue is a prerequisite of greatness. Right. You know, can you get people to follow you because they truly believe in it, not because they're self interested and think that it just will benefit them?
- [42:53]
B
Yeah, yeah. But in a larger way. You know, this struggle about can you be good and be great? I mean this. I raised this issue in my introduction. This is the big question about Demis. And you know, his mum was Chinese, Singaporean, you know, raised in Singapore, was an orphan on the streets for a bit before a relative rescued her and, you know, was quite religious because of all that and brought up her son to pray in the evenings. And to believe that you must never control other people, that it's very important to respect them, to be good to them. In the early days of DeepMind and a test they made for new hires was, was the person coming in to be interviewed, was that person polite and friendly to the receptionist? If they weren't, forget it, they're not going to be hired. So they really filtered out the assholes. All of this stuff is extremely important to Denis. He would tell me repeatedly, I hate to control people. I don't like controlling people. I don't like controlling. Dude, you run a big company, you do control people. What are you talking about? And he says, I don't like it. I do it because I have to, but I don't. He struggles actively. So in a way, Demis is a case study in your question.
- [44:03]
A
Yes. All right, well, Sebastian, thank you so much. This was fantastic. Again, the books, the Infinity Machine, demis Hassabis, the DeepMind and the quest for Superintelligence. Highly recommend it and thank you again for joining us.
- [44:14]
B
So fun to talk to you. Thanks a lot.
- [44:20]
A
Firewall is recorded at my bookstore, PNT netware, located at 180 Orchard street on the lower east side of Manhattan. We'd love to hear from you with questions, feedbacks or idea for a guest. Just email me at Bradley Firewall Media or find me on LinkedIn. And to keep up with what's on my mind and my latest writing, please follow my new substack@bradleytus.substack.com thanks again for listening.