← The Times Tech Podcast
The Times Tech Podcast
Anthropic vs Pentagon: How AI is changing war
00:39:36
Who gets to decide how powerful AI systems are used in war?
Loading summary
Transcript135 lines
- [00:00]
A
This episode of the Times Tech Podcast is sponsored by Adobe Acrobat Studio.
- [00:05]
B
A lot of us lead very busy lives, including the tech bosses you hear from. On this podcast. We're dealing with more information than ever before, often across disconnected tools and working in hybrid teams.
- [00:16]
A
If this sounds familiar and you're someone whose life involves scrolling through contracts, reports or pitches with important data buried in long documents or different sources, then you'll know it can sometimes be a struggle
- [00:27]
B
to keep up, or at least to do so in a timely way.
- [00:30]
A
Adobe, the company that invented the PDF back in 1993, is changing that with Adobe Acrobat Studio. It's a modern workspace designed to help you work smarter and save time.
- [00:40]
B
Acrobat Studio isn't just for essential PDF tasks like editing or signing documents. It's an AI powered space built to help you use simple language prompts to get information quickly from multiple documents and sources, including PDFs, Office 365 files and web pages.
- [00:56]
A
It's for people with busy, demanding jobs who want to spend less time on
- [01:00]
B
admin and more time on the work that really matters.
- [01:03]
A
Check out Adobe Acrobat Studio and its new features now.
- [01:06]
B
This episode of the Times Tech Podcast is sponsored by PwC.
- [01:11]
A
How do you use AI in ways that drive real business value? It isn't just about the tech, it's a new way of thinking, going further, faster. This is where PwC comes in. Shaping AI to fit your business to help you innovate with impact, empower your people and accelerate growth. Working with you side by side, bringing deep industry know how with expert change management to unlock real value and drive lasting impact.
- [01:37]
B
PwC knows how your business works so you can put AI to work for your business. Discover more at pwc.co.uk.
- [01:52]
A
Hello and welcome to the Times Tech Podcast where every week we unpack how technology is reshaping business culture and everyday life. I am Danny Fortson out here in Silicon Valley.
- [02:03]
B
And I'm Katie Prescott. Katie in the city, the city of London. And this week we're talking about how artificial intelligence is being used to wage war.
- [02:13]
A
Yeah, nice and light. Fair this week, indeed. Last week, US President Donald Trump launched a major air attack on Iran. And according to some reports, AI tools built by Anthropic, the 380 billion dollar company run by Dario Amade, friend of the pod, did play a role in those attacks. But just hours before the strikes, he had declared, that's Donald Trump. That the federal government would end its use of those very same tools. And he posted in his Truth Social account, quote, we don't need it, we don't want it and we will not do business with them. Again, exclamation point, just to underline the point. Yes, exactly. So this feud between his administration and Anthropic has been months in the making and centers on how its AI models can or how they do not want them to be used by the Pentagon.
- [03:10]
B
Yeah, he's not backwards in coming forwards, is he, your President? Some of those messages on Truth Social were quite extraordinary. He described those at Anthropic as left wing nut jobs. So this is not just some sort of boring contractual to and fro between AI companies and the government. I mean it really speaks to something bigger. Who gets to decide how AI incredibly powerful AI systems are used in war. And on one side you've got this private business which is develop one of the leading developers of AI and on the other side the Pentagon. And it sparked this, a huge debate really about what ethical means when it comes to national security.
- [03:56]
A
That's right. And in a moment we're going to bring on a great guest to talk about this. His name is Sean Gourley. He's the founder and former CEO of Primer, which is a San Francisco based AI defense company. He was also a former advisor to the Pentagon and current advisor to AI defense startups. So he'll be helping us unpack what it all means. So our big question that we're going to explore with him and right now is how is AI remaking war?
- [04:27]
B
Well, let's get into that big story. I think we all knew that AI was remaking war, as you say. But it's only when something like this happens that it really sharpens the mind and you can see it. So just going back to that row before the strikes on Tehran this weekend, the row between Anthropic and the US Department of Defense. They have been negotiating over exactly how the Pentagon could use the firm's technology. And Anthropic was the first AI company to have its tech used by the Department of Defense. Its model CLAUDE is used for intelligence analysis, cyber operations, operational planning and things like simulation and modeling. So Anthropic said you can use all of our AI, that's fine, but there are two key areas where we don't want you to use it, and that is domestic surveillance in the US and also operating autonomous weapons without any human control at all.
- [05:21]
A
That's right. So those are the two red lines. And Pentagon officials sent Anthropic their kind of final agree or else last week they had till 5pm last Friday to accept this final offer. Anthropic refused. They're saying we could not in God conscience grant the military unrestricted access to its technology. It all went pear shaped. Trump terminated this $200 million deal that anthropic had with the Pentagon, ordered all federal agencies to stop working with the company. And amazingly, the Defense Department or Department of War designated Anthropic, an American company based in California, a security threat and a risk to its supply chain.
- [06:07]
B
Which is something that happens to Chinese businesses like Huawei, but not, as you say, an all American Silicon Valley AI champion like Anthropic. Anthropic has responded to all of this. Dariama Day did a very lengthy interview on CBS News last weekend, but in a statement they said that being designated a supply chain risk would both be legally unsound and set a dangerous precedent for any American company that negotiates with the government. And they have vowed to sue the government over this. But then things got really interesting and there was another twist in the tale because just after Anthropic's deadline had passed, its great rival OpenAI, led by Sam Altman, friend of the pod, swooped in and said that they'd struck their own deal with the Department of Defense. And Sam Altman put out a blog saying its red lines were even stronger than Anthropic's, which was very confusing trying to follow it and trying to work out what the red lines actually were when it came to the use of this technology or not. It's probably worth going back to look at how this row really started, because it was actually back in January when Claude was used by the US military in its raid to capture the president of Venezuela, Nicolas Maduro. Back in January, Yeah.
- [07:25]
A
So Claude was used apparently in that operation in conjunction with Palantir, which is another Silicon Valley company, or formerly Silicon Valley. But at the time of the kind of this Maduro abduction, Anthropic didn't comment, but said any use of its AI tools was required to comply with its usage policies. And at the same time, Pete Hegseth, the Secretary of War, called on all AI companies to hand over their technology to the military for use without restrictions. And he was even like, there's stories of around the Pentagon, he's put up these old school recruiting posters, but they're generated by AI and it's like, I want you to use AI. This meant that the AI companies working with the Pentagon had to renegotiate their contracts because there was this kind of new three line whip from on High that we needed all the AI, we needed unfettered access. And that led to some renegotiations, which really kicked off this row with Anthropic.
- [08:30]
B
I think it's fair to say a lot of this story really hinges though on the personalities of the characters involved, perhaps even more than those red lines and what the tech can and can't be useful, which is still, I think, incredibly confusing. So it's probably worth picking apart those three big personalities. Daario Amadei, the boss of Anthropic. Sam Altman, the boss of OpenAI. And another figure, Emil Michael, the Undersecretary of War, who's basically the CTO of the Pentagon. So, Danny, why don't we start with Emil. Just tell me sort of who is he and what was his role in all of this stuff?
- [09:11]
A
Yeah, so I think what's so interesting about all this, because we're talking about AI and these powerful systems and how they're going to be used and all this stuff, but ultimately this comes down to humans and humans getting in a room deciding yes or no, what they agree, what they don't agree on and do people like each other. Right. So Emil Michael is kind of at the center of this. He was the former chief business officer at Uber back in like Uber's crazy hyper growth days where they were like warring with regulators all over the world. They were expanding into China and Russia, like all over Europe, London. And he's seen as like through a couple different lenses. One, he's like this world class deal maker. So like he helped Uber raise something like $15 billion back when $15 billion was a lot of money, you know, pre AI. Pre AI.
- [10:05]
B
Pre AI. Hundred billion.
- [10:07]
A
Yeah. And so Uber was like the most lavishly funded startup around. It was doing all this stuff and Emil Michael was like the guy really driving that right alongside Travis Kalanick, the CEO.
- [10:18]
B
So this is a businessman, essentially.
- [10:21]
A
Businessman, but also a tech guy. He was based in Silicon Valley. Right. And he. So he knows also Altman and Amadei through just being a Silicon Valley tech guy.
- [10:34]
B
Yeah.
- [10:35]
A
And so the other thing that he's really known for during his time at Uber is he was kind of seen as like the exemplar of Uber's really aggressive, what was later dubbed, quote, unquote, toxic culture. So back in 2014, he spoke openly about the idea of spending like a million dollars, for example, to dig up dirt on journalists and critics, because Uber had a lot of critics at the time because of the way they were operating. It should be noted that that Michael later apologized for his comments. And then also during a business trip out to Seoul, Korea, this is like this infamous trip. He and a bunch of executives went to a hostess bar. There was an HR complaint filed. He asked a female employee to help him cover it up. It became this huge controversy at this very, like, I don't know if you remember, you know, Uber at the time was like in the news every day over its culture.
- [11:27]
B
What's a hostess bar?
- [11:29]
A
I believe,
- [11:31]
B
asking a man, yes, you.
- [11:34]
A
I know I have never been to Korea, but the implication was like, there are women there and you can kind of go off with a woman of your choosing.
- [11:45]
B
All right.
- [11:46]
A
To play Parcheesi, presumably. I see, you know. Yeah, you know, so anyway, it became this big controversy. He left amid, like this really crazy time at Uber. And he is the person who was hired last year, brought in to be kind of the right hand man to Pete Hegseth. And his goal was, or his kind of mandate was to basically inject AI into the veins of the Pentagon, to kind of bring it up to speed for the modern world in the way
- [12:16]
B
that Donald Trump does. He turned to the business world to find someone who could work within the Department of Defense.
- [12:25]
A
Yes, exactly.
- [12:25]
B
To push the AI strategy. So he's there in the Pentagon, and I guess then part of his job has been negotiating with the frontier AI labs like Anthropic.
- [12:36]
A
That's right. And so here he is speaking to Bloomberg last week before breaking off the talks with Anthropic. For any AI system we might use. Are we using it to protect our war fighters in the right way? Are we using it to sort of give them the best tools to be efficient and to be lethal when they have to be lethal? And that's the primary thing in Secretary Heg says, hey, head. And we told them that ultimately, at the end of the day, we follow the law, all laws, but we can't let any one company stand between us and the warfighter because they don't make the rules. Congress makes the rules, the president sign them, we execute them, and we do so safely.
- [13:18]
B
So I guess what he's saying is we don't like being dictated to.
- [13:23]
A
We don't like to be told what to do. You're a vendor. Then we do what we want with it. Yes. So he's one character and then who are the other two guys?
- [13:31]
B
Sam Altman, CEO of OpenAI, who probably needs no introduction in all of this, but I would say he has misjudged this situation in many ways. So when this was all blowing up initially. And this is obviously pre the Tehran strikes. Altman and others in Silicon Valley actually seemed very supportive of Anthropic stance against the White House. And it was seen as an ethical stance. And I think there was a sense, not necessarily about the rights and wrongs of what they were saying, but really that a tech company shouldn't be bossed around by the President in this way. And so his decision then to come and accept the contract with the Pentagon within hours. Within hours caused a huge amount of blowback. And we saw Claude rising through the ranks on the App Store and people
- [14:28]
A
deinstalling or uninstalling chatgpt and, and he did say, he's like, look, I don't think Anthropic should be classified as a supply chain risk. He's like, I don't think that's a good thing. By the way, I am taking this contract. And so a lot of people just saw it as just kind of ick,
- [14:48]
B
profiteering, you know, not really standing up for what is right and didn't smell great. And it's probably worth explaining, isn't it, the relationship between Sam Altman and Dario Amedei, the guy that he kind of screwed over here by taking that contract because they used to work together. And then Dariamaday spun out Anthropic from OpenAI, didn't he?
- [15:11]
A
Yeah, he and his sister and five others left OpenAI to start Anthropic. And as we mentioned last week on the pod, he set it up as we're going to do it differently, I. E. We're going to do it better. We are the AI safety lab. They saw things within OpenAI they didn't seem to like that didn't jibe with their moral ethics, whatever. And so they left, set up their own stall and said, we're going to do this differently. We are the AI safety lab. And back in December 2024, I actually met the entire top team of Anthropic and sat down with Daria Amadei at their, at the headquarters here in San Francisco. And this was on the back of a first big essay he wrote about all the wonderful things that AI could bring to the world if this quote, unquote, all goes right. I'm someone who actually takes the, you know, the potential for, for, for AI and where it could go very seriously. It's possible that I'm wrong about, you know, where AI is going to go and what's going to happen. But, you know, I think, I'm not sure that, you know, the Impacts are actually going to be very serious. And so let's think about how this can go. Well, what could AI actually do for humanity that, you know, looking back a decade or two from now, people could say, wow, that really actually made our lives better. That's him, you know, the glass half full. But obviously even then, what he's saying there I think really kind of hints at, you know, he's also thinking deeply about how this might not make our lives better or there's like very much a downside to these systems.
- [16:48]
B
Yeah, you can really hear the seeds of the argument with the Pentagon and what he just said, can't you? The real focus on safety, the fear as well, about where the technology could go. And you can feel him grappling with that. So I don't think it's surprising at all, actually, when you think about the discussion that, well, discussion is the wrong word. The argument, the fight.
- [17:10]
A
And that was December 2024. Now we're in, what, March 2026. Yeah, just, you know, life comes at you fast. And as we mentioned last week, just to muddle the picture further, they actually just recently got rid of one of their core safety pledges, which was, we will not put out AI systems unless we are certain that our guardrails are sufficient to kind of keep bad things from happening. And what's really interesting is that you mentioned Sam Altman. He kind of realized what was happening. As you start to see, Anthropic goes to the top of the charts in the App Store, people are uninstalling ChatGPT and he's like, oops. And he did put out a mea culpa online that. And I think he said, quote, one thing I did, I think I did wrong. We shouldn't have rushed this thing out on Friday. The issues are super complex and demand clear communication. We were genuinely trying to de escalate things, really. Okay, that's just my editorialism.
- [18:16]
B
Don't get involved if you're trying to, when things are literally blowing up.
- [18:20]
A
Yes. And avoid a much worse outcome. But I think it just looked opportunistic and sloppy. Good learning experience for me as we face higher stakes decisions in the future. So clearly he saw this blowback and was like, oh, my goodness. But basically, where we sit now, Anthropic is on the outs. The Treasury Secretary said, same thing. We're not using Anthropic anymore. Get rid of it. OpenAI has stepped into the breach and now they're after the fact, trying to be like, actually those red lines that Dario was Really kind of dying on that hill. We're trying to impose those kind of after the deal has been agreed. So that looks like they're trying to do that to kind of inject some of those same controls. But again, it gets back to, like, didn't sound like Emile. Michael really liked Dario. He called him a liar with a God complex.
- [19:14]
B
Yeah. And Trump calling him left wing nut job.
- [19:17]
A
Left wing. So he's a liar with a God complex. He's a left wing nutjob. Like all of this stuff. So there's just this real kind of personal animus that has been dropped into the middle of all this. What is a very big, big question of, like, how are we going to use these things because they're really powerful. And now we're in something that looks a lot like a hot war in the Middle East.
- [19:38]
B
Yeah. Well, two things. Firstly, it's another example of tech just being split down political lines. Right. So people who fall into the left wing nutjob camp are siding with Anthropic and downloading it and shunning OpenAI. And it's very much made OpenAI look like Trump's partner in all of this.
- [19:58]
A
Yes.
- [19:59]
B
But for me, the big question is, what do we do globally about this issue of AI safety? Because I don't think personally that these sorts of decisions should be made by Dario Amadei or Sam Altman or even your friend Emil Michael.
- [20:15]
A
Not my friend. Well, sorry, never been to a hostess bar. Thank you very much.
- [20:21]
B
You've never been to a hostess bar? Not with him joking, obviously. But we have still failed globally to come up with any sort of body who can look at what's going on with AI and actually try and make these decisions across the board. And I think that's really important. And we had the lead writer from the AI Safety Institute on not that long ago, and they write an annual report. And it's, you know, as I think you said at the time, it struggles to catch up with the speed of change with this technology. The UN last month has launched its own AI safety body to look into the technology, but there are still no guardrails around it. And I don't think it's very clear that any have really, frankly come out of this discussion.
- [21:04]
A
I think we have the perfect person to help guide us through that and give some kind of, you know, I think we know some stuff, but let's say we're not like experts in the weeds of the AI and defense world. So on the program now, Sean Goerli, he's The founder and former CEO of Primer, which is a San Francisco based AI defense company. He's advised senior officials at the Pentagon, now works as a kind of an advisor to AI defense startups, and he's going to help walk us through some of this.
- [21:39]
B
Hi Sean, welcome to the podcast. Thanks very much for joining us.
- [21:42]
C
Great to be here. Thanks for having me.
- [21:43]
B
Can you tell us in your own words what it is you do?
- [21:47]
C
Yeah, look, I've done a number of things from sort of studying the mathematical patterns of insurgency in places like Iraq and Afghanistan. As part of my PhD work, I've built and sold to AI technology companies. And the most recent one was a company called Prima. And we built and deployed language models for the Department of Defense and a large number of the US intelligence agencies running on classified documents in order to help them approve their processes.
- [22:14]
B
What's it like dealing with the Department of Defense as an AI business?
- [22:18]
C
It's changed a lot. Like so, so when we started selling into first the intelligence organizations and then the Department of, then the Department of Defense, you know, I think we were, we were doing the first kinds of AI deployments. This is things like Project Maven coming online. And, you know, it was like any organization perhaps even more advanced than Most organizations, circa 2017, 2018, it was the first AI deployments into a large bureaucracy. And so that had all the kind of the challenges that you would get with working with a large bureaucracy, but it had also the other kind of layer which was working on top of classified documents. Right. And so classified documents, you have to have all sorts of safeguards, you have to have all sorts of checks and balances, and you have to run that in a way that's disconnected from any kind of outside interference. And so there's an added layer of complexity on top of, you know, navigating a large bureaucracy. And then you've got the whole dynamic which has changed again, but was still very much prevalent, which is you've got to have the social license to operate. And that's not just inside of, inside of Silicon Valley. It's also the engineers inside of your organization have to be on board with that. And I think as we look at some of the dynamics with anthropic particularly, I think you're seeing them struggle with all of those components. As we were doing, you know, eight years ago when we started this journey
- [23:35]
A
as well, you have a great kind of vantage point for this discussion because we were just, before you came on, we were talking about the whole blow up between anthropic and the Pentagon and OpenAI coming in. But if we get to start at like 30,000ft, what we're trying to kind of unpick, which is a little difficult because the nature of a lot of these technologies, it's kind of classified how they're used. But where are we or how should we think about the importance of AI in defense in war? Like the role it can play currently or does play and where things are going.
- [24:09]
C
Look, I think you've got world leaders saying those who control artificial intelligence will control the geopolitical sphere through military dominance. That's one way of looking at it. I think you've had Putin say similar things to that nature. I think here, the fact when Anthropic says perhaps we don't want you to have unfettered access to all of our artificial intelligence, immediately the Department of War cracks down and says, you're going to have major issues if you take that position. I think that tells you how important artificial intelligence is. Right. The scale of the reaction to any kind of limitations being placed on it, I think suggests very clearly that this is a primary technology to enable military dominance.
- [24:52]
A
The background or the backdrop to all of this is China. What is your sense of, can we give like a comparison, you know, right and left of like where we are kind of where the technology is here relative to China? Because if you talk to certain people in Silicon Valley, particularly like we had last year, we had Vinod Khosla and he was slamming meta who were trying to open source things. He's like, this is like open sourcing the Manhattan Project. This is insane. And this is an existential race against China. But I'd love to get your sense of what's happening there relative to what's happening in the West.
- [25:29]
C
Yeah, look, I think that the dynamic of this is, again, my sort of mental model going through is this is artificial intelligence represents the third technological offset in military parlance. Right. So the first one was nuclear weapons, the second one was precision munitions and stealth weaponry. And the third is artificial intelligence. And an offset for the, you know, sort of the. Those that maybe don't track. Military terminology is a technological advantage that is so great that it means that if the opponent doesn't have it, it's not worth even showing up for battle because you'll be defeated before you start. And I think that's the right frame. Right. We've had two of them. This artificial intelligence is the third. What does that mean? Right. It means if you take some of the scenarios, you have a human piloted F35 against a machine piloted autonomous drone. Right. The autonomous drone with similar kinds of mechanical capabilities will beat the human pilot again and again and again.
- [26:24]
A
And those are war games that have already happen. Happened. Right.
- [26:26]
C
So these are tests and simulations that happened, are now running live inside of specially modified aircraft. And the dynamic on this is just humans are slow. We have risk tolerances that machines don't. Machines are much faster than us. And machines are also able to play and practice these games many times over. So in certain scenarios like that, which is now several years old, in terms of research, which is ancient, in terms of artificial intelligence, machines are clearly better than us. Right. And I think the trajectory that we're all seeing in this is that they're getting better and better at a very much faster rate. Now, that's not to say that they're better than us at everything today, but I think the writing is very much on the wall, is that if you own the best artificial intelligence and you can deploy that inside of your military with autonomous weapons, you will defeat a human controlled conventional weapons system. And I think for those, you know, tracking this stuff, it's like, it seems pretty clear that that's where things are going. And so as you look at this, you say, well, should we open source this technology? And it brings with it all sorts of kind of questions around, well, are you giving up your military supremacy for the sake of your commercial benefit or are you not willing to make that trade? And I would probably argue that the primacy of military supremacy is something that should be taken very seriously as, as a first, as a first citizen in that debate.
- [27:52]
A
Let's pause there for a moment to talk about today's sponsor, Adobe Acrobat Studio.
- [27:57]
B
Acrobat Studio helps documents work harder, organizing projects, extracting insights quickly, and creating professional content from the files you already have.
- [28:07]
A
It's designed for individuals, teams and businesses who need AI powered tools that are easy to deploy and use.
- [28:13]
B
And Adobe says your data stays secur and isn't used to train its AI models.
- [28:18]
A
So if you want to rethink what PDFs can do, check out Adobe Acrobat Studio and its new features Now.
- [28:25]
B
This episode of the Times Tech Podcast is sponsored by PwC.
- [28:30]
A
Every leader is looking for that next opportunity for growth. But as technology and AI reshapes entire industries, the path forward isn't always clear. This is where PwC comes in. Helping you put AI to use in ways that drive real business value. Working alongside you, bringing the deep tech expertise, AI capabilities, strategic insight and industry knowledge you need to innovate with impact. Empower your people and accelerate growth.
- [28:57]
B
PwC knows how your business works, so you can put AI to work for your business. Discover more@pwc.co.uk
- [29:10]
C
Howdy, howdy ho, and welcome to Fantasy Fanfellas.
- [29:13]
A
I'm Hayden, producer of the Fantasy Fangirls podcast and your resident lover of all things Sanderson. And I'm Stephen, your bookish Internet goofball, but you can call me the Smash Daddy. And we are currently deep diving Brandon Sanderson's fantasy epic Mistborn. But here's the catch. Steven here has not read Mistborn before. That's right.
- [29:31]
B
Hey.
- [29:31]
A
Hey. So each week you'll get my unfiltered raw reactions to every single chapter. And along the way, we'll do character deep dives, magic explainers, and Steven will even try to guess what's next.
- [29:41]
B
Spoiler alert.
- [29:42]
C
He'll be wrong.
- [29:43]
A
News flash. I'm never wrong. Episodes come out every Wednesday, and you can find Fantasy fan fellows wherever you get your podcasts.
- [29:57]
B
So how do you think AI then should be considered within this idea of an offset? I mean, it sounds like you're saying it's essentially the same as nuclear power. What does that mean for governments, but also what does it mean for vendors like Anthropic who are selling the technology?
- [30:15]
C
That's a big question to sort of unfold. What does it mean for a company? So, firstly, you've got to understand that the corporations are building very, very powerful technology, not necessarily for military applications, but it has very, very strong military applications, or it could be repurposed for military applications. And that is one of the most significant military technologies that we've seen, certainly in the last 30 or 40 years. So what does it mean for those companies? Firstly, you can't, you can't ignore. It's like if you were building a nuclear weapon, you can't be like, well, you know, I'm just going to ignore the fact that this could be used for military and just, I just pretend that that doesn't exist. Right? Like somebody's going to come knocking on your door at some point and say, the technology you've built is so powerful that you have to consider the military applications. And so I think getting into this sort of game. And look, Anthropic took steps. They deployed the technology on classified networks as we did. Once you're in that, you're in the game. And I think to pull out halfway through and say, oh, I don't quite like this game. I think at best is naive, but at worst is poorly kind of thought out. So I think that realizing ramifications of getting in this game is that if you get into the game and sell technology to the Pentagon, you play by the Pentagon's rules. And one of those rules is if you are so fortunate as a company to build a technology that has immense military advantages, you don't just get to hit stop and say, I'm out. I think that's sort of the dynamic here that I think America is wrestling with, which is what happens when a private company builds a technology so powerful it changes the military balance of the globe. Right. And that's maybe just a little bit beyond kind of where we are today. But if you project that forward 12 months or 24 months, like, you know, these companies or private companies are very much in a place where that is the reality that they're sitting with. And I think at that point, you need to figure out how to work with the Pentagon or stop building the technology.
- [32:19]
B
I was interested that earlier you used the phrase social license. Do you mean by that? Basically getting staff buy in for the fact that what you're selling is going to be used in war. And do you get a sense that maybe that's what Anthropic hasn't got with it, with its team, and that's maybe why some of this has happened?
- [32:39]
C
Yeah, we saw. We saw this early on. There was. There was a company involved in the Maven project that was a computer vision
- [32:43]
A
company out of New York, Maven, just for people who don't know, there was this drone project.
- [32:47]
C
It was the first AI project the Pentagon took on and represented. And I think history will record that as being a very significant contract and vehicle to bring AI into the Pentagon.
- [32:59]
A
And Google eventually threw in the towel because people revolted. People walked out, signed letters, and these are the employees saying, we don't want to be in the business of war.
- [33:10]
B
So this is the social license you were talking about?
- [33:12]
C
Yeah, 100%. So Google protests. There are people marching out of the company and saying, if you do this contract, we will not work here anymore. And so there are a number of companies that were actually involved in that Maven project that lost their social license to operate with their employees, where their employees said, oh, I never signed up to be part of a defense company. And I think this was sort of part of the Silicon Valley sort of belief that defense wasn't needed, it wasn't necessary. We live in a world of peace, and can't we all just get along with each other? And a very naive view that sort of permeated the valley, which is changing, but it's still very much like, oh, we just don't need defense. And if I put my head in the sand and believe that it's not there, then it just goes away. So I think that there's a big kind of change of bringing people from that position to. No, defense is an imperative. If you want the society that we're living in, you're going to need to have the ways and means not just of defending it, but also projecting that force to further the interests of the nation. So that trajectory has to happen and all the employees have to be brought along with it. In such a competitive kind of war for talent between some of the teams that are building these foundation models, losing 10, 20% of your top researchers could put you back six months compared to your competitors, which is equivalent to hundreds of billions of dollars of market capitalization being lost. And so I think one of the dynamics here is, you know, if your employees aren't with you, it's very, very hard to go into the defense space. And particularly if you've started your company on the premise that we will build artificial intelligence that will do no harm to humanity, it's a hard line to come in and say, oh, by the way, we're going to build killer robots. Right? Which is, which is there. And I think, look, I think the leadership of Anthropic has walked a pretty good line up to this point, and they've certainly been involved in doing some very, very interesting work with the national security organizations. But I do sense that they lost that buy in from their employees just, just recently. I think the Maduro stuff down in Venezuela was probably a tipping point for that. And I think that the big kind of takeaway on this is if you're going to go in and deploy technology to the Pentagon, which I think is a good and a strong mission to have, then you need to also defer and say, I trust that the institutions of this country are going to use it in a way that is consistent with the law. And I also defer the definition of what the law is to the democratic structures and processes of the country. So the moment you go in and say, look, I want to write the law and that law is what I believe it should be, then you know, you're saying, I'm protecting democracy, but you're undermining it at the same time.
- [35:51]
B
We were talking about this a little bit before you came on, and it feels like trust in the institutions has been eroded by having a very erratic president in power. And I wonder if the answer to all of this is to have something like we do for nuclear, which is some sort of global standards around AI which various institutions have tried and failed to put in place.
- [36:14]
C
Yeah, I had this conversation at a meeting with the Pentagon back in 2015 as the office of Net Assessment. They brought together a bunch of thinkers back in 2015, put us up in West Point for a week and said like let's, let's war game out how all of this stuff is going to unfold. And I had this conversation with one of the other attendees who was there and I won't say who it was, but they were saying look, can't we just regulate this? And I think my response then as it is now is regulation of this is known impossible. Like it's incredibly difficult to regulate a set of weights that can be downloaded onto a thumb drive. How do you regulate the use of that? So I think regulation is very, very difficult. And regulation across an international set of organizations that can't even kind of get around the table at the UN Security Council and agree on, on what to do about anything is, is sort of gone. I, I think the reality of the world that we're moving into today or, and I think the last few days with Iran has shown is that the rule based order that we've been living in, international rule based order is, is very quickly being replaced by might is right. And you know, any kind of international kind of coordination across that is being trumped by you know, those that have the, the ways, the means to get their way.
- [37:30]
A
If we put this in the context again of, of like the third offset and you know, comparing it to nukes in that context, the answer would seem to be have the biggest nukes or have the biggest arsenal or have the most powerful weapons if you want to quote unquote, win.
- [37:51]
C
Yeah, to bring it, to bring it back to the military side is like don't lose the AI arms race, right. The speed at which AI evolves. Right. So that's first thing is the speed that evolves. If you get a six month lead on your opponent, you may have twice the kind of capabilities that they have, which may mean you have half or a quarter of the kind of the, the, the, the, the attrition or any kind of conflict. So if you can get a six month lead on your opponent, but there is very much this kind of AI arms race that's unfolding where if you fall behind, you expose yourself to your opponent coming after you and exercising that advantage to tremendous effect.
- [38:28]
A
And this is Very real. Because I was just looking in the past, I think it was in the past week. Anduril, you know, Palmer Lucker's company, they just put out a video. I think it's the world's first fully autonomous fighter jet.
- [38:40]
C
Yep.
- [38:41]
A
And they just have this video of like, okay, you don't need a pilot anymore. And to your point, like this is going to be better than Tom Cruise and Top Gun because it's a machine and it doesn't get tired or lazy or whatever.
- [38:54]
C
And it'll take risks that humans won't. It's got reaction times that humans, you know, will never biologically get to. And frankly, our appetite for losing one of those systems is a lot, is a lot higher than for losing a piloted system inside of that. If whoever, you know, if China were to wake up tomorrow and have, you know, a fleet of a thousand of these autonomous systems ready and available, they would probably be able to knock out the entire air kind of force projection of a US carrier group. Right. And, and this is the major projection of U.S. power in the world. If China were to kind of have a thousand, or Maybe, maybe it's 5,000 of these systems up running and built tomorrow, what would stop them from going and taking that and saying, look, we've taken Taiwan, come and stop us. And you'd run the calculus on that and you're like, yeah, we're going to lose that every time. So you see how close we are. It's, I love, love that the US has got their first with, with, with Anduril and getting that stuff up. We've got to get a lot more of them in action. We've got to get, you know, that stuff up and running. But if China were to get a six month lead on that, you can bet that they would take advantage of that and use that to say we are now the number one military in the world. Because I think whoever owns that is going to own the skies and is going to own the spaces they choose to project that power.
- [40:13]
A
You're freaking me out, Sean, but I know we're trying to just lighten the mood. Yes, Katie?
- [40:17]
B
I mean, nothing is going to look as good as Tom Cruise in Top Gun. Guys. Come on.
- [40:23]
C
But the reality is an automated system would have got that done with far less drama. It would have been a much worse movie.
- [40:29]
B
Didn't we love the drama?
- [40:31]
A
Yeah, but like, would you want to watch an AI play volleyball greased up in jeans?
- [40:38]
B
You know, it's just not the same.
- [40:39]
C
That's, that's the next Chinese Humanoid robot.
- [40:42]
B
Oh, God.
- [40:43]
C
Horrible humanoid robots playing volleyball and smacking each other on the ass.
- [40:48]
B
Sean, thank you. Thanks so much for coming on and taking the time to speak to us about all of this.
- [40:53]
C
My pleasure. Thank you.
- [40:57]
B
I have to say it was fascinating to hear from somebody within the defense industry with that perspective because I sense that I'm probably more on the left wing, nut job side of pacifism. So I'm sure Trump would put me in that box. So it was interesting to hear his view. I think that point about a social license and bringing your teams with you and people not having signed up to Anthropic to make weapons is really crucial. And it's fascinating to hear Dario Amadeus grapple with this stuff. What on earth do you do with this power? Because with great power comes great responsibility.
- [41:35]
A
If you're working at one of these foundation model companies, I feel like you have to sit down and have a very serious conversation with yourself before you start. Because to his point, these things are very powerful. And as they've said, if you listen to Sam Altman, if you listen to Dari Amade, they know how powerful these things are. They know what they can be used for. They know it can be very dangerous. And of course, like in the world of like the great power politics and all of that stuff, of course these things are going to be used in ways that are deeply uncomfortable. They're going to be used to kill people, like to put a fine point on it. And you have to be like, okay, am I okay with this?
- [42:20]
B
And if you don't like the person who's in charge of the country, who is making decisions on this.
- [42:28]
A
Yes.
- [42:29]
B
Then it becomes problematic.
- [42:31]
A
Yeah. Mix in a bit of the culture wars. The left versus right, the west coast progressives versus the Pentagon is tricky. It's tricky, but that is it. I think. For this week's episode of the Times Tech podcast, if you're enjoying the show, drop us a line to let us know.
- [42:48]
B
And we'd love to know your thoughts about the AI and defense. And who decides the guardrails? Is it the military? The government? Silicon Valley bosses? Let us know your thoughts and any questions that come up as well.
- [43:02]
A
Yes, and you can email us@techpodtimes.co.uk that is TechPod, the times.co.uk. we will see you back here next week in Katie. I hope. Hope your case of the sniffles has. Has moved on.
- [43:17]
B
Yeah, the never ending virus. Thank you.
- [43:19]
A
Goodbye. Rest up. Bye bye.
- [43:22]
B
Today's episode of the Times Tech Podcast was sponsored by Adobe Acrobat Studio Acrobat
- [43:28]
A
Studio is about more than just the PDF.
- [43:30]
B
It's a collaborative workspace designed to make working with colleagues across teams and time zones faster and easier.
- [43:38]
A
One of the features is PDF Spaces, which brings PDFs, Microsoft 365 files and websites in shared spaces.
- [43:46]
B
It helps centralize notes and draft documents so you can see the full picture of a project all on a secure platform.
- [43:53]
A
There's also an AI assistant which can summarize information, pull out key insights and help you understand what's in your files without reading everything manually, which helps if
- [44:02]
B
you're dealing with a lot of information but are short on time.
- [44:06]
A
And with integration with Adobe Express, you can turn documents into presentations or branded content without switching tools.
- [44:13]
B
So it's not about creating more paperwork, it's about about making your paperwork work harder.
- [44:19]
A
Check out Adobe Acrobat Studio and its new features now.
- [44:23]
B
This episode of the Times Tech Podcast is sponsored by PwC, AI that delivers
- [44:29]
A
it isn't just about the tech. It's a new way of thinking and it's empowering your people to create value in new ways. This is where PwC helps you shape AI to fit your business, delivering real value working with you, combining strategy, industry insight and deep tech expertise to help you innovate with impact, empower your people and accelerate growth.
- [44:52]
B
PwC knows how your business works so you can put AI to work for your business. Discover more@pwc.co.uk.