← The Times Tech Podcast
The Times Tech Podcast
AI in security and defence: is tech making us safer?
00:42:51
BAE Systems' Julian Cracknell on killer robots & a new era of AI deterrence
Loading summary
Transcript99 lines
- [00:05]
A
Hello and welcome to the Times Tech podcast, where every week we unpack how technology is reshaping business, culture and everyday life. I'm Katie Prescott covering all things tech here in the city of London.
- [00:18]
B
And I am Danny Fortson out here
- [00:20]
A
in Silicon Valley, which is, Danny, the place to be this week. I am so jealous. You are at what I think can only be described as the tech trial of the century. The millennium.
- [00:35]
B
The millennium.
- [00:37]
A
Elon Musk versus OpenAI. The beef between them is Elon Musk and Sam Altman Co founded OpenAI, among others. And Elon Musk said, we founded it as a charity and he's turned it into something commercial in very basic terms.
- [00:54]
B
Yeah, basically, I got bamboozled. I was a prime funder. I put in $38 million to kind of get it off the ground and fund it in its early years. And then they fell out over strategy. Elon Musk left and said, and this is in an email, gave them, quote, unquote, exactly zero chance of survival. Once he left, he's like, yeah. And of course, they went on to very much survive, turn into a for profit, have a, you know, establish a for profit subsidiary, which is now worth $800 billion. Launch ChatGPT, Kickstart, the AI revolution. And so this fight is really around. He's like, hey, I donated this money to a charity. Now it's. Now it's one of the most valuable companies on the planet. That can't be.
- [01:44]
A
Okay, pony up, guys.
- [01:47]
B
Exactly. So he's wanting just a cool 134 billion. I mean, fine.
- [01:53]
C
You know, that seems reasonable.
- [01:56]
A
What was he like in court that first day with Elon Musk giving evidence, like, and what's been going on?
- [02:04]
B
Yeah, so it was a total zoo. So I got there at like 8am and there was a line of maybe 75 people waiting to get into the building, all kinds of camera crews, et cetera. So just a real spectacle. You got in there and there was like no room in the courtroom because it was fully packed. You have to show up kind of 5, 6am to try to get a seat inside the actual courtroom.
- [02:30]
A
Then they have like, queuing for Wimbledon.
- [02:32]
B
Exactly, exactly. But without the strawberries. But then you get. But they also have an overflow room that. That has a live video feed and audio, and that's just like literally down the hall. So I was in the overflow room with probably, I don't know, at least 100 people, a lot of journalists, there's a lot of lawyers, and in the foyer are all these, like, Beefy looking guys with earpieces, which are all of the security detail for the various billionaires who were in the room that day. So they're all just kind of hanging out, looking kind of big and buff and mean. But you had Elon Musk in the room. You had friend of the pod, Sam Altman in the room.
- [03:16]
A
Wow.
- [03:17]
B
You had his co founder Greg Brockman in the room. And so I was there for opening arguments of both sides.
- [03:24]
A
Astonishing. Well, we should say OpenAI has strongly disputed Musk's claims and characterizations. Anyway, Danny, it's really exciting to hear what's going on and I can't wait to hear more over the coming weeks. But let's move on to today's episode where we're going to be talking about all things defense and the price we pay for security both globally and at home. Because this has been a massive issue at the moment, hasn't it?
- [03:51]
B
Yeah, exactly. And this is all of course, germane because what we're really talking about is AI and how does this all work, this very interesting dilemma around national security, safety at home and abroad in this age of AI. And the reason we're looking into this is because several stories have come together at once in the last couple weeks that really shine a very bright light on this moment in tech and defense. So this week, Alphabet, which is, you know, Google's parent company, signed a deal that allows its AI to be used by the Pentagon for quote, unquote, classified uses a little vague, but one could speculate. And this was despite a letter of protest signed by more than 600 Googlers, delivered to CEO Sundar Pichai. And in signing the deal, they join really a pretty fast growing list of Silicon Valley companies currently making agreements to kind of do work for and sell stuff to the US military.
- [04:49]
A
Yes, it does feel like quite a moment as most companies grapple with what their relationship should be with government when it comes to defense. There's one company that doesn't grapple with that and that's pretty clear on what their view is, of course, and that is Palantir. Yeah, zero grappling, no dilemma, just a quite a firm view. And they put out this post on X, which has had about 32 million views, setting out its manifesto, which is just worth going into a bit. It sets out a future basically where AI warfare, it says, is inevitable. And it calls on Silicon Valley tech bosses to, and I quote, participate in the defense of the nation. I mean, it sounds like something from World War II.
- [05:32]
B
Yeah. And it positions companies like Palantir it's kind of now helping engineer national security for this kind of new AI era. And they envision a world that grants, you know, really a handful of unelected tech executives huge influence over military strategy, domestic safety. And critics have called us. You know what, we're kind of, we're a step toward, if not already in a surveillance state. So this week we're looking at how AI is transforming defense and security and asking the critical question, are we actually being made safer because of AI?
- [06:09]
A
And in keeping with that theme of defence, one of the most controversial aspects of AI use, our big interview is with the chief technical officer of defense company BAE Systems. Julian Cracknell is going to talk to us about what the future of defence looks like, and if it involves killer robots going to war and scarily making their own decisions. But should we start off by talking about Google's deal with the Pentagon? Danny, can you, can you just take us back to what's happened?
- [06:38]
B
Yeah. So this week, Google has reportedly signed a deal with the Pentagon to use its models, its AI models for classified work. And the agreement allows the Pentagon to use it for, quote, any lawful government purpose. You may recognize that terminology because that was the kind of the key phrase when OpenAI and also Xai Elon Musk Co. Signed similar deals to kind of start selling their wares to the Pentagon. Anthropic famously said, we're not comfortable with this because effectively they don't trust the government in its current form to only use it for, quote, lawful purposes. And the agreement sparked employee pushback inside Google. 600 members of staff signed a letter and sent it to Sundar Pichai. In it, they say, quote, we want to see AI benefit humanity, not to see it being used in inhumane or extremely harmful ways. This would include lethal autonomous weapons and mass surveillance, but extends beyond, end quote.
- [07:42]
A
And this is somewhere Google's been before and grappled with before. So back in 2018, they scrapped a contract with the Pentagon project maven after thousands of employees protested against the use of AI tools to analyze drone footage. So now, eight years on, they've made another agreement, and it feels like employees are pushing back again. Let's have a closer look at what the deal entails. So essentially, it says Google has to adjust the company's AI safety settings and filters at the government's request, and it doesn't give Google the right to control or veto lawful government operational decision making.
- [08:26]
B
Yeah, the Pentagon has not commented on this. And it's very similar to what, you know, we had Sean Gurley on last month, talking about the whole anthropic kerfuffle. And he's like, look, if you're selling something that can be used as a weapon or for weapons or with weapons to a military, you can't say, hey, don't use it to fire a gun in that way. You know, like, you're kind of handing over power of these tools. You can't set rules about how it can be used. You're basically selling a product and then, you know, the government, the military, whomever is going to use it however it wants. As you mentioned, this is really significant because back in 2018, and I covered this at the time over the Project Maven AI drone contract, 3,000 people signed a letter of protest. It was a really tumultuous time in Silicon Valley, especially around defense. And then at that time, eight years ago, the company backed down and they basically declined to renew the contract. And who picked it up? Palantir.
- [09:32]
A
And what's really interesting in a minute.
- [09:34]
B
Yes, indeed. And what's really interesting, back then, 3,000 people signed this letter of protest. Today, Alphabet is twice as large, so it's now 190,000 people. Only 600 people have objected in this way so far. So I'm not, I'm not a mathematician, but that feels like on this very
- [09:55]
A
unscientific numbers even ChatGPT could get its head around.
- [09:58]
B
Exactly. Just in this one unscientific metric that's like, you know, one tenth the outcry, at least in this form. And again, I think it speaks to this moment of just this vibe shift we've been taught. We've, we're talking a lot about a year ago where it's just whether it's due to the war in Ukraine, what happened in Israel, and Trump being in office, like, there's been this real turn. It used to be turning away from Washington, and now there's a real turn toward Washington when it comes to, you know, big tech. Like I said, it's the vibe shift. And if you talk, and if you look at, like, Andreessen Horowitz, one of the biggest investment firms out here, venture capitalists out here, they have their, what they call the American Dynamism Fund, which is really all about funding a whole new generation of American defense startups, which, again, if you go back five years, that idea would just not really even be talked about in polite company out here. And now they're, like, stepping into that and be like, yeah, we're going to build a new generation of defense, and we're proud of it, you know, so again, it's just, it's quite a change.
- [11:08]
A
It does also feel naive, doesn't it, to think that you could sell technology to the government and then put these guardrails about, about how it uses it on, onto it. It's. I think it's either. It's a. It's quite binary really. And it also begs the huge question, should US AI companies be allowed to say no to Washington? And Palantir, which is the company that interprets massive amounts of data, shared this manifesto a few weeks ago when it said, no, absolutely not. Silicon Valley owes a moral debt to the US for making its rise possible. And led by its boss, Alex Karp, said national service should be a universal duty. I mean, it's worth saying this sparked so much controversy when it came out, and some critics called it the ramblings of a supervillain.
- [12:00]
B
Palantir put it out on X. It's 22 points, kind of point by point about kind of really. It's a summation of the Technological Republic, which was the book that Karp put out some months ago, which kind of laid out like basically his life story. And also building Palantir and his kind of worldview, which is, has long been, let's call it, distinct from everyone else. Like, you know, Palantir.
- [12:28]
A
Delightfully diplomatic of you.
- [12:30]
B
Palantir was basically created by him and Peter thiel back in 2003 in the wake of September 11th, because if you remember, Peter Thiel, he was the founder of PayPal. They had created all these anti fraud systems and he's like, maybe we could use these to solve a problem of like, you know, finding those needles in the haystack of intelligence, helping governments, you know, use their data better, find the bad guys before they do bad things. Thus was born Palantir, and it's been a pariah from day one.
- [13:04]
A
It's worth just talking a little bit about the scale of Palantir, I think. So just before we came on, I saw its market cap is currently $326 billion. Shares are up more than 1,000% since it floated and just 17% in the past year. But probably even more interesting than those numbers is the boss, Alex Karp, who is, should we use the word, distinct again?
- [13:28]
B
He's distinct.
- [13:30]
A
You've met him?
- [13:32]
B
I have.
- [13:32]
A
What's he like for people.
- [13:34]
B
Yeah.
- [13:34]
A
Who don't know him.
- [13:36]
B
So I, I did a profile of him a few years ago and I went down to their office in Palo Alto where I met him. And I walk in and he's like, he's doing. He does kata, which is a form of kind of martial arts. And what's. You kind of moving, very deliberate movements through the air, kind of looking like, you know, something out of a movie. He was doing tai chi or kata in the foyer, and people were just kind of veering around the boss, you know, the billionaire boss of their company, as he's like, very deliberately doing his Tai chi in the middle of the space. And then, you know, we go into his office. He has this wooden sword that he goes and, you know, talks in great length about this wooden sword that he has and what he uses it for. I would say he does not suffer from a lack of clarity of his views. He views Silicon Valley as this, like, island of moral bankruptcy where all of the best people, all of the best talent, at least this was a few years ago, was going into optimizing ad spend, getting people hooked on apps. And he's like, this is just all garbage. You guys are giving up your duty to the country to actually get involved, help the government, you know, help, quote, unquote, war fighters and equip them with the best kind of 21st century tools that you all know how to build, and you're choosing not to. And so Palantir had been the subject of a bunch of protests in, you know, because Silicon Valley, famously progressive. So eventually he left. He relocated the company to Colorado. He now lives in New Hampshire, out in the middle of nowhere by himself. And he runs this company that has just grown dramatically in the past four or five years, basically, since. Since he and I sat down with, with the wars in Israel, Ukraine, now Iran. And really they've been one of the kind of stars of the stock market. And also his star has risen, especially with, you know, the advent of the Trump administration. They're kind of, let's call it more muscular approach to international relations.
- [15:48]
A
And they process vast amounts of information, yes, as you said, for defense and for governments, but also for organizations like banks. And over here in Britain, people probably know them best because they won an enormous contract with the British government to process medical data for the NHS worth 330 million pounds. So some campaigners have said, we're not sure. Is that dangerous? Is this the sort of company we want to be handling our public data? Louis Mosley, who's the chief executive here, has pushed back very hard on that, saying, look, you need to see what we're doing. He said he welcomes the extra scrutiny that the company is getting, and it is getting a lot of scrutiny. Here at the moment, but, yeah, it's growing and growing. Well, that feels like a good time to introduce this week's guest, who's from the largest defence contractor in the UK and Europe, BAE Systems. And they just work well with everyone. The UK Ministry of Defence, US Department of Defense, Denmark, Sweden, Saudi Arabia. With a market cap of £60 billion. This is a really incredibly topical interview. Given the conflicts going on around the world, defense isn't something that is easy to discuss. And I think it's fair to say, Danny, this is certainly a different side to a lot of the consumer and B2B tech that we talk about.
- [17:09]
B
Yes. And I just want to call out, you did make a small mistake. It's not the Department of Defense, it's the Department of War. It's been rebranded. Don't you know?
- [17:17]
A
No way. It's do not. Dod. Okay.
- [17:21]
B
D O, W department. Anyway, so who better to answer that question and about what the future of defense and national security looks like than Julian Cracknell. He is the CTO of BAE Systems, and we started by asking him just to go back a pace or two and explain exactly what the company does.
- [17:45]
D
We're the biggest defense company in Europe. Pretty much depends slightly on how you measure it. One of the largest in the world. And the way I like to think of it is from outer space down to the seabed. So from a space perspective, we build satellites for communications and observation. Then in the air domain, we have the combat air programs that we're delivering at the moment, as well as all sorts of uncrewed vehicles and drones and similar things on land. We're the largest provider of land vehicles in the Western world. We provide complex warships, we build submarines, as well as other uncrewed vehicles that float or go on the land or under the sea. And then there's a whole set of systems that connect those things together, whether that's communication systems, command and control systems, things like electronic warfare. So it's a very kind of broad portfolio and that's delivered internationally. So very large in the uk, big presence in the us, very large in Australia, as well as lots of other countries around the world.
- [18:48]
A
And give us a sense of what your role as Chief Technology Officer entails, because I just can't even begin to wrap my head around that. I mean, the research and development must be enormous for BAE before you get on to everything else.
- [18:58]
D
Yeah, so I'm responsible for four things. The group strategy overall and thinking about how we run the strategy process. A number of the operational functions, so project management, engineering, manufacturing, Quality technology. Now, that's not every bit of technology we do. Clearly there's a lot that happens within the businesses and within the sectors, but my team are thinking about some of the areas we might go next, particularly how we incubate new ideas and think about taking those, you know, establishing whether they've got legs and might go somewhere. And then also I'm the CIO as well, so the it, cybersecurity, all of that kind of stuff.
- [19:35]
A
How has AI changed what you do over the past few years? Because I imagine AI has always been integrated into what BAE Systems does, but with the advent of generative AI ChatGPT launching in 2022 for everybody, things have changed quite dramatically.
- [19:51]
D
I mean, you're absolutely right that AI has been involved or defence has made use of AI for an enormously long time. If you think about the Typhoon platform, which you may recognise as a combat jet that's used by the uk, it has two very distinctive little flaps at the front of it that keep it stable because the platform itself is inherently unstable in order to be more performant. And those flaps are controlled by an AI system. And that's the only reason that the plane stays up and is capable of flying is because there is an AI augmenting the pilot at all times. So the kind of use of AI is definitely something that's been prevalent in defence. Things like generative AI and LLMs are clearly having impact in some ways, in similar ways that you would see in other non defence organisations, you know, helping people write reports and similar, but also in things like controlling drones, being able to interact with them with, you know, natural language and ask it to perform a function and they'll go off and task the drone to behave in that way. There are other ways that we're making use of AI, so things like physics based AI models to allow us to design complex components more quickly, rather than having to do lots of iterations with humans trying ideas out and then testing them, using the physics based AI models to try and get to an answer more quickly. So it's definitely something we're making lots of use of in defense.
- [21:17]
B
I just want to get your sense
- [21:18]
C
of kind of the moment we're in, because if you go back to 2018, it was another moment out here in Silicon Valley in particular, where there was this very kind of infamous project Maven drone contract with the Pentagon that Google had. It was revealed there was a big walkout and you had all these employees being like, we don't want to be in the business of war. Google Gave up the contract, Palantir picked it up. Palantir famously left Silicon Valley because they're like, basically, you all need to be more patriotic, stop worrying about better ads, make weapons of war to defend the west, et cetera, et cetera. And there's been a real shift in, especially the last, I would say 18 months to two years, where all of a sudden kind of defense is seen as something that is okay to do. Now, from where you guys sit as one of the biggest defense contractors in the world, how has the playing field changed in terms of what the big tech companies in Silicon Valley are doing and how they're showing up in your universe compared to say, five, six, seven, eight years ago?
- [22:26]
D
I mean, clearly that question of whether you're comfortable to work in defense is one that we've thought about a lot and we're really proud of our role in defence. Our motto, if you like, is we protect those that protect us. And I think it's a very inspiring thing for us as an organization in terms of access to technology. My sense is it can only be a good thing that more technology is available to be used in the defence space. So where we are today I think is clearly helpful. I think the risk is if that appetite changes and it starts to take away some types of technology that we have started to rely on. But certainly from my perspective, it's clearly a good thing that people are both applying their technology towards defence and also actively investing in are the tech companies,
- [23:15]
C
whether they're startups, because there's a bunch of new kind of American defense startups like Anduril and others. Are they showing up in pitches in a way that they just weren't there before?
- [23:25]
D
There are a lot more companies interested in defense and I think that has changed fairly significantly over the last probably five years or so. I suspect you tie it back to the events in Ukraine and that has, in some ways it has re established the reality of the threat, made it clear to governments around the world that they need to invest in defence and therefore that has attracted investment and technology into the space.
- [23:53]
A
And how important is it now then in warfare to have the right technology? I mean, is it a question of whoever spends the most, has the best tech can win wars? Or as we've seen with, I guess, Iran recently, and some of the cheaper technology that they're using, which has been relatively successful in some fronts, Is that, you know, do you feel it's still an open playing field or is that sort of daft question?
- [24:16]
D
Well, it's not a daft Question. There is a balance to strike between kind of exquisite tech at one end and the mass of volume that you can get at another end. And, you know, the focus on affordable combat mass is something that I think has become much more apparent coming out of Ukraine. In the Strategic Defence Review in the uk, which was published a few months ago, they talked about a balance between exquisite capability, reusable capability and attritable capability. And exquisite meaning generally top crude, crude C R E W E D with people inside it and therefore with the protection that's necessary around that to protect people. But typically the view expressed in the SDR is probably about 20% of your capabilities would sit in that space, probably about 40% reusable and 40% attritable. So, you know, you're certainly seeing a recognition from governments again, driven to some extent, I think, by what's coming out of Ukraine, that more focus on volume and the kind of lower cost capability is necessary to support and augment some of that more capable technology.
- [25:26]
A
And apart from the government's review of this, what do you think BAE has learnt from Ukraine?
- [25:31]
D
We're very proud of our support of Ukraine. The pace of technology adoption is really high. There's been a great or significant use of commercial capability in Ukraine and you see that in a number of different places in the use of commercial drones, in the use of things like commercial satellite imagery, commercial communications. But those technologies tend to get competed out very quickly and we're talking about kind of days and weeks rather than months and years for those technologies to get effectively sort of out competed by the opposing technology. And so that pace of evolution, the pace of adaptation, to be able to respond to new threats, embody that in hardware and software and then deploy, deploy it is a real change. I think there's always been that pace element in defence and we talk about urgent operational requirements, which is essentially, you know, the customer has a burning problem that needs to be solved. And you know, defence companies are very good at being able to respond to those things. But I think what we're seeing is probably more of that on a more consistent basis.
- [26:44]
C
And, you know, obviously you make bombs and bullets and tanks. You also make bugs like cyberbugs, you know, because we're in this moment of AI and we just had all this strum and drang about Mythos preview and how it can hack virtually anything and everything. And those are quite powerful weapons, right? Especially when you're talking about, you know, things that have been deployed to, like hamstring Iran's nuclear program and things like this. Is that part of your operation of like beyond the kind of the big kit that you're also building kind of cyber weapons?
- [27:20]
D
We've got a really broad portfolio, as I talked about earlier on, and I'm not going to get into any specific capabilities, if that's okay.
- [27:27]
B
Classified, I can neither confirm nor deny. Right.
- [27:33]
C
But I think what feels like really interesting and different about this moment is that it's kind of everything is being democratized. When you talk about a $200 drone that can go do real damage versus a $2 million Tomahawk missile, that feels like we're in the midst of this kind of qualitative and quantitative shift in how wars are fought. Is that how you guys view this moment?
- [27:54]
D
I mean, that comparison is clearly true. You know, relatively low cost drones are being taken out by relatively expensive interceptors. I think what gets missed in that comparison though, is what's the thing that you're defending? You know, if you're protecting a nuclear reactor or a city, actually you're going to spend a lot of money to do that regardless of the cost of the thing that's coming at you. So the straight kind of pound for pound comparison in terms of the cost of things, I'm not sure tells the whole story. That said, it's clearly important that given this volume game of large numbers of relatively low cost attack drones, that we can respond to that. So one of the pieces of research that we've been doing and have just announced actually is taking a low cost rocket, attaching a laser guidance system onto the front of that, and then firing that from both a drone and also combat air Typhoon platform. So suddenly you've got a really low cost interceptor that's able to respond to that. And I think that that kind of. I was going to say arms race. That's exactly the right term. That kind of back and forth between technology and then the counter to that technology is going to keep going and is one of the things that drives innovation in the defense sector.
- [29:10]
C
And to Katie's earlier point around, you know how AI has changed things. When there was the blow up between Anthropic and the Pentagon, we had a guest on Sean Gourley out here and he said a lot of people like within the defense space view AI a little bit like nuclear weapons. Like if you have clear superiority in this very powerful technology, it is what they call the next kind of offset. Like you render your opponent defeated before you even have to fire a shot. Because they just know and you know that you're just going to Win and win decisively and cause real damage. Is that accurate or is that how you guys view the world?
- [29:54]
D
I think what's really being talked about there is speed of decision making. So you know that concept that if you can make decisions faster than your enemy, that you can get inside their decision cycle, outthink them and therefore outmaneuver them and at some point get them, you're so much faster than them that they're unable to respond. That's been a constant in warfare forever. And I think what we're seeing here with AI is the opportunity to really accelerate that. I'm not sure I would describe it in quite the same way in terms of that. That kind of, you know, life is fundamentally different, but clearly what it is doing is creating opportunities to make decisions much more quickly, to take in volumes of data that would have been harder previously and, you know, synthesize those, bring them together in order to make better decisions faster. And that does clearly have a strategic benefit.
- [30:51]
A
So we are in a geopolitical arms race then, when it comes to developing AI, sounds like, from what you're saying, to try and get ahead of other people, to get it, as you said described it, get inside their heads. I mean, that's sort of what the.
- [31:04]
D
We've got to make use of the best technology that we can in whatever space that we can in order to be able to out compete our adversaries. And AI is clearly a significant element of that. I don't think it's the only element. There are lots of other areas where technology is advancing.
- [31:20]
A
I'm going to ask you a question that you'll probably hate, but I have to ask it. How has your job changed with Donald Trump in the White House and the sort of split in the special relationship in the last year or so.
- [31:36]
D
I do hate that question. So, look, I'd look at it from a couple of different angles. We have a very large U.S. presence because of U.S. federal legislation that is entirely separate from the rest of the. The business. And so they are working with the US administration and working through the impact of that for the rest of the business. We're thinking about where our customers are going in terms of their decision making around new equipment, the types of capabilities they want. I think it's not entirely related, but the issue of sovereignty, who has control, particularly who has the ability to make decisions about how you use equipment, how you can modify equipment. You know, that again, tours ever thus in defence, it's always been a concern, but certainly in the recent UK sdr, Strategic Defence Review, you see a much greater focus on sovereignty and the element of industry as part of the defence enterprise. And again, I think that's drawing on the lessons of Ukraine, that deterrence is not just about the capability that you can deploy today, it's about how you can regenerate that, how you can iterate that, and how you use industry and a close relationship with industry as a critical part of the overall defence estate.
- [32:56]
A
I guess we'll be right back.
- [33:07]
C
Gillian, you mentioned control, which brings to mind autonomous weapons. And Palmer Luckey, who I'm sure you know runs Anduril, which is like now worth $13 billion. You know, this defense tech startup.
- [33:22]
A
Yeah, I love that you describe as a startup.
- [33:24]
C
Exactly. It's very big now, but he's, his kind of line is I love killer robots. Like that's what they're building. And I'm just wondering how you guys think about that. Like where are we on the continuum of just. You talk about decision making, control, and especially in the context of China where they have this huge manufacturing base where they're building more drones than the rest of the world combined and all of it and they're racing again ahead in AI. Is this where we're going and how do you guys think about killer robots? This idea of just letting the AIs go to war, make decisions and blow stuff up.
- [34:08]
D
I don't really love the phrase killer robots. What we're seeing at the moment is actually a lot greater use of remote controlled systems. So you've got human in the loop in most of the systems that have been deployed, certainly in Ukraine. And that reflects the fact you've got a very mobile battlefield that's changing on a regular basis. And therefore having humans, you know, actively engaged is the right choice. We will inevitably see more autonomous systems. And indeed, you know, we are doing a lot of work looking at common autonomy systems and the deployment of those in all sorts of domains, land, sea and below the sea as well. You know, the use of autonomous systems is going to be a political decision as to the acceptability of that and the role that humans play in that decision making process. Whether you have a human in the loop or a human on the loop who is making the decision about where you deploy the system and the rules of engagement that are put around it. So I think we clearly will see more autonomous systems over time. I have to say I'm not sure I do share the vision of armies of autonomous systems fighting against each other to determine who wins. I don't think that makes sense actually. In A political context, but being clear around the rules that you can put around systems so you can have confidence in how they're going to be used. That's where an enormous amount of thought in government and in ministries of defense is going on. And I think we will see that
- [35:34]
C
presumably this is like, I don't know, self driving cars. Like the technology is already there. Right. It's just the question of how do we roll it out. In other words, there has to be a decision to your point by governments or the west or whatever to cross that Rubicon to actually be like, okay, we're going to take our hands off the wheel and maybe that moment never comes.
- [35:58]
D
Well, we do have autonomous systems now. So, you know, defense of a maritime platform will be done, some of it will be being done autonomously. Then as a missile comes in, there will be systems that spin up, see that, and fire a response faster than a human could engage with. But there's a lot of testing around that clear understanding of the situations in which it will respond. But the Rubicon in that sense has been crossed. There are autonomous systems. I think the question is how far you then go with that. And ultimately that's a policy decision.
- [36:29]
A
And does it mean the end of boots on the ground?
- [36:32]
D
One of the things we can use autonomous systems for is trying to take humans out of harm's way wherever possible. But, you know, does it eliminate boots on the ground? At some point you have to be able to demonstrate political will and you know, that's about being able to hold ground.
- [36:47]
C
Katie always gives me flack because I love a robot.
- [36:50]
A
It's just a male thing.
- [36:52]
C
That's. I love robots. The humanoids, there's so many humanoids and you know, the humanoids are coming, etc.
- [36:58]
B
Etc.
- [36:59]
C
And it feels a little bit ridiculous in science fiction, but to the point around boots on the ground, if you have build up an industrial base where you can pump out 500,000amillion.
- [37:10]
A
Yes, in Star Wars.
- [37:11]
C
Yeah, exactly.
- [37:13]
A
Stormtroopers.
- [37:15]
C
Is that a realistic thing? Is that like, you know, when you talk to the Pentagon, they're like, yeah, we're just, we really love to have a droid army in 10 years where we don't have to and that. It sounds ridiculous, but also it's keeping humans out of harm's way.
- [37:29]
D
It's a little bit hard to know how to respond to that. We will clearly see more autonomous systems. I think it's unlikely we will see armies of humanoid robots because why would you design them to look like humans in the first place? For a start, you were designed for the mission, but you will see more autonomous land platforms, more autonomous air platforms, at working conjunction in order to take humans further back from danger where you can. But that will be. I go back to my earlier point, that is in conjunction with other systems. And that element of in the moment decision making, I think, will always be. Will always be necessary.
- [38:06]
A
What sense do you have about the way politicians and governments are leaning on that issue of sort of trying to use robots more and taking boots off the ground? Like, which way do you think we're going? If you're a betting man.
- [38:22]
D
The SDR was clear about more use of reusable and attritable platforms where you would expect to see autonomy. The Navy is talking about its future vision for the Navy, which is the hybrid Navy, which will have more autonomous platforms alongside leaner crewed vessels. So I think we will see that direction of travel continue, but I don't think we'll get to the point where the humans aren't there.
- [38:49]
A
Well, I thought he was incredibly diplomatic. I mean, it is a really difficult thing to discuss. And that line that he said, you know, in defence, our motto is, we protect those that protect us. I mean, a lot of people, I think, will wonder about that issue of protection. But if we put that aside for one moment, the future that he painted of the defense industry. Industry was. Was really interesting.
- [39:14]
B
Yeah. And like I said, there's just a whole new wave of companies in defense, like a whole new wave of defense startups. And it's just, it is quite extraordinary how dramatically has changed. And the one thing I would say two lesson is this is really the start of Silicon Valley out of, you know, coming out of World War II, there was a bunch of Pentagon contracts. You know, Lockheed Martin was out here. Like a lot of the kind of the tech industry started with big military contracts building stuff, whether it's chips or missiles or bombs or radar systems or whatever kind of gear. And then over the course of the years, the kind of industry was like, yeah, we don't want to really do that anymore. If we do, we're not going to talk about it. And now it's kind of come full circle and you have this whole new generation and this whole new attitude that is kind of proudly patriotic about this, whether people can have their own opinions about it. But it is quite extraordinary how the kind of the shift has happened, I would say again, over the last five years, going from big walkouts and protests to, hey, check out our new cool, like, you know, bomb, robot drone thing. Or whatever it may be on the,
- [40:34]
A
on the tech front, actually. I mean, obviously bomb, robot, drone, things are sort of interesting. But the most interesting thing to me was the, the question that he didn't answer when you asked him about cyber weapons and you said, are you building cyber weapons? And he said, we've got a really broad portfolio and actually wouldn't go into the details of that because I think you're spot on in terms of the future of warfare. As quantum computing develops, you know, the acceleration of generative AI, it is absolutely that cyber warfare that not only governments, but obviously businesses and whole organizations absolutely terrified by. But yeah, he dodged that question quite neatly.
- [41:19]
B
Yes, indeed.
- [41:20]
A
I think we can infer what he really meant.
- [41:22]
B
Yes, exactly. And he has no kind of, what's he gonna say, right? He's like, yes, actually we are building cyber superweapons, but clearly that's part of the mix.
- [41:31]
A
Do you know what? I'm sorry to say I'm not sure we have answered that big question. Is AI actually keeping us safe? I don't know how I feel about that. I actually feel like it's.
- [41:41]
B
Yeah, I don't think we know. I don't think we know. I think it's just like, it's like everything with AI actually going back to something that Elon Musk Musk said this week in court. He's like, AI is double edged because you can do all of this new stuff, theoretically, you can have all these scientific breakthroughs, but you can also build really dangerous things. And I think that we're in the middle of this process of trying to figure out what we enable, what we're okay with, what we're not. And you know, obviously when you're talking to a BA which is the largest defense company, one of the largest in the world and certainly in Europe, you know, they're right at that, at that bleeding edge of figuring out, okay, what are we to use this for? What level autonomy are we good with? What are we not? All that stuff. So it's really tricky. It's a really interesting, weird time to be alive. If you guys like the show, tell us, go to techpod@thetimes.co.uk that is Tech Pod at the Times. Leave a comment, good or bad, whatever we look, we love, we love feedback. And we will see you back here next week.
- [42:48]
A
See you next week.
- [42:49]
B
Bye bye.