A (3:04)

from the beginning, this was Silicon Valley helping. They were the ones that were providing the support.

C (3:09)

This was the hope. But both those things were difficult. So the initial aims were actually much broader than just bringing computer vision to drones. It was, I learned through the process of researching this book, to bring AI and put it at the heart of How America Makes War. And it was always seen by its mavensbachers as a stepping stone to autonomy, to actually removing humans ultimately from the loop, or bringing humans and machines together not just for computer vision, but for multiple different types of intelligence and combat operations. So the scope was very wide.

A (3:43)

And was Google the only company working with the Pentagon at that point, the initial point, or were there others?

C (3:51)

One of the other key things that Project Maven was trying to do was bring in Silicon Valley. So for years the Pentagon was depending on the big defense primes, Lockheed Martin, Bowen, all of these ones that people know and they needed, they felt cutting edge AI. And that meant Silicon Valley. The chief of Project Maven, who was one of the founding visionaries for it, desperately wanted Google.

A (4:15)

This was Robert Work.

C (4:16)

This is Robert Work. He was the backer as deputy Defense Secretary at the time, he saw AI as a stepping stone to autonomy. But the man he appointed to run at Day to Day, the chief of Project Maven was a Marine Corps colonel named Drew Cucor, whose story I tell. He wanted Google DeepMind and he didn't get them. Demis Hassabis is very publicly against AI warfare. Certainly at the time in 2015, he'd signed onto a letter, he couldn't reach them, he couldn't convince them. So he moved to his next favorite option, which was Google Brain you. He couldn't get them either. So then he moved to what was told to me was described as Team B at the Google, which was Google Cloud. They were just getting going at the time Project Maven came to them, I think they only had four customers, so they were much more willing to work with the Pentagon. And they started trying to really bring Drew Cukor's vision to life, which was one to identify objects, but also he had this vision to create objectives, almost Google Earth for war. Google Earth was already used regularly by military operators, but the platform itself hadn't been adapted and that's what he wanted. He also went to multiple AI startups. Some of the best brains were running, would you believe it, a company that worked with a wedding blog. And so they were using their computer vision algorithms to identify the tears on a wedding cake, bridal veils, suits of a groom, And Drew Cuckoo got on the Amtrak. He came down to New York and said, I need you to work on war. I believe your algorithms can save lives. And he made this pitch to this company, Clarify, and they started working with the Pentagon.

A (5:57)

So, I mean, before we talk about things that are concerning about the integration of AI into war fighting, I mean, to be clear, it's almost inconceivable that you wouldn't use AI to help the Defense Department when you're using it for literally everything else. I've heard for decades that there's too much information that is collected that humans can't possibly sift through it all. It's an overwhelming task to understand how to track down a terrorist, how to understand what threats to the United States might actually be. Those vectors you're going to want to use compute. So, I mean, how much of what they have been, of what initiated this project was something, in your view that makes a lot of sense and that if the Americans didn't do others were going to that are adversaries.

C (6:45)

The Department of Defense likes to say they've been using AI for 60 years. So whatever AI was back then, they were trying to adopt it. The main problem that AI was trying to solve in 2017 or so was this problem of too much data. So it wasn't. Which was a real problem, a real problem for them. Drew Cukor himself had been deployed to Afghanistan in 2001, soon after 9 11, and had found that as an intelligence officer, he could not get information to operators on the front lines who were very soon suffering the consequences of improvised explosive devices. And nobody was tracking sufficiently data that might help them figure out where the next IED might be laid. Eventually what they did was they brought in data analytics over the coming years. But until then, they were relying on PowerPoint. They were sometimes logging incidents just on paper. They would create wheels, circles on the wall like pizza slices and try and work out chunks of time when attacks happen and then start correlating it even with the Moon, because it turned out that the Taliban was laying these bombs according to the weather systems. Now, data collection, bringing that information together, logging which house was which began to be the beginnings of what the US military saw as this possibility that data could help them plan better for war. That was beginning to happen before Project Maven. That didn't need AI, but AI needed compute, as you mentioned. It needed cloud, which didn't exist at the time Project Maven started going. And it needed really good data, accurately labeled data, and no one had tried to do that before. In fact, for the first few years of Project Maven, they're going around different commands, different units, asking for data, being shown cupboards with old footage that they can't even figure out how to translate. So there's been this enormous effort to digitize the US military that's still underway.

A (8:46)

Now, when I think about companies, there's a lot of resistance to use AI senior management. Frequently you'll have someone who's evangelical about it, but others that are like, oh, it's going to take my job. I'm not so interested. Bureaucracies can be famously slow in this sort of uptake. The military is one of the world's largest bureaucracies. How effective has rolling AI out across the Department of Defense actually been over the course of the past decade?

C (9:14)

At the beginning of Project Maven, the team trying to bring AI to this large bureaucracy felt like they were fighting a rearguard action, an insurgency inside that very big office. And they encountered resistance really everywhere. They couldn't get the services even to play with them, never mind to start funding this effort. They were constantly experiencing military operators who said, I don't need this stuff, I don't trust this stuff. I have my way of doing targeting, for example, one of the most critical and consequential decisions that the US military makes. And they didn't want to go near AI. It's begun to change, but I would say there's still enormous concern and debate inside the Pentagon. And even for Drew Cukor who was leading it, he told me that he was told he could fire whoever he wanted in order to get this done. He always felt that getting AI into the department would be a knife fight and it would depend on adoption, testing and really getting it out into hot wars, doing something very controversial in order to incrementally improve it and adapt it to what people actually needed.

A (10:21)

Now, when we look today, would you say that the principal military primes, the big military industrial companies, are they adopters? Are they saying, yeah, we're fully on board, we know that our systems need to be integrated with AI because that's important, or are they acting like legacy providers that are trying to defend big spends on older systems and keep these new guys out?

C (10:47)

It's a little bit of both. There is no defense prime that is not using AI. Everyone is trying to do it, everyone is adopting it. But you're also beginning to see tie ups. So Palantir, which makes the user interface for Maven smart system, that's the platform that comes out of Project Maven.

A (11:03)

And is it Fair to say they're the principal beneficiary as a private sector company so far, of AI utility in the United States military.

C (11:10)

I think that's absolutely fair. It wasn't always going to be that way. But when Google drops out, it decides not to renew its work on Project Maven. The backers of Project Maven seek help from Palantir and they end up making this interface like a Google Earth for war, on which you get a display of friendly forces, foe forces and then potential targets, other information, including elevation, location, precisely what's there. Well, that's the aim. And it's developed over the years to something where you can click through to an element called target workbench, pair a target with a weapon system and then allocate that to be fired at. So it's becoming an integrated digital system and Palantir has the contract for that. So it became a prime contractor for Project maven back in 2020 and they're

A (12:06)

now working with the Lockheed's, the Northrops.

C (12:08)

Exactly. And particularly as the second Trump administration came in with these huge threats that the defense primes need to ref form, that they want to adopt AI, they want to go fast into autonomy. All these words that people weren't particularly prepared to say in public before. The second Trump administration is saying, we want to put more than $56 billion towards drones in the next budget. We'll see if Congress says yes to that, into autonomy, into autonomous warfare. They've even created a group called the Defense Autonomous Warfare Group, or dog. But they have been changing the way in which they require defence contractors to work. And so you began to see a tie up between the palantirs, the andals of this world and some of the traditional defence primes, as everyone was jostling for the new contracts. And to suggest that they are viable not only at hardware, but also at software.

A (13:04)

Now, when we look back at the original desire, which is integrating all the data that's coming in, being sure that when you have intelligence, you can actually assess it. Would the Defense Department now say that with AI they are able to assess the intelligence that's coming in, or is this a never ending problem?

C (13:27)

Some US agencies are even producing intelligence documents solely using AI that no human eyes ever look at. So they are very proud of this effort in the past few years to dive exactly into this process. There are multiple ways in which AI isn't satisfying even them, never mind the detractors who are worried about it. The computer vision still doesn't work right. They haven't Figured out cloud. They haven't figured out how to link together sensors and shooters. But this is the big all encompassing project of what the Defense Department is moving towards. And Maven is at the heart of that effort.

A (14:08)

What is AI doing today in war fighting in the United States that people would be sure surprised about?

C (14:14)

The one that surprised me the most, this is a narrow project, but I discovered one called Whiplash. This is a program to put AI into automatic target recognition and into autonomous navigation on something that America makes a lot of that China doesn't. Jet skis. This is autonomous jet ski robots armed with explosives. And the idea, the concept behind it is that once you have an autonomous weaponized vehicle, you don't need to worry about jamming. So if you don't have your communications link at risk, it, if it's reliable, can go and find a target and execute against that target. The scenario in mind for that is the defense of Taiwan. So if China ever decides to make an invasion attempt of Taiwan, and if the US ever decides to defend Taiwan, that could be the sort of vehicle that might help stave off an invasion over a short period of time.

A (15:12)

These are usable.

C (15:12)

Presently, these are in production. In the book I report that the CIA smuggled some very rudimentary versions of them to Ukraine in support of Ukraine. And one jet ski armed with explosives washed up on the shores of Turkey. And this sparked consternation inside the Pentagon that their scheme had been discovered. In Navy budget documents I found that Whiplash is in low rate production and there is an effort to expand what the US is doing currently in all sorts of autonomous drones. Not only that, but also something. There's a new project starting in 2026 to make voice controlled autonomous drone swarming tech. This is the idea that a commander could say something like left and then a group of drones would take on board that instruction. It would be translated using an LLM and the drones would then be able to move as a swarming group.

A (16:11)

Now, now my understanding is that as of today, what this technology is doing for targeting is it's helping to identify targets, it's making recommendations, a human being is giving the go, no go, and then the weapon system is deployed, is that correct? With these jet skis, for example.

C (16:34)

Is that the way it works with the jet skis? I'd say that's under development. But with Maven smart system, where that process you explain is happening, AI is being used in two ways. You've got the computer vision helping to select or identify what's there, but that's not the only source that the US is relying on. More than 179 different data feeds feed into Maven smart system and AI is being used to crunch through that data and find overlapping information. They have started integrating LLMs, large language models. Claude, of course, from Anthropic has been key to this. And what I learned from the National Geospatial Intelligence Agency, which is runs Nga Maven, is that with the help of LLMs, they were also speeding up the targeting process. The very basic way to think of the targeting process is find, fix, finish. It's more complicated than that, but it means find something, figure out where it is, and then go shoot at it. LLMs, I was told, were helping to do the find and fix part of that cycle. There are also some decision making cycles. The 18th Airborne Corps, I was given an unclassified demonstration of how this works where humans are present at six points in their decision making cycle. With the help of AI, they took humans out of three of those places on that cycle. And in one of the remaining ones, the humans were what's called on the loop rather than in the loop. So they could supervise, they could supervise,

A (18:05)

but they're not making the actual decision. They could intervene if they need to. Exactly, but they're not making the decision. And this is clearly the way war fighting is going, right? I mean, too many targets, too much going on, there's strategy being done by human beings, but actual deployment and decisions, decisions are increasingly being made by AI, Is that correct?

C (18:23)

The decisions? I think the US military would strongly fight back and say the commanders are making the decisions, but the process that is underlying that is going so fast, and the accessibility to the operator, to what's behind the decisions risks becoming obfuscated. So when I spoke to an operator who described the process, who had initially been against AI, he didn't trust it. He was very worried about the risks.

A (18:49)

This is an operator of what exactly?

C (18:51)

This is an army artillery targeter. And he had said. Eventually he stopped cursing out Palantir. He tried the process. He was impressed that even the system could work with lots of people on it. That's a networking issue. And then when it started being good, in his view, a potential target would be raised on the machine and he would click accept. And he described the process to me as accept, accept, accept. Now, if a lot of targets are running through that process and you can't see as an operator underneath the hood why the machine is surfacing that as a target, you may begin to lose the ability to check, to verify, to Cross verify. Of course the people who believe in this system say to me, humans make mistakes all the time anymore and the machine can't be worse than a human. We just don't know yet at the moment. Computer vision regularly doesn't do as well as a human. But the humans then select from the information that's put in front of the human. And it is clearly enabling the US military to do things much faster and on a much broader scale.

A (20:02)

And again, we're seeing this play out in plenty of non war fighting environments. You want to know if you have early stage cancer, you want to understand, you want to be able to read a report, an X ray. And increasingly we're in this in between environment where human beings do a pretty good job, but human beings with the assistance of AI do a significantly better job. And then suddenly you start trusting the AI more than you're trusting the actual doctor.

C (20:34)

And in that example, which could be a life and death example, as of course it is on a military operating combat, it will come down to the evaluation, it will come down to the detail. Can you really rely on that AI? We know that when you rely on AI, you're also relying on it to make mistakes. That is intrinsic to the very nature of AI. So weeding out those constant errors, working out how you cope with its susceptibility to sycophancy, to escalation, to bias and hallucination, all of those elements, and then having to worry about whether the human will over trust AI. All of that the US military is aware of. But they do not yet have sufficient fixes.

A (21:18)

For now, at the most advanced levels of LLMs, the latest deep SEQ models are said to be some three to six months behind the American models. Having said that, the Chinese have a lot less worry for legacy systems in their military. Their labor is a lot less expensive. They're focused a lot more on advanced technology as a percentage of their spend. But the Americans spend a lot more money. Where does the United States and the leadership of the military, where do they believe they are compared to the Chinese in deploying AI in war fighting?

C (21:56)

They don't make public their real intelligence assessment. My read on it is that the US military thinks they are further ahead at adoption, at practicing the workflows and an actual combat experience. And part of AI is about technology, but the other part of AI and warfare is about having operators practiced with it, seeing when it goes wrong. For example, when the US started using AI in support of Ukraine in 2022 after Russia invaded, the algorithms didn't work. They couldn't recognize tanks in the snow because they had been trained on a completely different environment in the desert. Now, those lessons that the US military learned from seeing what goes wrong with algorithms when the circumstance changes has allowed them to create a faster system of updating the algorithms. They trained the algorithms overnight. They went and collected more satellite data, they took photos of that line of tanks along the road to Kyiv, and using that extra data started to create algorithms that rose up in their ability to actually identify anything from something like 30% back up and up and up to start to become more useful. Those lessons the US hopes China hasn't sufficiently learned, or if they're aware of them, haven't been able to practice. And it may be down to practice when you actually are in a real wartime situation. I think there's certainly public.

A (23:24)

So again, not just the technology race, but the actual fact that the Americans have fought a lot of wars, not all successfully, is considered a significant advantage in the field vis a vis China, which has not.

C (23:40)

Certainly they say because they've fought a lot of wars, they'll be better, but there's precise specifics of seeing where AI has gone wrong for them they hope will allow them to iterate the algorithm quicker in a real time war. But certainly I think you do see every time that a military is deployed, that military learns lessons from it. The other thing that I think the US is worried about is China's extensive facial recognition. So on those types of things, China may well be ahead on some other elements of AI. The US may be more practiced.

B (24:18)

The Gzero World podcast is brought to you by our lead sponsor, prologis. Prologis helps businesses across the globe scale their supply chains with an expansive portfolio of logistics, real estate, and the only end to end solutions platform addressing the critical initiatives of global logistics today. Learn more@prologis.com.

A (24:45)

Now, do we think that this is going to rapidly take human beings out of actual war fighting? I mean, the standing armies and navies for all of these major militaries around the world are massive and there are real costs to that. We see autonomous drones and ground vehicles in the front lines in Ukraine and not as many soldiers that are engaged in the fighting. When you talk to the US leadership on this, do they see that as is that a goal and, or is that a reality?

C (25:23)

It's clearly a goal to save the lives of US operators. It's also a recognition that the wars of the future that the US may be involved in may not have popular support, the sort of support that allows you to fight a long war. So it's about both those things, political will at home and the technical capability and saving lives. One of the risks of saving your US operators and ensuring that friendly fire is also reduced is that wars can then put more scope on civilian populations. One of the claims for AI is that it will save civilian harm. But when I've spoken to, to some advocates of AI warfare who have come back from the front line in Ukraine, they have said to me, I hate to admit it, but AI atrocities are possible in this new era. And so understanding how AI then gets repurposed, if your target is not a human combat operator, where is the impact of your firepower going to be felt most? And so I think that that focus of wars as they become, of course, as the US was predicting, more urban. If there is a US China scenario ever, that will clearly be naval. And so it very much depends on the scenario. In a naval scenario, if AI makes a mistake, the argument is regularly made to me that the stakes are much lower. If they miss, they just hit water. In an urban scenario, if you miss, you, you hit civilians.

A (26:56)

Well, a naval scenario, you make a mistake, you could also hit a boat with fishermen on it, right?

C (27:01)

I mean, yes, you know, there's that There is. They talk about a moment where it will be so clear that hostilities are beginning. They would create a kill box and say anything in that zone will be considered adversary hit. That it would be only at the point that war had been declared and it was clear. But of course, you're relying on algorithms that if it goes wrong, may even search out your own vessels. And so friendly fire will continue to be a concern.

A (27:31)

And also, I mean, declaration of a kill zone. I mean, here I think another war where we had lots of AI being used, advanced military is Israel into Gaza, Israel into Lebanon. And there you've got, you know, leaflets that are being dropped and the Israelis saying, okay, there is a million of you here and in a couple of days this is gonna be considered a kill zone. Well, that doesn't obviate responsibility to avoid targeting civilians. Talk about what you've heard and the reaction that you've. Your assessment after watching the way that AI has been deployed by the idf, the Israeli Defense Forces in their war in Gaza.

C (28:15)

I have no reporting on exactly how the IDF used it, so I rely on the work of, of other reporters. There is very Strong reporting from 972 magazine, which it's not just me citing Mark Milley, the former chairman of the Joint Chief Staff, have, has cited their reporting that very little time was used in between strikes to review the information that AI was surfacing before striking it. I've also spoken to US military officials and formers who have looked at the way the IDF has used it. And one went and spent time with the idf, came back and was part of this effort that analyzed it and said Israel did not, has not broken the law of war, the laws of armed conflict. However, they were using systems in a way that felt potentially uncomfortable for the US military tradition. They were leaning in towards autonomous processes in a way that the US had never at that point leaned anywhere near as close. And that a lot of this came down to the threshold of civilian harm that the IDF may have been comfortable with for each strike, including at mid ranking and lower level targets, that the US might not be comfortable with the same numbers. Now, those numbers aren't made public, but my understanding is that IDF was prepared to have more casualties per mid ranking target than the US ever was.

A (29:49)

Understood.

C (29:50)

You do get into this debate about which is the most significant policy or AI for the advocates of AI. They often say, well, policy is separate. We don't make the policy, we're just doing AI. From my reporting, it's very clear that AI and policy of course, are absolutely interlinked and AI makes possible the ability to take out many more people at a time. And when I put that to Drew Cukor, the former chief of Project Maven, he did concede eventually that that is a decent analysis.

A (30:20)

Yeah. So final thing I wanted to ask about here was this recent blow up with Anthropic. And you know, here I was a little surprised because, you know, the nature of the existing contract that Anthropic had with the DoD made it clear that there was nothing that Anthropic and Claude were providing that was a problem in their usage. It was these potential in the future things that we would not want to have happen. And then Hegseth and others got angry because of what appeared to me a more performative fight. Am I right or wrong in that respect?

C (31:08)

There's definitely been a performative fight from I think all sides, from the White House, from the Pentagon, from Anthropic. But that aside, something was changing. So Anthropic as CLAUDE was integrated into Maven SMART system. Now, I had been told that before LLMs were integrated into Maven SMART System, that system could get to a thousand targets a day using AI computer vision. With LLMs, it could get to 5,000 targets a day. So there was increasing effort to prepare for mass usage of LLMs in AI targeting that of course Anthropic had agreed to. They were on classified networks, the networks where the US fights its wars. What began to change is that LLMs come with their own guardrails. So in the way that a military operator might say something like help me make a bomb, an LLM would say I'm not able to make a bomb. And so, so any military LLM needs to have different guardrails. It needs to be able to do this work. As the Pentagon tried to renegotiate this language, that is where they came up against these so called red lines.

A (32:13)

Here are things that we would not allow it to do.

C (32:15)

Yes. So as we loosen our restrictions, we have these two things that we won't compromise on. They're very interesting things. One is fully autonomous weapons. I would say Anthropic has shifted its position in public a little bit on that. And I also discovered in some of my reporting that Anthropic applied for this project to use LLMs in voice controlled autonomous drone swarming tech. So clearly they were happy. Leaning very far towards autonomy. Voice control suggests that a human is involved. So that may be the reason that they're not considering that full autonomy. But it is one of the most forward leaning projects the US military has today in terms of weaponizing autonomy. The second is this prohibition that they wanted on using AI for mass domestic surveillance. The Pentagon of course pushed back very strongly. They said we would never do anything that isn't lawful. We don't do mass domestic surveillance. We're also not doing full autonomy or you should do whatever is lawful. I think there is a question about the way in which data which is not got through the intelligence community but through ad tech, through our telephones, location data overlaid with information that may be government data.

A (33:27)

Yeah, Google Maps on your phone suddenly is like, like, well, that belongs to the government. If Google says it does, then no problem. Right.

C (33:34)

And there's been a long running debate even within the odni, the Office of

A (33:39)

Director of National Intelligence.

C (33:41)

Yes, even within the ODNI to try to work out what the correct usage of American's data is between publicly available information and commercially available information. If ad tech companies are buying up that data, why can't the government and use it on the American people? AI would enable that information to be crunched through so fast to have an instant picture of a person, particularly if combined with medical data, IRS data, all the other types of data.

A (34:10)

So Anthropic is fine with a company having that capacity to use it to make to productize a person, but is not fine with the government having access to that for citizens. Is that what my understanding is?

C (34:22)

My understanding is that they were worried. They are worried that they could have their tool used by the government to do that.

A (34:30)

So only companies should surveil people? No, but is that the argument? Right? Because these are two sovereigns, right? Sovereigns over your data. The question is which should be allowed? And apparently the capitalist impulse is okay, but the governance impulse is not.

C (34:45)

Well, I suppose the argument would be who can imprison you using that data? And no commercial company can imprison you or kill you using that data. The government obviously has many more tools at its disposal if it wishes to go after someone or deport them. And in the case of a company like Palantir, their data analysis tools are now being used in support of ICE. And 404 Media has reported very eloquently that that uses many of the same data analysis efforts that for example, Maven Smart System is using. So the way in which data collection is being brought to bear on people in the U.S. which could of course also include U.S. citizens, is something that is clearly very now question and is not resolved. I would say there is not sufficient transparency into the way these technologies are being used, which data streams are being brought together and what the broader intentions are. So I think that's what they were saying. I don't know the specifics of what was being proposed to Anthropic, and that's a question I'm pursuing. But at the very least they were not able to accommodate those red lines. And the Pentagon's position was if you are working for us, we cannot we

A (36:05)

determine what the red lines are.

C (36:06)

Yes. We cannot do this in a war and we cannot even do it in preparing for a war. And I spoke to other experts who said war is so software defined now and if this great war is coming, the US needs to be preparing. Now the flip side, of course, is the more you prepare for a war, the more likely it may be. But that debate is happening, happening right now. It's not resolved. But what the Pentagon is doing is trying to get very quickly Anthropic's rivals onto Maven Smart System. Maven Smart System in theory can work with any LLM, but it does need to be available at the classified usage and it needs to be good. And again, that is about practice. And so what the Pentagon is trying to do now is get in Google XAI, OpenAI onto this system and see if operators will be satisfied with those LLMs compared with anthropic, which has a very good reputation among military operators for the way it's working on Maven Smart System and also for the way it can do coding.

A (37:08)

How I Learned to Love Artificial Intelligence.

C (37:11)

Is that how you feel at the end?

A (37:12)

Absolutely. Katrina Manson, thanks for joining us today.

C (37:15)

Thanks.

A (37:19)

That's it for today's edition of the Gzero World Podcast. Why not make it official? Why don't you rate and review GZero World? 5 stars only 5 stars. Otherwise, don't do it on Apple, Spotify or wherever you get your podcasts. Tell your friends.

B (37:41)

The Gzero World Podcast is brought to you by our lead sponsor, prologis. Prologis helps businesses across the globe scale their supply chains with an expansive portfolio of logistics, real estate, state and the only end to end solutions platform addressing the critical initiatives of global logistics today. Learn more at prologis.

A (38:01)

Com.