Loading summary
Transcript35 lines
- [00:00]
Indeed Sponsor
This episode is sponsored by indeed. You just realized your business needed to hire someone yesterday. How can you find amazing candidates fast? Easy. Just use Indeed when it comes to hiring, Indeed is all you need. Stop struggling to get your job post seen on other job sites. Indeed Sponsored Jobs helps you stand out and hire fast. With Sponsored Jobs, your post jumps to the top of the page for your relevant candidates so you can reach the people you want faster and it makes a huge difference. According to Indeed data, Sponsored Jobs posted directly on indeed have 45% more applications than non sponsored jobs. Sometimes the hardest part of hiring is just working out where to start. Indeed makes it clear, simple and quick to reach the right talent. Plus, with Indeed Sponsored Jobs, there are no monthly subscriptions, no long term contracts, and you only pay for results. How fast is Indeed in the minute I've been talking to you. 23 hires were made on Indeed according to Indeed Data worldwide. There's no need to wait any longer. Speed up your hiring right now with Indeed and listeners to this show will get a $75 sponsored job credit to get your jobs more visibility at indeed.com intelligencesquared just go to indeed.com intelligencesquared right now and support our show by saying you heard about Indeed on this podcast. Indeed.com intelligencesquared terms and conditions apply. Hiring Indeed is all you need.
- [01:45]
State Farm Sponsor
This episode is brought to you by State Farm. Checking off the boxes on your to do list is a great feeling. And when it comes to checking off coverage, a State Farm agent can help you choose an option that's right for you. Whether you prefer talking in person, on the phone or using the award winning app, it's nice knowing you have help finding coverage that best fits your needs. Like a good neighbor, State Farm is there.
- [02:11]
Mia Sorrenti
Welcome to Intelligence Squared, where great minds meet. I'm producer Mia Sorrenti. In today's episode, host Adam McCauley sits down with Professor Antony King to shed light on AI and the rise of the military tech complex. King is a professor of War Studies and Director of the Strategy and Security Institute at the University of Exeter. He wants to debunk the science fiction tinged narrative of AI's military potential, arguing that AI will not replace human strategic judgment in war. Instead, he takes a more grounded approach exploring the actual applications of AI by the armed forces over the last decade. Let's join our host Adam McCauley now with more.
- [02:54]
Adam McCauley
Welcome to Intelligence Squared. I'm your host Adam McCauley. The latest advanced technologies seem to emerge as solutions to near impossible human problems. These technologies or Technological fixes tend to bubble up at the sharp frontiers of knowledge, often precisely where uncertainty about the future casts its shadow over the present. In recent years, the story of artificial intelligence has featured narratives around the coming singularity, those first warnings of an emergent superintelligence, and in our present age of mounting geopolitical tensions as a technology that might revolutionize, automate and mechanize how we fight war itself without relying on the trope of Skynet or feeding the furor around Terminator style warbots. Our guest today, Anthony King, leans on evidence, investigation and firsthand testimony to better understand how AI is influencing some of our most critical institutions, notably those of the military and national defense. King is professor of War Studies and Director of the Strategy and Security Institute at the University of Exeter. He is the author of the 21st Century General and Urban Warfare in the 21st Century. And I'm so pleased he could be with us today to discuss his latest book, AI Automation and War the Rise of the Military Tech Complex. Welcome to Intelligence Squared, Anthony.
- [04:21]
Professor Antony King
Oh, thanks for having me. It's a pleasure to be here.
- [04:24]
Adam McCauley
So I hope we can start with the basics, at least briefly. Can you explain what artificial intelligence means to you and what it can do in the present moment, just to situate sort of expectations?
- [04:35]
Professor Antony King
Well, it's a strangely difficult question because everyone uses the term artificial intelligence, but strangely difficult. So artificial intelligence is a. A diverse field of computer programming that involves various types of function. What I would summarize it as is a form of computer programming able to process large amounts of data to produce useful but not entirely predicted results from its. From its programming. So it exceeds the actual initial inputting and initial human programming to produce results that are actively useful and unexpected. Now, they're a little bit on intelligence. So computer scientists will say artificial intelligence, the intelligence part is anything, a computer generating anything that we might impute or describe as a human intelligence having certain form of intelligence. I prefer a definition that focuses more on the data processing side, but what I would use is a functional, untechnical, commonsensical definition of the term.
- [05:56]
Adam McCauley
I think what's so interesting today, and I think so timely about your book, is that it can be read almost as a disclaimer perhaps against some of the common claims or assumptions that some maybe of our loudest proponents of AI are often caught championing. And so you write that much of the conjecture around AI's potential, and again, AI as you understand it, is a pretty wide space in which we're operating here, but the conjecture around AI's potential to change, and in your case, to change warfare, which is what this book is very much about, often projects a vision of the future based on some understanding or maybe we should say some imagining of what AI truly is today. And so when it comes to this intersection, given your background and your knowledge, maybe you could give us a little bit on how you sort of came to this topic itself, but also give us a sense of the debates or the lines of argument being advanced today around what AI seeks to do or could possibly do, or might potentially do to the world of warfare itself.
- [07:02]
Professor Antony King
Well, thanks for that, Adam. And you put your finger on some of the key things. So, you know, I've described, given a definition of artificial intelligence. But of course, the key thing, if we look at contemporary debates and we could extend these back 10, 15, 20 years in some intensity, and of course they go back to the 50s in a looser sense, is the notion that on the basis of that automatic processing of massive data computer programs, we are able to generate results that humans can't see and can't develop. And eventually AI will develop general intelligence. I be able to apply its programming across a wide range of activities to basically outperform humans until it reaches a level of superintelligence. Now, that kind of argument has filtered into security and strategic studies very strongly. So the essential argument is this, that if artificial intelligence is developing so quickly, if it's exceeding human intelligence so quickly that it is moving towards the bounds of general intelligence already and superintelligence is only five, ten years away, well, what is the implication for warfare? Well, the implication for warfare must be that a entity, a program that knows more than a human commander in general, that can process that knowledge and action, that knowledge more quickly, more accurately, will eventually displace human commanders, human leaders, human generals, human commanders in chief to prosecute war, to decide when, where, how to fight wars and who to kill on its own. In short, the argument across a lot of literature is we are on the cusp of some significant automation of strategy and warfare itself. On the basis one of this belief in the teleological development of AI and its application, and essentially the work that I've just published, the book I've just published, it's a refutation, a skeptical, hopefully not a refutation negative way, but a recalibration of the debates about AI strategy and warfare. And what I seeking to do is in place of a teleological belief that, look, artificial general intelligence is about to happen, artificial superintelligence is about to happen, and therefore we are already on the road to that endpoint and every piece of evidence points to it. Instead of that, I just wanted to take that endpoint away and look at the evidence itself and provide a skeptical view where we looked realistically at what the functions are and will be in the short term and how they're going to apply. And my central argument was, look, AI is already a very significant factor in strategy and warfare, but it's miles away from the vision of automating war, war and strategy.
- [10:16]
Adam McCauley
I think what your book does so well, and you've just spoken to this, is that it does reframe sort of our present moment and offers us at least an opportunity to glance at the evidence or to take a look at the evidence as to how these systems are actually employed, what they seek to achieve, what they can achieve in our present moment. And then to be a little bit more dubious, perhaps instead of listening to the hype often offered from those who create the systems, but actually think about how these integrate with systems, some of these long standing institutions, particularly in the military and defense space, where the stakes are so high and the questions we might want to ask might be fundamentally different from those that we would find in our everyday lives. And so one of the things that I think is useful within the book and maybe for our conversation right now, is that in the context of generative AI, which for most listeners and viewers will be the one they're most familiar with, you write that this is inherently an inductive system. It's constitutionally brittle and. And so the smallest bias or the gap in the data set can generate incredible errors in subsequent performance. And these details, I think, to your last point, speak to perhaps a challenge we would then have as to imagining these systems in a position of command control, or as something that sort of automates something essential, particularly around the question of violence or otherwise. And so I wonder if you could just tease out what these hard obstacles are at the present moment for how these tools actually engage with some of these spaces and give us a sense of what that delta or gap looks like.
- [11:49]
Professor Antony King
Sure. Well, thanks again for that and thanks for that very insightful question. Yeah, so I mean, as many people know, second generation AI, including generative AI, including large language models, operate on a probabilistic inductive system. That is, they process large amounts of data, I mean, when I say large, absolutely vast amounts of data, to identify mathematical statistical correlations in that data. When the data is good enough, when it's accurate enough, when it's big enough, those inductive processing produces correlations which tend towards nearly 99% accuracy. Basically they're 100% accurate in terms of identifying correlation. That is unbelievably useful and powerful. And it's been shown in various fields from Alphago, plan, goal, go to medical research, to looking at cancer screening, that where you have predictable, established good data, AI, artificial general intelligence and machine learning can produce results which frankly exceed humans. The issue is, is that how strategy works, Is that how armed forces make decisions? Is that how decisions in war go on? And there is no doubt there are some decisions that are made in war that are inductive, that are based on correlations between existing evidence, on the base of which in the past humans, I.e. commanders in their headquarters, their staff officers made inductions from that, from that basis. But the notion that Straffordian war is reducible to pure statistical probability, it's simply false. So one of the critical things that any armed forces, any military commander, any general needs to do before the start of an operation and during an operation is to define what they're actually doing. What's the purpose of this operation? Why are we fighting in Gaza, why are we fighting in Ukraine? What are we trying to achieve? Those are constitutive of the action of the war itself. They're definitional, they're not an answer that second generation, even the most sophisticated generative AI model can give to a human group. It is, it's the humans and the collective, their political enterprise that determines what those decision making decisions are. Now there's loads of other practices in war which also in terms of the manipulation, the coordination of warfare, where there's two things that exceed AI, namely there's lots of decisions we need to make for which there's no real data, as in binary digital information processable by a computer. There's lots of evidence, but there's not data. And there's other elements to warfare which involve moral judgments, ethical judgments, leadership, which actually influence the way we define the situation and define the mission, define the operation, define what we're trying to do, which can't be reducible to AI as it's currently constituted or conceivably constituted on its current basis in inductive statistical systems. So what I'd suggest is the powers of AI. You know, every single person needs to be aware of these powers, but in terms of their application to warfare, it's complicated. Now then the next question comes is so what can you apply that system to? And here the application is clear and very potent and for me it's not the entire sphere of warfare being automated. It's the automation of data processing for intelligence purposes and situational awareness purposes. And this then distills into three key functions. Firstly, AI can be used and has been used to help planning, to help logistics. Secondly, and most controversially but most successfully, it's been used to target. We've seen that in Gaza, we've seen that against ISIS in terms of U.S. models. And finally, it's used for cyber operations, cyber defense, offensive cyber operations, and also information operations, where AI, sometimes in very simple forms like bots, is able to generate content, misinformation, illusion and lies, frankly propaganda, psychological material. So what it suggest is it's very easy to make hyperbolic jumps of what the capabilities are to assume a complete colonization of what war involves. It's important to avoid that, but also to recognize these key areas where AI has made a difference and, and I would anticipate as models become better and more accurate, they will become more effective in those sub functions.
- [17:18]
Adam McCauley
What you do so well, I think, in this book, is you speak to the nuance, as you've just laid this out, right? You don't discount the importance of technology, but you do make key distinctions as to where we might see its effects. And I think that's incredibly helpful for the reader. And one of the things I did want to sort of hold you on, as it were, given your last answer, is that you highlight in the book that both Gaza and the conflict in Ukraine might very well, in your words, be a turning point in the history of war. They mark the moment when AI first became essentially indispensable for military operations. And I'd love for you to, I think along those three lines, and maybe not so much the cyber line itself in this first cut, but, you know, I was struck by the microworld experience or example you use from The British Army 3rd Division, for instance, and then maybe if you could speak a little about AI targeting in Gaza and what that's enabled. I think these are two sort of tangible empirical spaces where we can understand what AI has really moved the needle on in some important way.
- [18:23]
Professor Antony King
Yeah, so thanks for that. Yeah, I mean, so, you know, I might say, yeah, they, you know, Gaza, Ukraine represents changes, maybe not radical revolutions, but they're important moments where I think the application of live data processed by certain forms of AI, sometimes quite simple forms of machine learning, but certain forms of AI have become routinely used by the military, and I think that is very important. As I say, is it a revolution? I'm skeptical. I'm always skeptical about revolutions, but it's a transformation now in terms of applications. So let's just talk to two examples, concrete ones that you've referred to. And one of them is very simple. It's a very small example where the British army following Covid, it started about 2019, but really post Covid began to want to explore whether they could apply some form of artificial intelligence. In this case, it was quite a simple form of machine learning to the issue of planning and in this case to the issue of route planning. Which routes can you actually move an operation along? And it sounds a totally mundane. It is a totally mundane thing, but in the past it's taken staff officers a long time to plan routes. But it was an example that the British army wanted to explore, to see the potential. Because here you had a simple bounded example in which there was very significant data. People knew the topography on which roads were the mapping was possible. It was possible to work out observation where the roads could be observed from anyway. So an institution in the British Army Enhanced Command and Control Spearhead developed a software program in a hardened. Eventually the hardware of it was in a hardened sort of box that could be movable, which essentially allowed for route planning, automation of route planning and. And within what would take hours for a staff officer and a team of staff officers took vet. She took seconds for this program to. To automate. Essentially it was a military form of Waze or Google Maps and it worked very effectively. It was a small and the army was completely realistic about this. This wasn't the final application of AI to the issue of planning. This was a controlled experiment for what the British army could do. But empirically, for me, it was a really important case. Why? Because the argument about the application of AI goes very big on rhetoric of the ease of application without specifying the problems. When we actually go to talk to practitioners, the identification of the problem to which you can apply AI is one of the key problems of applying AI. And what was very interesting to me in this case is here was a case where the British Army TR much they're much further behind than the US army tried to identify a case study, a case example for the application and successfully within about eight months executed and developed a program that seemed to be a very useful model, not just for the British army, but me as a scholar of what it will look like to apply AI and just repeat, just to say you need a clear problem, you need a clear bounded set of data that you can apply to that problem and you need effectively seeking a solution that you know you've got that solution when the AI generates and you know when it's wrong. So this was a really clear case, and there's no doubt this was a very simple case. There's others, more confidential, more classified, that have already been in place. Western armies generally, the US Army, US Forces generally are way ahead and are applying much live data and AI to these problems, to logistic problems much more widely. But that case study shows the procedure, the targeting issue is critical. And the targeting obviously is critical because at the end of targeting are people getting killed. And so obviously it's a critical military issue. And it's a critical issue that obviously lawyers and ethicists have been concerned about, because that's where nobody particularly minds about AI organizing a column of tanks moving or column of vehicles moving up a road. But when we're talking about the delivery of lethal ordinance into a civilian building, at that point questions become asked and we. There's two obvious examples, project maven, the US example, but the famous one now is lavender. The US sorry, the IDF's system where they've collected huge amounts of data on Hamas in particular, but other Palestinian terrorist groups based on mobile phone data, sensor data, presumably satellite images, probably digitalized human source images, to basically create a huge data archive of the Hamas operators. Hamas structures, Hamas GEO locations. And on the result, on the base of this programming Lavender, the IDF were able to generate, well, the figure that is in the public domain is 37,000 Hamas targets within those first couple of months of the war. So October, November, December of 2023. So once again, what you see here is a clear problem. The IDF wanted to target Hamas, a clear set of data, they could use it, and a clear output that they wanted. Now, we can certainly talk about the ethics and legality of this, but the process seems to me to be an extremely useful one in terms of understanding the application. As I say, there's got to be a bounded problem. There's got to be data to which you can apply that problem before you can begin to develop these kind of programs. I'd say one other thing about both things, which I'm sure we want to come onto. People have suggested that Lavender is responsible for although, you know, large numbers of civilian deaths in Gaza somehow automated the war itself. I totally disagree with that interpretation. Lavender generated targets that Israel could prosecute. Israel decided to prosecute those targets because they had decided on a strategy in which collateral damage and civilian casualties were allowable. They changed their strategy in Charlie from previous interventions. Note, this comes back to my previous point. Lavender had not defined the Gaza War hadn't defined the operation. Netanyahu, his cabinet senior generals in the IDF defined what that operation was. Lavender came in as an effective targeting system to generate targets for them. And we might do a thought experiment here. What would have happened if IDF had not had Lavender? They in my view, would have bombed Gaza very heavily on the basis of poor intelligence. And so the results might have been worse or, or about the same. So the idea that Lavender had agency in this process, it had capability, the idea it had moral, legal, military agency, decision making agency, I don't agree with. It's a very serious system, but it's a system like a weapons system or a satellite system.
- [26:33]
Adam McCauley
It is a critical sort of intervention you make here. And I think the book makes a strong and acute argument around technology's ability to shape organizations as much as anything else, particularly within the military. And so this targeting example sort of gets us into that territory. And I will circle back towards the end with some questions around kind of the cultural impacts of technology in these spaces and what it means, but this collision between tech and institution. I think one of the other examples you tease out is the American joint all domain command and control concept, JADC2 and you write here, and this is, I think quoting American sources, that it uses artificial intelligence and machine learning to help accelerate a commander's decision cycle, automatic machine to machine transactions, extract, consolidate and process massive amounts of data and information directly from sensing infrastructure. So JADC2 here is an example of an AI system that produces almost an infrastructural impact on human organizations. Right? It's IT conditions in some ways the decision environment in critical ways. And I just wondered if you could talk a little bit about the dynamics that you see in the organizational sense here. What are these changes? What kind of pressures and impacts does. I don't want to call it automating, but certainly programming a kind of understanding of the world. What does that mean for that command and control space? A space that I think, you know well, given your own background studying it, what changes in that decision environment?
- [28:00]
Professor Antony King
So yeah, I mean this is a completely critical point. So if I could just sort of answer this question slightly with a slight oblique sort of biography, autobiography. So when I first started the work on AI, I was seriously concerned as a sociologist because on my reading, if what various scholars were arguing about AI, namely that it could automate strategy, the decision to go to war, how to make war, if it had actual agency in a genuine human sense. Well, I'm a sociologist. Essentially I'm out of business because Then the issue of agency moves into computer science. To me, it's not about social agency, it's not about humans, it's about programs. And, and I actually was seriously worried. What became clear as I did this work, which I, you know, it's, you know, I see this project as the beginning of a journey for me, which I found fascinating is exactly this. The organizational effects of AI and more, I might put it the other way around. The organizational transformations that need to go on in order that you can get AI doing things like targeting and planning for you, or with the joint or command control system. Of course, this is an aspirational system, a full multi domain, situational fairness system for all of US forces. This is aspirational. I think it will be very creaky, actually in the end. But the key point is what are, what are the organizational requirements in order that a military force can apply live data, mass data, and AI to process that data to get results that it wants? And here I must say I found the whole transformation that we've seen over the last 10 years, we could stretch out to 15, possibly 20. But really we're looking at a very recent development. Phi 10 is a very strange partnership with Silicon Valley and the tech sector, and especially with tech defense companies. The armed forces, powerful that they are, have no experience of data at this scale, computer programming. They don't have the human expertise in terms of computer scientists, data engineers. They don't have the computing power in terms of hyperscale centers. So they are completely reliant on the tech sector and to affect tech companies and tech defense companies a way that they've not been in the past. Now, of course, they've been reliant on industrial companies, arms manufacturers to produce materials for them. Yes, but arms manufacturers make the weapons, deliver it to them. The armed forces decide how to use them. The very odd thing about artificial intelligence is that in order to utilize it and operationalize it, there needs to be a deep integrated relationship between the armed force and the tech sector so that the software and data is developed together and that the tech companies understand the immediate problem that the militaries are facing. And then once they developed AI programs, machine learning programs, generative AI models, even those need to be tweaked in an operational environment. So what you see is something that I think organizationally is utterly fascinating, but also very important and actually potentially quite dangerous politically. And that is a fusion of the military and the tech, a deep partnership of the tech sector and the military, and a deep partnership that goes into military operations. Themselves. So for instance, in terms of targeting in Ukraine, companies like Palantir have been involved in those targeting systems. Elon Musk's Starlink was totally critical to Ukrainian military effort. It also gave him a nefarious, notorious influence over the campaign in 2022, 2023, when in fact in September 2023, he shut the system off because he wasn't happy about Ukrainian plans to attack Sevastopol. So what you see, that's an extreme pathological case. But what you see is a configuration, a civil military configuration, a private sector, public sector configuration, which is highly unusual. It's novel historically, there are parallels that go back to the early modern period, but it really is very different from 20th century military industrial relations. And obviously, as a sociologist, I found that both fascinating and quite comforting. Though of course there's a political bit that is not comfortable, that's quite concerning. But for me, the issue of AI as the most radical technology we've seen for a while is paradoxically, the key issue is organizational. We should focus on the organization of AI, not on AI as automation.
- [33:43]
Adam McCauley
What's fascinating about this discussion, I guess, of the fusion or this new marriage of necessity, as it were, between the military and the technological sector is in some ways a market response. You tease out in the book that the type of talent required to both innovate in and then maintain these systems is just rewarded in the marketplace at such a level that it's impossible to sort of seed those same individuals back into military or public institutions in some enduring way. And so part of this question becomes, is this just a correction that will have to be sort of smoothed out over time? Are we seeing sort of a bubble in this tech sector which eventually can't hold and there's going to be an onboarding, as it were, of talent back into the public sector? But I'm just curious to know from your time in this space, is this a persistent incompatibility in terms of how we right size talent across these spaces, or is this very much a purposeful arrangement or construct that the tech industry is more than happy to support through high investment in R and D and everything else to make this dependent seem more or less enduring?
- [34:55]
Professor Antony King
Yeah, it's great. It's a great issue, great question. So, and there you are at some point asking me to slightly predict the future and project and as I said.
- [35:06]
Adam McCauley
Very awkwardly so, you are welcome for this.
- [35:09]
Professor Antony King
Yeah, but it's, but it's such a great question. Let's try and let's try and think about it. So my position on this would be this. You're absolutely right in that what we see is market concentration and actually a monopolization of certain bits, key bits of capital, the compute power, the data, the people, into very isolated monopolizing sectors of the. And it is, in the case of the west, is the US economy, Silicon Valley, and a handful, the tech primes. And then in the defense sector, there's a few others, Palantir, Alan Drill, who've come to be important players. So absolutely, we have a sort of organic, unplanned concentration of economic and human and computing capital in a set of institutions locked together actually in a tight geographical area in Silicon Valley, essentially. I mean, okay, Amazon's in Seattle, but, you know, apart from, you know, it's a tight, tight thing. Do I see a rebalancing? No, I don't. I think one of the most important things, as I say, and this goes back to my organization point, is that one of the most interesting, important things about digitalization and AI is the political, economic, organizational transformation, which speaks to me to a profound reconfiguration of Western politics, economy and society around the concentrations of these extraordinarily powerful and rich companies. And I just don't see how that capital, that extraordinary concentration of capital, is going to be prized from the hands of those companies and the supporting tech sector and democratized and fragmented out, or that it's going to be nationalized and regulated in the West. Now in China, it's a different question because the links between the tech sector and government, there's almost no boundaries, but in the west, and if anything, I see a consolidation of this process, we look at the whole emergence of OpenAI, its ability to influence regulation. I mean, one of the interesting things about over the last two years been interesting is how much discussion there's been about the requirement for the regulation of AI and a large number of people who are saying, oh, AI is dangerous. We're on the cusp of artificial general intelligence. Artificial superintelligence is key figures in the tech sector. And you look at their actual political intervention and what are they asking for? Here's my view. They're asking for regulation of AI because they think general AI is going to appear. And on the threat of the existential risk of general AI and artificial superintelligence, what they're doing is entrenching their monopoly around Silicon Valley. And I read a lot of their statements, just they're terrified that a small tech startup is going to generate an AI program that actually does what they can do with Small amounts of data, less computing power. So that is a slightly polemical set of statements I just made. But why do I say them? To emphasize that I really don't see a rebalancing of the sector. On the contrary, I think that the monopolies that Silicon Valley have had since the kind of digital revolution from the 80s, 90s and accelerating, I just don't see how you can, how that is going to be reversed. And indeed the evidence the last 20 years is that even universities can't hold onto the top people. Geoffrey Hinton, Yann Lecun are all going over to the tech sector to become their AI gurus and chiefs over there. So the actual leeching is the other way out of the public sector into the private sector. And for me the issue will be how will states and governments, how will military forces negotiate and create an agreement and a partnership with those companies that are so powerful and whose functions are totally indispensable to military operations today and still preserving some Democratic Republican hold over them? I think that is a key issue. And of course the problem here is that China doesn't even begin to have to answer this question. They have a huge, not AI advantage. They have a huge organizational advantage which may manifest itself in an AI advantage when it comes to any hopefully avoidable future conflict. So that's. So my, my message on that unfortunately is quite negative and pessimistic, I'm afraid.
- [40:24]
Adam McCauley
Even the evidence you provide in the book, I suggest that there are just two different games or a bunch of different games being played at the same time. You know, I reference the 2023 numbers you show here. 5 Tech Primes invested 4. $45.3 billion in research and development, whereas the Pentagon is sort of coming in at a measly $1.1 billion of investment in AI research and development. So there is this sense in which those strong or those largest entities are keen to stay ahead of everybody else. And I think to your point too, it's probably not impossible that what we're seeing in the regulatory space is more or less kind of fence building around those who have already made it to the greenest pasture and don't want to see anybody else sort of hone in on their supply. But I think you have. What's fascinating too is you've kind of got a two level game going on in the international and domestic space at the same time. So you reference 2018 as potentially a tipping point in the strategic environment, as it were, where emerging tech really starts to bolt itself into the story of national security in critical ways. And it might be a moment where they align themselves maybe against a traditional understanding of a libertarian ideology and very much in line in the American context with US national interests. I also reference here Palantir CEO Alex Karp's new book, the Technological Republic, which is, if anything, a manifesto around the nationalization of technology in some essential way. What this speaks to, and I think you've teased on part of this answer already, but we've got the Beijing sort of model here or China's model on civil military fusion, and there seems to be at least a number of the tech entities who are maybe advocating for that same understanding that what's good for us in a business sense is what's good for the country. And can't we write an enduring check as a result? I wonder if you could speak a little bit about those dynamics and how they play out or how they supercharged treat AI's influence in this military sort of complex as you understand it.
- [42:39]
Professor Antony King
Yeah. So thanks for that again. Yeah. So I found this process of the realignment of Silicon Valley, especially, I think as a British citizen, I found it extremely interesting that the fact is, from its origins in the late 50s and especially in the 70s and 80s, and despite its dependence on the Pentagon at that particular time, for quite a bit of its funding, Silicon Valley have always distinguished itself from east coast industries, especially from defense industries. And that was especially from 2000 onwards, it saw itself as a global libertarian order, which is the absolute pinnacle of the kinds of things that President Trump absolutely hates. Absolutely borderless globalization, in which markets and money is everything and states and people don't count and all that mass is enrichment through borderless markets. And I mean, that undoubtedly influenced that first wave of digitalization. The great companies, you know, especially Microsoft and then and similar Microsoft, Facebook, the var, Google, the various other key primes. What was very interesting to me, of course, is the this and you talk there this period, 2017, 2018, and the protest at Google in March 2018 where Google has got the contract for Project Maven, the digitalized machine learning targeting project from the Pentagon, where the idea was that an AI model, a program, a machine learning program, will be taught to recognize digital signatures in digital video footage. Result, it was successful. Google got the contract. And there was a protest in Google in 2018, which then Google relinquished the project, relinquished the contract. And at the time and subsequently, people taken this as evidence that Silicon Valley is not trustable. It's a global organization, it's a globally orientated libertarian organization. But to Me, that was the last moment of that period of genuine global libertarianism in Silicon Valley and thereafter, 2018 onwards, and actually with increasing force, especially now with the election of President Trump. Silicon Valley has aligned itself very closely with US national interests. I mean, this is a general move in the US generally on both sides, both Democrat and Republican sides of the House and the spectrum, with US national interests in general against China and with their security interests. And so what I'd suggest is what we see is Silicon Valley still is going to maintain its economic, financial, business, intellectual independence in a way that Chinese companies won't and never have enjoyed. But it seems to me clear that they have aligned themselves with the political establishment, especially with the DoD. And that will continue and deepen. And it will continue and deepen because look at the geostrategic situation of increased competition with China especially, and increased economic and military and political tension. So for me, I see a deepening of that, of that alliance. But it won't be a Chinese solution, as you rightly say. In my view, it will be a Western Republican model of governmental, commercial, military, civil cooperation and integration, rather than any sort of incorporation and direction of, of Silicon Valley by government. And here's the thing, maybe that will mean that Silicon Valley retains its more creative, innovative, it's famously innovative ethos that by having that it's working for and with government, for and with the Pentagon, but as an independent entity. And maybe that will give them some competitive advantages. There's disadvantages of the American model in comparison with the Chinese on other things, but maybe that will give them some competitive advantage. But that's the way I would envisage it. And you cite Alex Karp. I think he's a key figure in this reconfiguration. Eric Schmidt, absolutely critical figure in negotiating this alliance between emergent tech and military and security needs for the U.S. other figures, of course, is Elon Musk. Yes, for sure, Peter Thiel, and actually, I'm Sure Sam Altman, OpenAI will be incorporated very soon into that kind of pantheon of important kind of entrepreneurs. Switch points between the two spheres, I think.
- [48:10]
Adam McCauley
Testament to the fact that you sort of identified this book as the beginning of this conversation in some sense, too. The big question that hangs over certainly my head, is the balance of influence in these spaces moving forward, what that looks like. I could probably spend the rest of the day talking to you on that vein, and I'll avoid it largely because I know we've got an audience listening as well. I'm going to take us back to two Questions to close us out. The first one is maybe a little bit cute or a little bit nuanced, but I do want to get your opinion on it because I do think it's important as we think about the private sector or these tech primes essentially creating the data architectures that inform militaries and then the decisions that they make. I think a lot about this in terms of kind of like a focal or optical power palantir terraforms, how governments collect, sort and organize their data. And then they offer tools to allow them to look through these layers at the operational space. And the idea here is that that serves as the lens through which those modern states navigate these operational environments. There seems to be an implicit argument here that by processing greater amounts of information we get greater clarity. But I also wonder if it also flattens complexity. Right. It fixes decision makers within shared operational bubbles. And that has implications for how they understand risk. It has implications for how they manage doubt or uncertainty, the extent to which they understand certainty and uncertainty. And whether or not this is a space where you creatively engage, where you're humble about what you can possibly know and hedge in different ways. I just wonder, given the time that you've spent thinking about command and control, what are some of the concerns that you have about this heavily datafied, technologically sort of enveloped space? Do we lose something uniquely human that matters matters in really critical ways?
- [50:08]
Professor Antony King
This is a lovely question and your description, I think is a really profound description of the way the influence of tech into the pre influencing understanding and decision making by structuring data and what's presented. I think I completely agree with you and think it's highly lucid and profound. Absolutely. I say first of all, I say two things. One, note the minute we start to talk like that, which is the way to talk in my view, we're not talking about automation, we're talking about human teams of experts and entrepreneurs fixing maps that then the military used to make decisions. And the question is, are the maps they're making accurate or are they just illusions on which the basis of terrible decisions will be made? In terms of the decision making, which is the next topic, I really want to do some, you know, this, I see this as a initial piece of work and I want to continue to work on this. And the issue of decision making under AI is for me the crux of the whole matter and one which I really want to invest some, some time in. So what would I say here? First of all, it absolutely is true that companies like Palantir, Curating military data, governmental data are making the maps and therefore pre influencing, pre structuring decision making. Yeah, so that's true, but I'd say a couple of caveats. The decision making, military decision making strategy, military operations involve decisions that go well beyond those curated data archives, those maps and networks and results that programs like Palantir's Gotham or Metaconstellation are producing, that they are sub parts of decision making. It doesn't just colonize the entire practice of command. What you've got is effectively a map that might influence certain elements of the design of a campaign and certain sub elements of decision making. So it's not in my view, as simple as a complete colonization. Secondly, so, you know, the influence I think would be more limited, real, but more limited. Second point, is this what your discussion points towards? And this actually goes back to a point that I kind of pointed to. But let me bring it out more fully. There is a concern that there'll be about something called the automation bias, that Palantir, for instance, will generate a program that gives pretty plausible results in a crisis. The minute you go right, let's just do that, that's the result. Let's just follow that. In other words, it's not that there's some kind of systematic AI power that takes over, it's just in a crisis, in a difficult situation where people don't have enough information, they grab for the nearest thing and they start to use and rely on AI in a way they shouldn't. And I think that's possible. In other words, I think it's possible that the application of AI that should be, in a small bit, so you've got a little bit of help on decision making in one area starts to be taken more widely and to direct decisions more widely. I think that's possible. But let me emphasize a point. Even in that case, it's not AI taking over, it's unseen. It's hidden tech labor and expertise being presumed to be correct. We shouldn't fetishize the AI into thinking it's taking over. Those decision makers have made a either implicit or implicit decision to collectively agree just to defer to the AI result. So what I suggest, I mean, I think what you're saying is absolutely right and I think that's happened already and will continue to happen. It presents challenges, presents challenges of accuracy. It presents challenges of, in inverted commas, automation. But I think the actual process will be more complicated and I think, as I say, it will be more complicated because even the best AI programs will not have obvious Answers to everything. Their application will be much tighter. I mean, as I say, many AI programs, I would equate them with a map from the early modern period, from 19th, 20th century, where you can't do a military operation with a map without the map, and it allows you to do different kinds of military operations. But can you explain Napoleon's UL campaign or his Austerlitz campaign, just by the existence of a map produced by the French government, by a branch of the French government? No. The map is indispensable, and it does structure the way that Napoleon and Berthier and his staff thought about things. But it didn't determine that they could do a march that they did in 1805. And that's how I think it's better way to think about that curation. It's not deterministic. It has an influence. We need to as scholars. And actually, if we're politically aware, we need to be aware of that inference and think about it. But it's not one of simple determination.
- [55:56]
Adam McCauley
With so much in this book, I mean, we come back to this enduring, I think, conclusion one that we should take away, which is the messiness of sort of the human intervenes in this AI and computational space in really critical ways. And I think I want to push us there as a final question. So I think a lot here about Neil Postman's work in Technopoly, for instance, a book that I think is prescient in many ways still. And he argues that a new technology does not add or subtract anything, but it changes everything. And what he's trying to suggest here is that there's an ecosystem at play in this delicate balance in social relations or otherwise, or the way in which we comport ourselves to the world that we live. That is changed when we introduce something like a map or a new technology into this domain. And so when I think about this in terms of working on military affairs or otherwise, I do think about it in terms of military culture. And so I want to take us back to the IDF experience or example for a moment, if only to suggest that there seems to be a conversation, and a conversation sort of in an academic sense, between the technology and what the command itself or actors within that command are navigating at that same time. So the IDF essentially changes its risk appetite. It may be changing its risk appetite because it has an incentive to go after Hamas in a very particular way after October 7, for instance. Or it may also be responding to the fact that it now has the technological affordance to go after Hamas in A fundamentally different way. And so I just wonder, as we hit this very awkward and complicated space where we have to navigate both the morality and the ethics of using these tools in different ways, how you think about this environment, how you think about asking, you know, better, more complex but more important questions, as it were, about how these systems interact or how perhaps technology collides with human institutions and military institutions in different way, different ways. I just wonder if there is a concern that we're slowly sliding towards a type of enveloped technological life that makes us far more willing to lean into the automation bias in some sense and just use the tools that seem to give us solutions to some of our hardest challenges. I just want to get your sense on how this changes us culturally in some sense.
- [58:23]
Professor Antony King
Yeah. So we finish on just a small question then.
- [58:26]
Adam McCauley
Of course. 100%. Yeah.
- [58:29]
Professor Antony King
This is interesting. So look, my answer here is going to be unsatisfactory. You've asked something that I think is, you know, really difficult and profound question of how does. How might we understand technology interacting with culture more widely and with the armed forces? It's. It's a really difficult question. How would I answer it? Let me. Let me just try and put some thoughts back, which I don't think do justice to the question you've asked, but maybe, maybe raise a few questions. So new technology changes things in a certain way, but we always need to remember that technology is just solidified labor in a device. So when we use a piece of technology, what we're using is consolidated, congealed human knowledge and labor which isn't co present with us. We're basically working with a bunch of humans who aren't co present with us. So absolutely, the existence of that new labor congealed, that new knowledge congealed into a machine allows us because we're effectively cooperating with now 10,000 people who aren't there to do new things. So what I'd say technology therefore does change what humans can do, it does change what organizations can do, and therefore it changes what organizations can do. It changes their culture. But of course it doesn't mean that for me, it doesn't mean the technology changes the culture. For me, it's congregated social relationship of what the organization is trying to do and what the developers of the technology have tried to do, which fuses. The second point is that I think possibly with something like AI, where it's not just like a machine gun or a tank which has to be driven and is a mundane, dull object, even though if you're on the wrong end of it. It's not so mundane or dull, but it seems to be sentient, it seems to be cognizant of the world, etc. Even though it's not. So what do I think about that? What I think is the key thing is this is the congealed expertise and knowledge inside of AI is likely to alter some aspects of military culture, how they think about operations. But here's the funny thing. In a certain sense, I don't think it's going to change much in terms of military operations, because what has AI already allowed human commanders, human militaries, to do, to do more complicated operations. In other words, it allows them to do things they couldn't do with the previous technology, setting them 100 more questions about how to do, how to use that technology and what to do. In other words, we end up on a sort of, you know, a self defeating machine where the point about automation, if we, if we decided we're going to do exactly what we did in 1985, but now we've got 2025 technology, we probably could automate quite a bit of military operations in 2025, in 1985, yeah, but we're not, we want to do operations in an adversarial situation. So one of the key drivers of military culture is the enemy and the operational environment and the application of the technology. This congealed labor in AI will be that operational environment which as I say, runs away from us. Every piece of technology we get right now, we can do something faster, better, quicker, right? What other 10 things can we now do? And if you look at the history of military history, that is precisely what happens. Nothing changes because it's an adversarial environment. And whether you're armed with spears or with machine guns, you're not doing the same thing. So the commander has a load of different decisions to make. And here goes back to my very first point. The commander has loads of decisions to make. Many of those decisions cannot be solved by induction. They are definitional, they are interpretive, they are acts of imagination. So the congealed technology, I think will is already changing some aspects of military culture. The structure, the practices, the understandings in tide of military headquarters that have been partnering with Palantir have changed from 10 years ago. 18th Airborne Corps is not the same organization. The structure of the careers is starting to change. The expectations of the staff officer is starting to change. But to say somehow that's just the technology and that there's going to be a base level, fundamental cultural revolutionary change I got to say I'm deeply skeptical because the minute we embed it in that context of military operations, adversarial strategy, adversarial warfare, suddenly the technology is a really important aspect, but not even the decisive one in many cases.
- [64:01]
Adam McCauley
Yeah. I think just to close here, one of the insights you have in the book, which I think is quite profound and a good one to take away, is that even in our most technologically advanced conflict at our present moment, the only thing that it's seemingly enabled is the slow, begrudging type of combat that we've seen in the past. And so you meet your adversary with those same tools on the battlefield, and perhaps it takes just as long to fight out that competition as it ever has. So I just wanted to thank you again, Anthony, for joining us today with Intelligence Squared. Your new book, AI Automation and the Rise of the Military Tech Complex, is available online and at a bookstore near any of our viewers or listeners. I'm Adam McCauley and from all of us here at Intelligence Squared, thank you for joining us today.
- [64:50]
Professor Antony King
Thank you.
- [64:51]
Mia Sorrenti
Thanks for listening to Intelligence Squared. This episode was produced by Margarita Valparto and it was edited by Mark Roberts.
- [65:11]
Vanguard Sponsor / Experian Sponsor
Join Vanguard for a moment of meditation. Take a deep breath. Picture yourself reaching your financial goals. Feel that freedom. Visit vanguard.com investinginyou to learn more. All investing is subject to risk.
- [65:27]
Professor Antony King
Vinyl chloride is under EPA investigation. This toxic chemical used in PVC pipes is being reviewed for its risks to health and the environment. Learn more@informedwaterchoices.com Dr. India woods here As.
- [65:43]
Figs Sponsor
A veterinarian, I demand a lot of my scrubs on the job. That's why whether I'm at the clinic or visiting a four legged friend out in the field, I show up in figs. Figs are made for the sometimes crazy situations I find myself in. It doesn't matter if I have to squat and sit, bend or even chase after one of my patients. FIGS moves with me and keeps me comfortable in the highest pressure situations. Figs are designed specifically for healthcare professionals and have a lot of pockets and zip compartments that never snag so I can carry a ton of necessities like treats, gloves, instruments and more. Figs are also liquid repellent and never fade after the wash. One of the best moments is taking a fresh pair out of the dryer before a long shift. As a vet, I spend a lot of my time worrying about my patients and the people who love them, and it's not always easy. Figs gives me one less thing to worry about and even help me feel a little more confident on the job if my patients could talk. They tell you to hurry up and get some figs. Show up in figs. Get 15% off your first order by visiting where figs.com and use code figs rx at checkout. That's wherefigs.com code figs rx taking care.
- [66:47]
Vanguard Sponsor / Experian Sponsor
Of your money is taking care of your mental health. That's why I use Experian, my big financial friend. With the Experian app, I can check my FICO score, explore savings tools, and get matched with credit card offers. Some cards are labeled no Ding Decline, which means if I'm not approved, they won't hurt my credit scores. That's confidence I can count on. Download the Experian app today. It's free. Applying for no Ding declined cards won't hurt your credit scores. If you aren't initially approved, initial approval will result in a hard inquiry, which may impact your credit scores.
- [67:15]
Figs Sponsor
Experian.