Loading summary
Transcript46 lines
- [00:00]
A
Dan I'm Dan Kurtzphelin and this is the Foreign affairs interview.
- [00:05]
B
If we stand still, I think we will cede a lot of the territory to China, who may not have the same standards for security and safety, who may not be interested in the sort of norms and guardrails that we want to put in place.
- [00:22]
C
Hi, everyone. I'm Kate Brannan, deputy editor at the magazine, sitting in for Dan this week. From killer robots to smarter logistics, artificial intelligence promises to change the way the US Military fights and develops weapons. As this new technology comes online, the opportunities are coming into focus, but so are the dangers. In a new piece for Foreign Affairs, Michelle Flournoy argues the US Military has no choice but to move forward with AI and to do so quickly. Flournoy served as the Pentagon's policy chief during the Obama administration. I talked to her about how the Defense Department will need to change the way it does business if it wants to integrate AI safely and responsibly. Michelle, thank you so much for joining us today and for your recent piece in Foreign affairs titled AI Is Already At War.
- [01:16]
B
Oh, good to see you. Thanks for including me.
- [01:18]
C
One of the most heated debates these days around artificial intelligence is about how quickly it should be developed and whether the United States can afford to slow down to put safeguards in place. You write in your piece that the Pentagon needs to accelerate, not slow down, its adoption of responsible AI. Why is it so important for the US Government to move full steam ahead?
- [01:40]
B
Well, I think it's important for the national security community writ large to move forward with AI, but to do so with an eye towards safety. But the reason that we have to keep accelerating and moving forward is that we're in a strategic competition with China first and foremost, but other countries as well, and they are very quickly developing AI models and applications, and they will be applying those in the field of the military and intelligence and so forth. And if there's a country that is a first mover, they will gain tremendous. So this is an area where the US Needs to compete. We're poised, well to compete, but we also have to do it responsibly and ensure that we're setting not only standards here at home, also trying to influence international standards.
- [02:36]
C
And when it comes to AI, what's the United States competitive advantage when you look at the two countries?
- [02:41]
B
You know, I think our competitive advantage is the incredible innovation ecosystem that we've spawned in places like Silicon Valley. But more broadly, we have free flow of private capital. We attract some of the best talent from around the world. More than half the founders in Silicon Valley are either immigrants or first generation Americans who've come to this country to develop their companies. So we have incredible capital flow, talent and culture of innovation. And so I think most of the AI innovation at this point is coming from inside the United States. That said, if you look at the human resources, the fiscal, the financial resources that China is throwing at the problem, they are definitely trying to keep up, if not surpass us in some areas. So if we stand still, I think we will cede a lot of the territory to China, who may not have the same standards for security and safety, who may not be interested in the sort of norms and guardrails that we want to put in place.
- [03:47]
C
And are there any other aspects of its system that let it move faster?
- [03:51]
B
Well, I think the primary aspect is that the state can direct capital to flow into priority areas in a way that just doesn't happen in the United States in our system. And they can also throw human talent at the problem. They can decide what, okay, now we're going to switch the number of engineers we create and put them all into, you know, AI and computer science. And so the state really controls the allocation of resources in a much more direct way than a free market system that we have here.
- [04:25]
C
I'm curious, were you ever an AI skeptic? And do you remember what tipped you into thinking this is really going to be revolutionary, especially when it comes to the military?
- [04:34]
B
I think I've always been concerned about the security and safety of AI, but I also see its tremendous potential to giving us an advantage ultimately to deter and prevent conflict. That really is the name of the game here. And so I think it is a place where we have to be a leader not only in developing new applications, but also ensuring the security of the data, of the models, of the applications and their safety, which is going to mean investing not just in the models and applications, but in the testing and evaluation infrastructure, in the cybersecurity elements, in the safety elements, you know, but that is, I think it's an opportunity for the United States to lead in responsible AI and to demonstrate to the world you can be a leader in AI and also a leader in safety and security.
- [05:28]
C
What's your understanding of when people talk about the danger of AI? What are they talking about and what are sort of these worst case scenarios that we, that we should legitimately be worried about?
- [05:38]
B
I think there are a couple of basic scenarios. One is that we get to a point of what's called artificial general intelligence, which is you have a System that starts making decisions entirely separate from the human beings who created them and somehow turns against the humans. You know, this is The Space Odyssey 2001 kind of scenario with Hal, the computer who decides to destroy the human. But I think the more near term and realistic concerns are applications of AI where we prematurely take human beings out of the decision making loop. I think the case that worries people is the application of AI in very high stakes situations such as a military situation where human life may be at stake without adequate human involvement in the decision making about when lethal force is used. And again, both as a matter of policy and as a matter of design, that's one we can control for. That's one where we have agencies to decide on how limited or how broadly AI is used and what sort of applications are acceptable or not in terms of things that go towards full automation or AI working without a human in or on the loop.
- [07:03]
C
Is that a live debate inside the Defense Department?
- [07:06]
B
Absolutely. I mean, not only is there a framework for responsible AI, there's a much more specific policy on the use of fully autonomous weapons. And so this has been something that the department has been debating even before the latest kind of AI craze or the bursting of AI into the public consciousness. And there are ethical aspects, there are legal aspects, and there are also just fundamental command and control elements that, you know, what do commanders feel comfortable with in the field? Where are they willing to rely on AI and where are they not willing to rely? And that's, that's going to evolve over time. But I don't think we're at any danger of the United States fielding killer robots that, you know, can act freely without human oversight. Not in my lifetime, probably not in my children's lifetime, maybe never.
- [08:00]
C
When people talk about AI in the military, it often does go to kind of that killer robots, the AI making targeting decisions like that. I feel like that's what the public is thinking about. But in your piece, you talk about all the ways AI can improve things like procurement, weapons, maintenance, logistics, which are giant parts of what the US Military does. Could you talk a little bit more about what are some of these sort of mundane tasks, but that they actually could be quite transformative when AI is applied to them.
- [08:28]
B
So the first is just really smart resource allocation decisions and understanding the second and third and fourth degree effects of making a certain budget decision or what have you. A second example is called predictive maintenance, where we're taking data off of weapon systems as they operate and we're using machine learning and AI to get A much better sense of a part that's going to fail before it fails. So you can do preventative maintenance and you can optimize maintenance schedules, and you're not doing things that aren't necessary, and you are getting to things before there's catastrophic failure. So that's already, in a documented way, saving huge amounts of money. It has potential to save much more the more it's applied. But where it has been applied, it's having a meaningful, creating meaningful savings for maintenance. A third example is predictive intelligence. So we have a number of situations where we're using AI to look at just vast quantities of data, to try to recognize patterns of behavior, patterns of data that an analyst may not be easily discerned by a human analyst, and to sort of nominate those patterns for analysts to go deeper. And what's happening in reality is we're seeing indications of warnings of future events much earlier than we did before. And so in practical terms, when a decision maker has much more warning, whether it's the invasion of Ukraine or an operation that's happening in space by an adversary, you have more decision time. You have more time to look at options. You have more time to evaluate the pros and cons of different reactions or responses. So that's already hearing and now and starting to happen. It's not replacing human judgment or analysis. It is facilitating and making that better across industries.
- [10:33]
C
People are worried about AI making their jobs obsolete, myself included. But in the next 10 to 15 years, are there jobs in the military or at the Defense Department? Or maybe it's a longer timeline that you think will become obsolete because of AI? Or is that a discussion that's taking place?
- [10:51]
B
You know, I don't think it's been a focus of the discussion yet, because I think most of the discussion in the Pentagon has been on operational applications. How is this going to help the war fighter, the decision maker, et cetera. But the Pentagon is one of the largest business enterprises in the world. And to the extent you start applying AI to certain back office functions, like any business, there is the potential for you to, in some cases, maybe replace some people in terms of jobs. But more likely, even more likely, it's sort of a. If you upskill your workforce, AI is a productivity enhancer for the workforce that you have in terms, it will help people be much more productive with data. I mean, the Department of Defense has massive amounts of data that it sits on, and most of that data is not actively used in the most productive way to inform the best resource allocation, the best decision making and so forth. And so there's real opportunity there. And I think with some upskilling, you're more likely to see a more greater productivity of the workforce rather than a dramatic shrinking of the workforce.
- [12:05]
C
Are there any areas of AI and ways that it could be applied that seem over hyped to you?
- [12:12]
B
I think the overhyping is really like assuming a sort of fully automated and automatic type of warfare, which I just don't see us as a democracy adopting and I don't see our commanders seeking to implement. What I do see is AI being applied in ways that give human beings more agency and more effectiveness.
- [12:38]
C
Let's talk a little bit about regulation and making sure that AI is developed and used responsibly and ethically. As you write in your piece. I'm quoting you. The essential effort to simultaneously foster AI and put guardrails around its use aims that are seemingly intention is underway. Is it fair to say that right now the fostering of new technology is really out ahead of the guardrails? Is there catching up that needs to be done?
- [13:03]
B
I think in the commercial world, with ChatGPT and other recent developments, that may be a true statement. I think there are not clear guardrails in place yet and people have raised very valid safety concerns, concerns of implicit bias, concerns about cybersecurity or data poisoning, unexplainability, lack of transparency, those sorts of things. But I think in the DoD context there is a very clear framework. What I argue in the piece is that we need to resource the full implementation of that framework. So we need more people who are capable of testing and evaluating and verifying and validating AI models before they're put into use in the field. We need more people who know how. That's not a one and done situation. Once something's in use, you've got to monitor it. Is it behaving as expected? Are there unexpected behaviors? If the latter, we need to take it offline, understand what's happening, correct it, etc. We need people who can recognize whether data has been poisoned or manipulated by the adversary to make the model behave differently. And we need people who are looking at how is AI being used to accelerate cyber attacks and how do we combat that. So there's just a lot of investment in human talent, both inside the government and leveraging outside talent, as well as rigorous protocols for testing and evaluating and monitoring AI when it is used at scale in the department.
- [14:42]
C
The Defense Department's been talking for years about needing to do a better job recruiting the best talent for its tech workforce. Have you seen improvements over the years in that way or what could actually make a difference in this area? So it's. I feel like that call is just always put out there, but I'm not sure what's changing.
- [15:02]
B
Yeah, I do think there have been some good pilots at a sort of local level. You know, the Air Force has a program called Kessel Run. The army has started a software factory, but they're small and they're sort of disaggregated. I think the first thing the military needs to do a better job of is to manage the tech savvy talent they have. So currently, if you have tech acumen or aptitude, that doesn't necessarily mean you'll be placed in a tech related job. Even if you are in a tech related job, it doesn't mean that there's a career path that will keep you developing as a technologist. In fact, there isn't one right now in any of the services. And so how we manage tech savvy talent that we already have is job number one. The second is how do we create more of that talent? And here, borrowing from something that Eric Schmidt and the National Security Commission on AI talked about, I think we do need something like a digital academy, like another service academy where if you'll get your education paid for at the university level and then you commit to some number of years of service to the government or to the national security sector at large. That's one model. Another is the notion of a digital reserve corps. So there are people in Silicon Valley and other tech hubs around the country who would love to serve their country on weekends or two weeks a year, whatever the commitment is. And there's no real pathway for them to do that right now. It's very, very minimal. And then there's just the fact that we're never going to fully compete as a government for the best tech talent with private industry. So we have to figure out better public private partnerships to leverage that talent from industry to help where we need it as well. So the talent piece is huge. Even as in the piece, it even extends to immigration policy. So much of our tech talent coming from outside the United States, studying in our universities, we want to widen that pipeline and we also want to encourage people to stay, build their businesses here, become American citizens and bring their talents to this country. And we've got to do some targeted reform for high tech talent to bring more of them in and allow more of them to stay.
- [17:37]
A
We'll be back after a short break
- [17:40]
C
in a World where knowledge isn't just power, but a means to ensure safety, security and prosperity. The need for skilled intelligence professionals has never been greater. At Georgetown University, our online Master's in Applied intelligence offers concentrations in homeland security intelligence, cyber intelligence, law enforcement intelligence, and competitive business intelligence. For more information or to apply Visit SCS georgetown.edu Security to stay on the idea of more fusion between Silicon Valley and the Defense Department. I used to be a defense reporter a million years ago, and when I covered the Pentagon, I wrote about that clash of cultures between the Pentagon and Silicon Valley. In your current role at WestExec Advisors, you often act as an intermediary between these tech companies and the government. And I'm curious what they say about their interactions with the Defense Department. What are their impressions? What are their kind of frustrations in terms of doing business?
- [18:44]
B
I like to describe us as coaches and navigators. We don't do any representation or lobbying, but we try to help them understand how, you know, working with the federal government is very different than working with their commercial customers and how to navigate the maze. You know, I think there, there is a lot of frustration that there's still many, many obstacles. You have a company with cutting edge tech that's really relevant to a priority operational need. You have people in the military and in the department who want the technology or the capability, but that is not enough to get them over the many, many hurdles to become a program of record and to have a scale, you know, a production at scale contract. And so we still make it much too hard for them. That said, I am seeing companies become more sophisticated in how they do navigate. I am seeing the department making an effort like reinvigorating and re empowering diu, creating the cdao, the office, you know, the Chief Data and AI Officer creating this Office of Strategic Capital. These are all efforts to try to close that gap and bridge the so called valley of death for these companies. And I'm also seeing in this more volatile international environment, more of the investors being willing to play in the national security space, more venture capital firms, private equity firms saying, hey, we're willing to be more patient capital because we think this is so important for the nation. And so people are starting funds that are national security focused, which just didn't happen five, ten years ago. So I am optimistic. I just want the pace of, I'm frustrated. I want the pace of change to be faster than it is.
- [20:44]
C
On the flip side, you used to serve in the Defense Department. What do they say about their private sector counterparts? And do they have their own set of frustrations.
- [20:55]
B
I think that most of the, there are many people who don't really understand the life cycle of a venture backed startup and so it's hard for them to understand why they're so impatient. They don't understand that they've got to show return on investment in a relatively short amount of time to be able to continue to attract capital and continue to exist and grow as a company. To be fair, I think what the department needs to do is actually create a separate cadre of acquisition professionals that are really educated and trained on commercial tech development and particularly software driven tech, which most of these are the agile development process, the venture backed ecosystem, to understand how to work with these companies more effectively, to have a different risk tolerance and to, you know, have our acquisition folks be rewarded and promoted for a different set of behaviors than the ones you want. The folks who are doing a 10 or 12, 15 year procurement of an aircraft carrier, you know, we understand why, we want them to be risk averse and focused on keeping things on schedule and to requirements and on cost. But when you're, you know, working with a company who's, you know, developing a product where each iteration is, you know, three to six month cycles, the current requirements driven acquisition system does not work for that type of capability. And so you need a different approach.
- [22:44]
C
When you look to Congress to regulate the industry, it's proven fairly inept or inactive when it comes to regulating tech companies so far, especially in the social media space. Do you think it will be any different this time with AI, Is it wrong to expect Congress to step in and make sure things are developed safely?
- [23:05]
B
I do think that there will be some move towards regulation you see right now, particularly in the Senate, but to a degree in the House, real serious efforts for members to educate themselves about the technology. Both the potential positives but also the dangers. And I think actually Leader Schumer is taking an approach that was very similar to what they did as a precursor to the CHIPS act, where they spent a lot of time learning about the semiconductor industry and the state of the competition and so forth. So I'm hoping that kind of process will yield some smart regulatory frameworks that make a positive contribution in terms of safety and security while not being overly heavy handed and thwarting innovation and the speed of adoption. So that's a, it's a threading a needle. And regulation, you know, legislation can be a blunt instrument, but I think if they set up some mechanisms for doing this over time, I think, you know, that could be a very Important investment for the nation.
- [24:26]
C
As I was preparing for this, I was thinking a lot about how important leadership is to maintaining this commitment to responsibility and safety. It requires responsible leaders in the private sector, responsible leaders in the military, responsible leaders at the White House. Does responsible AI depend on having responsible leaders in business and in government?
- [24:50]
B
I do think at this moment, when we don't have all of the systems and mechanisms in place, the leadership component is even more important because we're in the infancy of figuring out how we're going to construct a system that keeps our applications of AI responsible, safe. I think the name of the game for the Pentagon is to bake it into their normal processes so that there's a certain approach to testing and evaluation. There's a certain set of requirements that are put into contracts in terms of data sources and transparency so that vendors have to meet those, as a matter of fact, for kind of first principles. So the more we can figure out how to bake this into our processes, the less it will depend on an individual leader focusing on this and driving the change, and the more it will become part of the systemic effort.
- [25:58]
C
Okay, but we're not there yet. We're at the sort of nascent phase,
- [26:01]
B
the very early stages of that. And I think trying to figure out how to, you know, what. What the guardrail should be and then how to systematize them is, you know, we're at the earliest stages of that before we close.
- [26:14]
C
I'd love to zoom out and ask a few bigger picture questions. There seems to be a consensus that we're at a geostrategic inflection point these days. You hear that term a lot. If so, I'd love to know what you think we're leaving behind and what will define this new era, if we know yet.
- [26:32]
B
Well, I think we are. We're moving into a much more multipolar competitive environment where you have numerous countries that can exercise some power and influence, whether it's a rising China or a declining Russia, but that's still powerful enough to, you know, start a war in Europe, which we had thought was, you know, Europe whole freeing, at peace. Well, no longer because of Vladimir Putin. So. And you see other countries not aligning necessarily with either coal or any, you know, on a permanent basis, but acting more like swing states. Like, you know, a number of countries in Asia want China as their. See China as their primary trading partner, but they count on the US as their most important security partner. You see Gulf states swinging between the US And China to see who's going to Give them the best deal on a given issue. You see the Global south also kind of wanting to stay out of these. The strategic competition between the US And China, but again, often aligning with China, which has invested heavily in their infrastructure through the Belt and Road Initiative, but also wanting certain protections and investments from the United States. So things are in flux. The international order, the inherited rules that we architected after War 2 are now being challenged by both Russia and China and others like Iran. So it's just a much more volatile, much more uncertain environment that I think will be more multipolar and more chaotic for a while.
- [28:30]
C
For you, what are the open questions of this new era? I guess the main one is sort of, where does, in such a volatile, multipolar time, who gets on top or who outpaces whom? But what are the open questions for you?
- [28:46]
B
I think the number one in my mind is how do we prevent the competition from becoming conflict? Because I think an actual war between the United States and China to nuclear armed powers would be catastrophic. I mean, not only for them, but for the region, for the world, the global economy. There's so much that's integrated now that everyone would feel the effects of something like that. So I think managing competition so that it does not become conflict is number one. Number two is creating some space for cooperation where we can. Because we cannot solve or manage climate change and its impacts without some collaboration between the United States and China and other great powers. We can't prevent the next pandemic without some cooperation. We can't prevent further nuclear proliferation without cooperation. So we have got to somehow carve out space for cooperation when it's in our interests to do so. And then I think the kind of discussion we've been having here on AI. AI is not the only disruptive technology out there. There's synthetic bio. And when you combine AI with synthetic bio, you can enter a world where an individual or a very small group of people can do very dangerous things like design a new pathogen for use as a bioterror weapon. So we've got to start thinking about the sort of safety and security guardrails across a number of disruptive technologies, not just AI.
- [30:41]
C
What are you worried about in the near term, even over the next six months? What are you sort of what's most concerning to you as you look at the world?
- [30:49]
B
I'm worried about conflict escalating as a result of miscalculation, be it in Ukraine or with Israel and Iran potentially. I don't think that's the likely outcome in either case. But I think it's a very real risk. I worry about our own degree of domestic polarization and dysfunction preventing us from doing what we need to do to keep competing effectively and to safeguard our national security. Concrete example, you know, if we live with a continuing resolution for all of 20, 24 series of them, those don't allow any new starts in defense. So any new investments that we are trying to make to give our commanders new tools to ensure deterrence and that Xi doesn't go to war over Taiwan, for example, none of those are possible with a continuing resolution. So we can be our own worst enemy if we're not careful. We have to make sure that we, we rise above our politics. I know this sounds naive, but it's really a national security imperative that we get beyond some of these political issues to focus on safeguarding national security. Another example is the hold that's been put on hundreds and hundreds of senior officers in the military. What that's doing to them, what that's doing to their families, but also what it's doing in the system, which is really preventing progress from being made in these areas where we need urgent progress.
- [32:40]
C
All right. Well, we'll leave it there. Thank you for the excellent piece in the magazine and thank you so much for joining us today.
- [32:46]
B
Thank you. Look forward to continuing the discussion sometime.
- [32:56]
A
Thank you for listening. You can find the articles that we discussed on today's show@foreign affairs.com the Foreign affairs interview is produced by Kate Brannan, Julia Fleming dresser and Molly McEnany. Special thanks also to Grace Finlayson, Caitlin Joseph, Nora Revenaugh, Asher Ross, Gabrielle Sierra and Marcus Zacharia. Our theme music was written and performed by Robin Hilton. Make sure you subscribe to the show wherever you listen to podcasts. And if you like what you heard, please take a minute to rate and review it. We release a new show every other Thursday. Thanks again for tuning in.