← The AI Policy Podcast
The AI Policy Podcast
Inside Anthropic's Standoff with the Pentagon and What It Means for Military AI
01:00:07
We unpack the Anthropic–Pentagon clash, Pete Hegseth’s ultimatum, DPA risks, AI’s economic impacts, and claims DeepSeek used Nvidia Blackwell chips.
Loading summary
Transcript63 lines
- [00:00]
A
Foreign.
- [00:11]
B
Welcome back to the AI Policy Podcast. I'm Matt Mand, and today I'm speaking with Greg Allen about some of the biggest AI policy news, including anthropic clash with the Pentagon, AI's economic impacts and a senior government official's claim that Deep Seq smuggled AI chips to train its latest model. Greg, it's been a while since our last news roundup, so it's good to be back.
- [00:29]
A
It's overdue, and I'm glad we're doing it.
- [00:32]
B
So let's dive into our first section. Over the past several months, the Pentagon has expressed frustration, to say the least, over Anthropic's refusal to remove model safeguards and user policy requirements. And this frustration has now reached a boiling point. Right before recording this, actually, War Secretary Pete Hegseth gave Anthropic CEO Dario Amade a Feb. 27 deadline to comply with his demands, threatening to label Anthropic as a, quote, supply chain risk, unquote, or invoke the Defense Production Act. We'll talk a bit about what this would mean and why it's pretty extraordinary in a moment, but first, could you explain how the Pentagon currently uses Anthropic's models and where negotiations currently stand?
- [01:14]
A
Sure. So, to begin, Anthropic has a contract with the Department of Defense valued at around $200 million, and that goes through the chief Digital and Artificial Intellig office. That's actually the successor to the organization that I used to work at. I was the director of Strategy and Policy at the Joint AI Center. That organization was folded into the CDAO organization when it was created back in, gosh, I want to say March of 2023. So Anthropic is on contract with the DOD. They also have a relationship with Palantir, which is a super important provider to the Department of Defense and the intelligence community for a whole host of things. Now, what exactly are they doing there? Well, the first thing to note is that a lot of what they're doing is classified and on classified networks. So when the DoD does things on computers, it does that across multiple networks. There's Nipper, Sipper, and jwix, and even more, you know, private networks beyond that. But those are the big three. Nipper is the unclassified one, Sipper is the classified secret one, and J WIX is the TSSCI network. And notably, Anthropic is the only AI model provider who is providing AI capabilities on classified digital networks. And that's been true for a while now. So other AI companies, Google, XAI OpenAI, et cetera. They are providing AI capabilities to the Department of Defense, but at least for now, they've only been doing that on unclassified networks, which really limits, you know, how impactful you can be, especially on applications that are closer to the sensitive activities of the Department of Defense.
- [03:02]
B
Yeah.
- [03:02]
A
Whether that's actually war fighting or intelligence and reconnaissance type stuff. Yeah.
- [03:07]
B
Let me just jump in here really quick because I think actually earlier this morning, or maybe it was yesterday afternoon, XAI signed an agreement which we'll discuss later, which will now put them on these classified networks. Is that right?
- [03:18]
A
That's right. But that's just an agreement that allows them to go on there. The reality is that Anthropic has been there and there are war fighting communities that have already been using Anthropic and developing capabilities integrated deeply with Anthropic's technical stack for a while now. So, for example, the Wall Street Journal reported not that long ago that the raid that led to the capture of former Venezuelan President Nicolas Maduro was specifically enabled by Anthropic's AI capabilities. I'm assuming that was related to cyber attack relevant capabilities, because one of the things that Anthropic capabilities are useful for is cyber, both on the defensive side and, as would almost certainly be the case in this instance, on the offensive side. So that's the reality is that CLAU is in a league of its own when it comes to right now, right this second, enabling the Department of Defense with frontier large language model AI capabilities. Of course, there are other types of AI capabilities, like the kind of stuff that Palantir has been providing through Project Maven, which is more about computer vision or other parts of the DoD and the IC which are using it for speech recognition. But where Anthropic really steps in is on its coding relevant capabilities and then on other things that take advantage of large language models.
- [04:37]
B
Yeah. And Pentagon officials have explicitly acknowledged how important these capabilities are, right?
- [04:43]
A
Yes. I mean, at least according to the reporting from Axios, which I have to give a shout out to Axios. They've been all over the story with terrific coverage. They're quoting an anonymous senior administration official that says competing models are, quote, are just behind when it comes to specialized government applications. Complicating an abrupt switch, and that switching cost is part of the story, I think, is actually a really important data point, not just for this instance and the Department of Defense, but in the broader AI ecosystem. Because I think what we've seen is that large language models, at least in the CHATBOT interface for individual consumers, the switching costs are comparatively minor. You might be using ChatGPT one month and then decide, try Claude, decide you like it better. Not a big deal for an individual user to switch over. But what this suggests is that once you're actually building applications that are taking advantage of these capabilities, perhaps at scale, using API interfaces, using agents, et cetera, actually you are doing a lot of building and integrating work that is tough to replicate and makes the switching costs high. That's why the DoD is so frustrated with Anthropic, because it really would mean something to kick them out.
- [05:59]
B
Yeah. So tell us more about where things are right now. I mean, Anthropic has, has been asking or forcing the DoD to comply with its usage policy, right?
- [06:09]
A
Yeah.
- [06:10]
B
Now, it's resulted in months of negotiations around this.
- [06:13]
A
Sure. So anytime you are using online software, you are probably used to, you know, clicking that I agree button where you accept the terms of service and conditions. And, you know, that's the commercial license, that's for the individual user license. What Anthropic has is the enterprise equival equivalent of that. And specifically for its government customers, those terms of service obviously are actually more allowant. You know, the government can do a lot more stuff that the commercial version of Anthropic is not allowed to do. So, for example, if you ask Anthropic's Claude Chatbot about bioweapons, there's a decent chance it's going to say, hey, I don't know what your interest in bioweapons really is, but I don't want to really engage in this conversation. Whereas in the DOD they have a lot of legitimate reasons to maybe want to ask questions about bioweapons. Bioweapons defense is a core function of the Department of Defense and understanding, you know, what might be signals of bioweapons activities or what countries have historically had programs. I'm just making this up off the top of my head, but the point is, there's a lot of legitimate reasons why people in the DoD can and should have conversations that would be restricted in terms of permissions and accessibility for the commercial version of Claude. But there still is that terms of service usage agreement and Anthropic being a very safety and security minded company, trying to stand out with its ethical promises and using AI for the benefit of humanity, for the benefit of democracies. Obviously they had terms of service going into that first DoD contract in July 2025, at least from the conversations that I'm having with folks who are part of these negotiations, the claim is that these terms of service have literally never come up, at least not since Anthropic relaxed them to allow for offensive cyber capabilities some number of months ago. And now, as the Department of War is renegotiating the contract as laid out in the Dowai strategy, they are really saying that we're not interested in working with companies that have elaborate terms of service. In fact, all we want is for you to agree that we can use any of the capabilities you provide us for any lawful use, or all lawful use.
- [08:45]
B
That's. That's the key language right there.
- [08:46]
A
Exactly. That's the key language. And what the Department of War and I'm using, you know, what they now call themselves, although Department of Defense is also still part of their legal name. You know, the congressional appropriations bill says Department of Defense, but I'll use them interchangeably here. The Department of War is saying that they need flexibility, and I'm sympathetic to that argument. Here's what that might mean. There are military capabilities that are controversial. There are use cases of military force that are controversial. And the Department of Defense does not want to prevent a debate on those use cases from occurring in a democracy. Right. But the right place to have that debate is in Congress or in the executive branch or in the public sphere. The Department of Defense does not want a debate with its contractors, with the companies providing those capabilities. You know, to give an extreme example, during the Vietnam War, the use of napalm provided by chemical companies was quite controversial. And there were some people who would argue that what the United States was using that napalm for in the war in Vietnam was unethical and counterproductive. And that's a debate that. I mean, I'm, I'm sort of speaking for the Department of War here. That is fine to have in Congress where Congress can say the dod, no, you can't use napalm for these use cases, or among the White House dod, you can't use napalm for these circumstances. But what the DoD does not want is the companies that make napalm putting restrictions on when and how they can use it. Like the DOD would argue. That's not your place. Right? That's not your role in this story. Your role is to provide us with the capabilities, and then our democratic institutions, our military institutions will determine what is the appropriate use of that. And now connecting it to the use cases that Anthropic is trying to restrict with its capabilities. There's. They've basically retreated on everything. There's. They're down to only two things that they are fighting should be included in the final terms of service, and that is mass domestic surveillance and use of autonomous weapons, attacking manned or crude things. So basically, autonomous, lethal autonomous weapons to a first approximation is what they're saying. And notably here, Anthropic isn't even saying that you can't use their technology to develop lethal autonomous weapons. They're saying you can use their technology to develop lethal autonomous weapons. What they're saying is that the technology is not sufficiently mature to support a decision to field such technologies to use them operationally. So what they're saying is we're willing to support the development of lethal autonomous weapons, but we cannot be on board with using the lethal autonomous weapons.
- [11:56]
B
Right.
- [11:56]
A
Based on our current assessment of what the maturity of the technology is, it would simply be irresponsible and perhaps even counterproductive to do that. So the anthropic position on lethal autonomous weapons is not a never, it's a not right now. The technology's too immature. So that is a quite modest set of demands on the part of Anthropic. Right. They're really only coming down to these, these two things. And the DOD says, absolutely not. We are not interested in engaging in this debate. What we want from the companies who support us is when we say jump, you say, how high.
- [12:38]
B
Yeah.
- [12:38]
A
And if you're opposed to the way in which we are using these capabilities, write your congressman. Right. But do not put it in your terms of service. That's not appropriate when our servicemen and women are in harm's way, when national security is on the line. We need the flexibility to make these decisions and just to connect it again to lethal autonomous weapons. Our colleague here at csis, Katerina Bandar, has a report coming out in the next few weeks that points out that Russia evidently is using lethal autonomous AI enabled weapons in the war in Ukraine right now. So you can imagine how the Department of War could say, how are you, some company, going to purport to tell us, the Department of War, that we can't have the capabilities that Russia is using on the battlefield right now? Like, we need to be the ones who make the decision as to whether or not we. We match Russia's decision? Maybe we will, maybe we won't. But that's our call, not yours. And Anthropic would simply say, look, you signed on to our first batch of terms of service. It wasn't a problem then. You know, we're much more forgiving now. We've, we've greatly relaxed you know where our terms of service are. These are our last two points of contention. We don't want to budge. Come on. Is effectively their position. And. Yeah, so. So.
- [14:05]
B
So I think you've done a great job outlining either side of the argument here. And they take these two sides into the meeting that happened a few hours before recording this, and they had reportedly a very tense discussion. And so, yeah, what came out of this discussion? Where are we now?
- [14:23]
A
Well, according to the reporting that's available on this topic, the Secretary of War, Pete Hegseth, gave Dario Amudei, the CEO of Anthropic, an ultimatum and said, as you mentioned earlier in this podcast, you have until February 27th, and then we're either going to cancel the contract or we're going to label you a supply chain risk, or we're going to invoke the Defense Production act, which might give us the legal authority to compel you to provide the capabilities under the terms of service that we outline. Now, the first one of those, canceling the contract, I think would be a unfortunate outcome, but a reasonable outcome. As I said, I'm sympathetic here to the Department of War position that that's a call they should make, not that companies should make, but the other two options on the table would be a shocking escalation and really one that I think would be a huge mistake on the Department of War if they were to engage in that. I mean, effectively. I was in the dod, and I remember these debates before I went into the DoD in the summer of 2018, Google, very prominently, very loudly, very contentiously pulled out of Project Maven, which was at the time the Department of Defense's flagship military AI initiative. And Google pulled out of that project. They issued this whole set of AI ethical principles which said they were not going to be involved or allow their technology to be involved in weapons development. That was a very painful moment for the DoD AI initiatives and really a terrible outcome for Google, at least in terms of their public opinion. I mean, the military in the United States historically has the highest approval ratings of any public institution, oftentimes, you know, 80% public approval ratings. While it's totally routine in the United States for a president to have approval ratings below 40%, in general, America loves the military. So when a company says they're not going to support the military for certain kinds of activities, as Google did, that can be quite contentious. And when I joined the military, then a very senior government official, I want to say it was the chairman of the Joint Chiefs of Staff at the time, General Dunford although I could be wrong on that, but certainly was one of the high ups on the Joint Chiefs of Staff. He effectively accused Google of refusing to help the DoD and while it was simultaneously offering to help China. That was a terrible day for the Google communications department. But here's the thing. Google was refusing to participate in what was a comparatively innocuous program at the time. I mean the early days of Project Maven were like identifying how many cars were in a satellite picture or a drone picture. Right. This was not some super advanced AI killbot kind of technology. This was pretty innocuous stuff. And Google pulled out of it and it was a big blow to the DoD's efforts to forge closer ties with Silicon Valley. Well, now you have a remarkable turn of events where a very prominent world leading AI technology company is enthusiastically participating with the dod. They bid on that contract. They allowed, at least according to the reporting in the Wall Street Journal, their technology to be used for offensive cyber attacks against a foreign country that led to the capture of that country's president. They are being deeply put into actual war fighting activity and they're the farthest along of any AI company. And the message that the Department of Defense would be sending if they really hurt Anthropic would basically be to any AI startup, if you are even thinking about dipping your toe in the DoD waters, we are going to grab you by the shirt and pull you all the way until you're soaking wet.
- [18:40]
B
Yeah, yeah.
- [18:42]
A
That is not a great signal to be sending.
- [18:44]
B
Yeah. And I think like, just for the audience, can you explain a little bit more about what being designated as a supply chain risk would mean for Anthropic as well as the use of the Defense Production Act?
- [18:56]
A
Yes. And for, for those who are not familiar with these terms, you might think it's like not that innocuous. Right? You might think it's quite innocuous. So what you might think is that they're saying, oh, this person might say no to you. So they're not a reliable supplier, they're a supply chain risk. That's like what the everyday English interpretation of that phrase would mean. That's not what it means. That's not what it means in this legal context at all. This is like the nuclear option in the DoD anthropic relationship. Designating a company as a supply chain risk. That's what we do when your owners are Russian or Chinese and we're worried that your company is basically just a front for foreign intelligence services. And we're saying that not only can the DoD not work with you. The DoD can't work with any company that works with you. We view you as that much of a security risk. Well, Anthropic is the type of company who, when I was in the DoD, we were begging these types of people to take us seriously and come assist us in our work. I mean, the AI talent pool on planet Earth is still pretty shallow. And having really talented companies wanting to work with you, that's a great US national security advantage. And so to go to that community and say, hey, people, we've been begging to work with us, actually, we not only don't want you to want to work with you, we don't want anyone to work with you. And that it probably would not be fatal to Anthropic's business, but it would be devastating to Anthropic's business. I mean, the $200 million contract that anthropic has with the do, that's decent money, but that's not huge money. I think Anthropic's revenue was in the single digit billions this year and more than doubling every single year that goes by. They can live without that $200 million spread over several years, kind of a contract. What Anthropic cannot tolerate is if every company in the US Economy that does business with the Department of Defense is suddenly forbidden from working with them. That would be devastating to their business. And this is why I say it is a huge escalation and overreaction on the Department of the War if they do in fact invoke these powers. And I would say it was an overreaction even to threaten these powers. And it's not just me who's saying this. I want to read a line from Dean Ball, who was previously a senior advisor in the White House Office of Science and Technology Policy in the second Trump administration. He was deeply involved in the drafting of the American AI Action plan. And he's a Republican who served in the second Trump administration. So not exactly a bleeding heart liberal here. Here's what he said on this story, at least what he posted on X. This designation seems quite escalatory, carrying numerous unintended consequences and doing potential significant damage to US interests. In the long run, I hope the two organizations can work out a mutually agreeable deal. If they can't, I hope they agree to peaceably part ways. But this really needn't be a holy war. This administration believes AI is the defining technology competition of our time. I don't see how tearing down one of the most advanced and innovative AI Startups in America helps America win that competition, it seems like it would straightforwardly do the opposite. The supply chain risk designation is not a necessary move. Cheaper options are on the table. If no deal as possible, cancel the contract and leverage America's robustly competitive AI market, maintained in no small part by this administration's pro innovation stance, to give business to one or more of Anthropic's several fierce competitors. I think that is very well put and I think he is right on this instance. I mean, just Anthropic and the other frontier AI labs. It is a great advantage of the United States that we have these types of companies here, that they are succeeding. And for the United States government to take one of those crown jewels of our technology industry, at least in the startup part of the story, and light it on fire over a dispute like this one, as like the first option you read to a huge mistake and I hope they don't make it.
- [23:23]
B
Yeah, there are some quotes on from Axios here that I want to read that kind of, I think might illustrate why we're seeing this. And I think it is a little vengeful by the administration. I mean, a senior official told Axios, quote, it will be an enormous pain in the ass to disentangle and we are going to make sure they pay a price for forcing our hands like this. Right? Like that's a pretty serious quote. And it's clear that they're not just doing this because they think, oh, this is like the rational policy move. Right? Like this is like, in a sense it would, it would be revenge.
- [23:54]
A
I certainly hope that's not what they're optimizing for. But as you said, that's a plausible interpretation of the quote. Now, the third option on the table here is the Defense Production Act. This is a law that goes back many, many decades. And the original purpose of the law is that in times of shortage, in times of national security crisis, the government needs to be able to tell private industry, hey, I know you could make a lot of money building limousines, but what your country needs from that factory right now is for you to build tanks. Because we're in a shortage of industrial capacity, you have precious capabilities, and we're in a national security emergency, so you're going to make that stuff. I believe this was used during the COVID pandemic to force certain companies to produce ventilators, for example, because hospitals had egregious shortage of ventilators. So if you use the Defense Production act hypothetically to force Anthropic to provide this capability. Well, just think about the cognitive dissonance that would be implied that both of these options are on the table. On the one hand, Anthropic is the supply chain risk and we can't use them because they're the equivalent in this nomenclature, in this designation as a Chinese company. You can't trust them with national security. On the other hand, Anthropic is so valuable and amazing to national security that we have to force them to work with us. So, like, which is it? Is it that we can't work with them or that we have to force them to work with us? Like, there's a real cognitive dissonance and that those are the two options that are being explored. And as I said, like, I'm sympathetic to the Department of Defense's position that they need to be the ones to make these decisions. They need to have the flexibility to decide how these types of things are going to be used. But you signed this contract in July of 2025. It's not like Anthropic is crazy for thinking that this is something that a company like them might have the right to request. Your administration, the Trump, the second Trump administration, approved such a request less than a year ago. So to go from we are willing to accept a really broad set of terms of service conditions to now, if you have any terms of service conditions, we're going to effectively label you as the security equival equivalent of a Chinese or a Russian company. That is just an egregious overreaction that will as Dean Ball said, have many, many, many unintended consequences and do really, I would say, long term damage to the national security Silicon Valley relationship, which is something that we've been working hard to repair for a very long time. I mean, when the Google project maven fiasco happened back in summer of 2018, I wrote an op ed that was published in the science journal Nature criticizing Google for pulling out of that deal and saying that they should work with the military. So I'm really on the military side here, but the options that the Secretary of War is outlining, at least according to the reporting that we have available, I really hope they don't pull that trigger. It's something that, that I feel confident in saying that the department will ultimately come to regret.
- [27:19]
B
Yeah, well, I guess we'll see because Dario Amadei has until Friday to decide. And we'll be keeping an eye out for the outcome of this story. But I want to move on to our next.
- [27:30]
A
There's actually one other quote that I want to read here, please. And that comes from Andrel CEO Palmer Luckey. He posted on X. It isn't a matter of punishing companies for not sharing political views. It is a rational response to a vendor trying to control the government via terms of service in the products they power. Now here's the thing. Number one, I've been on the record praising Palmer Luckey in the past for embracing bringing Silicon Valley venture capital and Silicon Valley technology to the Department of Defense before it was cool, before it was fashionable. As a privately wealthy individual, he invested a lot of his money to try and build something that was useful for national security. And I give him a lot of credit. And I'm also sympathetic to his argument that the right place to have this kind of debate is in Congress or in the executive branch or in the court of public opinion, not in terms of service. I'm sympathetic to that argument, but I'm not sympathetic to labeling anthropic a supply chain risk. I'm not sympathetic to using the Defense Production act to force them to produce here. I just think that that would do a lot of unnecessary damage to the US military technology industry relationship. And gosh, slow down, slow down is my advice to this administration.
- [28:56]
B
I'm right there with you. Well, let's move on to something else that's not slowing down, but actually speeding up, which are the economic impacts from AI. And I want to discuss several recent stories that signal economic disruption, the potential for economic disruption that AI is poised to create, starting with a relatively ironic story from earlier this month. On February 6, the Financial Times reported that Big Four accounting and auditing firm KPMG pressed its own auditor to lower fees, arguing that the use of AI should be making the work cheaper. Can you walk us through what happened here and how it connects to the broader economic impacts of AI?
- [29:34]
A
Yes, I thought this story was really revealing. It was one of those moments when you see behind the curtain and learn a lot about how the ACT actors actually view the performance they're putting on for the world. So kpmg, one of the biggest accounting and auditing firms in the world, and they of course, are a big financial entity. So in addition to being auditors, they are also themselves audited independently. And what's so interesting is that they have been watching the AI revolution as a beneficiary of it. And so they're going to their auditor and they're negotiating what should the price be, and they're bringing up AI's impact on auditing. So here's a quote from the Financial Times article, kpmg, one of the world's largest auditors of public and private companies, negotiated lower fees from its own accountant by arguing that AI will make it cheaper to do the work, according to people familiar with. The matter goes on. Quote, the Big Four firm told its auditor, Grant Thornton uk, it should pass on cost savings from the rollout of AI and threatened to find a new accountant if it did not agree to a significant fee reduction, the people said. And then finally it said KPMG International paid $357,000 in total for the 2025 audit, according to the filing, down from $416,000 in 2024. Whoa. Right? This is a big auditing firm saying that that the price of auditing should go down because AI has made their work much more productive, much more efficient. And those cost savings should not be profited as additional profits by the auditor. They should instead be passed on to the consumers. Well, I'm inclined to think that every single customer of KPMG auditing services sent an email when they saw this article saying, I'd love to negotiate our 2026 auditing rates. Based on your company's own internal analysis, it's clear that you think that the price for auditing should go down due to the beneficial. The beneficial productivity gains of AI. Tell me how you're going to lower my cost for that. And this was. There was a great quote from Derek Thompson, who's a writer at the Atlantic, among other places, quote, this looks like a company accidentally announcing to the world that its business model is under attack.
- [32:07]
B
Yeah. Oh, that's exactly what it is. And that story broke after the release of another story, which was several new plugins from Anthropic automating tasks across legal, sales, marketing and other sectors, which triggered an $830 billion sell off of software and services stocks. One investor called it, quote, a manifestation of an awakening to the disruptive power of AI. Then Anthropic followed up with another product release that led to a drop in cybersecurity stocks this time. So could you tell us what's happening here? Is this an overreaction by investors or is this a genuine sign of disruption of white collar work?
- [32:45]
A
Well, I think it's worth pointing out that stocks are based on predictions about the future. And so even a change in the probability of different scenarios for the future can legitimately drive a change in stock price. I mean, just to give you an example, right. If you and I are going to play a betting game where we're flipping a coin and heads you win $50,000, tails you win $10,000.
- [33:16]
B
I like that game.
- [33:16]
A
Well, if we change that game to now be, you know, a six sided dice where you only win $50,000, if you roll a five or a six, you still might get that. But we don't know that at this stage. So the question is like, how much would you pay to be able to play this game? And that's kind of what I think is going on in the stock market. It's not a guaranteed outcome at this stage that the software industry is going to be obliterated by AI. That's not a guaranteed outcome, but it's a lot more plausible of an outcome at this stage. You'd be very hard pressed to look at the fact pattern available to us right now and say there's no way that that could happen. And I want to point out this is a little bit of a weird analogy, but I think it's a helpful one. The software industries in the United States and China are quite different. And that is because in the United States the software as a service business model is overwhelmingly prevalent, right? You have companies like Salesforce, which provides customer relationship management type software. You have companies like Asana that provide team collaboration and productivity type softw. There's a million companies like Westlaw which provides legal research and services enabled software. There's, there's a million of these companies who provide custom software for specific industries. And they don't just provide it to, you know, in the case of Westlaw, one law firm, they provide it to dang near every law firm, right? That's the software as a service business model. In China, the software as a service business model is much less prevalent. And, and a big part of the reason for that is that whereas in the United States we have had this dramatic shortage of software engineers for the past few decades and software engineers are extremely highly compensated because they're such a precious resource. Well, it therefore makes sense for those software engineers to sort of pool their resources and provide the same solution to many, many, many, many customers. So you sort of sacrifice customization in order to increase economies of scale and availability. In China there hasn't been that shortage of software engineers and the salaries of software engineers are much, much lower. And so what that means is that actually companies are a lot more willing to just hire their own in house software engineers. So rather than using a software as a service business model, you know, a law firm might hire its own software engineers and they might develop a custom solution just for that company. And this is, you know, What I just said about legal services. True in a bajillion industries in China. So the key here, the key difference between the two ecosystems appears to have been the shortage of software engineers. Well, what if, what if AI agents in the very near future increase the productivity of software engineers? That America's software ecosystem looks more like China where suddenly adding software engineers to your team is really cheap and pretty easy. So you can actually afford to build stuff that is highly, highly customized and it actually will be reliable, it actually will be helpful. You know, you don't have to suffer through and I'm picking on Westlaw here even though I've never used their service. Maybe it's awesome, but you don't have to suffer through like the trade offs of Westlaw trying to be good for customers whose needs are very different for yours. Your software stack can be perfectly enabled to deal with the problems exactly as you and your employees deal with them. That could be a very different reorienting of the software market and the fact that Claude Code and now Claude Cowork have sort of reset expectations in the industry as to what agents are possible, what's possible with AI agents right now and what is likely to be possible in the next 12 to 24 months? Well, it's not a guarantee of a software industry apocalypse, but it certainly has increased among all possible future scenarios. It has increased the probability, the likelihood, the plausibility of a software apocalypse. And that's what I think is driving this massive sell off.
- [37:42]
B
Yeah, I think that's a really important point because I have friends who say, well look like AI can't do these things yet, so why are people selling off their stocks? Right, right. But it's not that it needs to be able to do those things, it's
- [37:52]
A
just for almost every stock. Yeah, almost every stock in the stock market. When you calculate, you know, what is the justification for the price of this stock? For almost every stock, the majority of the value of the company is all the money it's going to be making more than five years from now discounted to the present value of those future cash flows. Yeah, so. So the future really has an impact on the present when it comes to stock prices.
- [38:22]
B
Yeah. Well, one last story I want to talk about relates to the entertainment industry. So many of our listeners have probably seen this AI generated video of Tom Cruise and Brad Pitt fighting. It's quite a funny video, so you should go check it out.
- [38:35]
A
If you haven't seen it, you got to go see it.
- [38:37]
B
Yeah, it's good stuff. But what are Filmmakers saying about the future of their industry in response to this. Because if I was a filmmaker and I opened up Twitter and I saw his video, it would be a pretty bad day for me.
- [38:48]
A
Yes. And I think going back to the golden days of Hollywood, right in the 1990s and 2000s, when stars in their prime like Bradley Cooper or Tom Cruise, could routinely make $20 million for every single movie that they were starring in, that was based on them having a precious and rare skill set. And, you know, obviously they had their brand and their reputation, and that sort of thing is separately precious. But even for the folks who are doing the audio on the movies or who are doing the camera work on the movies, or who are doing the set design on the movies, in many cases, these were pretty well paid jobs. And a good chunk of the reason for that is that they were precious and rare capabilities. Well, if you look at what AI is generating for movies, it is not quite Hollywood level. But if you go back just a few years ago to the first diffusion model generated videos, to the multimodal models integrated with LLM capabilities generating videos, wow, have we made a lot of progress in the past three years? And so you have to ask yourself, where might we be next year? Where might we be in a year after that? And what we're seeing is companies betting on this technology being useful now and getting really powerful in the future. So, for example, Amazon, which has a large film and television production studio, at this point, they're one of the biggest players in the game. According to reporting by Reuters. At the Amazon MGM studio, veteran entertainment executive Albert Chang is leading a team charged with developing new AI tools that he said will cut costs and streamline the creative process. I mean, if you're a big movie studio at the high executive level, and you hypothesize that there is a technology out there that in the next few years could cut the cost of producing a movie by twofold, by tenfold, by 50 fold, who knows? You can't really afford to avoid that technology because if you sit out, all your competitors are going to jump in and you're going to be destroyed as a company. So the writers, the directors, the actors, the stick, you know, the crew, they're all worried about how AI is going to take their jobs. But the production companies, they have to use what works. They have to use what's economically viable. And I think thought some of the most interesting commentary on this actually comes from Rhett Reese, who was a writer and producer on the Deadpool movies, among others. And I think Deadpool and Wolverine was like the number one movie in the year that it came out. Probably certainly the number one R rated.
- [41:47]
B
Definitely an an action movie that like relates to.
- [41:49]
A
Exactly. So this guy is a big player on big movies that make a lot of money. And what's so interesting is that he posted on X. I hate to say it, it's likely over for us. That was his comment on that Brad Pitt, Tom cruise, just 15 second clip from sea Dance. And I'm gonna read his other posts because I think they're all a remarkable sort of moment in time about what folks are saying in Hollywood.
- [42:19]
B
Sure,
- [42:22]
A
in next to no time, one person is going to be able to sit at a computer and create a movie indistinguishable from what Hollywood now releases. True, if that person is no good, it will suck. But if that person possesses Christopher Nolan's talent and taste and someone like that will rapidly come along, it will be tremendous. And then one other thing he says, quote, I suspect could be wrong, that many screenwriters are using AI heavily in their writing and many execs are using it heavily in their analysis of writing. So hilariously, all the people are sitting back watching as AI critiques what AI just created. And I have to say, there was like a making of documentary for Stranger Things Season 5. And at one point they show the writer's computer and if you look at all the tabs they have open, there's a bunch of chatgpt tabs open. So even on like the number one show on Netflix, like AI is already contributing to the creative process. And at this stage, you know, it's mostly in the writing idea generation, but it's very easy to see the writing on the wall is going to be really amazing stuff. Okay.
- [43:37]
B
Yeah.
- [43:37]
A
The final quote I have to read from Rhett Reese because as I said, he's doing some amazing writing on this and it's a really interesting voice for the conversation. Quote a post to clarify, I am not at all excited about AI encroaching into creative endeavors. To the contrary, I'm terrified. So many people I love are facing the loss of careers they love. I myself am at risk. When I wrote It's Over, I didn't mean it to sound cavalier or flippant. I was blown away by the Pit v Cruise video because it is so professional. That's exactly why I'm scared. My glass half empty view is that Hollywood is about to be revolutionized, decimated. If you truly think the Pit v. Cruise video is unimpressive slop, you've Got nothing to worry about. Out. But I'm shook.
- [44:24]
B
Yeah.
- [44:25]
A
Wow.
- [44:26]
B
It's a good quote. Yeah. I mean, watching that video, I was pretty shook too.
- [44:33]
A
Yeah. So I think to connect that to the KPMG conversation. Right. The question is, is AI going to make people so much more productive and awesome, or is it going to make people worthless? Because everything that they do that is difficult and hard is now going to be easy. And I think the result for the consumers is going to be great. Right. There's going to be way, way, way more movies about, way, way, way more topics at a much higher level of production value. But for the producers, at a minimum, the business models are going to have to change. Right. The structures of. Of when can you ever justify making Avatar at a $300 million budget? It seems difficult for me to understand how those business models are going to survive. Maybe the industry will survive and the industry will just be different. There'll be a lot of smaller teams of people making pretty high quality stuff in partnership with AI. But it seems very likely to me that there is a moment of disruption headed to Hollywood. So way. Absolutely. Just like there's a moment of disruption headed for the auditors.
- [45:48]
B
Right, right.
- [45:49]
A
And. And the legal profession and all these other professions. Yeah.
- [45:52]
B
Well, these are all some really interesting stories, and I will continue to cover the economic impacts of AI over the coming weeks. But I want to move on to this, this last section we have on export controls, because on February 23, a senior Trump administration official told Reuters reporters that Deepseek's forthcoming model, which I think is coming out in the next few weeks, was trained on Nvidia's Blackwell chip chips. And as our regular listeners will know by now, these chips are and have always been illegal to export to China. But before we get into the smuggling implications of all this, can you tell us exactly what the official is claiming?
- [46:24]
A
Yes. So this is an anonymous official interviewed by Reuters. So they're just cited as a senior Trump administration official. We don't know who is the person here, but they're saying, saying, quote, we're not shipping Blackwells to China, or at least not shipping them legally to China. So this is a story of smuggling. I suppose it's also possible that it's a story of remote access in the cloud. But as written in the story, the implication is that the chips are in Chinese soil and it is smuggling. And this would be a violation of U.S. export controls. And according to the Reuters report, it goes on to say, quote, the U.S. believes deep seq will remove the technical indicators that might reveal its use of American AI chips, the official said, adding that the Blackwells are likely clustered at its data center in Inner Mongolia, an autonomous region of China. So again that's part of China, even though the name is Inner Mongolia. That's, that's a part of China. And this suggests that it is a smuggling story, not a remote access story. Now one other part of this is the distillation part of the story. So here is again is quoting from Reuters quote the model they helped train likely relied on the distillation of models made by leading edge US AI companies including Anthropic, Google, OpenAI and Xai. Echoing allegations made by OpenAI and Anthropic, the official added, end quote so here's the thing. Distillation to refresh folks memory. We've talked about this in the past. We talked about it when Deepsea, the 3.0 I think it was, was the January 2025 model that kind of blew everyone's socks off along with R1. And distillation is essentially think about the training data to create a model for the first time in the largest models today that training data is basically most of the Internet is the training data. Then the output of that training phase along with the the post training and yada yada yada is a model. Now you can then have a series of conversations with that model and then those conversations become a new set of training data. And because that new set of training data is going to be even if you have millions and millions of conversations with that AI to generate more training data data, it's still going to be a much smaller training data set than the whole Internet or most of the Internet. And so what you get is a model that has most of the performance of that first model but only a fraction of its computational cost to use at inference and only a fraction of its computational cost to create. Now every computation company engages in distillation, right? Anthropic is distilling its own models, Google is distilling its own models, OpenAI is distilling its own models and they're doing that to create smaller versions of their models that are cheaper to serve to customers. What Anthropic and others are alleging here and what this senior US administration official is alleging here is that deep SEQ actually the reason why they're having such high performing models is that they're successfully using distillation to sort of ride on the tailwinds or the slipstream of the research that companies like OpenAI, Anthropic and Google are Doing so. Anthropic has been public about this. Quote. I'm sorry. On February 23, Anthropic published an announcement saying, quote, we have identified industrial scale campaigns by three AI laboratories, Deep Sea Seek, Moonshot and Minimax, to illicitly extract Claude's capabilities to improve their own models. These Labs generated over 16 million exchanges with Claude through approximately 24,000 fraudulent accounts in violation of our terms of service and regional access restrictions. Now, what they mean by that is terms of service. You're not allowed to distill Anthropics models without their permission and regional access restrictions. These models aren't even supposed to be able to be accessed from China. So they're using VPNs or something else to do that.
- [50:52]
B
Yeah. So can you walk us through some of the most significant implications if these claims both about smuggling and then about distillation, are accurate?
- [51:01]
A
Yes. So I think there's a couple things going on. Number one is it does show that China still wants America's advanced chips. And Saif Khan, who has been on this podcast before and is now at the Institute for Progress, he told Reuters, quote, quote, Chinese AI companies reliance on smuggled Blackwells underscores their massive shortfall of domestically produced AI chips and why approvals of H200 chips would represent a lifeline. It also shows, you know, as. As Chris McGuire, who's also come on this podcast, pointed out that China's companies are absolutely willing to brazenly violate U.S. export controls. So why would we trust them, you know, to. To abide by export control restrictions such as not providing capabilities to the Chinese military, for example. So I think the next thing that Chris McGuire said, and this he said on X is worth quoting at length. Quote. When Deepseek releases its new model and claims to have trained it from scratch using 2000 H800 chips, hopefully the world will recognize that as a lie. The fact is, Deep Seek is almost entirely dependent on banned American technology and ip. It is trained in its models by illegally using US chips and illicitly stealing US ip. These actions must have consequences. And I think Anthropic said something that was kind of insightful. They were commenting on what this means for the value of the export controls. So here's what their announcement said. Quote. Without visibility into these attacks, the apparently rapid advancements made by these labs, talking about Chinese labs, are incorrectly taken as evidence that export controls are ineffective and able to be circumvented by innovators. In reality, these advancements depend in significant part on capabilities extracted from American models and executing this extraction at scale requires access to advanced chips. Distillation attacks therefore reinforce the rationale for export controls. Restricted chip access limits both direct model training and the scale of illicit distillation. So that, I think, is a really interesting point here, which is to say, say, obviously, if we could wave a magic wand and make China unable to create its own models and also wave a magic wand to prevent distillation, that's sort of the ideal outcome. But what Anthropic is pointing out here is that effective chip export controls are still useful even in a world where distillation exists. Because even though distillation is far computationally cheaper than training a model model, it's still computationally intensive. But I want to say something else on this story, and it's something that I wrote in my report on Deepseek back when it came out in March 2025, and that was, we're talking about the fairness or the unfairness of distillation and whether or not they're stealing American ip. And I think we just need to be honest with ourselves that they're gleefully stealing American ip. And the question about distillation, it's not really about whether or not it's fair. It's about whether or not it's going to work and whether or not it's going to continue working. And what the strategic consequences of that are. We kind of need to ask ourselves, is AI as an industry more likely like developing advanced fighter jets, where China literally stole the blueprints over a decade ago, and they still can't make a jet as good as the F35 that they stole the blueprints for? And that's because a lot of the secret sauce is not just in the blueprints ip, it's like in the organizational, tacit knowledge of the people who build the F35. So even if you steal the IP, you haven't really stolen all of the secret sauce, although certainly gives you a leg up. So do AI models look like that, or do AI models look like pharmaceuticals, which, like, if you didn't have patent protection, no one would invest in pharmaceuticals, because once you know what the molecule is, you can make your own copy and you can incur, like, literally 1, 1 millionth of the research and development costs, and you can get all of the benefit in terms of it. And that's just because. Because it's really, really hard to keep the secret of the drug when you're putting it in the hands of potentially millions of consumers around the world. And so we rely upon fair play and intellectual property restrictions to make pharmaceuticals a viable business model. Well, as we've said, as the Trump administration has said, AI is like the space race and boy, oh boy, did the Soviets not respect our intellectual property in the space race case, nor vice versa. Right. If we could steal stuff from the Soviets, we would happily use it. I, I don't think that actually ended up being a, a major part of certainly not the moon program. But you know, in principle we would have been happy to, to, to do so. You know, American policy is that we try not to use industrial, we try not to use espionage for commercial advantage, but for military relevant sectors, heck yeah, we'll do that. So China's going to keep on trying to distill and I, I, I, you know, Chris McGuire is advocating for sort of coercive retaliation to try and persuade China to put a stop to the behavior of its companies. Anthropic is advocating for more strict enforcement of the export controls to try and do it. But I am unfortunately a little bit pessimistic that either of those would actually do it. And the kind of, the question becomes, can you make distillation not work? And in talking with folks at the labs, you know, what I've heard is, is so far at least, the current state of technological play is that you can put in safeguards that raise the cost of sort of distillation attacks by a factor of 10. But if that's raising the cost from like 2 million to 20 million, whereas like American AI companies are spending hundreds of billions of dollars to train these models, to build the infrastructure to train these models, boy, does that still make distillation attractive. And this comes back to, you know, what we talked about on the last podcast episode about Michael Kratzios speech in New Delhi where he was effectively saying, you know, America wants to export chips, we want to help you make your sovereign AI. And I said that one of the plausible interpretations of why the United States might take that position is that they actually think that the business model for closed AI models is not that good compared to open source. Especially not if that open source is just ripping off American IP with distillation. Like if Chinese companies were just happily willing to steal American pharmaceutical recipes and violate all of our intellectual property, the business model of American pharmaceutical companies would still work, but only until, you know, like, once China starts saying like, hey, every country on earth will sell you American researched and developed pharmaceuticals at a tiny fraction of the price, a lot of countries are going to take that deal. And that's a very dangerous moment. For the US Pharmaceutical industry in that scenario, and that might really be the scenario that the AI industry is in. So I do agree with strengthening enforcement of the export controls. I do agree with agree with exploring what are punitive measures that we could take against China, and I certainly agree with continuing the line of research about how can we make models more resistant to distillation type attacks. But I also think companies and countries need to be thinking about what's the optimal strategic position if distillation continues to work, because that's a plausible future at a minimum.
- [59:15]
B
Interesting. Well, I'm glad we were able to talk a bit about export controls and distillations today because we haven't talked about export controls, I think in over a month, which is a long time for us.
- [59:24]
A
Everyone's shocked out there.
- [59:26]
B
People are praying for it. Oh my God, thank God we have export controls. But yeah, thanks for joining me despite the probably brutal jet lag you're experiencing from coming back from a trip to India. And thank you to the audience for tuning in.
- [59:39]
A
Thank you Matt. Thanks for listening to this episode of the AI Policy Podcast. If you like what you heard, there's an easy way for you to help us. Please give us a five star review on your favorite podcast platform and subscribe and tell your friends. It really helps when you spread the word. This podcast was produced by Sarah Baker, Sadie McCullough and Matt Mand. See you next time.