← The AI Policy Podcast
The AI Policy Podcast
White House Greenlights H200 Exports, DOE Unveils Genesis Mission, and Insurers Move to Limit AI Coverage
00:52:55
We talk White House’s decision to export H200 chips to China, the One Rule and Genesis Mission executive orders, and major insurers relationship with AI.
Loading summary
Transcript48 lines
- [00:00]
A
Foreign.
- [00:09]
Policy. Welcome back to the AI Policy podcast. In this week's episode, we'll be discussing several recent developments coming out of the Trump administration, including a reported decision to allow H200 exports, updates on the preemption executive order, and the Genesis mission. We'll then close out by discussing how insurance companies are thinking about AI. I'm Matt Mand and I'm here with Greg Allen. Greg, good to be speaking with you again.
- [00:31]
B
Great to be speaking with you again.
- [00:34]
A
So, Greg, we recorded a whole episode this morning, but we're having to go back and add in this section because while we were recording, Semaphore reported that the White House has instructed the Department of Commerce to allow H200 exports to China. Can you start by giving us some context here? What is the H200 chip and how do we get into this scenario?
- [00:52]
B
Yep. So H stands for Hopper. So that is the best of the last generation of Nvidia chips. We're currently in the well era and the Hopper is still a great chip and the H200 is the best Hopper chip that money can buy. So we started with the Trump administration banning the sales of the H20, a hopper generation chip that is not so good for training AI, but is pretty good for inference. And then they unbanned it, allowed those sales, but then China banned it as part of a negotiating gambit to get the Trump administration to sell them even better chips. And, and now it appears that that gambit has worked. The Trump administration did sell that they were not did say that they were not going to sell the most advanced chips, the Blackwells. That is something that Nvidia had asked them to do that the Trump administration and President Trump himself very publicly said was not going to happen. But last week at csis, we hosted Nvidia CEO Jensen Huang, who was sort of making the case for allowing advanced AI chip exports to China. And earlier that same day that he came to cs, he was actually meeting with President Trump to make the case for allowing H200 exports. And there is another great paper out from IFP, the Institute for Progress, authored by Saif Khan and some of his colleagues, including George Adamson, who we had on the podcast Talking about the B30, the Blackwell generation chip that Nvidia wanted to sell. And surprise, surprise, almost all of the arguments for not selling the Blackwells also apply to the H200. This is giving China a massive degree of AI computational capability that they would not otherwise have. It's going to give Nvidia a short term injection of revenue, but it is not going to change China's mind about getting off of Nvidia as fast as possible. I don't think I'm surprising anyone who listens to this podcast when I say that this is a strategic mistake. This is going to help advance China's AI efforts to a significant degree, and it is not going to lead to the outcome that the administration is seeking in terms of strengthening Nvidia and sort of weakening China by making them dependent upon the Nvidia stack. I would not expect to see a single change in any Chinese indigenization policies as a result of this. They're still going to move ahead with all of that. So I was testifying before the Senate Foreign Relations Committee alongside a great group of folks, Chris Miller, the author of Chip War, Tarun Chabra, who now runs National Security Stuff at, and James Mulvanen, who's at Pamir. And we were testifying before the Senate Foreign Relations Committee and basically pointing out everything that I just said about what China wants, what China is pursuing, the means through which they are pursuing it, and how this is really not in America's best interest. Overall, I think one thing that I am heartened by that came out of that hearing is that there is now draft legislation called the Safe CHIPS act, which was introduced by a bipartisan group of senators led by Senator Ricketts, a Republican of Nebraska, and Senator Coons, who is the ranking member on the Senate Foreign Relations Committee and the chairman of the the relevant subcommittee, whatever the opposite of respectful, respectively is. Coons is the the ranking member on the full committee. Ricketts is the chair of the subcommittee that we were testifying before and that a pretty interesting hearing. There was, I think, five Democrats, five Republican senators, if my memory serves correct correctly. And there was unanimous agreement that the United States should not be in the business of transferring this advanced technology to China. To just say one thing that I said during my testimony.
- [04:57]
The Trump administration in the AI action plan, which is still sort of the definitive policy document on AI put out by this administration, said that the race with China in AI is the equivalent of the space race during the Cold War. Can you imagine if we were selling rocket technology to the Soviets while we were trying to beat them to the moon? How could that help? In what scenario does that possibly help?
- [05:24]
It just doesn't. And I think that is really sad that where we are now with the AI race with China is we're going to be giving China the equivalent of Gatorade and Nike super shoes at the same time that we're trying to race against them. A. It's a really unfortunate state of circumstances, but this congressional. Congressional legislation would effectively take the decision out of the Trump administration's hands, would put the technological restrictions, basically saying above what performance thresholds chips are and are not allowed to be sold would be a congressional decision and not a executive branch one. I hope that legislation moves forward. I expect it'll be good. But I think this move on the H2 hundreds is a very interesting test of whether or not it was a negotiating tactic by China. Because if China is to hypothetically ban the purchases of H200, which I do not expect them to do, but let's just say hypothetically they were, I think what that would show is that of the various constituencies in China, you know, you have folks like Alibaba, Tencent, Baidu, they want to be able to buy Nvidia chips. They want to be able to buy as many as they can, as fast as they can. But then you have other constituencies in China like Huawei, like Cambercon, like smic, and they actually do support banning Nvidia chips because they're thinking about what's best for their companies, not what's best for China's overall AI ecosystem. And so in that regard, you'll see which constituency really has the most pull and power over the Chinese government. If they ban the purchases of the H200s, that would mean that Huawei's concerns and priorities over indigenization are in ascendant. And if they allow the sales of the H2 hundreds, that will mean that indeed, banning the H20 was just a negotiating gambit to try and get the Americans to offer up something better. Well, here is something better. We're running the experiment. I'm officially predicting that China is going to allow the H200 imports, but we'll learn a lot based on whatever their decision is either way.
- [07:30]
A
Yeah, well, as you mentioned, Jensen Huang, the CEO of Nvidia, was here at CSIS the other week. What were some of the arguments he made in favor of allowing chips like the H200 into China, and what are your reactions to those arguments?
- [07:42]
B
Well, I think it's worth putting in the context of his overall presentation. So the interview was conducted by our CEO John Hamry, who has led CSIS for 25 years. And before he did that was he was the Deputy Secretary of Defense. So when you come and speak at csis, you really are speaking to the Defense National Security, Foreign policy establishment of the United States. And as Nvidia CEO Jensen Huang has said some things in the past Several months that have really set off that community. For example, he said that being a China hawk should be a badge of shame, meaning taking China as a serious threat to US national security should be viewed as a badge of shame. That set a lot of people off, including a lot of Republicans and conservatives in Congress, as well as conservative media personalities like Steve Bannon, who said that. I think, I think Steve Bannon accused Jensen Huang of being a Chinese agent after, after those remarks. So he was dealing with that, the pushback against his remarks there. And then there was also the, his remarks to the Financial Times in which he said that, quote, you know, China will win the AI race. China will win the AI race. So you can imagine that that's not something that the United States wants. As the AI action plan said, you know, losing that race would be a disaster for the United States. And so Jensen was trying to say that the AI stack is like a five layer cake. And the bottom of that cake is energy. Then there is chips on top of that, then there is infrastructure. And he uses a pretty broad definition of infrastructure, which includes not just CUDA software, but also all of this networking and physical infrastructure on data. Then above that is foundation models, and finally at the top is applications. So he was kind of walking back his China will win argument by emphasizing what are China's strengths at every layer of that five layer cake that he mentioned. So China really stands out as a leader in energy. Even though Chinese GDP is officially smaller than that of the United States, they actually produce about twice as much electricity as the United States. They're very strong. And Jensen specifically said, you know, the United States, it takes us years to build a new great data center and the associated energy infrastructure. And China can build a hospital in a weekend, which I do think is noteworthy and actually true. I mean, these are real advantages that China has. Then he said, in the AI models, we're leading, but only in closed source. In open source, he would say that China is leading. And then finally in applications, he thinks China is very aggressively driving the adoption of various use cases and applications. And that's another advantage. And then a final advantage that's not part of that five layer cake, but sort of threaded into all of them is AI talent. He says that something like half of the world's AI developers are actually Chinese and based in China. So for all of those reasons, he thinks China is a formidable competitor, but also a market that the United States cannot afford to avoid. And there I think that it's just China is a different kind of market than Other markets. The reason why Nvidia is doing better than AMD in the United States, Europe, most of East Asia, Latin America, Africa, etc. Well, those don't have the industrial policy backing of China. China has said in made in China 2025, a policy that goes back to 2015, and I think it was document number eight, which came out in 2020, which advocated spending unlimited amounts to pursue semiconductor self sufficiency. And then finally in the 3, 5, 2 policy which specifically mandated that China get off of foreign advanced semiconductor technology for all government enterprises and for all state owned enterprises within three years. So we have already run the experiment of what happens when you're willing to sell all of your most advanced chips to China. We had that policy in 2015, we had that policy in 2020. And in both cases, China's policy was get off of foreign technology as fast as possible and spend whatever it takes to do so. So we've already run the experiment and China is not willing to be addicted to American technology and they're willing to spend vast amounts of money, they're willing to force customers to adopt Chinese technology, and they're willing to embark upon extraordinary state backed industrial espionage. So the toolkit that helps Nvidia win in Europe, that helps Nvidia win in Latin America, that helps Nvidia win in Japan. It's just not an apples to apples comparison when you're talking about China. And the reality is, as I said in my testimony, all we are doing is building China a bridge to get them to the future that they have already said will not include the United States. And I think the wiser policy is to not build on that bridge, to not make it convenient and easy to decouple with the United States on their schedule, on their terms. But instead, if they're committed to decoupling with the United States, we should make that as expensive and complicated as possible. And I wish that this wasn't, you know, China's policy. I wish that that didn't mean that we needed to have this policy. But since Xi Jinping took power in 2012, since he made himself effectively dictator for life in 2017, it's just a different ball game and we need to wake up that the tactics that we're going to use need to be grounded in that reality. I mean, consider companies like Tesla, which have gone into China and have had no export controls. And now effectively Tesla has built a supply chain that is in the process of enabling all of these Chinese competitors who are putting Tesla out of business. Nvidia says that they have been really hurt by these export controls because they've lost this revenue opportunity. But the reality is they're being hurt by Chinese policy and it is export controls that have that saved Nvidia in China. As I mentioned in my testimony.
- [14:17]
Huawei was already planning on introducing a 7 nanometer GPU in 2020. Even before Nvidia was planning on introducing a 7 nanometer GPU, they were both going to design it in house and have it manufactured by contract at TSMC of Taiwan. It is export controls from the first Trump administration that blocked Huawei's access to TSMC and stopped them from having a 7 nanometer GPU then. And when Huawei then turned to domestic manufacturers like SMIC as an alternative to tsmc, it is export controls that blocked their purchases of advanced semiconductor manufacturing equipment and thus prevented them from being able to substitute for tsmc. So Nvidia says that they're being hurt by these export controls. The reality is the last years, the last five years of Nvidia dominance in China on the AI chip market have been underwritten by export controls, which prevented Huawei from being a vastly stronger competitor than they would have otherwise been and prevented them from getting access to tsmc. So that is what a policy of, hey, China, if you want to decouple from us, we're going to make that expensive and complicated. Looks like it's the one that Nvidia has been benefiting from. And for them to now say that, you know, export controls should apply to my suppliers but not apply to me, I think it's just unwise and disingenuous and not going to help America's national security or foreign policy. So I'm disappointed by this development, but I am hoping that Congress can take measures through the Safe Chips act to take measures into its own hands and reassert some control on this issue.
- [16:03]
A
Yeah. Well, you talked a little bit about where Nvidia would be without export controls. I was just hoping you could talk a bit more about where the US Would be in the race with China without export controls, particularly as it relates to data centers.
- [16:16]
B
Yeah, and I think this is a great point. Thank you for raising it. If you look at the largest data centers that exist this year, you know, we're already in the hundreds of thousands of chips era per data center. And that's up from, I want to say, GPT3 was trained on maybe 10,000 chips of several generations ago. Now we're in the era of hundreds of thousands of chips. By the end of 2026, we're going to be in the era of 1 million chip data centers, and those are going to be beefy Blackwell chips.
- [16:50]
If we hadn't had this export control policy, I think the largest AI data centers in the world would probably already be in China. In this era where Google, OpenAI, AWS, Microsoft are all spending, like Apollo Moon program levels of capital to build AI data centers. The reason why the Chinese hyperscalers haven't been doing the same thing is that they don't have viable chips to buy because of these export controls. In a parallel universe where we had never adopted AI chip export controls, given China's ability to rapidly construct new energy generation facilities, given their ability to rapidly build data centers, I think right now in 2025, we would already be in a situation where China has the largest AI supercomputers on earth. And because of that, we know compute is such an advantage in model training and development and even in innovation, because the more innovation you want to have, the more experiments you need to run, the more experiments you want to run, the more computing facilities you need to have, because those experiments cost computing resources. And so China might literally be ahead in both computing resources and model quality, which means they might also be ahead on global adoption. So when we give these Nvidia AI chips to China, we are effectively arming Chinese AI companies to go up against OpenAI, Google, Anthropic, et cetera, in all of the global markets. So the idea here is that we're going to get the world addicted to the American AI stack, but actually what we're doing is putting China in a position to get the world addicted to their AI models and their AI applications. And I just don't think it's going to go well for us, frankly.
- [18:52]
So that's the unfortunate state of affairs that we're in now with this new decision.
- [18:57]
A
We'll follow that state of affairs closely over the coming weeks. But in another big announcement from this morning, just a few hours before recording this, President Trump posted on Truth Social saying he'll be signing an executive order preempting state AI laws sometime this week. We're currently recording this on December 8th. So for those interested in what might end up in the executive order, I highly recommend you check out our analysis of a leaked draft in our November 21st podcast. Again, that's a leaked draft, not the final draft. But, Greg, what exactly did President Trump say in his post today on Truth Social and what has changed since we last spoke?
- [19:33]
B
Yeah, so I think, as always.
- [19:37]
You gotta just quote from the post itself, because it is monstrously quotable. So this comes from President Donald Trump in that truth social post. There must be only one rulebook if we are going to continue to lead in AI. We are beating all countries at this point in the race. But that won't last long if we are going to have 50 states, many of them bad actors involved in rules and the approval process. There can be no doubt about this. AI will be destroyed in its infancy. Exclamation point. I will be doing a one rule executive order this week. You can't expect a company to get 50 approvals every time they want to do something that will never work. Exclamation point. So my assumption here is that that draft leaked executive order was legitimate and was reasonably close to whatever the final text of that executive order is going to be when the real thing comes out. We'll of course talk about it more. But I think the big context here is that there is no preemption in the National Defense Authorization act, which, you know, was an attempt made by some folks in Congress. So the fact that that does not going to be included in the National Defense Authorization Act. We thought, I mean, you and I, Matt, speculated on our last podcast that the executive order might be a backup and even potentially a source of negotiating leverage for the administration if that NDAA clause fell out. It has fallen out. And now the Trump administration is looking to move forward. I do kind of wonder, like, are we going to see lawsuits from states attorneys generals? Are we going to see any other kind of backlash to this kind of a story? I have a tough time, you know, imagining that California, Colorado are going to swallow this unchallenged. It's a, it's a pretty interesting moment for the balance of power between states and the federal government. But here's the thing. Even if the Trump administration loses in court, takes a long time to lose in court. And so they, I am sure, will have an impact on all state legislatures in the meantime, both the ones who are implementing laws they have already passed and the ones who are drafting laws and contemplating how to move forward. And when those lawsuits ultimately come to a head, there's going to be multiple debates going on simultaneously. Like, for example, what is the nature of the interstate commerce clause as it applies in this instance? What is the nature of the balance between federal and state power in the nature of this instance? And so it's an important moment in AI regulation, no matter how this turns out.
- [22:22]
A
Yeah, I think for those who don't remember when we had the moratorium and the one big Beautiful bill. I think there was a coalition of about 40 attorneys general who signed a letter opposing that. So that gives you a sense of the magnitude of opposition to preemption of state AI laws from the state attorneys general. So it'll be interesting to see.
- [22:41]
B
Thank you for raising that, Matt. Great point. And I think just one thing I'll add is it's not like those 40 states are all run by Democrats, like most state governments. Majority of state governments are run by Republicans, some of them with single party control. And so this, this policy is controversial on a bipartisan basis. But ultimately, as I said, I think it's going to be resolved in the courts and that could go either way.
- [23:09]
A
Right. So as we said, the preemption provision didn't make it in the ndaa, but another interesting AI related provision did. The provision creates an AI Futures steering committee with several responsibilities. Who will be on the steering committee and what exactly are they expected to accomplish?
- [23:27]
B
Yeah, so normally I don't always talk about stuff that is in the NDAA before it passes because I like to wait until it's actually finalized and then tell you what's real as opposed to speculating on what is. But I thought this was just too interesting to gloss over.
- [23:44]
So the people who are going to be on this, it's going to be co chaired by the Deputy Secretary of Defense, AKA Deputy Secretary of War and the Vice Chairman of the Joint Chiefs of Staff. So it definitely has a strong national security type of bent. But then there'll be a number of other DoD officials and representatives and the responsibilities is where it gets super interesting. It's focused on AGI, artificial General Intelligence preparedness and adoption. And I think it's just worth quoting from the provision here, quote Formulating a proactive policy for the evaluation, adoption, governance and risk mitigation of advanced artificial intelligence systems by the Department of Defense that are more advanced than any existing artificial intelligence systems, including advanced AI systems that approach or achieve artificial intelligence. Approach or achieve artificial general intelligence. And then it goes on to give two other key things. Forecasting. So here's what it says about forecasting. Quote Analyzing forecasted trajectory of advanced and emerging artificial intelligence models and enabling technologies across multiple time horizons that could enable artificial intelligence. And then the other one is adversaries AI capabilities. Quote Assess the possible technological, operational and doctrinal trajectories of adversaries of the United States with respect to the uses of AI capabilities by such adversaries across various time horizons, including any pursuit or development by such adversaries of artificial general intelligence. And they have to meet every three months and not less than every three months and submit a report to the congressional defense committees by January 31, 2027. That is pretty interesting. When you are basically ordering these individuals to spend a good chunk of their time thinking about artificial general intelligence, competition with China, thinking about how they're going to win. It's a pretty interesting directive. It's not telling them what to do, but it is telling them what to focus on. And it's a pretty interesting instruction in that regard, a pretty interesting signal from Congress in terms of what they think is important.
- [25:53]
A
Nice. Well, we'll be keeping an eye out on both that preemption executive order as well as that provision in the NDAA as both of those stories develop. But in the meantime, let's talk about another significant AI related executive order coming out of this administration. On November 24, the Trump administration launched the Genesis Mission. Greg, what exactly is the Genesis Mission and also what is its goal?
- [26:17]
B
Great question. So I think this mission is still in the early stages and they have a general course and heading, but I think a lot of the specifics have yet to be determined. So you know, what I'm saying is what's available publicly at this stage, which is the high level concept for the mission. However, they acknowledge in the executive order that they're going to pursue an appropriation from Congress. And if that appropriation from Congress is $30, we'll learn a lot about the scope of the Genesis mission. And if that appropriation is $300 billion, we'll learn a lot about the scope of the Genesis mission. So the basic point is that a lot of the future is not yet written. Here's what is written so far. To begin, I'm just going to quote from the the new website, which is snazzy and has a bunch of nice AI generated visuals, which I think is a something that you haven't seen from that many government websites, certainly not in the a year before today. But here's the quote, the Genesis Mission is, quote, a national initiative led by the Department of Energy and its 17 national laboratories to build the world's most powerful scientific platform to accelerate discovery, strengthen national security and drive energy innovation. And the goal of the mission is to, quote, double the productivity and impact of American research and innovation within a decade. And how is it going to do that? Quote the Genesis Mission will create a national discovery platform that unites the world's most powerful supercomputers, AI systems and emerging quantum technologies with the nation's most Advanced scientific instruments. It goes on to say it will generate a new class of high fidelity data to train advanced AI models, empower researchers to solve the hardest scientific challenges and accelerate discovery from years to months. So here's how I interpret all of those things coming together. We have a few existence proofs in the scientific literature where AI has made it possible to do things faster and remarkably, in a way that was not true before the sort of current generation of AI. I think an obvious example here is AlphaFold, which was using artificial intelligence to sort of unlock the problem of protein folding. It used to be like a PhD to determine how a given genetic sequence created proteins and what the structure of that protein was going to be once it all folded. AI took the sort of library of all the folded proteins that we had identified and what their structure was, and then, you know, by applying artificial intelligence to that training data set, was able to create a model that could take any given DNA sequence for a protein and predict what its structure was going to look like in three dimensions. So that is an area where like there was this really important subset of scientific discovery that was meaningfully, dramatically enhanced in productivity by a astute application of AI capabilities to that problem. Other biological models, including biological foundation models, which we've talked about in a prior podcast, are likewise showing really interesting results.
- [29:53]
Now, not all of this is large language models type technology, but the basic point about like foundation models in different disciplines of science, broader application of massive computing resources with AI capabilities to science, is sort of showing that certain types of things.
- [30:12]
Certain categories of discovery and of productivity are possible now in a way that was not true 10 years ago. So how do you energize the entire scientific establishment of the United States to take advantage of that? You know, to a certain extent they already are. You know, there's really cool stuff going on in the Department of Energy related to material science related to fusion. Notably, DeepMind had some work on how to contain and preserve fusion reactions so that they are stable for long period, longer periods of time. That was using artificial intelligence, machine learning. And the basic point here is I think that the, the Trump administration is directing the Department of Energy. Just throw a brick on the accelerator of all of that work. And I think back to, for example, when the Trump administration released its first skinny budget, before they had the sort of full President's budget request, Something that really stood out to me was in the National Science foundation budget request, there was like, cut, cut, cut, cut, cut, cut, cut. But by the way, none of these cuts shall apply to AI stuff. Right. Which just shows you that there's this sort of overarching theory of the case in the Trump administration as it applies to science, that the AI stuff is important, worth preserving, worth doubling down on. And here I think that's obviously the case with the Genesis mission. So I've talked about, like, what existence proofs we have in the scientific literature thus far that demonstrate why this opportunity appears to be out there, why there appears to be this opportunity worth seizing. And then I think it's worth, you know, saying that the Department of Energy has a number of areas where they're interested in pursuing this, and that's stuff like fusion, energy, materials science, grid resilience, molecular chemistry, sort of on and on and on this list of scientific domains where they think there's an opportunity to make a meaningful, perhaps doubling the productivity of the overall scientific enterprise by applying AI in clever ways. And I don't think they come with sort of pre existing hypotheses or not hypotheses, but like, they do have hypotheses, but I don't think they come with, like, really strong opinions on what applies. Where, you know, there's a lot of different terms of art here, like using AI agents, like automating various capabilities, like applying these AI foundation models. Basically every single area in which somebody has proposed how AI might include scientific productivity shows up on this website somewhere. And so what actually is going to be applied? Where I think that's going to be determined probably at the discretion of the Energy Undersecretary Dario Gill, who is leading the Genesis mission from the perch of the Department of Energy, and the Director of the White House Office of Science and Technology Policy, Michael Kratzios, who's leading it from the perch of the White House. So it's a, it's a pretty interesting undertaking. I have talked to folks in the Department of Energy who have told me that by you know, moving money around in existing programs, there's going to be something like $200 million available for this initiative right away.
- [33:31]
That still leaves you asking, you know, what is the, what is the real final size of this program? Which, as I said, is going to depend upon congressional appropriation.
- [33:40]
A
Yeah. Well, that brings me to my next point. The administration has repeatedly compared the Genesis mission to the Manhattan Project as well as the Apollo program. Are these comparisons, like, accurate? Is it at that scale?
- [33:51]
B
I think the honest answer is that we don't know. It is certainly not at that scale just based on the reprogrammed funds that they have right now, whether or not it could become the Manhattan Project scale or the Apollo Program. Scale really depends on Congress and how much money they want to kick in. So, you know, I mentioned in my Senate testimony last week that the cost of the Apollo Program was in the tens of billions of dollars between 1960 and 1973. And then if you adjust all of that for inflation, you know, you get to something like $300 billion. Well, right now Genesis has like $200 million. And that's a thousand times less, right? More than a thousand times less than what the Apollo Program had over the entire life of that 13 year program. So if you want to get to Apollo scale, you know, you're going to need a lot more money. Maybe not exactly as much as Apollo, but certainly a lot more. Now, one thing, I guess to be fair to the administration is that they announced a lot of partners in this initiative. Companies like Amazon, like OpenAI, like Google, others, all listed as named partners in this initiative. I guess I should probably just read the whole list here. Right? So, AWS, AMD, Anthropic, Google, Microsoft, Nvidia, OpenAI, Oracle Scale, AI. So those are companies that are dropping hundreds of billions of dollars in AI investment. Those are companies that are investing at the scale of the Apollo program. And so I guess we don't really know to what extent it is appropriate to grab the activities and initiatives of those companies and lump them under the Genesis mission. You know, these companies were already planning on spending hundreds of billions of dollars before there was a Genesis mission. So what part of their activities will be new and additional because of the Genesis mission? We don't yet know at this stage.
- [36:05]
A
Yeah, well, it'll be interesting to see how that plays out, especially the appropriations side of things. Let's move on to our last topic here.
- [36:12]
B
Can I say one more thing? And.
- [36:15]
I mean, actually this is your point, so you're the one who should make this. But I think there's a issue on the concreteness of the goals around the Manhattan Project, the Apollo Program and the Genesis mission. So, Matt, you made a good point here. You want to just elaborate on what you think is the difference in the goals between these various initiatives.
- [36:34]
A
Yeah, I think for the Manhattan Project, there was a clear goal to develop nuclear weapons. The Apollo Program had a clear goal to land on the moon. Now, the Genesis mission has the goal of doubling scientific productivity. That is a hard goal to measure. And so it'll be hard to know whether this mission succeeds. I mean, in some sense we can know based on progress in science, but in another sense, there's no clear race towards a specific target here. I mean, they frame it as a race with China. I guess we could be racing with China to doubled scientific productivity, but it's definitely not as clear as the past programs.
- [37:09]
B
Yeah. And there are metrics out there that have sort of attempted to codify declining scientific productivity. There's a very famous paper, the name of the author escapes me, but the title is Good Ideas are Getting Harder to Find. And it basically just says that like, wow, when you invent the wheelbarrow for the first time as a caveman, like planet Earth's productivity goes way, way up with the invention of the wheelbarrow. And you know, you're like tripling the productivity of every laborer on Earth effectively. It's really hard to triple the productivity of every laborer on Earth. And so, you know, as you keep reaching up for new ideas, it's harder and harder to find equivalently good ones as the sort of low hanging fruit that you picked. Now, the various metrics that are used there to codify the quote, unquote, amount of scientific productivity, there's not universal agreement in the field on what this is. I assume this being the Trump administration, that they're not targeting twice the number of published papers. I think LLMs have already delivered that. And I think to the chagrin of the scientific reason, terrible papers written by LLMs.
- [38:17]
A
Right.
- [38:18]
B
So, yeah, like what is the scientific productivity that we're doubling? It probably is going to be different in every different field. You know, like I said, they're going after molecular chemistry, they're going after biology, they're going after nuclear fusion. So what constitutes success? Maybe we'll get more specified metrics as more of this undertaking gets specified and laid out in terms of a roadmap. But I think you make a great point, which is, man, it was really convenient that there was one unified, universally agreed metric for success on the Apollo program. Something that's a story. I don't know if it's apocryphal or if this actually happened, but there was a story of when John F. Kennedy was visiting some NASA facility and he saw a janitor doing something. And he asked the janitor, you know, what are you doing, sir? And the janitor responds to jfk, I'm doing my part to help put a man on the moon, right. Which is like in his case was making sure that everybody else had a clean workspace so that, you know, they could be maximally productive as they were putting a man on the moon. And the fact that like everybody from the janitor to the head honcho understood that what we're working on here is putting a man on the moon that was such a unifying and conveniently unifying mission and vision and doubled scientific productivity. I think it's a worthwhile goal. I think we should pursue it. I think it's just hard to measure and perhaps hard to unify everyone around.
- [39:52]
A
Yeah, and it's not a goal that the issue here isn't isolated to just the Genesis mission. I mean, the goal of developing artificial general intelligence to win the race against China is also an even more perhaps vague goal.
- [40:05]
B
No, I think that's a, I think that's a acceptably specific goal, even though I know there's like violent disagreement on what it means. But part of the reason why there's violent disagreement on what it means is the fact that so many lawsuit outcomes depend on what it means. So of course they're going to disagree on it. I actually think, you know, as smart or smarter than a human on everything that a human can do is, is good enough for, for my purposes to understand what we're talking about.
- [40:27]
A
Cool. Well, let's move on to our last section. On November 23, the Financial Times reported that several major insurance companies are seeking to exclude AI related risks from their corporate policies. This is pretty interesting news that has several big implications for AI governance. But before we get into those implications, could you just start us off with a crash course on the underlying logic of insurance here as it applies to a high risk technology such as AI?
- [40:53]
B
Yeah, I think this is really, really interesting and really, really important. So folks who are interested in AI, I could forgive them if they got into this so that they wouldn't have to learn about insur insurance. But surprise, it turns out insurance is crazy important on planet Earth and in the economy and in AI specifically. So let me start by just pointing out like why insurance matters in regulation. Here in the United States everybody is familiar with like the fire code and like at your office, at your school, I'm sure you've done fire drills at various points in your life and there's like a fire exits and all these kinds of things for fire safety. You know that. What you might not know is that there's like almost no fire safety regulation being handed down by governments forcing people to do that. Almost all fire safety regulation in the United States is coming from insurance companies who are basically saying you cannot get insurance from this building unless you follow the fire code. Which like a bunch of insurance companies and a bunch of fire experts sort of convened and created this thing called standard, you know, the fire code for like, what it means to build a building that is safe and resistant, you know, to fire. And then what governments do is they will sometimes say something like, you're not allowed to build that building unless you have insurance. Or maybe they won't say that explicitly in regulation, but they'll do other things that basically make that obviously the correct and only acceptable financial choice. And so the basic point here is that like, insurance companies, not just in fire, but in like a million other industries are like pseudo governmental entities in terms of their de facto importance for regulation, standard setting and governance and all of that stuff. So it matters a lot in AI what insurance companies think and what they're doing. And as you know, we, we learned from this Financial Times article what insurance companies are trying to do is to get out of the business of insuring AI, because to them it is just a big, big challenge that they are not sure that they can effectively get enough premiums, basically insurance company revenue, to justify the, the potential risk they are taking on by ensuring the activities of AI companies. So like, what are they trying to exclude and why are they trying to exclude it? I mean, they're trying to seek exclusions from deployer insurance policies. So this is companies like aig, Big important insurance company, Great American WR Berkeley, are among the insurance groups that have recently sought permission from U.S. regulators to offer policies excluding liabilities tied to businesses deploying AI tools, including chatbots and agents. So it's like, I still want to offer you an insurance policy, bank of America, for all these things that might go wrong. Or I still want to offer you an insurance policy, Exxon Mobil, for all these things that might go wrong. But if things are going wrong specifically tied to your use of chatbots and agents, like, I'm not going to reimburse you for whatever costs are incurred related to risks associated with that kind of deployment. And they're saying it's because, look, you AI researchers have frequently described AI as a black box where the risk is quote, unquote, unknowable. So like, how am I supposed to price that in?
- [44:27]
And I think that's so interesting. There's also the fact that like these, these AI models are so general purpose and engaging in so many different activities. And so here's a quote from aon's head of cyber, Kevin Kalanich, who said that the insurance industry can afford to pay a 400 million or $500 million loss to one company that deployed agentic AI that delivered incorrect pricing or medical diagnoses, but quote, what they can't afford is if an AI provider makes a mistake that ends up as a 1,000 or 10,000 losses, a systemic, correlated, aggregated risk. He added, and I thought that was really insightful because just like these AI models only cost, you know, 20 bucks a month, 200 bucks a month, whatever it may be, but if they're being deployed in tens of thousands of different applications, tens of thousands of different customers who are using it for powerful, meaningful activities where it matters if something goes wrong, it's really tough for the insurers to know, like, what should we charge for insurance to ensure that we are profitable at the end of that story?
- [45:32]
A
Well, you spoke a bit about why insurance is so important as a de facto regulation mechanism, but there's also a flip side of this, which is that it enables adoption. Can you talk a little bit about what the consequences would be for the adoption of AI if insurers don't want to participate in AI insurance?
- [45:51]
B
Yeah, I mean, let's go back to my fire example. Let's imagine a world in which I said, fire insurance is illegal. You can't get fire insurance. What's going to happen to construction in America? It's going to plummet, right? Because there's a lot of people who would say, like, I'm not going to take the risk of building a building if I can't be compensated in the event where it burns down. Or maybe I'm only going to build my building if I have a trillion billion fire safeguards, you know, installed in it, because I have no other way of mitigating my risk, you know, financially, because insurance is not available to me. So if insurers exit the market when it comes to AI, that means that there's going to be no means of mitigating your financial exposure to deploying AI, which I think in practice is going to mean a lot fewer institutions are going to want to deploy AI. So the de facto outcome is almost identical to the case of extremely burdensome regulation. This is a point that I have been making for years, right? The people who are anti AI regulation are not necessarily the people who are pursuing the policy that is going to most rapidly accelerate AI adoption. Because for AI adoption, you know, to proceed rapidly, you need insurers to feel comfortable or you need some kind of liability safe harbor clause. And it's very difficult to have a liability safe harbor clause if there is no guardrails that are mandatory and in place so that people transacting in the economy for AI related services and products can, you know, point to. Okay, well, they took those things in place. And here I want to quote from a article that was published by. I might be getting this name wrong, but it's Anat Lior. It's a spring 2022 paper published in the Harvard Journal of Law and Technology. And this is obviously a law review article. And he's pointing out, like, why companies using AI will want insurance. Quote, it is unavoidable that the vast majority of businesses with a significant effect on the public will purchase liability insurance in the long run. This is because it is highly likely that they will be suede. So they will need to acquire some sort of hedging mechanism, either because the courts will impose strict liability or out of a fear that they will do so. So let me just take a moment here to explain this term, strict liability, which I think matters a lot in this story. So it means if strict liability applies, it means that the injured person does not need to prove negligence or wrongful conduct to win a court case to win a suit against you. So, for example.
- [48:53]
If I, like, give you.
- [48:56]
A product and it breaks, let's say I sell you a car, right, and the engine of the car explodes. If strict liability does not apply, then what that means you need to prove is that the car exploded because of my negligence, right? Like, I had a bunch of workers who were drunk making the car engine, and I had no safety oversight of their lousy construction of the car engine. That would be sort of like, you're winning because of my negligence. I have wronged you through my negligence, and that is why you have a financial claim to sue me about. If strict liability applies, it means even if you did everything right, even if you are not negligent, the fact that your product was defective is enough that I can win in court and sue you. So that's like, on a policy preference, not so much out of fairness, but out of, like, where is the ideal place in the marketplace to allocate risk based on who can most effectively mitigate that risk, and the safety incentives can be appropriately applied. So coming back to, like, why are we talking about strict liability? Because if there is no AI regulation that says, hey, here is the risk management framework that you need to apply to ensure that AI is appropriately governed and that reasonable safety precautions are taken in your enterprise, in your market, in your activities, et cetera, if you don't have some kind of governance standard in place, then how do you justify giving companies a safe harbor from legal liability? And we know that like safe harbor clauses in regulation that protect firms from legal liability has been really important in incentivizing adoption in other industries. And so this, I think is kind of the big unanswered question in AI regulation right now is that all the folks who are anti regulation, anti regulation, anti regulation, what they're doing is they're putting a lot of the burden on insurers. And insurers are basically going to say like, well, if there's no guardrails in place here, why would we insure any of this? And so they're preferring to exit the market rather than develop those guardrails themselves. And this is why I think for all the justified criticism of the European Union and, you know, taking a highly imperfect approach to AI regulation, you still come back to like, it would be really good if we could put in place some high quality standards that had the sort of legitimate government stamp of approval or some kind of other multi stakeholder regulatory organization or independent verification organization, something that could justify this sort of legal liability safe harbor clause. Because the de facto outcome, maybe not the de jure outcome, but the de facto outcome otherwise is the equivalent of heavy handed regulation. Okay, hopefully I persuaded you, Matt, that insurance is actually interesting.
- [52:06]
A
I'm persuaded. I hope our audience is persuaded as well. I think AI and insurance can be a slept on topic in the sense that maybe it doesn't sound that prominent or significant, but I think, Greg, you did a good job making clear why it is. So that does it for today's episode. Thank you to all that tuned in. And Greg, it was great talking to you again.
- [52:27]
B
Always a pleasure, Matt, thanks.
- [52:31]
Thanks for listening to this episode of the AI Policy Podcast. If you like what you heard, there's an easy way for you to help us. Please give us a five star review on your favorite podcast platform and subscribe and tell your friends. It really helps when you spread the word. This podcast was produced by Sarah Baker, Sadie McCullough and Matt Mann. See you next.