← The AI Policy Podcast
The AI Policy Podcast
The Energy Cost of AI with Joseph Majkut
01:25:49
Joseph Majkut joins to take an in-depth look at energy's role in AI.
Loading summary
Transcript277 lines
- [00:00]
A
Foreign. Welcome back to the AI Policy Podcast. I'm Gregory Allen. Today we've got a special treat for you. We've got my colleague here at csis, Joseph Mikat, who happens to be one of my favorite people at csis. He's incredibly knowledgeable and a real fun hang. So I think we're going to love this. And where Joseph specializes is on all things energy policy. He is Mr. Energy Policy at CSIS, although he has a real title. And his real title is that he is the Director of the Energy Security and Climate Change Program here at csis, where he leads a team of scholars who are doing research on all aspects of these things. And the reason why we're having him on this podcast is that over the past six months or so, he has written three really fun papers, really awesome papers, along with his team at the intersection of the AI and energy issue. And so today, Joseph is going to give us the kind of soup to nuts overview of where things stand on AI and energy and do that in a way that is approachable and understandable no matter where you are. So if you know a lot about this topic, I think you'll hear a lot of cool insights. If you're just starting, I think we'll bring you up to speed. So those three papers, and I want to highlight them because I do think they're, they're well worth reading. I certainly enjoyed reading them. So from March 2025, we've got the electricity supply bottleneck on US AI dominance as well as the AI power sur scenarios for gen AI data centers through 2030. And then just in September of this year, hot off the press, we have AI for the grid opportunities, risks and safeguards. So Joseph's going to talk about some of the fun findings in each of those three papers. So let's turn it over to him. Joseph, thank you so much for joining the AI Policy Podcast.
- [01:56]
B
Thank you so much for having me, Greg. Very nice compliments. To start off this conversation, I can only hope to disappoint the audience, but I'm really, it's a pleasure to talk to you about this. You've been such an intellectual actual leader on AI issues here at CSIS and we've had, I hope this conversation roughly 50 times in the coffee room, and now we get to have it for your audience as well.
- [02:17]
A
Yeah, it's like I've already been benefiting from this conversation, so we might as well put it on tape and give it, give it to the world as well. So before we dive into the nuances of this important topic I want to ask you, Joseph, like, how did you become, you know, such an energy policy wonk and such a force in Washington, D.C. what's the arc of your career that led you to this moment?
- [02:37]
B
Of course, yeah, I would have been hard to predict. I, I did my undergraduate education in mathematics. I had thought I wanted to be an engineer, and then I was like, terrible at engineering classes, but I really enjoyed math. And so I, I, that's where I started. And then I was really interested in usage of risk and uncertainty modeling, quantitative decision tools in particular for like, social and environmental issues, and like a very formative experience, which I actually think you would, you'll enjoy hearing about. In my, I think, senior year of college, I took a public speaking course and our final assignment was to give a position speech utilizing technical, economic and political arguments on some public policy matter of the day. And I did mine on the Yucca Mountain nuclear repository.
- [03:34]
A
Oh my gosh. The hill that Harry Reid almost died on.
- [03:37]
B
Yeah, exactly. And like, we don't need to get into that other than it was a federal program to build a facility that would hold U.S. waste products from U.S. commercial nuclear reactors. And you can imagine that's like a very difficult scientific challenge. Right. How do you isolate things for 10,000, 50,000, a million years? What are our moral obligations to people far into the future? How do we, how do you even communicate that this is a dangerous place to go digging when you're not sure what society is going to look like in the future? There was a lot of really interesting things involved, but I really enjoyed that project and it probably more than anything else, is responsible for launching me into a career that eventually ended in public policy.
- [04:22]
A
I love that. And you'll have to send this podcast to that professor to let him know that it all culminated here.
- [04:28]
B
I don't want him to know that. No. And then I was kind of doing then an applied math degree in the Netherlands where they have developed a lot of the tools, the best tools we use for sort of uncertainty quantification for problems that you don't want to realize. And that's because in the Netherlands there are legal requirements around the reliability or the failure rate of their dike system. Right. Certain areas of the Netherlands can be. Every area in the Netherlands has a like, flood frequency that is legally allowed.
- [05:03]
A
Right. And this is a country whose sea level, or, sorry, who's like, average height is below sea level. And it's all protected by an artificial system that keeps the ocean out of the country. They care a lot about Floods over there.
- [05:16]
B
Yeah. And so it's not the whole country, but it's a large. It's a large portion of it that. That is kind of under sea level. When I lived there, I lived 2 meters under sea level, interestingly enough.
- [05:27]
A
Yeah.
- [05:28]
B
But it was a. You know, while I was there, the ipcc, the Intergovernmental Panel on Climate Change, published, what would it have been their fourth assessment. This is like mega reports about climate change. What are the hazards? What do we understand about human cause, human causation of climate change? What is our climate outlook? And as a young, arrogant graduate student, I was reading this and seeing that the physical scientists who were writing this were making, like, trivial mathematical mistakes.
- [06:00]
A
Whoa.
- [06:01]
B
And by that, I mean, they were not really thinking about how to reason with uncertainty. And that's exactly what I was studying. So I thought, well, I'm going to go do that. And there was probably some opportunism there. This was like a rising issue. It was very aligned with what I was interested in intellectually. Al Gore had just made this movie, and I wanted to bring some of the decision tools and sort of more rigorous ways of examining uncertainty to the climate science enterprise as a means of understanding this kind of big social, political, economic problem. I did a graduate degree in atmosphere and ocean science.
- [06:39]
A
Happy, this is your. I'll say it so you don't have to say it. This is your fancy PhD from Princeton, right?
- [06:45]
B
Yes, that. That is where I did my graduate degree. And in that context, I was like, kind of minorly successful in. In kind of applying those tools that I trained with to a couple. A few different issues that the community was facing. But I didn't really want to be. Toward the end of my graduate career, I had decided I didn't really want to be an assistant professor, and so I had to find an alternative.
- [07:10]
A
But you were building like, hardcore mathematical and computational models.
- [07:14]
B
Oh, yeah, yeah.
- [07:15]
A
Weather, climate. But also the interactions between those things and like, the human environment, which I was right.
- [07:21]
B
Yeah, I was like, right at the intersection. So I was. I've run large climate models Right. On supercomputers owned by the Department of Energy that we were. We had access to.
- [07:31]
A
Yeah.
- [07:31]
B
And then I was also, at the same time building economic models of the human response to climate change because my. My core intellectual interest, since you asked, was understanding the value of infra. Of Earth observations, Earth system observations. So there's like this huge uncertainty umbrella around future climate change outcomes. A lot of that is driven by how much greenhouse gases emit. A lot of it is driven by uncertainty in the system response to greenhouse gas emissions. And if you could reduce uncertainty in system response by making scientific breakthroughs, by observing the climate better now, by using AI tools to improve your modeling techniques, presumably that information would be very valuable. Yeah, I was actually running studies trying to understand how valuable would it be.
- [08:21]
A
Yeah. So this is like, if you could predict the impact of climate change, you know, not on a country level, but like on a zip code level or on like a street level, like, I'm sure to every insurance company in America that that knowledge would be worth ridiculous amounts of money.
- [08:36]
B
Something like that. Yeah. So this was like I was doing something that was like, vaguely related to that, but much more about, like if you were able to. For instance, the US at the time was about to launch a satellite which looks downward on the Earth and measures the radiative balance of the atmosphere because.
- [08:56]
A
Oh, the Orbiting Carbon Observatory.
- [08:59]
B
No different satellite, but it. But if you. That use the example, though, so the oak. So say you wanted to observe carbon really well. Right. There is profound uncertainty remaining today over every year as we emit greenhouse gases, or CO2. Roughly half of that, not at this in a rate term, is absorbed by the oceans and by plants on land. This is actually very important because it's protected us from what have otherwise been a higher degree of warming. If that process were to degrade because of changes to the climate affecting, like where plants can grow or the circulation of the ocean, both of which have been hypothesized, then you would be more concerned about climate change than you would have been before. And the question was, could you use a satellite to reduce uncertainty in our present understanding of the climate system in such a way that it gave you valuable information today? And what you find when you do these studies is that the value of information from Earth observations is like 10 or 100 times the cost of gathering them. Very important information for today's administration. And. And that value is. Is sort of like even agnostic to whether or not you use it, because there's opportunity created by that value.
- [10:16]
A
Wow.
- [10:17]
B
Anyway, as you can see, like, it got deep, but I was interested in pursuing something else. And in part, I had felt that a lot of my best work as a graduate student I'd done in like, you know, I'd had an insight, I'd figured something out in 72 hours, and then two weeks later, I'd done most of the work and that took me a year and a half to publish it.
- [10:37]
A
Yeah, academia can be frustrating.
- [10:39]
B
I found that frustrating. And so I found public policy because it was related to the issues I was working on. Climate's a big issue in, in American public policy as it is in places around the world. And the, the feedback cycle on the work that we do is much faster. Right. You can, you know, you can, you can tweet an idea and you know by the end of the afternoon whether or not it's going to be a good one or not, you can write a paper and it gets or it doesn't. And so my own intellectual work has really benefited from being able to work at sort of a faster clip than the traditional academic system allowed me to.
- [11:18]
A
Well, I certainly agree that your work has been impactful and timely and all that stuff at csis. So thanks for, thanks for giving us the story of how you came here. Now, I want to start by sort of level setting the audience on, like, what is the American electricity grid that the AI boom is walking into? And in one of your papers, I found a fact that I did not previously know that kind of blew my mind. So I'll just read it from your paper. Quote, the US Electric power sector is facing a stunning and sudden paradigm shift. For roughly two decades, top line national electricity consumption has stagnated, growing at a compound annual growth rate of nearly zero percent since 2007. We did it. Yeah. So for like, for nearly 20 years since 2007, like, we've had no additional power demand. And now the AI boom is walking in like, hey, can you give us like 500 gigawatts by tomorrow? So it's, it's just an amazing contrast. So, like, talk to us, like, what was going on in the US Electric. Well, first off, I guess, what is the US Electrical grid? And then like, what was going on in the US electrical grid over the past 20 years that resulted in this, like, no growth situation. Sure.
- [12:39]
B
So in the US Electric grid, everyone is familiar with its fruits. Right. You flip the light switch, it goes on. And unless you live in a sort of intentional community far away, you largely are, are connected to that grid and, and benefit from it. It is one of the world's largest and most complex machines.
- [12:57]
A
Right.
- [12:58]
B
But you can roughly think about it as having three components. There's the generation. There are generation units. This is like coal plants, natural gas power plants, solar fields, wind farms, et cetera. There's a transmission system which carries power from the generation units down to what we call substations, is the borderline, and then beyond that, it enters the distribution system where it goes to homes. Now that's like sort of A rough topology. There's a lot of exceptions to that. And now we have a lot of distributed energy resources like rooftop solar and batteries that are starting to blur the lines in between these different things. But those are the big system components. It has a lot of strange nuances, right? There are regional disparities in how we govern the electricity system, the rules that utilities operate under, whether or not utilities have, you know what, what kinds of utilities have monopoly power and not. That's a lot of really complicated stuff we can maybe get into because I think it does have implications for where AI is going to get built and price implications for consumers and things. But it's basically a very large, highly capital intensive, and thankfully quite reliable system. By and large. Everyone's had a power outage, but by and large we keep the thing running outside of extreme conditions. Now, over the last couple decades, there's been a few things that have happened. One, we've done very well around the country at having a higher level of energy efficiency. This is by and large good. Right. Energy inefficiency is waste. And so by changing building codes, by updating appliances, LED bulbs actually have played a huge part of the story.
- [14:45]
A
Sure.
- [14:46]
B
We've managed to continue to see economic growth while not seeing incredible increases in electricity demand. At the same time. The generation mix really changed, Right? So like, you know, you and I left college in the early 2000s. Like at that time, the US was emitting a maximal amount of greenhouse gas emissions. Coal plants were still the biggest provider of electricity. Now through a shift, you know, we had the shale revolution. The cost of wind and solar panels has come down. So you had conditions where there was like flat demand. And then you were substituting for all of this old fleet of coal generators with much cleaning, more cleanly operating and cheaper mixture of natural gas and, and wind and solar. We also, that was like a time period of relatively, I have to say, stagnant economic growth. So like, all these factors combined sort of define the power system up until a few years ago, a couple of years ago.
- [15:49]
A
So. So I think, I think, like one thing that's interesting, there is part of the story I feel like is a, is a good news story, which is about efficiency, Right. It used to require a tungsten filament to light a room. That requires a lot of power to light a room. Now we use LED bulbs. The room is still perfectly lit, but we're using a lot less power. So that's like a deficiency in conservation kind of story.
- [16:11]
B
I mean, some of us have problems.
- [16:12]
A
With color, temperature, but yes, but I think there's a second story that, like, critics would argue, and I'm curious, you know, how much truth you see to this argument, which is. It's part of the story of quote, unquote, like, American de. Industrialization. So, you know, it's. It's good that the amount of energy required to generate GDP growth has gone down and we've still had, you know, GDP growth, although not like, you know, breakneck GDP growth. But if you contrast, like the China story since 2007, you know, their electrical generation was less than US electrical generation in 2007, and today it's more than twice as big as us. So, like, the Chinese grid is more than twice as big, even though it was starting from a smaller baseline. And so it's like all those, you know, factories that maybe would have been built in America, did we, like, regulate that away? And so, like, the. So part of the story is the efficiency story, but the other part of the story is, like, the changing composition of GDP and that growth occurring in the services sectors that are relatively less energy demanding versus, like, the manufacturing sectors, which are relatively more energy demanding.
- [17:24]
B
There is definitely an alternative state of the world where you had a higher degree of industrialization, where you would have used a lot of electric, you would have used more electricity in the United States. Absolutely true. And. And, and in fact, that. That is the story of AI confronting this. What has been the stagnant power sector. Right.
- [17:41]
A
Yeah. Because, like, all of these factors have combined to create a U.S. electrical grid system that knows how to change, like, less coal, more solar, but does not actually know how to grow yet. Yeah.
- [17:55]
B
So if you're a. Yeah. If you're an executive leader at, like, a United States utility, almost your entire career has been defined by a period of no growth.
- [18:06]
A
Yeah.
- [18:06]
B
Now, if you're looking at national statistics that can cloud that. There have been some shifts. Right. So there's growth in Texas, there's growth in the Southeast, but none of the. It's like, when you think about just sort of like an industry confronting a new problem.
- [18:18]
A
Yeah.
- [18:19]
B
Our utility industry. And the utilities are, are the, you know, is shorthand for those that generate and distribute electricity is like, they are. They are untrained.
- [18:30]
A
Yeah.
- [18:30]
B
And inexperienced in a growth environment. And that is actually part of the challenge that we're facing.
- [18:36]
A
Yeah. And, you know, we've talked about how there's been a lot of change on the composition. More solar, less coal. But, you know, in the big three components of the grid that you talked about, generation, transmission, and Substations. I think, like, transmission is the one that really has had like, no growth. And your paper, correct me if I'm wrong, but like, your paper talks about like, the wait times to get connected to the grid are just like, horrific. So, you know, what does it mean to be connected to the grid? And like, why is it so formidable an opponent? If you want to build something, a data center, a factory, whatever that wants to use the grid, like, why is it so bad?
- [19:20]
B
So here's why. The grid is tricky.
- [19:22]
A
Yeah.
- [19:22]
B
And it has to constantly be in balance. What is unlike other elements of the energy system. Right. If I'm, if I'm an oil producer and I make too much oil one month, I can store it, put it.
- [19:37]
A
In, put it in a giant, you know, facility.
- [19:39]
B
You put it in a tank. Now that system does break down during COVID We had an ex. We had a. We had a moment where we actually like, ran out of physical storage for crude oil in the United States. Price went negative in Oklahoma for the first time.
- [19:49]
A
Time. Wow.
- [19:50]
B
So that, that is, you know, but that's crazy. A once in a century exception, right?
- [19:54]
A
Yeah.
- [19:54]
B
But with electricity, you actually cannot have big system imbalances or even like, particularly small system imbalances, to be fair, without it causing real problems.
- [20:04]
A
So, and this is, this is like if you have a light bulb and you run it at 100 watts, you get light. But if you take that same light bulb and you run it at 1000 watts, it explodes and like starts a fire. So you have to constantly make sure that the amount of power that you're delivering is the amount that is being consumed. Because there's no giant oil tank where you can dump excess power.
- [20:23]
B
Yeah. I think like, an electrical engineer might find a couple problems with that analogy, but, like, the conclusion of it is not bad.
- [20:30]
A
Yeah.
- [20:31]
B
Like, look what happened in Spain earlier this year, right? There was a cascading grid failure. Because we don't know the exact reasons, but the diagnosis appears to be there was voltage irregularities in one section of the system, and then to protect itself, the rest of the system sort of shut down in a cascading way. That's like the electricity system. You have to do that. Otherwise if it, if it, if it is allowed to go too out of balance, then like your turbines break down and you have mechanical failures that don't take days to restart from. They take months to restart from. So.
- [21:03]
A
Yeah. So my favorite, like, story about balancing the electrical grid, which I forget if I got this from you or something like that, but it is how electricity demand in the UK is tied on a minute by minute basis with the commercial schedule of the English Premier League because so many people in the UK go off to boil a pot of tea during the commercial break.
- [21:31]
B
Oh, right.
- [21:31]
A
And if you take like 10 million teapots and you turn them all on at once, that's actually like a non trivial amount of electricity. It translates to like, like generators having to be turned on.
- [21:42]
B
Right?
- [21:42]
A
And so like balancing the grid on like a minute by minute basis means turning on power generation, turning off power generation. And, and all of that stuff is crazy, you know, one of the most complicated machines humanity's ever built. And like, yeah, it's a crazy challenge.
- [21:58]
B
And by way of analogy, this actually really helps us explain this sort of like connecting problem. Right. So like there is by way of analogy, like I. But a person who's operating a power plant at very low capacity and then when the Premier League goes to halftime or a third time or whatever, soccer players take a break and all those kettles are going to find they spin that, that generator up to higher capacity to match that anticipated demand. Right?
- [22:24]
A
Yeah.
- [22:24]
B
So let's say I'm like, I'm a large data center and I go to, I want to build, I want to like plug into the grid and run all these fancy chips and make cool videos and, and dramatically change the economy. I need to be able, I go to the grid and I say, well, I'm going to need 500 megawatts of pretty consistent power because I want to run these chips all the time. That grid person then has to figure out can I serve that load reliably and without disrupting the service of all the other incumbent users. And what is the generation portfolio going to look like at times of peak demand where I have to serve that 500, but I've also got all these kettles going on or people coming home at the end of the day. And likewise, if a new generator is coming online and they say, hey, we're planning to sell power into the grid and we're a solar farm, so we can only do that when the sun is up and down and we're going to sell more at noon than we do at 6am or we're a gas plant and we're going to sell under these market conditions, then the regulators and the operators need to sort of pencil out that when they connect that stuff, it's not going to cause large system balances, they're not going to have problems.
- [23:38]
A
Right.
- [23:38]
B
And that process can be a very slow one. And so in the United States, in part because, like, it's done by different people. There are consultants that operate here. There's humans sending each other's PDFs, right?
- [23:52]
A
Oh, God.
- [23:53]
B
This is clearly a problem that AI can help with. And there are tech firms working on solutions in this context.
- [24:00]
A
I mean, I guess before I learned all this, my mental model would have been like, there's an ultra awesome computer simulation of the grid, and you just plug stuff into your simulator, like a new data center, and it's like, nope, wouldn't work. But no, you're telling me, like, this is a lot of humans working months of studies.
- [24:19]
B
There is a human that has very sophisticated grid modeling software. That's one element in this chain. Right. But the issue is that we're facing because the sort of, like the fundamental problem, it's not a problem because there's a ton of economic benefit that we expect comes from this, is we want to connect. Now we have this data center thing happening, which is like, I think you can analogize it as an industrial load, right? These are factories of intelligence. They make tokens. They require a bunch of electricity to do that. And we want to build them really, really, really fast. Because the country's in a race with China, because all these tech firms are in races with each other to kind of own a large part of this new piece of the economy. That the system had sort of like lost muscle memory for growth. It's grown quickly in the past, 1950s, 1960s as examples, but we need to, like, you know, revisit some of that. And the systems and the processes that were acceptable at a time of low growth are no longer, I think, going to be functional now. So a lot of the function, the.
- [25:29]
A
Emailing PDFs and taking 6 months era is just not compatible with the growth pace that we need.
- [25:35]
B
No, I don't think so. And it's not going to help us meet national goals, and it's going to introduce lots of other problems along the way.
- [25:42]
A
Okay, so help us understand this is fabulous context in terms of what the grid is and what the situation is now. Well, let's change gears a little bit and just talk about, like, AI as a share of the story. And so your paper, the Electricity Supply Bottleneck, you know, offers a range of scenarios, one of which you produced with your colleagues on the team. And it says that US AI data centers were roughly 4 gigawatts of demand in 2024. And you're forecasting that by 2030, that's going to grow to 84 gigawatts which is 2,100% growth. Now, how big is 84 gigawatts? Help, like, contextualize, because obviously that growth amount is huge. But how big is it in the context of the grid?
- [26:33]
B
So this is where this does actually get a little bit nuanced. So 84 gigawatts is like in terms of generators that people are familiar with, a large nuclear reactor generates about a gigawatt.
- [26:49]
A
Okay.
- [26:49]
B
So, you know, we've built, I think, one of those, two of those. Excuse me, in the last 30 years in the United States.
- [26:57]
A
Oh, two for 30 years?
- [27:00]
B
Yeah. Like a really big solar facility might make 30 megawatts at peak production. No, excuse me, 300 megawatts.
- [27:09]
A
300 megawatts.
- [27:10]
B
Okay. Yeah. At peak production, you know, like, you know, you get a few gas turbines, you can approach a gigawatt. My favorite, and I've shared it with you before, is the Hoover Dam. It has a capacity of 2 gigawatts. It generates about a gigawatt right now.
- [27:28]
A
At any given on, like the river levels, right?
- [27:31]
B
Yeah, totally. Depending on the level of the Colorado River. And so, you know, you're talking about for every gigawatt, a fairly significant amount of information power generation infrastructure that has to back that.
- [27:42]
A
Yeah. And then. And then 84 gigawatts, of which 80 would be novel growth in your prediction between 2024 and 2030. For context, that's more than the UK grid. The UK grid in was 71.7 gigawatts in 2024. In Saudi Arabia, which loves energy, the grid is 91.1 gigawatts. I don't know if you know off the top of your head like, what the US or China grid is, but clearly this is like adding a UK of electricity demand to the United States is not a trivial matter.
- [28:19]
B
No. So like, the other way to think about it is in terms of percentages. So gigawatt is a statement of capacity, and then a gigawatt hour is like how much you use. Right.
- [28:30]
A
Yeah.
- [28:31]
B
And so data centers as a load category, tend to run at high capacity, meaning they're using most of the energy they're able to use most of the time.
- [28:41]
A
Yeah. And just to put that in AI terms, right. Nvidia chips are like $35,000 a pop. So you want to get a return on that investment. If you're building $10 billion worth of Nvidia chips, data center capacity, you know what is the return on investment is basically determine of the minutes in the year, what percentage of the minutes were Those Nvidia chips being used to generate toke tokens, which basically means you want very high utilization. You'd love those data centers to be running flat out all the time. That would be your ideal situation.
- [29:14]
B
So there's another calculation in our paper that kind of helps ground it because it's like right now, data centers as an entire class, this is sort of traditional data centers, storage data centers, and the sort of nascent class of AI, generative AI data centers use about 5% of all the electricity generated in the United States on an annualized basis.
- [29:38]
A
So this is the, the LBNL data point where you say there that all US data centers, not just AI ones, in 2023 was 20 gigawatts, which you're saying is 5% of the US electrical.
- [29:51]
B
Demand, but in total, in annualized demand.
- [29:55]
A
In annualized demand, yeah. So not, not generation capacity, but in annualized demand, which is the difference between a gigawatt of capacity and a gigawatt hour of actual electricity consumed. Right? Yeah.
- [30:05]
B
And then if you kind of look at that at the annual scale we used in the paper, terawatt hours as the data point or as the unit. But if you think about that sort of, if you think about that now, anticipated growth, we see the growth having like data centers of all types approach 10% of U.S. electricity demand by the end of this decade. So that is a substantial growth category for like total demand. There's also like a couple of the data points I like from the paper. I know that you've been looking at the table. I can see you doing it over the. Over the zoom is if you look at the McKinsey example that we cite, they look at global data center capacity. So right in. This is not just AI data centers, but this is like all data centers, there's like 55, maybe 60 gigawatts, depending on which consultancy you ask about. You ask in just like total data centers around the world, the growth we expect to see puts that somewhere between 130 and 200 gigawatts by 2030. And our model says 80 of that's going to happen in the United States.
- [31:20]
A
Wow.
- [31:21]
B
So like in the modeling we do, which I will admit is optimistic. Right. You're looking at an incredible concentration of AI as an industrial activity in the US and it being basically doubling the share of data centers in the electricity system.
- [31:39]
A
Yeah. So I think we talked how that looks.
- [31:43]
B
Can I make one more comment? Because this is wildly underappreciated. We are. Now, what's so funny, we are casually talking about what are going to be the largest electricity loads? Single point electricity loads, like in the country, if not the world. I have yet to find a verified example of something that is taking 2 gigawatts of electricity in a single concentrated facility.
- [32:08]
A
Can you talk about that? Why is it interesting or important that they're single point loads?
- [32:13]
B
Oh my gosh. They're like black holes of electricity.
- [32:17]
A
You.
- [32:17]
B
Know, and so if, if this was like a very highly distributed new demand.
- [32:21]
A
Signal, like this is not another Los Angeles with another million homes spread out over many square miles. This is like a campus of. That's a black hole of demand. That's what you're getting at.
- [32:33]
B
I would think. I would, I would draw a different distinction. I would say like, you know, we, for a long time in the climate world have known that the US would need more electricity because of the shift we anticipate to see in public, in personal transportation, going from gas powered cars to electric vehicles.
- [32:48]
A
Yeah, right. Clearly less overall energy demand by switching to EVs, but higher electricity demand.
- [32:54]
B
Exactly like that. That demand is going to be mediated by the electricity system instead of by fuels. And so we anticipated a rapid or a large growth in electricity demand, but that is like very diffuse demand.
- [33:08]
A
Right, right.
- [33:09]
B
It's all cars. Yeah. And it charges at different times. And it's not all sort of like syncopated. What happens when you bring a 2 gigawatt data center to a service area is you've suddenly created this immense source of, of electricity demand and one that is like amongst the largest source. I mean, it's like, you know, large steel plants.
- [33:35]
A
Yeah.
- [33:35]
B
LNG export facilities. Take this kind of aluminum smelters on the shores of Lake Baikal. Take this kind of power. Right. This is just, just, it's not just like a new factory that is, that is doing some light industrial work.
- [33:47]
A
And so, and so what that means, like in terms of regional grids, I'm sure is crazy because you're talking about, you know, national demand for data centers increasing by like hundreds of percent, you know, or maybe thousands of percent, depending on how you count. But most of the data centers in the United States are in California, Virginia and Texas. Right. And I think most of the new construction is going to, is expected to happen in Virginia and Texas. So the grid in those regions is not going to go up by 2000%. It's going to go up by like some crazy unfathomable number, I'm sure.
- [34:26]
B
Right. And this carries a few implications. I mean, it really does have implications for the kind of infrastructure that's needed.
- [34:31]
A
Yeah, right.
- [34:31]
B
And you can kind of, you can see that guiding the decision making of data center developers where they want to put things. It means that if you want to have some sort of like on site generation as a backup, that has to be a very large system system. And, and so like the. I think the, the thing to appreciate is that while these bulk numbers are large, I argue they are manageable against the scale of the whole electricity system we have in the US So we don't need to panic, but we do need to be attentive. We need to pay attention to the fact that these are coming in very large concentrated units.
- [35:09]
A
Yeah. So we already have some pretty big AI data centers in this country. One construction project that I've talked about on this podcast because it's, it's just such a remarkable undertaking and also because they've been pretty public about it is the XAI facility construction in Memphis. So can you just tell me like what, what to you stands out about that story as it interacted with the U.S. electrical electrical system and the U.S. electrical regulatory system. Like tell the story of the XAI data center.
- [35:43]
B
The electricity. The energy story XII is, is actually quite interesting. Very elon coded in that they. So the. I think the first wave, if I recall correctly, is about 300 megawatts of data center capacity.
- [35:57]
A
Right.
- [35:57]
B
I can't remember how many chips that is, but they built that facility.
- [36:02]
A
The first, the first round was 100,000 chips and then they doubled it to 200,000 chips and the from 0 to 100,000 chips was 122 days.
- [36:12]
B
Right. So like the building of a data center itself is not a long process.
- [36:16]
A
Right.
- [36:16]
B
You build a shell, you install a bunch of cooling equipment, then you bring in racks of servers. Right. I'm sure there's more to it than that, but it is not a years long process. If you need 300 megawatts to power that data center, it could be a years long process to add 300 megawatts to the grid. In the context of the US power sector as we have been running it.
- [36:39]
A
Yeah.
- [36:39]
B
And so what, what the XAI did was they found a bunch of portable generators that run on I believe natural gas.
- [36:47]
A
Why do these exist? Are these like what you send to like when a hurricane goes through Florida? These are the generators you call for help or.
- [36:54]
B
Oh yeah, yeah, construction sites. I mean you use them like you know, large power site, large power plants might go down, out of service or be restored. You might need to use them in that context.
- [37:04]
A
Okay, okay.
- [37:05]
B
But you Know, it's like, like it's a portable generator. Right. It's, it's a very large version of a very familiar consumer technology.
- [37:13]
A
Yeah.
- [37:14]
B
And in that context they, they got an air permit waiver so that they could run them all the time to power this data center.
- [37:21]
A
Ah.
- [37:21]
B
And, and then, you know, there was a lot of utility resistance not just to kind of getting this thing connected and then dealing with the expansion because Colossus 2, I think is like in excess of a gigawatt of capacity and it's being planned in the same area. The story there is actually remarkably interesting because what they ended up doing was buying an older, near defunct power plant across state lines from Memphis and packing it with again, large portable gas generators, at least to start, as well as a bunch of Tesla batteries and other things, and then building a dedicated line to bring it over into Memphis to power this new large data center. And in that process sort of averted resistance from the Tennessee based regulatory authorities, but brought in dedicated power.
- [38:14]
A
Okay, so this is kind of amazing because I think this example sort of highlights where some of the bottlenecks in the system are right. It sounds like to me, tell me if I'm wrong here, but like a lot of the bottlenecks in the system are not like the engineering and construction side of the story. The bottlenecks are in grid connection and regulatory type issues. And if you're willing to do whatever it takes to make those go away, the speed of construction could be much, much higher. But this is obviously, you know, just like one example. Does that scale up to the size of America, like building 84 gigawatts of data centers?
- [38:53]
B
I don't think it does. I can't remember. I was trying to REM the proportion of the portable generators that XAI is using. But it's an incredible. It's a very large proportion of it.
- [39:03]
A
It's been a great year for the people who make portable generators, I'm sure.
- [39:06]
B
Oh, it's the people who own them and can rent them out right now.
- [39:09]
A
Yeah.
- [39:10]
B
And so like it doesn't quite scale. Yeah. But you do see, however. But two things I guess. One, that model, I suspect we will continue to see where sort of more sort of duct tape together, temporary solutions are important for speed to power in the sort of gig in building gigawatt scale facilities.
- [39:30]
A
And can you just. Because speed to power, I feel like is a term that people in this field use all the time. But for the rest of us, what is speed to power? Why do we matter?
- [39:38]
B
You can build the infrast, you can build the shell of the data center and pack it with chips in three months. It's going to take you years to get it interconnected. That's the speed to power.
- [39:45]
A
Speed to power. Okay.
- [39:46]
B
Yeah. And so, you know, I haven't heard a lot of stories. There's some anecdotes floating around of like, data centers that are buil and are not, not empowered. But I think I'm starting to believe more and more of the story that data centers are not being developed because of delays in, in being able to connect them to the.
- [40:04]
A
Oh, yeah, because, I mean, like just, you know, as a guy who went to business school. Right. If you, if you take out $100 billion in loans, you, you know, every year that, that $100 billion is not in the stock market making money is like a horrific failure. So to have built data centers, to have bought Nvidia chips, and to not be generating a return on invested capital on that at these numbers is like a horrifying failure. And so if they start construction, they're kind of making a bet on what their speed to power is going to be. And if it turns out that that is longer than intended, the unrealized opportunity costs of that is horrific. Yeah.
- [40:51]
B
Oh yeah. I mean, it's like existential. I'm probably.
- [40:54]
A
Yeah.
- [40:55]
B
So. So I think what you'll see is a lot, a lot of these kind of like what I would say is like lose, you know, technical term, duct tape together solutions because eventually they're going to have to backfill all those, those portable generators with some sort of permanent generation solution.
- [41:08]
A
Yeah. When there's another hurricane or another whatever.
- [41:09]
B
Yeah, well, it's not just that. It's like, you know, those are expensive to run. They're, they're less efficient. Right. You want, you, don't you. Rather than have a bunch of portal generators, you want a substantial, you want a reliable, an inexpensive grid connection.
- [41:22]
A
They want to be on the grid.
- [41:23]
B
Oh yeah, totally. And so like that example I think doesn't scale, but I think the model maybe scales a little bit more. And then we've seen a lot of attempts, or we are seeing a lot of attempts to build different models for large data center and compute hub development that would increase speed to power. And so a good example for folks to have in their head is if I had a couple large natural gas turbines, I could build those and then co locate. Unfortunate terminology because colocation has another meaning in the data center context. But locate nearby a data center and principally serve that data center with this new generation in sort of an off grid sense, like in the way that your weird uncle generates his own power. Right. You could do that for a data center.
- [42:22]
A
Is that being done? Are there any companies who are known to be pursuing that strategy?
- [42:25]
B
It's definitely, yeah. So we're definitely seeing gas turbines go in at large compute facilities. Whether or not it's going to be 100% of the power is. I don't think that we have yet seen that. And then you've seen a few especially actually, interestingly, the big oil and gas majors look at that as a, as a service they can provide. And this is like when we did an epo, we did a podcast episode with the CEO of Chevron, Mike Wirth, where he kind of explained why his company is looking at doing this. Oftentimes they're running very remote large refineries and other facilities that are also large electricity consumers. So they'll do on site electricity generation and they're trying to port that model over to the data center context. So you'd have a campus that was principally powered by on site large generation that could be a mixture of natural gas turbines. In the future, it might be small modular reactors or, you know, in some people's vision, it could be an incredibly large field of solar panels and batteries as a way of sort of getting around this issue that it takes so long to connect to the grid that it's going to. You're going. The opportunity cost is going to be too large if you've got chips orders and you need to be using them.
- [43:44]
A
Yeah. Okay, so our fantasy is USAI leadership, which requires constructing a lot of data centers, which requires a lot of new additional electrical generation capacity. And we've talked about some of the bottlenecks that you encounter when you're trying to do this. If you're trying to go fast, fast, fast, fast, fast. So like Elon Musk and Xai have demonstrated one way to go fast, but you top out when you buy all the rentable portable generators in Americ, know, so you can't get to 84 gigawatts that year. So let's talk about the rest of the bottlenecks in this journey as America, you know, and all these companies keep trying to build more and more, like, what are the challenges that they encounter and what is like the. I'm thinking out loud here, but like what is the order in which they will encounter them?
- [44:32]
B
So I think we're already starting to see political opposition arise as an area of, of concern for data center developers. Right. And for reasons that should be kind of obvious when you sit and think about it for a second. Right. We've already talked about how these are incredibly large electricity loads that you sort of like airdrop into existing service regions and that might have price impacts for the people in that area. Right. It might forestall other opportunities. Because if, even if you're able to meet all this new demand load, all this new load demand, you might have wanted to have a new steel plant or a new semiconductor fabrication place. And so the extent to which we're sort of limited in our ability to grow the electricity system, you start getting into zero sum politics around who gets to use the next marginal unit. So I think that we are already starting to see that. And there's a lot of tension in between sort of governors and regional leaders that want to play host to data centers and large data centers. Utilities actually would like to, I think play often like to play host. Because very large loads like data centers, like once you can get them empowered, tend to be kind of good for the system efficiency.
- [45:50]
A
Right.
- [45:51]
B
But if the perception, rightly or wrongly, is that these are increasing costs on consumers who are also voters and are foresawing other economic opportunities, I think that there will be large political challenges.
- [46:05]
A
Yeah. And we're already seeing that in the XAI example where as you noted, they chose to build a lot of their electrical infrastructure across state lines in Mississippi. And that was specifically to avoid political challenges because Mississippi was more willing to accommodate them.
- [46:22]
B
Brilliant move.
- [46:23]
A
Yeah.
- [46:24]
B
Whether or not Memphis was chosen with that possibly in mind in the future is like a. Going to be a business school case study at some point.
- [46:32]
A
Point. Yeah.
- [46:34]
B
But yes, so I, I think those will be issues. I mean, obviously there are things that you're more familiar with than I am and, and we talked about with our authorship team. It could be that this demand doesn't realize. Right. Like the, that this is a huge source of, of capital accumulation in the US right now, but whether or not it will be in a year.
- [46:52]
A
Yeah, a lot of, a lot of debates going on in the industry right now. Now I think, I think everybody thinks that AI is for real. But you know, Palantir is currently trading, I want to say their stock price, their market valuation is like 60 times revenue. And Google in like 2005 was trading at like 10 times revenue. So like their valuations are incredibly high. That's part of the valuations have grown an enormous amount. That's what's driving a lot of investment in the sector. But like, feels kind of bubbly to some of these folks. Even if AI is true over the longer term, again, when you spend $100 billion, you really need to start seeing some return on investment. Not too short fashioned. So one scenario is like the demand does not materialize at the pace anticipated. Another scenario is like we encounter all these political bottlenecks. What are some of the other big bottlenecks in the story?
- [47:44]
B
Like the other kind of potential bottlenecks? Right. The ones we talk about in our paper is like growing the electricity system en masse to meet this growing demand. Right. So that's sort of, you have like lots of micro questions about individual data centers, but then like, how is the US going to tackle having this much demand while meeting the other things that we need? The other priorities we need to meet with our electricity system, which is maintaining affordability, maintaining resilient service. And for a lot of people around the country, though not the present administration, like continuing to make progress reducing greenhouse gas emissions.
- [48:20]
A
Right.
- [48:20]
B
And so that's really a statement about like being able to add enough generation that you can serve these new loads of a variety that helps control costs and reduce emissions. And it means being able to build out a system of transmission that allows us to recognize, to realize these economic opportunities associated with new data centers. And know transmission is a very tricky element of infrastructure because the way that it is paid for is like highly arcane. The way that it is like planned and operated is like also incredibly difficult to grok. But I think the important thing for your audience is to think about it as it's a tool that provides really, really valuable grid services. And if you want to connect a very, very large load to the grid system, you need very large transmission capacity to connect into.
- [49:19]
A
Yeah.
- [49:19]
B
Otherwise you need to be connect, you need to have a large generation source very near your facility.
- [49:25]
A
Yeah. So your paper AI for the Grid talks about some of these interconnection challenges and has some great, you know, fun facts for here. So let me just say, let me, let me just read from this report.
- [49:36]
B
Yeah.
- [49:37]
A
In recent years, the median time for from interconnection request to commercial operation has stretched to about about five years. More than 2.6 terawatts of proposed projects were waiting in interconnection queues at the end of 2023, enough generation to power more than 145 million customers nationwide. So like this existing system, before the AI boom times already had a horrifically slow, you know, speed to power interconnection queue. And now you're asking that system to say yes to 2,100% more projects or whatever the actual real number is per year.
- [50:17]
B
Yeah, I mean, there's enough like that stat is used both responsibly and irresponsibly. I think we use responsibly, but.
- [50:28]
A
Well, help me understand why a lot.
- [50:31]
B
Of those projects are probably fake or wouldn't be realized. Right. Because the system has gotten so constipated that if you're a project developer, you need to hedge your bets across a bunch of different places because you're not sure where you're going to get interconnection.
- [50:45]
A
So you're saying, Mississippi, can I build a steel factory in your state? Louisiana, can I build a steel factory?
- [50:50]
B
Everything on the list we're talking about is generation.
- [50:54]
A
Right, okay.
- [50:54]
B
And so like, can I build a solar plant here? Can I build again, Can I build a wind farm here? Can I build this new pack of batteries that I want to add in, you know, near Dallas or whatever? Those are all areas where you're going to see a lot of the, where there's been a lot of delay.
- [51:09]
A
Okay.
- [51:10]
B
We are starting to see interconnection cues like a line form on the demand side in a few places around the country as we're trying to navigate this load growth challenge.
- [51:21]
A
Yeah. So now we talked a little bit about the speed and the, you know, the transmission growth problem. Let's talk a little bit about the generation problem and like the mix of energy in the United States that we have today and the mix of energy demand that the AI boom could bring about because the Biden administration, I mean, I've talked to folks in the Biden administration like Ben Buchanan, who ran, you know, the AI policy portfolio at the White House from the chief of staff's office. And one of the things that he said, because there were some folks in the Biden White House who basically saying, oh my gosh, you know, we're trying to reduce energy demand in this country. How can we possibly, you know, get enthusiastically on board with all of this AI boom? And he's like, no, you don't understand. A hundred, like hundreds of billions of dollars of capital are coming. This is exactly the capital infusion we need to power the green revol. So we're going to make all these waivers available if you build green energy. The Trump administration obviously has this different theory of the case there. I think some, some would argue that they're an all of the above energy strategy. I think others would argue that they're just straight up pro fossil fuels and anti renewables. But yeah, Talk to us about like, what is the, what is the mix of energy in this country now? What is the mix of energy uses that are going into these AI data centers? And how might the US generation mix evolve in the various AI boom scenarios?
- [52:46]
B
There is no more fun game than. And when you're talking about this issue than trying to elicit from someone what they think the proper and right generation mix for the United States is.
- [52:57]
A
Okay.
- [52:58]
B
It tells you you can really get under somebody's skin if you say instead of 45%, we need 50% of this or that source. In our paper, we take a fairly strong editorial stance. Finance. Looking at market data from the last few years and what project developers are able to build and finance in the United States today and argue that all demand growth over the next four years is going to be met principally by natural gas, new natural gas generators, by solar, and a little bit by batteries. Batteries are not a generation technology, but you can, it's like, it's like not a bad thing to count them as a generation technology in this context. There's a, there's a part of the nature of the power system. Is it like we talked about earlier or implied earlier, doesn't run at full capacity all the time. A lot of the, a lot of the planning concerns around having like large new loads is being able to serve them at times of peak demand. And batteries at times of peak demand can be deployed as they. As if they were generators so long as they were charged beforehand.
- [54:06]
A
Got it.
- [54:07]
B
And so you can kind of think of them as, as generation in this context, even though it's like strictly not correct. And, and like that is the package of stuff that's going to get built.
- [54:16]
A
So natural gas, solar and batteries are like 90 plus percent of the story for the next five years.
- [54:23]
B
Yeah. Now wind may occupy some bit of that as well because we've been building wind power.
- [54:27]
A
Well, and I think, I think one thing that I learned from you a while back back is, you know, the fastest way to add power to the grid is to not subtract power from the grid that you were planning on subtracting. So. Right. Like there was a bunch of coal plants that were at one point scheduled to be decommissioned and now I think they're going to live longer than they would have otherwise if the AI boom hadn't happened. Right.
- [54:48]
B
Yes. So we did have, we have about like roughly 100 gigawatts, say of, of coal plants that sort of would have closed over the next few years. I think it's unlikely that we will see the closures of most of those plants.
- [55:03]
A
Got it.
- [55:05]
B
A lot of them are in the sort of Midwest where we're seeing data centers, data center development and sort of like in the Midwest where we're seeing some, some like new proposed data centers and, and just for reasons of grid resilience, it's probably unlikely that they're going to be closed. The Trump administration is very comfortable with that. But the real problem and where I think our work has been trying to make some head roads is what happens after that five year period because we're talking about immediate business decisions.
- [55:42]
A
Yeah, you have this lovely line in the paper. Five years is tomorrow in the energy grid totally because of how long ahead you really need to plan.
- [55:52]
B
And it's not just for these kind of messed up interconnection issues. It's like this is large infrastructure whose cost is largely socialized. So you actually do need to be like sort of judicious in how you do go about expanding it. That doesn't mean slow, it just means smart. And if we're truly in a sort of second industrial revolution, then the 80 gigawatts we project for 2030 is going to be exponentially higher in 2035 and 2040. And if we're sort of redlining the system to get to 2030 and we're unprepared for a continued pace of growth in the 30s into the 2000 and 30s, then we've got additional problems because like that coal plants that we might be able to keep online for grid security purposes over the next five years, they are aged, right. They're not perfect, they're going to have to be closed eventually. And so you need like, you're sort of like that in, even in activating that as a, as a way of keeping things online, you're sort of setting yourself up a bigger cliff in the2030s than you would have otherwise had.
- [56:53]
A
Right, right. Because we are going to, we are going to pull all the slack that exists out of the system, you know, over the course of the next five years. And so to be like, haha, we can do it is like actually missing the point, like the gains. Because I think people look at like Xai and Elon Musk, right? They're like, okay, so it takes 122 days to build a hundred thousand tier facility. It's like no, no, no, no, no, no. That is like what it looks like to build one of these where you can pull all the slack out of the system. If you want to build a, you can't use the, pull all the slack out of the system, you know, route, you have to actually solve the fundamental underlying issues of the system.
- [57:33]
B
Yeah. And so this is where our paper sort of talks about some of the solutions we think are going to be important long term.
- [57:39]
A
Yeah.
- [57:39]
B
And that is the US getting serious about nuclear power again as a generation source. Low carbon, very consistent or firm in the language of the electricity system.
- [57:51]
A
I think it's worth dwelling on this because like non energy nerds might not know that, but like, you know, if you think about demand and supply, there's like how consistent your demand is and how consistent your supply is. So like solar, very inconsistent in supply. Tea kettles during the soccer commercial, very inconsistent in demand. And AI is very, very consistent in demand because they want to run those Nvidia chips 24, 7. And nuclear is very consistent in supply because it's actually a real pain in the butt to turn off a nuclear reactor and to turn on a nuclear reactor. Right?
- [58:26]
B
Yeah. The types that used and built and built in the United States, you want to run them relatively consistently.
- [58:32]
A
Yeah.
- [58:32]
B
And so they, they sort of are like a natural pair for large data centers.
- [58:36]
A
And, and natural gas is very, very easy to turn on. Very, very easy to turn off. And so a lot of the, you know, excess capacity, the flexible capacity in America's system is performed by natural gas. So your point here, right, that like nuclear is a logical fit for AI, I think makes intrinsic sense. I'm just a little, you know, terrified of like, you know, you talked about how nobody alive remembers the grid growing and has ever like had a portion of their career with a growing grid while in nuclear. You know, like those, those folks are. Yeah, really, really, you know, kind of need to, to shake off the rust. Right.
- [59:13]
B
And so, you know, we make a series of recommendations in there about sort of in our paper about what we think is necessary to do, particularly if you kind of view this as a like urgent national priority. The administration has not adopted all of those recommendations. They've adopted some and, but they are, they seem fairly keyed in on nuclear, like what they call the nuclear renaissance. They want to see 10 gigawatts of new nuclear under, under advanced development or under construction by, by the end of the administration. That's in line with what we think is possible. And talk about in the paper, you know, that doesn't cover the whole 80 gigawatts that we foresee in potential demand, but it would be a huge addition because if you can do the 10 and have those under construction, then in 20, 35, you might be able to do, you know, 20 or 40 if, if that ends up being what we, we need to have.
- [60:10]
A
Right, right. Because just in the course of building that 10, you would be fundamentally changing the nuclear power plant construction industry of the United States. Is that the, the, the, the industry that we will become in order to achieve that 10 is the kind of industry that could plausibly do 20 or 40? You know, totally.
- [60:30]
B
I mean it's just like, I mean, you know, it's like any other industry. You just need like, you need a value chain, you need firms that can reliably deliver componentry on time. You need welders that can meet the requirements that we hold for construction of a nuclear reactor. Right.
- [60:45]
A
Ultra, ultra, ultra high reliability. Yeah. Welding jobs.
- [60:49]
B
Yeah, yeah. If my think tank career goes, goes bottoms up. That's my, that's my backup plan.
- [60:53]
A
20 bucks says robots are doing it in 20 years. But you know, yeah, could, could well.
- [60:58]
B
Be, or AI assisted, you know, you got the glasses on and AI assisted human.
- [61:03]
A
There you go. It's telling you what to do. Okay, so, but like, but that, that.
- [61:06]
B
Is, you're, you're right. You correctly identified why this is important. The point is not just we're going to tackle all this low demand with new nuclear.
- [61:15]
A
Nuclear.
- [61:15]
B
The point is we might find ourselves in the 2000 and 30s really wishing that we were again a nuclear superpower and we should use this opportunity to help develop that. The former colleague from the Biden administration, they saw incredible finance running into the development of this industry and wanted to lash the green agenda to that money stream. To that money stream. And I actually think that kind of like gets the, I think it gets it like a little bit wrong and a little bit right. A little bit right in that, yes. The hyperscalers and the tech companies, they want low carbon power. They don't want their revolution to sort of cause a backsliding on climate. And so they are willing to spend on a little bit, not, not a, not a ton in my, in my view for, for cleaner options. And like they're tech forward people, so that's cool. And we're seeing sort of equity investments from them in various nuclear providers and other things where what I think it gets a little bit wrong is like we need proven solutions for immediate effect. And, and so a lot of what we need to be able to demonstrate is the capacity to deliver on time. Time. And in today's system, what delivers on time is natural gas, solar and batteries. And so public policy interventions, in my view, need to demonstrate that we can deliver on time, on budget, predictably. This other class of solutions, transmission nuclear geothermal, is another power source that you could consider using in this context. I don't really care if we're using, using coal, if it's, you know, for doing something about the emissions like, you know, clean coal or carbon, Carbon, managed coal plants are, are a total viable option. We just need to devote ourselves to making sure that they're proven.
- [63:13]
A
Yeah. Amazing. So I have one more question I want to ask you on AI in the United States before we take it international. Yeah. And that is your, your most recent paper called AI for the Grid. So we've been talking for a long time about like, like the grid for AI. Now we're going to talk about AI for the grid. So what is this paper about? What did you find?
- [63:36]
B
So if you believe that this technology is going to transform our economy, absorb a trillion dollars of finance between now and 2030, and is of such economic and national security import that we treat it like we did oil in the 20th century, it seems obvious that we would use it in the electricity in the energy sector, which occupies like 7%.
- [64:00]
A
Well, they're, they're, they're, they're promising. They're going to revolutionize every sector. So energy should certainly be on the list.
- [64:05]
B
And like, and a lot of the problems we've talked about already in the podcast are, I think, amenable to a lot of AI solutions. Right. You've got large unstructured data sets, you've got optimization problems. So there is an objective criteria that somebody somewhere, maybe assisted by an algorithm is, is looking to optimize.
- [64:25]
A
Yeah, can I, can I dwell on that optimization problem thing for a second? Because, yeah, I mean, this is where machine learning, which is, you know, the main story of the AI revolution over the past 15 or so years is the machine learning revolution really shines in optimization problems, in complex super multivariable optimization problems. So if you think about Google, for example, Eric Schmidt, back when he was the CEO of Google, that's when they bought DeepMind, you know, which is their big AI subsidiary now. Now Google was already really, really good at data center efficiency because they ran more data centers than anybody else on planet Earth. And so additional efficiency in terms of, you know, how many computations per watt of electricity can you get out of these data centers, that just goes straight to Google's profits, right. Every dollar that they can wring out of that system. And Eric Schmidt was of the opinion that, like, look, Google is the best in the world at this. We already have the smartest people in the world working on this. Like, you know, sure, DeepMind, we're going to let you loose in our data centers to run some fun experiments, but like not really anticipating that you can juice, you know, squeeze more juice than we've already squeezed.
- [65:35]
B
Algorithm, sure.
- [65:36]
A
And then they showed up. DeepMind, I want to say I can't remember the exact numbers, but it was something like they gave them two weeks and they increased efficiency by 10%, which was like completely unheard of. Just by applying machine learning to like all the variables in the equation, when you run the cooling, when you turn on, you know, the lights, blah, blah, blah, blah. Just analyzing the awesome data set that Google had already collected, they found 10% of additional efficiency. And like most industries do not see 10% efficiency gains per year. So if that's like what it looks like in the, in AI optimizing the energy demand of a data center. What about AI optimizing the energy demand of the grid or like of, you know, the whole universe of other industries where this optimization has not yet taken place.
- [66:19]
B
Totally. Right. So like this is the big, this is like a big important question. And so what we looked at in this paper is not, I wouldn't say overall efficiency, but basically where are AI applications already being developed or demonstrated that'll help resolve the big challenges we face today? Growth, resilience and affordability.
- [66:37]
A
Yeah, so what, what were like some of the most exciting opportunities that you saw in those areas?
- [66:42]
B
So you can use AI to attack this interconnection queue challenge?
- [66:45]
A
Oh heavens yes. Yeah, that one really needs solving.
- [66:49]
B
Yeah. And there's a couple firms working on doing that. Google, there's a Google sponsored firm called Tapestry, there's a couple other ones that are running at various levels of publicity that I think like this is a problem that is amenable to software. Instead of looking at, you know, plant by plant studies you can look at, you can batch them. Instead of having humans email PDFs to each other, you can push these things as software. Right. Like there's just, there's, there's, there's going to be both like computational and logistical speed up that I think we can accomplish there. So that's one A lot of the transmission infrastructure, these are like very long power lines. Bear in mind again, this is like a big complicated machine. Right. Their capacity is weather dependent on a very hot day, the lines stretch and their capability in their capacity goes down.
- [67:45]
A
Oh my God. This is. Okay, I, I thought you were going to say like air Conditioning demand. But you're talking about the thermal expansion of the metal wire, adding miles of length to the grid. Oh, my God.
- [67:55]
B
Yeah. And so like, and so, you know, because we, the, the number one thing the power grid can't do is fail.
- [68:02]
A
Yep.
- [68:03]
B
We have relatively, I would say conservative small C conservative rules about how we assess capacity or how we rate those lines.
- [68:12]
A
Yeah, these guys are pretty paranoid, the people who run the grid. Right.
- [68:15]
B
Yeah. The sanction against failure is high. Right. And, and like the, and the UIS pays a lot of money to have a really resilient system. And so like. But if you could use, you know, if you instrument those with a bunch of temperature sensors, you can get it, you can build a model which will tell you. Well, we, we actually, you know, the old rule would have had us run 20% less than we're actually able to run on this day because it out turns, it turns out while it's hot, it's also windy. Right. Or whatever conditions you've got. And then now what AI allows you to do because it's a really useful tool in weather forecasting. And downscaling a weather forecast to regional conditions is instead of having to instrument the whole grid, you could say, well, we would be able to use algorithms to estimate what line rating could be without having to have the full coverage of, of the, of instrumentation like we traditionally would have had. So the value of that information or the value of an individual observation or with a sparser, you know, sparser data set can be, can be utilized. And in that context, there's like a lot of ways we could use the existing system much more efficiently.
- [69:23]
A
Okay, so like, like, like, yeah, yeah. Basically we never want it to fail, so we run it, you know, based on conservative assumptions. But AI could allow us to be.
- [69:34]
B
More dynamic and managed. Yeah.
- [69:35]
A
Be closer to the truth of the situation. And so our assumptions don't necessarily have to be as conservative if they are as grounded in reality as AI is inferring it from the available data.
- [69:46]
B
Exactly. Earlier in this conversation, you mentioned solar as a sort of a randomly occurring or a not consistent source of power. Right. Because on a cloudy day, the output of your solar plant can go down 40% if like just a cloud rolls over it. Right. And weather forecasting is imperfect. Well, it turns out that we have all sorts of reserve margins and power plants we keep running for such a condition. Again, AI tools can improve the weather forecast we use and help respond to the intermittency of the grid of all this solar and wind generation that we have. On the grid now in a much more sort of optimized way. And because machine learning and optimization and sort of general intelligence tools can be applied to different parts of this problem, both the optimization routines as well as the decision criteria. I think we're still probably far away from decision making happening in an agentic way, but you could definitely see power grid operators in the not distant future being sort of agent assistant in their management of the whole system. And if you can use the system more efficiently, then you can both meet new load and reduce the cost of the system or the growth in the system. And that would be very positive. Lastly, this is something your audience probably has heard quite a bit about. These very large data centers, can we turn those into instruments of grid reliability rather than risks to it? So if you're able to introduce a little bit of flexibility in the demand of a data center, such that when everybody else needs power, you can go from a gigawatt down to 700 megawatts or 500 megawatts for a short period of time, just on a really hot afternoon in Texas as an example, then that actually can. Is as much a grid service as you are a draw on the system.
- [71:45]
A
Interesting, because, yeah, I mean, my starting assumption, right, was that these things want to run 24 7, but you're basically saying like, no, the grid, you know, is basically saying, hey, there's a ton of demand right now. You would be saving us X if you could, you know, wait until tonight to run that analysis. That's kind of interesting. Yeah. And I mean, the AI system, they want to make money on the chips, which means they need to use the chips. But there's a difference between the demand when everybody's awake, right, which is like serving customers, and demand, like running training experiments where conceivably, you know, you could schedule those for nighttime. So I think the sort of background assumption of everybody has been that these things introduce like ultra stable demand into the grid. But your point here is that, well, now we're starting to learn, you know, whether or not these companies could live with more flexible demand schedules and be. Be assisting in helping the overall grid resilience.
- [72:35]
B
And that's not, it's not a simple proposition position, right? And so like, you know, these are complicated places and algorithms like, I know I don't understand all the details, but I know it's not so easy as just to say, well, let's run at 70%.
- [72:47]
A
I mean, my starting hypothesis is that the answer is no, because they'd rather just, you know, run.
- [72:52]
B
And I Would I would say like even from our conversations with industry players, that was the response a couple years ago. But two things have happened since. One, there was the realization that our power grid is going to struggle to grow fast enough to meet all this new level load with new generation. Yeah, that's just like that. The bottleneck. That's the paper we wrote. And while we have some ideas for helping relieve that, that's going to be a persistent problem. And so if a data center can set or a company can say we're going to build this data center, we're going to offer 100 hours a year of grid flexibility and we've got a model for the electricity tariff, the bill that we pay or that's going to allow us to skip the line and get off on, get empowered faster. I think there's an economic incentive because the opportunity cost of waiting for not having to be flexible 100 hours a year might be two years. And you'd much rather run for two years at minus 200 hours than you would have to wait. Secondly, there was a very, and this is like think tank inside baseball, but there was a very influential paper that came out of Duke University. It's like so influential now. It's known as the Duke paper.
- [74:06]
A
Oh, that's Mount Olympus in think tank world.
- [74:11]
B
Absolute home run in the takes game. But it showed that if you were able to do this under certain conditions, you could add something like 100 gigawatts of data centers onto today's grid.
- [74:25]
A
Wow. And I'm sure that got everybody's eyebrows up. Yeah.
- [74:28]
B
Oh absolutely. And we don't yet know all the economic incentives. We don't know how this, how regulators are going to like, are they going to trust that the data center will actually turn itself down.
- [74:38]
A
So, so basically the reason why they might say yes is that if they can do this, the grid might interconnect them a lot more like it might say yes to their construction request a lot more than they would have otherwise. So that makes sense to me.
- [74:50]
B
That is the leading incentive that we're seeing right now. This has been proposed by some as not, not just something that we're going to use as an incentive, but there would be regulations or rules around this interest.
- [75:01]
A
Interesting.
- [75:02]
B
That has encountered a lot of industry blowback, I think, you know, but it's a space to watch in that in a sort of, in an ideal case you would have, because these are very large loads, you might actually be able to sort of use the whole system at higher capacity if they're able to be managed flexibly at certain times or points of the year in different locations. Right. That's the optimal case. And so some combination of those things is, I would assume, what we're going to end up realizing is like some data center flexibility, some optimization of the electricity system, and then a boatload more electricity generation, which I will note for your audience, the United States needed to do anyway.
- [75:47]
A
Yeah, yeah, we needed to do anyway just to like electrify vehicles and to electrify so many different segments of the economy.
- [75:54]
B
But that makes sense, you know, Reshore industry, you know, meet a lot of other, A lot of other national economic.
- [76:01]
A
Yeah, I mean, a lot of, A lot of economic and strategic priorities among both Democrats and Republicans.
- [76:06]
B
Yeah.
- [76:06]
A
So now I want to end on an international note. So, you know, China has quadrupled their electrical generation capacity between, gosh, just 2005 and 2022. Right. So they know how to build a ton of electrical generation capacity in, in data centers. One of the biggest deals that was cut recently was with the UAE and Saudi Arabia where they're basically saying we want to be in the AI game. And what we're bringing to the table is an awful lot of energy and an ability, a demonstrated ability to build quickly. Like, what do you make of. Of these developments and how energy is a part of the AI story not just here in the United States, but around the world?
- [76:51]
B
I think, I think energy definitely is a, like a sort of a leading component of the story. And so the, you know, if you think about the concentration of data center demand that we talk about in our paper, this is really, this is like the most American way to approach the problem.
- [77:09]
A
Right.
- [77:10]
B
Is we're just going to throw an incredible amount of compute at it.
- [77:13]
A
Yeah.
- [77:13]
B
And we're going to burn everything we have to to do. So, you know, if we fail to do that, that I suspect we will see a lot of offshoring of data center activity because there's. The chip pipeline is relatively well defined and the economic incentives appear to be there. Yeah, the, you know, the places you've talked about, the uae, Saudi Arabia, they see a lot of potential. The, you know, we even some countries like in Europe are starting to say, hey, we want to build a gigawatt scale facility like the Greeks. There's a new project announced in Norway. Way, you know, there. I would say that. And then the last one that you missed is Japan, where there is a lot of nuclear capacity that was throttled after the Fukushima incident and could plausibly be restarted. There's like Social license and other issues that you'd have to deal with. But, but if in a world where we are underserved, served in compute because we don't have enough electricity, those are the, those are the main places to watch. You bring up China, China's remarkable story, their, you know, their ability to build and add to their power sector shows all the muscle memory that we are trying to regain here in the United States.
- [78:33]
A
Right.
- [78:34]
B
They have, you know, there are licenses that are available to Chinese planners that we don't have here. Right. Property rights and a variety of other things. But our model has grown in percentage wise ways that are not inconsistent with Chinese growth over the last 20 years early in the 20th century. We just need to regain our ability. To do it requires a sense of optimism and confidence in the electricity sector. And if you kind of make a point by point comparison, in China is building now a very diverse generation fleet. They build a lot of coal plants, they have a lot of coal plants coming down the pipeline. They sort of use coal plants in their system like we use natural gas. They're building more renewable capacity every year than like the rest of the world combined. They have distinct programs for running data centers in the west western part of China where they're, they're rich in renewable resources and, and they're very, very efficacious in bringing online nuclear power and other things. So we want to chase all that. Interestingly though, their leading models are quite energy and compute efficient. And so I think one of the vectors that you and I need to talk about or we need to talk about more is like, is the right strategy go whole hog on compute and does that kind of create a decadent set of model training exercises when, when some efficiencies would be useful?
- [80:07]
A
Oh, I'm, I'm, I'm on the other side of this argument. You are. So, so I would say that the model efficiency over the past 10 years, I don't know the exact number, but it's like probably around a thousand fold. So the idea that China alone is driving the efficiency story deep seq is a point on the exponential curve of model efficiency growth and that exponential. All the other points in that curve are American and uk. You know, it is just the first time China has been a part of the story of crazy growth and model efficiency. The Americans have all the exact same incentives that the Chinese do to drive additional model efficiency. And on the Chinese side, it's probably about to go thrown in reverse because a lot of the hypothesis there is that if we won't sell them Nvidia chips, they'll have to use lousier Huawei chips. And the question is, can they overcome that by building more of those chips, using more power to get to a comparable level of overall compute? I think is where the Chinese story is headed in the future. The future.
- [81:24]
B
Okay, interesting. Well taken. I, I do think that, that that is a key area to watch is the sort of, you know, how well can they shift and they, they can do lots of stuff in China. Right. Like, you know, they have excess capacity in steel and you can just like turn plants off to.
- [81:39]
A
Right, right.
- [81:40]
B
And like just things that in the United States we don't, we haven't traditionally practiced.
- [81:44]
A
Yeah. So, Joseph, this is like exactly the kind of conversation I wanted to have. Thank you so much for coming on the AI Policy podcast. Let me just ask, what do you got coming down the pike? You just got this paper out, which is exciting. We'll link to it and your other work on this topic in the show notes. What else should folks look at coming out of your program in the not.
- [82:03]
B
To'S in future in this context, there's like three things that we're working on that will come out over the next, I would say six months. I actually do want to do a more rigorous study of what's going on in China with powering data centers.
- [82:15]
A
Great. Love that.
- [82:16]
B
And with incorporating AI in the, the grid. In writing the paper I published in September, I did a little bit of surface research on how are these applications that I'm talking about being deployed in China. And there's an interesting body of literature that seems to indicate that they have a much more centrally managed electricity grid. No surprise, but that actually can help in deploying new tools quickly. And so I think I want to, to learn a little bit more about what's going on in China with respect to grid optimization and the use of AI in, in the, in the, in the energy system itself. We're doing. I want to do some more regional studies. So what we, you know, you talked earlier about the clustering of data centers in Virginia, Texas. You know, we see looking at the semi analysts or other data, semi analysis or other data like new concentrations in sort of the Midwest, like Ohio, western Pennsylvania. There's a lot of spare transmission cap up there, some spare generation capacity. I want to understand how this story is unfolding in different regions and what lessons might be learned. You know, like not every state's going to be Texas, but are there, are there sort of durable lessons that, that we can apply. The DOE has an open request for information on sort of how it can use its programming to help speed to power. And I think that there's, there's going to be some opportunities to answer that question with more regional stuff studies. And then lastly, I'm really interested in the question, more of a macro question. Does an AI rich economy get more energy intensive or less energy intensive in the United States? That's interesting because for the past 20 years we had a declining energy intensity of productivity in the U.S. right. And I don't think we have enough. We know. And I've asked like every really smart energy modeler I know know, do we anticipate AI sort of the efficiencies that it can bring to the economy improving our, our use of energy such that even as it adds both its own energetic demands in terms of data centers and the energy demands associated with hopefully another point or point and a half of GDP growth.
- [84:31]
A
Right, Right.
- [84:33]
B
Will we be able to through efficiency outpace that growing demand? Or do we need to build an energy strategy for the US that foresees using a lot more energy in sort of like more energy intensive modes of production re industrialization, everybody riding around in their own personal EV toll? I actually don't think we have a good sense of the macro picture in the same way that we've developed it around for instance, labor productivity.
- [85:04]
A
Yeah. And there's just a range of scenarios there and it'd really be worth understanding like what, what each of them would mean. Those are three awesome papers that I am very excited to read of yours. So Joseph, thanks again for, for coming on.
- [85:20]
B
Thank you, Greg.
- [85:20]
A
All right, take care. Thanks for listening to this episode of the AI Policy Podcast. If you like what you heard, there's an easy way for you to help us. Please give us a five star review on your favorite podcast platform and subscribe and tell your friends. It really helps when you spread the word. This podcast was produced by Sarah Baker, Sadie McCullough and Matt Mand. See you next time.