← Shift Key with Robinson Meyer
Shift Key with Robinson Meyer
Nvidia’s Case for Why AI Will Cut Emissions
00:48:42
Rob sits down with the Josh Parker, head of sustainability at America’s world-leading chip designer.
Loading summary
Transcript60 lines
- [00:00]
A
This episode of Shift Key is brought to you by heatmap Pro. You already rely on heatmap for daily
- [00:06]
B
reporting and commentary on the energy transition.
- [00:08]
A
That's why you listen to this show. Well, Heat Map Pro brings all of our research, reporting and insights down to the local level. It's a software platform that tracks all local opposition to clean energy projects and data centers. It forecasts community sentiment and it guides
- [00:25]
B
data driven engagement campaigns.
- [00:27]
A
Go to heatmap News Pro to book a demo and see the premier intelligence
- [00:32]
B
platform for project permitting and community engagement.
- [00:36]
A
That's heatmap News Pro. Hello, it's Tuesday, May 26, and the first unofficial day of summer here in the United States. Or maybe the second, depending on how you count. And at least as of when markets closed on Friday, the chip maker Nvidia was the world's most valuable company. It currently has a market cap of around $5.3 trillion. The next biggest company, Alphabet or Google, is worth 4.6 trillion. Just last week, Nvidia released its financial results for the first quarter and they were another blowout. It turned in almost $82 billion in revenue, which was up 20% from Q4 of last year and up 85% year over year. It's now beaten Wall street's expectations for 14 quarters in a row. Now, if you follow AI, you know that's because Nvidia's chips are still the best option or one of the best options on the market for companies that want to power a large language model like Claude or ChatGPT or another Frontier AI model. I go into all of this not because Shift Key is a technology business podcast. We are not, but to illustrate the centrality of Nvidia to the current artificial intelligence boom, and I think to the broader American economy too. Nvidia produces the physical infrastructure of the AI and data center explosion. And since that boom is the biggest story in electricity and climate and even energy right now, I think you could argue that Nvidia is one of the most important companies in electricity, climate, and even energy. After all, America's tech companies are tripping over themselves to build solar panels and batteries and gas turbines specifically to power warehouses that are specifically built to hold Nvidia chips. Utilities are tripping over themselves to power Nvidia chips. Communities, as we've reported, are fighting those warehouses that are built to house Nvidia chips. Nvidia's chips are like where America's dominance of the global artificial intelligence industry meets America's physical economy. The actual electrons and copper wires and wooden poles and infrastructure that runs through American towns and cities. So on that note, I'm really excited today to welcome to the podcast Josh Parker. He is Nvidia's head of sustainability, a role he's held since 2023. Before that, he was head of sustainability at assisted General Counsel at Western Digital. Josh and I had a good conversation last week. We talked about why he thinks AI is a net good for climate change, about whether AI and Nvidia are already cutting emissions on the power grid, and about Nvidia's work with clean energy companies as well as fossil fuel companies. It's a very interesting conversation. I learned a lot from it. I'm Robinson Meyer, the founding executive editor of heatmap News, and it's all coming
- [03:17]
B
up on Shift Key. Josh, welcome to Shift Key.
- [03:24]
C
Thanks. I'm thrilled to be here.
- [03:27]
B
So you joined Nvidia in August 2023, which was. Right a few months after ChatGPT came out and completely changed the AI conversation. What did you walk into at the time and where was the internal conversation around sustainability and climate at that moment? In Nvidia.
- [03:44]
C
It was a really unique and wonderful time to join Nvidia. You know, the company was just doing amazing things. The whole world was starting to wrap its head around the fact that AI was useful and was finally here in ways that would transform the world, transform the economy and really our existence. And so the timing was, yeah, fantastic for me, really thrilling just based on the where the company was, what it was doing, and the whole conversation around it. The sustainability conversation was one of growing interest at Nvidia. Jensen, our CEO really has this vision of technology helping to solve the world's biggest challenges. And sustainability is, of course, one aspect of that. Things like climate change and materials resources and water conservation. And he believed that AI had a very critical role to play in sustainability in the near future. And the company was looking to expand its sustainability program and efforts. And so I was very fortunate to come in at a time when the company was really trying to accelerate that program and find new ways to use tech for good and also to be a responsible organization ourselves.
- [04:56]
B
How do you think about Nvidia and sustainability today? What are the goals that you have? Because obviously at this point depends slightly on the day, but recently it's the world's most valuable company. It's driving this enormous infrastructure boom. Nvidia provides the physical infrastructure of the AI boom. And so to some degree, it's in every sector of the economy. Economy story. And I wonder, given the company's enormous importance right now, how do you think about its sustainability goals and what you focus on?
- [05:26]
C
Nvidia is a pretty unique company, just across all the metrics. The culture here is very unique, very dynamic and we could get into that and have a whole podcast on it, but the sustainability space follows that same pattern. We have a very unique approach to sustainability, I think based on Nvidia's role in the ecosystem. One of the first things that I did when I joined Nvidia was to start some analyses, some incredible third party validated products, carbon footprints for some of our high volume projects, to figure out what does the data show us about where our lifecycle impacts are. So if you look in gaming or in AI or 3D modeling pro visualization, what are the kind of soup to nuts, beginning cradle to grave hotspots for emissions in particular, and then other impacts as well. And when you look at that, you very quickly realize that Nvidia's direct footprint, and this is something most people would understand just conceptually, Nvidia's direct footprint is a tiny, tiny fraction of the total lifecycle impacts of our products. So while traditional sustainability programs, especially tech companies that involve manufacturing and perhaps downstream use as well, really focus on their own footprint, if we focus myopically on our own footprint, we're missing the forest for the trees. So very quickly realized that Jensen's vision about sustainability and about AI's potential to impact sustainability issues was much, much more significant than Nvidia's direct impacts through our operations. And so as a result of that, we've been focused from day one, really on trying to unlock applications of AI for sustainability and to work with our value chain partners, both upstream and downstream, to decarbonize, to manage impacts, et cetera, across the value chain. So it's a lot more outward focused sustainability program than most, which I think makes a lot of sense based on where Nvidia sits in the ecosystem.
- [07:25]
B
And can you talk a little bit maybe about why that is? Because I think what many listeners I expect will understand. But just to be clear here, Nvidia designs its chips and it operates them as well, but it doesn't actually produce the chips. The chips are usually produced by TSMC or another outside chip fab. And so I guess from your standpoint then that makes these external projects especially important. But like where did the emissions, I guess, in Nvidia's world come from? Do you focus on emissions as, you know, a key metric here?
- [07:54]
C
Yes, our top issue for sustainability since I arrived at Nvidia has been emissions and climate change. So that has Been the top focus for us. And yeah, if you look at the value chain emissions and those product carbon footprints that I mentioned, we've published summaries of those that are cradled to gate. So they start from the very beginning of the value chain and end kind of when we ship our products to our customers because we don't have as good a visibility to how our customers are using our platform. But we are as a company. Historically it's accurate to say we were a chip design company. Nowadays we're more of kind of a platform infrastructure solution company. But we are focused very much on the design. So on the AI side we do very advanced networking. We have CPUs, GPUs, data center architecture, we co design things like cooling solutions for data centers and we publish reference designs for those. And then we work with manufacturing partners, contract manufacturers to actually build the systems and then to sell them. And then we do operate some data centers. But most of our business is really selling the tools, the infrastructure to the company. Companies that go out and build great things with that infrastructure.
- [09:11]
B
What's the most important metric to focus? I mean, we were talking about emissions, but in terms of understanding kind of Nvidia's sustainability goals, what's the most important metric to focus on?
- [09:22]
C
I think at a moment when AI is growing, rapidly transforming the world, the most useful metric is one that takes into account both the footprint and the handprint, so takes into account the impacts of, as well as the potential offsets, the benefits, the transformational impacts down the road. Now consolidating that into a single metric is really difficult, but there are some studies that have tried to look at least the net impact on greenhouse gas emissions of AI broadly. So that's, I think the best indication of is AI a hero or a villain or somewhere in between in terms of climate change and greenhouse gas emissions in particular. And the very rapidly growing consensus is that AI is most likely to lead to net emissions reductions, especially if it's deployed broadly. So organizations like the International Energy Agency, World Economic Forum, Boston Consulting Group, Grantham Institute have all come to that conclusion that AI, because of its transformational impacts on other sectors, in particular around energy efficiency and so forth, is poised to drive net emissions reductions. So if I were to pick a metric, I would say what's the net impact on emissions that AI is creating? And it's really a positive one if you look at those studies.
- [10:42]
B
So I think that this gets at to some degree the question that I want to talk about while we have your time, which is that there's enormous focus on the energy use from AI. Right. And of course the energy use from chips. And we can talk about chip efficiency and what Nvidia is doing there. And I think it'd be good to talk about it. But it does seem like to kind of step back that we are in this moment of massive infrastructure investment in AI and that infrastructure investment is going to happen regardless. At this point, I think it's just AI is too valuable, it's too obviously useful for that infrastructure investment not to happen. And what we track at heatmap and we look at data centers get built across the country and we become aware, for instance, that there's a lot of off site behind the meter gas being built to service these data centers. Obviously there's going to be a surge in electricity demand and there's ways in which electricity demand increases can be good. But just as we like think through the next five years, given that at this point the AI investment boom is happening and to some degree, you know, the AI story is a foregone conclusion, what needs to be true for AI to have been good for the climate or for Nvidia's efforts here to have been good for the climate?
- [11:57]
C
The biggest variable in that analysis of what's the net impact of AI is really, again if you look at those studies that I mentioned, including the International Energy Agency, is how broadly we apply it in the near term. So yes, the infrastructure is getting built, it's getting used. And contrary to what most of us consumers conceive of as AI, the vast majority of the really useful cases of AI is not the chat bots that you're engaging with. It's not the dogs surfing in Hawaii videos and photos that people create and in their spare time. It's the commercial applications where AI is saving energy, it's saving material resources and so forth. And that infrastructure is being deployed for that purpose in addition to the chatbots. And the real opportunity for us is to say, okay, we've got these amazing models, you've got Claude, you've got Gemini ChatGPT X. They're really, really powerful and obviously just growing in capabilities month over month. There's so much potential there for those to transform manufacturing, for example, with digital twins. And we see proof points of AI reducing energy in manufacturing by around 30% across the board. If AI is deployed to optimize manufacturing for energy, that happened at one of our manufacturing partners in Guadalajara, Mexico, for example. 30% reduction in energy. And so the opportunity is, and the risk is that if we build out all this infrastructure and we don't use it effectively. If we don't apply the AI to these big problems, then we may miss out on those significant emissions reductions. So what needs to be true? The biggest variable here is are we taking advantage of what we've built because the infrastructure like you said, is being built and it's being used, but can we deploy it more broadly and can we bring in some of the sustainability focused organizations to deploy it for good? How do we intentionally use AI for good? In addition to the kind of regular efficiency, revenue and cost driven allocations that are happening very naturally and have very, very significant gains across sustainability, they're also very purpose driven applications of AI that can have big impacts as well.
- [14:17]
B
Do you think that AI by itself increases efficiency where it's applied in that if you apply it to manufacturing for instance, or another one of these industri uses, that it's going to just increase the efficiency of that process by dint of its application and being very intelligent and finding ways to streamline processes or skip processes or augment processes that maybe wouldn't have been considered otherwise? Or does it need to be applied in an intentional way where people say we need to look at this for efficiency or for emissions? And that should be our main focus here.
- [14:54]
C
That's the beauty of the concept of efficiency in free market is that the incentives to reduce costs are really well aligned with sustainability goals of reducing impacts, reducing consumption and so forth. And so what we are seeing, and I think this will even grow more over time once we get out of this kind of Cambrian explosion of tech innovation that we're in right now, which is a little chaotic, is that you'll see optimization of okay, using a huge LLM for this problem might be good, but it might not be the best tool for that particular task. Can we use a lighter weight model? And you see tons of innovation in this space. Mixture of experts has been around for a long time. We're seeing a lot more innovation around how to use more efficient models and target them to specific applications. But the market and kind of customer demands and everything is really driving us well. Plus supply constraints, compute constraints are really driving us towards efficiency and to optimized allocation of those resources. And if AI doesn't end up being the right tool for every task, then it won't be used there. And we can continue to use traditional techniques. But efficiency does happen to be one of AI's kind of low hanging fruits, one of its superpowers that is really easy to unlock and unlocks value immediately across the board. So it is very fundamentally true in general that AI does drive efficiency very, very rapidly in most areas.
- [16:26]
B
I think what I hear you saying is that a lot of the good that will ultimately come from this build out though will be done from intentionally applying AI to intentional sustainability problems. Is that wrong or is it also just the diffusion? I mean we were just talking about efficiency, so I guess that's on the other side. But in your kind of first answer, I did hear a sense that a lot of the most important work on sustainability will come from Nvidia intentionally applying itself technology to sustainability problems.
- [16:56]
C
I would say that's important mostly because it does require us to think about it and to do something. It's not being driven necessarily automatically by existing incentives and market dynamics. So the market dynamics and the efficiencies that are being driven by that, like a 30% reduction in manufacturing efficiency, it's really mind boggling when you think about were concerned about the energy that is being consumed by AI. AI still represents less than 1% of total electricity consumption worldwide now. It's obviously higher in some regions, higher in the United States, and it's about
- [17:33]
B
to go up a lot too. Is the.
- [17:34]
C
Yeah, no. Expected to double by 2030? So it's growing very rapidly. But if you think about AI's existing footprint again, less than 1% of global electricity right now, even if it doubles, doubles again, doubles again, it's still going to be a small share of global electricity. If as we're seeing the proof points for it can reduce energy in much, much larger energy consuming sectors like transportation, like buildings, like industry, which are each in the 20 to 40% range of global electricity, then those savings dwarf AI's footprint unambiguously. And that incentive is there because companies want to reduce costs, they want to reduce their energy consumption. When we're in this environment of energy constraint, particularly in the United States, the incentives are there. So that is going to happen. I think that's kind of inevitable because it's an opportunity, there's value and there's sustainability. It's good for everybody and the stars have aligned. The additional piece is applications of AI intentionally for sustainability. And that's where maybe it won't happen unless we think about it, unless we try to apply it there. And the potential is just phenomenal. When you think about the way AI is already transforming drug discovery and healthcare and material science, there's potential in nuclear fusion, advanced fission, geothermal and carbon capture and storage just across the board. When you add intelligence to these sustainability challenges, you arrive at this Wonderful inflection point where we might finally have a technology that can sufficiently complement policy to help us actually prevail on some of these sustainability challenges, help us to kind of reverse things and make progress that we otherwise wouldn't have the opportunity to do.
- [19:29]
B
There's two types of AI that we're talking about here and I wonder if we can disambiguate them a little bit, in part just for my understanding. So there's the large language models which I feel like are the charismatic megafauna of AI. This is Claude, it's ChatGPT, it's Grok. Those are the models that I think people are most likely to have experience when they think of AI. But there's also this whole other set of AI applications which I feel like you've alluded to, applying it to manufacturing, applying it to drug discovery, applying it to energy. And my sense is that type of AI, it doesn't look like Claude or it doesn't look like ChatGPT, it might have the same kind of organic structure where it was trained on a large data set and kind of allowed to self train itself on that data, but it doesn't have the same interface. It's much more kind of machine brained than maybe the LLMs of the world. And to the extent you could share this data, to what extent is AI demand and Nvidia's demand and energy use coming from the LLMs of the world like Claude and Grok and ChatGPT versus these other AI applications?
- [20:34]
C
So it is true there are very different applications of AI depending on the sector and the consumer facing chatbots that you see are one small use case and not where you see the biggest opportunities for advances in sustainability through AI, of course, things like digital twins for example. And that's a really interesting marriage of Nvidia's expertise in 3D modeling and AI. And that is a very fundamentally valuable concept and technology for things like the manufacturing optimization that I was talking about.
- [21:09]
B
You build a digital simulation of a real life factory or physical space, right?
- [21:16]
C
That's right, yeah. And they become, it's a lot more than what it sounds like at first blush, just, you know, a 3D rendering of a building. You actually can simulate robots, you know, going through this factory, simulate the airflow through the factory and the cooling system and all of the impacts of various factors on it. So it's, it's very complicated and the emulations enabled by the AI really make the technology as valuable as it is today. That's one example of something that is Obviously not a chatbot that is fundamentally just extremely valuable when it comes to sustainability applications of AI, but there is actually substantial overlap. So when you see anthropic training, Claude Opus, and devoting all of these resources to training that huge LLM, so many parameters, and same thing with ChatGPT and Gemini, those very large language models end up being really useful tools for helping us create more bespoke, lighter weight, custom models as well that can do other things. So the multimodality functionality of modern day LLMs is just going through the roof. And the result of that is that these foundational models become even more valuable for lighter weight, more tailored applications of AI. So it's true that the actual application of them in other areas probably won't be the exact same model that was the huge foundational model that you started from, but through distillation and other techniques, you may end up using that as the basis for one of those other models.
- [22:57]
B
There's been a lot of excitement, and I believe Nvidia has invested in a number of companies, or at least emerald AI companies, that are looking at whether data centers can be flexed up or down to meet the grid needs of the moment. So instead of data centers simply being a huge energy suck on the grid, they could modulate their usage and they could modulate their compute and therefore their energy usage to kind of meet the grid's needs. I know Nvidia's invested in this. Can you give us a sense of how, where does that project stand right now in between, say, white paper and, like, deployed scale?
- [23:35]
C
So we are actively deploying this technology at our data centers. We're building a data center right now in Virginia that will come online, I believe, later this year. That is, we think, the world's first entirely flexible data center for AI. And we do see this as the future because it leads to a situation where we're making better use of existing energy resources. And this is something that's really, I think, underappreciated, and it might be a little nuanced for most people who don't follow this to appreciate, but the concept of AI data centers becoming grid assets is really powerful because they're being deployed rapidly, they're using a lot of energy, and if they end up being good citizens of the electrical grid, then that can have actually a profound reductive impact on energy prices for retail consumers like you and me. The concept here is you have a grid that is built for peak load. So in the middle of the summer in Texas, when everybody's running their AC units, and you're consuming the maximum energy that the system can deliver, that is what the system is designed for. So when you're not at peak load, what does that mean? That means that all of those resources that you've built for the peak load are being underutilized. And so this leads to the conversation about smart grids and virtual power plants, where I think everybody that looks into this closely wants to get where we're saying, okay, how can we be more flexible, both primarily with our demand, but also on the variable generation side, how can we make better use of wind and solar that aren't firm power sources? And data centers play a huge role in that, especially as they become a higher percentage of electricity consumption in the United States. If a data center could say, okay, I'm in Texas, I'm in the ERCOT region, and it's hot day in late July, everybody's running their ac, I'm going to curtail my electricity draw slightly for a few hours until the system can get back to below peak load, and then I'll ramp back up. That ends up becoming a net asset because you're able to soak up the electrons when they're more available and then reduce your load when they're less available, which means we're paying money for electricity that's otherwise being unused with existing grid infrastructure. So it's fantastic for consumers, it's fantastic for the energy sector, and it's good for data centers because it means we can build them sooner and take advantage of existing resources. And one last comment on this. You may know that the concept of emerald AI in this dataset, or flexibility, ties back to a study last year by Tyler Norris at Duke University, who
- [26:24]
B
said there's a hundred gigawatts and a shift key listener. I believe so, yes.
- [26:28]
C
Anyway, as am I. Yeah, just. I want to get that in there as well.
- [26:32]
B
Thank you.
- [26:33]
C
Yeah, no, it's fantastic work that you do. Shift key and heat map. So 100 gigawatts, that is a ton of energy that could be accessed if we just ask data centers to be flexible for 1% of the year. And so that's the concept here. It's making the energy sector electrical generation more efficient, which leads to lower prices over time and better utilization.
- [26:55]
B
I think when Tyler's paper came out last year and when there was the initial wave of discussion about flexible data centers, the thought was that data centers would be flexing their compute, that they would change the operation, the programming, or the level of training that was happening in the data center at that moment to match real life grid conditions. Since then, the focus has shifted more to data centers flexing how much energy they draw draw from the grid. But maybe the training itself or whatever compute is happening being more stable. It just the question is whether the facility is drawing from the grid or from battery storage that's on site. When you talk about this data center in Virginia, or when you talk about flexible data centers going forward, are they flexing the compute mostly, or are they mostly flexing their grid use and their where they draw electricity from? And sometimes they're drawing electricity from the grid and sometimes they're drawing it from on site batteries. But most of the flexibility per se is coming from where the electricity is coming from and not how much electricity is being used.
- [27:57]
C
It's really a mix. And where we end up will really depend on what customers the data center is serving, whether it's a mix, whether they're being served locally, whether it's focused primarily on training versus inference. So what we'll end up seeing is there will be a wide variety, I think, of data centers with different types of flexibility, perhaps based on the needs of the data center. So if you have a data center that is running critical infrastructure and needs to be available even at peak load, then you may have more incentive to build out a large array of batteries so that the grid so that you can continue to use that compute even when you're at peak load on the grid. And you can still be a good citizen of the electrical grid by reducing your draw from the grid. But there are three different types of flexibility that we're building into this framework. One of them is what you mentioned with batteries, where you can say, okay, grids at peak load, I'm going to use my batteries now temporarily instead. Good citizen. The second is also what we've been discussing, which is when you just ramp down your compute, you can say, some of the workloads that I have, I can pause on for a couple hours without deteriorating service or having any significant problems, it's okay to pause right now. The third type of flexibility that doesn't get spoken about as much, but that is rapidly developing, is geographic flexibility. So if you have workloads that are really vital, but maybe you don't have the battery storage on site to keep your compute running full steam all the time, you could actually transmit that workload to a different geography. Maybe somewhere in the Pacific Northwest, they're not experiencing the same heat wave that they are in Texas. And the way a lot of interaction with AI works, that additional latency due to the different geography isn't a huge factor because there's already some delay built into the computer.
- [29:48]
B
So is that training or inference that you would move geographically? Like, would you send the inference out to the Pacific Northwest, or is this. You would actually send a training task out to the Pacific Northwest, and then it doesn't matter in some ways because training doesn't happen on a scale that the customer is always aware of.
- [30:06]
C
Technically, either is possible. Training, because it's kind of a large workload, chunking it up into discrete bits and then moving the data to the location where you need to continue the training does have some additional complexities to it. Inferencing is a little easier to move because it's smaller chunks, smaller amounts of data, and either one, again, because of the different latency requirements for AI compared to a traditional data center service, are feasible for a lot of workloads, some inference workloads, the latency does matter. If you're doing real time robotics and things like that, you do care about latency. So I don't want to overstate this, but there's a lot of inference that can happen where the latency is not a huge issue. And so those types of workloads could be shifted.
- [30:51]
B
In some ways, the geographic flexing kind of addresses this. But when we talk about flexing compute or flexing grid use and turning data centers into grid assets, I do have to ask, I mean, are data centers getting built in the places where that capacity or that flexibility is useful? Because it often seems like, especially at this point, they're getting built in places where there's just energy that's efficient or profitable to use because compute and energy are so constrained at this moment, and maybe not in the places where, say, that flexibility is useful. Do you see that changing? Or are we going to go in and maybe make existing data centers flexible in places like, say, the Mid Atlantic or Texas, where that flexibility could be actually useful to customers?
- [31:40]
C
Again, I think there is a. We'll end up with a mix. So right now, especially because of the challenges that we see in getting access to energy in the near term as we're rushing to build AI because it's so valuable and so important to us, you do see data centers being built just where they can get online, where there is electricity available. And you do see increasingly some of these companies bringing their own energy, building new solar farms because they need it, sometimes bringing online new gas. But the good news is this flexibility is available in the future when we need it. And the companies that are Building, bringing their own energy to their data centers. I haven't heard of any that really want to be off grid. It makes a lot of sense economically and conceptually for data centers to be part of the grid so that they can be assets, they can take advantage of the shared resources, offer benefits to the grid through improved utilization, et cetera, especially with the flex technology. So I think where we end up will be a highly innovative, interconnected mesh of data centers that can flex and can transmit data. But we do have some hurdles that we need to cross to get there, especially in the United States. So permitting, reform, transmission, of course, the things that we always talk about in the energy sector. This could be the golden moment where there is enough consensus around the importance of AI from an economic development, national security, scientific discovery, sustainability perspective that we can find a way to make progress on these important issues and break through some of those backlogs. If we can do that, what we'll end up with is a smarter grid, more robust economic development, more sustainable outcomes. It really will be good for society generally and help with energy affordability as well.
- [33:35]
B
So the data center that we were discussing earlier, you said is set to come on later this year. I think a lot of this conversation about data center flexibility is future focused, is looking at improvements that could happen in the future. Is there a substantive example of using AI on the grid right now to improve the supply side or the overall efficiency of the grid?
- [34:00]
C
If you're asking about kind of the data center flexibility piece, we have run several pilots in conjunction with Emerald AI in Chicago, Virginia and the UK to demonstrate that this is viable and works. I'm not aware of it being implemented fully at a data center yet. I think this Virginia one that we're building now is going to be the first one that is really built around that concept. But the pilots that we've run the demonstrations have been really impressive. They've kind of hit all the metrics that we were hoping to achieve. So we think that it's been demonstrated conceptually and we're excited to see it work in real life with this, with this new Virginia facility.
- [34:38]
B
So when I think about the AI electricity and AI energy use story, thinking back almost to 2023, I think when AI was forecast or projected to be a very large user of energy, frankly from a lot of folks I talked to, including guests we had on very early episodes of this podcast, there was a lot of skepticism because if you go back 10 or especially 20, 25 years, at the end of the dot com boom and the beginning of the aughts, there was a lot of fears that electricity, that computers, personal computers in that case, and server farms to a lesser extent as we called them then, were going to be a major user of electricity across the US and they really weren't. Those concerns really never panned out. And that's because the actual chips, the computers themselves, got more efficient. Now, of course, it's become a big user of electricity. It's totally transforming the energy system. We're compute constrained, we're energy constrained, we're in a very different moment. And that has put these efficiency gains that Nvidia has made in its chips in a totally different light. And so Nvidia has unlocked enormous efficiency gains in recent chips. The new AI chips are far more efficient, I think 95% more efficient than previous generations. But this seems to be contributing to a dynamic like a so called Jevons paradox where we're using them more. I wonder how you think about the Jevons paradox and AI and do you think we're going to get to a point where the raw efficiency gains from AI ultimately do lead to a leveling off of energy, or right now are just all those efficiency gains from Nvidia going basically to just using AI more?
- [36:20]
C
So I love Devin's paradox in this context because I think it says something really fascinating about the unique moment that we're in. So absolutely, the efficiency gains that we're seeing in AI are just astounding. And I'm not aware of any technology in history that has seen the type of efficiency gains, the magnitude of efficiency gains that we've seen in AI over the past decade or so. We're talking 100,000 time improvement in energy efficiency in the past decade. And the iea, their estimate, which is actually a little lower than ours, is that on average we see a 10x improvement in energy efficiency year over year with AI. And that improvement, which means, by the way, if you're running an AI task now and you run the same AI task in five weeks, on average, it will use half the electricity in just five weeks. Again, aggregate and average if you're doing the same task. So that is a huge countervailing variable in terms of aggregate energy use by AI. But of course, the reason we're building out more data centers and we need more energy for them is because AI is so incredibly valuable that even despite those energy efficiency gains, we need more of it. The scaling laws are holding so that more compute does translate into significantly more intelligence. And that intelligence is what is driving value across sectors in so many different areas. So to answer your question about where do we end up, I think it's very clear based on what we've seen over the past couple of years, aggregate energy is growing, that's focused on AI. Still relatively low baseline, globally again, but it's growing and we expect it to continue to grow rapidly. Now the question is that a problem? And I think if, if you look at it, there's again this risk of losing the forest for the trees. On the sustainability front, do we care if AI uses more energy consumption if at the same time it's reducing energy in other sectors at a much faster rate? So what we care about with emissions is net emissions. What we care about in energy, it's actually less clear because sometimes energy growth is actually a good thing for sustainability through advancements in clean energy and so forth. But if you just look at the emissions side, what matters globally is the net. And even if AI grows, doubles, doubles, doubles and doubles its emissions as well, which I don't think is the case based on the data, you'll end up in a world that has net emissions reductions because of the huge impacts that it's having positively in other sectors.
- [39:06]
B
Is there a current sector, though, where we can point and say emissions reductions are happening on a scale commensurate to the increase in data center electricity use
- [39:16]
C
in the near term? At the sectoral level, I don't think that's true, and that's because we're not deploying AI rapidly enough. Back to the earlier point about what is the key variable to capturing those emissions reductions. And again, going back to the manufacturing case, that kind of makes sense, because for the economics of energy efficiency to convince you to tear down your existing manufacturing facility and build a new one that's optimized, that's a much harder case. But as everything gets naturally upgraded as you're ready to build a new factory, because the old one is ready to come offline, AI is undoubtedly going to be utilized in those circumstances. So over the course of the next decade, we will see entire sectors, I think, driving those net reduction that we're already seeing the proof points for.
- [40:06]
B
But it does sound. We are kind of in an interesting moment here where we are making a big infrastructure bet. And I understand why we're making this infrastructure bet. And it's kind of irreversible. And we think there's a benefit on the other side, but we don't fully know that yet, at least on the emissions front.
- [40:21]
C
I would say that's true, but I don't think I haven't heard any arguments that suggest that the fundamentals don't compel us in that direction. So again, sticking with manufacturing. But transportation and buildings are similar. If you're building a new building and you have the option of using AI to manage the H vac, manage the energy consumption, and you expect a 15 to 20% reduction in your bills, of course you're going to use it, and the economics just work out. So I don't think it's a question of if, it's just a question of how rapidly the AI gets used for those purposes.
- [40:56]
B
Nvidia is working with a lot of companies and industries who I think have a very natural and mechanistic interest in improving their efficiency and who are very interested in improving their efficiency. Nvidia is also working with slb, which I think of still as being called Schlumberger, putting together an AI factory for energy and for conventional energy and unlocking more fossil fuels. And it does seem to me that this is the place where AI could run against some of these sustainability goals, that instead of improving efficiency everywhere, it could cause, in the same way that we're talking about Jevons Paradox, it could cause a general acceleration and unlock more fossil fuels and unlock more oil and gas and have those fuels be cheaper and have them crowd out the clean energy that I know Nvidia is also working with clean energy companies too. Can you talk about how your work with SLB fits into the sustainability goals? And it does seem to me, doesn't it kind of push against this idea that AI applied to every industry is going to make everyone more sustainable and reduce our emissions?
- [41:56]
C
Yeah, so that's a good question. And the truth is, AI really does, back to your original point, drive efficiency very easily across whatever purpose you're trying to apply it for. So if you want to be more efficient at extracting fossil fuels, it can help with that. Now, where we end up again, if the important thing is the net, then we need to look at, okay, is AI poised to accelerate fossil fuels more than it's poised to accelerate clean energy adoption? And I think the data pretty clearly demonstrates that clean energy is likely to benefit at least as much as fossil fuels. Not least because clean energy is already in, in many cases, if not most cases, the most economic and most secure form of energy that can be used. And then when you layer in things like this growth in energy demand that's being driven by AI, the companies that build out those AI data centers, by and large, are looking for every clean electron they can find their Commitments to clean energy are world leading. And so the demand that AI is creating itself, itself is very much focused on clean energy. That's what Microsoft and Google and Meta, that's the type of energy they want. And then you factor in the concepts of smart grids, VPPs, which AI can enable, and the demand flexibility of data centers themselves. That makes variable generation like solar and wind at least incrementally more valuable relative to fossil fuels. So I think it only accelerates and improves the economics of clean energy relative to fossil fuels. So I think if agreed, AI can I think help fossil fuel companies be more efficient in their operations. But I think the overall demand picture and the economics of clean energy are driving us unavoidably in that direction. And the last thing I'll say on this is AI is a fantastic complement to policy. It's not a replacement. AI is technology agnostic. It helps you be more efficient at whatever you're doing generally. But if we want to want policies that drive prioritization of clean energy and things like transmission and permitting reform and smart grids will lead us down that road naturally, then the policies we should focus on the policies that unlock that feature.
- [44:17]
B
I agree with it. The current set of companies use it. They're using a lot of Nvidia's chips. Most of Nvidia's chips and are applying AI, especially in the United States are very focused on these clean energy goals. That's not true of globally, right? I mean that's not true of China, it's not true of the Gulf states, which I think are the next buyer of some of Nvidia AS chips. Does this mean when we think about how to regulate AI, focus on keeping it at these American tech companies that have these clean energy goals?
- [44:44]
C
I'm not our political specialist, so I won't be able to comment on the geopolitics of everything. But I will mention that I think the trend towards net emissions reductions enabled by AI to me looks almost unavoidable at this point because the technology fundamentally helps us take better advantage of the resources that we have. So even if in the near term we see an increase in emissions globally due to the build out of AI, I think in the medium and long term we will end up with net reductions for all the reasons that are covered in those papers that I mentioned.
- [45:19]
B
So heatmap has been tracking what to us has been a very sudden and shocking rise of local pushback against AI data centers. And of course this has become a larger meme over the past few months as it's gotten more attention. For instance, we think about 50 AI data centers or data centers broadly were canceled last year after facing local pushback. And we think more than 50 have already been canceled this year. Are you seeing that at all at Nvidia? I mean, it doesn't look. Your quarterly results came out yesterday and they were, they absolutely blew out expectations. And so evidently it's not affecting demand yet. But do you hear it from customers? Is this affecting Nvidia's business at all? And how do you think about, about it as a risk going forward?
- [46:07]
C
So I, I'm aware of the sentiment, the paranoia around AI, mostly on a personal level because I see it on social media like other people do as well. I'm not, you know, aware of any direct impact on our sales, so I can't comment on that. But what I will say is I, I do think it's particularly tragic because this technology has the potential to be the most beneficial both for environmental goals and for social goals. So things like education and healthcare and kind of across the board social issues benefit from AI as well. And the concerns about AI, a lot of them are based on either erroneous data or old data. And I worry that some people don't fully understand the net impacts, the positive as well as the negative of AI. Plus we have the uphill battle of it's really hard if a data center is being built a few miles down the road to tie that data center, which they don't always look beautiful and things like that, to the benefits that the whole world is going to get from AI. So if, you know, obviously not promising this, but AI could unlock cancer cures or cures to other diseases and we're seeing trends in the direction of cures and treatments and drug discovery and so forth. But it's really hard for us as humans to draw a line between the infrastructure that we see down the street and especially the speculative, the moonshot benefits. But even the more fundamental ones like the benefits that in productivity that we're seeing in potential for wage growth and education and so forth, even those it's hard for us to draw the line between the infrastructure. So it's understandable. But I do think it's tragic and I think it's our responsibility in the tech industry to help people see the bigger picture and to address people's concerns head on about environment, environmental impacts and social impacts. Because the data really does demonstrate that by and large these data centers are pro sustainability. They don't have the impacts that most people are concerned about and they're manageable and most data center operators are trying to operate them in a sustainable way.
- [48:13]
B
Josh Parker, so much more to talk about, but we're going to have to leave it there. Thank you so much for joining us here on Shift Key.
- [48:18]
C
My pleasure. Thanks, Rob.
- [48:24]
A
And that will do it for us on Shift Key today. We'll be back soon with another episode. Until then, Shift Key is a production of heatmap News. Our editors are Gillian Goodman and Nico Loricella. Multimedia editing and audio engineering is by Jacob Lambert and by Nick Woodbury.
- [48:36]
B
Our music is by Adam Kramelau.
- [48:38]
A
Thanks so much for listening. See you next time.