← NVIDIA AI Podcast
Loading summary
Transcript50 lines
- [00:11]
Brad
Hello and welcome to a special GTC edition of the Nvidia AI podcast. This is the third of five episodes on the road to GTC Live in Washington D.C. bonus conversations you won't hear anywhere else. We've gathered leaders from across the energy and infrastructure sectors to discuss how they're building the backbone of the AI economy. The unseen network of data centers, power systems and partners that powers each and every breakthrough. Enjoy the conversation and when you're done, visit AI-podcast.invideo.com to browse our library of over 275 episodes of the Nvidia AI Podcast.
- [00:49]
AI isn't just transforming tech. It's becoming a new driver of the American economy. From manufacturing to medicine. It's rebuilding our industrial and scientific foundations that are driving new productivity and new jobs.
- [01:06]
Nowhere is that impact clearer than in the rise of data centers. They are the new factories of the AI age.
- [01:12]
America's AI leadership depends on more than algorithms and chips, although those are awesome. It runs on infrastructure. Data centers, power grids, supply chains are becoming the backbone of intelligence, the foundation of a national strategy for innovation. Even when we assume we need, let's say, 1000x more and there's forecasts that go up to a million times more, the build out is tremendous. And here to talk about building that foundation is Gio Albertazzi, CEO of Vertiv, Olivier Blum, CEO, Schneider Electric, Krishna Janalagogada, CTO GE Vernova and Chase Lockmiller, co founder and CEO of Pruso. All right, Gio, let's start off with you and let's pretend for a second people don't know how big of a challenge that we're looking at right now. Can you talk about the scope of the power and the cooling that we're going to need by let's say, 2030? Well.
- [02:24]
Gio Albertazzi
Think in terms of all the announcements that we have heard about investment in data center, certainly in the US but globally we have an imagination how much IT power is behind that. For every kilowatt megawatt gigawatt there is an equivalent amount of power and thermal infrastructure. And that power and thermal infrastructure needs building at scale. Now that has been traditionally a very kind of a construction, labor intensive industry. And we're all thinking about how can we change that and scale it to absolutely unprecedented industrial proportions. Because without that, and that's what I think, I believe, and I know everyone in this panel and beyond are doing, without that, we will never be able to scale AI at the speed that we have seen in the video and that you have explained to us. So I think we are at a very important inflection moment in this juncture.
- [03:33]
Brad
So Olivier Schneider has been a hallmark inside of data centers. I get a lot of data center tours and I see your logo on a lot of the boxes in there. One question I have for you is, sure, it's build more and more and more, but this is also an efficiency play. Can you talk me through the balance of efficiency and just more power?
- [03:58]
Olivier Blum
Yeah, you're absolutely right. You know what is very interesting? We are at a time where AI depends on compute, compute depend on energy. The very interesting part is energy availability and efficiency depends on AI.
- [04:09]
Brad
Okay.
- [04:09]
Olivier Blum
Because, you know, at Schneider, I think we have been an advocate of energy efficiency for many, many years. And we strongly believe that the combination of electrification, automation and digitalization will solve the energy transition in every part of the world. Now, 10 years ago, frankly speaking, it was not possible. Electrification automation were available. But the type of technology you have now through AI help you to make energy more efficient. So on our side, of course, we are excited because altogether we are providing the infrastructure to make the compute and therefore AI available. But we are even more excited to leverage AI to make our overall industry more efficient. And if you think through, we have been, all of us through the industrial and digital revolution, actually what we are doing, it was in your introduction. We are talking more and more about AI factory and we are implementing exactly the same principle. So going through the full life cycle of this AI factory. Start from the design building digital twin where you can simulate power and cooling efficiency, to the development, the construction, the maintenance, the operation. This whole life cycle is completely powered by AI at the end.
- [05:17]
Brad
Krishna, I think that Vernova's gas combustion engines are really a secret weapon in America's AI buildout. But you're sold out. And talk to us a little bit just about how you're changing your own business processes to drive more production, to get more online and how you see that playing out over the course of the next few years.
- [05:41]
Sure, Brad. So like you said, the production is, you know, the demand is massive. We are sold out through 28, maybe through 29. But we are investing heavily in capacity expansion of gas. And so we are quadrupling our capacity of, you know, number of gas turbines delivered by 2028 compared to 2020. And this is all types of power is also an answer to this. It's not just gas. So we are fortunate to have basically all kinds of power that's available to us. This is gas, this is nuclear, this is solar Wind and hydrogen. So we are looking to leverage every type of power source that's available to grow the electrical infrastructure. And power generation is a part of it. A big part of it is also we'll talk about later is the grid. Once you actually generate these electrons, how do you get them from source to use in a very complex environment with new grid resources and so on. But power generation is definitely and grid is a big bottleneck right now and we are working on easing that.
- [06:44]
Help us understand how your power generation capabilities capacity evolved from 2025 to let's call it 2029.
- [06:55]
So if you look at the demand side, for about 20 years, the total power demand in the US has been flat, no growth, 0.3% growth. But in the next 20 years it's expected to grow 50% and about a third of that is going to come from data centers. And to meet the demand, you essentially need to leverage all kinds of power that's available. Like I said, it's gas, it's nuclear. We have an SMR that's coming online in 2029 in Ontario. Our first SMR application at TVA is went in this year. So there is significant growth in nuclear that's coming up. From a gas perspective, depending on the gas turbine, sometimes heavy duty is the answer, sometimes it's your aero derivatives, sometimes it's different types of turbines. And also future proofing some of these gas turbines so that they can run on hydrogen in the future. So it's gas is a fantastic bridge right now. So electrify now, decarbonize later. So I think that that's something we can do as well. And we are also looking at wind and solar. So all of them belong in the energy equation. And I think that's what we are working on. It's what the market needs. Energy abundance is what we are chasing here. And energy abundance is really an all of the above kind of answer right now. Right.
- [08:16]
You know Chase, you guys have been at the forefront of what Jensen talks about, extreme co design. Right. Moore's Law is dead. The data center is now the unit of compute. This is about power in and tokens out. Crusoe is building some of the most capable data centers in the world in Abilene and other places in partnership with Nvidia. Talk to us a little bit about what you see happening in that world of extreme co design and why you've been so successful along with companies like Core Weave in pushing the frontier on how Nvidia brings its chips to market.
- [08:51]
Chase Lockmiller
Absolutely. So you know, CRUSOE is an AI factory business. And the thing about AI factories is this incredible composition of all of the greatest industries humanity's ever produced. From, you know, everything the gentlemen next to me are working on to, you know, from, from a cooling, a power generation, a storage perspective all the way to, you know, everything that we've done in the, in the silicon space. And you know, most importantly, our partner with Nvidia, our partnership with Nvidia. But you know, this, this process of taking electrons and turning them into tokens is just one of humanity's greatest opportunities and challenges over the next decade. And we, we, we see it as, you know, just a very complex challenge in terms of just the scope and magnitude of these investments and it's requiring just an incredible amount of resources across the entire economy.
- [09:45]
Brad
So tell me a little bit, I mean again, I'm out of my league, out of my depth a little bit here, but the 800 volt data center kind of architecture that I know Nvidia has been pushing as just one example of this extreme co design. What are the things that you see happening and coming down the pipeline that's going to drive that equation in terms of power in and tokens out?
- [10:09]
Chase Lockmiller
Yeah. So you know, I think the 800 volt rack design is like a great example of us, you know, thinking through from a first principle basis like okay, data centers are sort of built in this way and that's how we built them 25 years ago. Is that really how we should be doing it today? At the scale of gigawatts? Maybe not. You know, when we think about this data center scale computer where it's not just a single server, it's not just a single rack, it's really the entire system working together as one cohesive unit that's thinking as one giant brain. And how do you think about that from a networking perspective? How do you think about that from a cooling perspective? How do you think about that from a, you know, from a like entire system perspective? Yeah, and you know, the power aspect of that, you get massive efficiencies from doing things like you know, changing, you know, changing from 40 volt to 800 volt, like that's like a huge efficiency gain that like all of us would benefit from. And so, you know, it's just one example of like this cohesive collective, you know, industry wide partnership that's required to bring this infrastructure to life.
- [11:13]
Gio Albertazzi
And if I, if I can kind of reinforce 100%, you know, this 800 volt is an example of how you can make the whole system not only just more energy efficiency but efficient but really scoop up kind of a power that is available because the system can be redesign from scratch. Again you were talking about the system think system, not just components. We've come from decades of data center will being built a component at a time. I was talking about an inflection point in the way we think about data centers. This is a fantastic example. There are many examples. Again we have to think that powertrain, the thermal chain, the whole prefabrication as ways to industrialize the way we build data centers like we've never done before. So believe the next 10 years, the next five years, three years will be very different in that respect than I have been so far. We've seen some picture of what we call vertivone core that is exactly that type of modularization system level. All parts integrally designed to work together and that is perfectly applicable to the 800 volt powertrain. So things are changing fast.
- [12:40]
Brad
So China is building faster, they're building bigger. You had mentioned the 10x on nukes to R0 or 1 if you count Georgia. And we're behind. I mean I started, I did China, I worked with China in 1995 and on buying sheet metal and plastic injection molding. And they sent me, I said hey, send me a picture of the site. And the picture had no roads, had no electricity. And I said well, how are we going to be up and running in 60 days? They said watch. And it happened. And China was willing to do whatever it took to build infrastructure. And here we are, it's 30 years later and it has continued. What is it that we could do to go faster? I was at the, I think we were both at the winning of the AI race here in dc. Chase, you were here as well on stage. What can Washington do to get this moving faster? Start with you.
- [13:45]
Olivier Blum
Well, look, when you see what happened in the past months had been very clear for the local government that has been big on the agenda and energy has been big on the agenda and the the two are really interconnected. So I think what they are doing right now to make sure, and it's a bit what my colleagues have said, it will take an ecosystem at the end of the day and you might not be aware, but whether it's Nvidia us, the four of us, you have more and more an ecosystem of people who are working together to support the local government to make it happen. We are working very closely also with all the hyperscaler and what is very, very important at the end of the day, and Joe talked about it, we are living a huge revolution in our sector. You know, I've been 32 years at Schneider Electric. I think more has happened in the past two years than in the past 32. So the fact that you have this ecosystem which is working together, working closely with the government, I think we have the good discussion, the real discussion, on what does it take to make it happen. And we are moving very fast. Us, by the way, when it comes to AI and data center is still by far number one in the world today. Yeah.
- [14:47]
Brad
President Trump signed an executive order which basically said removing some of the barriers to that. Are you currently experiencing that?
- [14:55]
Olivier Blum
Yep, we're working on all of that, for sure.
- [14:58]
Brad
Good. I mean, that's, you know, you nailed it. I mean, Secretary Wright and Secretary Bergam, every time I've seen them, heard them when we're in dc, they're actually asking us, what is your recommendation? How do we go faster? What do we need to do? I think it's a complete change in kind of the mindset because now it's been elevated to a national security issue.
- [15:23]
That's right.
- [15:24]
Right. This is no longer just about economic security, it's about national security and consequently moving faster than ever. Which brings me to this question of where can startups help? Right. When you think about, we're sitting here with some of the largest industrial companies, companies in the ecosystem, what, if anything, do you see happening out there in the startup ecosystem in Silicon Valley? And power generation, certainly, obviously Crusoe and the work of Corweave and others, neus in terms of building out these new data centers has been critical. But anything else that you see is interesting that people in the crowd might want to pay attention to.
- [16:06]
Absolutely. So if you look at, I think startups are the lifeblood of our innovation ecosystem, you're here. And when you look at what we're doing, I mean, Ge Vernova is 18 months old, spun off as an independent company, and our CEO intentionally headquartered it in Cambridge, Massachusetts, because of the innovation ecosystem with MIT and the universities and everything surrounding there. And we are investors in a lot of these startups, but startups are bringing.
- [16:33]
Olivier Blum
So.
- [16:33]
Brad
So when I look, let's go to the grid, for example. Our grid was designed for, you know, 50 years ago, relatively stable spinning load, spinning sources of power, relatively stable loads. It's a completely different world right now with the AI workloads, hundreds of megawatts in milliseconds. And then on the source side, you have wind and solar, where these Resources, you know, they're changing constantly, they're fluctuating. So having a grid that can actually deal with this is really, really important. So we are working it, all our competitors are working it. But the startups have a really, really important role to play there. There's the hardware aspect of it and also the software aspect of it. When it comes to hardware, we have power electronics, innovative solutions. I've seen companies that are cooling the transmission cables. I mean just fascinating stuff, you know, so that you can increase the capacity to transmit power. But how do we leverage power electronics, statcoms, fax solutions to make sure we can leverage the infrastructure? More startups are playing a huge role out there and they're also playing a very massive role in the software ecosystem. How do we predict what's going to happen? How do we make sure the right resources are there at the right time, at the right place? And how do we manage in real time from millisecond scale to minutes and hours scale, how do we dynamically manage the grid across, across the country? These are all big, challenging problems. We are working it, but a lot of the innovation in that space is coming from startups. So we are seeing, again, I don't want to name a specific startup, but we are seeing lots of innovation coming from there. So I'm really excited to work with the startups and see how we can kind of accelerate this transition.
- [18:23]
Maybe the same question for the two of you.
- [18:24]
Olivier Blum
Yeah, maybe. If I can. One point, if I come back to what I said before. You know, AI depend on compute, compute depend on energy and actually energy depend on energy intelligence. What I mean by that, at the end of the day it's about data. You know, if we are, even if we are Schneider, I think historically more hardware company, actually today we are both an hardware and in the digital company. And back to your question, what we love to do with startup. The more you have startups which are coming on this data intelligence and looking at the different assets you can have in the AI factory because it's a very complex one. You will have switch gear, you will have racks. We are doing a lot of R and D, but the more they are taking those user case on how you can make it more efficient, how you can capture the data, we are helping to structure the data, we are helping to create an ecosystem, but our customers are asking for even more. So the more you have startup coming in our own ecosystem, which is completely open by the way, not protected, the more we deliver solution and the more we can accelerate energy intelligence in our industry. So we love having those startups taking a strong interest. We need more and more smart people and capital coming from all of the startups.
- [19:27]
Gio Albertazzi
Absolutely, 100%. And if I look at it from the infrastructure technology, the industry infrastructure, data center. Infrastructure technology has been static for many years and then it started to accelerate. Crazy. In the last three, five years. And it is going to accelerate more when you have this kind of acceleration. People like us, that scale, I mean, scale is absolutely essential. But you cannot just scale in this market. You have to be an innovator and scale. Innovation happens organically, innovation happens inorganically. There needs to be a base of startups, inventors, creative minds, engineers out there that define what the new technology will be. Because the new technology cannot be just nurtured within the Jeevanova, Schneider or Vertiv. It needs to come also from spring, from the, let's say, entrepreneurial fabric of a country. And that the two things combined and scale. And that's what we see very auspicious times for right now.
- [20:45]
Brad
Chase, where are the most exciting areas you see innovation? When you look ahead two, three years, you say, I can't wait to bring this into the AI factory.
- [20:56]
Chase Lockmiller
Yeah. Just adding to this sense of urgency. And I think this huge opportunity for startups is just this sense of startups have the benefit of being able to move very, very quickly. And we're at a moment of just incredible urgency across the world in terms of bringing this infrastructure to life. I think to your question around things that we're really, really excited about, what's changing in the data center is. It's unbelievable how much change is happening over such a short period of time, from the cooling architectures to the power densities of the systems. I think when, if you look 20 years ago, a single rack in a data center might have been 2 to 4 kilowatts. Today, with the GB2 hundreds, you're looking at 130ish, 140 kilowatts, depending on peak demand. And then you play it forward and you look at Vera Rubin, you look at Feynman. These are 1 megawatt racks. This is a. A thousand homes worth of power in a single rack in a data center. That requires tremendous innovation across the cooling systems, the network systems, in terms of cabling these things together, which we're seeing a lot of tremendous innovation. And I think there's a lot to be said about just the overall memory optimization in the systems. One thing Crusoe is very focused on is not just the hardware aspect of bringing these AI Factories to life. But actually the software aspects of actually how do you operate the AI factory to produce intelligence very, very efficiently. That requires getting data and moving data from, you know, things like, you know, your object storage, you know, you know, bigger storage platform to into, directly into hbm. And how do you do that, you know, ultra efficiently. And we're really excited about some of the innovations taking place across the networking stack and the overall software innovations that are, you know, bringing data to compute way more efficiently so we can actually utilize the GPUs and get the most intelligence per unit of investment in compute.
- [23:02]
Brad
You know, maybe appropriate because we're here at gtc. Nvidia has been an active investor across the space. Their forward engineering, working with partners like you guys to help drive this, you know, this extreme co design in the data center to get us the benefits of Moore's Law. Now that Moore's law is dead or even more, can each of you just give us an example of, you know, some of the things you're doing with Nvidia in particular that are helping to drive this flywheel of innovation and how this unique partnership might be accelerating the path that we're on.
- [23:39]
Olivier Blum
No, no, please, please go too.
- [23:40]
Gio Albertazzi
We can start. Exactly. We were Talking about the 800 volt DC as an example and that's an area where we are actively, actively working together. So how will we make that technology available? And what we have to do as infrastructure provider, we have to make that infrastructure ready ahead of the GPUs, the silicon that will land because that infrastructure needs to be in place. So that's an area. Another area of great importance is still going to continue to evolve is everything around, around liquid cooling. Let's not forget the transition that has occurred and is still occurring in the industry from air cooled to a, to a liquid cool. And that's, it's a colossal change if you will. There's still air. It's a mix, it's a, it's a hybrid. But more and more loads are liquid cooled. So all these things require technology partners that are ahead of the pack and then the industry can follow. But we have to pave those ways together.
- [24:52]
Chase Lockmiller
Yeah, we have a ton of deep partnerships with Nvidia across the stack. And it's so amazing that this company just spans so many aspects of the economy from power generation to you know, the most cutting edge artificial intelligence development. But you know, we're working with Nvidia very tightly on like AI factory reference design. So how do you actually build these gigawatt scale data centers? And do it efficiently from a, from a power generation, from a, from a cooling, from a, you know, power perspective. And so that's one big aspect that we're investing a lot of time with Nvidia. Another is on the software layer. Right. So, you know, working with them on, you know, what the, you know, things like Lepton, which is basically a distribution of like getting compute in people's hands so that they can run their workloads very efficiently across, you know, various NEO cloud partners like ourselves, as well as things like Dynamo, which is basically accelerating inference and creating. How do you actually get more utilization and a lot of these memory optimizations that Crusoe is focused on pushing and innovating on so that people actually get more tokens per, per unit of compute from their system.
- [26:00]
Brad
Let me, let me just interrupt by asking, do you see any other chip company out innovating?
- [26:05]
Right.
- [26:05]
There are a lot of other chip companies. You guys work with other ones. But Nvidia seems to be playing a really unique role in pushing the ecosystem forward. Maybe, maybe a little commentary on how it's similar or different than others.
- [26:19]
Chase Lockmiller
Yeah. The thing I would say about Nvidia that they've done so incredibly well and kudos to Jensen is just the ecosystem itself, right? Nvidia. I forget the exact headcount. It's 35,000 people, something like that. The amount of leverage that they're able to get from those 35,000 people and then the ecosystem of builders is absolutely incredible. And that's kudos to Jensen in terms of building out all these different work streams and workflows across energy data center design, all these different SDKs that they've released that have just helped unlock this collective intelligence of society and the startup ecosystem to big enterprises.
- [27:02]
Brad
Just to kind of build on that a little bit, by the time you're done with all four of us, maybe you'll get a view of the scope and breadth of the ecosystem that Nvidia is playing. We talked about software optimization, AI, we talked about cooling, we talked about 800 world rack. We are more on the power generation and grid side. So we just released a white paper with Nvidia. We're working with Nvidia on reference architectures for power gen inside the. And how do you manage it? So we just talked about the grid a few minutes ago, how complex it is, different source of power, load balancing and so on. Now the power is not available today at the grid. Sometimes when folks like you are building the data centers so there's behind the meter power, there's bridging power. But when you do that, you're dealing with all the complexities of the grid inside the data center. So how do you design and architect it so that you can actually manage the power flow with battery energy storage, with power electronics, fax, all kinds of different systems, working with people who were in the medium voltage and low voltage space that my colleagues here are doing. So it's an ecosystem where all the way from the power gen to memory to algorithms, cooling and the entire thing, we're all working with Nvidia. So to me it's amazing to see how they're curating the entire ecosystem and moving us all forward, helping us move forward.
- [28:22]
And maybe we'll end with Schneider's commentary on this. I don't see anything else out there that has this level of systems thinking. It's extraordinary the breadth. It gives me a lot of encouragement about the gains yet to become or yet to come in terms of America's re industrialization around around power and AI generation. But how is Schneider working with Nvidia to drive this forward?
- [28:51]
Olivier Blum
Well, look, a lot has been said, but for me, what is very unique in the vision of Jensen is first of all to work with partners. What we love with Nvidia team, they share a lot with us. They speak about the omniverse, they speak about the next generation of chips, the next generation of gpu and when you can work as a technology partner upfront and in a very open manner, that changed the game. What I would say also, you know, Jensen has this famous sentence about the speed of light. You know, he's forcing the entire industry, but in a very positive way, to work at a very different speed. And for us, for Shannon, I think I can tell you it has changed a lot of suffering the way we are managing the company. Joe mentioned the shift from air cooling to liquid cooling. We discover that by working, we knew it would come. The question is, was when we discovered two to three years ago that it could come much faster than expected. But if you ask me today, what does it change? We have to completely rethink the way we design those data center. And I like what Chase mentioned in his sentence. He said design, he talked about maintenance operation. So you have to completely rethink the way you build those AI factories from the design stage, you know, to the build stage, to the operate to the maintenance. So it's about building a digital twin. And what Jensen and the Nvidia team are forcing us to do is to think about those next generation. And guess what software is playing A big role. And software is powered by AI. So again it's a loop on which we are working all together.
- [30:19]
Chase Lockmiller
One thing I want to touch on, follow up there. Just in terms of the system wide innovation needed to invent solutions to support this infrastructure at scale. One of the conversations I was having backstage with Olivier is just around this notion of how the utilization of the data center scale computer impacts the power swings. Because you have this massive large variance in terms of like when you have a gigawatt scale data center that's operating as a single brain, you have this problem where it's pushing data to the GPUs. All of the GPUs are running their back propagation or their computer know their matrix multiplications and then they're publishing the results to every other GPU on the cluster. And when you have that, you actually have these massive power swings that, you know, it's like whatever 6 hertz somewhere in that range where it's, you have these massive load oscillations. And that creates a lot of problems for utilities and on site power generation. Like, you know, GE Vernova doesn't like it when we have these massive power swings in the actual power utilization of the turbines. So it's actually created this, you know, system wide partnership where we need to work with, you know, our on site generation, we need to work with the utilities and most importantly we need to work with, you know, folks like Schneider to innovate these solutions where we can actually create a buffer through battery systems that can absorb a lot of this load oscillation that we're seeing from these large scale training workloads and these system wide workloads that are running on these massive scale computers.
- [31:49]
Brad
I mean one of the things that blows me away if we think about the Manhattan Project that cost about $4 billion over four years, right? Inflation adjusts that, that's about 40 billion as a percentage of GDP is 1% of GDP. So let's call it 400 billion. Jensen has said we're going to spend 4 trillion over the next five years. So private industry is going to spend 10x what we spent on the Manhattan Project. And just listening to the decentralized coordination really where Nvidia plays this role in pulling everybody together and pushing everybody forward. It reminds me like the speed of that is very commensurate with what we think about the speed of, you know, the Manhattan Project. I don't see it happening anywhere else in the world. I think that all of your companies are critical to the success of, you know, of the American AI race, and it's great to have Nvidia helping push it all forward. I know we got to jump, so I'm going to throw it to you.
- [32:51]
Chase Lockmiller
No.
- [32:51]
Brad
That was a great ending. Gentlemen, I just want to thank you so much for coming up here, and I hope everybody out there listening has a better understanding of the challenge that is ahead of us. And also, Washington can hopefully clear as many roadblocks as possible to let all of you cook. Thank you very much.
- [33:10]
Chase Lockmiller
Thank you.
- [33:40]
Olivier Blum
Sam.