Loading summary
Transcript150 lines
- [00:03]
Narrator
Latitude Media, covering the new frontiers of the energy transition.
- [00:08]
Stephen Lacy
So the people want to know. Caroline, did the robots plan your trip to Scotland?
- [00:13]
Caroline Golan
Well, you know what, it did a pretty good job. But like with any trip, you just got to find some good locals and they will tell you what is the heart and soul of the land that they occupy. Thank you for letting me have the break.
- [00:28]
Stephen Lacy
You've been swearing a lot this morning. Did you pick that up in Scotland?
- [00:31]
Caroline Golan
Hey, you're not supposed to say that on air. Yes, that, and I'm just very jealous.
- [00:41]
Stephen Lacy
From Latitude Media. This is open circuit. First it was a compute bottleneck, then it was a power bottleneck. Now, as AI agents start burning through tokens, we're back in a compute shortage. And it's getting harder than ever, ever to build the data centers to alleviate it. As both constraints get worse, we're seeing a shakeup in who builds the infrastructure to serve AI and possibly how that infrastructure gets built. First up this week, Nextera wants to buy Dominion. It would be the largest utility merger in American history. What does it mean if one of the top clean energy builders takes over the utility serving the world's biggest data center market? Then, with compute rationing hitting AI companies, a new debate is emerging over the next wave of buildout. Will a distributed inference model start to emerge? And we'll close with some new ideas on how to solve the compute crunch. Home inference hubs, wave power data centers, even water heaters that serve AI. That's all coming right up.
- [01:43]
Narrator
OpenCircuit is brought to you by FlexGen. Batteries are becoming an essential part of the data center energy stack. And FlexGen is helping operators use them to solve some of their toughest power challenges.
- [01:53]
Ken Ron
So at FlexGen, we like to say that the grid doesn't have a 24 hour problem. It's a four hour problem. It's really that choke point that needs to be solved for these data centers. And batteries are solving it.
- [02:01]
Narrator
FlexGen VP Ken Ron spends his days explaining batteries to a wide range of customers, including data center operators. And lately, he says the conversation has changed dramatically.
- [02:11]
Ken Ron
We certainly have seen more of those light bulb moments. Everyone's saying, I need a battery, so please come explain why I need a battery. So the conversation is shifting in the right direction.
- [02:19]
Narrator
Later in the episode, Ken breaks down how batteries can smooth volatile AI loads, speed up access to power, and help data centers become better grid. Participants. Learn how FlexGen is powering more strategic energy systems at flexgen.com.
- [02:38]
Stephen Lacy
Welcome to the show, everybody. I'm Stephen Lacy, the executive Editor of Latitude Media. The three legs of the stool are back. Caroline Golan is the Chief Growth and Policy Officer at nrg. She is the leg that keeps us both upright. Welcome back. How are you?
- [02:55]
Caroline Golan
I'm great. I am, I am tired. So I, I'm hoping that you two will carry or have the longer legs
- [03:02]
Narrator
I guess of the stool today.
- [03:06]
Caroline Golan
That, that, that analogy doesn't really work, does it? We still topple, I guess.
- [03:10]
Stephen Lacy
Yeah. What does that make you, Jigger? Are you the wobbly leg that you need to fold a piece of paper?
- [03:14]
Jigger
I'm, I'm the one with, with the adjustable like thing in the bottom.
- [03:20]
Stephen Lacy
Once you get it right, it's stronger than ever though.
- [03:23]
Caroline Golan
That's true.
- [03:24]
Stephen Lacy
How's it going, Jigger?
- [03:25]
Jigger
It's going great. It's going great. I have like, it is just such a busy week, which is interesting.
- [03:33]
Stephen Lacy
What's busy about it?
- [03:34]
Jigger
I mean we had our good friends at Voltus like finally announced their big deal with Google, which is great. Right on demand flexibility. I think that the PJM has decided that they have no choice but to just do all flexibility all day. Right. You've got huge launches of product around battery storage and other things. And so like just keeping up is a full time job these days for sure.
- [04:02]
Stephen Lacy
Absolutely. And so let's turn now to some big utility news that unfolded when we were apart. Nextera wants to buy Dominion. This is the biggest clean energy developer in the country with an enormous pipeline of renewables and storage. Wants to take over the utility serving the world's largest concentration of data centers. It's an all stock deal that would value Virginia based Dominion at $120 billion. The combined entity would serve 10 million customers. And executives said that they had a combined pipeline of 130 gigawatts of projects, much of it driven by data center growth. So it raises a bunch of questions. What does it mean for the resource mix? What does it mean for electricity rates? What does it tell us about how AI is shifting the corporate landscape in the power sector? What would this combined entity look like? Let's start with the basics, Caroline. This merger would not only create the biggest power company in the US but also America's third largest energy company behind Exxon and Chevron. What would this entity be able to do that like Nextera and Dominion couldn't do on their own?
- [05:13]
Caroline Golan
I'm not even sure that Nextera knows the answer to that. I think Nextera has been in the hunt to expand its IOU portfolio for quite some time and it hasn't Been successful. It's been tried to buy what is now Dominion, South Carolina several years ago, Santee Cooper and, you know, tried to buy Hawaii and, you know, is now sort of looking for its next step. I think it knows the Southeast. It likes the Southeast. I think it also knows PJM in the way that it's developed in its renewables arm. And so it's a little bit of the devil it knows. I will also say that in terms of, you know, the IPP landscape, NextEra has been very aggressive, and rightfully so, in developing more of a platform business for its data center pipeline. And so I think that putting two and two together at a very basic level, it's looking to expand its regulated book, it's looking to capitalize on the investments it's made in terms of its data center pipeline. And Dominion operates in a hybrid market. But the truth is Dominion operates much more like a traditional IOU than most entities in pjm. And so it makes sense from sort of checking those boxes. I'm not sure, and I'm not privy to it, but I'm not sure that there is a great big master plan. I think it's been batting at trying to expand its IOU book. It sort of just moved up the East Coast a little bit. You know, it started with the.
- [07:09]
Jigger
Didn't they try to buy Duke at one point?
- [07:12]
Caroline Golan
They did, yeah, they did, yeah. They tried to buy Duke, South Carolina, Evergy maybe. Yeah, and they tried to. And they tried to buy Santee Cooper. And there was really, you know, the pushback in South Carolina was that they didn't want more big corporate. You know, South Carolina is a very interesting state. And, you know, it's one of those states like Ohio, like California, like Arizona, which is just sort of unique to its own energy ecosystem, culture. And so they really. They push Nextera out, I think, despite some very aggressive, persistent bids. And we'll see. I mean, I give it more than a 50% chance that it goes through, but I wouldn't give it. I wouldn't say it's locked down. There's a lot of pushback and there's a lot of populist pushback across the board on all energy issues. And when you are mixing energy with data centers, I think you could stir up a lot of local fervor that maybe five years ago, six years ago, wouldn't have. Wouldn't have been there. But, you know, I think that, I think it's really more just about. They've been trying for a decade and this was the next one up the east coast. That made sense.
- [08:33]
Stephen Lacy
So they landed.
- [08:34]
Caroline Golan
And Dominion has a great portfolio for them for what they know in terms of what they know how to operate and what they know how to deploy.
- [08:40]
Stephen Lacy
Yeah, jigger, you're not one to hold punches. So you called Dominion the worst run utility in America. You've also said that this isn't really an AI story, it's a competence.
- [08:49]
Caroline Golan
They're not the worst run utility in America.
- [08:52]
Jigger
I make the case versus if you just look at their stock price since 2019, outside of PG&E and Hawaiian Electric that went bankrupt, Dominion was the worst performing stock of any single utility company in the country. That's all I said. And it's true. Like, I mean, when you think about the fact that Bob Blue likes, you know, like, bless his heart, I mean the guy like was brought in to turn around Dominion, right, in 2020. And so he has had nothing but heartache since he's been CEO. He had to sell his natural gas assets for a fire sale to Berkshire Hathaway. The amount of money he made there didn't even pay off the debt that they had taken on there. And so like, it's been a tough slog for him, right, to figure out what to do with Dominion. And now they've got, you know, like the offshore wind farm that's still not completed and probably will be completed in 27. So you've got that overhang, right, because that's in construction until it becomes operational. You don't get revenues from it. They've got a ship that they're building down in Louisiana for like, you know, like Jones act compliance. And so it's a lot. But then I also gave it to Nextera too. Like Nextera, when you think about it, they were already more valuable than ExxonMobil in 2020 and 2021 during the COVID pandemic. Right? Then they basically were upside down in their yield co. Right, because interest rates went up and they didn't, they were long debt. So then, you know, they ended up in a situation where their stock price went down by like 35 or 40% a few years ago. Now they're like climbing back out. They separately had to fire their CEO of Florida Power and Light two years ago for rampant basically corruption in Tallahassee. And so like, you know, like, I mean, you know, like people deserve each other.
- [10:39]
Caroline Golan
So listen, first of all, Dominion is not the worst run utility in the country. But what happened that forced Dominion to go from what was, you know, a pretty bankable and profitable utility to One that was struggling were a couple of things. First, the bcea, which everyone rallied as like the biggest climate coup for clean energy at the state level, which was forcing Virginia to go 100% clean energy. But it was doing so in a way that really required a significant amount of confidence and Dominion to build out offshore wind and to build out a lot of resources that frankly probably was not the best path for that utility. Right. And we could probably spend a whole show on that. And the other big thing that happened was that the commission started to pay a bit more attention to Dominion's returns and tighten the belt. And so when those two things happened in concurrence, you don't have quite the cash flow that the company had before. And I think Bob, for what it's worth, I have a lot of respect for Bob and I think he's done the best he could with a utility that was probably inevitably going to be carved up given the data center onslaught, the PJM changes, the clean energy requirements and the historical return schedule that they had been used to that all of a sudden they hadn't got anymore. So I'm not surprised that Bob took Dominion down this path. I do wonder what it means in South Carolina more than actually in Virginia, because if you look at the portfolio, I think that the Dominion South Carolina folks have always wanted to be their own utility. They've never particularly been interested in responding to Richmond. And so I think that's an easier sell than necessarily the whole portfolio. Even if everything's up on the board. I wouldn't, I wouldn't be shocked if at the end of the day Dominion South Carolina goes to Nextera and Virginia stays. Virginia, to be honest. But you know, we'll see. We can make a bet.
- [13:09]
Stephen Lacy
Yeah. So Jigger, are there any advantages that, that this combined entity, I mean, to
- [13:16]
Jigger
be crystal clear, I don't know that I care. Like, I mean, in general, like, I
- [13:20]
Caroline Golan
just, I mean, it's a good answer, right?
- [13:22]
Jigger
I mean, it's just, I mean it was so weird. Cause I just posted something on LinkedIn about like, you know, the fact that Dominion had challenges. And I think Bob Blue's great. So like, I'm not badmouthing him. I think he was brought in at a particularly difficult time and then they had to write off like their natural gas pipeline investments in like, was it Mountain State or whatever that didn't get completed. And so like there was just a lot of stuff going on. Right. And then like, you know, Nextera had a lot of stuff going on, so that's all I. That's all I posted. It's not like I don't have this like desire for all of the utility companies in the country to get aggregated up into one. And if I did have that desire, it wouldn't be Nextera that would do the work. It would be like BlackRock or JP Morgan's infrastructure fund that would like buy up everybody and turn us into a copper plate. Right. I mean, I think the bigger challenge I see right now is that like when you think about how many laws have been passed at the state level in Virginia, right? I mean, and I was on the other side with DOE with you know, the grid deployment office and loan programs office, et cetera. I mean, Dominion is not known for its extraordinary ability to, you know, free up capacity using modern technology for the data center. Alley folks, like, they still haven't deployed grid enhancing technologies fully. They're still piloting line vision even though they love them. Right. They're still not building out advanced. When you look at what Gil Quinones is doing in ComEd's territory, he is deploying all of that stuff at scale super fast. And you're like, why is Dominion not doing that when they have an eight year backlog of data centers? You're like, this is literally the easiest goddamn thing to do. And Dominion's like, ugh, this is so weird for us.
- [15:09]
Caroline Golan
Well, I mean it does beg the question, like FP and L is not exactly the hotbed for data center development. It's a different mountain to climb and it creates a different question of growth and management. I think Nextera made a ton of money on renewables because it had the capital to meet the economies of scale needed to do one of two things come in under avoided costs for IOU programs. Right? Which it did throughout the Southeast very successfully. And to meet sort of the threshold for an energy plus REC deal for corporate buyers across the U.S. that's a very simple business. It was a pretty simple business model and they had the cash and jigger, you probably know more about this than I do. They had the cash to be able to treat it more like a portfolio of investments which most developers didn't have that just deal by deal trying to make ends meet when renewables were taking off. That's a different business model than being able to take full deliverability risk, being able to take on capacity and power supply in a way that from Nextera's renewables arm. I'm not saying they can't. It just hasn't been the business model and it hasn't been the way they've made returns. I think they're moving into that. So in an FP L they don't operate, it does not operate in a wholesale market. PJM is what it is and will continue to be the head scratcher of the Mid Atlantic, but it is going to also continue to be very demonstrative in the way things get built out. So the reason why I think we should care is the question of fit. Like Nextera has the cash, clearly they have the ability to develop renewables and to sell renewables under a certain business model. But whether or not they have the institutional knowledge and capability to turn around Dominion, I don't know. I wouldn't say that. Like what they've done historically says, oh,
- [17:36]
Jigger
but that's not what this is based on, Caroline. Given that there's like.
- [17:40]
Caroline Golan
I know it's not what it's based on.
- [17:41]
Jigger
No, but I'm just saying, given that there's four previous failures by Nextera. Right. This is all about their bedside manner, right? Their bedside manner was terrible territory. Like, can they sweet talk people in Richmond? I don't know, but they better start learning how to sweet talk people in Richmond. This is not about like your capabilities and all this other stuff. It's about like, does the governor of Virginia actually think this is good for the citizens of Virginia?
- [18:12]
Stephen Lacy
Yeah. So is there a difficult regulatory pathway here then?
- [18:15]
Caroline Golan
I think so. I mean, I think so. I, like, I, I, like I said, I don't think this is a slam dunk. But if it does happen, I mean I go back to the same one, which is like, will they turn Dominion around? Will they, will they do the things that Jigger is talking about?
- [18:31]
Jigger
The governor of Virginia is in her first year, right. So she will say like, my sense is she's probably going to be against it. Right? But like let's say she was like, I could be convinced to be for it. She will. You have to finish the offshore wind farm. You have to like fully built out grid enhancing technologies. You have to agree to the grid utilization law that we passed. You've got to fully build out the 300 megawatt VPP that like is in law. You have to do all these things and the next era will say anything you want if we get a permission to.
- [19:04]
Caroline Golan
And then she won't be governor in a year and the legislature will flip and they'll pass a different law. I mean that's the way for sure
- [19:11]
Jigger
that this stuff can all get done in 12 months.
- [19:13]
Caroline Golan
Well, maybe. But what I'm saying is that Virginia is notorious for the flippy floppy.
- [19:19]
Stephen Lacy
Well, what about rates? I mean this is an area of contention. Nextera is offering $2.25 billion in bill credits, about $550 per customer as a sweetener. Consumer advocates are skeptical. They see this as a temporary measure. What are the possible scenarios for rates?
- [19:40]
Jigger
I think that what's really important in this moment is just how smart the staff is around the governor. I think, you know, like, I mean Abigail Spamberger and Mikey Sherrill for that matter in New Jersey had to run on rates in their governor election. So when you think about all of the people that joined her transition team and then also now she's elevated the position of energy into the governor's office, right. Which Asifas who's joined in, Right. And so like I think that, I think they've got a team that can negotiate a much better deal than for instance, like Mayor Bowser did for the Pepco merger with Exelon where she just gave everybody a one time $50 credit and just got rolled over by Exelon. But like when you think about, you know, like just how smart the team is around the governor, like I think they're going to ask for a lot like, and Nextera may balk and say, well, we're not going to give you all that and therefore we're not going to do the deal. But ultimately I think the governor has a plan. She passed it into law within 90 days of coming into office. And so she is going to demand that they implement that law. That's what she's going to say. She's going to be like, Great, take your $2 billion and implement the damn law. Get utilization rates up, do grid enhancing technologies, deploy advanced conductors, do all the stuff that we put into the law. And if they say yes and they start doing it within 24 months, great. If they say no, then she'll tell them to pack.
- [21:08]
Stephen Lacy
So one last point here. What does this tell us about consolidation in the power sector around the AI load growth thesis? We saw BlackRock's global infrastructure partners buy Minnesota Power. A BlackRock consortium is buying AES. Now this, what does this mean for sort of the corporate landscape and power generation?
- [21:30]
Jigger
I think it's a lot different when a financial player is buying a company like Elite, for instance. Bethany is amazing. And I think they've already invested $500 million into her utility, which is like almost more money than Elite has ever raised as a publicly traded company in its history. And so like when you think about Just like, you know, how important it was for her to switch out her public shareholders to a more, you know, deep, you know, pocketed infrastructure owner. Right. That was really important for the people who live in Elite's territory. Right. And so I think that similar vein is going through, you know, Public Service of New Mexico. There's other places as well where these utilities just don't have the ability to raise the amount of cash necessary to do the things that the governors want them to do on data centers or anything else. Right. And so having a financial owner who has deep pockets is going to be critical. This is a little bit different. Like Nextera does have more money than Dominion does, so that's obvious. But I don't think they're viewed as a financial player in the same way that like GIP or you know, JP Morgan infrastructure or some of the others would be viewed.
- [22:47]
Caroline Golan
So a couple points here. One, I completely agree with Jigger. It's a very, this is very different. It's, it's apples and oranges in this case. Nextera is an outlier and they've been an outlier for, for a very long time. And so I wouldn't say because Nextera can do it, it can be replicated. It's, it, it's, they have operated differently. To be fair, the, you know, the firewall between their regulated and deregulated arm has been permeable. And so I think they've been able to grow at a level that others have not been able to grow and take that for what it is. The other thing here is that what I'm interested in, and I don't have any answers to, and maybe one of our listeners will have an answer to, is that the vast majority, and actually Jigger and I talked about this in London a little bit, was that the vast majority of investors, traditional IPP investors, still want to see fairly traditional deals. They want to see big centralized power plants built. They want to see long term offtake. They want IOUs to be building what they know how to build and they want to see load growth with a lot of political and regulatory capture. And we have to just be honest about that. The investment community does not want to evolve, does not want to see a ton of new capital stack needed to launch innovative or what we would say non traditional returns right in this space. So Nextera has gone, has been a bit of an outlier in the fact that they've proven some of that naysayer wrong. Like to be honest, when Nextera launched a lot of its renewables, I'M not sure anyone expected it to be as successful as it was. They were sharks in the field to their credit, and they were able to grow on a business model that, you know, I'm not positive all the ipps in the country could have done Right. So does investment change because Nextera does this in as much as Nextera proves out new return models? Yes. So if Nextera goes and says we're going to invest in distributed, we're going to invest in grid enhancing technologies, we're going to invest in utilization and figure out how to commoditize that and return it to shareholders, then the investment circle may change and then you may see this stir something if they don't. I think investment stays the way it is and the margins on conversion for cash flow is going to keep everything pretty clunky in the way that it's been. And so I hate to say the same thing that Jigger says. I don't know how much this matters in the big scheme of things. I think it could matter. If Naxtera takes a very different approach to building power for Dominion, it could, it could be a game changer because everyone will be watching it. I don't know that they will though.
- [25:58]
Jigger
You know, I was surprised that the Sun Sentinel came out against the merger. Nextera's like hometown came out against everything.
- [26:06]
Caroline Golan
I mean, they come out against Groundhog Day. I think they don't like anything. I mean, but you're gonna see, you're gonna see that. And the truth is like if you see two or three more botched big data center investments, like, I don't know, I mean, I don't have a crystal ball on the populace around data center pushback, but it's bleeding to big corporate. It's bleeding across the board. And so again, I don't think it's a slam dunk.
- [26:41]
Narrator
AI data centers are exposing every constraint on the power system, from grid congestion to power quality to slow interconnections. And batteries sit at the center of nearly every solution.
- [26:52]
Ken Ron
Every corner you turn over or every rock you turn over. Batteries can solve that problem. And batteries with sophisticated software specifically solve that problem.
- [27:00]
Narrator
Ken Ron is the VP of Marketing and Customer Solutions at Flexgen. Every data center developer and operator he talks with faces the same challenges. The first challenge is power quality. AI training workloads can behave like an industrial facility that's constantly switching on and off, creating rapid spikes and dips that neither servers nor grid equipment are designed to handle.
- [27:21]
Ken Ron
And if you put a battery there, it can sit right in the middle and basically absorb power, push power, absorb power, push power. So the electricity line that's coming out of the compute looks crazy, but the grid around it, the equipment around it, the gas turbines around it are suddenly shaped like a nice flat fluttering line instead of this violent spiking AI transient mode.
- [27:40]
Narrator
Then there's the race to get connected. Many data centers have the capital and the land, but not enough power. Utilities can often serve them sooner if they're willing to curtail load. That's where batteries change the equation.
- [27:52]
Ken Ron
Utilities are saying, I can't give you consistent power today, but if you wait eight years, I can update the local substation and then I can guarantee that. But if you insert a battery into the middle of that conversation now, you can say, okay, utility, I'll stop taking your power for 1 to 3% of the year, and I'll just provide my own power through this battery for four hours a day.
- [28:13]
Narrator
But deploying the batteries is only part of the story. Inside each battery container are thousands of individual cells, each producing data and performance signals that have to be managed in real time. That's where software comes in.
- [28:25]
Ken Ron
So you have to have really sophisticated software that can talk to every one of those individual batteries and make sure that it's in a safe condition, that it's doing what needs to do, that you still deliver what in this case the data center needs or the grid needs while keeping the battery inside safe. So it is really important to make sure it's highly sophisticated software, but it's also a really great partner.
- [28:46]
Narrator
And that partner is FlexGen. FlexGen's data center solutions and hybrid OS platform help owners and operators source batteries. Commission systems quickly manage performance and keep critical energy assets online when they're needed Most. Discover how FlexGen is powering more strategic energy systems@flexgen.com or click the link in the show notes.
- [29:12]
Stephen Lacy
So that's the picture for at least the corporate landscape and serving all this new demand. Let's actually talk now about how this demand might show up. The last few years, of course, have seen this steady increase in large, training, focused data centers. But something's shifting as the demand for inference picks up. We're seeing the explosion of agents. Companies are blowing through corporate AI budgets. AI labs are running against their compute limits. And so as a result, we're seeing this shift in the conversation about how and where we're going to serve all this new inference. Are we entering a phase where distributed computing could overtake the massive centralized data centers that we're seeing built out right now? And in that World. How do energy needs change? Jigger has been firmly arguing that we are undergoing this shift. I'm a little skeptical about how quickly it's happening, but it's a great conversation. I'm dying to get Carolyn's take on this, so let's roll into it. Jigger, you recently wrote that Google meta AWS are quietly converging on the same conclusion. The gigawatt campus model is hitting limits and that distributed grid connected computing is the future. Make that case.
- [30:33]
Jigger
Well, right now I don't have to make the case, right? Right now you've got 60, 70% of all gigawatt scale data centers that are behind schedule, many of whom haven't even started construction that were supposed to be completed in 2026. And so they're just behind. And so what has happened is Google and all the other hyperscalers and frankly, they're being bypassed directly by anthropic and OpenAI, who's going directly to other owners of data centers, are coming forward and saying, we have a 200 megawatt data center. And people were like, yeah, we could do 200 megawatts. Then they're saying, we have a 50 megawatt data center. They're like, ah, that was small, but we're kind of desperate, we'll do that deal. Then they're coming to like 20 megawatt data centers. They're like, actually, we might do that. Then like, you know, a startup company called Rune just like, you know, raised their round of capital and we're greatly oversubscribed and they're putting up 5 megawatt data centers. Then Nvidia signed up with Prologis and Prologis is now signing up 5 megawatt data centers everywhere, right? And now you've got like, you know, the, you know, like the pilot with Span, right? And so what's happening is we had a major chip shortage in the country, right? And Nvidia was like selling out to 27 and 28. Turns out a lot of those chips are now available on the secondary market because the data centers that they were supposed to go into that were 1 GW in size are not yet up and running. And so everybody that I'm supporting on the distributed side is like, oh yeah, I just secured Blackwells. I just got more Blackwells. They just got offered to me, I just bought more Blackwells. And so the GPU prices have not gone up and in fact they've stayed pretty stable and folks are just unloading it from a cash on cash basis and then they're getting their new shipment in whatever it is, like six months from now, that'll back backfill it. So I don't know that people are aggressively saying that they're against one plus gigawatt data centers, but they are saying that those data centers are not going to come online in time to meet their compute needs. And so they have dropped their standards that they were once sitting on and they're, they are signing contracts for all of this distributed compute now in a way that they were saying that they wouldn't just like six or nine months ago.
- [32:50]
Stephen Lacy
Caroline, is this the same story you're seeing?
- [32:53]
Caroline Golan
Well, I back it up a little bit and say there were some technical advantages to concentrating compute or training. Okay, concentrating training, which is that training was a bit of a mess in the first year from a harmonics perspective, reliability perspective. It was blowing through servers in a way that I remember when I saw the first test at Google. I was like, oh gosh, if that was at a gigawatt scale, we'd blow the grid. And so there was actually a lot of thought and logic behind. We really need to work out the kinks in this. And if you try to work out the kinks of a new technology across 100 different point sources, it makes it very hard just from a scientific engineering process perspective to do that well. Right. So the idea of concentrating training to work out how training is going to work made sense before you even talk about power and self supply and all that stuff. It made sense. I think what a number of entities are finding is that, hey, building power plants and operating power plants is hard. So going off grid or even behind the meter and building our own power plants and trying to do that is not our core competency. And unfortunately it's not a core competency of a lot of utilities in this country anymore. And that's just because that was the nature of load growth being flat. And I don't know if I've used this analogy before on the podcast. I know I've used it in different areas, but I go back to the idea of the military base concept, which is that we are always going to have concentrated large campuses for AI training. Those are going to be your gigawatt plus campuses. Those are going to exist. I don't want to send the message that that's going away. I don't think that's going away. Is it going to happen at this scale that we expected? I don't think so, because chips are working themselves out, right? And then you have, you have training and then you have your outposts, and your outposts can be large or they can be very, very small and strategic. And what I think what Jigart is saying is happening in the field, which is that entities are just desperate to deploy their chips, which they've already bought and are depreciating and are on their books and need to generate tokens or else they're going to go under. That is a major motivational factor. The other motivational factor is that we're starting to work out demand for AI as a product. And you can't actually work out the, the value of that until you are able to integrate it at the cloud level. And integrating it at the cloud level requires proximity, and that proximity requirement requires flexibility about where you put it. So for the large hyperscalers, the value of AI was always going to be who can build the best cloud product. But at the end of the day, cloud is geographically important in proximity to its end use. Right. And so that transition, I think this transition was inevitable. What is interesting, like when I was at Google, I mean we always knew that we would switch back and inference would look much more like the early days of cloud, where we did have 2050, 100, nothing more than a 500 megawatt data center. What I think is so interesting about what's going on now is that the technology capabilities, and I think we'll get into this in a second, that allow for even more micro compute, is really changing the way that the hyperscalers thought about scale and mass procurement. Because originally I think we thought what does procurement at scale look like? It looks like big chunks of concentrated power. And now they're being challenged in saying, well, potentially procurement at scale could look like 1,000 distributed chunks of power. Right. And that is something that is newer and I think has a lot of potential. The devil of course, is going to be in the third party developer. And whether some of these development arms are able to manage at that portfolio level. And if they are, and if the analytics and the grid interoperability and everything is there, great. But there are a lot of holes that have to be or a lot of hoops that have to be jumped through to be able to manage what is an incredibly important and critical asset across thousands and thousands of data nodes. So that's the big question and sort of why I see the shift in the way it is. But I'm for it. I'm 100% well, I mean everyone who
- [38:25]
Jigger
knows anything about the physics of the grid is for it.
- [38:27]
Caroline Golan
Right. That's the other thing which is like we've always, and this is one thing Jigger and I have always agreed on, which is that like the quickest and best way to do this is to figure out how to actually improve the distribution system. Because the transmission system is clunky and in reality built to be a one way power flow. You know, it is built that way. But the distribution system has the ability to, to advance far quicker than transmission does. What I'm more interested in is like, which comes first? Do the entities like Tapestry and whatnot come first to open up this? Or does this force the improved analytics, improved interoperability, improved load plan? Like one is going to come before the other. So one's either going to break the system so the system improves, or it's going to put so much pressure where we just finally start doing the distribution planning that we should have been doing forever. And you can see what's happening in the UK as up on the hills in terms of the onslaught, but you can see how that regulatory paradigm, how those changes in data analytics allowed for a much more flexible distribution system.
- [39:43]
Jigger
That's why Caroline was actually in Scotland. That's exactly why she was citing data centers where the offshore wind farms were coming in. There's all this extra power there. She was like, this would be a good place for a data center where
- [39:56]
Caroline Golan
I rode my horse. I rode my horse to check out more data center sightings. Exactly. No, I mean, but it's true.
- [40:03]
Stephen Lacy
I guess what threw me off about your argument, Jigger, is that it seemed like you were implying that the age of the gigawatt scale data center is coming to an end.
- [40:11]
Jigger
Oh, I am more than applying that. Yeah.
- [40:14]
Caroline Golan
So I think that's where we differ.
- [40:16]
Stephen Lacy
Yeah. And I don't agree.
- [40:16]
Jigger
Nobody has to agree with me right now. I mean, the thing that's interesting now is that, I mean, all roads lead to my predictions at some point. But I'm just saying that like right now they don't have to agree with me because the contracts are getting signed. It's not like they have the choice. The only choice they have, if they want near term inference is stuff that can be built in months, which are the smaller formats. Right. And so now the question becomes, what do the investors do? And I've talked to them a lot and they are genuinely worried. Right. Because what's happening is that when you look at these 1 GW data centers, you can imagine the number of these 1 GW data center buildouts that have already hit final investment decision is a very large number. It's like 30 to 40 gigawatts of 1 gigawatt data centers. So now when Mr. Wonderful goes on TV and says, I feel wronged for my 9 gigawatt data center in Utah, and everyone is saying, I don't think we need Mr. Wonderful's 9 gigawatt off grid data center. Right? Because to Carolyn's point, it was so hard to begin with to justify that gigawatt data center. And then you're going to build it off grid. And then he's like, well, we're never going to connect it on grid. And then you're like, wait a second, are these people really that dumb? And so now you're starting to see a lot of these frothy 1 gigawatt data centers starting to lose all traction. All of their investors are pulling out and saying, you don't have a hyperscaler that really wants your stuff. And that's why Fermi ended up going under. Remember, Fermi had already figured out, okay,
- [42:01]
Caroline Golan
that is not the reason why Fermi went under.
- [42:03]
Jigger
No, no, but I mean, no, but I want to make sure that we're clear about what Fermi could have done. Right? They raised $700 million in the IPO. They had proprietary access to natural gas turbines, but not enough. Right. They could have just built a 500 megawatt data center right away and done exactly what Elon Musk had done at Colossus. Right? They could have said, let's just do a 300 megawatt data center. Let's do a 500 megawatt data center. Let'S just force it. Right? And they had 200 megawatts already of grid capacity, but they were two chickens. And so they were like, we need to get project finance, so we need to get off takers. And to get off takers, we need to do this. And they never got that stuff done right. And so part of what makes Elon Elon is that he said, screw it, I'm not going to do any of that stuff. I'm just going to go and do this merchant. And now he's got 1.5 billion a month from Anthropic. Right. I just think that the vast majority of these 1 GW plus data centers are looking for a project finance structure. And that project finance structure is proving harder to put together than they thought.
- [43:07]
Caroline Golan
I think there's another layer here which we don't talk about a lot, but at the end of the day when we talk about AI and AI training, it's a GPU and a TPU world. And everyone was building infrastructure to integrate chips divorced from the trajectory of chip evolution because they were advantageous. Right. And so there will always be a sector of this market that needs a concentration area, concentrated area, to figure out the evolution of chips and training and development. So I don't think the concentrated space is going away. I think it's going to get smarter, I hope. And part of what I hope to do with NRG is I hope it gets integrated as an asset to the grid as opposed to just this suck from the grid. But I don't think that's going away because I don't think anyone is going to continually blindly test out training in this distributed manner. It's not a smart way.
- [44:25]
Jigger
But we're not arguing that, Caroline. Not a single person is arguing that. What everyone is arguing is that we have 30 gigawatts of 1 gigawatt plus data centers already under construction. They might be behind schedule. And when you talk to Epoch AI Research, they're saying that even in 2030, Anthropic is saying its largest large language model only needs 4 gigawatts for training.
- [44:50]
Caroline Golan
But that's because of chip evolution is what I'm saying. Is it like.
- [44:52]
Jigger
No, I think we're saying the same thing. The question then, does someone do the 52nd gigawatt of 1 gigawatt plus data centers and the 64th gigawatt of 1 GW plus data centers? Because that's what's getting funded right now by development land developers called Mr. Wonderful. And they're the ones pissing everybody off at the local level. Right? And they don't care about water and they don't care about this stuff because they're just trying to flip these land positions that they have and make way more money. And what I'm hearing from all the hyperscalers is they're ready to tell all those guys to stop pissing everybody off. We're not buying your 1 gigawatt computer anymore. We're just finishing up the 30 plus gigawatts that we've already committed to.
- [45:38]
Caroline Golan
I don't think the shift is completely in the other direction. I don't think the pendulum has completely swung. I think most the hyperscalers are looking at the fact that they need options across all of the infrastructure. What I will tell you what has shifted is that a year ago it was heads down, concentrate, compute, build as much as you can in a few choice areas because that was the economies of scale and question that we were all trying to figure out. Now it's broader than that. And so I think that the bubble, if there is a bubble to Jigger's point, is around the idea that everything's going to be concentrated in these gigawatt plus data centers. I think that bubble was all, I think anyone who was actually working in this industry knew that that was always a bit of a bubble. You just have to follow where the revenue stream was going to come from. Google Nvidia was never going to make revenue just by training. They have to make revenue in cloud.
- [46:46]
Jigger
Well, you say that as if it's obvious, but I've been saying this very loudly from the open circuit podcast platform and got a lot of hate mail for saying it. Even though, like it's pretty obvious over
- [46:58]
Caroline Golan
here that 90% of all computers with such a more calm tenor, with grace. And I don't call people idiots.
- [47:08]
Stephen Lacy
Yeah, I mean, I think we also have a ways to go before we see this market where inference represents 80% of compute. Right. Like that's.
- [47:18]
Jigger
No, I mean Epoch AI research is
- [47:20]
Stephen Lacy
saying 90% five to seven years away.
- [47:22]
Jigger
No, they're saying 90% of all compute in 2030 will be inference. That is like what Epoch AI research is saying. And they're, I mean they're saying that even, even for training, they're saying only the final training runs have to be done in 1 GW plus data centers. That when they do their R and D training runs, they can do them in batches in 100 megawatt and 200 megawatt data centers.
- [47:46]
Caroline Golan
But that is for existing chip schedules. And what I'm saying is that they're saying in 23, but that. Right, which is if your chip's coming out right now in 2026, you have a 3 at most 4 year run rate. So what I'm saying is that that run rate will continue to evolve. It will continue. And if, I think, if everyone thinks that like Google Nvidia are like, yeah, we're good, our chips are good, we're only testing.
- [48:12]
Jigger
No one's saying that. And we have a big IPO this last week that proved otherwise. Right, Right. It's not Cerberus, it's like Cerebras. I don't know how do you.
- [48:20]
Caroline Golan
That sounds Scottish anyway.
- [48:21]
Jigger
But whatever it is like, but they had a huge ipo, right. And they do the wafer, you know, S chips and like, you know, 900 times better on the tokens than traditional chip architecture. So I think we're getting better. I just think that like, like what governors want to know is that we have 30 plus gigawatts worth of data centers already under construction that have already hit final investment decision. Some of them are like four months delayed, eight months delayed, whatever. But they're already like under construction and will be online by 2030. Right. Epoch AI research is saying that the largest large language models are going to need 4 gigawatts of training for 2030. Now after 2030 they might need more. Right. So the question is how many more crazy disruptive 1 GW data centers do we want to greenlight this week or do we want to see how this stuff plays out and really focus on the distributed data centers until we figure out how this stuff plays out?
- [49:19]
Stephen Lacy
Well, let's talk about the distributed data centers. There's a lot of ideas out there about what the nodes will look like, how you build out this network. And we have just a few minutes left. So I want to turn this into a segment that we are calling short circuit. We've got a few examples here. We want to get quick.
- [49:38]
Jigger
I feel like that's usually a negative thing.
- [49:40]
Caroline Golan
I was going to say short circuit. None of these technologies are going to short circuit the grid.
- [49:45]
Jigger
I want to call it electrocution.
- [49:49]
Caroline Golan
Trial by fire. Yeah.
- [49:51]
Narrator
Okay.
- [49:53]
Stephen Lacy
You got a better name?
- [49:56]
Jigger
Lightning Round. Lightning Round. Let's just do it.
- [49:58]
Stephen Lacy
Oh, everybody uses Lightning Round. All right, we're going to.
- [50:00]
Jigger
Okay. Okay, got it. Here we go.
- [50:02]
Stephen Lacy
Short circuit.
- [50:05]
Jigger
Mars Shot.
- [50:06]
Stephen Lacy
No. Okay. The first one I want to talk about, which Jigger mentioned was, is Span's XFRA Node. This is basically Span's plan to turn your home into a data center. It was announced at our Transition AI conference in April. Span is a smart panel company piloting this thing called XFRA Node, which is this liquid cooled inference GPU module that gets installed on the exterior of a newly built home. They charge 150 bucks a month for electricity and Internet and then they own the hardware and sell the compute to hyperscalers. Is this a compelling concept? Does it seem viable? Caroline, what do you think?
- [50:50]
Caroline Golan
Yeah, I mean, I'm a big, big fan of this concept. Where I am personally and professionally most interested is generally evolution of the concept of the smart home. What that means. I think the vast majority of customers, residents, voters, however you want to denote them in this country, do not want to think about their electricity use in the same way. I don't want to think about my WI fi server. Like I don't want to think about any of that stuff. I still don't even know how to turn my TV on properly because it's so complicated and it frustrates me that I have to think about which remote to use and which button to press. And so I think the vast majority of households don't want to think. I think there are some cool techie, you know, 2% that are interested in how time of use works, interested in sort of solar and storage and all those things. And so this becomes a very interesting partnership with the general smart home development and where smart home is going to go. And I think, I don't think this is something new, but Amazon meta Google. A huge part of the growth in AI is also being able to be integral into the way a home operates. So you can imagine being able to pre program your home, you know, from your Gmail, right. Or you know, set through Amazon how your, your home is going to operate for the next month and allowing that bot to be able to do that. If that takes off in a way this becomes an absolute no brainer. Right. Because then each home is its own inference. Right. And even broader than that, the question for me is would anyone adopt this if they weren't already adopting the smart home concept? Right. And that's where an integration across those two is incredibly interesting. And I'm 100% for it. I just think that there is a customer acquisition model that has to be thought through and there's going to be a lot of competition in that space over the next couple years.
- [53:23]
Stephen Lacy
Yeah, Jigger, what do you think?
- [53:26]
Jigger
So right now in the early days, they're able to pay the homeowner $2,000 a month in rent to put this thing at their site, right? So like, which is very difficult. There's a monetary, yeah, there's a monetary like reason to do this and they'll pay, pay their electricity bill and their Internet bill for them, right? So like, so there's a value there. The other piece I'd say, which I didn't fully appreciate until I got deeper into it is that, you know, we're just moving away from Claude and ChatGPT and there's just so many much smaller LLMs that are focused on the medical industry or the dental industry or this industry or that industry. And they're really easily, you know, hosted at these data centers and so these small ones. Right. And so, you know, like you think about the 4 gigawatt, like Claude LLM in 2030 and then how much memory you're going to need to be able to like, you know, host it and all that stuff. But there's so many other LLMs that like actually don't they're more tailored.
- [54:29]
Caroline Golan
Yeah, yeah, they're more tailored. And that tailored approach lends itself to a micro distributed approach. Yeah, I agree.
- [54:36]
Jigger
Yeah.
- [54:36]
Stephen Lacy
I mean it's a super interesting economic case. I think Span's pitch is that 100 megawatt data center takes a few years to build, costs $15 million per megawatt, and Span says it can deliver compute across 8,000 homes in six months at $3 million per megawatt. So we'll see if they can do it. The one thing that stood out to me was like theft, like putting these valuable GPUs on the outside of your house. It seems like it's ripe for theft.
- [55:04]
Jigger
They only live in gated communities in atlant.
- [55:07]
Caroline Golan
No, I mean it's the same customer acquisition as I go back issue that you had with the smart home. Jigger's completely right. Getting paid to do this as opposed to saying it pays for itself in a couple years and then you save 400 bucks a month or a year is very different. But ownership over this long term is a very interesting question because you can imagine, and this is, is where I think Jigger and I have always agreed is like the grid, the future grid is far more micro, is far more distributed. And with agentic buying and with the ability to have real time data and treat electrons as they should be physically treated in, in real time, the whole system changes. Right. And what is interesting to me about this as well as the other technologies is this provides a level of prosumer engagement that we've never seen before. And the question is, what happens to the role of the utility? What happens to the role of the electricity provider if all of a sudden the customer, the residential customer, becomes a net asset to the grid and a net asset to economic development? That's never happened before. We've never seen that happen before. It completely changes the equation and flips it on its head. I'm for it. I, I'm not, I don't think I'm so concerned about things like theft. The bigger thing that I would be concerned about is at the end is insurance. Like how are you insuring this against residential? Like, I don't know the answer to that. Maybe some one of our listeners do. But one of the more interesting things for me is like, like I've, and I've been saying this for a long time is that if you want to know where the future of data center infrastructure is going, you need to look at forward curves on the insurance market. Oh, we don't have forward curves. On the insurance market. So no one can really know where this is going, but I'd be interested in the way that insurance sort of appraises this and where that happens.
- [57:20]
Stephen Lacy
Well, this was supposed to be a lightning round. Now Caroline's forward curves on the insurance market.
- [57:25]
Jigger
No, totally.
- [57:26]
Stephen Lacy
I love it. I love it.
- [57:27]
Jigger
I told you it was a march.
- [57:29]
Caroline Golan
I personally would have rather talked about this than, you know, Nextera buying Dominion is far more interesting to me. Sorry.
- [57:37]
Stephen Lacy
I'll do better next time. I'm sorry.
- [57:39]
Caroline Golan
It's okay. I'm tired.
- [57:39]
Stephen Lacy
What do you think of this startup? What do you think of this startup in the UK called Heda? They're installing a small server in your home that processes certain workloads and then reroutes the waste heat directly into a hot water cylinder. Free hot water in exchange for hosting the hardware.
- [57:56]
Caroline Golan
Yeah, interesting that. No, no, no, no.
- [57:59]
Jigger
I'm just saying pay me $2,000 a month. I don't want your free heat. Like, I'm just saying, like all this like, you know, co Jad. I'll just give you free heating and cooling in Exchange for making $50,000 a month off of you.
- [58:13]
Caroline Golan
No, I think that the waste heat recycling is probably more a product of the policies and data center requirements that were passed at the EU level that the UK bought into. So my guess is the reason why they started with that is because there was a policy requirement to do that and that was the box they checked.
- [58:35]
Stephen Lacy
All right, the third one. Panthlassa. This is a Peter Thielback company. They're building an 85 meter steel structure that will get towed out into the deep ocean, flipped vertical and it will bob with the swell. And there's compute sealed inside. It's cooled by seawater. It's connected to the world via Starlink. And this is for like long running inference and reinforcement. Learning stuff that doesn't need low latency. This has been a tough space. Marine energy is like a notoriously very difficult space. Can they get the economics right? Jigger, is this a serious contender or science project?
- [59:12]
Jigger
Short the stock?
- [59:14]
Caroline Golan
I'm so glad you asked. Jigger. I have no idea. I mean, I looked at this.
- [59:18]
Jigger
You do know you are not putting your own 401k into this.
- [59:22]
Caroline Golan
I am not putting my own 401 into this.
- [59:25]
Jigger
No. It is crazy town. Like first we go to space and then when space didn't work out, they're like, we're going to the bottom of the ocean.
- [59:31]
Caroline Golan
I think so.
- [59:32]
Stephen Lacy
Well, he's been doing this since 2016. They've been working on this.
- [59:36]
Jigger
I mean, people have been working on all sorts of crazy ideas this way. Space. Yeah.
- [59:39]
Caroline Golan
I mean, we still don't have a lot of floating solar plants either. So, you know, I still don't have
- [59:44]
Jigger
a lot of space solar that's like being beamed to Earth. I still don't have like a ton of.
- [59:50]
Caroline Golan
I think space solar still happens. I mean, I'm not sure it happens at this camp, but I'm going to put, I will put my money on space solar.
- [59:55]
Jigger
I think we just tested it. I think we just tested it. I think both China and Japan now have working space solar units for like, whatever it is, like $50 a kilowatt hour.
- [60:07]
Stephen Lacy
But sure, yeah, sure. Well, if you want to dig deeper into the concept, because we don't have a lot of time to talk about it here, watch the recent episode of Catalyst with Shayl Khan. He talks with the co founder and CEO of this company and it's really fascinating. It's the first concept in the marine environment and marine energy that actually made me like prick up my ears and think that they're being very thoughtful about it and about their design and the limitations of other marine energy devices. So we shall see. And if any of these become the next big solution or the next idea to fizzle, you'll hear about it right here on Open Circuit. Thanks, everyone for being here. Caroline, thanks for joining us. Hopping off a plane and coming right here. Good to see you as always, Jigger. Great to see you, too. Are you traveling this week? Are you staying put?
- [60:57]
Jigger
I'm staying put. I am just trying to figure out how we actually get. My fourth grader graduated on Friday.
- [61:07]
Stephen Lacy
Oh, nice.
- [61:07]
Jigger
It's like a whole production and so like we're. He's going to middle school. It starts in fifth grade now, which I think. What?
- [61:13]
Caroline Golan
Why?
- [61:14]
Stephen Lacy
Yeah, yeah.
- [61:15]
Jigger
That's the way they do it.
- [61:16]
Stephen Lacy
Confusing.
- [61:17]
Jigger
I mean, you know, no one asks me for my advice on schooling.
- [61:21]
Caroline Golan
D.C. y' all always have to do things differently.
- [61:25]
Jigger
I know, I know.
- [61:31]
Stephen Lacy
Thanks everyone for being here. Hit the subscribe button if you don't already subscribe to the show on YouTube. Open Circuit is produced by Latitude Media. The show is edited by me, Sean Marquand and Ann Bailey. You can find the audio version of this, of course, wherever you get your podcast and find us on The Latitude Media YouTube page to watch the videos. We will be with you next week. We really appreciate you being here every week with us and we'll see you next time.