Loading summary
Transcript35 lines
- [00:02]
A
As AI's potential and popularity have grown in recent years, so too has its notoriety. From its economic boom to its ethical and intellectual concerns, to the underlying question, how much will this technology change our lives? To understand this is to also understand what makes it so complex. While many of us are used to the intuitive nature of ChatGPT, Gemini, and Claude, it's the large underlying layer of elements that make AI so complex, and the advanced software design, the chips and the data centers.
- [00:33]
B
So the AI that you're using, your social media apps, it doesn't exist in a cloud, it exists in a data center. Data centers are everywhere.
- [00:43]
A
If AI is shaping the future, then data centers are what's powering that future. They can be large and seemingly uninteresting infrastructures that essentially hold computers, servers and storage drives. And as the name implies, they hold data, process it, and ensure things run smoothly. Data centers have recently become a point of attraction as major tech companies such as Meta and Amazon have invested heavily in the development of new and current centers. In the tech world, they're like the physical libraries that hold digital information. As the demand for tech grows, so too for data centers. If AI is enabling the future of virtually every industry and data centers are one of its core pillars, what do we need to understand about them to continue AI development? I'm Jennifer Strong, a tech journalist for more than 20 years. In this episode, we'll explore the role of data centers and how they've become such an integral part of our everyday lives. Welcome to the next innovation.
- [01:47]
B
Data centers are kind of the infrastructure. They're basically the infrastructure that supports AI and supports different types of technologies. So the data centers are like a warehouse and they are getting built bigger and bigger, but they're a warehouse and they have many servers inside and all of the electrical equipment and cooling equipment needed for those servers. So basically what's happening now with a lot of these really big data centers that are getting built, and they are often to use to train artificial intelligence, so train the large language models, all of that, that work is really big right now. And with the idea of eventually training those and kind of rolling them out to the world.
- [02:42]
A
Leila Kearney is a U.S. power correspondent for Reuters. A few years back, she started focusing on data centers when major tech companies, known as hyperscalers, started promising more and more spending. By early 2024, almost every utility company was on board in that first year.
- [03:00]
B
It really was mostly about the excitement, all the money that was going to be spent, lot of it going into electrical infrastructure that's needed to power the data centers. Data centers are measured by their capacity, the amount of electricity basically that they can consume. So that first year was like w all of this power demand in the US which had been kind of stagnating. Not much change in power demand growth over like 20 years. And extraordinary things were happening.
- [03:39]
A
January of 2024 was a major month for AI. It was the month OpenAI announced its ChatGPT store which meant users could create custom versions of GPT for specific tasks. Tesla rolled out its automated driving mode and major corporations began adopting generative AI into daily oper, which brought the question do we have enough resources to power all these endeavors? As demand for data centers grew, so did the demand for energy supply. Data centers are generally powered by massive amounts of electricity, usually from the local power grid. This can be a potentially dangerous strain on the grid. Not only that, but data centers need to run efficiently and reliably. 247 any power outage could prove catastrophic as servers and technological infrastructure around the world is interrupted. Which is why so many companies rely on mission critical infrastructures. They're a reliable form of energy specifically designed for operations that require on the call energy supply like defense systems or emergency response or communications. These industries like data centers are incredibly sensitive to power outages from cyber attacks to natural disasters. We wanted to get a better sense of how mission critical power is supporting data center operations. So I spoke with Niall McFadden, CEO of CEL Critical Power, an Irish mission critical power supplier.
- [05:05]
C
So we have to design redundancy into the system and we have to design backup and we hook into all those systems. When the power comes from the grid through a factory wall, it's usually high voltage or medium voltage. And we will supply transformers. We don't make those, we procure them to step it down to low voltage which is what is the usual operating voltage level. And then we will clean it. So there's things called harmonics, sort of like a lot of technical electrical jargon. But the basically it needs to get from high voltage to clean low voltage. And then it needs to get distributed around the facility which we use ring mains for. And then we use to bring in the switchgear that will just that will keep distributed like in your house where you got it under your desk, you got a four way socket for your computer and all the ancillaries, we do that up to 64 different ways and distribute the power all the way down to the rack. The design will be led by the requirements of the data center operator. What sort of electrical load they want to run and how they want to control it. And they will give us what's called a single line drawing, which is like a big electrical diagram. They say that's what we need. Can you please fit that in to a cabinet that fits our footprint and deliver us the specification we require? So there'll be different breakers, different metering, multiple channels. Your power will come into your switchboards and you will have to then push it back out again into lesser rated panels all around the facility. So it's about balancing it up and creating an environment where you have redundancy if anything fails, that there's alternative routes to supply the power and also that there are ability to do maintenance and shut off particular lines and all that sort of stuff. So we just really provide the flexibility for the operator to keep the facility running and provide the protection for their equipment and provide the backup should anything fall over.
- [06:54]
A
CEL focuses on building and maintaining capacity, which is energy supply. For over 40 years they've been serving high level clients, including the world's top 10 pharma companies and most recently 4 out of the 10 top data center providers. The company delivers these solutions using a design for manufacture approach with a focus on meeting customer needs from concept to delivery. This approach supports the operational requirements of AI compute workloads where high power density, continuous availability and future proof designs are all essential. They collaborate with hyperscalers and operators to design and produce power distribution units and modular systems that contain switchgear and cooling infrastructures, among other things. The modules are delivered as plug and play systems that allow customers to rapidly add power capacity without lengthy on site construction. These systems safely route and manage large volumes of electricity through industrial facilities and data centers. But the demand for AI and other tech ventures has created an even bigger calling for data centers. To keep a data center running, you need land permits, energy, cooling and electrical distribution, which is why the right location is an invaluable asset in this regard. Earlier this year, CEL opened its first large scale manufacturing facility in Northern Virginia. That's not a surprise location for what tech journalists call data center alley. It's actually the global epicenter of the Internet. This area is home to roughly 300 data centers, powering nearly 70% of the world's daily Internet traffic.
- [08:30]
B
Northern Virginia is the biggest biggest data center market in the US or hub in the US just in terms of how much capacity of data centers there is there. And so they're a mature market in Northern Virginia. There's a lot of fiber connectivity and it's right near Washington D.C. so big, big U.S. populated markets, and that area is continuing to expand as a data center market. There's still a lot of demand from data centers there.
- [09:04]
C
Virginia is number one data center alley in the world. So you've got a lot of skills right across the board. That's the first thing. Secondly, it has a great supply chain. So we have the data center supply chain, we have the Newport News shipbuilding region, and then we have the Norfolk Naval yard and all their hinterland. So we have all the skills and supply chain sub assembly people in the region who can do everything from like, we build up electrical baseboards with all of our metering and control wiring and everything. So that's a job which we give to the specialists. We get all our metal built locally. So we build our. A switchgear is the size of a, what you call a wardrobe over here at closet. So our switch gear is the size of multiple closets, and it's built like mechano sets. Remember mechano sets growing up? So we bolt these cubicles together and then we panel them, we clad them with panels, and inside it goes our copper arrangements, which distributes all the power. We put our breakers in there. So the power comes in on an incoming breaker and then is distributed through the switchgear and goes out on outgoing breakers called feeders. So we do all of that work and we will typically look for local people maybe to have it fabricate the complex copper. If we need it, we can do that ourselves. But to get scale, we sometimes will spread it out. We have the metal people and then we have the transformer suppliers. There's 10 transformer suppliers in the US we can use.
- [10:31]
A
It's important to note that data centers have been around since the inception of the Internet. And in the last 30 years, Virginia has pretty much remained at the top of the market for development. Why companies choose Virginia has always been indicative of the kind of resources amenable to developers.
- [10:47]
D
Data really can travel at the speed of light.
- [10:50]
A
Tommy Pan Fang is an assistant professor of strategy at Rice University. He recently penned a study that examined why data centers choose to locate where they develop, specifically the economic incentives driving those decisions.
- [11:04]
D
Virginia has had a very long history with data centers, originating really from academic efforts, proximity to the government. Over time, it has continued to grow in terms of the number of data centers, and this was driven by AOL and Amazon in recent years. There are a lot of benefits on the supply side of being located in Northern Virginia, both in terms of the availability of land, cheap electricity prices, local tax abatements, and incentives that are offered by the government. And I think that creates a very attractive area or opportunity for data center firms to choose to locate in Northern Virginia. It also has benefits in terms of its geographic position because of the rich infrastructure in the area related to IXPS and broadband, as well as being able to be located close to to a number of large commercial hubs in the local area, which can reduce latency for certain firms that might require lower latency. There has been a long history of the underlying infrastructure dating back to the 1960s with Arpanet. And then as it became more interesting to sort of be able to look at ways that the Internet could connect different areas, the government sort of provided more funding as well. And so I think that is the backdrop. In the 1990s, we saw the first data centers being established in Virginia. And I think there has been a lot of support from the local government in terms of incentives, in terms of the availability of land that led to more clustering in that area area. One of the other things that is advantageous is that Amazon has built quite a few data centers, hyperscaler data centers in Virginia. And so that has led to some degree of spillovers in terms of they're being trained, you know, skilled human capital in the area that knows how to build data centers already. Right. So there's less of a learning curve if you are trying to build in Virginia. Data centers do require high reliability. And so having abundant energy, making sure that there is very low downtime for these data centers can be quite important. And so I think this is also part of the historical reason for why data centers have been located in Virginia. Because of this confluence of different factors that are favorable, both on the supply and demand side.
- [14:07]
A
Tommy's study found that while rural areas with vast land opportunities can be attractive to hyperscalers, it's often the case that they choose to develop in urban, highly populated areas. His research indicates that the biggest association is between population density and the size and number of data centers in the area. According to his research, nearly every metropolitan area with a population above 750,000 had at least one data center. But those that were in heavily populated areas tended to be much smaller and less powerful.
- [14:39]
D
Where these data centers are is related to the availability of local demand as well. So if you have a large hospital system, if you have high finance nearby, there may be reasons why those types of firms want to have a data center close by. Right. It might be related to security concerns, compliance with certain data standards, latency related to data, as well as other Concerns such as being able to troubleshoot things if they need to. And so having a data center nearby seemed to be a bias that some managers prefer to have.
- [15:21]
A
Still, building data centers means having access to a reliable labor force, one with the right skill set. Developing a data center can be perceived as a valuable source of economic boost, but ensuring that the right people can actually work is also a key to its success.
- [15:36]
D
I think here there's an interesting twist in that in order for a data center to be operational, you only need a few dozen or at most a few hundred skilled employees. Right. And so unlike other types of technology, let's say a software firm, for example. Right. There's not going to be the same type of skilled technical human capital that is required in order to maintain the day to day operations of these data centers. One of the other points that has been discussed as well is whether there are positive spillovers from these data centers in terms of will there be more AI jobs where data centers lead other types of skilled human capital that might relocate to where these data centers are.
- [16:26]
C
So all of our supply chain is now local in the U.S. all of our employees are local. There's no Irish employees in the us and they come from a mix. Probably half, which is our objective, come from the DoD skillbridge program or they're vets. So either they come from DOD SkillsBridge or else we hired vets anyway because they're just better trained and very easy to work with in my experience, and have a high quality approach to life and high standards. And we mix them with the other half of our worker population, which is people from graduate programs and people coming from business and people coming from engineering colleges. The people, the supply chain and the fact that Virginia is number one in the world for data centers made it an easy place to go.
- [17:10]
A
The Skillbridge program was introduced by the Department of defense back in 2011 as an opportunity for transitioning service members to participate in civil training programs. Companies like Amazon, Lockheed Martin and John Deere have previously served as Skillbridge sponsors. Much of the notoriety behind data centers also stems from its high dependence on energy supply and the resources it relies on for continuous power generation, like water.
- [17:37]
B
Water plays a very important role in the most recent iteration of data centers because they are used to, to cool down the data centers to sort of whisk away the heat that's being generated by the servers. They are tremendously powerful servers and that power is literally powered by power, electricity. And the more you use, the more heat is generated. And so you can't just have all that heat in essentially a warehouse, which would basically melt everything at the end of the day if you didn't cool things down. So the water is used to create sort of a river, like whisking away of power from the servers, as opposed to sort of older fashioned methods or something you might use in a smaller data center where you just use air, essentially powerful air conditioning like we would have in our homes, or many people, at least in the US Would have in their homes.
- [18:41]
C
Like, if you think about Iraq, in a data center today, like, typically they would be under 10 kilowatts, the power in a rack, which is like three kettles in your kitchen. We're now talking about going 100, 200, 300 kilowatts and a megawatt. So, you know, a 300 kilowatt rack from Nvidia is 100 kettles. Now if you suddenly plug that into your kitchen, the whole thing will fall apart on you. Like, part of our challenge is to work out how to, how to move our designs and our configurations for customers from being able to handle 10 kilowatt racks to be able to handle 300 kilowatt racks. And further, I mean, they're talking about a megawatt rack in three or four years time. So you've got to fit it in the same footprint. And energy, electricity creates energy, heat.
- [19:27]
B
These data centers are being built overwhelmingly with liquid cooling systems. So that's where the water comes into play. So running these servers, they're really powerful and they generate a ton of heat and that heat needs to be removed. And so it used to be that there were kind of ambient cooling systems where you would just cool down the air in the data center. But now liquid cooling is used and that is a more efficient way of basically creating rivers and that carry away the heat. And so that's when you're thinking about data center water use.
- [20:10]
A
The high level of energy consumption has become a considerable point of concern for both AI proponents and its observers. The truth is that as AI capabilities expand, so does its footprint. Raising the question, is there any alternative? Current research is studying the ways in which different forms of data centers could be a productive alternative in the near future.
- [20:33]
E
My name is Igor Bargatin. I'm an associate professor of mechanical engineering and applied mechanics at the University of Pennsylvania. I do a lot of interdisciplinary research. I'm a physicist.
- [20:44]
A
He's leading a research project that examines how solar powered data centers can operate in space. The goal is to reduce the weight, power consumption, and overall complexity of terrestrial data centers. The advantage is 24,7 solar power, continuous power and no heavy energy cost.
- [21:02]
E
So when we use solar panels on Earth, we can only do it when during the day obviously, and then also when there are no clouds. And that severely limits the output. So you have to greatly oversize the solar panels. You have to have some storage capability. If you only can use solar panels. That's one of the major challenges is generally with adopting source cells more widely on Earth, in space, at least in some orbits, you can avoid all of that and basically have the sunlight hit your solar panels all the time. And that's very convenient, not only because you don't need a battery, or at least not a large battery. It's also convenient because there is something called thermal solution stresses that affect all satellites. If a satellite ever goes into the shadow and is not exposed to sunlight, its temperature will drop. And that causes all kinds of small changes that can wear out the mechanical structures in the satellite over time.
- [22:08]
A
The idea of orbital solar powered data centers is currently being explored by major companies like SpaceX, Anthropic, Nvidia, Google, Google and others. It's a response to how we can continue to expand and invest in an AI powered future with more efficient sources of energy. But launching orbital data centers is hard. It can be highly expensive and difficult to design and build.
- [22:31]
E
Right now there are individual satellites that weigh maybe about a ton and can power produce what is these days called compute power, meaning electricity that goes directly into the chips that do the calculations, or maybe a kilowatt or so. A typical data center would probably start from several megawatts and can go to tens or hundreds of megawatts. We need to launch thousands of these satellites to really make it equivalent to a terrestrial data center. And we will need to launch millions of satellites to make a dent in the build out that's already happening on Earth because we are currently planning to add gigawatts of data centers on Earth every year. And therefore if you have take 1kW to do a gigawatt, we would need to launch a million satellites. So the numbers are just gigantic. So you know, if, if you take a smallish data center that is let's say 10 megawatts on Earth, then you have to launch 10,000 satellites. And each of them weighs a ton. So that would be 10,000 tons. And at $1,000 per kilogram, I guess that's a million dollars per kilogram ton that we're talking about. $10 billion. If my math is correct.
- [24:03]
A
The lighter the satellite, the cheaper the mission, the closer the satellite in Orbit. The cheaper the mission, the higher in orbit, the higher the radiation, making it potentially difficult for AI chips to operate. But developing a light orbital data center is also hard. Terrestrial data centers are made of concrete and steel. Satellites are mostly made of aluminum and aluminum alloys, which are lighter materials. But ensuring the right amount of material for a satellite can still be quite expensive, let alone operating Internet and AI capabilities.
- [24:35]
E
So that's one of the problems that we're trying to solve is how, how do you decrease just the total number of things that you have to track in space from. From millions to thousands, which we currently do track thousands of objects. So presumably thousands is something that is more manageable. The other thing is, is how do we make everything as lightweight as possible? Because we talked about the fact that launching things into space is $1,000 a kilogram, roughly, and maybe that will go to hundreds of dollars per kilogram with, with the starship coming online. But it's still going to scale with the mass. So the lower the mass, the cheaper it's going to be, the more competitive it's going to be. And there we, we try to do a radical redesign where we, instead of relying on the traditional rigid trusses, the whole architecture is actually like a string. And in space there is an interesting effect of something called gravity gradient orientation. And when you have a string in space, it orients itself towards the center of the there. That is something we can use to maintain the right orientation towards the sun and minimize the mass at the same time. So it's much easier to create a structure that remains under tension. You can make it much lighter than a structure that ever is compressed. So if something compresses and typical satellites would have to survive compression during any orbital maneuver. If you can get rid of it, you can make all the structural components that maintain the orientation, for example, much lighter. So that's, that's kind of the goal. We're trying to make it much lighter by using some physics tricks that come from gravity gradient. And then another thing that we're using is actually solar pressure. The sunlight exerts a little bit of pressure. It's actually a tiny amount. For most practical purposes you can ignore it. But in some cases, especially if you try to make the structure as light as possible, it starts to matter. One of the directions of orientation is solved by the gravity gradient, but the remaining one is solved by solar pressure. So we designed our structure to self orient its solar panels towards the sun completely passively, without using any active propellants, any rockets, any thrusters. And that's another way to make the structure much lighter.
- [27:12]
A
It's going to take several years before we start seeing or even seriously discussing data centers in space. The work Igor and his research team are doing contributes to that conversation. There are still countless concerns when it comes to satellite data centers, some that even rival those of terrestrial data centers. Deploying millions of satellites would massively worsen space debris, which is a growing under regulated problem. Terrestrial centers have known problems, but operate within regulatory, labor, and community accountability frameworks. The data centers we know on land can be noisy and obnoxious, and orbital data centers are also vulnerable to satellite traffic and collisions that can destroy them. But the scientists and engineers building these infrastructures continue to work on solutions that could mitigate these concerns, be it on land or in space.
- [28:00]
C
It is pretty clear to all of us who are involved that there's a huge amount of capacity required for AI, and the question is just how much that capacity is that capacity that's required from how much is that figure? And I don't think anyone really knows.
- [28:14]
A
The possibilities are always diversifying, which means there isn't really a shortage to how we can think, design and develop a future where technology is efficient and reliable in every sense of the word. Thanks for listening to the Next Innovation. This series was produced by Situation Room Studios and Powered by Enterprise Ireland. Investing in the next wave of innovation. Our executive producer is Christine Barata and our senior producer is Sharon Barreiro. Additional production assistance by Global Situation Room. I'm your host Jennifer Strong. Until next time, it.