AWS Podcast Episode #716: Concrete, Cooling, and Compute: Reinventing Data Centers for the AI Age

Release Date: April 14, 2025

In Episode #716 of the AWS Podcast, hosted by Simon Lesh and featuring guest Stephen Callahan, Senior Principal Engineer at Amazon, the discussion delves deep into the transformative changes AWS is implementing in its data center infrastructure to meet the burgeoning demands of the AI era. The episode offers insights into scaling challenges, innovative cooling solutions, sustainability initiatives, and the underlying philosophies driving these advancements.

1. Introduction to AWS Data Centers

Simon Lesh opens the episode by highlighting the often-overlooked yet critical role of data centers in AWS's operations. While AWS typically emphasizes its services accessible via APIs, the physical infrastructure supporting these services remains a cornerstone of their reliability and scalability.

Simon Lesh [00:30]: "Servers have to run somewhere. It has to run somewhere... technology is made of hardware and software, and those things have to come together."

Stephen Callahan reflects on his 15-year tenure at Amazon, emphasizing the rapid evolution of data center technologies and the continuous innovations required to stay ahead.

Stephen Callahan [00:47]: "It's been a roller coaster... feels like we've been through three or four different variations since then."

2. Scaling AWS Infrastructure

The conversation underscores the exponential growth AWS experiences, necessitating a shift in scaling strategies. Callahan humorously notes the progression from gigabits to terabits and now petabits, illustrating the relentless pace of technological advancement.

Stephen Callahan [02:11]: "It's just a different letter, you know, It'll always evolve."

Lesh emphasizes that such scaling efforts benefit customers by ensuring AWS can deliver robust and scalable solutions.

Simon Lesh [00:47]: "Every year is like three to five over here. But that's a good thing for our customers..."

3. Impact of Generative AI on Data Center Design

A significant portion of the episode examines how Generative AI (Gen AI) workloads are reshaping data center architectures. Unlike traditional, diverse workloads, Gen AI demands high-density, power-intensive compute clusters with minimal latency. This shift necessitates:

• Concentrated Compute Clusters: Entire racks or clusters of racks designed to handle large-scale AI models.

  Stephen Callahan [06:00]: "We're talking entire racks are now systems... having that concentration helps these models."

• High-Speed Networking: Ensuring low latency and high bandwidth between vast numbers of GPUs.

  Stephen Callahan [06:56]: "We're able to get things closer to each other... under a couple of microseconds latency."

4. Power Consumption and Advanced Cooling Solutions

As AI workloads escalate power consumption and heat output, AWS is pioneering advanced cooling methodologies to maintain efficiency and sustainability.

Multimodal Cooling Systems

AWS has transitioned from traditional cooling methods to multimodal cooling, which dynamically adjusts based on real-time data center needs. This flexibility allows AWS to:

• Optimize Energy Use: Combining chillers with evaporative cooling or shutting down chillers when not needed.

  Stephen Callahan [09:43]: "We've been moving to systems called multimodal cooling... depending on the needs of the data center at that time."

• Integrate Liquid Cooling: Directly cooling GPUs and accelerators to manage concentrated heat loads effectively.

  Stephen Callahan [10:00]: "We're now pushing liquid cooling all the way to the GPU or the accelerator itself."

Adaptable Data Center Designs

AWS designs data centers tailored to diverse climates, ensuring optimal cooling efficiency worldwide—from temperate regions like Ireland to humid areas like Mississippi and arid deserts in the Middle East.

Stephen Callahan [11:58]: "Each building... has to be the best data center for the scenario that we have."

5. Sustainability and Renewable Energy Initiatives

Sustainability remains a top priority as AWS scales its data centers. The company not only achieved its goal of pairing data centers with 100% renewable energy by 2023 but continues to explore deeper efficiencies.

Strategic Location Selection

AWS strategically locates data centers near renewable energy sources—such as solar and wind farms in Mississippi or nuclear plants in Pennsylvania—to minimize carbon footprint and optimize energy sourcing.

Stephen Callahan [13:41]: "Instead of pulling the power from the source and dragging it all the way to Northern Virginia, we're able to put our data centers in those locations."

Innovative Building Materials

AWS actively seeks ways to reduce carbon emissions through sustainable building practices. This includes:

• Concrete Mix Optimization: Replacing a portion of cement with slag, a byproduct of metal refining, to lower CO₂ emissions.

  Stephen Callahan [17:44]: "We reduced the amount of carbon in the cement mix by 35% by substituting 40% of the cement with slag."

• Eliminating Unnecessary Concrete: Removing concrete toppings from mezzanine floors to save on CO₂ without compromising structural integrity.

  Stephen Callahan [25:12]: "We saved 115 metric tons of CO₂ per data center by not putting a concrete topping on the mezzanine floor."

6. Electrical and Mechanical Control Enhancements

To further optimize data center operations, AWS has reimagined traditional electrical and mechanical systems:

• Electrical Distribution Simplification: Reducing connections from transformers to racks from seven to five to decrease complexity and enhance reliability.

  Stephen Callahan [22:50]: "We reduced the number of connections from the transformer to the rack from 7 to 5."

• Decentralized Battery Backups: Moving from centralized UPS rooms to integrating battery backups within individual racks, thereby minimizing blast radius and improving recovery times.

  Stephen Callahan [23:20]: "Now a lot of racks have battery backups in the compute racks themselves... lower likelihood of failure."

7. Philosophy and Mental Models Driving Innovation

Underlying these technical advancements is AWS's core philosophy centered on customer obsession and extreme ownership. This mindset drives teams to continuously reassess and optimize every facet of data center operations, ensuring they anticipate and meet evolving customer needs.

Stephen Callahan [18:31]: "We're trying to obsess about where the customers are going and we're looking to anticipate their needs of the future."

This approach encourages:

• Deep Technical Engagement: AWS engineers, like Callahan, delve into granular details, such as sensor telemetry, to enhance data center performance.

  Stephen Callahan [21:12]: "We have the right number of tools in that tool belt to build the best data center for the scenario that we have."

• Continuous Improvement: Regularly reevaluating past decisions to align with current scale and technological advancements.

  Stephen Callahan [04:14]: "We may reevaluate a decision you made in the past because now suddenly we're 10 or 50 times larger."

8. Real-World Applications and Benefits

The innovations discussed translate into tangible benefits for AWS and its customers:

• Enhanced Reliability: Decentralized systems and optimized electrical controls reduce points of failure and improve uptime.

• Increased Efficiency: Advanced cooling methods and sustainable materials lower operational costs and environmental impact.

• Scalability: Adaptable data center designs facilitate the rapid deployment of AI-focused infrastructures to meet demand spikes.

Stephen Callahan [24:36]: "We have three or four wins... it's worth the benefit or worth the interesting."

Conclusion

Episode #716 of the AWS Podcast offers a comprehensive look into the intricate and innovative efforts behind AWS's data centers. Stephen Callahan's expertise illuminates the complexities of scaling infrastructure for AI workloads, the imperative of sustainability, and the relentless pursuit of operational excellence. Through customer obsession and extreme ownership, AWS continues to redefine what modern data centers can achieve, ensuring they remain at the forefront of technology and environmental stewardship.

Simon Lesh [27:44]: "There are teams of people like Stephen who are obsessed with this stuff... It's been great to hear some of the results of that obsession."

Key Takeaways:

• Adaptability is Crucial: AWS continuously evolves its data center designs to accommodate emerging technologies like Generative AI.

• Sustainability is Integral: Strategic location choices and innovative building materials underpin AWS's commitment to renewable energy and reduced carbon emissions.

• Deep Technical Ownership Drives Success: By owning and optimizing every layer of the data center stack, AWS ensures unparalleled reliability and efficiency for its customers.

• Customer-Centric Philosophy: AWS's decision-making is deeply rooted in anticipating and meeting the future needs of its vast customer base.

For more insights and updates, visit aws.amazon.com/podcast.