The Lawfare Podcast: Scaling Laws – AI and Energy: What Do We Know? What Are We Learning?

Date: October 17, 2025
Host: Kevin Fraser (AI Innovation and Law Fellow, Texas Law; Senior Editor, Lawfare)
Guests:

• Musharaf Chaudhary (Associate Professor, University of Michigan; Director, ML Energy Lab)
• Dan Zhao (AI Researcher, MIT, Google X, Microsoft; AI for Science, Sustainability & Energy Efficiency)

Episode Overview

This episode explores AI’s rapidly increasing impacts on global energy and resource consumption. The hosts and expert guests discuss public misconceptions about AI’s energy use, clarify the distinction between training and inference, and cover efforts in research, industry, and policy for creating more energy-efficient AI systems. They consider whether rising AI-related energy demands should cause alarm, debate the limits of individual impact, and envision the future of energy use in AI as both a technical and policy challenge.

Key Discussion Points & Insights

1. Common Misunderstandings about AI’s Energy Consumption

• News Hype vs. Measurement Reality
  • Musharaf Chaudhary (04:03): Early reporting exaggerated AI energy use due to poor measurement tools:

    “At the beginning there were no good tools to precisely measure... so people were using estimations to get a sense of rough order of magnitude… multiplying all these big numbers… you end up with a very large number, which have been reported [as] bigger than Netherlands and Ireland... There is a lot of overestimation.”

  • Sensational stats (“AI uses more energy than Ireland”; “as much power as 100,000 homes”) make headlines but are often misleading.
• Distinction Between Training and Inference
  • Kevin Fraser (05:22): These viral comparisons overlook different energy uses between training (building a model) and inference (serving user requests).

2. What Drives AI’s Energy Use? Training vs. Inference

• Training Requires Intensive One-Time Computation
  • Dan Zhao (06:16): Training entails processing huge datasets once, using powerful hardware in large data centers.

    “Large behemoth models… undergo something called pre-training… not accessible to your common folk… large companies, large labs, basically training these models on tons and tons of tokens.”

• Inference: Small per Use, Massive in Aggregate
  • Musharaf Chaudhary (09:12):

    “Training… happens only once… when it gets deployed… hundreds of millions of people are using the same model… each inference request consumes a small amount of energy, but when you multiply it by hundreds of million, they add up to very big number… the ratio between energy consumption of training and inference depends… but many of the numbers… ranges from 30/70 and 40/60… the bigger one being inference.”

  • Result: As LLMs reach hundreds of millions of users, total inference energy often surpasses that from training.
  • Implication: Growth of AI agents (autonomous, multi-step tools) means inference energy needs will skyrocket.

3. What’s Actually Happening in Data Centers?

• Why AI Uses So Much Power and Water

  • Dan Zhao (12:19):

    “You walk in [a supercomputing center], and the first thing you notice, it’s quite hot… GPUs go brrr...even when not active, transistors have static power leakage… not many people know how to efficiently get every single drop of efficiency out of GPU usage.”

  • Keeping GPUs busy is incentivized both economically and technically, but many (especially new users/startups) use resources inefficiently.
  • Analogy: Like novice vs. professional farmers using equipment (16:18–16:44).

• “Tragedy of the Commons” Dynamic

  • Limiting GPU power can save energy, but demand often fills saved capacity:

    “If [users] realize we’ve capped their power… they might still order more jobs… energy you spare might still get used up anyway.” (14:46)

• Growing Modalities & Complexity

  • New AI capabilities (e.g., video generation, multimodal agents) further increase energy needs, often in ways not originally anticipated (17:43–19:09).

4. Measuring and Improving AI Energy Efficiency

• Filling the Data Gaps

  • Musharaf Chaudhary (20:10): His team created ZEUS, a tool for precise measurement of AI energy use at the hardware level, leading to a “ML Energy Leaderboard” for open source models:

    “We have built… a tool that automatically finds this critical path and precisely computes how to slow down everybody outside the critical path… [saving] up to 20, 30% of energy consumption of training.” (23:30)

  • Optimization: ZEUS can not just measure but also optimize usage, especially in distributed (multi-GPU) settings.
  • Industry Impact: Big labs reference ZEUS results in their own reporting (e.g., Google).

• Transparency & Closed vs. Open Models

  • Measurement is much easier for open-source models; for major closed models (e.g., ChatGPT), researchers rely on proxies (parameters, observed emissions, etc.) but these are imperfect (30:40–33:47).
  • Dan Zhao:

    “Only really the large labs themselves, when they do it on their own hardware, on their own network… will they know the precise numbers.”

5. Individual vs. Systemic Energy Impact

• Is Not Using AI the Solution?
  • Dan Zhao (38:39):

    “If you won’t submit that single query, someone else will… as an individual submitting a query… it’s a negligible difference. It’s a tragedy of the commons and a coordination issue…”

  • Individual restraint matters little; coordinated action or policy makes a difference.
• Public Communication Challenges
  • Attempting to relate AI energy use to familiar benchmarks (e.g., “like running a microwave for a second”) is helpful but usually trivializes the issue.

    “People want something that people can relate to… but really, each individual query is so small…” (40:55, Musharaf Chaudhary)

6. Bottlenecks & What Holds Progress Back

• Major Barriers:
  • Information Asymmetry: Not everyone understands or implements energy-saving techniques (16:44, 44:24).
  • Public Education Needed:
    • Dan Zhao (44:24):

      “Having people understand what a GPU does… making based on that understanding… they’re going to get more efficiency… but it takes that fixed cost to overcome for people to learn because it’s not easy.”

    • Many researchers or startups prefer to chase “sexy” improvements (new models, state-of-the-art results) rather than deep efficiency work.
• Resource Bottlenecks:
  • Lack of access to closed-model usage data makes benchmarking and research harder.

7. The Future: Optimism & Policy

• Reasons for Optimism
  • Musharaf Chaudhary (47:59):

    “I have this vision… energy-optimal AI… AI is going to happen. What we want to do is to… get to the same level of accuracy… but with minimum energy… At every layer—model, software, hardware—innovation is happening to make it more efficient.”

  • As AI is democratized and commoditized, efficiency and cost-effectiveness will increase.
  • Upgrades will keep energy-per-unit-of-AI-task dropping even as aggregate demand rises.
• Room for Policy, but Context Needed
  • Dan Zhao (51:53):

    “Innovation is still going to be key… we probably don’t want a one-strategy-fits-all. [Labs use] different model components, architectures, constraints… When it comes to state legislators, what’s in their own backyard… might be most important… and that understanding might be lacking right now.”

  • Cautious about blanket mandates; nuanced, targeted, and expert-informed policy guidance is best.
  • Universal change is more likely by influencing human behaviors and incentives (e.g., behaviors during ML conference deadlines), not just technical mandates.

Notable Quotes & Memorable Moments (w/ Timestamps)

• On Overblown Energy Stats

  “It led to many news articles which are... [understandably concerning].”
  – Musharaf Chaudhary (04:03)

• On the Inference Boom

  “As more and more people use it, [training energy] easily gets dwarfed by all the people and all of their requests.”
  – Musharaf Chaudhary (09:12)

• On Inefficiency in AI Use

  “It’s in some form a tragedy of the commons… even if you efficiently allocate, demand will fill the vacuum.”
  – Dan Zhao (14:46)

• On Benchmarking Efforts

  “We have built… a tool that automatically finds this critical path… and that allows us to save up to 20-30% of energy consumption of training.”
  – Musharaf Chaudhary (23:30)

• On Individual Impact

  “If you won’t submit that single query, someone else will. It’s a negligible difference… It’s the tragedy of the commons and the coordination issue that comes up in aggregate…”
  – Dan Zhao (38:39)

• On the Communication Challenge

  “People want something physical so they can relate to… but none of [the analogies] seem perfect.”
  – Musharaf Chaudhary (40:55)

• On the Path Forward

  “For the fixed amount of AI that we want, the energy cost I think will keep going down.”
  – Musharaf Chaudhary (47:59)

• On Policy & Human Behavior

  “If we can somehow induce human behavior to change… I’m probably happy with that.”
  – Dan Zhao (54:39)

Timestamps for Key Segments

• [03:49] – Guests introduced; framing AI’s energy question
• [04:03] – Myths/truths about AI’s total energy use (Chaudhary)
• [06:16] – Training vs. inference explained (Zhao)
• [09:12] – Why inference now dominates energy usage (Chaudhary)
• [12:19] – Data center resource use, inefficiencies, and analogies (Zhao, Fraser)
• [19:09] – Energy impact of agents, multimodal models grows
• [20:10] – New tools for precise measurement and optimization (ZEUS) (Chaudhary)
• [30:40] – Challenges benchmarking closed vs. open models (Zhao)
• [35:15] – Guests’ own AI use and thoughts on individual impact
• [38:39] – Why individual energy restraint is limited in effect (Zhao)
• [44:24] – Education, technical/informational barriers, cultural factors (Zhao)
• [47:59] – Optimism for energy-efficient AI future (Chaudhary)
• [51:53] – The case for policy nuance, innovation, and context (Zhao)

Conclusion

This episode dismantles headline-grabbing myths about AI’s energy demands and replaces them with data-driven perspective. Training does consume enormous resources, but as AI applications proliferate, routine inference by millions worldwide is the larger, growing factor in total consumption. Technical innovation—across hardware, algorithms, and measurement—shows promise for major efficiency gains. Yet, individual restraint is negligible in the tragedy-of-the-commons dynamic: Large-scale progress will hinge on transparency, industry incentives, smart policy, and better public/technical education around the true costs and tradeoffs of AI adoption.

Host Contact: scalinglawsawfairmedia.org
Subscribe: Find Scaling Laws wherever you get your podcasts.