NVIDIA AI Podcast: Powering the AI Inference Wave with EPRI's Ben Suter (Ep. 292)

Release Date: March 4, 2026
Guest: Ben Suter (Director of R&D, EPRI)
Host: Noah Kravitz

Episode Overview

This episode explores the transformative role of AI microdata centers in today’s energy landscape. Host Noah Kravitz sits down with Ben Suter, Director of R&D at the Electric Power Research Institute (EPRI), to discuss the interplay between data centers, AI inference, and the power grid. They delve into how distributed, smaller-scale microdata centers might solve power delivery challenges, support the growing demands of AI, and help utilities adapt to an inference-heavy future.

Key Discussion Points & Insights

1. Introducing EPRI and Ben Suter's Background

• EPRI’s Mission: A nonprofit R&D organization supporting more than 400 companies in 40+ countries, focused on ensuring reliable, affordable energy through innovation.
  • Ben: Over 20 years at EPRI, witnessed multiple technology shifts in energy, from solar to EVs to today’s AI boom.

  "I've had... several lifetimes here because I've been here for 20 years and so working through different areas and now kind of in this AI space, which is obviously just accelerated everything about 10x." — Ben Suter (04:03)

2. The Rise of Data Centers: Training vs. Inference

• Data Center "Flavors":
  • Traditional, massive “behemoth” centers power model training (multi-gigawatt, centralized).
  • A coming shift: Inference will outpace training as the primary compute/power demand.
• Staggering Statistic:

  "Only about 20% of [a model’s] compute capacity and thus its power consumption is in the training side. 80% of it is in the inference side." (06:33)

• Inference Boom: As real-world AI applications explode (smart devices, translation, autonomous vehicles), inference must accommodate a broader, more distributed demand.

3. Energy Consumption & Load Implications for Inference

• Load Diversity:
  • Training jobs: Massive, sudden spikes in power demand, highly centralized.
  • Inference: Initially thought to follow user-centric daily patterns, but agents/agentic AI can generate demand around the clock.

    "That's doing all this work at night now... It completely changes the paradigm because now it's running at night while I'm sleeping." (08:56)

• Varied Load Shapes: Different applications (consumer-facing, financial institutions, internal models) create varying energy demand profiles.

4. The Microdata Center Solution

• Why Microdata Centers Now?
  • Distributed, smaller centers are a better fit for inference because of latency needs and proximity to users.

    "Having the huge mega data centers that are centrally located don’t necessarily make sense for the inference data centers because they are more consumer-centric and user-centric." (10:50)

  • Mirrors the evolution of streaming and game servers: localized nodes improve performance for end-users.
• Sizing & Siting:
  • Most substations have 3–10 MW spare capacity (some up to 20 MW seen as a kind of upper boundary).
  • Clustering several microdata centers in a region can aggregate 20–25 MW capacity in a distributed fashion.

    "If we go to a regional area... each data center maybe has 5 megawatts of capacity. Now we've got 5 data centers at 5 megawatts... 25 megawatts of capacity." (17:19)

5. Grid Impacts and Broader Benefits

• Win-Win Potential:
  • Microdata centers can use underutilized substation capacity—no major new infrastructure, faster deployment, reduced costs.

    "If we can get extra usage out of existing assets, then that's sort of a win for everyone... we're not having to put new steel in the ground." (19:00)

• Clean Energy and Flexibility:
  • Possibility of pairing with renewables and storage.
  • Microdata centers can ease grid strain during peak times by redistributing computational loads.

    "If you can engineer it so that you can have flexibility to reduce your load... you actually have a lot more envelope that you could potentially use." (21:12)

6. Real-World and Emerging Applications

• AI-Driven Use Cases:
  • User-driven: translation, smart glasses, autonomous vehicles.
  • Grid-focused: smart lenses for field workers, AI-powered grid diagnostics, improved restoration times, digital twins.

    "Just here at EPRI... can you use smart glasses to analyze your poles and transformers and, and things in a substation and make, you know, your line workers smarter, more efficient and safer all at the same time." (22:37)

• Scaling Up:
  • Industry waiting for large-scale, measurable enterprise AI deployments; proof-of-concept work is ongoing.

    "The thing everybody is waiting for is that scaled demo of where there's this application and it's measurable and we've scaled it out to the entire enterprise." (25:02)

• Historic Data Unlocks: AI can help digitize, organize, and analyze previously inaccessible historical records (e.g., microfiche archives).

  "There was somebody that had access to this huge repository of microfiche... started using the models to ingest it all and just created a monster data set and so cool." (28:03)

7. The Road Ahead: Success & Open Questions

• Hopes for the Next Year or Two:
  • Build and monitor many micro inference data centers, gathering data to inform broader roll-outs.

    "Hopefully in a year or two years we've got a pile of... micro inference data centers built out and we're monitoring and measuring them and that's helping educate us..." (29:06)

• Continued Acceleration:
  • Landscape is evolving rapidly—unexpected breakthroughs (like agentic AI) are reshaping the field almost monthly.

    "Every year there's new things that come out, completely change things... I don't know what it's going to look like, but I'm hopeful and... it's going to be exciting..." (29:40)

Notable Quotes & Memorable Moments

• "I've gotten to have several lifetimes here because I've been here for 20 years... now kind of in this AI space, which is obviously just accelerated everything about 10x." — Ben Suter (04:03)
• "Only about 20% of [a model’s] compute capacity... is in the training side. 80%... is in the inference side." — Ben Suter (06:33)
• "That paradigm [user-driven load curves] is sort of evolving now and I'm having to change my hypothesis... now it's running at night while I'm sleeping." — Ben Suter (08:56)
• "If we can get extra usage out of existing assets, then that's sort of a win for everyone..." — Ben Suter (19:00)
• "Can we make the control center of the future smarter? Right. And get smarter about restoration times and all these different things on and on." — Ben Suter (23:00)
• "[With AI]... you can go back and ingest all that old data... how much hidden information there is in old analog film scans... AI image analysis is able to extract now." — Noah Kravitz (27:50)
• "Hopefully in a year or two years we've got a pile of... micro inference data centers built out and we're monitoring and measuring them..." — Ben Suter (29:06)

Key Segments & Timestamps

• [01:14] — EPRI's mission & Ben's background
• [04:03] — Technological shifts observed in energy R&D at EPRI
• [06:33] — Compute/power split: AI model training vs. inference
• [08:56] — How user agents shift inference energy loads
• [10:50] — Why inference needs distributed, consumer-centric microdata centers
• [13:08] — Technical/design challenges: sizing, geography, grid impact
• [17:19] — Aggregating capacity across multiple substations
• [19:00] — Existing infrastructure leverage, grid benefits
• [21:12] — Flexibility & integrating renewables/storage for peak usage
• [22:37] — Real-time, field-level AI applications and potential
• [25:02] — Enterprise AI impact: proof-of-concept to scaled demo
• [28:03] — AI’s power to digitize historical data (e.g., microfiche story)
• [29:06] — Success metrics for microdata centers in the near future

Further Information

• Learn more about EPRI and their work on epri.com and their LinkedIn page.
• For AI and data center developments in the electric sector, follow EPRI’s updates and NVIDIA’s industry news.