Ion Stoica has done what almost no academic ever does — repeatedly turned university research into billion-dollar companies. He co-founded Databricks (now valued at over $100 billion), Anyscale, Arena AI and Conviva, while his Berkeley lab produced the open source projects the entire AI industry runs on: Ray, vLLM, and SGLang.

In this episode, we ask him how it's actually done. His answer is surprisingly unromantic: solve a problem people already care about, build an artifact good enough that they adopt it, and pay attention to the moment users start asking "who maintains this after the students graduate?" - that's when a project becomes a company. He's also insistent that the credit belongs to his students.

From there, the conversation goes deep into what he's watching now: why the AI stack has become an order of magnitude more complex than the Hadoop/Spark era, why maximizing GPU utilization is "the name of the game" for any enterprise, and why coding agents will struggle with distributed systems long after they've mastered web apps. He shares a memorable reward-hacking story — a load balancer that maximized throughput by dropping requests — explains why the gap between open and closed models sits at about six months, and closes with his case for regulating AI by outcomes, not capabilities.

Timeline

00:00 — Introduction: welcoming Ion Stoica
01:21 — The playbook: how research projects become companies
05:22 — Will vLLM and SGLang stay open source?
07:47 — The real bottleneck in the AI stack: complexity, not just hardware
14:31 — Should algorithms follow infrastructure, or the other way around?
16:13 — Can AI coding tools write distributed systems and GPU kernels?
21:09 — Verifiers, harnesses, and the limits of outsourcing understanding
25:41 — Reward hacking: the load balancer that dropped requests
25:58 — How should enterprises consume GPUs? Utilization as the name of the game
30:23 — GPU scarcity: will the compute crunch ever end?
35:27 — Hyper-optimization and the risk of locking in today's architectures
37:17 — Open vs. closed models: why every company wants to own the stack
40:35 — The six-month gap, and the rising cost of training frontier models
43:58 — Kimi, Qwen, and who's incentivized to keep open models alive
45:39 — Regulation: outcomes, not capabilities
47:41 — Self-regulation, concentration of power, and auditing open models
48:32 — Wrap-up

Music

"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About

The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.