Practical AI Podcast – Autonomous Vehicle Research at Waymo (November 13, 2025)
Episode Overview
In this episode, hosts Daniel Whitenack (CEO at Prediction Guard) and Chris Benson (Principal AI Research Engineer at Lockheed Martin) welcome Drago Engelov, Vice President and Head of AI Foundations at Waymo, to discuss the current state and future direction of autonomous vehicle (AV) research at Waymo. Drago reflects on developments since his last appearance five years ago, delving deep into technical, operational, and societal aspects of AVs, safety, public trust, modeling advances, simulation, and the promise of machine learning for global-scale deployment.
Key Topics and Insights
1. Waymo’s Progress Since 2020
- Expansion of Service: Waymo One, initially launched to the public in Phoenix (2020), has now scaled to five major metropolitan areas: San Francisco, Los Angeles, Phoenix, Atlanta, and Austin.
- Operational Scale: Serving “hundreds of thousands of rides a week to paying customers” ([03:37]), with plans to expand to more cities and possibly internationally.
- Safety Performance: Over 100 million autonomous miles driven; studies show Waymo’s AVs are “five times less likely to get into accidents with critical injuries and over 10…12, potentially less likely to get into collisions or injured pedestrians” compared to human drivers ([03:37]).
- Partnerships: Recent integrations with Uber (Austin, Atlanta), Lyft (Nashville), and DoorDash for delivery experiments.
Quote:
“I think in 25, I would say a lot more people have had and continue having the opportunity to try Waymo. …We are contributing probably, I would like to think, the most advanced version of an embodied physical AI today that you can do without.”
— Drago Engelov ([06:14])
2. City Selection & Operational Design Domain (ODD)
- Combination of Business and Technical Factors: Chosen cities are large, car-centric metros with varied traffic and infrastructure, helping Waymo challenge and refine its system under different driving conditions.
- Incremental Approach: Progression from more controllable environments (suburban Phoenix) to complex, urban scenarios (San Francisco’s fog, hilly streets, pedestrian density).
- Highway and Adverse Conditions: Focus on mastering highways and future expansion into cities with snow and left-side driving (London, Tokyo).
Quote:
“You want to tackle these challenges in some order, not just try to do everything at the same time.”
— Drago Engelov ([10:57])
3. The Autonomous “Driver” System in 2025
- Physical Architecture:
- Sensors: Camera, lidar, radar, microphones (for sirens, external cues).
- Compute: Substantial on-board computing power, more than a smartphone.
- Safety Redundancy: Layers of contingencies and redundant hardware for safety-critical functions (steering, brakes, compute failover).
- All-electric fleet by design.
- System Design Philosophy: Emphasis on robustness, redundancy, and environmental sustainability.
Quote:
“It’s a robot designed for safe transportation from the ground up…they need redundancy and robustness to make sure that if any system goes wrong…you have contingencies.”
— Drago Engelov ([12:18])
4. Building Public Trust in AVs
- Challenge: Statistics alone don’t shift public perception—actual user experience is vital.
- Conversion Through Experience: Most people quickly adapt and feel safe after an initial ride.
- Transparency: Commitment to publishing safety data and incident reports, third-party validation (insurance studies), and proactive engagement with city authorities and community groups.
Quote:
“People do not feel statistics…What people feel is when they get into the vehicles…My mother-in-law sat in it just a few weeks ago…‘This car drives much better than me.’”
— Drago Engelov ([14:54])
5. Modern AV Stack: Models, Architectures, and AI
- Perception & Decision Pipeline:
- Perceive environment → world representation → behavior prediction/planning → action selection.
- Evolving Model Structure: From modular ML to fewer, larger models; offboard training (foundation and vision-language models); on-car deployment optimized for compute/latency.
- Emphasis on Vision-Language-Action Models: Combines sensors (including modalities like lidar/radar) into “foundation models” that are increasingly multi-modal and capable.
- Safety Harness: Real-world deployment is guarded by systems to detect and constrain hallucinations or out-of-domain model behavior.
Quote:
“Generally the trend has been to have few, and in some cases people claim they have one, large AI models on the car…We also have some notion of, as you know, VLMs also have this weakness of hallucination. So we have the safety harness around them…”
— Drago Engelov ([22:39], [25:22])
6. Simulation, Validation, and the Representation Challenge
- Dual Problems in AV R&D:
- Building the onboard driver (model itself).
- Testing and validating it with exhaustive realism.
- Unique AV Simulation Needs:
- Real-time, multi-modal inputs (billions of sensor readings/second).
- Interaction with diverse agents (humans, other AVs).
- Must cover rare events that real-world driving alone cannot expose.
- Covariate shift: Models encounter scenarios not seen in training; must generalize.
- Emerging Solutions: Use of LLM-inspired action prediction, advanced simulators (GENIE model for controllable videos), and creative domain adaptation techniques.
Quote:
“Simply having drivers on the road is not a particularly scalable solution if you want to keep iterating on your stack because some events happen once in a million miles or more and you would much rather test them in the simulator.”
— Drago Engelov ([32:49])
7. Fast Progress and Open Challenges in AV Research
- Machine Learning Power-Up: “Every couple years our capabilities with AI and machine learning dramatically expand” ([36:59]).
- Scaling Laws in AVs: Parallels with language models but unique challenges (motion tokens less diverse, requiring more data for same parameter sizes).
- Fusing Modalities: How to elegantly combine vision, language, lidar, radar, and actions.
- Simulator Realism and Efficiency: “How do we get the maximum possible simulator realism and how do we get the maximal possible scalable simulator?” ([48:51])
- Modeling System 1 (fast, reflexive) vs System 2 (slow, reasoned) behaviors: Tradeoffs for latency and reasoning.
- Hallucination/Failure Detection: Active area of research—detecting out-of-domain circumstances and failures; importance for safety-critical deployment.
8. Swarming, Coordination, and Collective Intelligence
- Multi-Agent Coordination: Interesting but not at the core of current operations; more relevant in vehicle management (e.g., charging logistics), with potential future impact on traffic flow.
- Research Outlook: As fleet scale increases, inter-vehicle collaboration may become a larger focus.
Quote:
“Controlling jointly vehicles starts becoming interesting now that we’re getting to some kind of scale…when you want to charge them…that’s one example of where you’re fairly swarmed…”
— Drago Engelov ([44:03])
9. Looking Ahead: Future Challenges & Opportunities
- Waymo’s Mandate: The proven improvement in road safety creates an imperative—and an opportunity—to expand.
- Scaling with AI: Core excitement is in leveraging the latest ML advances to speed and scale the deployment of safe AVs worldwide.
- Research Frontiers:
- Expanding VLMs to new modalities (e.g., lidar, radar).
- Building ever-more realistic and scalable simulators.
- Efficient, cost-effective simulation for millions of virtual miles per day.
- Sharing progress with the community (Waymo’s research portal publishing ~100 papers since 2019).
Quote:
“Even just on the US roads alone…40,000 people die every year from accidents. That’s a lot. And I think these gains are starting to become somewhat meaningful. So it starts becoming thinking, hey, maybe we have a mandate to expand…”
— Drago Engelov ([46:11])
Notable Quotes & Memorable Moments
-
On Public Trust:
“People do not feel statistics…what people feel is when they get into the vehicles…”
— Drago Engelov ([14:50]) -
On Simulation:
“Simply having drivers on the road is not a particularly scalable solution…some events happen once in a million miles or more and you would much rather test them in the simulator.”
— Drago Engelov ([32:49]) -
On AV System Architecture:
“It’s a robot designed for safe transportation from the ground up…need redundancy and robustness…”
— Drago Engelov ([12:18]) -
On Progress:
“Every couple years our capabilities with AI and machine learning dramatically expand…this innovation train has not stopped.”
— Drago Engelov ([37:04])
Key Timestamps
- 03:37 – Waymo’s expansion, safety data, partnerships.
- 07:58 – Technical/business reasoning for city selection.
- 12:03 – High-level “robot on wheels” system breakdown.
- 14:50 – Building public trust; why experience > statistics.
- 21:35 – Model architecture: perception, planning, decision; foundation models.
- 27:43 – The unique challenge of simulation in AVs; need to test rare events.
- 36:59 – Advances in machine learning for AVs; challenges and scaling.
- 43:28 – Multi-vehicle swarming, coordination, information sharing.
- 46:01 – The future: mandate for expansion, AI as the core enabler.
- 50:42 – Waymo’s research portal and open science.
Resources & Further Exploration
- Waymo Research Portal & Open Data Set ([35:07])
- Papers: MotionLM, scaling law studies, VLM fine-tuning (“EMMA”), Genie model by Google.
Closing Thoughts
This episode is a rich, deeply technical yet accessible exploration of AV research led by Waymo—a company at the leading edge of robotics and AI deployment in the real world. Drago Engelov highlights the scope and depth of advances in AI and simulation, the operational and social realities of how Waymo selects and deploys in cities, and the multifaceted, ongoing challenge of realizing safe, trustworthy, and scalable autonomy for society.
“Let's not make it five years next time. We'll try to get you on and hear the update sooner than that for sure.”
— Chris Benson ([51:06])