Every capability in an agent needs its own evidence and release bar. A model-provider slip, an incorrect tool call, and a wrong fertility-benefits answer should not be held to the same pass rate.

William Horton, Staff AI Engineer at Maven Clinic, joined us the day after Maven Assistant reached its first external users. The agent helps members inside Maven Clinic’s women’s and family healthcare platform find providers, manage appointments, navigate Maven, and get basic health information. William had spent much of launch day reading chat traces and turning the surprises into product decisions and tests.

William shows how a production failure moves through Maven’s system: the trace becomes a regression case, code handles deterministic checks, and LLM judges cover behavior that cannot be reduced to exact outputs. Human labels calibrate those judges, while the consequence of a wrong answer determines whether the capability ships. You can apply the same release workflow to the agent you are building now.

“For a lot of our tool-call evaluation, I’ll accept that it runs ten times and passes nine times. Going for that ten out of ten is just not worth the effort.”

— William Horton, Staff AI Engineer, Maven Clinic

You can also find the full episode on Spotify, Apple Podcasts, and YouTube.

👉 Want to build agents from the ground up? Registration is open for Build AI Agents from First Principles, a live workshop on the loops, tools, context, harnesses, and engineering decisions behind useful AI agents. You'll learn how to design agent systems from first principles, with enough structure to decide which harness patterns your product actually needs. Sign up today with vg-code for 10% off 👈

In This Episode

* The architecture behind Maven Assistant. A stronger lead agent routes requests to four narrower specialists for appointments, provider search, health questions, and Maven support. Hard guardrails run before dynamic routing.

* Why an enterprise healthcare assistant only needs 15 to 20 tools. Maven divides a manageable toolset across its specialists instead of exposing one model to hundreds of choices. Existing APIs become safer agent tools, with user identity and other application state injected by code.

* Turn failures into the cheapest reliable eval. A response claiming the agent was “made by Google” became a string check, tool calls are verified deterministically, and LLM judges handle clinical accuracy and other qualitative behavior.

* Set release thresholds from the consequences. Nine passes in ten can be acceptable for a cheap failure. Maven withheld benefits answers that could influence tens of thousands of dollars and routes self-harm language directly to human support.

* Let production change the product and the test set. Early chats changed the roadmap, became regression cases, exposed weaknesses in the judges, and supplied realistic opening messages for simulated users.

Join the Four-Month Follow-Up

This episode was recorded live inside our Building AI Applications course the day after Maven Assistant reached its first external users. By the follow-up four months later, Maven will have a much larger body of real conversations. William will return to compare the launch assumptions with what members actually used, which evals changed, and how newer models altered the system.

Resources

* Maven Clinic

* Maven introduces Maven Intelligence

* Google Agent Development Kit

How You Can Support Vanishing Gradients

Vanishing Gradients is a podcast, workshop series, blog, and newsletter focused on what you can build with AI right now. Over 70 episodes with expert practitioners from Google DeepMind, Netflix, Stanford, and elsewhere. Hundreds of hours of free, hands-on workshops. All independent, all free.

If you want to help keep it going:

* Become a paid subscriber, from $8/month

* Share this with a builder who’d find it useful

* Subscribe to our YouTube channel

* Join one of our other workshops here

Get full access to Vanishing Gradients at hugobowne.substack.com/subscribe