Podcast Summary: Scrum Master Toolbox Podcast

Episode: Quality 5.0—Quantifying the "Unmeasurable"
Guests: Tom Gilb, Simon Holzapfel
Host: Vasco Duarte
Date: December 11, 2025

Overview

In this fourth episode of a five-part series on building and evolving engineering organizations, Vasco Duarte hosts Tom Gilb and Simon Holzapfel to explore the concept of “Quality 5.0”—with a focus on how to quantify the seemingly unmeasurable qualities in software and organizations. The conversation dives into practical strategies for clarifying, measuring, and improving critical attributes like security, usability, and trust, drawing from Gilb’s influential “evo” process. The discussion weaves together methodology, human factors, and systemic trust, delivering actionable insights for Scrum Masters and Agile leaders.

Key Discussion Points & Insights

1. The Essence of Quality: Clarification and Quantification

Timestamp: 02:06–04:58

• Tom Gilb emphasizes that while quantification is essential, clarification is the core objective—making sure all stakeholders have a shared understanding.
  • “Quantification is not the main idea, but it's a key idea. The key idea is clarification so that the executive team understands each other and the board understands the executive team, things like that.” (Tom, 02:06)
• Many organizations, even prominent ones, lag in quantifying values, clinging to outdated methods.
  • “I noticed something. They had no idea of how to quantify their values. They're 50 years out of date…these are the advisors to the U.S. department of Defense. So I'm shocked.” (Tom, 02:24)

2. Defining Scales of Measure

Timestamp: 03:35–06:57

• The first crucial step is to define a scale of measure for each critical variable (e.g., “security,” “flexibility”).
  • Compare this to measuring speed in MPH or km/h; the act of defining the scale contextualizes and operationalizes what matters.
  • “When you define your scale of measure, you have chosen what you're going to work with and live with and what is important…there are usually at least 100 different options.” (Tom, 04:10)
• Many attributes (usability, security) often require multiple scales (sometimes 10+), not just one.
  • “Usability in practice is 12 different scales of measure…I’ll just say, maybe it is as simple as one scale of measure. Things like security, all the important things probably are decomposable into order of magnitude 10 different aspects.” (Tom, 04:58)

3. Establishing Metrics: Past, Tolerable, and Target Levels

Timestamp: 06:39–07:23

• Set three critical metrics for each scale:
  1. Past/Status Level: Where are we now?
  2. Tolerable Level: What's the minimum acceptable threshold?
  3. Target/Goal Level: What does success look like numerically?
  • “The first number is…Where are we? …Next, what’s the minimum we have to get to to survive? …Then we set a third number…I call it the target or goal level. That’s where we have to be to succeed.” (Tom, 06:39–07:23)

4. Measurement Process vs. Testing

Timestamp: 07:41–08:24

• Measurement is about continual assessment—not just “passed/failed” as in traditional testing.
  • “A measurement is a form of test, but it's essentially different from saying, is there a bug or not kind of test process.” (Tom, 07:55)
• Continuous measurement allows for faster feedback, learning, and adaptation.
  • “We need a measurement process early and frequently so we can learn where we got it wrong…pivot or adjust very rapidly. We don't waste time going down a bad rabbit hole.” (Tom, 08:10)

5. The Role of Trust in Organizational Learning and Change

Timestamp: 09:44–12:01

• Simon Holzapfel introduces trust as the “human growth hormone” for effective feedback loops and system learning.
  • “I think about what is…the human growth hormone to this change…that hormone is trust.” (Simon, 09:44)
  • “My wife often talks about any organizational change moving at the speed of trust.” (Simon, 10:02)
• In low-trust environments, information is hoarded; in high-trust environments, information flows freely and learning accelerates.
  • “Once there's trust, information flows…Until there's trust, you have the Soviet problem.” (Simon, 10:18)

6. Practical Actions for Middle Managers

Timestamp: 10:27–11:44

• For managers, trust is built through visibility, alignment, and learning cycles.

  • “What I literally do tomorrow is make the work visible to my manager to alignment check with my manager first, then go do something and then show them the result. If I live as the learning cycle myself…It'll be awesome. Until that exists, I'm withholding.” (Simon, 11:11)

• Delivering incremental value builds trust and amplifies impact.

  • “If you manage in fact, to deliver increased critical value every week, you will build trust.” (Tom, 11:44)

7. Trust as a Systemic Quality

Timestamp: 12:01–14:22

• Trust acts like a chemical property—it decays over time if not replenished through positive interaction.

  • “The Longer you go without talking to someone, the lower the trust becomes…trust is a little bit like that. It's like temperature. Like when you get away from the heat, you lose heat.” (Vasco, 12:01–12:39)

• Building trust through the “evo” process (early, frequent involvement and feedback) is crucial—and must start immediately.

  • “If we do it early enough, we need to start doing it today. Not tomorrow…Because the trust is something that we build slowly. We might lose it quickly, but we build very slowly.” (Vasco, 12:57)

• Simon proposes a unique metric: net team age (time since last personnel change) as a macro measure of organizational stability—and chaos.

  • “What is the net age of your teams? In other words, what is the time since last personnel change? …In most organizations, it's going to be less than three years. Think of a three year old. And we wonder why we feel like we live in chaos.” (Simon, 13:31–13:49)

Notable Quotes & Memorable Moments

• On the importance of clarification:
  “Quantification is not the main idea, but it's a key idea. The key idea is clarification so that the executive team understands each other…”
  — Tom Gilb (02:06)

• On outdated practices:
  “I noticed something. They had no idea of how to quantify their values. They're 50 years out of date.”
  — Tom Gilb (02:24)

• On measurement vs. testing:
  “A measurement is a form of test, but it's essentially different from saying, is there a bug or not kind of test process.”
  — Tom Gilb (07:55)

• On trust as a systemic enabler:
  “My wife often talks about any organizational change moving at the speed of trust.”
  — Simon Holzapfel (10:02)

• On the practical path to building trust:
  “If you manage in fact, to deliver increased critical value every week, you will build trust.”
  — Tom Gilb (11:44)

• On trust decay:
  “The longer you go without talking to someone, the lower the trust becomes…trust decays fast with time.”
  — Vasco Duarte (12:01, 14:06)

Important Timestamps

• [02:06] – Tom Gilb on quantification vs. clarification
• [04:10] – Defining a scale of measure
• [06:39] – Setting past, tolerable, and target levels
• [07:55] – Measurement process and its distinction from testing
• [09:44] – Simon on trust as the “human growth hormone” for change
• [11:11] – Simon: How a middle manager can start building trust and visibility
• [13:31] – Metric for team/organizational chaos: net team age

Conclusion & Teaser

The episode clarifies how “unmeasurable” aspects of quality can (and must) be made measurable, and how trust catalyzes learning and change within organizations. By combining measurement, visibility, and trust, Agile teams and leaders can consistently navigate complexity and drive improvement. The conversation sets the stage for the final episode, where these threads will be woven together into a practical system for organizational learning.

For listeners hungry for actionable, science-backed Agile practice and organization evolution, this episode delivers both principle and technique in a lively, experienced tone—making “Quality 5.0” a tangible goal, not a buzzword.