Quality 5.0—Quantifying the "Unmeasurable" With Tom Gilb and Simon Holzapfel

Summary

Podcast Summary: Scrum Master Toolbox Podcast

Episode: Quality 5.0—Quantifying the "Unmeasurable"
Guests: Tom Gilb, Simon Holzapfel
Host: Vasco Duarte
Date: December 11, 2025

---

Overview

In this fourth episode of a five-part series on building and evolving engineering organizations, Vasco Duarte hosts Tom Gilb and Simon Holzapfel to explore the concept of “Quality 5.0”—with a focus on how to quantify the seemingly unmeasurable qualities in software and organizations. The conversation dives into practical strategies for clarifying, measuring, and improving critical attributes like security, usability, and trust, drawing from Gilb’s influential “evo” process. The discussion weaves together methodology, human factors, and systemic trust, delivering actionable insights for Scrum Masters and Agile leaders.

---

Key Discussion Points & Insights

1. The Essence of Quality: Clarification and Quantification

Timestamp: 02:06–04:58

• Tom Gilb emphasizes that while quantification is essential, clarification is the core objective—making sure all stakeholders have a shared understanding.
  • “Quantification is not the main idea, but it's a key idea. The key idea is clarification so that the executive team understands each other and the board understands the executive team, things like that.” (Tom, 02:06)
• Many organizations, even prominent ones, lag in quantifying values, clinging to outdated methods.
  • “I noticed something. They had no idea of how to quantify their values. They're 50 years out of date…these are the advisors to the U.S. department of Defense. So I'm shocked.” (Tom, 02:24)

2. Defining Scales of Measure

Timestamp: 03:35–06:57

• The first crucial step is to define a scale of measure for each critical variable (e.g., “security,” “flexibility”).
  • Compare this to measuring speed in MPH or km/h; the act of defining the scale contextualizes and operationalizes what matters.
  • “When you define your scale of measure, you have chosen what you're going to work with and live with and what is important…there are usually at least 100 different options.” (Tom, 04:10)
• Many attributes (usability, security) often require multiple scales (sometimes 10+), not just one.
  • “Usability in practice is 12 different scales of measure…I’ll just say, maybe it is as simple as one scale of measure. Things like security, all the important things probably are decomposable into order of magnitude 10 different aspects.” (Tom, 04:58)

3. Establishing Metrics: Past, Tolerable, and Target Levels

Timestamp: 06:39–07:23

• Set three critical metrics for each scale:
  1. Past/Status Level: Where are we now?
  2. Tolerable Level: What's the minimum acceptable threshold?
  3. Target/Goal Level: What does success look like numerically?
  • “The first number is…Where are we? …Next, what’s the minimum we have to get to to survive? …Then we set a third number…I call it the target or goal level. That’s where we have to be to succeed.” (Tom, 06:39–07:23)

4. Measurement Process vs. Testing

Timestamp: 07:41–08:24

• Measurement is about continual assessment—not just “passed/failed” as in traditional testing.
  • “A measurement is a form of test, but it's essentially different from saying, is there a bug or not kind of test process.” (Tom, 07:55)
• Continuous measurement allows for faster feedback, learning, and adaptation.
  • “We need a measurement process early and frequently so we can learn where we got it wrong…pivot or adjust very rapidly. We don't waste time going down a bad rabbit hole.” (Tom, 08:10)

5. The Role of Trust in Organizational Learning and Change

Timestamp: 09:44–12:01

• Simon Holzapfel introduces trust as the “human growth hormone” for effective feedback loops and system learning.
  • “I think about what is…the human growth hormone to this change…that hormone is trust.” (Simon, 09:44)
  • “My wife often talks about any organizational change moving at the speed of trust.” (Simon, 10:02)
• In low-trust environments, information is hoarded; in high-trust environments, information flows freely and learning accelerates.
  • “Once there's trust, information flows…Until there's trust, you have the Soviet problem.” (Simon, 10:18)

6. Practical Actions for Middle Managers

Timestamp: 10:27–11:44

• For managers, trust is built through visibility, alignment, and learning cycles.

  • “What I literally do tomorrow is make the work visible to my manager to alignment check with my manager first, then go do something and then show them the result. If I live as the learning cycle myself…It'll be awesome. Until that exists, I'm withholding.” (Simon, 11:11)

• Delivering incremental value builds trust and amplifies impact.

  • “If you manage in fact, to deliver increased critical value every week, you will build trust.” (Tom, 11:44)

7. Trust as a Systemic Quality

Timestamp: 12:01–14:22

• Trust acts like a chemical property—it decays over time if not replenished through positive interaction.

  • “The Longer you go without talking to someone, the lower the trust becomes…trust is a little bit like that. It's like temperature. Like when you get away from the heat, you lose heat.” (Vasco, 12:01–12:39)

• Building trust through the “evo” process (early, frequent involvement and feedback) is crucial—and must start immediately.

  • “If we do it early enough, we need to start doing it today. Not tomorrow…Because the trust is something that we build slowly. We might lose it quickly, but we build very slowly.” (Vasco, 12:57)

• Simon proposes a unique metric: net team age (time since last personnel change) as a macro measure of organizational stability—and chaos.

  • “What is the net age of your teams? In other words, what is the time since last personnel change? …In most organizations, it's going to be less than three years. Think of a three year old. And we wonder why we feel like we live in chaos.” (Simon, 13:31–13:49)

---

Notable Quotes & Memorable Moments

• On the importance of clarification:
  “Quantification is not the main idea, but it's a key idea. The key idea is clarification so that the executive team understands each other…”
  — Tom Gilb (02:06)

• On outdated practices:
  “I noticed something. They had no idea of how to quantify their values. They're 50 years out of date.”
  — Tom Gilb (02:24)

• On measurement vs. testing:
  “A measurement is a form of test, but it's essentially different from saying, is there a bug or not kind of test process.”
  — Tom Gilb (07:55)

• On trust as a systemic enabler:
  “My wife often talks about any organizational change moving at the speed of trust.”
  — Simon Holzapfel (10:02)

• On the practical path to building trust:
  “If you manage in fact, to deliver increased critical value every week, you will build trust.”
  — Tom Gilb (11:44)

• On trust decay:
  “The longer you go without talking to someone, the lower the trust becomes…trust decays fast with time.”
  — Vasco Duarte (12:01, 14:06)

---

Important Timestamps

• [02:06] – Tom Gilb on quantification vs. clarification
• [04:10] – Defining a scale of measure
• [06:39] – Setting past, tolerable, and target levels
• [07:55] – Measurement process and its distinction from testing
• [09:44] – Simon on trust as the “human growth hormone” for change
• [11:11] – Simon: How a middle manager can start building trust and visibility
• [13:31] – Metric for team/organizational chaos: net team age

---

Conclusion & Teaser

The episode clarifies how “unmeasurable” aspects of quality can (and must) be made measurable, and how trust catalyzes learning and change within organizations. By combining measurement, visibility, and trust, Agile teams and leaders can consistently navigate complexity and drive improvement. The conversation sets the stage for the final episode, where these threads will be woven together into a practical system for organizational learning.

---

For listeners hungry for actionable, science-backed Agile practice and organization evolution, this episode delivers both principle and technique in a lively, experienced tone—making “Quality 5.0” a tangible goal, not a buzzword.

Transcript

[A]

Hey there, agile adventurer, just a quick question. What if, for the price of a fancy coffee or half a pizza, you could unlock over 700 hours of the best agile content on the planet? That's audio, video, E courses, books, presentations, all that you can think of. But you can also join live calls with world class practitioners and hang out in a flame war free and AI slop clean slack with the sharpest minds in the game. Oh, and yes, you get direct access to me, Vasko, your Scrum Master Toolbox podcast. No, this is not a drill. It's this Scrum Master Toolbox membership. And it's your unfair advantage in the agile world. So if you want to know more, go check out scrummastertoolbox.org membership. That's scrummastertoolbox.org Membership. And check out all the goodies we have for you. Do it now. But if you're not doing it now, let's listen to the podcast.

Welcome back everybody. This is our fourth of our five part series episodes on building and evolving engineering organizations. And this week we have with us Tom Gilp and Simon Holzapfel. Simon and Tom, welcome back.

[B]

Thanks so much.

[A]

So we're building on what we discussed yesterday. Of course, yesterday we talked a lot about testing and of course we're testing something. There is a process to this that Tom describes in his books about evo. So we've talked a lot about what quality is. Tom, you introduced right off the bat when we started this week, the idea that we need to quantify attributes. So let's dive into that a little bit more, give us a couple of examples and how do we help our listeners to really grasp and apply this idea that quality is about quantified attributes?

[C]

Okay, so now quantification is not the main idea, but it's a key idea. The key idea is clarification so that the executive team understands each other and the board understands the executive team, things like that. And there are many tactics for clarification beyond quantification. Okay, but let's focus on quantification. Okay, Now I was listening last night to a Software Engineering Institute lecture on better requirements, and I noticed something. They had no idea of how to quantify their values. They're 50 years out of date. I've taken action, I've written to them, but these are the advisors to the U.S. department of Defense. So I'm shocked. Okay, so there is a big cultural problem. Now let me cut to the chase. There's an incredibly simple idea. When you have a critical variable like you want Better security or better flexibility. Just to take two simple examples. Your stage one with me and my clients, within five or ten minutes is can we define a scale of measure?

If you, if you say, no, no, that's fluffy and you know, it's soft and you can't do that, well, then I already said, you know, one tactic is ask AI It'll tell you, okay, but now defining a scale of measure is like saying, I want a very fast car and say, well, can we define a scale of measure? Well, extremely fast, you know, turbo fast. Wait, wait a minute. Have you ever heard of miles per hour or kilometers per hour or light speed or something like that? You know, so scale of measure is really simple. Everybody knows something about it, okay? But defining a scale of measure is defining the concept itself in a practical way. You know, when you define your scale of measure, you have chosen what you're going to work with and live with and what is important. And you can define scales of measure so they are irrelevant and not going to get you anywhere bad scales of measure and which you can define those that really home in or really counts for people. So there's an art to it. But so the question, one, define scale of measure. By the way, there are usually at least 100 different options. Ask AI number one. It's not like there's zero, which is amazing how many people think like my friends at the Software Engineering Institute last night, okay, like zero.

Now the, the other possibility, by the way, is that take a thing like usability, very important topic for us in my book, Competitive Engineering, which you can give everybody a free digital copy of chapter 5, you'll find that usability in practice is 12 different scales of measure. I won't go deeper. I'll just say maybe it is as simple as one scale of measure. Things like security, all the important things probably are decomposable into order of magnitude 10 different aspects of usability or security. Okay? And people haven't learned that either, that maybe they should look for a set of scales of measure, not just one scale of measure. Okay? Now if you got one scale of measure, there's a second step I want to at least mention. It's defining what I call the meter. What's the meter? What's like speedometer, volt meter. Was that it's the device for measuring along that scale. Was that. Well, we talked about it in previous episode. It's not a test process. It is a measurement process. Okay? So, so I, I demand fairly early that we define scales of measure to have shared clarity of Purpose and to agree that this is relevant. It's not a bad scale of measure. It's relevant to our stakeholders. Right? That's the starting point. Then we set a number on the scale of measure. First number is, I call it the past level or status. Where are we?

Wow. People don't know. People haven't even found the number. But it's very revealing, you know, how bad are we in security today or usability today? Then we take another number, which we call the tolerable level. This is a scalar constraint.

What's the minimum we have to get to to survive? Survival level. We put a number on that. You know, that in two years we have to be at 67 or we die because the competitors will beat us or the enemy will beat us. Then we set a third number. I call it the target or goal level. That's where we have to be to succeed. That's actually the numeric definition of success.

Now, that's what we're. That's our North Star, that goal level. That's. And we may have 10 of them, or we may have 10 times 10. If each of the 10 are decomposed into 10, who cares? But we have a very clear set of North Stars.

[A]

And I think that the next step, which is critical is okay, but now that we have those numbers, we actually have to have a way to continuously assess where we are regarding those and then make decisions.

[C]

And that's called a measurement process, not a test process. A measurement is a form of test, but it's essentially different from saying, is there a bug or not kind of test process. So, yeah, we need a measurement process, and we need a measurement process early and frequently so we can learn where we got it wrong or slightly wrong and pivot or adjust very rapidly. We don't waste time going down a bad rabbit hole.

[A]

Talking about rabbit holes, I can think of a couple right here. We will talk more about how this whole thing comes together in tomorrow's episode. But Simon, there is something that seems so obviously simple, yet often completely ignored and sometimes in some organizations, even completely impossible, which is to create that feed loop from what Tom just described, Right? Like, because it's not enough to have that North Star, it's not enough to have the measurement process. We still need to act on that measurement, right? So when you think about complex systems, how do we build structures that allow us to not only collect the information doing that measurement process, but then also decide based on the information available and then actually have a way to deploy that decision? Because, you know, if A manager somewhere makes a decision, certainly we can't hear it fall in the middle of the forest. Right. It's like the tree in a forest that nobody's listening to. Did the tree really fall? If a decision is made by a manager and nobody gets the decision, did the decision get made? Right. Like, how do we work around this aspect? That organizations are actually completely, unpredictably adaptable systems.

[B]

Yeah. And so really, I think about what is the kind of like human growth hormone to this change? Because if we think about a hormone to a system, it can be fast, it doesn't have to be expensive, it doesn't require a user's manual. That hormone is trust.

My wife often talks about any organizational change moving at the speed of trust.

[A]

So give us an example of what you mean by that. Trust as a hormone. Great metaphor. But how does that translate into something we can.

[B]

Once there's trust, information flows. Once their information flows, the systems comes to life and can learn until they're stressed. You have the Soviet problem.

[A]

Okay, so I'm thinking I'm a manager in a department somewhere in the middle, right? Like, not at the top, not at the bottom, because people at the top have superhuman powers. Like, when they say something, everybody listens. And the people in the middle are kind of lost in the middle. They can't influence very much upwards and they're not sure how to influence downwards and all of that. So I'm a manager somewhere in the middle and I wanted to implement Tom's ideas and I actually defined the qualities that we wanted to achieve. I established the measurement, I got the data, and now I need to make a decision.

What do you mean by trust or speed of trust? Like, I'm in the middle. Like, what do I literally do?

[B]

Yeah. What do I literally do tomorrow? What I literally do tomorrow is make the work visible to my manager to alignment check with my manager first, then go do something and then show them the result. If I live as the learning cycle myself and I have a modicum of trust with my boss, it'll be awesome. Until that exists, I'm withholding. Right. A low trust environment hoards information. Efficient, lovely environment. Share.

[C]

A remark. If you manage in fact, to deliver increased critical value every week, you will build trust.

[B]

Bingo.

[C]

Yeah.

[A]

And of course also visibility to the progress, even when it's bad, as long as it's early.

[C]

Right.

[A]

That's very important. One of the things that I very often realize with, like in organizations when we talk about trust, there's something that is very simple to see is the Longer you go without talking to someone, the lower the trust becomes. You can think of trust as almost like a chemical property of the system. Right. Like if you put some entropy into the system, there's entropy, but the entropy will dissipate and it will spread and it will be less visible at each part of the system. So if you think about that thrust is a little bit like that. It's like temperature.

[C]

Right.

[A]

Like when you get away from the heat, you lose heat. Right. And what I was thinking is if I take Tom's ideas about establishing the metrics for the value qualities we care about and then the measuring process, I think that.

Together with the evo process, where we start cycling through the whole cycle by involving the right people, that starts to build trust. And I want to put a caveat here, which is if we do it early enough, we need to start doing it today. Not tomorrow, not in the next strategy cycle, but today. Because the trust is something that we build slowly. We might lose it quickly, but we build very slowly. So we need to start exercising that neural connection. Yeah.

[B]

And if you want a nice metric for managers across a huge organization, just say, what is the net age of your teams? In other words, what is the time since last personnel change? In most organizations, it's going to be less than three years. Think of a three year old.

And we wonder why we feel like we live in chaos.

[C]

Right.

[B]

That's a systems level rule. Saint Donella Meadows told us this simple inspection with Tom's methods and the leaf methods let these obvious things precipitate out of the cloudy solution.

[A]

In terms of trust, we could even ask, when was the last time you had a trust moment with the person? That could be just a meeting, a positive phone call. You help them, like whatever that would be, because trust decays fast with time.

[C]

Yeah.

[A]

So we talked about quantified attributes, we talked about quality in terms of the value delivery, which is quality 5.0, as Tom likes to call it. And next we're going to bring it all together and talk about how the Agile organization can become a learning system. So stay tuned. Tomorrow we will talk about that. Tom and Simon, thank you for being here.

[C]

Thank you.

[B]

Thanks, guys.

[A]

All right, I hope you liked this episode. But before you hit next episode, here's the deal. This podcast is powered by people like you, the members who wanted more than just inspiration. They wanted real tools and real connection to people who are practicing agile. Every day we're talking access to over 700 hours of agile gold, CTO level strategy talks, Summit keynotes live workshops, E courses, Deep Dive interviews, books, and if you're into no Estimates, we got the pioneers of no Estimate in those Deep Dive interviews as well. Agile Business Intelligence, creating product visions, coaching your product owner courses. You name it. You'll get invites to monthly live Q&As with agile pioneers and practitioners, plus a private Slack community which is free of all of that AI slop you see everywhere. And of course, without the flame wars, it's a community of practitioners that want to learn and thrive together. It's the best place to connect with community and learn together. So if this podcast has helped you before, imagine what you will get from this podcast membership. So head on over to scrummastertoolbox.org membership and join the community that's shaping the future of Agile. And we have so much for you. So check out all the [e-mail] membership because listening is great, it's important. But doing it together, that's next level. I'll see you in the community.

[C]

Slack.

[A]

We really hope you liked our show. And if you did, why not rate this podcast on Stitcher or itunes? Share this podcast and let other Scrum masters know about this valuable resource for their work. Remember that sharing is caring.