Mouli Berry (5:16)

Absolutely. The aspect of the heat map is actually rather interesting because depending on how we lay out the information, you can actually pinpoint the areas of the organization where more support is needed and therefore justify more investment into that particular area of the organization. As I was looking at your heat map when you were presenting that, the immediate idea that I got was, hey, for sufficiently larger organizations, of course for small organizations you may not need this level of complexity, but for sufficiently larger organizations, you can overlay the heat map on the different countries, whether it is the whole globe or just two countries, whatever. And you can actually find where do I need to put more emphasis? Right, like, where do I need to bring, I don't know, somebody to talk about TDD or somebody to help the teams work on the CI CD pipeline, or somebody to work with the product owners to improve the way requirements are documented and followed up on and so on. And it really helped me kind of imagine how we could use this as an information radiator for leadership to then decide where to allocate the investment that they have, but of course need to prioritize for improvements in the Organization.

Mouley Berry (6:40)

Yeah, I think there are many ways of using, and I've seen many ways of using this data. Think about a proper management team that is trying to drive Agile mindset. And everyone wants to improve. And so imagine a good retrospective where everyone comes in with their plans to improve. One team wants additional funding for automation, another one's funding for a different tool, another wants to rewrite a certain part of the infrastructure, another wants to replace the CICD infrastructure and so on. And you as the manager is very difficult to decide which one of those is a proper investment, which part of the organization is already at a good enough level and which part needs, needs help. And furthermore, you have five, 10 teams under your responsibility. Maybe one of them is already very good at test automation. And all you need to do is borrow the test architect from their team and have them run for an iteration in the other team. So these are all discussions that today happen at a very intuitive level. Okay. It's more or less the person who has the most convincing power at the retrospective or closest to the budget or the manager's ear will get the approval to invest in their specific improvement. And I think we've seen many cases where teams go into rewrites and they go into replacing tools and so on, and no one really understands the rationale behind that because there are many other areas without which are more burning in terms of the need to improve. So I've seen this visualization used and I spoke about Philips earlier, but maybe I can speak about Aptiv, which is a large automotive company. So a dashboard describing the entire software landscape of Aptif was presented on monthly basis to the CTO of the company. And this is a $20 billion company. So the CTO doesn't have daily touch with every team and can't really say which team is doing well or not, but they trust the effectiveness index. And so they look at this heat map and they say, well, we see teams here that are improving, we see teams here that are struggling on dynamic analysis, are struggling around on automation. We need to fix that. And so let's put a specific effort to fix that for the next review or the next iteration and so on. And it also gives high level management where they are not exactly software experts, it gives them the ability to quantify how effective the teams are so they can say, well, I trust this index. And so this team is 50% effective and I want them to be 60% effective. And so please team, here is the index, go and rise up your effectiveness towards the next year or the next iteration. And so on. And so it becomes almost like an internal language and everyone can use, which allows the teams, on the one hand, to improve their effectiveness in any way they see fit, but on the other hand, gives management a way of monitoring the entire process and actually proving to themselves that it's actually working.

Mouli Berry (10:29)

Yeah, and visualization creates that internal language of improvement. Of course, the whole set of things that we choose to be in the model we're using will also affect that. But the visualization does help. One of the aspects that you mentioned was this kind of even intuitive discussion about what needs to be improved next. Right. And I know you have a great example about the quality of code reviews, which sometimes gets discussed, hey, you know, code reviews take too long, or we don't get enough information, or we don't have enough people available to do whatever the improvement might be. Right. Like, people complain about it, but they don't have a concrete and disciplined way to go about checking that it is actually being improved. And I know you have a great story, so can you give us that example of how this team was able to effectively focus on how to improve the code reviews and actually measure the progress of that improvement?

Mouley Berry (11:27)

Yeah, yeah, sure, sure. The code review is an interesting example, because if you ask every. Any team, are you doing code reviews? Almost everyone will religiously say, sure, of course we have to do code reviews. It's mandatory. We have. We do it for every change. We are very passionate about code reviews. And so then if you ask a team, how well are your code reviews done? The conversation kind of shuts down. And some people will say, well, we think it's okay, but we don't really know. And the story that I wanted to share started when there was an initiative. Someone was at the conference and came back with an initiative to double the time that we invest in code reviews. Okay. They heard somewhere, they read somewhere that this is great practice. It will immediately find much, many more issues and then make the code quality great. Double the time that's invested in code reviews. And the problem was no one was able to quantify whether this is a good investment or not. It sounds okay, but if you do it, and let's assume you've done it for an iteration, how do you prove that this is actually a good investment? And why not quadruple the time spent in code reviews? There's no data behind that discussion. And so we came up with the concept of measuring code review escapes, which is basically an exercise every team we work with is doing. At the end of every iteration, you take a Sample a subset of the defects found in the iteration and you gather the team together and you ask for each of these defects. Only one question. Could we have found this defect if we did a better code review? That's it. And if the answer is yes, we could have found that if we did a better code review, then this is a code review escape, which means this is something that should have been identified in a code review and wasn't. And that of course needs to lead to some improvement in your code review process. If you say 3 out of 10 defects could have been found if we did a better code review, you have a 30% code review escape. And now whatever actions you take, you can double the time you want to invest in the code reviews. That's fine. At the end of the iteration, measure that code review escape ratio. Again. If it's still 30%, doubling the time didn't help. If it's now 20%, okay, now you can calculate an investment and benefit ratio. So this led to this concept of measuring COD review escapes. And now we have the same concept around requirement escapes and design, yeah, design design escapes and architecture escapes. And each of every time we go into discussion of we want to improve something, it's based on measuring what are the number of issues that escaped that specific phase in the software development life cycle. And so we are able to consciously decide whether we need to do more on code reviews or maybe this is already good enough.

Mouli Berry (14:57)

What I really like about this story is how first, yeah, let's invest that double the time in code reviews, but let's measure it. How would we measure it? Right. Just asking that question is already very productive because of course we know what better code reviews mean. But we do need to go through the trouble of defining that clearly measuring it. And I think that's the most important. Be accountable to ourselves and also learn from the process. Right? Because, you know, let's say that we did double the time in code reviews, but maybe we got a 5% improvement in code review escapes. Is it worth it? Right, we can ask that question, and that's a very important question. Now, of course, the challenge here, and this is a challenge also that I'm sure you have faced, the challenge here, is that we might fall into this idea of top down improvement mandates. Right? You shall improve your code reviews by 5%. How does the shift, how does your method, instead of, you know, focusing on this top down improvement requests, which traditionally we know don't work very well except in very specific organizations that invest a lot in internal auditing, how does your system help to not just make the improvement visible, not just make it measurable, but also shift the accountability to the teams so that they keep themselves accountable and also that they take advantage of learning that the process would give you while keeping that organizational alignment that usually top down mandates are there to give.

Mouley Berry (16:40)

Yeah, that's really a nice challenge to solve. Maybe I can start responding first by telling you a short story from my time in Microsoft where we had one very senior VP of engineering that was really into unit testing. He thought unit testing is the solution to all of the world problems. And so there came one day a directive that everyone needs to move their unit test coverage up from, I think it was that time 70% to 80%. And everyone, regardless of how good the code was, how proficient the team was, everyone needs to invest the time to up their code, review code coverage metrics. And that is an example on the one hand of someone who is knowledgeable somewhere up the management chain, but on the other hand a top down directive that is not relevant to many teams because some teams the main problem is not the unit test coverage. And so with our approach, what we basically try to agree on is the definition of effectiveness through our effectiveness index. And this effectiveness is a composite of this. We have 35 competencies derived from the software development lifecycle all the way from, as I said earlier, from requirements to deployment. And so the overall effectiveness is some of your different parts of effectiveness. And so when we say to a team, your effectiveness is now at 40% out of 100, they have many, many different ways of moving this up the scale. They can move it to be more effective. Some teams, for instance, they don't have easy access to the live production environments, it's expensive equipment, it's changing too often and so on. For those teams, maybe it makes much more sense to invest in unit testing. Some other teams, most of their issues come when they integrate into a larger system. And for them maybe integration testing and functionality testing is a much more sensible investment. And so they can choose to stay very basic in their effectiveness on unit test, but make it extremely effective on integration test and functionality test. From the team's point of view, they are doing the proper effort and investing in the right thing for them. From management point of view, it's a win win because the team is improving on the effectiveness index, they are getting better quantifiably, they're generating less defects, delivering faster to market, higher quality and so on. And so it's basically a win win. The team gets to define their own path based on what makes sense for them. And management can see that the overall effectiveness of the organization is growing.

Mouli Berry (19:58)

Okay, so I need to ask this follow up question because obviously, you know, the improvement of effectiveness is as you describe it, opening the door for teams to decide, you know, where should we invest? What are the areas that we as a single team need to focus on. But it's very tempting for management to still say, hey, but I want everybody to improve. Test driven development or unit testing or customer interviews or whatever like that manager you described a moment ago did. So how do you work with organizations to try to open up that possibility? Right. To really delegate, but also shift the accountability to the teams on what to improve? Like how does that happen in practice? I'm sure you've had difficult conversations on that topic with the people that you've worked with.

Mouley Berry (20:50)

Yeah. So I think the example I gave from Microsoft is actually the more rare example because I. So I've done an initial scan of the full organization in Philips that was 2015 or 2016, I don't remember. And then I presented the outcome to the CTO of Philips at that time. And they were not software, software experts. But he saw that the effectiveness of the full organization is, let's pick a random number, 60. And he asked only two questions. He said, everyone, do you agree this is a good way to describe your effectiveness? And the teams under review said, yes, this is a proper way of describing our effectiveness. And then he said, okay, so if you agree your effectiveness is now 50 or 60, let's go into a three year program of moving up the effectiveness scale. 10 points every year. Every team, define your own path. I don't care. I don't care if you, you're, you're currently at 50%, which is low. So you have to be better. Now, between you and me, when, when you get to 80% effectiveness and you want to take it to 90, there aren't a thousand ways of doing that because you're already very good at many things. So you have to improve the last remaining points. But when you move from 40, 50, 60% effectiveness, there are many ways of doing that. Maybe I'll take an example. Let's assume you are an Olympic athlete and you want to improve your fitness. There are probably few ways to do that. But if you are couch potato and you don't move a lot and you want to get in shape, it doesn't matter if you walk three days for half an hour every day, or if you go swim or if you bike or you golf or play tennis. It doesn't matter. Each of these will make you in better shape. And so, so I think this, this actually takes a load of upper management because if they focus on one thing, they have to also be sure that this is the right thing to move the organization. And very often they don't really know what's holding back the organization. And so they can say, well, I have this external independent view that says you are only 50% effective. Well, just work on it and become 60% effective, 70% effective, and then let's see where our issues still remain.

Mouli Berry (23:42)

Yeah, I remember I interviewed David Markey a long time ago here on the podcast, and he, he shares a story of how they're in an exercise. He was the commander of a submarine and they are in an exercise. And he gives a, an order to the person controlling the machinery and says basically the same or equivalent to saying, hey, you know, let's move to fourth gear. And then the person in control of the ship says, sir, this ship does not have a fourth gear, sir. And this, I think, is a beautiful example of the fallibility of management. Right? Like management might have the right intention, the right direction, but they don't necessarily know what is the detailed action to take in every specific team. And that's also one of the things I really like about the model that you present, which is the idea that, you know, you have this menu of things that are considered important for software teams to be effective. And then the teams themselves in their context acknowledge what is it that they need to improve next. And I think that just the list of items is a great tool for a retrospective. Right? Like the team can self grade in all of those items and start from there. Now, of course, when we do it at the level of the organization, it gives us the ability and the benefit that we might see systemic problems that the teams cannot see because they only see their own context. And I think that the visualization we discussed at the top of the show is a very important aspect of that.

Mouley Berry (25:12)

Just one point about the list of topics, I didn't mention that before, but the list of topics that we measure is derived from an ISO standout for software development, Life Cycle. And so it's supposed to cover everything that a software team needs to do. But as to context, when I developed that, I needed to grade different teams, remember in the Philips case? So Philips had teams building software that goes into espresso machines and teams that build software that manage entire hospitals or run MRI scanners or CT scanners. And so very different type of software. And of Course, for every different type of software, there is a different consideration of what's important, what's more critical, what are the critical components. And so this framework really allows every team to define for itself what's important and still allows top management to drive the mindset of continuous improvement and measure that the organization is indeed improving in some fixed rate. And the top management can prescribe the rate of improvement. I've had customers where the management says we think we're more or less okay, let's try to move up the effectiveness ladder slowly. Five points a year, that's fine. In some places I've had teams say we need to move 10 to 15 points of effectiveness every year and invest more. So this is where management can weigh in in terms of the investment, the attention to improvement. But still, every team defines their own software improvement plan and they are the ones driving it. But they of course have to demonstrate that the heightened effectiveness.

Mouli Berry (27:09)

Yeah, and another aspect that you just kind of mentioned in your talk about the Philips case is that of course you work with many different teams, but also many different industries. And here in the podcast, the question that is really interesting for everyone listening, I'm sure is okay, but how do you connect the studies and the improvement follow ups that you have done with patterns of successful agile adoption across this different diverse contexts? Pardon me?

Mouley Berry (27:41)

Yeah, the. The. So. So I've worked a lot in healthcare with Philips and Roche and then a lot in, in automotive with, with Ambition and ZF and Harman and Aptiv of course, for many years. And at the end of the day the differences are not that great. Of course every software team believes they are unique and to a certain extent they are. But we all have the same profession. If you ignore for a second the features and the specific industry context, then everyone goes through the same pain. Understand the requirements and try to make sure they really reflect what the customer needs, convert them into some design which will be resilient to future changes, write some code, test it, and so on. Now, of course there are slight variations. Some industries are more into code generation and some more into integrating third party products. Some are slower to release, some are faster to release. But the, the birth of this framework was from an environment where Philips was so diverse that it had to accommodate very different types of software. So we had to keep it flexible enough for every team to define their critical components. And for instance, when we measured the effectiveness of dynamic analysis. Dynamic analysis is about measuring behavior in runtime, but whether you measure CPU usage or resource usage or whatever, that really depends on your Context. And so when we work with the team around dynamic analysis we ask them to define what are the components they want to monitor in real time and then these have to be monitored and properly acted upon and so on. But the team is the one responsible for defining the context and defining the also the urgency. If you're working in as part of a team that is responsible for releasing the some parts of a CT scanner and you release once every four years, continuous integration has a different meaning for you than it has for someone who is releasing a new update every 24 hours. So the flexibility is there. It's up to the team to define the critical to success components, but it's not up to the team to prove that they are becoming more effective as, as they work.

Mouli Berry (30:55)

Well. One of the things that is really interesting from what you just shared is that once we realize that this is like what you, the model that you are working on is kind of a, a process of evaluation, measurement and follow up for continuous improvement, we can easily start seeing other aspects that might be coming in. Now of course you based your model in the SDLC or software development lifecycle, but we as agilists we can bring in perspectives to the same model that reflect our contextual reality. Right, the company we're in, but also the need to adopt Agile in certain aspects, for example time to release or time to respond to customer requests or availability of product owners or all of that stuff. So what I'm thinking is, and I would like to hear your thoughts Mouli. What I'm thinking is that we as a community can also contribute to that model to try to represent the same style of high level components of the model, but that also represent a perspective on how the organization works rather than how the software gets developed only. What do you think about that?

Mouley Berry (32:08)

Yeah, I can see where you're coming from and I can understand your thought process. I'm a software guy and have been my entire life. But if we zoom out for a second, the nature of Agile is a mindset of continuous improvement and that's fine, we agree and I think there is a large community promoting continuous improvement. But I think where we see a lot of issues is around fine, we understand we need to improve, but where to improve by how much? And is that specific improvement the right thing to do now? And so trying to quantify the investment is, is for me a very basic, a very basic requirement in any improvement effort. And I, I this, this framework is definitely flexible enough to also accommodate other domains. It needs to have the domain expert that can Plug in the, the rationale into it. But essentially, if you're trying, if you're, if you're trying to make the team do a better daily scam, okay, even saying these words, this needs to be based on some proof that your daily scrum is not working. And so here comes the question, what do you use to measure the effectiveness of the daily scan? Is it the attendance? Is it the number of features delivered? Is it the number of rejects at the end of the sprint? Whatever it is, and I'm really not the expert on that, you can't ask a team to improve the daily scrum without showing some quantitative evidence that it needs to improve and then being able to measure it at the end of the next iteration and show that it has improved. So to that extent, I think this type of discussion that the way I sometimes call our framework is we are supercharging Agile with the quantitative data, which helps the discussions. I'm sure many of your listeners have faced the frustration of trying to convince the team to do a better daily scam. And the team is saying, well, we think it's okay, we think it's going well. Look, everyone's coming, everyone's talking, and so we think it's okay. And to break through that, you need a metric. You need something to be able to show the team if it's working well or not and also show the team the side effects of not improving.

Mouli Berry (35:16)

Absolutely. That's a great point. Right. And there are some trends within the agile community. For example, evidence based management is one trend that has started there. But I think that plugging some of those things into the model that you have would help us to also get a holistic view of the organization. And of course, we can only start where we are. Right? Like if we're the CTO of an organization, we can go and look at the whole organization. If we're a tech lead or a QA lead or a Scrum master in a team, then we need to start in our team. And you know, the best time to start continuously improving was 10 years ago. And the next best time is today. So let's get started with the continuous improvement. Mouli, thank you very much for sharing your knowledge and your experience with us. If people want to know more about you and the work that you're doing, where should they go?

Mouley Berry (36:09)

Well, the best place to go is to go on our website to bettersoftware.dev and look up what we're doing and some case studies and also reach out to me directly, can find me through my LinkedIn page or the different ways, but basically go through our website, reach out. And we are also don't do too much promotional, but the way we work is that every, every team that is interested can do a proof of concept basically for free, has to try out the framework. And I think many of the aspects of this framework can be developed by every team on their own. And we do have several cases where teams have done a proof of concept and said, well, we understand this makes a lot of sense, we will take it from here. So we really invite people to experiment, try it out. And I agree with you that teams should just start, I think today maybe other than I wouldn't say they need to start improving. I think everyone is maybe improving too many things too much at the same time without really measuring what gets improved and what is already good enough. And so I would say start taking the improvements under a controlled view and really understand where you need to invest and where are the organization's weak points.

Mouli Berry (37:54)

Yeah, absolutely. And that's the other aspect, right? When we go across teams, we can look at the organization rather than just our team and how it is doing. We'll put the link in the show notes bettersoftware.dev so everybody be sure to check it out. And there's a few videos that Molly has available also on the website if you want to know more and dive into deeper details. Molly, it's been a pleasure. Thank you very much for your generosity with your time and your knowledge.

Mouley Berry (38:21)

Thank you for hosting me. This was a good experience for me.

Vasco (38:30)

Hey friend, thank you for staying here. Is all you need to know about the Global Agile Summit. If you've ever suffered or know people who are suffering from agile fatigue, this event is for you. Agile fatigue is that feeling that settles in when we can't really see a light at the end of the tunnel. We get discouraged, especially when conversations revolve.

Mouli Berry (38:55)

Around the same old frameworks, the same.

Vasco (38:57)

Old buzzwords and theories. We don't feel that energy anymore. Well, the Global Agile Summit is a different kind of event. We're bringing you real life first person stories of agile succeeding out there in the real world that will inspire you to take action and transform the way you work. The Global Agile Summit will happen In Tallinn, Estonia, May 18th. That's the workshop day. Then 19th and 20th, the conference day. And Tallin, Estonia is one of the most innovative tech hubs in Europe. The Global Agile Summit is hosted together with Latitude 59, which is kind of a citywide celebration of software startups and groundbreaking ideas and we'll have a shared ticket for you to attend those events as well. So who will be speaking? Well, we've got an incredible lineup of thought leaders, leaders in software and agile. For example, Clinton Keith, the person who wrote literally wrote the book on game development with Scrum and is busy bringing Agile to the world of game development. You must check his session. The very famous and well known Jurgen Apello, Author of Management 3.0 will be talking and exploring about AI's impact on leadership. We also have Goiko Atsic, who's taking an unconventional look at product growth with his Lizard Optimization keynote. Other speakers include, e.g. sig Sven Dietz, who's challenging everything we know about software development by ditching, literally ditching contracts and estimates. Can you imagine his teams deliver software before the their competitors are even done with a contract negotiation? How agile is that? But there's more. We'll cover engineering practices in our developer track with talks on for example AI assisted test driven development, developing products in minutes with a different approach to how we develop, configure, deploy platforms and much more. We also have a product track where we cover cutting edge ideas around product discovery, delighting customers with product delight frameworks. We'll have a talk about that. And we also have an Agile Business track where we will talk about for example Open strategy, a very agile approach to managing organizations and delivering software faster to clients faster than you can even write a contract. Literally. I mean, I already told you about Svendeet's story is amazing. It definitely is a must see. I'm sure you'll be inspired and get a lot of ideas for your own software projects and software delivery. Now whether you're a business leader, a product innovator or a developer, you'll definitely find find value in our three focused tracks. That's Agile Business for those working with businesses and organizations, Agile Product for product managers, product owners and innovators, and Agile Developer for the builders making Agile work in practice. The coders, the testers, the designers, the producers, the Scrum masters, you name it. If you join, you will meet over 200 agile professionals from all over the world. People who just like you, want to grow, want to share and want to learn by challenging the ideas that don't work anymore. At the Global Agile Summit, you'll get new connections, fresh ideas and the energy to take your own Agile to the next level. And who knows, maybe even find your next career opportunity. So don't miss out. Check out the full full program and grab your ticket now at Global Agile Summit, Dot com. I'm really looking forward to seeing you all in Tallinn, Estonia, in May.

Mouli Berry (43:08)

I'll see you there.