← Scrum Master Toolbox Podcast: Agile storytelling from the trenches
Scrum Master Toolbox Podcast: Agile storytelling from the trenches
AI Assisted Coding: How Spending 4x More on Code Quality Doubled Development Speed With Eduardo Ferro
00:32:45
Loading summary
Transcript55 lines
- [00:04]
A
Hey there, agile adventurer, just a quick question.
- [00:07]
B
What if for the price of a.
- [00:09]
A
Fancy coffee or half a pizza, you could unlock over 700 hours of the best agile content on the planet? That's audio, video, E courses, books, presentations, all that you can think of. But you can also join live calls with world class practitioners and hang out in a flame war free and AI slop clean slack with the sharpest minds in the game. Oh, and yes, you get direct access to me, Vasko, your Scrum Master Toolbox podcast. No, this is not a drill. It's this Scrum Master Toolbox membership. And it's your unfair advantage in the agile world. So if you want to know more, go check out scrummastertoolbox.org membership. That's scrummastertoolbox.org Membership. And check out all the goodies we have for you. Do it now. But if you're not doing it now, let's listen to the podcast.
- [01:12]
B
Hello everybody. Welcome to this very special bonus episode in our AI bonus week. And for this episode we have Eduardo Ferro. Hey Eduardo, welcome to the show.
- [01:24]
C
Hey, thanks for inviting me. I'm really happy to be there here.
- [01:28]
B
It's a pleasure to have you here. So let me tell you a little bit about edu, edu or efero.net you will see the link in the show notes. Edu is the head of engineering at the Data Platform and. Data Platform, sorry. At clarity AI with nearly 30 years experience, he helps teams deliver value through using approaches like Lean XP or extreme programming, DevOps and blending technical depth with product thinking. Recently he's exploring AI assisted product development which is why he's here talking about and sharing his experiences on his website effero.net and you will see the link in the show notes so make sure to check it out. Edu, let's. Let's get started with. I guess I could call it a level set question. How do you define vibe coding and how do you differentiate this vibe coding idea from other types of AI? AI assisted coding?
- [02:32]
C
Yeah, for me VY coding is flow driving, a curiosity based way of building software with AI. And it's less about meticulously reviewing each line of code or more about letting the AI to steer the process. Perfect for quick experiment, side projects, MVPs, prototypes and so on.
- [02:56]
B
Do you use that more flow based, as you said, more AI driven approach in your own work?
- [03:04]
C
In general, yes, but with a lot of different engineering practices. So I try to use the same base about always having like a flow working in super small steps. But always having a conversation, but at the same time introducing all my knowledge about okay, architecture decisions or just give me like different options for the design, let's talk about the coupling or so I have this kind of conversation. So it's flow based, flow driving, but is introducing like a lot of other stuff that is not so common in what we can.
- [03:43]
B
So and how would you differentiate this Vibe coding as you said, more flow based with other types of AI assisted coding. You've shared already that you add a bunch of engineering practices on top of this Vibe coding. But, but is there like, are there like different types of AI assisted coding in your practice and how do you make the difference between those?
- [04:04]
C
For me it's about, it's more about knowing the context. So it's not the same that I am developing like a one shot script for certain things or like an MVP or I am developing like production code if I am doing like a product increment for anything that I'm doing for, for my, in my daily work I always try to work more on the or in the plan or in the specs, trying to identify like the edge cases, trying to define like all the acceptance criteria. In the end I also work in small safe steps. So I always do like vertical, vertical slicing. But it's not the same as when I am by coding a side project that I am more learning, exploring, seeing what is happening. In the other case I have a goal and this is what I want and I already split the process in small safe steps. And at the same time could I.
- [05:06]
B
Then, if I understand you right, could I then express it like Vibe coding is more exploring, it's also coding but we don't care so much about the perfection of the implementation, even if we might use test driven development or whatever. But Vibe coding is more like exploring a problem space. You called it a side project, but it could be an MVP or a proof of concept or a prototype and then we use then AI for production code in a different way or you use production AI for production code in a different way which is already more understanding of the context and you go in smaller steps, you put more effort into the spec and understanding the edge cases. Is that how you describe it?
- [05:51]
C
Yes. But at the same time I also think that even for vive coding, if you are making this experiment bigger, you need some sustainability. It's more about the levels that you impose for each one of the practices and the level of flexibility that you have to explore and change things. But yes, in any case I think that we are going in the direction that more and more we will.
- [06:20]
B
Work.
- [06:21]
C
Less with code at the low level. So we are going in that direction. So for me the difference will be not about by coding or using different approaches, it's more about, okay, what is the context is an internal application is something that is in front of a lot of customer is critical. It's not critical is having huge or super core side effects, for example, or not. So it's about managing the risk. And I think that this is also super important in agile development. It's about managing the risk and this is why we iterate and work in super small steps.
- [06:57]
B
So yeah, but, so what have you learned? Like what are the concrete practices, the approaches that you have learned that help you make AI assisted coding work for you? And specifically I want to ask about production code. Of course you can talk about the side projects and so on because they often give us ideas for. But when you think about production code, what have you learned about AI assisted coding that makes it work for you in production code?
- [07:25]
C
So I think that the most important thing that I learned is that what already work for humans work very well for AI. So all of these best practices that we define about having like process for doing tests, having like a tight feedback loop, working in a small safe step, validating all the time, all the things work super well also for AI and also anything about doing the clean, like meaningful or clean in the sense that you can see like the domain, express the domain and that's super, super important for AI. At the same time it's also cheaper to introduce all of these practices. So it's a matter of the discipline of introducing them to validate.
- [08:18]
B
What do you mean by cheaper? Let's be specific about that. What do you mean by cheaper?
- [08:22]
C
For example, the last two months I did an experiment with code for my company. So real production code, checking the kind of commits and changes that I did during like I don't know, 11 weeks or something like that, okay. And I see that I was investing four times more in refactoring, cleanup, deleting code, introducing new tests, improving the testability, security analysis than in the generating like new features. And at the same time globally I think that I more or less double my pace of work, even if it's not my main work because I'm not a developer, I'm doing other stuff. So I was doing a lot more of these best practices that in the past just because it's easier. And it's like, for example, you do the analysis and define a plan, you start implementing it after that I can code review just automatically. I can't analyze the security. I can just say, okay, from these problems that I see, I will prioritize this one, this one, this one, and this can be in a cycle that is in an hour and in the past, perhaps this was today. So perhaps in the last moment I did a code review, but it was not going so deep because I already did tdd. So okay, who cares? Because perhaps it's working. Or I do like another strategist to try to optimize my time, but now is super cheap. In fact, I have like a small agent that I put in an automatic way generating, okay, let's analyze this code. As an XP developer, try to prioritize potential refactoring and select one and implement. And this automatically generates a merge request. So in that case, even you can connect in the morning and have like two or three things that are improving the sustainability of, of this.
- [10:32]
B
So you used four times more time on code review, writing the tests, executing the tests, checking if things were working but you still were faster. Is that what you're saying? Did I understand it correct?
- [10:48]
C
Yeah.
- [10:49]
B
Wow. So what do you think was the aspect of it? Because, okay, so one argument is that AI writes codes, writes code much faster, which is true, we all know it. Anyone that has tried it knows that. But writing the code is just the typing, right? Like, and I don't think that the bottleneck for software development, even in production, perhaps even specially in production, has been the typing, right? So for you, in that moment when you were developing that system, what do you feel was giving you that much faster pace of delivering new features, even though you were still using four times more time in the, let's call it infrastructure tasks like test spec and so on than in actually writing the new features.
- [11:34]
C
The thing is that I, for example, in one week I can generate like two different experiments that are put in front of the customer. And this need to be not having problems and be robust. And at the same time I'm having feedback. So for example, in this experiment, two of the projects in reality we remove code. So we did remove all features just because it was so easy to use the product analytics and say, okay, but these commands are not being used. So let's remove, let's remove the complexity. And whenever I remove these commands, next step is automatically try to see, okay, can I simplify or I just remove this thing that is the only one that require this part of the architecture and can I simplify this part removing that and this in the past was not so easy or not so evident. And I didn't have a thinking partner that is capable to analyze and identify these opportunities. So I put all the details of this experiment in my blog post. I think that is the last one that I published. And it's okay. The number of commits, the distribution of the different type of comments. And at the same time I'm feeling that I am developing more and having more feedback because sometimes some of these increments I remove the next week if I see that it's not getting what I was. What I was trying to. To the impact I was trying to get.
- [13:19]
B
Yeah, this is really great because one thing that you said, and I want to reiterate that because it's a very important aspect, is that AI can also make removing code faster. Removing code makes changing architecture easier. I can change architecture. So if we remove code faster, we make the architecture changes easier. We can then remove even more code, which makes the changing of the architecture easier. So we're actually introducing a cycle which humans are very reluctant to do, which is this cycle that as we go, we spend time effectively removing code, which is legacy, but it is also liability. Right. Existing code breaks and we need to fix it. So there's a liability there. And that makes the whole process even much faster. So if I understood you right, it's like you remove code which makes architecture changes easier. Because architecture changes is easier, you do it more often. That makes adding new features easier, which makes them even faster. So it's kind of a positive spiral of consequences.
- [14:28]
C
Yeah. In reality, I think the AI is an amplifier. If you have bad practices, the collapse is super fast. If you have good practices, I think that you can even accelerate. So for me, yeah, it's about looking for these cycles of improvement and seeing what is working well for the AI. For example, I have one of the projects that I work on, have five years and we didn't have all the architecture decisions documented as architecture decision records. We have some gaps. So something that I did was, okay, let's analyze the commits for the last year and see if we can have a gap in our architecture decision records so we can introduce this information. And later with this information, it's easier to say, okay, now that you know all of these things that we have decided and I want to generate like this new functionality, how should I change my system to make like super easy to introduce this one? Not this message from Kembek about no making the change. No, make the easy change. And later to the chain in two phases. Always prepare the system to make this functionality easy to develop in the system and later introduce the functionality. So you are adapting the design in that direction.
- [15:54]
B
Exactly. And I think that what you just described is actually the point that really strikes me is this. We are more free to do things we wouldn't do before. So things like what you just described, you prepare the system before you introduce the change. We now can do it because it's so much faster. So we can do that, and we can do that carefully with all the practices and so on, and then introduce a change. And when we prepare the system, if we remove code, we make the change easier, the performance better, and the architecture more maintainable. So it's kind of this positive spiral. But of course, not everything is positive with AI. So, Edu, what are some of the anti patterns that you've seen in using AI, either from your own experience or from everything you've read and analyzed when preparing your own journey to learn about how to take advantage of AI assisted coding? What are those anti patterns you've seen?
- [16:55]
C
I will not say anti pattern in this case. I think that is more about how we change our way of thinking. And this is super difficult. We already see in the agile community about managing the changes is difficult and you require a different way of thinking about the problems in order to use the AI as an amplifier. This is difficult and I'm seeing a lot of push from the companies and the CEOs and so on to just introduce AI without the proper preparation. And I think as proper preparation is, how are your practices or even the code base, how prepared is to have super fast feedback loop. So if you introduce without having this feedback loop or without having test, you can have a problem and it can generate a lot of bugs and so on in a fragile system. Another problem that I see is about expectations because there are some people thinking about that, okay, this is magic, this will solve everything. And I think that no, we need to have a lot of engineering expertise to steer the AI in the right direction and creating all the tooling and all the or taking all the architecture decisions. So for me it's a problem about expectation and thinking that vive coding as I probably is saying that having in production things that have a lot of complexity. So I assume that we will have in the near future like a lot of huge collapse and problems and at the same time we will have other people that is literally doing stuff that was unthinkable like one year ago. No, it's true that in general I think that we are going in the direction to Raise the level of abstraction and thinking more about acceptance tests or about specification or about what the system need to do. But at the same time, we, we need to take decisions about architecture, about, I don't know, the practices, about the flows and so on. So we will need all of this expertise. And there are people that doesn't really understand the nature of software and are the same that generate problems in the past. So yeah, it's an amplifier. So you have the different approach.
- [19:26]
B
So if I understand you right, like if taking that AI as an amplifier, as a metaphor, if I understand you right, what you're saying is eventually the people who already knew how to develop software well will continue to develop it well and faster. And the people who did not know how to develop software well will probably get into troubles much faster than they would otherwise. That's basically what you're saying, right?
- [19:51]
C
Yeah, that's for sure. And at the same time, there are things that was not even, or considered impossible or so for the people that is doing the things right that now are not just possible, but you can even experiment with different solutions. So this will generate like a lot of stuff. This is another problem. So because we will saturate all the marketplaces and all the things with a lot of different products. And I think that we, the importance of discovering the real thing to do or even the distribution channel will be even more important in the future. Because we will have too many different options for anything.
- [20:35]
B
Yeah, absolutely. And I think that having too many options is also an issue, right? Because for example, a very simple example, just out of my head, understanding what kind of technology fits, what kind of problem space is important. So when we use AI, especially for those who never really cared so deeply about the technology choices, it's very fast, it's very quick, rather to make bad architecture decisions, but it's also very easy to make bad technology use decisions like what frameworks to use, what kind of hosting platforms to use, and so on. So what I'm thinking about here is that architecture. So by this I mean the decisions that are hard to change later on, right? Architecture becomes so much more critical. Like for example, do we use Node JS or do we use Java backend, or do we just use HTML with JavaScript and it's a single page app or whatever. Like all of these technology decisions have huge impacts down the line. And I think that with AI making things so easy and so fast, it also makes it easier to make the wrong decisions faster. How do you see that part? Like the real core, the real bottom of the decision layer when using AI.
- [21:54]
C
Yeah, the thing is that I completely agree that the architecture decision will be more important and you will need to do more architecture decisions per week or per month. In the past, you just decide this once every one year, two years. And so at the same time I think that will be easier to change if the decision was wrong. But you need to have the taste to detect that the architecture is the problem or so. So you also need this knowledge. So I will assume that it's easier to solve the problem you have generated, but at the same time you need to take a lot more architecture decision and this is not so easy and also require some expertise. So yeah, there is like side effect that is that I think that some technologies will commodity size and I don't know, centralizing. I don't know, you will decide that to use this framework just because it's easier for the AI and have more training data and so on. And at the same time, if you want to do something completely different, using the same thing will not be the solution. So it's not is not so easy to know in what direction this will be evolved. But sure that anything related with architecture will be more important than now. If not that it's more important now, it's at the same level of importance. But the pace is so fast that you need to take more of these kind of decisions.
- [23:29]
B
Talking about trends, when you look at AI assisted coding and how it's used right now, what are the trends that you are already seeing when how do you think this approach will evolve in the future?
- [23:43]
C
I think that we will learn more about AI assisted coding, augmented coding and best practices. Some of these practices will be incorporated even inside the tool. So it's not something that we will need to introduce ourselves. The tools will absorb these kind of things. In general, I think that we are going in the right directions. Assuming that we can divide the different kind of solutions or things that make to be sustainable in the time, things that are more temporary. The thing is to identify in which kind of solution we are creating and which are the goals. I will see a huge trend.
- [24:31]
B
About.
- [24:32]
C
Having like a huge bottleneck with product decisions, business decisions and validating the market and discovery. So for me this will be more important than the thing that we will implement is like we will change the bottlenecks of this. And also if coding or implementing is not the bottleneck, we will need to see how they will evolve, like even our profession. I think that this will go in the direction of product engineers, mixing roles and not Having so fixed roles for different people in the company.
- [25:18]
B
Yeah, the roles part is actually very interesting. I think this would be a whole new episode or even a podcast series about how AI coding changes the concept and the structure of teams. Like a very simple example that I face every day because I do a lot of small but yet important production projects for the work that I do. Like software that helps me with specific tasks that I need to do with clients. One of the things that I see is the collapse of the roles. Right now a developer is a strategist, a product manager, designer, a coder, a tester. Like everything is being done so fast that the roles start collapsing into one person. Right. How do you see that? Edu?
- [26:06]
C
Yeah, I think that generally is always. I think that can have like a huge impact in this kind of environment. Just because you will mix like different roles, I expect like super small teams in more fluid organization. Let's see how we will manage all of these changes. But yeah, I think that the difference between a product manager, a product engineer, for me, the only thing that is super clear is that the people that is just thinking their self about someone that just code, I think that these people will have a problem because the code is losing the value. The low level code, let's say someone that thinks about being a translator. I think that we need to always understand the goal, the product, the impact.
- [27:05]
B
And that's a real challenge even now when you have multiple people doing these different roles. Now imagine when you have one person doing all of these roles.
- [27:14]
C
Yeah. In that case, I don't know how we'll be the solution. But I expect super small teams working in missions that start and end and moving around with different profile because you always have a preference or something that is your focus, even if you know about other things. So you will cover with AI as a thinking partner for the things that you don't know and with another person or two, but I don't know, teams of three, four, but not six, seven or so.
- [27:53]
B
Yeah, that's definitely something we will all be looking for with much interest and attention. Because I'm sure that the team structures will change as AI gets more and wider adoption. So Edu, we're getting close to the end, but before we do, we always ask our guests, is there a resource, a book, a video, a paper, a tool, a course that you think every practitioner interested in coding with AI should check out?
- [28:23]
C
I share all my recommendations in my blog. So I have a site just for the talks and so the ones that I put there are already Recommended. And I'm falling super close. The things that Kemp the Farley modern software engineering YouTube channels are doing related with AI because it's the same people in the strength programming that is exploring the way this can evolve.
- [28:55]
B
Very good, very good. And we'll put the link to those on the show notes and of course we will also put the link to edu's website efero.net where you can follow his adventures because he does write a lot about his experiments with AI.
- [29:10]
C
I just remember the Vive coding book from Jin Kim and I don't remember the other author. I like it. But in reality they are not discussing about pipe coding in the sense of they are also thinking about engineering practices and the flow and it's okay. The only thing is that it's evolving so fast that perhaps in six months is already outdated. But I think it's a good book.
- [29:37]
B
I think it's always good to add that even your AI assistant can help you find more information and test things out. Right? Like with the help of the AI assistant as well. And it's been a pleasure. If people want to connect with you, learn more about you and the work that you're doing, where should they go?
- [29:56]
C
I think that my blog is the best way. I have contact there and also LinkedIn or so I don't use X too much. I use more Blue sky. But I have all the information about contact in my blog.
- [30:13]
B
So yeah, absolutely. We'll put the link to those in the show notes. Edu, it's been a pleasure. Thank you very much for your generosity with your time and your knowledge.
- [30:23]
C
Thank you for inviting me. It was great.
- [30:29]
A
All right, I hope you liked this episode. But before you hit next episode, here's the deal.
- [30:34]
B
This podcast is powered by people like.
- [30:37]
A
You, the members who wanted more than just inspiration. They wanted real tools and real connection to people who are practicing agile. Every day we're talking access to over 700 hours of agile Gold, CTO level strategy talks, Summit keynotes, live workshops, E courses, Deep Dive interviews, books. And if you're into no estimates, we got pioneers of no estimates in those Deep Dive interviews as well. Agile Business intelligence, creating product visions, coaching your product owner courses, you name it. You'll get invites to monthly live Q&As with agile pioneers and practitioners, plus a private Slack community which is free of.
- [31:29]
B
All of that AI slop you see everywhere.
- [31:31]
A
And of course, without the flame wars, it's a community of practitioners that want to learn and thrive together. It's the best place to connect with community and learn together.
- [31:45]
B
So if this podcast has helped you.
- [31:47]
A
Before, imagine what you will get from this podcast membership. So head on over to scrummastertoolbox.org membership and join the community that's shaping the future of Agile. We have so much for you, so check out all the details@scrummastertoolbox.org membership because listening is great, it's important. But doing it together, that's next level. I'll see you in the community.
- [32:19]
B
Slack we really hope you liked our show. And if you did, why not rate this podcast on Stitcher or itunes? Share this podcast and let other Scrum masters know about this valuable resource for their work. Remember that sharing is caring.