← Scrum Master Toolbox Podcast: Agile storytelling from the trenches
Scrum Master Toolbox Podcast: Agile storytelling from the trenches
AI Assisted Coding: Stop Building Features, Start Building Systems with AI With Adam Bilišič
00:37:27
Loading summary
Transcript53 lines
- [00:04]
A
Hey there, agile adventurer, just a quick question.
- [00:07]
B
What if for the price of a.
- [00:09]
A
Fancy coffee or half a pizza, you could unlock over 700 hours of the best agile content on the planet? That's audio, video, E courses, books, presentations, all that you can think of. But you can also join live calls with world class practitioners and hang out in a flame war free and AI slop clean slack with the sharpest minds in the game. Oh, and yes, you get direct access to me, Vasko, your Scrum Master Toolbox podcast. No, this is not a drill. It's this Scrum Master Toolbox membership. And it's your unfair advantage in the agile world. So if you want to know more, go check out scrummastertoolbox.org membership. That's scrummastertoolbox.org Membership. And check out all the goodies we have for you. Do it now. But if you're not doing it now, let's listen to the podcast.
- [01:12]
B
Hello everybody. Welcome to one more episode in this special AI bonus week. And for this episode we have joining us, Adam Bilisic. Hey Adam, welcome to the show.
- [01:26]
C
Hi Vasco, how are you doing?
- [01:28]
B
It's a pleasure to have you. So let me tell you a little bit about Adam, then we'll start right away with the questions about AI assisted coding. So Adam is a former CTO of a Swiss company with over 12 years of professional experience in software development, primarily working with Swiss client clients. He's now the CEO of Node on Labs where he focuses on building AI powered solutions and educating companies on how to effectively use AI tools like coding agents and how they how to build their own custom agents. The link to his company is in the show notes, so be sure to check it out. Adam does a short intro. Tell us a little bit more about yourself and how do you these days define vibe coding? Let's just say for context that we are about what, nine months or even a year after the introduction of the word term vibe coding. So it's a pretty recent thing, but how do you define it? Adam?
- [02:27]
C
Yes, thank you again, it's a pleasure to be here. So this is a very good question because as you said, it's very new term and people often throw it together in one bag with AI assisted or AI augmented coding. Let's start from the vibe coding perspective for me and this is the interesting part because right now everyone is starting to or trying to define these terms and there is no like very global and agreed upon definition. And it's because the technology is changing so fast, right? And one of the things which I would say about how I would define by coding is that the person who is actually creating the app doesn't have to have in depth overview or understanding of how the app works in the background. So they can see and evaluate what they see on the front end, for example, like where the data is showing up or if the specific feature doesn't work well. So they're like essentially like manual tester of their own application. But they don't know how the data structure is within their application, they don't know how it should be. Like what are the best practices, right. They don't understand the security aspect of everything, which is obviously very important. So obviously Wipe coding can get you only so far, but it has also like huge advantages right now for product owners. So in the past product owner would give you just a big backlog of Jira and you would hope that you actually understand what they meant by the tasks. But right now what they can already do is use this kind of like by coding tools and give you sketches of how specific page should look like or would look like. And taking it even step further, this can be used even when you are starting a new relationship with the customer, let's say in your application and you are trying to help them grasp the idea they have in their head and put it actually then not on the paper, but in your computer. And it's much easier when they actually then see what they are talking about and then they can be more specific what they like, what they don't like. And this is a huge case in my opinion for Vibe coding. So this is the current state of Vibe coding and then the other part which is this AI assisted or augmented coding.
- [05:22]
B
So you're making a distinction, right? It's like Vibe coding is one thing, but then there's this, what you call AI augmented or AI assisted coding. And what makes that different?
- [05:34]
C
So the AI augmented coding is different because this is what we as software engineers need to learn. All this new tool set and all these coding agents we have at our disposal right now. And thanks to those, we don't have to now do a lot of repetitive work, which we used to do in the past. But we understand what is going on in the background. And the first level obviously is about starting to write prompts and prompting the coding agent and then checking like the results. But the further levels, as you go, as your knowledge gets deeper, you realize that you can automate much more than you, than you maybe thought you can. And this enables you then to, instead of now building features Change your thinking and start building systems. And that's the fundamental change which you as an engineer can.
- [06:41]
B
What do you, what do you mean about that? Like, walk us through your distinction between features and systems so that we can grasp better what you mean.
- [06:50]
C
So imagine that you have a task, right? Let's say you need to implement some kind of form within your web application and you will be focusing on implementing that form based form based on what was described by your product owner. This is obviously repetitive work and you are just essentially translator of requirements into the code. And you can use coding agent as a chat, you can write there your prompt, like how this form should look like and what kind of like examples it can find within the application and so on. But the problem is that at this point you are now trying to solve only this specific feature or task. When you start building systems, instead of thinking like how can I solve this feature? You are thinking of like how can I let's say create either skill, command, sub agent or other things which these tools offer to then do this thing consistently again and again without the need of your repetition.
- [08:15]
B
Okay, so you mean like you create almost a set of rules or styles or design strategies that then the tool will implement over and over again, even when developing different features, Is that what you mean?
- [08:32]
C
Exactly. So you are essentially trying to get your thinking into reusable, like recipe. That's how I would probably simplify it.
- [08:43]
B
And that recipe can then be implemented in different ways like as, as a configuration in the AI's own configuration tool or as a you called it agent or slash command. Those are all things that current AI coding tools have. But there are more like skills and so on. So it's about kind of what you're saying is like, it's about almost like translating our coding practices into something that then the AI can repeatedly execute for any new feature.
- [09:18]
C
Yes, on one hand it's like one of the first levels is practices. So best practices how the code should look like, what is the architecture that it should follow. But later on you can even start to implement the way you think or how you solve specific problems. And this can relate to let's say refactoring. So so once you figure out how to efficiently refactor specific react component, or let's say NestJS, API, endpoint or SQL query, like this knowledge, if you close that chat and go home is lost. Right? But if you start to think like, okay, so what was my thinking process of deciding like which way to go and how to refactor it? If you document it and essentially create this kind of recipe, then you can reuse it again when you are refactoring the specific thing.
- [10:17]
B
Yeah, it's almost like designing your own colleague, as it were. It's like, because when we work in a team, we have working agreements, we have coding standards, maybe we have practices for decision making. And it's almost kind of replicating that, but through multiple tools like skills and agents and slash commands and so on, so that the AI system can reliably deliver features in the way we expect and also with the right decision points. Because of course, at some point the AI is making decisions for us and some of those decisions we may want to delegate, while others we may want it to check with us before implementing.
- [10:58]
C
Exactly. And you can see this kind of development towards this approach even in the current releases of, let's say cloth code, where it will. Instead of choosing the specific route, like when there is really a questioned, like there is no good practice about the decision, then it really like asks you, and I saw it actually like a few days ago that they even implemented, not only that it will ask you, but if you use the extension, not only the CLI tool through the terminal, it will even show you like the question and the like options like abcd and you will just click on it. So you can see the development of these tools, how it's trying to improve to suit the needs of the developer. It's not trying to replace him, it's trying to really augment him.
- [11:52]
B
Yeah, and actually that leads very nicely into the next question, which is from your own experience, what do you think? And in your experience, what has actually helped you make AI assisted coding work well for you? So we've talked about knowing about and using these tools like MCP server skills and slash commands and whatever else will come, because these things are changing all the time. But beyond that, because that's very high level, what are like the concrete coding practices that you have learned from experience really work for you? And you have taken and used in all, or perhaps only most. But anyway, the most critical coding projects you've done with AI.
- [12:38]
C
So my journey with the coding agents and the whole augmented coding started very early because I actually built my own coding agent when the first GPT came out and also GPT API. I'm talking obviously about the. I think it was 3 or 3.5, which came public and already started to change the game. And that was the point when I started to ask myself like, okay, can I delegate? Or how much can I get out of this new tool? And Automate what would be just repetitive work, right? So I didn't become a software engineer. I didn't start coding when I was 14 because I wanted to write text. I was just amazed what I as a kid can build with these tools which are at my disposal. So. And I really much doubt that a lot of engineers are there just to write text. They want to solve problems and they want to build things. And so as I was like building up this first concept of the agent, because I was like, okay, so just using regular chat interface was not enough. I started to build my own template system which had loops, which had pattern recognitions and reactions. And as you go through this, you start to understand in depth aspects of this, even modern coding agents. And there because you have to overcome them, right? Like you have to overcome, especially at the beginning, the very limited context size. I remember like I was fighting with that a lot and tried to find the right solutions, like how to manage it. Like you could not simply get changes within your file and it would spit out the whole file especially it was too big or put into context multiple files. So you needed to start to at the beginning like make your own techniques how to overcome it. And this is where I learned that and started to understand that managing the context is everything, right? Like you need to know what, how first of all, like how much of the context is filled. So because the more of the context is filled, the higher the likelihood is of hallucination. You need to know like at which cases it's good to reset the context. What is good to learn is also like to get into the context only the right information or also get out of the agent only the information which is for you needed. And obviously the second part is of this whole experiment before these coding tools exploded and overtook the market was was prompting like it sounds very basic, but when you learn how to write good prompts, you start to get much better results. Obviously right now the agents are much more sophisticated and they have contact between those much larger. Oh yes, of course, yes it is. It is obviously much larger. But they also have this chain of fault, like okay, so what user actually wanted? What was he asking? They try to like find out. And if they don't, if they are not will ask you, right? But it's good to understand like that how it works in the background and how to write the good prompts which then yield you better results. And then the level further is that it's not only about writing the prompt itself, it's about reusability of the prompt. You don't want to be writing the same prompt again and again all the time. Right.
- [16:35]
B
But now, okay, so you're talking about context, which is the context window of the LLM. So how much information can it process at the same time in one prompt, basically? And then you're talking about the prompt, which is kind of an interaction with the LLM that uses both the prompt, the context window, and then of course the LLM's own weights. So what have you learned specifically, like when you're doing this AI assisted coding that helps you to think carefully about how you're managing the context window and also how you're either defining new prompts or reusing old prompts. Like what, what are some of those practices that you apply today with these concepts in mind?
- [17:23]
C
So let's, let's say one for, I will, I will say one for the management of the context and one for the prompting. Okay, Just so we have some kind of actionable tips for the listeners. So one which would be for the context management is that right now, as you probably heard about Contact or like concept of mcps, people have this tendency to install everything which they see on Reddit and they never check like what, what is then loaded within the context just when they open the coding agent. And you can actually check it by, I think in cloud code is slash context. And then suddenly you can see like, oh my God, like even 40 or 50% of my context is now taken just by MCPS. And I. And you, you didn't do anything yet. Right? Which is, which is then once you realize it, the concept of sub agents is amazing because when you create a sub agent, then you can specifically define which MCPS it will have available. So if your orchestrator is calling a sub agent which is focused on front end, it will need let's say playwright MCP to open the browser and go through it. But if it's a backend agent, then it doesn't need it. Right? So this is like a very tricky thing and something which you can use right away is to take a look at what kind of MCPS you have installed and how much context they are eating and do you actually need them for the specific thing you are doing right now. And then the, the part about the prompts is that what you'll. What, what, what I preach and what we teach other companies is that you should create specific template of the prompt which you then like the structure which you then reuse across all of your prompts. First of all, it helps you define like what these either command skill, sub Agent or whatever it is like what, what is it doing? Right. And second of all, it will, you will not forget about all the other aspects of what the agent should follow, either if it's like how that it should test its work in the end. Right. Or what kind of MCPS it should call in which cases. So this kind of structure is very important. And it's also when you start to scale it within teams, which we obviously, which we obviously see with our customers, they, they need to be unified or the whole code base becomes a huge mess. And that's kind of where we are right now, that many companies like experiment with this stuff in a way that they maybe allow developers to use whatever and however. But there needs to be some kind of order in this, in this mess in order to be sustainable.
- [20:38]
B
Exactly. And we are all learning how these tools will be used in the future. This is why we're doing these interviews and interviewing people like you that are actually using them. But I'm sure you have some ideas of how this will evolve. So when you look at how AI assisted coding is being used right now, what trends do you see and where do you think this approach is going in the near future?
- [21:05]
C
So I see different types of people and different kind of categories or levels of where, where they are at right now with this AI augmented coding. So what I saw from the beginning is that there is a huge, there was at least there was a huge resistance with software engineers against AI. And I understand it because the first results were not great and it could do only very limited things and the developers could say like, oh, I can do it already much faster on my own, I will not use AI. The problem with that is that if you're looking at things what this kind of evolving tool cannot do, then you are missing out on what it can do and all the knowledge which you can gain by using it. And so these people really missed out on like in last two years, or maybe it's already even free on learning a lot of skills which then would be helpful right now. Right. And still there are software engineers that's like the category zero, I call it, which just are really resistant and don't want to use these tools at all. And I think that if they didn't caught up in the last or changed their opinion in the last two, three years, I would say that we are getting to the point where it will be kind of last chance to do so because the technology is evolving so fast and the companies usually don't already hire people who don't have These skills. So we are getting really kind of. It's strange territory, but we are there. So this is like the category zero, the people who are completely resistant. And then we have like, I define them as like five levels of augmented coding. And I actually also prepared for your listeners like a PDF where I describe it in much more detail. And it can really help you to understand like on which level you are and what you can and should learn to get to the next level and what these levels then actually offer you in the end, when you learn how to master this augmented coding. So I think we will put the link in the description.
- [23:30]
B
Yeah, absolutely. The link is in the show notes. So just check it out everybody.
- [23:34]
C
So the first level, I call that it's like a master of prompts. So this is where everyone starts, right? So you start to. You are now convinced that, okay, AI coding is here to stay, we need to learn how to utilize it. And obviously you open the chat or like a CLI and start to prompt the agent. And here you maybe start to see like, okay, so with some specific prompts I get better results. With some specific prompts I get worse results. So it's kind of all over the place. And this can be also very demotivating, especially at the beginning when you start and you don't have systems built. It can be that you spend then in the end maybe equal time as if you would be doing it by yourself. Not. Not really anymore. Especially if you use like let's say latest cloth code, Opus 4.5. That's as the. As right now of the recording of this podcast, the latest model. And it's amazing for coding. But you are, you are kind of on that like starting level where you realize that when I'm implementing a new feature, I always have to remind that I'm using SheTCN and node material UI and don't mix those two. And my architecture of the application looks like this. And why did you put it in this folder and why the code looks like that? So those are really the drawbacks of this first level, which is like this kind of prompter, like the developer who learns how to prompt, but then he. He doesn't progress any further with just prompts because it's not possible. You need to start thinking bigger. And this is where you start to take a look more at the documentation and start to take a look at what kind of settings can you use.
- [25:42]
B
Basically what you're saying is that if we stick to the prompt level or being good at prompting, it means that we are keeping Every past information, the best, the lessons learned and all of that, we need to keep that all in our memory and we need to remember to type it every time we type a prompt, which can be of course tedious and it can take a long time to do those quick interactions with AI.
- [26:07]
C
Yeah, I think you summarized it very well.
- [26:09]
B
So what's the next level like? When you think about a prompt being level one, what's the next level?
- [26:15]
C
So the next level is obviously that you take a look at the configurations. Right? So you will take a look.
- [26:21]
B
And by configurations, what do you mean? Can you give an example?
- [26:25]
C
So every coding agent, like for example, you can use, let's say something built in into your ide. The IDE by itself is cursor. That's very popular, even though it has had its ups and downs with pricing and people. People paying too much money unintentionally for, for their services. But hopefully those times are gone. And they have it right now under the reps. But they have specific file which is called cursor rules. And there you can define like what kind of rules the agent should follow every time when you prompt him. Right. The advantage is also disadvantage. So this file is always included within the context. Yeah.
- [27:15]
B
And the bigger ends, the worse the prompting becomes. Because then there's too much context, not enough.
- [27:21]
C
Exactly.
- [27:22]
B
Based for the code itself.
- [27:24]
C
Yes, exactly. Or for other information which the agent needs to actually then implement the features. So in this case you have also the same thing for Claude code. There is like a cloud MD file. And there you define these rules. And here you. Essentially the problem is that this file then starts to grow right, on this level and the developer starts to just add stuff and then it just overtakes the whole context. So then you move on to the third level, which is mastering the context. Right. So here you understand like, okay, so maybe cloud code file should be smaller. Right. It should not include all the information about my code. There you learn how to manage context better. So let's say, and also augment it with mcps, which we already mentioned. Right. Like you learn like which MCPS to use when. And this starts to make a significant difference because you essentially enabled the agent to have eyes to have access to information which it needs. Right. So this is the third level, like this kind of mastering of the context. And you start to create also markdown files which then store important information and you can reuse them and you start to spread the word, let's say maybe even through multiple agents, still manually. Right. So then the next level above this that's fourth is the mastering the automation. So now at this point, you really don't want to be prompting all the time and opening like new tabs and like spinning up the new coding agents. So what you learn is that these tools have evolved quite a lot and they support things like subagents, like creating your own custom commands and skills and when to use them because they have very specific behaviors. For example, subagent cannot spin up the sub agent. So if you would create the orchestrator sub agent and you would be like, oh, then use this sub agent when you need to implement front end feature. It would not do it because it cannot. But funnily enough, it will not tell you that. It will start to simulate it. And those are then complex completely different concepts and you can expect. Com. Expect completely different behaviors. So there you really start. You. You need to learn like how to. How to then set everything up. And so it's reusable. So then you already delegate. You are starting to create your own team of virtual developers with specific focuses.
- [30:15]
B
It's important to reiterate that like it's. It's not. Not a team of orchestrators, it's one orchestrator that uses a team of narrowly focused agents.
- [30:26]
C
Yes. So and there, there you are already going to the, to the last level which is exactly I call like the orchestrator. Right. So that one is already. That you don't have to orchestrate and specify in the prompt like which sub agent to call or which what. What they should do. But you essentially create one orchestrator which is. Which has access and awareness of these sub agents. And the great part about this is that the sub agent he runs in his own context window so it's not polluted by whatever the. The orchestrator was doing. It's also tricky because you need to. Orchestrator needs to give him enough information.
- [31:07]
B
So the Right. Correct.
- [31:09]
C
Yes, exactly. But once you master this, then it's amazing. They do their job, they return you then tell you, okay, so here are the results. They are in the files. And then the orchestrator can be like, okay, so now let's spin up the tester and he will go upon playwright again using the MCP which only he needs, let's say not backend agent if there is like a backend change and then he can go and test it. And if there is something wrong, you will return it to the orchestrator and Orchestrator will be like, oh, there was a mistake. Let's spin up again front end agent to fix it. Right. And it gets complex. But you are now creating systems and orchestrating. And obviously it's still important that you check what the agents are doing and monitor them and even sometimes step in and like okay, so this is not the direction where I wanted to go, but it's evolving so fast that.
- [32:07]
B
And it's important to be able to backtrack, by the way. And that's a practice you should use with all of the five levels, not just the last one. Use git or whatever version control you have and always backtrack when something goes wrong because it might be well. And that happens to humans as well. Right? We get into a change path that eventually becomes so deep and so complicated that you have to just backtrack and get back to where you were and try again. Adam. Okay, go ahead, go ahead.
- [32:39]
C
And finally enough. I just wanted to mention that as developers are looking at the vibe coders like oh, that's bad and dangerous and so on, as we are speaking, they are learning how to use git so they are not clueless already to version control. And this is interesting because this will evolve in the future and it will be something completely different, maybe more safe, maybe more reliable. We will see.
- [33:11]
B
Absolutely. We'll be here to see how it goes and maybe talk with Adam again and do a quick report for all of you adventurers out there that may want to dip your toes into AI assisted coding. Before we go though, Adam, I did want to ask, is there a resource, could be a book, a video, a YouTube channel, whatever that you would recommend for people that want to get deeper into this AI assisted coding topic.
- [33:38]
C
So I would for sure recommend if you are interested in cloud code, obviously official documentation the cloud code subreddit and there is also very good youtuber which is called indie dev then and he is making videos and tips like how to especially like build very good prompts and also inspires you like what kind of agents and skills and commands you can build. So that's very, very interesting and I for sure recomm to check it out.
- [34:10]
B
Absolutely. And we'll put the link to all of those in the show notes just to make sure that everybody can easily find them. And Adam, how about you? Where do we find you and learn more about the work that you're doing.
- [34:23]
C
So you can find me on LinkedIn so just search my name Adam Bilisic and I should pop out because I am also writing there regularly posts educational ones like what to use and how to use in this augmented coding area. So if you are really interested in that, feel free to give me follow or you can also just check our website nodionlabs.com yeah, absolutely.
- [34:49]
B
So check that out. Why not interact with Adam on LinkedIn, ask a few follow up questions. I'm sure there are plenty after this episode. Adam, it's been a pleasure. Thank you very much for being with us and for being so generous with your time and your knowledge.
- [35:04]
C
Thank you Vasco, it was a pleasure and thank you.
- [35:11]
A
Alright, I hope you liked this episode. But before you hit next episode, here's the deal. This podcast is powered by people like you, the members who wanted more than just inspiration. They wanted real tools and real connection to people who are practicing Agile. Every day we're talking access to over 700 hours of agile Gold, CTO level strategy talks, Summit keynotes, live workshops, E courses, Deep Dive interviews, books, and if you're into no Estimates, we got the pioneers of no Estimates in those Deep Dive interviews as well. Agile Business Intelligence, creating product visions, coaching your product owner courses, you name it. You'll get invites to monthly live Q&As with agile pioneers and practitioners, plus a private Slack community which is free of all of that AI slop you see everywhere. And of course without the flame wars. It's a community of practitioners that want to learn and thrive together. It's the best place to connect with community and learn together.
- [36:27]
B
So if this podcast has helped you.
- [36:29]
A
Before, imagine what you will get from this podcast membership. So head on over to scrummastertoolbox.org membership and join the community that's shaping the future of Agile. We have so much for you, so check out all the details@scrummastertoolbox.org membership because listening is great. It's important. But doing it together, that's next level. I'll see you in the community.
- [37:01]
B
Slack we really hope you liked our show. And if you did, why not rate.
- [37:07]
A
This podcast on Stitcher or itunes.
- [37:10]
B
Share this podcast and let us other Scrum masters know about this valuable resource for their work. Remember that sharing is caring.