← Scrum Master Toolbox Podcast: Agile storytelling from the trenches
Scrum Master Toolbox Podcast: Agile storytelling from the trenches
BONUS How AI Is Reshaping Software Teams From the Inside With Dwarak Rajagopal
00:36:20
Loading summary
Transcript57 lines
- [00:04]
A
Hey there, Agile adventurer, just a quick question.
- [00:07]
B
What if, for the price of a
- [00:09]
A
fancy coffee or half a pizza, you could unlock over 700 hours of the best agile content on the planet? That's audio, video, E courses, books, presentations, all that you can think of. But you can also join live calls with world class practitioners and hang out in a flame warfree and AI slop clean slack with the sharpest minds in the game. Oh, and yes, you get direct access to me, Vasko, your Scrum Master Toolbox podcast. No, this is not a drill. It's the Scrum Master Toolbox membership. And it's your unfair advantage in the agile world. So if you want to know more, go check out scrummastertoolbox.org membership. That's scrummastertoolbox.org Membership. And check out all the goodies we have for you. Do it now. But if you're not doing it now, let's listen to the podcast.
- [01:12]
B
Hello everybody. Welcome to a bonus, very special bonus episode of the Scrum Master Toolbox podcast. And joining us is Dwarak Rajagopal. Hey Dwarak, welcome to the show.
- [01:24]
C
Hey, great to be here.
- [01:27]
B
So Dwark, you've spent quite many years in the whole AI field. I mean, you've been at Google, meet Meta, Uber, Apple, and now VP of AI Engineering and Research at Snowflake. So today we're going to talk about something that hits close to home to every software team and of course the people listening to this podcast today, which is what happens when AI agents become part of the of the development process and some would argue part of the team itself. So you've taken this front row seat now working at Cortex AI at Snowflake. Before we get into the questions, tell us a bit more about your background. Like how did you even get started with the whole AI in software development topic?
- [02:14]
C
Yeah, that's a great question. So I mean to. Quick intro about myself. So I lead AI engineering and research at Snowflake. I've been working in the areas of AI for the past two decades, actually in companies like Google, Meta, Uber, often in the intersection of generally the research and production. What's become, I guess, clear to me is that in AI, the hardest part isn't just the models itself, it's just making them use in real environments where data is messy, fragmented, governed. And that is something we think a lot every day across our AI engineering and research teams. Back I started my career as a compiler engineer, which is about 20 or more than two decades ago. I used to be an open source GCC compiler engineer worked on I guess at that time one of the critical elements is like when the hardware is getting faster and faster, how do you kind of feed it and kind of optimize it. Then I worked on graphics after that at Apple and then from there about 2011, 2012 when Alexnet started running on GPUs which is kind of like our first neural Networks. Running on GPUs is when I got into AI where optimizing AI workloads for GPUs within Apple graphics software stack and then worked in Uber on self driving cars for a few years. Then worked in Meta where I led the Pytorch core team which is one of the open source AI frameworks in the industry. And then since then worked at Google, worked on some of the other frameworks like Jax, TensorFlow, Pytorch and also getting like Gemini training on TPUs. But like I said, I think this is all more on the model side. And one of the things that was very exciting and kind of I think two years ago for me was that how do I use these models in like real world enterprises. And one of the critical element is data which is generally very messy and it's kind of like the oil for the engine in some sense. And so that was one of the reasons why I started working at Snowflake, which is kind of like the place where we have a lot of enterprise data. And so how do I kind of give value, provide value for the enterprises from here? So that's my quick background on how
- [04:41]
B
I started a lot of potential threads right there in what you just shared. Now you're someone who builds even AI tools for developers to use like Pytorch as an example, I'm sure many more. So I have to ask how is AI already changing software development as a process for you and your teams? And I mean not the hype, but what are you actually seeing inside the engineering organizations that you work at and right now in your team of AI engineering in Snowflake?
- [05:14]
C
Yeah, I mean AI is definitely accelerating development. I mean that's kind of a no brainer at this point. But the biggest shift is that engineers are spending more time on system design, validation, production, reliability. The real value generally comes when AI is grounded in enterprise data context. So it becomes it's important to support real workflows not just generating code faster. So that's one of the key elements that what we are seeing is that engineers spending now more time actually doing more thinking in terms of how to design systems, validate and Less time actually doing the implementation itself because AI is helping that on that front. Even at Snowflake, what we see within our teams are like the teams use tools like Cortex Code, Snowflake intelligence and other LLMs to generate more codes and tests because the cost of the friction of development has dropped so much. And that kind of gives us a big kind of speed in which how we can go to try out many ideas. And we see that in our customers as well. For example, Whoop, which is the fitness band company where they're actually using Cortex code with conversational data assistants and agents to kind of reduce the development work cycles from weeks to hours, freeing teams to focus on high value work. What is also interesting here is that when the team spend more time doing design, they also need to spend more time doing evaluation platforms. Thinking about how do you kind of get the AI get the LLMs work longer, longer horizon tasks? I kind of focus on those kind of problems where you can do more things parallelly. Context switching is a problem in the sense that now engineers have to do more context switching, but it kind of lets us to do more things faster is kind of like the key element here.
- [07:05]
B
Okay, so the acceleration of delivery is something we've heard before and we've done a series on AI assisted coding here on the podcast, again with first person real life stories of how this is already influencing massively how very experienced developers develop in fact their workflow. But it also changes how companies think about the economics of software. Team sizes, of course, timelines as you discussed, but the structures around the work that needs to be done around the workflows also change. What are you seeing in the teams that are working in your organization as well as potentially some of your previous organizations? How are you seeing the real life in the trenches impacts of AI software engineering?
- [07:54]
C
Yeah, I think it's a great question. I think the workaround changes. It also changes the team dynamics as well. One of the things that I see is that it amplifies how teams already works. So clear systems ownership communication becomes even more important. Teams that are strong in actual structure and data context generally are able to move faster because AI can operate them within those systems. In practice, this means basically when teams actually start to actually look at problems, they want to look at ways where they could kind of structure the problem down into different elements and then actually let things more go in parallel. The other thing, what it also helps is that sometimes we have to make choices in the past where we need to decide either we should do this way to design or this way to design systems and because it usually is bottlenecked on the time it takes to actually build these systems. Now what we have seen is that teams actually are able to go, okay, let's try both and actually do two different designs and do two different things in parallel and then decide which one. So basically there's a lot more chances of like you don't have to think about this versus that do both and then figure out what is the best. But this also means that teams have to be ready to kill stuff more faster because that's also sometimes as a software engineer myself, you tend to put a lot of kind of personal your baby, right? It's your baby, right? So here now you need to figure out how to kind of kill some of the projects faster because you are also building more things. So that's a very interesting dynamics that I've seen as well.
- [09:41]
B
So in those teams that are already embracing AI coding agents today, what's working and what's breaking? Can you give us an insight into what you've seen in the field?
- [09:54]
C
Yeah. So things that are generally working is, I think there's a thin balance between platformization, which is doing things multiple. One example is that we have a mobile app that we are building for Snowflake Intelligence. And there are these different choices whether you want to build a platform and then write one and then use both separate app for Android and iOS usually. And people have used Flutter, React, native and other frameworks to do this. But now there is a clear choice whether you can actually build two native apps now faster. So things like that being able to kind of do, being able to have an open mind to kind of do those things. Generally you now can build native apps for iOS as well as Android kind of from grounds up at the same speed the same way and kind of manage these across agents. So you kind of build once and then ask the agent to kind of go and build the other version and then be in the loop. So that generally works quite well. From our experience, things that are still challenging is that when the teams are bigger, there is a lot more coordination that needs to be done. Now what do you mean in practice?
- [11:08]
B
Can you give an example of that?
- [11:10]
C
Yeah. So for example, one of the things, I mean we have done many experiments. One experiment is that we used to have functional teams where there is a backend team, there is a front end team and there is a feature team. Usually this is how generally when you want to build a feature, you kind of get from the feature team to define, hey, this is what the requirements for the back end, this is what we need from the front end. And there is a dependency across these different teams. What we've seen generally is that now instead of having separate functional teams, what we've tried was having project based teams where like there is one backend person, one front end person. The thing is that we, there is a lot more things you can build vertically end to end faster than to kind of do it functionally if you arrange, if you arrange the teams that way. So that's one kind of interesting way to kind of see that. For example, the things that brought people Snowflake Intelligence was essentially built in that methodology.
- [12:14]
B
So basically what you're saying is like smaller teams but cross functional, so they cover the whole stack with a smaller team. Is that what you mean?
- [12:22]
C
Exactly, exactly. So what this means is that you could build multiple products now faster with the same but different smaller teams across these things.
- [12:35]
B
The opportunity cost is basically disappearing with the help of AI. So you can do more things even if you know that some of them might not really deliver the value. But that's okay because you're trying out multiple bets and figuring out what actually works in practice.
- [12:53]
C
Yep, exactly.
- [12:55]
B
So one thing that when you think about these teams, the increased coordination cost as you say, like for example, just having multiple things going out in parallel already increases the coordination cost. Many of these things kind of drop into the team lead or the engineering leader stable as. Okay, but now I need to recruit people that can actually do this. Right? Like last year I went out and I recruited a front end specialist, a backend specialist, and maybe we had a couple of full stack developers to kind of, you know, be more agile, more nimble and adapt to the projects coming in. But if AI can write the code, what do we actually hire for? Now you're a VP of Engineering at Snowflake. So this is a question you need to ask yourself. So I'm really interested. How have you tackled this question yourself?
- [13:45]
C
Yeah, great question. I actually strongly believe the fundamentals matter more than ever now, especially systems thinking, judgment, the ability to work with complex data and production systems are super critical. AI can generally assist with execution, but the teams will still need people who can actually define the problems clearly and ensure systems behave correctly at scale. So as AI becomes part of the workflow, we look for engineers who understand data context end to end system, not just code. One example here is that before we would ask engineer to come and code a particular specific search algorithm to see whether they could do, for example, if we hire a search engineer for search systems, now we ask them to build a whole search system because you could actually go build the whole entire end to end system within an hour. And so this kind of helps us to actually kind of understand whether the person fully understands the end to end system defining kind of structural thinking. So I do think it's super important from that perspective, from a research perspective, the focus is shifting towards building systems that combine reasoning, context, execution, which requires strong engineering discipline. So it's actually what is kind of interesting is that before it used to be like researchers kind of do one specific kind of deep area, which is still the case, but then now they could actually go and like test this across a product much more faster. So I think the fundamentals of structured thinking, critical thinking is like super critical.
- [15:26]
B
So what you're saying, if I get you right, is that the ideation to development and production deployment cycle is kind of compressing into one person or at least a small team. Is that what you're saying?
- [15:42]
C
I think what I'm saying is I think we could do more things now. So for a specific thing you could do it with lesser people because you could basically kind of ideate faster but then you could do more parallel things and basically it kind of accelerates the number of products that folks can go and build. And you could kind of test it. I've seen interviews where a person actually goes, builds end to end system, which actually almost like production ready, which is kind of like amazing to actually see these things working.
- [16:18]
B
Absolutely. Of course from the perspective of the people joining these teams, it completely changes the expectations for a junior developer. So for the junior developers out there that are thinking of a career in maybe even working for Snowflake at some point, what advice do you have for them?
- [16:38]
C
Yeah, so to be clear, like we actually hire a lot of junior engineers. We've actually seen a lot of kind of ways junior engineers are able to kind of ramp up very fast. One of the things what we have done is focusing on like, in fact we have like onboarding is becoming now much more easier because now we have like skills. So our team specific skills are being built. And so if a junior engineer comes, they could actually go. I think the rate at which generally an engineer kind of grows from building a small system to building end to end production system is now much faster. That I do see junior engineers kind of accelerating in their doing things. In terms of advice, I mean the best way to kind of do these things is that now the ability for junior engineers to actually go build things are much more easier. The cliff to climb is much more smaller before you need to kind of actually work in a company to actually go do these things and understand you could actually now do it at school, starting from school, where you could go build these things end to end. So I do think that's like a critical element, building, learning by building is like more true than ever now and it is much more easier to go build as well. So I do think that's kind of going to be like the critical element for engineers to kind of grow.
- [17:58]
B
So what you're saying is don't be afraid, get your feet wet, try to build a system end to end, whatever that is and get used to the idea thinking from idea to production and operation. Really?
- [18:12]
C
Yes.
- [18:14]
B
So which skills do you think are going to become more valuable for teams? So of course for developers, but for a team in general, like what do you think are the core skills that a team needs to make sure they have within their skill set to be able to thrive in this AI era?
- [18:32]
C
Yeah, I think one of the critical element is because like the, the models are getting better, the field is changing very fast. It's important to constantly how do you keep track of what you're learning as a team and being able to share across the team. And one of the things that like I mentioned is that we have these team specific skills that we actually build and kind of iterate almost every week. So when someone in the team actually learns or does something, we automatically kind of pass this and update the skills for everyone else within that team. And so having that kind of inner loop is super important. The other critical part is most of the kind of the ability to do things is actually bottlenecked on data in terms of what is available. So if somebody who's been there for a longer time in a company, they have a lot more tribal information. And what it actually means is that they have a lot more data across different systems that they know of. And one of the things that what we have spent at least both within Snowflake and also for our customers is to actually building the context for each team and what data that they have automatically from both structured as well as unstructured information as well. So that's one of the key elements where like how does the team be able to get all the information ready and being updated continuously? That's one aspect. The other aspect is how do you kind of do a lot of keep the lights on things more automated. So there is one of the key elements in the past is that you need to, especially if your Team is like running a service. Usually 40 or 50% of the time is mostly spent on operational stuff because you are actually serving this for thousands, if not like a lot of users. And what AI has done is that it kind of helps you kind of building run books easily. And once with that, you could kind of do a lot of these things more automated. So that's one of the things that I've seen. Teams which actually go and do those things more faster are able to not just kind of do more interesting things, but also have lesser issues as well. In terms of like the build issues and in terms of like the how you kind of validate. These things are much more faster if you can kind of spend time to doing this again, these things are also changing very fast that you need to have. What we have done is we need to have a specific person who is focused on operational aspects of things, on using AI tools. Operationalize that. Similarly, on the development side, we have a specific focus person doing that within the team. So there's multiple aspects of these things.
- [21:13]
B
Yeah, absolutely. There's so much to learn. And of course one of the areas that we need to learn more about is what happens every day inside a team. So in your own teams over there, in your department at Snowflake, how has the daily work changed? I mean, do they even have standups anymore? How is the normal daily work for an AI native team today?
- [21:37]
C
Right. So the way we do is that we have every Monday we have a. The team actually comes together, decides what to do for the week. We of course have like quarterly okrs, yearly kind of goals and all that stuff at a higher level. But every week we have have this Monday figure out what to do. Friday we have demo days essentially. So this is. And it's not just demo, but also like testing. The team actually goes together, sits in a room, what they built, test it and figure out what all the things that are actually missing. So it's actually, if you think about it the way it used to work before sprints, we used to have like features for a couple of weeks. Then there's like a test sprint week. What, what's happening is that you basically decide what to do on Monday and you're kind of like testing actually together as a team on Friday and getting the feedback and giving it back for the next week. So do I think stand ups are definitely there, but it's usually happens at a much more faster kind of loop overall.
- [22:46]
B
So. So you're saying that standups happen, but they might have several stand ups a Day, because the. The call it Sprint cycle is now compressed to a week or so.
- [22:57]
C
Exactly. The other thing we also see is that. So we have. So we have teams across different geographies as well. And so I have teams both in Bay Area, Seattle as well as in Poland, for example. What we have done recently is automatically getting. So we. So the teams working in say, for example in the US Working in the morning, by the time they finish automatically, there is a skill that actually goes through the code and kind of summarizes what has each person done in a joint Slack channel. So by the time the folks in Poland come, they know exactly what has happened and what are the things that they could. So there's like a pretty automated way of doing these things more faster. So every day people know what's happened. So it's a great way to kind of understand, to kind of reduce the kind of also the overhead in terms of communication in some sense.
- [23:57]
B
Yeah, that sounds amazing because I mean, one of the core things that we've always struggled with in distributed development is of course, when you have multiple time zones, you end up having a big gap in communication. And it sounds like you guys are already thinking about that and putting in place some tools that kind of try to cover that gap so that the work can continue with minimal disruption in different time zones.
- [24:22]
C
Absolutely, yes.
- [24:23]
B
So I think it's the time of the episode Dvarak, where we need to look at the future and kind of point to where is this going? And you've been at the frontier of AI and of course also recently AI engineering for 20 years. Give us your honest read. Where is this heading for software engineering? And I don't mean like, you know, 20, 30 years. What is happening right now in the next two, three years? What do you think will be the big trends to watch out for?
- [24:54]
C
I think we are moving from AI as a tool to AI as a part of a system where it supports more and more of our development lifecycle. The focus is also shifting from just getting answers from AI to actually enabling actions. One of the key elements for data engineers and data scientists is working across both structured and unstructured data across these elements. So how do you kind of do it in a more automated way? Those things are going to actually happen more. For example, within Snowflake, we have, with Snowflake Intelligence, our users can ask questions in natural language and take actions directly, whether that is running or triggering workflows. Similarly, with Cortex code, you could go build data pipelines automatically. So I do think, and this is something not just we are seeing within internally, but also seeing with our customers as well. So this is in terms of things that are happening is that there's going to be more and more reasoning and system level capabilities that AI is going to kind of work at scale. So that's one of the things our research team is also heavily focused on as well.
- [26:14]
B
Yeah, absolutely.
- [26:15]
C
This is going to be an exciting couple of years.
- [26:17]
B
It's so exciting. So we had Philip Hsu, who used to work at Amazon and Meta also in the past on the show and he had this provocative idea. He called it lights out code bases. Code bases where humans don't go anymore because AI can do at least as good. And some would argue with the mythos bruja about security recently, perhaps even better than humans could do. What do you think about that idea of lights out code bases?
- [26:49]
C
I do think definitely in some areas where it is pretty clear where things are going to be. There is already examples of things that have been done. I think the AI could automatically do these things and you have code bases. And there is also the other aspect where any code base that even doesn't matter if it's AI built or human built or mixture of this is that having this MD file everywhere. And so generally AIs would basically use it. I do strongly believe that there is going to be more agents actually writing. I think this is probably already true in some sense, agents writing more code than humans already. So there will be source code where there's a lot of that stuff being done by agents. But there are challenges in this aspect here. I mean one of the key elements to kind of do these across enterprises that do you have governance, do you have secure governed way to actually use it? And that's one of the critical elements in my opinion for us to get to that lights out stage. Because you need to have observability, you need to have traceability, what actually happened. And also there are things where especially if you want to take actions, if you want agents to take actions, which of these actions cannot be taken back? Or do you have this concept of committing actions or rolling back? So I do think there are these systems that are, I mean it is going to be going towards that direction. But I think those are the things I do think is like important for the enterprises to actually go and use this at scale.
- [28:24]
B
Okay, so one last question. There's a lot of people thinking about how this is going to develop. I myself, I'm also doing that with my clients and also in the work that I'm Publishing, but you have the hands on like you're working with this every day. So what do you think that most people in the industry are getting wrong about AI and its future impact on software engineering and software teams?
- [28:52]
C
Yeah, I think one of the critical elements, what we've seen is that it's very easy to build prototypes, it is very easy to go actually build even end to end systems. But it's very hard to actually get it to working in like enterprises where the data is so messy. There are data across the elements, for example, that is invoice data, you have data that is coming from your factories. How do you combine these things and actually give insights to actions for the AI is one of the key challenges, the big challenges that we actually think a lot about at Snowflake. And we built systems like context code, stuff like intelligence and products like that to kind of understand and do this in a governed, execute, in a governed execution, interoperable and control. I do think that is one area where I often see people hitting bottlenecks, roadblocks when they want to scale this through across the organization or across the industry as well. And I think that is something super important to kind of focus on.
- [30:01]
B
And very much in that vein. What could we be doing differently? Is there somebody whom you think they are seeing this more clearly? They have some solutions proposing who is someone or a resource, a book, a video, a course that you could refer us to that you think would be useful as we map out this new territory.
- [30:26]
C
Yeah, I can give comments on any book, but all of these become stale pretty soon because the things are moving very fast. Like I said before, I think the best thing is to actually use the latest tools across these companies. Like if you are into data, I would highly recommend folks to give it a try on partax code which is very easy to go start using it. Essentially it will automatically understand your data pipelines and also across your data science workloads and stuff. That's one example. Of course using all of the LLMs from other LLMs is super critical. I do want to, I mean it might sound boring, but the way to learn is basically building. That's probably the best way to learn here.
- [31:11]
B
Go build. You heard it here folks, go build. And isn't it wonderful that we're seeing a resurgence in the importance and of course also the impact of software. And as people talk about the potential impact of AI in many different professions, which no doubt it will have and we don't even know how now for me it just looks like we will have a Lot more software in the future than we have today. What do you think about that?
- [31:38]
C
D Absolutely. I think that one of the key elements what AI does for software is that it makes it easier to customize for your own self. So I do think there is going to be the place where there's going to be a lot of personal software or personal customized, easily customized software that's going to come out and it's going to be easier for everyone to go build it. And that's kind of like the most exciting part.
- [32:06]
B
Yeah. Coming back to the data, like, one of the things that I see at the clients I work with is that everybody's building, salespeople are building, managers are building, and of course they're all talking about data, where to get it, how to get it, how to manipulate it. It's all going back to again.
- [32:23]
C
Absolutely. I think we definitely see, like some of the business folks that enterprise companies, our customers have. They are actually now becoming more builders. They don't have to go and like, wait for some other team to go build it. They are actually able to go build it themselves. And it's kind of fascinating to see, especially when we shipped Cortex code, it was kind of very, very fascinating for us to see how different ends of different work functions are using Cortex code, which you would have never thought about even a year ago, to go build systems that actually without even fully need to know about the actual coding aspects of things. And that's kind of what democratizes here. Yeah.
- [33:12]
B
And that's already an example of lights out code basis right there. Dvarak, thank you very much for being here with us. Before we go, where can people find out more about you and the work that you're doing at Snowflake?
- [33:26]
C
We have it in snowflake.com there is specific look for Snowflake intelligence Cortex code. These are the things where the teams are shipping almost like everyday new stuff here.
- [33:41]
B
Absolutely. And we'll put the link to those in the show notes. Do check out Dvarek's team's work and of course, interact, give comments. We're all exploring the whole AI landscape. Darek, it's been a pleasure. Thank you very much for your generosity with your time and your knowledge.
- [33:59]
C
Thank you.
- [34:04]
A
All right, I hope you liked this episode. But before you hit next episode, here's the deal. This podcast is powered by people like you, the members who wanted more than just inspiration. They wanted real tools and real connection to people who are practicing agile. Every day we're talking access to over 700 hours of agile gold CTO level strategy talks, Summit keynotes, live workshops, E courses, Deep Dive interviews, books, and if you're into no Estimates, we got the pioneers of no Estimates in those Deep
- [34:43]
B
Dive interviews as well.
- [34:45]
A
Agile Business Intelligence, creating product visions, coaching your product owner courses, you name it. You'll get invites to monthly live Q&As with agile pioneers and practitioners, plus a private Slack community which is free of all of that AI slop you see everywhere. And of course without the flame wars. It's a community of practitioners that want to learn and thrive together. It's the best place to connect with community and learn together. So if this podcast has helped you before, imagine what you will get from this podcast membership. So head on over to scrummastertoolbox.org membership and join the code community that's shaping the future of Agile. We have so much for you, so check out all the details@scrummastertoolbox.org membership because listening is great. It's important. But doing it together, that's next level. I'll see you in the community Slack
- [35:56]
B
we really hope you liked our show. And if you did, why not rate this podcast on Stitcher or itunes. Share this podcast and let other Scrum masters know about this valuable resource for their work. Remember that sharing is caring.