← Software Engineering Daily
Software Engineering Daily
Organizational Context for AI Coding Agents with Dennis Pilarinos
00:49:21
AI agents have taken on a growing share of software development work, so much so that the hardest problems are shifting away from code generation towards something new, context. The challenge is now contextualizing why systems work the way they do,
Loading summary
Transcript73 lines
- [00:01]
A
AI agents have taken on a growing share of software development work, so much so that the hardest problems are shifting away from code generation towards something context. The challenge is now contextualizing why systems work the way they do, how architectural decisions were made, and the sources of truth that exist outside of the code base. As teams adopt agentic tools, gaps or inconsistencies in context have emerged as a primary reason why software fails to meet production standards. Unblock is a startup focused on solving this context gap. Their context engine aggregates and reasons over organizational knowledge, spread across source code, pull requests, documentation, chat systems, and production telemetry. By acting as a context engine for both developers and AI agents, Unblock aims to improve AI code quality and review, reduce interruptions, accelerate onboarding, and enable safer, more effective agentic workflows. Dennis Pilarinos is the founder and CEO of Unblocked. Previously he helped build Azure at Microsoft, worked at AWS, and co founded BuddyBuild, which is a mobile CI platform acquired by Apple. Dennis joins Kevin Ball to discuss context engineering, reconciling conflicting sources of truth, permission aware AI systems, the shifting bottlenecks in the software development lifecycle, and and what it means to be a software engineer in an increasingly agentic world. Kevin Ball, or K. Ball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow Kabal on Twitter or LinkedIn or visit his website Kball LLC.
- [02:05]
B
Dennis, welcome to the show.
- [02:06]
C
Thanks very much. Thanks for having me.
- [02:08]
B
Yeah, I'm really excited to learn about what you're up to, but let's start a little bit with you. So can you give our listeners a little bit of your background and what led to Unblocked?
- [02:18]
C
Sure. I'm Dennis Pulledrinos, I'm the founder and CEO of Unblock. I've been building developer tools for the better part of 20 years. I'm my career started in earnest at Microsoft where I was one of the earliest people to think about shifting on premise computing APIs to cloud hosted APIs. So was part of the team that built a lot of the Azure platform. I worked in aws. I started a company called Buddy Build, which is a CI platform for mobile app developers which was acquired by Apple. So about three years after we started it, I worked at Apple for about a year and a half and basically through that series of like lived experiences and life experience. We took a lot of that to go and build Unblocked. And so unblocked helps engineers get the context that they need to get their jobs done. So that's really kind of my background and a little bit about what we're trying to do at Unblocked.
- [03:00]
B
Yeah, so let's talk about that word context, because that's become loaded in the last year or two. We use context to mean many, many different things. How are you meaning it in the context of. Oh, we help engineers get the context they need.
- [03:13]
C
It's interesting. Yeah. Even setting context about context feels very meta. But yeah, it is one of the things that I think is a term that is very overloaded. I think certainly within the last few years we've seen this evolution from prompt engineering to context engineering. So if I think about it purely from a technology perspective, it's all the information that the machines need in order to make the right decision so that they have what I would consider decision grade context from a person. It's all effectively the tribal knowledge that exists as a consequence of working in an organization and in order to get your job done. So if you're new to an organization, your context for how that team works is probably relatively insignificant. Right. You don't know a lot on day one. And getting up to speed is really that context loading. Why do we do certain things, certain ways, why have things been done, certain ways, who do I talk to about things, all of that kind of stuff. You know, it's this interesting problem because it's the intersection both of like people and their context and the machines and their context. And I think there's actually a lot of analogies. I think they're actually quite similar in many ways.
- [04:16]
B
One of the things that's kind of interesting here is about the legibility of that context. Like, is it living just in people's heads? Is it scattered? So what are the different places that you're thinking about as you talk about loading up that context for developers and tools?
- [04:32]
C
Yeah, I think again, they're probably the same in many ways. I saw an interesting LinkedIn post recently where someone had said, you really need to spend a lot of time planning before you implement. And I was like, well, are you talking about that within the context of agents or are you talking about that just in standard software development lifecycle? So I think it's the same, right? We talk about these things. It's if they're new or unique to AI. But I think if you've been building software for a while, these things resonate with you. So where does context live? I think context lives both for AI agents and for people and all the systems that people use to build applications, right? So it's in Slack, it's in teams, it's in your source code and the history of that source code, it's in the pull requests that exist there, it's in your bug trackers, it's in your documentation systems, it's in your runtime production systems, right, like in Sentry and Datadog. It's basically all the places that your application uses to operate and all the places that you used in order to figure out what to build and how to build it. That's where that context layer, I think, lives.
- [05:35]
B
So then moving into what you do with that, right? Because like there's stuff all over the place and if we dial back and I think you're spot on to say, like, hey, the practices we were doing as humans, those are also what's going to help with agents. I think that's there. But so thinking about that, there's like all of these different layers that go into that and often involving back and forths. And I know this part of the human and background context and I'm going to propose a thing and someone else is going to come in and say, hey, you're missing this system over here, like what's going on? So what does that end up looking like in today's world and how is unblocked helping with it for sure?
- [06:09]
C
I think one of the hardest problems is to really understand where the context lives. So finding it and then that's effectively a form of a search problem and then trying to understand, well, what is actually the truth? Because what you'll find is that you actually get drift across these different systems, right? The source code will say one thing, a Slack conversation says another, JIRA ticket says another. And so being able to reconcile or do effectively conflict resolution in a way that doesn't lead a person askew is actually a really hard problem. And so, I mean again, we keep talking about it within the context of AI agents, but imagine talking to someone who just constantly lied to you around the questions that you had, right? Like you're not going to trust them. They quickly become out of the loop of people you chat with to get information. And so that person, what are the characteristics of that person? While they've typically been around the code base or the application for a while, they, they've participated in a lot of those conversations. They understand why things have been done, they understand what the truth is for all of those things. So in the same way that you'd want to replicate that person, the software that Unblock tries to build basically does the same thing. We'd like to be thought of as like a member of your team who's been aware of every conversation that's ever been had for every part of your code base. So you can ask questions and get answers from it, basically.
- [07:27]
B
I think there's two different angles we can come at this. We can come at what does this end up feeling like from a developer perspective when they use it. And then I think we can also come at the end. Well, that's a complicated software problem that's worth digging into how you're doing it. But let's maybe start with the developer perspective. What does it feel like for a developer if they want to start using Unblocked?
- [07:45]
C
For sure, there's a handful of different expressions on how people would realize this context problem. Unblocked chronologically started basically as a Q and A platform. So you could, as a developer ask a question and get a high quality response in a very short period of time. And so you'd have a web experience. We have a standalone Mac app you could ask questions in. The idea of. We found that our integration with Teams and Slack is actually one of the kind of killer scenarios. I think almost every organization has a channel or a place where people ask questions around how to get things done. Your infrastructure support channels or your feature channels. And so what Unblock does there is listen to questions that people pose to their coworkers. And when it has confidence that it can provide a high quality answer, it replies typically in a few seconds with that answer. And people find that incredibly delightful. They can get that answer without having to wait for their coworkers. And especially that's a super painful experience if those coworkers are time shifted or geographically distributed. There's all sorts of human behaviors that we think are amazing. We have lots of teams who speak different languages, right? Where they might have a team that's in Portugal, another team that's in the UK and the Portuguese folks are maybe not necessarily as fluent in English as they feel comfortable to ask these questions. So they'll ask Unblocked in Portuguese and, and get a response in Portuguese. And they don't have to worry about getting any kind of like judgment as a consequence of their language skills. So there is this Q and A experience for people to get the context they need to get their jobs done. Obviously what we've seen is this historical trend of people being the entity that writes code to now agents writing code. And so we've augmented a lot of the agent so that they have very similar context. So as part of their reasoning experience, they have the full end to end context to make the decision for when they're actually going and writing code.
- [09:38]
B
So from an integration standpoint, does that look like an MCP server? Does that look like a set of skills and an API? What does that play into if I wanted to integrate this into my agent workflow?
- [09:49]
C
Yeah, absolutely. So I mean, the first thing to do is obviously connect the data sources that you're interested in. Right. So if you think about that knowledge might exist across a whole bunch of these different data sources. Slack notion, your source code, you connect them so that unblocked can reason across all of them. And then how that's exposed, I mean, we're recording this on a Monday, I think by Friday this will change once again. Right.
- [10:10]
B
But yes, that is the world we're in these days. Yeah, exactly.
- [10:14]
C
It's mcp, it's a skill, it's a cli. There's a strict API. I was joking this morning with the team, just for trolling purposes, we should just expose like a WSDL endpoint for those folks who've been around for a long time, just to see. See if we can bring it full circle. But yeah, so it's really. I think of these effectively as protocols and depends on the CL who's trying to use those. Right. So for an agent versus someone who's building an app, who wants more determinism, they can use the API or CLI or so on and so forth.
- [10:41]
B
Another part of that user experience we can talk about both for humans and for agents. You highlighted how often there are discrepancies. The code says this, the docs say this, the original ticket says this. There's been drift or just failure to implement or what have you. What is the experience for me as an end user when I encounter one of those areas where there's a lot of discrepancies?
- [11:04]
C
Yeah. I mean the experience for you really is that you just get a truthy answer. Right. You just get the answer that you're looking for. So what we've seen is typically both agents and people are asking two types of question. How does it work today? Or what are the decisions that we made in order to get here? Right. So if you're like, I just need this information so I know how I can re platform versus we're trying to make some decisions, what are some of the learnings that we've had in the past. Those are the types of questions. There's a temporal aspect to the type of question. And so you can always ask clarifying questions depending on the temporal nature of the question that you're trying to get an answer to. So typically the experience is your ide, your code editor, or your CLI will call into the unblock context engine to get information that it needs and you can see obviously the output that happens there. And you can ask, well, how did you come up with this? Or why did we decide to do this? Or any of those types of things. So it does feel like another person on the team.
- [12:00]
B
So that maybe gives a window into maybe we can start diving into some implementation things a little bit. Like, one of the things that I have observed out in the world is that LLMs tend to be bad about time. That temporal nature that you're talking with, just out of the box, LLMs don't seem to do very well with it. So how do you think about that? Represent it in your system and expose it?
- [12:23]
C
Yeah, I think that's a very astute observation. I think that's actually gotten progressively better as we've moved from like these static rag implementations which have semantic representation but really fail on a kind of a temporal and more to like agents who can reason more a little bit about where they are at a point in time and how to move forward or backward or what have you. When we think about conflict resolution and time, you basically have to take all the artifacts and weight them accordingly and come up with what you believe to be the truth at that point. I kind of allude to this. People typically ask questions or agents, people and agents ask questions around how does it work now? Or how did we get here? So there's a temporal part to it, the thing that you know. And if you think about your human interactions quickly, what you realize is that you also need to know who the person is that is asking the question. Right? Because they typically work with a group of people or work in a certain part of the code base. So that helps you narrow down some of the things that they're looking for. And then obviously the third part of it is like, well, what do they actually have access to? So some organizations have very specific rules around who can access what information. So all of those together need to factor into what happened over the last 30 days, or summarize the Slack thread for me, things like that.
- [13:35]
B
Yeah, that makes a lot of sense. Well, let's talk about what are the layers on the technology side that go into Building that. Like you mentioned, access control. Access control is near and dear to the hearts of everyone having to deal with agents. Right. Like what should these things. They're essentially just running stuff for us, like what can they see? So how do you approach that?
- [13:55]
C
Yeah, it's a great question. The thing that we realize a lot of our customers care a lot about who can see what information. And it has to be done at runtime, right. It has to be done at the time of the query being executed. So we have. It is a permissions aware model. So if you don't have access to the underlying data source, the context engine doesn't use it as part of its response. Right. So we have this great demo where someone asks a question about Project Tantalus, some arbitrary projects, what's the release date for Project Tantalus? And that lives in a Google Doc that I currently have access to. Our demo is very straightforward. You ask the question, you get the response, we revoke access. In that Google Doc, you ask the question and unblock. Can't answer the question. It says based on the information about you and what I can find, I don't know what the release date is for Project Tantalus, even though it exists in the underlying knowledge base that exists there. So at runtime you have to be able to understand who that person is. So a strong sense of identity, what access control they have across those disparate data systems. And there's some really hard problems associated with that in terms of this balance between making sure that you can provide answers while ensuring that it's not used as an escalation of privilege.
- [15:06]
B
Yeah, for sure. Thinking about that, you have this access control layer, actually, how do you engineer this whole context engine? Like, is it layered? You have access control at the bottom. Is that interwoven? Like you mentioned, interpersonal awareness, Is that. Do you have a relationships layer of like, oh, this person works with this team? Like, how does this all work from a software perspective?
- [15:27]
C
Yeah, it's a good question. It's probably most interesting to start like at the very beginning of like, what's the data that you need to use? Like, what are some of the data problems that you need to solve? Because like, that is kind of the substrate in which things are actually built. In my career, I've never met a team or an organization that says all my documents are super well organized and up to date. Right. It just doesn't exist. And so I think that is the predisposition that you need to have when you build a product like this either as A company like ours, or if you're building something internally so you get a lot of noise, and expecting a team to organize and keep their documents up to date I think is a bit of a fool's errand. No one's going to go to the heavy lifting, both to start and ongoing to maintain that. So I'll give you an example of one of the things that we do just at that data layer for people who ask historical questions. We actually look at every single pull request that's ever been opened within that source code, within the repositories that we've been granted access to. And what I found in my experience is that most developers aren't great at writing pull request descriptions, right? They're usually relatively brief and they don't give you a ton of context. So what we actually do is a diff on the pull request and embed that and basically build a background knowledge graph effectively for that organization. So we actually up level the quality of their knowledge based on something that they already have but is basically underutilized. So that's an example of some of the data enrichment that we would do such that you have a lot richer materials by which you can actually go and query from there. Like I said, you need to have a strong sense of identity. Who is the person who's asking the question, Then the identity is actually Difficult because your GitHub identity might be different than your slack identity might be different than your notion identity. And you need to be able to bind all those things together such that if your organization does care about access control and information seclusion, that you don't violate any of those things. So managing that and getting people an experience where they can ask a question or their cursor or claude or what have you is actually writing code that makes sense is another thing that you need to be able to manage. So who that person is making sure that they're logged in and what have you. And then underneath is basically a hybrid rag system with agentic. I hate using all these terms because they feel like word sounds, whatever, but it's an agentic but underneath hybrid rag system that does both lexical and semantic search. So increasingly what you're seeing is that like rag has run out of its limitations and that a lot of these agentic tools can actually effectively meta go and get the answers that they need themselves. So it's interesting to see, I was talking to someone about it this morning, where you think about Paul Graham's famous software is eating the world, right? Everything basically has an API and If the tool can call into that API and it does it in a way that respects the permissions model or what have you, you can start to really light up a whole bunch of these scenarios. So we expose a series effectively of tools across this so that those orchestrators, those agents can actually go and call into that. So that's kind of like the three layers, as I like to think about it. There's obviously a lot more detail in terms of, like, how you do conflict resolution, how you do accurate ranking and searching and things of that nature. But yeah, the other thing that I
- [18:35]
B
think is big when we talk about these sort of agentic systems is observability and debugability, because they're off doing all this stuff and going down and you're like, as you highlight, it's not a simple, like, we're doing one rag query and running one inference, right? This thing may have tried this API call, gotten back, things being like, oh, I need to adjust my search query. Let me search over here, let me go do this. And at the end of the day, you have something that either does or does not work. And I mean, it's shocking how often it does work, but when it doesn't, what tools are available to the end user to say, like, what is broken in my context? I mean, because maybe it's a problem in the engine, but maybe it's just like, I've got some document that is very confusing for the agents or something like that. Like, how do I trace back where this came from?
- [19:21]
C
Yeah, so for unblocks customers, really, it's kind of funny that you talked about it. Either it worked or it didn't. We actually have this in product experience, which is like, do you find unblocked helpful? Because at the end of the day, that's what people kind of care about. And so, like, we have tens of thousands of people who've answered that question. 95% of them say yes, they have the remaining 5%. Kind of. Funnily enough, about half of them just hate answering surveys. So they just say, like, why are
- [19:45]
B
you asking me this? Go away.
- [19:47]
C
Yeah, yeah, yeah. It's like, I don't like answering. So it's remarkably consistent to see over a large data set. So at the end of the day, what people care about is like, did it help me? Right? Does it help me accomplish implementing this feature or getting this information or what have you in terms of debugging it? The underlying technology continues to get better, where you can start talking to it like a person and it will explain Itself. If you think about even two years ago, people were deeply concerned with hallucinations. And I think that people are just less sensitive to it. I think it's a consequence of two trends happening. One, people are developing a sense for how to figure out when the thing might be overly assertive in terms of its confidence. It's like, you're absolutely right. You're like, well, okay, I'm going to have to double click on that. So knowing how to use the tool a little bit better. And then second, obviously, the technology has gotten a lot better in terms of being able to check its work, as it were. We actually have this interesting. I love the human factors of building software. And so we built this feature where you get to decide how quick or slow you want your responses to be. And the way we surface that is, do you want fast answers or do you want thorough answers? And there's a lot of internal debate as to, like, what people are going to want to do and where people would set those defaults. We're a very metric company. It's an overwhelming majority of people who are willing to wait for a higher quality answer than get a fast one. That's wrong. Which in retrospect seems obvious, but you would think some people would say, oh, I'll weed through the wrong answers. I just want them quickly. I think increasingly, I think people are seeing. And you see this with this orchestration of agents. I'd rather have it more things going for longer periods of time so I get better results.
- [21:22]
B
Yeah. So this is maybe a slight tangent on this, but I feel like one of the biggest, maybe not totally solved problems in this agentic coding space is how we as engineers update our brains on what the agents are doing. Right. How do we keep track, do we even need to keep track of the evolution of the code base and things around that? So you've talked about how unblocked can be used by humans. Q and A. It can be used in an agent as probably also kind of Q and A or things around that. How do these things meet in the middle? Is there something around like, oh, my agent. These are the questions that asked and these are the answers that it got. Or like, this is the space. What is this realm of. I have this autonomous use of unblocked, but I, as a human want to understand what's happening.
- [22:10]
C
Yeah, it's a great question. I think what we're seeing in the industry is like traditional software development is now progressively relegating the task of generating software to an autonomous agent that will go and do that for you? How do you make sure that it's actually correct and what is done through that process? Right. So I think that's where we're starting to see things that both help the person and I think the agents really get on the same page. Right. So when you think about plan generation, I think that's really solving two problems. Making sure that contract between the person, the agent, is well understood, such that when the agent goes and executes the plan, there's no surprises at the end. So it's fascinating. We see a bunch of our customers who use unblock as part of that planning process. So again, going back to that mental model of like, of course you would spend more time planning as you go to implement a feature. And I'm intentionally being like vague around, am I talking about a person or an agent? So when people are using agents to create plans and they augment or context enable that the plans are dramatically better because it understands the coding standards, it understands some of the architectural decisions, it knows all the things that you would get caught up in your pull request or in your code review process. You just skip all of that left of the software development lifecycle, so you're not getting caught there. And then, yeah, unblocked actually helps you with your code reviews as well, which is kind of a fun experience for you there.
- [23:32]
B
So talking about lifecycle, we can talk about the software development lifecycle and that's interesting, but your world is context. There's kind of a context life cycle also. So one example I'm thinking about and I'm curious if this is an area you're doing. So say you pull a set of sources and you're like, okay, this is coming from jira. I have this, I have a doc over here in my notion or whatever and I have the source of code and I'm an agent and I'm going off and working and I'm updating the code. So one area of this context has changed. I as a human developer would then say, oh, I need to go and comment on that JIRA ticket. Oh, I need to chime in in this Slack thread that this has been done. I need to update that notion doc because it is no longer the correct source of truth. And in some ways the process of gathering the context also gives me the dependency graph that I need to go and update or chime in and. Or those sorts of things. How does that work if my agent is doing this all on its own?
- [24:29]
C
Depending on what you want that end user experience to be, I think sometimes you Will have those agents go and effectively say, this has been done or this has been redressed or whatever it might be. Depending on the company and their level of comfort around someone going and writing back to their systems. They might or might not allow that. Maybe if you think about from where you end up when that code gets written and that feature is implemented or what have you, we do place a decent amount of weight in this conflict resolution strategy on the source code itself, right? Like, at the end of the day, source is the truth, right. Of what's actually happening at that point in time. So if there are discrepancies across these different things, especially as that work has been done, when you go to reconcile them, the source code is a really, really strong hint for what actually went and got done. And increasingly you can look at, well, what are the changes that got you there? So there's almost like a series of breadcrumbs that you can use to reason about that. I think for people, we probably have a natural inclination to want to see all that kind of documented and closed off. I think agents can reason about that, and I think we want that because we want to get to the answer very quickly. I think agents don't necessarily need to have that there because they can reason about it very, very quickly.
- [25:45]
B
Does that introduce more drift?
- [25:46]
C
Yeah, I mean, I almost think of, like, if a series of discussions happen that are not codified in any kind of written way, how do you reason about those? Where is that context captured and how does that. How does that flow in? I think you get effectively direct and indirect context in that way. Right. And so the direct or things that you can actually derive are those that have been written down or codified, the indirect or ones that you have to infer based on the evolution or the change in that knowledge graph. Right. So as these artifacts evolve, the source code evolves. Pull requests, related pull requests related to those systems have changed and we're building that indirectly, then we can infer a good chunk of it. But it's an interesting thought experiment, right? Which is like, if you had a company that solely only ever communicated in person and only just worked, like in source code, never used Slack, never wrote any documentation, never wrote any pull requests, just like git, push F to main. I'd be curious to see how well that company does from a technical debt perspective. But, like, it'd be kind of curious, like, it's an interesting thought experiment to say, like, well, how well would agents perform there over time? Like, as you think about a Greenfield application, it does great because it doesn't need to understand any of like system outages. And you know, the root cause analysis that happened as a consequence of that system outage to feed into. Okay, that was a really bad pattern. We're not going to introduce that pattern as we go and write more code or as we design another system. Right.
- [27:19]
B
Yeah. Interesting.
- [27:21]
C
Does that make sense?
- [27:22]
B
It does. So this gets to kind of the key question here around like there is. And one of the potential limiting factors broadly of Magentic moving more and more work into agents, which is not all work that we have is explicit linguistic and legible. So how well are they able to operate in areas where there is a lot of this context that is missing, not really there. And what do we need to augment to enable us to automate more of this?
- [27:52]
C
Yeah, I think again, if you go back and think about what is the evolution of the last five, six years. Right. As a consequence of even like the COVID pandemic, I think people were forced to basically have more things written down and have kind of gotten into that habit, which has been very fortuitous because a lot of these tools require that to be written down. You know, having worked at companies that are very much in person versus other companies who are very comfortable doing remote, they both have their pros and cons. And I think depending on the project and how old it might be, you benefit from one approach versus the other. So my time at Microsoft was a long time ago, but Microsoft is a very Redmond focused company. Apple is to this day very much a Cupertino based company. Right. And it's, I think quite difficult if you're not present in a lot of those conversations day, day in, day out. Amazon in my time there, you know, occupied many buildings across downtown Seattle. And so it was very common for you to. You heard about the six pager documents or the press releases or when you hop into a meeting that you start the video conferencing software. I think a lot of the behavior and cultural predisposition of organizations factor into like how good some of these tools can be there. Not in those specific companies, but just in general. And I think additionally what you end up finding is that as a consequence of some of the recent events, people are writing a lot of that stuff down. It's very fortuitous for these types of tools.
- [29:23]
B
That leads me to wonder and I'm curious what you're seeing as you're working with companies, I assume kind of all across the spectrum in terms of how well documented their various things are. Like does this create a schism in terms of productivity and ability to really embrace agentic workflows?
- [29:40]
C
Yeah, I think if I were to be crude about it, I would. You know, you could create a hashing function which basically says you're starting a brand new application. So a greenfield application, those don't need a ton of context. And it's, you know, the feeling of like asking a machine to do something and it coming back and, and blowing your mind and just like almost reminds me of a slot machine, right where you put the thing in and you're like, oh my God, it almost came. And you're like, I'll try it again, I'll try it again. That's why people are up. Leading myself to like three or four o'clock in the morning, being almost just like this one little thing. So for those greenfield applications, it's incredibly addictive. When I think about our customer base, folks who have legacy software teams who have like thousands of engineers working on these code bases that have been around for decades at times there's no way that someone can like one shot, you know, significant refactors or changes or what have you without a lot of that context. Accurately, I think you can reduce the repl time basically right by having that context where you probably don't need that for a greenfield application. It's interesting to see how those two things will reconcile with each other as time progresses.
- [30:44]
B
Let me go back to something you sort of mentioned as a throwaway earlier. So one of the things that I've observed in my organization and a few other organizations where I'm talking to folks is the bottlenecks in the software development lifecycle are shifting. And one of the big bottlenecks with agentic generated code is code review. What does that look like? How does that approach? How do we handle the incredibly increased volume of code that is getting written? You all have a code review tool. What is it that you're doing and how do you help address that bottleneck?
- [31:17]
C
It's interesting, I heard this funny quote which is like, there's been more technical debt created at big banks in the last six months than there has been in their entire history of existence. I was like, that's a great, great way of describing it. So yeah, now you have a bunch of these agents and we've seen, you know, even the backlash in the open source community where people are like, I don't want AI generated code because it just creates this maintenance and this burden for the maintainers trying to like accepted into, into their projects. The code review Tool that we've built is built on that context engine, right? We think about it as what are the standards and best practices of that organization or that code base that would make that a lot more palatable. And we've seen a significant adoption rate relative to other tools because it's aware of all the. I'll give you a concrete example. I'm building an internal tool. I was using cloud code to do it. That internal tool would require us to do some secrets management in production. Claude, just as an experiment, without knowing how unblocked work, basically said to secure the API key in a way that would absolutely, like, break us into jail. But I was like, oh, let's just see what happens, right? We'll just like stick it in an environment variable. And I'm sure nothing bad will happen when someone compromises your infrastructure, right? It's not the first thing they do is dump out the environment variables anyway. So I'm like, okay, great, off you go. Open the pull request, unblocked code review, looked at it and said like, no, no, the way that you're supposed to do this is actually like encrypt it in vault, decrypt it, sits in memory, so on and so forth. And so what's my natural inclination as a software engineer? Would I want to do that and say, like, oh, okay, I have to go make that change. Instead, what I asked was, you know, Claude, go take a look at this code review comment and like, just go and implement it. So, like, what does that repl look like now? Right? So what's the origin of pull requests? Well, it's to build some shared knowledge around what the code base does and how it's evolving, make sure that it's meeting the standards of the organization. I am actually kind of curious, like, does that get short circuited? Right? Do you need pull requests in the future where the bot can write the code? You have that context layer to evaluate that it meets the coding standards of the organization. Like, as a secondary check, it goes and addresses those comments, reissues it. You know, you have automated tests that can evaluate it and address any test failures. Can we tighten up that repl loop? And what does that mean for the knowledge that, you know, its initial intent behind call requests and code reviews, of distributing that and understanding how the code base evolves. It's a. I'm very curious to see where PRs are in two years. Do people accomplish the evolution of understanding how their code base is shifting using this technique or something else?
- [34:08]
B
Yeah, well, and this gets to this mental model question of how do we update our mental models when the agents are doing all of the actual writing of code?
- [34:15]
C
Yeah, it's cliche, but it is an incredibly exciting time to be building software these days. Right. Because a lot of the things that we just took as fundamentals, as a byproduct or as a consequence of building software are being completely reevaluated. You know, we started this call by saying, you know, I've been building software for the better part of 20 years, same way as you have. And there's these interesting as a consequence of how the software or how software is being built now, where you're like, well, how do I make sure that I understand how it's evolving?
- [34:42]
B
Yeah. I mean, I feel like things have changed more in the last three years than in most of that 20 years that I had. It's absolutely wild.
- [34:50]
C
Yeah, yeah. You know, I've seen the evolution basically from like on premise to cloud, cloud to mobile, and arguably like this wave. And, you know, you see these rates at which companies and people adopt this technology as of November, December of last year, when Claude became a fun thing for you to hack on over the holidays. Like, again, it's a stepwise function. Like, I think that will be a significant moment in time where you make it very easy for people to build a lot of software very, very quickly.
- [35:20]
B
So what would you say are the big, like the frontier of unsolved problems, maybe staying in this world of like, context, engineering and understanding what is needed in order to build things?
- [35:32]
C
That's a great question. I don't know that I have an answer for that off the top of my head. Like, I'll just be completely transparent. I could wax poetic about it or what have you, but I think the human factors part of it for sure. Right. Some of the things that we've just talked about, which is like, as you have increasingly these agents that are taking away the knowledge that's required, how do people keep up to date? It's an interesting thought experiment when you think about that sdlc. I'm writing code, I'm shipping code, I have to manage it in production. I think every kind of DevOps engineer or on call engineer's nightmare is that they have to support something that they don't know how it works. And so I think people, when that gets to a place where it's like completely resilient, that I think will probably force us to reevaluate the role that Software engineer has and how you interact and work with these systems. But I think having worked in large Scale production systems. There's no way you could possibly know the end to end inter system dependencies that would potentially take down your service. So the idea of having to support something you don't understand is very disconcerting when you become on call. But if you take away that problem, how much of the system do you actually need to understand? Is it just you guiding it in the right way? So I don't know. We'll know in a year, I suspect, or less.
- [36:49]
B
Yeah. Well, it is kind of wild, right? Is projecting forward. Are there software engineers?
- [36:55]
C
Yeah, I mean, I always think of this example of a person who, you know, frames a house like a house framer, started with a hammer or maybe some even predated that with some other blunt object to nail the house or what have you and then quickly got a nail gun. Now, do you think there was this existential question which is like, what do software engineers do? Well, you're just getting more work done, you're framing more houses because the tools that you have have improved dramatically. You still need to have the fundamental skills to understand like what is the geometry of the house? Will it be structurally sound? Will it stand up to like the weather conditions that you're building for? Will the doors swing cleanly? You don't want it to be tilted as an example, Right. So just because you have the tools to build that scaffolding faster or that end product faster, doesn't mean you don't have craftsmanship in order to accomplish it.
- [37:47]
B
This comes back then to the question of how do we understand what it is that we're building? Right? Like what is the software metaphorical equivalent of being able to know if the doors are going to swing the right way?
- [37:59]
C
Yeah, Well, I think in the same way that you like, you think about a door which is like, does it accomplish, I mean, what software, at the end of the day, it's trying to solve a very specific business problem, right. And so if it doesn't ultimately solve that business problem, you can just, it's turtles all the way down. You have to kind of go and figure out, well, why is it not doing that? I think we've probably all been in a house where the door closes by itself automatically. It's infuriating, right. It's not probably an environment that you want to live in. And so you quickly pull the pins out of the hinge and bend them so that it holds open or whatever you are replummet or whatever it might be to get to that solution. I think you really have to do look at the end result and say like, is this what we're trying to accomplish when we started off? And then the rest is kind of like you only need to understand it to the degree that you need to fix it.
- [38:44]
B
Taking us back to thinking about context engineering. So we've talked a lot about coding agents and software development lifecycle and software engineers asking questions. But as you highlighted, like this context question of like, what is it that I need to know to do my job is much broader than that. Software engineering is probably the place where agents have gone the furthest to date. We're seeing all sorts of different things going on with what is it? Claude, cowork and these other different things, which is like agentic tools for non software developer knowledge workers. Is that a space that you all are playing in in terms of like context for non software work?
- [39:24]
C
That's a very insightful observation. What we see is typically an engineering team loves unblocked and it saturates the engineering team as part of their day to day workflow very, very quickly. And then engineering adjacent organizations also find a lot of utility in the tool. And so if you work in product support, instead of having to interrupt the engineers to understand how something works or solutions engineering or any number of these engineering adjacent organizations, they also will use unblock to get answers to their questions as well. It was interesting yesterday. I was accomplishing, I was like looking at a task and I was reviewing a piece of content and I was like, oh, this content feels like it's a little bit different than my style of writing or what have you. So just on a whim, always trying to try these little micro use cases, I asked Claud, the desktop app, not the cli, to do an analysis of my writing style, tone, phrasing, whatever it might be, and summarize that and compare it relative to the writing that I was just evaluating. And what it did is it actually called into the unblocked context engine and dug through recent Slack conversations. It had dug through some of the notion documents. I read, it dug through some of the. We happened to host a bunch of our blogs in our scm. It pulled all that together, right? And said, ah, this is how Dennis writes. This is how I would characterize these things, right? And then now that I have that as a premise, I'll go evaluate this other text. So completely non software engineering related, but cuts across all these different systems to have an opinion for, you know, what context it needs to accomplish that task. Dennis writes like this. This text is written in this way. Here are the differences. These are the changes you'd want to make if you wanted to make them the same. So again, we see these scenarios pop up all the time.
- [41:07]
B
Yeah, that's fascinating. So if we were going to look out, and I hesitate to look too far because the pace that we're talking about here, we used to say if you were going to look out a year or two, what would you see coming? What's the right event horizon now? 3 months, 6 months? What's coming down the road for Unblocked?
- [41:25]
C
Increasingly, I think that the rate at which people are building software is going to actually increase dramatically. Like in a way that we've never seen it in the past. A lot of the scenarios that folks are building for, certainly within businesses need to understand, like to have that end to end context. So making sure that Unblock shows up as a tool that they can use as they need to context enable their applications, their workflows, their agents, their tools, that is definitely a priority for us. So if I were to think about how unblocked, you know, the chronological evolution of Unblocked, we had this Q and A platform for people. We made that Q and A platform now available for their tools, for their agents. And then we realized that one of the things that we have that's quite powerful is this context engine that powers that Q and A workflow for people and their tools. We built a code review product that also uses that context engine. Increasingly what we're seeing is organizations are building their own tools, their own applications that need that context engine. So then we expose it for folks so they don't have to solve a lot of these hard problems that we've done in the past. So I think that is the evolution. I think increasingly more and more people are going to build apps within their enterprise and Unblocked can play a significant role in reducing employee onboarding, customer support, a whole bunch of these types of scenarios.
- [42:48]
B
Yeah. So if I'm hearing it was sort of the evolution from Q and A product to oh, we have an engine here we can do more interesting things with. Let's do something interesting to now. Maybe engine as SDK/SaaS app that people can just kind of plug into.
- [43:04]
C
Yeah, I mean, so you can experience Unblocked using the apps that we've built already. So we have a Q and A product. We have a code review product that Q and A product is great for people asking questions in Slack. It's great for developers who are doing code gen. Right. So in the same way that a person needs context to figure out why Something was done a certain way, an agent can have that. So there's like that questions and answering like why do we do it, how should I do it this way? So on and so forth. We have that code review product, but that's built on this engine that we will expose effectively as an SDK or an MCP or CLI or any whatever, your favorite programmatic interfaces for you to build your own apps. So you can have out of the box solutions basically for those workflows, or as you build your own workflows, you can leverage that context engine.
- [43:49]
B
So is that far enough along we can dig into what that might look like? Because I'm imagining what levers do I have for exposing. I have a set of contexts that is unique, it's different. How would I integrate there and give you, oh, this should be taken as a source of truth or what have you.
- [44:07]
C
Sorry, it's the question like, can you do this now? Yeah, yeah, absolutely. You can? Yeah. It's in its infancy in terms of its public facing facade. We have customers who are using it quite actively and we're slowly having built, you know, dev platforms for a very, very long time. And API is one of those things that like, you just don't want to break. So as we've gotten more and more comfortable with those scenarios and what, you know, knobs people want as an example, how to prioritize one data source versus another. We're definitely making that available.
- [44:37]
B
Yeah, fascinating. That actually makes me realize we haven't talked about this, but like you highlighted, you've been doing this for a long time, all these things have changed. What hasn't changed? What are the lessons from like building cloud platforms 10 years ago that are still relevant as we're talking about building these LLM applications?
- [44:57]
C
It's probably an unsatisfying answer purely from a technology perspective, but like at the end of the day, the software is a means to an end to solve a specific problem for someone. So early in my career I was introduced to this model called bxt Business Experience Technology. Right. So what's the business problem we're trying to solve? What's the experience that best enables that solution? And then what's the technology that provides that experience? And so that's how we think about it for everything that we've done is like almost all the projects that I've ever kind of started up or worked on are as a consequence of my frustration with the inability to do something right. So I'm just like, I'm like, I have a very specific Business need. So, like, for Buddy Build, I had a very, very straightforward workflow git push. A build happens, it gets deployed to my device like CI at the end of the day, right, cicd. And it was incredibly difficult to do that in the mobile ecosystem. Well understood problem mobile, completely unsolved, really in earnest. And so the lack of a very polished experience was what caused us to create Buddy Build. Increasingly, like when we started to build Unblocked, it's the same 10 people, almost, almost the exact same 10 people who worked at BuddyBuild who wanted to build Unblocked. So we literally got the band back together as a consequence of this frustration of trying to get the information we need to get our jobs done. So you work at a company like Apple. I can think of the specific individual, I know his name, who knows how code signing works for an iOS and a Mac app. Right.
- [46:30]
B
Let me tell you, everybody who's trying to ship iOS apps wishes they knew that guy.
- [46:34]
C
Yeah, yeah, I know the guy. And so still think of him.
- [46:37]
B
You could like sell his name, just be like, all right, here we go. That's a business in itself.
- [46:42]
C
Yeah, but that poor gentleman had to just sit there and explain that thing over and over and over and over again. He just happens to be a subject matter expert. And so it's like we called the company Unblocked, but the original name, I think it's still in the code base, was bothered, which is like, you don't bother me, I don't bother you. Right? Which is like, I just want to be able to get answers very, very quickly. And so the business problem is like, I need to move quickly and remove this inefficiency in getting that information done. And so again, increasingly, as people are not writing as much code and agents are, they need just the same amount of context to go and write code that solves a business problem as people do. And increasingly people are going to build more applications, give them that context layer to make sure that those agents and those applications have the information that they need to solve those business problems.
- [47:29]
B
Well, and I think this gets back to our question a little bit about, like, what is the equivalent of knowing about the door and where it's swinging and those types of things. Like, what does it mean to be a software engineer in this world? You need to be thinking about that business and experience layer. You can't just be thinking about, oh, I have a well specified problem and I get to translate into machine readable code.
- [47:48]
C
Yeah, that's exactly right. It's like, you should have an opinion for how wide that door is and which way it swings. You know, like the two kind of dimensions that you'd immediately think about, but like most software is a lot more complicated in terms of the dimensions you need to think about. And so I think that is one of the things that will truly differentiate the ability for it. While apps are being built very rapidly, which ones are are going to be commercially successful or what have you is the ones that have great craftsmanship.
- [48:16]
B
So we're coming close to the end of our time here. Are there any things that we haven't talked about yet that you think are important to discuss before we wrap?
- [48:25]
C
Something I can think of. I think, yeah, this is a super fun conversation. It is remarkable how quickly things change and what things the engineering community cares about shifts rapidly, like every six months. It's remarkable to see how advanced some people and organizations are in terms of their adoption and how some folks are still reticent or potentially confused or concerned around what that means. But I think if you look through history, every time there's been one of these kind of like transformative changes, this is how it typically works. The early adopters are super excited, and the folks who are unclear around what that means for them have some apprehension.
- [49:04]
B
Sounds like a good close.
- [49:05]
C
Let's wrap that awesome.