← Software Engineering Daily
Software Engineering Daily
AURA and Open-Source Agents for Production Operations
00:53:04
AI agents have transformed how software gets written, but the operational side of running software in production has not yet experienced a similar revolution. The same teams responsible for keeping systems healthy, investigating incidents,
Loading summary
Transcript47 lines
- [00:00]
Narrator
AI agents have transformed how software gets written, but the operational side of running software in production has not yet experienced a similar revolution. The same teams responsible for keeping systems healthy, investigating incidents and managing reliability are still doing much of that work manually. Mezmo is a production AI company that makes autonomous operations fast, efficient and safe. Their open source project, Aura, is a declarative agent framework specifically designed for SRE and platform platform engineering workflows. It takes a Kubernetes inspired approach where teams define what they want agents to do rather than scripting every step of how to do it. Andre Elizondo is the head of product at Mesmo and he has a background in systems engineering, SRE and observability. In this episode, Andre joins Kevin Ball to discuss what makes SRE agent workflows fundamentally different from coding agents, how Aura handles context engineering, Aura's declarative configuration model, the spectrum of agent autonomy, and where the role of the SRE is headed as agents take on more of the operational work. Kevin Ball, or K. Ball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow K. Ball on Twitter or LinkedIn or visit his website K. Ball LLC.
- [01:47]
Kevin Ball
Andrej, welcome to the show.
- [01:49]
Andre Elizondo
Thanks for having me.
- [01:50]
Kevin Ball
Yeah, I'm excited to dig in. So let's start with you. Can you give us a little bit of your background and how you came to be at Mesmo?
- [01:57]
Andre Elizondo
Yeah, absolutely. So my background at my core is, you know, I started my career running systems in a lot of different forms, both at enterprises, small companies, both as a practitioner as well as in the ecosystem at various different companies across cloud, infrastructure, observability, security. And so my view on the space is primarily from an operator or an sre. Back in the day we called them sysadmins. Right. And over the last few years I've been heavily involved in the AI ecosystem, especially around agents. And so you know, here at Mezmo, what I'm focused on, I head up our product team and we're small but mighty team that's very focused on building, you know, really the the ecosystem around how we enable SREs, platform engineers, really everybody in operations to gain the same benefits that we've seen with coding agents over the past few years rather for how we better use them to reinvent how we build, maintain, troubleshoot everything around production.
- [02:58]
Kevin Ball
Nice. So can you give us actually just that, like, quick, high level, what is Mesmo? And we're talking a lot about agents and we'll talk a lot about agents. But like, Mesmo is a company, what do you do?
- [03:08]
Andre Elizondo
Yeah, so when you think about what the software factory is that a lot of folks are talking about right now, usually that's around, hey, how do I use something like a coding agent to build software faster, test it faster and get it to production faster? What Mesmo is focused on as a company and we have a, you know, a long past around observability. Making observability really useful, really kind of like toned in on making it kind of like a next iteration of how we use observability, where we've really focused our time within the last six to eight months or so is really around how we take it a next level further. Right. So when we think about how things like observability as a, as a good anchoring point change in the land of agents, well, not only do you need agents that know how to handle that observability, but the observability systems look a lot different. Right. Instead of an agent looking at a dashboard, it's going to prefer the raw data, it's going to prefer insights, it's going to prefer, you know, like specific tools around how it uses that, that data. And so, you know, when you think about the mission of Mesmo, we're really looking at the latter half of the software factory, right? Like creating that software factory and ultimately the dark factory for operations. Not just how we get software into production, but how do we really change the entire life cycle around? What does it mean as a, you know, as an operator? To make sure that that software is healthy, to make sure that that software. I can optimize performance, I can optimize cost. There's a big topic that I bring up a lot which is like, you know, what if you had an SRE with an empty JIRA queue, what would they do? And there's a lot of things that fill that particular topic. And so, you know, at masmo, we're really focused at, on not only building the agents, but building the operating environment, building the ecosystem for how we enable teams to do that today.
- [04:51]
Kevin Ball
Totally. I feel like, yeah, as we automate more and more of software delivery, that makes things like observability that much more important because you need to be able to understand, hey, my agent said this was fine, but is it actually behaving? How is it going?
- [05:04]
Andre Elizondo
Totally, yeah.
- [05:05]
Kevin Ball
So I could see a whole company focused around agent legible observability, you went more than that. So how did aura come into, and maybe just a very high level, what is aura and how did it come out of what you're working on?
- [05:17]
Andre Elizondo
Yeah, great question. You know, when we first started to go down this path of the next iteration of observability, right? When you think about like how do you distill patterns and logs and traces and metrics, we focused on initially like most folks, like exposing our MCP server in a way that people could consume it through agents. And you know, that meant that instead of your agent consuming an entire, say like hour of traces or week of traces, which would immediately bloat the context window, you, we wanted to make sure that that was tailor made for agents being the primary consumers more than humans in the long term. Now where we saw some problems internally as well as how folks were consuming that MCP server is that S3 workflows look a little bit different, right? So when you think about the types of workflows, the types of context engineering, the types of human in the loop and what that looks like for things like coding agents and things that are just building and testing software locally or in a controlled environment, that looks a little bit different than some of those problems. Like I mentioned that, you know, you have an S3 handling a large swath of data in front of them or her to figure out, hey, what's going on. And there's a lot of, you know, kind of like looking at graphs, looking at dashboards, looking at all these different pieces of information. And what we saw again from our own experience as well as in various different partners in the space is that, you know, what folks typically were doing is they were taking a very general purpose, very unopinionated agent framework, right? And like trying to put together that framework in a way that, you know, hey, I can do some pattern recognition on logs, I can do some, you know, pattern recognition or RCA on, on traces or metrics or something like that. And really what's, what's interesting about why we went down that path initially is, you know, we really just wanted the outcome of getting better RCAs faster RCAs be able to troubleshoot our own systems, better use that internally better. And what we found was like our experience of going through that journey and some of the folks that we initially partnered with were very similar to a lot of the things that we've heard from various different SRE teams and platform teams in the space is like, you know, it's like the building of agents Especially for managing production, is very much like the iceberg analogy that we've seen for a lot of different things like kubernetes like cloud, etc. Where, you know, everybody's thinking about the tip of the iceberg. And like, you know, hey, how do I just take a model, wrap it in a simple agent framework and give it a prompt. And in reality there's, you know, a lot of things that are needed to actually make that reliable, right? Like how do you make sure that the model can consistently call tools and that those tools don't just like randomly fail? Or how do you orchestrate something, you know, fairly complex like a multi agent workflow without making somebody kind of get dragged through glass to get to that point where they're able to actually realize that outcome of solving that use case. And again, when we saw that with operations workflows specifically, we just saw more and more of that kind of like upfront effort in order to make that reliable workflow and is really like hampering people's ability to be successful with those frameworks. And so, you know, we really just wanted to kind of like shorten the circuit, right? And so that's really, you know, going into where Aura came into play is like it was our own learnings. We looked at, you know, a lot of different frameworks and harnesses at the time in the space, and, you know, a lot of them were focused on either being insanely, you know, general purpose, kind of like one harness for everything, one framework for everything. And they typically didn't make those hard things, like I just mentioned, very easy. There were like a lot of Lego blocks that you had to figure out what the Death Star looked like at the end of it. And what we wanted to do with Aura was, you know, kind of like what the experience that a lot of, you know, S3 teams or platform teams are familiar with, which is like, hey, if I'm deploying something in Kubernetes and I do a Kubernetes deployment, well, you know, I tell the kubernetes like what the outcome should look like. I don't necessarily need to tell it imperatively every single step that it needs to do to manage those workflows, scale out those workflows, ensure that it's reliable at the end of it. And we wanted to create that same type of experience for how ops teams built and manage and maintains agents. So again, like shortening that process to getting to the outcome and ultimately making it so that, you know, the tools that we expose, like our own MCP Server and others just become much more valuable because you can get, you know, you can get your outcome closer to day one than if you were to kind of go through that process of building all the individual pieces that are frail and kind of like have the tendency to fail on their own. You know, we wanted to make sure that that was a, not only a shortened workflow, but a very reliable workload to get to that outcome.
- [09:43]
Kevin Ball
Yeah, so that's, there's a bunch in there and I'd love to dig in a little bit. But let me, let me pinpoint one of those things because you mentioned SREs are used to working in something like Kubernetes, which is essentially like a declarative environment. Right. I declare my relation, my, my objects, I declare their relationships and the system is responsible for bringing them up, making sure that they get into that state and kind of getting there. And talking about that with regards to agent definition is fascinating to me because like, I think most of us are familiar with like coding agents of different sorts. That's probably the most widely spread agent harnesses out there are things like Claude code and that sort of thing. And there I think a lot of us are still being pretty declar or imperative where we say I want this, I want that, you know. So what does it look like to have a declarative agent definition?
- [10:27]
Andre Elizondo
Yeah, I think the proof is really, you know, the benefit of us open sourcing aura is that you can look at the repo, you can look at exactly what the examples are. And like we operated very much on like a lot of the Linux fundamentals or philosophies of like having things like sane defaults, you know, where hey, if I don't need to tune every single part of the process, you know, I can operate on the defaults built into the system, but at the same time I have the ability to you know, tune the reasoning budget, tune the quality score where as the agent is maybe executing its tools or going to an end state and trying to solve a problem, like how does it self evaluate itself and self correct itself as part of that loop also, you know, enforcing the loop itself. Right. Like ensuring that we were kind of opinionated on those things underneath, but also gave the folks that were using aura the ability to kind of customize them if they saw fit. And again it's like same thing with a Kubernetes manifest, right? Like I may just want to deploy nginx or Redis and that may just be all I need to do or I may need to scale that out. I mean, do like, you know, a bunch of different pieces along the way in order for that to be very tailored to what I'm working on internally, to my workflow. And so we wanted to do the same thing for agents. And so again, when we looked at the ecosystem, there were different ways of defining the agent in a framework. There were different runtimes, there were different kind of pieces along the way that were very unopinionated and very much like, hey, you can do this, you do a little bit of anything. And again, with the types of problems that we were seeing with the industry, we wanted to really just operate on the path of, hey, you can start using Aura, you can pick up our container tomorrow, you can run it in your cluster or even on your laptop and give it a simple config file. That's also the beauty of Aura is when you look at how you build these agents declaratively. Again, going on the manifest ideology, here we use a simple TOML file, and in that TOML file I define the agents. Whether I'm saying, hey, here's a simple agent, maybe with one tool, and maybe that tool is my observability tool, maybe that tool is my cloud provider and a simple system prompt that says, hey, how should you know? What should you do or what should you focus on? But I don't need to define, again, the reasoning loop, I don't need to define the self correction loop, I don't need to define anything along the way. But as I'm starting to get into some of these more kind of complex things where, hey, maybe I'm dealing with a tool behind that agent that I know is going to return a lot of data, right? Like, maybe there's like different MCP servers. Like, for example, the Prometheus MCP server. I'm sure a lot of people have tried to use that MCP server and you just end up getting this context bloat of like, you know, every possible thing that's returned by that MCP server. And so, you know, again, where we wanted to make things easy was we introduced different concepts along the way. Like, for example, Scratchpad. Scratchpad is something that's built into Aura natively, and it's something that you could very easily turn on with a single line in that config file and say, hey, start using Scratchpad. And basically what it does is it acts as kind of a layer in between the tool and the model to give the model the ability to interact with the outputs of that tool via like a simple file on disk. And then Ensures that, you know, it'll kind of like, look at whether or not that tool needs to be recalled, whether or not you can kind of respect the previous tool call. Like, you know, how you can kind of share that tool call across different agents. And so, again, it was really operated on that idea of same defaults on day one. Make it really, really easy to get it started, but also get it running and ultimately achieve the outcome, but also not alienate folks that really, you know, cared about tuning every part of the process along the way. That that's actually something probably worth calling out is like, you know, when we also looked at folks in the space, especially a lot of, like, commercial solutions or things that are, like, built into, you know, the various different ops tools that people are using today, like there's an, you know, an embedded agent inside of anything. A lot of those were black boxes. Right. And so one of the really important things for us in, like, our day one, you know, opinions on how we've built and released and continue to develop on Aura is really that we wanted to make sure that all of those things that we're doing are fully transparent to the user. You can not only see what they're doing, you can interact with it, you can tune it. And so that's transparency on day one was also an incredibly important thing for us along the way.
- [14:40]
Kevin Ball
Yeah, that's super cool. And I mean, even just the tiny example of scratchpad is really nice because I feel like I operate a lot in coding agent land, and it's like, you learn these patterns of, oh, let's dump this to a file and let's interact with it, or, oh, let's dump this into a SQLite database and then query rather than just raw, putting everything in context. And when you find those patterns, if you make it easy declaratively, just say, hey, agent, here you go. You know, how to use it, run with it. It's really powerful. You know, the target with Aura is this is an agent framework for SREs for kind of production use, which is once again, like, different than most of us first encounter agents nowadays, encoding agents or things like that. And maybe we're building our own somewhere, but let's flesh out a little bit more of those differences. So we talked some about the declarative versus imperative. What are the other big differences in building an agent framework for a production environment for SRE as compared to, I'm running this as a coding agent on my laptop.
- [15:40]
Andre Elizondo
Yeah, there's really, like three big things that I look at across the landscape of like, what does it take to have a really successful SRE agent for any team? You know, a big pillar across this is how we handle context engineering. And so again, like when teams are just kind of picking up a framework and just putting a model and putting a simple prompt, like oftentimes they're not thinking about context engineering on day one. And you know, for folks that maybe aren't super familiar with context engineering, right. We always have the limitation of any model, you know, big or small, is the context window. Right. And so we're always operating within that context when I'm sure many of your listeners know about this, but just to be verbose on that topic, and you know, in reality operating within the context window, it can be very simple for very simple use cases, right. And it looks very different for things like coding agents where, hey, the context window is just maybe the code or the understanding of the code base that's like right in front of me. But for SREs, right? That, that context engineering looks very different, right? Like, you know, when you think about the amount of runtime data, everything, you know, understanding the code, understanding of the infrastructure, understanding of a lot of the observability data, right? So like traces, metrics, logs, et cetera, you know, you can't just say, hey, go give me like the last week of traces and logs and metrics and I'm just going to shove it into the context window and hope that the model not only has the ability to have all that within its context window, but also doesn't like hallucinate along the way. Right. And so the thing about context engineering for these types of workflows is they automatically become like, at scale, right? Like typically because of the, the nature of, you know, us operating this in production, we're dealing with scale and those don't present themselves on day one. And so context engineering is a really big pillar of that. The other piece is really like how we handle things like, you know, multi agent orchestration. And so when you think about like some of the advanced patterns that are starting to pop up that are becoming really, you know, useful in this whole agent ecosystem is like, hey, how do I have, you know, as part of any different type of agent? Maybe that's like an agent that's doing, you know, root cause analysis or whatever, like I may want different agents focused on different tasks that, you know, can collaborate with each other and that's going to look very different in different types of use cases. Maybe I want a security reviewer, maybe I want somebody who's, you know, just focused on investigating the cluster, just focused in on investigating the code base. And that helps me to, you know, do this in parallel. So it's actually a lot faster to do that. But it also makes sure that like each one of those agents, you know, isn't, isn't dealing with again like a single context window as its limitation across the way. Right. And so like orchestrating multi agent workflows is you know, typically a non trivial thing in pretty much any harness or any framework. And so again that's like usually a day three if, you know, if anything, maybe a day four type of problem that not everybody's thinking about on day one. And so it's really limiting them the ability to gain those benefits until they've actually hit the problems with like, you know, what they're doing in the simple agent or maybe the simple agent plus one or whatever. But we just wanted to make that again like very easy to solve on day one because we believe that the nature of these types of production SRE workflows benefit a lot from this orchestrated multi agent workflow. And then the last one is, you know, how do we handle things like human in the loop memory, things that are opinionated to the task. And again that's where you know, we think about like what memory looks like in coding agents, what memory looks like for pretty much anything else. And, and it's very general. It's very like you can kind of have the agent provide memory and then the memory system is like either just a simple graph or maybe it's just a simple, you know, kind of collection of like summarizations across the context window. Whereas, you know, what you'd want in, you know, an incident investigation is really a memory system that's looking at the context of hey, what are the previous incidents that I looked at? Like what were the previous resolutions to those incidents? What happened along the way? Right? Until like this is also another problem that like is unique to these S3 workflows is really how we handle those longer horizon tasks, those longer horizon things that the agent is dealing with. And then the last piece I'll just kind of highlight as part of that is things like human in the loop. You know, how we have agents that we can give a little bit of autonomy or a lot of autonomy to. Right? Like we believe that should be not an either or, but really like a graduated process, right. Where I maybe want to start with something that looks more like a copilot and then be able to very easily grant some more autonomy where it looks a little bit more like an assistant. And then once I truly trust that assistant, and I actually want to change the way that I'm running production to really be agentic and really be autonomous, well, then I want to graduate to more of that autonomy that looks like really that dark factory when we feel like we're comfortable with turning off the lights. And so we wanted to make sure that Aura kind of took in all those different concepts that I mentioned and just make them kind of like available on day one and something that we have our opinions baked into the harness. But at the end of the day, you can plug that harness into our tools. You could plug it into your own tools, your own observability systems, cloud providers, et cetera, behind the scenes, and really kind of provide that for the community to build on as a common set of practices that we believe will enable this like, industry to move forward.
- [21:01]
Kevin Ball
You have highlighted like some of the key things, so it's okay. I'd love to dive another level of detail deeper on probably each of these. The one I want to start with is orchestration, because as you know, like, I feel like this is a problem everybody's trying to figure out right now in different forms or another. And you have people building their own orchestration frameworks, you have it being built into different things. So first, from a developer experience standpoint, what does orchestration or multi agent look like in Aura? And then maybe we can peel back the hood a little bit and be like, and just sort of look at what are the primitives, how is it built? How would, if I wanted to customize it, what would I plug into?
- [21:37]
Andre Elizondo
Yeah, so the way that we handle orchestration is meant to be like as simple as possible.
- [21:41]
Kevin Ball
Right.
- [21:41]
Andre Elizondo
So, you know, there's a lot of different things that folks can customize. Again, if I just picked up like a general purpose framework like LangChain and said, hey, I want to do, you know, multi agent orchestration. While I'm thinking about the type of reasoning loop, I'm trying to think of like the type of retries or resiliency across the way and the way that it looks like with an Aura is again that simple TOML file that's driving the whole process. I define an orchestrator or a top level agent. That agent is very simple. It's, you know, anywhere from like 5 to 20 lines of like, you know, how detailed I want to be on my system prompt. Basically that's like the majority of, of the context there. And then I basically just kind of like note that as my orchestrator agent and then what I can do is I can define various different workers. And so again, like just another subsection of the TOML file where I'm saying, hey, this is a worker that is, that is named this and I'm going to give it this description in this preamble. And I can also limit its tools so I can give it access to everything or I can say, hey, you know, these are the tools that I just want this worker to have. Thus, again, like limiting some of the potential of context bloat or you know, kind of limiting the ability for that agent to be really good at one task. And like again, it can look basically as simple as that. Where I define the orchestrator, I define the LLM that I want to use, I define the workers. And, and you know, everything behind the scenes is actually driven by aura out of the box, right? So I'm not having to think about how the orchestrator is going to handle its planning loop, right? So like the first thing that the orchestrator will do is kind of create a plan, create a go forward. Like what am I going to do? Right? Like how am I going to fan this out? What workers are available to me? It's going to go ahead and provide that to the workers where it's basically going to go into kind of an execution mode and then it's going to go ahead and kind of like self evaluate, right? So like as I get the results from those workers, hey, do I have enough of a confidence it'll actually sell for itself along the way and say, hey, you know, this is a confidence I feel comfortable with moving on to the next step. Or you know, you can kind of make it either very high of a, of a quality threshold or very low, depending on kind of what you want to do with it. Typically we see like, you know, 70% confidence is what, what we see a lot of folks kind of define initially there's. But you know, I'm not having to think about like how the tools are provided to those subworkers. I'm not having to think about how that planning and self correction loop is defined for the orchestrator. I'm not having to think about like how the orchestrator provides the relevant context to just the workers that it's currently.
- [24:02]
Kevin Ball
That was kind of what I was going to ask about is like, what is the communication path back, forward and back. Like is it just dumping files in place for them? Is it, you know, standard in and standard out? Like can it hand off more complex things? Like if I have a sub agent that pulls together a database for me, can I hand that to a different sub agent? Like, what are the tools available in the tool chest?
- [24:21]
Andre Elizondo
Yeah, so we kind of go for like a single path in, single path out. For right now, that'll change as the harness develops. But you know, it's meant to be, hey, the orchestrator does the planning and then hands it off to the workers and the workers hand it back. And then the orchestrator can hand it to different workers. If it determines that, hey, I've built some context, but I need to clarify it. I need to get a little bit more context, execute another tool, or maybe execute the same tool again, because it didn't give me the result I was looking for. And so the orchestrator becomes kind of the important part of that process. But the thing that we're starting to see more and more of is like, you know, these multi agent workflows, you know, they have a lot of tendency to become really complex really quickly. And so we wanted to make sure that was simple on day one. But like, as we see more and more of these patterns emerge, we definitely want to make sure that aura kind of enables you to gain the benefits of either performance or reliability out of what that orchestration flow looks like. One thing I'll also call out with orchestration, which is pretty interesting, is that when you think about how you are gaining better economics out of the tokens that you're using in some of these workflows, again you're dealing with scale. And so you want maybe the bigger model, maybe you want an opus on how you're doing the planning and the reasoning in the top level orchestrator. But then on those sub workers, you can actually declare a different LLM, a different model behind the scenes that you know, operates as more of a delegated task worker. Right. Like, hey, maybe I want to use haiku, maybe I want to use an open source model. And so that actually is also something that aura makes insanely easy to do using our declarative format so that you can say, hey, I want to not only use a multi agent workflow, but I also want to use smaller agents for the workers and ultimately get better kind of like token economics out of the whole process. And that's kind of like one of the things that we solved on day one that we've seen some pretty surprising, like pretty exciting results out of just enabling that to happen and kind of embracing the idea of a multi agent system with a lot of different models in the mix.
- [26:17]
Kevin Ball
Totally. Yeah. There's a lot of classic like data categorization extraction Et cetera, that is trivial to do with a small model and you really don't need to throw opus at it, but you want that as a sub piece of your, your broader thing. So yeah, making that easy is, is super helpful. So to playback, making sure I understand you're sort of right now operating on like sort of single orchestrator set of available workers, the orchestrators deciding what and just sort of spawning things out. And then it can dynamically decide, hey, I got this from here, but actually I want to send it over there doing things, but it's kind of single level of delegation, is that exactly?
- [26:51]
Andre Elizondo
Yeah, exactly. And we're to your earlier point too where we're making it as simple as possible. A lot of our interactions are just utilizing flat files on the system. And so, you know, that's something that pretty much every LLM, every agent is good at interacting with is a file on disk. And so we're not introducing a lot of like, you know, custom processes. We're kind of operating on the simple, most reliable thing that when breaks, you know why it breaks. But ideally it just doesn't break because it's a simple enough mechanism.
- [27:18]
Kevin Ball
I want to come back to talking about memory and human in the loop because those are interesting. But that points in another direction that I think is really interesting to explore, which is like the observability of the agents themselves. Like we've been talking about, you know, how you're using these to tackle observability data and things like that. But like what tools does Aura have for making itself observable so that as you said, you know, when it breaks, you can tell why it broke and how to fix it.
- [27:42]
Andre Elizondo
Yeah, being an observability company, this is like one of the things that we made a very hard requirement for us to, to solve. Basically, you know, even when we didn't ga Aura, when we were kind of in a pre release, like an early access type of thing, we instrumented everything in aura natively with OpenTelemetry. So we're heavily invested in like OpenTelemetry for the agents. We define everything. Again in the readme you can see exactly how we're instrumenting that, what pattern we're using for opentelemetry, which were built on top of open inference for opentelemetry. And so you can see exactly in the spans of that trace of the agent interaction, you know, hey, how is it doing? The reasoning for the next step was the tool call successful, et cetera. So we organize that automatically and we build on top of opentelemetry to make that happen. And the other thing that we have natively built into Aura is a really rich set of server side events that can be returned back to the client that can really show you like what's happening live as part of the investigation process or like what is the agent doing behind the scenes. So definitely didn't want to get to the point where it's like, hey, I give this really complex task to maybe a multi agent workflow and I just kind of like wait on the spinner or anything like that. Like as that's executing, we have a CLI built into Aura. So you can very easily just like start interacting with the cli. And that is, you know, it'll very easily show you all those different events that I just mentioned. Right. So like you can do slash stream, you can do slash expand and it'll show you like, hey, what are all the activities that I can see from the agent happening, you know, behind the scenes as I'm interacting with it? And then, you know, because it's built on top of open telemetry, I can send that to my observability system. And that makes it really easy for me to understand, you know, the auditability of like the interactions from the agent. What did it actually do and why. Right. So like the reasoning steps across each process are also captured in that trace. And so we just wanted to make sure that, you know, one, it's built on open standards, you can send it to anything that supports opentelemetry and it becomes a great way for you to start building trust in your agents. Right. Again, like this is a big barrier in the industry. I'm saying is that, you know, for, for SREs, like anybody who's dealing with like a production use case, you really have to understand exactly what's happening behind that agent in order to trust it. Otherwise, you know, you're not going to put it into production, you're not going to, you know, offload more tasks to it, you're not going to really integrate it into your workflow. And so the observability or the transparency of what's happening behind the scenes, like that was a insanely important piece that we wanted to solve on day one.
- [30:09]
Kevin Ball
One of the big patterns I'm seeing right now, once again doing the metaphor to coding agents, because that's where I live a lot of my time is like looking at traces essentially of how are things going and using that to iteratively improve. And you know, if you're talking about an observability agent or an SRE agent, it needs the same thing. But you can't just be looking at files on your laptop anymore. You've got to have those deep observability traces.
- [30:31]
Andre Elizondo
Totally. Yeah, exactly. Because we're built as an observability tool. You know, our platform enables you to send those traces directly to us. So obviously we have the ability to pull those in, visualize those, show you the different service mappings and the interactions. Most people have an observability tool in place and so it's easy for you to take Aura and just point it. Whichever OpenTelemetry provider you decide to use, the visibility that you're going to get into those workflows is not going to be limited to you using our commercial product.
- [31:00]
Kevin Ball
Let's come back and talk a little bit about memory because I think this is also a really interesting domain and I'd love to get once again one or two levels deeper on how you approach it. As you commented, a lot of memory implementations out there are pretty naive. I'm going to use an even simpler example. You look at ChatGPT memory or something and it remembers everything. It doesn't give me any visibility into what it remembers or why. And you know, when I'm working on some thing for my son's project, it says, hey, because you're a director or you're a VP of Engineering, you should be thinking about it this way. And it's just this one big undifferentiated message. Obviously that's not what you want in a production SRE context. You mentioned examples of exposing incident logs or incident histories and things like that. So like, how does your memory architecture work once again, first from a user perspective and then like diving in under the hood.
- [31:55]
Andre Elizondo
That's a good question. And you know, a lot of this you can actually see in our PR is like how we're developing the next iteration and what we're. What we're already starting to improve within Aura. But the way that we look at it is like there's different levels of memory and so, you know, there's memory like more turn based or session based, basically like that you want the LLM to be able to recall, you know, what was the last tool call that happened on the last couple seconds?
- [32:18]
Kevin Ball
Right.
- [32:18]
Andre Elizondo
Like, or what was the previous run of this agent able to come to as a conclusion that maybe is is right in front of the agent at that moment. Right. And that's more of the turn based or session based memory, which is kind of like the lower levels of where we want that really fast. Kind of like when you think about memory architectures with like, you know, having DDR is like, you know, my, my RAM versus a hard drive versus, you know, something that's like off site storage. We kind of took the same approach with Aura where there's very simple memory that's built for turn based and session based, that again you can tune along the way. And then what we're looking at, like the really interesting problem is when we start to get into some of those like longer horizon memories. And this is actually where like, you know, we are working on this within the platform itself, within the Mezmo platform, to offer this as something that, you know, is a little heavy to do in line, but it's actually great to do kind of as an async or background process that just gets exposed to the agent that it doesn't have to waste tokens on. And so this is where we're looking at like again, more of the kind of needle in the haystack incident over incident investigation, but like tailored to the types of workflows that SREs would deal with where those memories are not just about the last tool call or the last iteration of this agent running, but it's like, hey, what happened in the last six months? What happened in the last 12 months? What happened with this service that I started to see maybe a little bit of service degradation and also a change that somebody pushed from GitHub and also started to see some funky stuff in the logs that all becomes a collective memory that we're kind of creating as a simple briefing to provide to the agent that just makes it easier for that agent to either solve the next thing the next time better, or potentially just short circuit the process entirely and say like, hey, you know, normally I would kind of come to this conclusion of doing this next thing, but I can see the last time I did this, this thing and I can kind of compare that to ground truth in the background. So I can kind of validate whether or not that had the intended impact that I needed. And maybe that'll change what I do the next time that this happens. Because now I've actually evaluated that in the background. And so that's kind of like where the longer horizon memory is like something that we're very interested in. Because again, if you look at like, you know, just taking either the graph, you know, memory MCP server that most people are familiar with, that you can just grab off of kind of the standard MCP servers or something like MEM0 or MEM palace or any of the kind of very general MCP, you know, memory servers. We wanted to make sure that that was again, shortening the circuit to the outcome or shortening the path to the outcome, where we know the types of tasks that SREs need to solve for and we know how memory becomes really important to some of those longer horizon tasks. And so we're just going to provide that on day one.
- [34:54]
Kevin Ball
What does that end up looking like technically then? Is that a kind of rag like implementation? Is it exposed via another. An mcp or it's a tool that. A separate tool that you can query or like, how does that end up actually playing out in the agent?
- [35:09]
Andre Elizondo
Yeah, so the way that we're exposing it today is just taking like the simplest, again, lowest common denominator across the board, which typically ends up being another tool. And so, you know, the way that we look at it, like we call them investigations. And so as I'm handling either a new investigation or looking at previous investigations, that's just another tool available to the agent that we expose through our MCP server. How that practically looks right is we're providing kind of a briefing as part of that tool call. So it's kind of like the summary and then like where the agent can go for the next steps. Something that again, like, because we're kind of focused on building on top of these, you know, lowest common denominators in the space, which a lot of this looks like, you know, just an MCP tool. You could plug what we're doing in the background for memory, or what we're doing in the background for context engineering, like longer horizon things into aura. And that's, you know, what, what we see as a great path to consume that. But you can also plug it into. You've already invested in that agent framework that's, you know, you've already gone through the glass to get to that outcome that we were just talking about before. Or you want to plug it into Claude, you want to plug it into wherever. You know, you can easily do that. We did experiment with a little bit. And again, you could see this in our PRs doing things like providing a virtual file system to the agent where we can kind of like seed in those memories without the agent actually having to do the tool call. And some of that's in kind of an experimental phase where we're looking at like, how can we better expose these memories that don't actually require the agent to do a tool call and is rather just like transparent to the agent where it's not wasting Tokens. But some of that stuff is like kind of in a research preview and again, you can see that in our PRs and kind of follow along with that.
- [36:42]
Kevin Ball
Yeah, no, it's, it's super interesting. And maybe this connects a little bit to the context engineering question, right, of like, what sets of things do you want to embed in your initial context? So it just kind of knows it versus make it on demand, go and call it versus. Yeah, have it in an explorable database or find, you know, some other way of finding it.
- [37:03]
Andre Elizondo
And there's also one last thing that I want to make sure I also layer on here that's related to memory. We don't always call it memory, but it's like on the same exact path, which is things like institutional knowledge, right, so like more of the institutional memory or institutional knowledge, right, like where, for example, every SRE team, right, has a set of runbooks that they have internally, maybe that lives in Confluence, maybe that lives in GitHub, maybe that lives in, you know, insert your favorite place to store runbooks, right? And so that becomes incredibly useful grounding for the agent. And so, you know, we also expose a really easy way for you to either connect us to where those runbooks are currently stored. If they expose something like a vector database, it's really easy to plug that in and be use that as grounding for the agent. So the agent will always kind of ground and respect itself based on, or respect the workflow based on what you've defined as the source of truth being your runbooks. We've also actually published a blog post on like, hey, if the agent has the ability to not only consume those runbooks, but as it starts to see things during its investigation process, how can it actually contribute back? And we've published a blog post on this, how we're actually doing this internally where we have our runbooks stored in GitHub. We actually use Aura for a lot of our own SRE workflows internally. And as we identify things that like, hey, this workflow has changed based on what's in the runbook. Instead of that becoming a human process where somebody's maintaining the runbooks and it's only a one directional, you know, like consumption by the agent, we've actually seen a ton of really positive success with providing the agent a feedback mechanism to basically iterate on those runbooks. Open a PR and say, hey, I did this investigation and I need to update the institutional knowledge. And that's just like a really good example of kind of the opinionated memory where like everybody has institutional knowledge in some way or form.
- [38:52]
Kevin Ball
Yeah, well, I love that because it closes the feedback loop. Loop. And it gets us to the last sort of pillar that you talked about that maybe we want to dive into, which is like the amount of human in the loop you have and sort of you described here a version of human in the loop, which is like I did a whole bunch of stuff, but here let me suggest to a human an improvement. But yeah, how do you think about. You mentioned which I love this idea of having the sort of gradient from almost entirely human with copilot going on to completely automated. But like what does that end up looking like, how do you end up configuring it in aura? And where do you see different types of SRE tasks typically falling out in terms of that level of autonomy versus human run?
- [39:35]
Andre Elizondo
It's interesting because the way that people are looking at human in the loop today is very dependent on the types of systems behind what the agent is interacting with. So that's one parameter or one dynamic. There's also like how confident are you in the workflow itself? And then there's also like how confident are you in the data that has been returned through the workflow? Right. So there's like an aspect of how you provide the ability for somebody to say like hey, these types of tools which maybe are read only or sorry, write, you know, they have the ability to write or they have the ability to do something that you know, it's kind of a one way door, right? Like I want to make sure that I provide a very hard kind of barrier like for the agent to be able to know that that is something that's insanely sensitive and it needs to ensure that it asks for help before continuing on. Like that's kind of one piece that, you know, we see everybody with some level of like, hey, there's remediation, but like I need to make sure that there's always a human that hits the go button before it does remediation. Because I know that that's a, you know, one way door that's really hard to come back from. That's kind of like one aspect of like limiting or providing human in the loop on the tools themselves. Then there's like, you know, how far do you want the agent to go in the process before it, you know, asks for a person to kind of let it continue on with go, right? So like maybe you're not limiting access to the tools themselves, but you're limiting access to like how far in the process the agent can get to, right? Like you can basically act as an assistant, but at the end of the day, the person who is implementing the remediation to what you've investigated is always going to be a person. I'm not going to provide you those types of tools and it's always going to be, you know, your, your read only. You're only getting me up to the point where you're creating a nice package for an SRE to go execute a script, remediate a problem. But you're coming to exactly what needs to happen before you know, that point becomes like a simple place for the agent to either determine that it needs to continue on, or you know, just completely hand off the process. And then the last one is, you know, where we start to really get into, I think the like the true benefits that people are looking for. Which is when I'm thinking about like a truly autonomous workflow, right? Where hey, instead of SREs handling this end to end, it's actually an agent handling this intent. That's really where we start to get into, you know, like an aspect of, hey, how do you use some of the data that we were talking about before with the observability data, right? And like grade that over time where even if you think that that's a pretty good workflow and you have the right data and you have the right workflow to find, you want to make sure that it is tested a certain amount of times before you give that the full like thumbs up, continue on, operate autonomously. Right? That's kind of like the graduation process. So like there's, there's those like three different areas there. Now how that looks, you know, specifically with an aura, it's all driven through the config file. Where it looks a little bit unique is like as soon as we start to get into those like longer horizon things again, where it's like, hey, I'm not just limiting access to the tool itself, but I'm telling the agent where it needs to go ask for help. That's where we're exposing a tool within our MCP server to basically provide to the agent. So like the agent doesn't need to go figure out, hey, you've told me that I need to go ask for help here. But like I gotta go figure out who's on call, I gotta go figure out how to get access to Slack, I gotta go figure out how to do, you know, xyz. Instead what we're doing is like, that's a process that we're seeing across, you know, a lot of SRE workflows that needs to happen. So like we're just exposing a simple tool for the agent to be able to ask for help. And then what we're doing behind the scenes is like some of that correlation that I mentioned, hey, who's on call? Who do I need to go ask? Trigger a notification via Slack, somebody give me the thumbs up. And then provide that to the agent versus the agent again, having to do that itself. So that's kind of like how we're not only defining the different types of human in the loop in the process. Like governed autonomy is kind of like what I think about it, but also make it so that our platform kind of provides the simple mechanisms for the agent to kind of like ask for help from a single place without having to bloat a bunch of context to go figure out all the possible places that it could go ask for help. And so like I said, the way that we operate with doing that, right, like Aura is open source. We always expect people to be able to use Aura with or without Mesmo. And so, you know, you can definitely point Aura at a different tool that's, you know, a different human in the loop process. And we publish kind of the spec there of like, if you wanted a different human in the loop process, here's how you can do it, here's how you can define it. And then we kind of like operate on, hey, we opened that spec, we've defined that spec, but then for the actual implementation, okay, great, we're going to make the simplest path for you to realize that implementation, which ends up being just, hey, our MCP server is going to naturally provide this to you on day one, so you can consume it right off the bat.
- [44:16]
Kevin Ball
Yeah, well, and I think that's a really nice way to have a pairing between a commercial product and open source. So if you say like, look, all the doors are open, you can do all the different things and here's some things that we know you're probably going to want to solve a lot of times that we solve well for you. So here you go. That kind of gets into an interesting question around sort of the future of Aura and in particular around like governance. Right. So is this, right now, this is a Mesmo run project and you all are doing most of the work. Are you looking to get it into third party foundation or how, how are you thinking about governance of Aura going forward?
- [44:50]
Andre Elizondo
Yeah. So right now, Aura is a small but mighty project. And I say that because, you know, the primary contributors are from Mezmo. We're starting to see more and more people interested in the space and the fact that we've gone with the Apache 2 license makes it a lot easier for folks to be able to contribute without having to jump through a bunch of hoops. We're a member of the aif, so the Agent foundation that's underneath the Linux foundation and we're active contributors there. And so, you know, there's definitely a future here where I think as we start to get more and more interest in Aura and you know, we have no intent of this just being like, you know, the Mesmo way to run agents. We want this to be as like general purpose of a best practice, a standard in the industry. And so I think like, you know, not to put the cart before the horse because we, we still want to get as much proof and partner with his people as closely as possible where we can very quickly, you know, make iterations where it makes sense. But I think as it starts to mature as a project definitely makes sense for you know, potentially contribution to. I think CNCF is probably pretty crowded. But like the great thing about AAF is it's, it's, it's fairly new, fairly green filled and you know, there's a lot of folks that are interested in kind of moving the space forward through that. So definitely looking at that as a good option potentially in the future here.
- [46:00]
Kevin Ball
Yeah, totally. Well and I think it makes a ton of sense to keep it tightly controlled to start with. Right. Like I've definitely seen people try to open things up a lot too early. Right. And it leads to chaos oftentimes. Right. Like the benevolent dictator for life. Even if the dictator is a company, like that's a good way to start a project and keep it, it moving. If folks have not used Aura, they are curious, they want to get started, maybe they want to try contributing or tweaking it. What is your sort of on ramp? What do you recommend?
- [46:29]
Andre Elizondo
Yeah, so one, we have a slack community. So you know, everybody from Mesmo is there. You can kind of get help if you have an idea. We're very interested in kind of like fielding, you know, what people want Aura to look like in the future and what ideas they have in the space. And you can definitely open an issue on the GitHub repo like you can for many things. PRs are definitely welcome as well if you're interested in donating your tokens. Right. Like, and we've tried to make that as coding agent friendly as possible where we're, you Know, heavily leaning on like the Cloud MD and AgentsMD and things like that. And so we're trying to make it as easy for folks to contribute to. But then again that Slack community is a great place where like, if you don't know where to get started or you're having some problems or you just need a little bit of help or have some questions, you know, pop into that community and very interested in help in, you know, helping where, where we can. And we're also very interested in partnering with people. Like if you're looking at how you want to change the future of SRE and you're, you're aligned to this conversation that we just had and you know, you want to partner with a company like Mesmo that's kind of like looking at building this, you know, firsthand and we have engineers actively working on this, you know, definitely reach out and we're like happy to provide any kind of support or like implementation, things like that to make sure you're successful. So a lot of different paths based on, you know, just community contribution versus like true commercial partnership and design partnership there.
- [47:47]
Kevin Ball
Yeah, I love that you're doing an agent friendly open source project. I know a lot of open source maintainers have been kind of resistant to agentic coding, but having that first and figuring out how do you design your workflows and your processes and your automations and everything to make that friendly, I mean that's, to me that feels like what makes sense in this world we're now in.
- [48:10]
Andre Elizondo
Totally now we're in this point where the code is cheap, but the ideas are expensive. Right. And so it's like the easiest thing for us to do is make that process, you know, as streamlined as possible. Right? Like we know that coding agents are the future and they're here to stay. And you know, it actually like lowers the barrier, right? There was a while where like in order to contribute to an open source project, you had to really be in depth in that project and like actually understand what's happening. Maybe be an expert in Go Rust or whatever, you know, because AURA is written in Rust. Not everybody knows Rust and we totally understand that, but every agent knows Rust, right? Like anybody who has access to OPUS or any of the coding agents, right? You know, you have an expert in front of you and you have your tokens and so, you know, now the donation to open source becomes, hey, if you want to donate your tokens. These projects like ourselves and other folks in the space, we just need to make it as easy as possible for you to do that.
- [49:01]
Kevin Ball
Well, I realized we're getting close to the end of our time here. Are there any things we haven't talked about yet that you think would be good to talk about before we wrap?
- [49:09]
Andre Elizondo
You know, there's a concept that we're really keen on advancing kind of in the industry and kind of how we operate as Mesmo, which is, you know, there are two kind of like axes on this in the space, right? There's what production AI looks like as far as like AI in production and like all these things that we're talking about, right? Like multi agent workflows, there's, you know, how you reliably call tools, there's memory, there's things like that. And that's all like, how does an agent operate well in production? And then there's another axis which is like AI for production, right? So that's where you think about like, what are the types of tasks, what are the types of skills needed, what are the types of tools needed that really enable these agents to handle these types of tasks that we see high value and something that's inherently hard to solve. And so we think about production AI and like how we're trying to shape the industry around like this as a problem. It's really the intersection of those two things where we want to enable SREs to not only own their own agents and kind of go through this process safely and reduce the risk to get to the outcome, but what we see as a healthy side effect, which I personally am really excited about just from being a practitioner earlier on in my career, is really like, this is our mechanism to also elevate the whole role of what SRE means, right? Where we're more, we're architecting reliability more so than responding to things that we've implemented like, like we're taking more of a design approach for how we can make systems reliable more so than being firefighters all the time. And so I think that's a really interesting opportunity that we have to move the space forward. And like I said, I'm just. We're really excited about like where this is already starting to show success and where we see this kind of advancing in the next months and years and things like that that really bring all the SREs today that are used to automation, they're used to, you know, handling these really high risk scenarios and just giving them the tools in the ecosystem to make that ready for the agent era.
- [51:04]
Kevin Ball
I love that it fits really well into something I've seen in a, in a bunch of AI spaces where like a Lot of people are trying to delegate their thinking to AI, but where it really shines is when you can use AI to elevate your level of thinking and really do something that's more impactful, deeper, more. Just leveraged all of those different things. So can you, like, maybe pull one example out of how using this at Mesmo has helped you elevate your SRE approach?
- [51:32]
Andre Elizondo
Yeah. So one is, you know, again, like, we've taken the approach of making it less like a firefighter and more like a reliability architect. So all of our SREs, right? Like, we have all of our engineers on call as well, right? Like, we take the approach of, like, the SREs are the ones building the guardrails and building the systems, building the workflows, but that makes it easier for other folks to also take that pager, right? Like, be on call for the systems where, you know, all that tribal knowledge that typically was trapped in an SRE's head, like, agents are a really good mechanism to make that kind of a next iteration of, like, as code, right? Where, like, we get that out of people's heads and make it more automated, make it more standard practices. And so that's really helped us to really iterate internally on what's the next iteration of SRE and really use our own ability to capitalize on our own harness and our ability to incorporate our own learnings there. And we really just elevated our SREs internally so much to the point where it's made the role really exciting and really given people internally the ability to see what the feature looks like, which then makes them also excited to talk to other SRE teams at other companies, which has been kind of a cool sight to see, is like, we're getting to the point where we can actually share knowledge, build a community, build, you know, kind of like these standard practices that extend outside of Mesmo as well.