Agentic DevOps at AWS

Summary

Software Engineering Daily — "Agentic DevOps at AWS"

Date: July 16, 2026
Host: Matt Merrill
Guest: Neha Goswami, head of Agentic DevOps at AWS

---

Episode Overview

This episode dives into the rapidly evolving world of agentic DevOps, focusing on how AWS is leveraging AI-driven agents to transform the operational layer of modern software delivery. Host Matt Merrill interviews Neha Goswami, a long-time Amazonian and leader of agentic DevOps at AWS. They discuss how AI agents are being used to reduce operational toil, increase deployment safety, automate incident response, and the cultural and technical changes this shift creates, both inside AWS and for customers.

---

Key Discussion Points and Insights

1. What Is Agentic DevOps?

• Definition: Neha describes agentic DevOps as the application of AI agents that reason over operational data, make decisions, and autonomously take actions in the software delivery lifecycle ([04:42]).
  • "My definition... is that you have a model in the mix which is doing the reasoning, but you’re really using the model to drive a lot of reasoning decisions and deciding which tools are the right tools to use for the agent to achieve a goal." (A, [04:42])
• Catalyst & Cure: Agentic DevOps is both a catalyst (matching code velocity with ops velocity) and a cure (reducing developer toil by automating root cause analysis, build/test failures, and injecting best practices into pipelines).

2. AWS’s Approach and Philosophy

• Customer-Centric Design: AWS intentionally does not aim to build walled gardens; integration with external tools (GitLab, Datadog, Splunk, ServiceNow, etc.) is seen as essential ([07:55]).
  • "Our mental model has not been about building a service just because we can. It's really been about...what do they really need to run their operations...and we work backwards from that." (A, [07:55])
• Choice and Integration: There’s a guiding tenet of meeting customers where they are; AWS builds higher-level orchestration rather than trying to replace every single tool ([09:42]).

3. Internal Use and "Dogfooding"

• End-to-End Ownership Model: AWS engineers are accountable for building, deploying, and operating their services — there are no hand-offs to separate QA or ops teams ([10:31]).
• Campaigns and Centralized Change: Large-scale changes (e.g., Java upgrades) are orchestrated centrally and now powered by AI-driven agents, saving enormous time and developer effort ([13:20]).
  • "One of the big changes...is how we run campaigns....The mindset change [from GenAI] is we can actually go much further to make the life of developers easy..." (A, [13:20])
• Dogfooding as Product Validation: Feature development for agents (e.g., DevOps Agent, Security Agent) is grounded in AWS's own use, with new tooling rigorously tested internally prior to customer launch.
  • "Before we launched this product, many of our internal teams were using DevOps agent....That gave us confidence we have something that we can take now to the public." (A, [16:13])

4. How DevOps Agent Works — From Alarm to Root Cause

• Incident Lifecycle:
  • As soon as an alarm breaches a threshold, a ticket is cut and the agent is activated.
  • The agent leverages pre-built topology maps (system dependencies, ownership, etc.).
  • It analyzes logs/metrics (from CloudWatch, Datadog, Splunk, etc.) and populates the ticket with likely root cause, hypothesis trail, and suggested mitigation steps ([25:23]–[28:14]).
  • Engineers “wake up to a root cause, not just an incident” and primarily act as auditors of the agent’s work.
  • Agent feedback and learning are tightly coupled with both automated and human feedback mechanisms ([19:52]).
• Customization: Teams can tune accuracy easily by adding runbooks and team-specific knowledge, reaching up to high-90s accuracy rates ([22:32]).
• Deduplication: The "triage agent" deduplicates incidents (e.g., coalesces five alarms with the same root cause) ([30:23]).

5. Determinism vs. Probabilistic Agents

• Core Issue: While DevOps traditionally values deterministic tooling (same input, same result), LLM-based agents are inherently probabilistic.
  • "DevOps is built on determinism...agents are probabilistic..." (B, [34:11])
• AWS Approach: Determinism is injected into safety/security boundaries (e.g., default read-only permissions), controlled action spaces, and deterministic tests within automated reasoning systems ([34:42]).
• Future Direction: Combining automated reasoning (symbolic/mathematical models of state change) and neural/symbolic hybrid techniques to provide more predictable, safe behavior ([36:30]).

6. Feedback, Evaluation, and Continuous Improvement

• Benchmarks:
  • Combination of internal/external benchmarks, “live” production data, and human review ([19:52]).
  • Engineers can provide explicit feedback (thumbs up/down, interviews, etc.).
• Iterative Improvement: Negative feedback and postmortem analysis are critical; all user feedback is looped back into product improvements ([37:19]).

7. MCP Integrations & Customer Creativity

• Extending Agents: Customers are using agentic extensions to do not just read operations, but write/update incident logs, modify tickets, or interface with workflow tools ([40:42]).
  • "Some of the more interesting ones… go and update tickets… or keeping track of what incident has happened and what is the next step… based on this agent’s result." (A, [40:42])

8. The Future of SREs and DevOps Roles

• Human Roles Evolve — Not Disappear: Agents will handle the trivial and pattern-matched incidents, SREs/operators will be called on for truly novel or complex issues ([42:05]).
  • "SREs will deal with more complex scenarios where the agent just fails and so the agent does half the work... I think that's actually a good thing..." (A, [42:05])
• Skill Shifts & Blurring Lines: Both PMs and managers increasingly prototype/coding; engineers more product-focused — traditional boundaries are blurring ([44:33]).
• Advice to Engineers:
  • Engage with agentic tools — let agents take grunt work, play with prototypes, and focus your time on more complex problems. Staying relevant means embracing, not resisting, these changes ([46:01]).
  • "Use them to make your grunt work go away... move up the stack to hire more complex problems." (A, [46:01])

---

Notable Quotes & Memorable Moments

• On AI-Driven Change:
  • "DevOps, the way I see changing with agentic development coming in, is it's both a catalyst for change and the cure for change."
    – Neha Goswami ([04:42])
• On Dogfooding:
  • "Dogfooding is, for me, essential. That's the way for me to get active feedback. And we have some of the most complex systems in the world. So what's a better way to get feedback..."
    – Neha Goswami ([18:58])
• On Determinism:
  • "I don't think determinism goes away at all actually. ...What is going to be fun and challenging is how do you inject determinism in the right way into an agent..."
    – Neha Goswami ([34:42])
• On the Future of SREs:
  • "SREs will deal with more complex scenarios where the agent just fails... The agent does half the work, maybe it’s incomplete. Those are the places where the SREs and developers will get engaged..."
    – Neha Goswami ([42:05])
• Advice to Practitioners:
  • "My hope and wish for all the SREs and the DevOps folks is just move up the stack to hire more complex problems."
    – Neha Goswami ([46:01])

---

Timestamps for Important Segments

• [04:42] — What agentic DevOps actually means in the context of modern dev/ops workflows
• [07:55] — AWS’s strategic choice to integrate, not compete, with existing tools
• [10:31] — Internal AWS DevOps practices and end-to-end ownership
• [13:20] — Example: Centralized Java upgrades with agent assistance
• [16:13] — Motivation and origin for DevOps and Security agents; "pager pain" & deployment velocity
• [19:52] — How internal evaluation, benchmarking, and feedback cycles work
• [22:32] — Internal learnings: successes, accuracy, and challenges
• [25:23] — The experience: from alarm to analysis and root cause for on-call engineers
• [30:23] — Triage agent: deduplication of tickets and cost management
• [34:11] — Determinism in a probabilistic agent world
• [37:19] — Transparency, observability, and agent learning
• [40:42] — Creative customer MCP integrations
• [42:05] — The future and training of SREs with AI/agents
• [46:01] — Advice for mid-career DevOps/SREs in an agentic world

---

Closing Thoughts & What’s Next

Neha and Matt reflect optimistically on the movement of engineers “up the stack” as AI agents take on more and more operational grunt work. Neha is enthusiastic about information synthesis and the promise of agents evolving to safely carry out more sophisticated actions, freeing engineers to focus on innovation and higher-value challenges ([47:51]).

• "I am really excited about what actions [agents] can take as they become more powerful... These agents will be taking a lot of actions on our behalf... I have more time to think of other complex, other creative things..."
  – Neha Goswami ([47:51])

---

For practitioners and engineering leaders, this episode provides both an insider’s look at AWS’s technical and cultural evolution as well as practical, forward-thinking advice for adapting to — and thriving in — an agentic DevOps future.

Transcript

[A]

AI agents have become capable of reasoning across large amounts of data, calling tools and taking sequences of actions autonomously. These qualities make them well suited to some of the most persistent pain points in DevOps, including the on call engineer woken at 3am to diagnose an incident, the build failure that takes hours to trace back to a root cause, and the operational toil of modern software delivery. AgentIC DevOps is the emerging paradigm that applies these capabilities to the full software lifecycle with the goal of matching the velocity of modern software delivery with an equally capable operational layer. Neha Kazwamy leads agentic DevOps at AWS and has been at Amazon for over 20 years. In this episode she joins Matt Merrill to discuss the AWS approach to agentic DevOps how Amazon dogfoods its own DevOps tooling, how their DevOps agent works from alarm to root cause, why determinism still matters in an agentic world, creative MCP integrations, customers are building, and what the future holds for SRE engineers as agents take on more of the operational work. Matt Merrill is a software engineering leader with over 20 years of experience building and scaling software teams across enterprise and product focused organizations. His background is in backend development, cloud architecture and distributed systems design. He currently architects and delivers SOFTW and leads a team of engineers at Dept. Agency. You can learn more about his [e-mail].

[B]

All right, hey everybody, I am Matt Merrill and I'm here today with Neha Goswami from Amazon Web Services. And before we dive into our topic today, which is agentic DevOps, Neha, can you tell us a little bit about yourself and what you do at Amazon?

[A]

Yeah, absolutely. Glad to be here Matt. Let me start with what I do here at Amazon. I lead agentic DevOps in agentic AI organization in AWS where my team is building products which bring agents into DevOps and application security aspects of development. I've been in Amazon for almost 22 years at this point and fun fact, I started my journey as an STE straight out of college in a team called Builder Tools. So I was building tools which was mainly focused on internal Amazon developers and I was there in the team for three years and then moved around in different parts of the company and came back to building developer tools four years ago and that has been like the most fascinating journey for me because I came back.

[B]

That's awesome.

[A]

Yep, came back to build the same tools I was building but for external developers and in the world of AI. And it's just fascinating to see how this World has changed in all these years.

[B]

Yeah, honestly, I've been really looking forward to this. I can't wait to talk about it. And what a great prep for dealing with outside developers. How many developers and engineers does Amazon have, roughly? Because, I mean, that's a heck of an audience to lead a set of tools for.

[A]

Yeah, I don't know the latest numbers, they keep changing, but thousands is what I can say. Yes, we have thousands of them. And to date I, I maintain this outlook that developers have been the hardest audience to please, whether internal or external.

[B]

Very difficult. Yes, yes. You have my utmost respect. Yes. And to the audience out there, we wouldn't have it any other way. Right. That's what the engineers do. So. Okay, cool. So today we're going to talk about DevOps in general and agentic DevOps. I can't wait to dive in to see what that progression has looked like, especially at Amazon. And I would love to take a view on this conversation of you'll hear me ask some questions about is Amazon dog fooding these things? They are drinking their own champagne, as they say, and how you release these features. So I can't wait to hear about this. So let's start. And I think agentic has become a really big blanket term for a lot of things. And I think like engineers have a very specific definition of agentic, but I think marketers have a much different one. Right. And I'm curious, from your perspective in the DevOps space, what are you excited about in the DevOps space with agentic and what do you think it really means in the DevOps space?

[A]

Yeah. So roughly speaking, my definitions of agentic is that you have a model in the mix which is doing the reasoning, but you're really using the model to drive a lot of reasoning decisions and deciding which tools are the right tools to use for the agent to achieve a goal that it's given. And we've now used agents a lot for coding. So we've had like so many coding tools available for many years now that are making it easy for people to generate a lot of code. Agentic for DevOps for me is more about, okay, now your code is ready and data is showing us that code is being produced at an unprecedented rate using these AI tools. What's the next step? And really DevOps, the way I see changing with agentic development coming in is it's both a catalyst for change and the cure for change. It's a catalyst because with all these code changes that are now lying around for code reviews, the challenge is how do you take them to production safely and you match the velocity at which the code changes are being produced in the first place. I think that's the problem and that's the catalyst of why DevOps had suddenly become much more important. The cure is, and this is the exciting part for me is agents pretty good at doing, reasoning on why our bills are failing, why our pipelines are stuck and what you should do to fix them. Agents will also help us solve some of these problems in the DevOps world. And that is really, really exciting to me. And we've had things like ise, we've always known those are really important parts of the DevOps cycle. Things like testing mean it has existed for a long time. Basic practices for good software development. But I think agents are going to make a really big change by taking these elements and making them easier for developers to inject them into their DevOps workflows. So that's what I'm super excited about is we've seen agents do a lot of work for coding, but what can agents do to make your pipelines better, to make your build failures go down, to make your changes safer? I think that's the exciting part.

[B]

And so I think that you have kind of a flagship product right now. It's called DevOps Agent. Pretty straightforward name, which is good. I like that, I like that about this AWS service. Cause I don't have to have this like, oh, what's that name that I need to remember? So anyway, but that what I was really pleasantly surprised to read about is that the integration list is like a who's who of the DevOps realm. It's GitLab, Datadog, Splunk, ServiceNow. And I saw you say something in a previous interview that AWS isn't building its own CICD platform or code repo. And so for someone like me who has a mental model of once we're in aws, we're going to stay within AWS because they have a service for everything. Feels like a pretty big shift. So is it? And how are you thinking about that? And is that a pivot or tell us more about the thought behind that.

[A]

Yeah, that's interesting that you say that because an insider view from AWS and Amazon in general. For all these years, our mental model has not been about building a service just because we can. Our mental model has really been about what do customer needs are and what do they really need to run their operations to build the systems that they need to build for their customers. And we work backwards from that and in working with so many customers over the years, what we found was the environments for software development, the tools that they use, whether it's the CICD pipeline or is it some policy related tools that they're using or even the code repos. There is a lot of variety in these development environments and the problem space was really on top of these environments. So the problem space was how do you connect the data or the process which is maintained in these disparate tools and give solution at that level to the customer. So that's why we have not been obsessed with building the next code repo, so to say. But it's really been about people are using these code repos to do their work today, then they're using a pipeline system to do the next step of that work. How can we help them at a higher level tie all these tools together to reduce their operational toil? So I'd say it's a much higher level goal that we are taking and if it so happens that in order to achieve this higher level goal of reducing the ops pain for our customers, we need to build a code repo, we need to build pipelines. Sure, we'll do that. But it's really about solving that. Where is the customer problem?

[B]

That's really nice. And by the way, just to clarify, that's my mental model. It doesn't make it right.

[A]

So thank you for quickly. I'll add one more thing to there, Matt. One of our tenets in AWS is also about choice. And you see that with things like Bedrock where we give people model choice and it's like you have open source models, you have the models from the large AI labs and the same principle applied to our developer tools as well. Like we want to give people choice. Okay, you're using these tools, which are not AWS tools. Great. We will meet you where you are. So that's another tenet that has sort of guided our mental models and what services we have built.

[B]

Okay, great. So I'm curious, this is morbid curiosity. Amazon Web Services at the scale it's at, how does Amazon Web Services do their own DevOps and how has that evolved, especially over the course of the past year or two?

[A]

Yeah. So Amazon and AWS in general, we have a very, very strong culture of end to end ownership for our developers. Our developers build services, they are also responsible for testing them. They're responsible for all the deployment processes and to actually run these services in production. So we don't have a separate ops team, we don't have a separate DevOps team or even a QA team for most of our organizations within Amazon. And I think that has been a big guiding factor to how the products, the AWS, DevOps agent and even the security agent came about. But our model is very much around centrally owning all parts of operations by development teams. We feel like by doing this and by having this mindset, our services are just designed better for sustainability. So developers have a vested interest and they have accountability to making sure that their services run well. And that has really benefited us as an organization and as a provider of these companies core services to other customers. Now we do have a central team which is responsible for doing a lot of centralized tooling and they provide platforms and tools which are used by a large majority of the team. For example, our pipeline system is owned centrally. We have policy management systems which are also owned centrally. Our build and deployment systems are owned centrally and we have ticketing systems. So we do have a fair bit of machinery and tools which are owned centrally, but a lot of the ownership around how to configure these tools and run them that is owned locally. So that's the way we've run DevOps throughout the years. And I think it's really been a very strong culture within our organization. Now coming to the side of how has it changed over the last few years, I would say for the central team, I have seen some pretty remarkable changes. One example that comes to my mind is around how we run campaigns. Campaigns are large scale changes. They could be API upgrades, they could be migration from one framework to another that impact a large number of teams. It's not localized to a couple of teams here and there. At our scale it really makes sense to run these campaigns centrally. And one of the big changes I'm seeing in the last couple of years is how these campaigns are running and how far we are taking these campaigns to the left in terms of the dev cycle. You might have heard the stat around how we were doing Java version upgrade centrally and saving millions and millions of dollars. Several.

[B]

I haven't heard about this actually, but that sounds interesting.

[A]

Yeah. So this was the stat we published when it was many ST years of work that was saved in doing a Java upgrade. And this is almost like 2 years old that we had this stat and that was done centrally and that was using the technology that we now have in our product called AWS Transform. And what was interesting about that was not just the time savings, it was just how far left we were going. Now you have a central team that is able to update so many software packages in one go produce a PR for them to review. They still own the last mile of making the code change happen. But those changes were also verified in the sense we would do a build of those changes to make sure that the build doesn't break. And we could go that far. We could do some testing for those teams on behalf of those teams. I felt like that is the change that Genai introduced in some of our central workflows. It was not just a tool change, it was a mindset change too. There's a mindset change that we can actually go much further to make the life of developers and developers easy who now have to accept these campaigns and make some service changes. And they have many of these campaigns happening at the same time. So I felt like that was an example of where it made a material change for individual development teams. Last two years have also been super interesting. A lot of teams are using Kiro to build bespoke agents that are doing things like they're basically putting together DevOps agent and some other agents that they have built into one consolidated workflow. That has been pretty remarkable to see. Many teams are of course using DevOps agent internally as well for managing their incident response, but they're also marrying it with their other custom homegrown agents that their teams have. And it's been pretty remarkable to see how they are doing that using tools like kiro.

[B]

Very cool. This was a few minutes ago, but you use the word toil and there was another interview I did on this podcast. If somebody used that term as well, they called it the Great toil shift. This grunt work now is like being pushed more to the right of how do you verify things with the speed of change. But what I like about what you're saying is like, no, this is actually pulling all that power and the change to the developers to be able to automate more of this with LLMs, if I'm hearing you correctly. So that's awesome. So I was tempted to ask the question what does the DevOps agent and security agent do? But I'm going to approach it from a different perspective of what is the origin story of these? And what I'm particularly curious about is was AWS or Amazon dog fooding or drinking their own champagne of these tools themselves before they were released to the general public? So yeah, what's its origin story? And that might lead us into what it does.

[A]

So I would say the motivation to build these agents definitely came from our internal development processes. We've had many years of focusing on what we call pager pain. How often are your SD is getting paged? How often is it pages at night?

[B]

That is definitely toil. Yes.

[A]

The other one which we have obsessed over is deployment velocity and deployment safety. So those have been the other two pillars in every year we would have teams and orgs take goals on improving that compared to the prior year. And I would say they've been top of mind for us for many, many years. And that is where the overall motivation for these two agents came from. It's like, how do we speed up the development cycle and where do we see the most operational pain for our developers? So the operational pain, obviously the pagers, was the top of the list. So that's where we said, okay, let's go attack that pain. And then it was, how do you get changes out faster? Application security reviews take a long time, and I would say rightfully so, because we want to be obsessed about security. But the fact of the matter is that they do delay releases. They introduce yet another part in the process that it takes for you to get your code out there. So the motivation came from looking at our workflows. And the one which got us most excited was definitely the incident response of like, what problem do we want to solve in ops? Is what we were looking at. And we're like, oh, look at the number of times our engineers are woken up. You must have heard that in some story. 3am in the morning and something has gone wrong in the system. Let's start with that. How can agents help there? So that became sort of our hero use case to get the product out there. And before we launched this product, many of our internal teams were using DevOps agent. They were seeing success, which was, I'd say between 85 to 95% depending on what the team was doing and how they had configured the agent. And that gave us confidence we have something that we can take now to the public. Because we had testimonials, we had enough evidence that this is something that can work for our customers as well. For Security Agent, our internal security team started to use that as well. And we got a lot of feedback from them around where the tool is doing well or how can we improve it. So we have Innovate. Dog fooding is, for me, essential. That's the way for me to get active feedback. And we have some of the most complex systems in the world. So what's a better way to get feedback than to have your.

[B]

And that is exactly why I was curious to talk about. Yeah, that's awesome.

[A]

Yeah. So that's about how we build these agents. The other thing that the reason why we wanted to dogfood these was the more we learned internally, the better flywheel we have. We found so many defects just because internal teams were testing our product as we developed it. And with every iteration of a release, internal release, we would make a lot of improvements. We'll find things that we were not great at in terms of quality. We'll go and systematically address those things and build that into our product. So I feel like it's a beautiful flywheel that we were able to create as a result of just being part of Amazon.

[B]

That's awesome. So this is something I actually ran into specifically today is just thinking about how to evaluate LLM based systems. How did that flywheel work? How are you evaluating that stuff? Was it humans voting? Was it? I'm just curious.

[A]

We do a few things. We have created an internal benchmark, I think on both sides for both our agents and in general, I would say methodology that we followed. For some of our AI functionalities, we create internal benchmarks. They're not just internal representation. So you'll have an internal benchmark which is for. In our internal system. So we would test on that. Then there is a parallel benchmark which is more representative of our current workflows internally plus externally, what do we learn from our customers that we can incorporate in our benchmark? And anywhere where you have public benchmarks, we would use that too. And I would say to start with, a lot of the evaluation was human, so we would have people actually verifying the output. We have cases where we also automated a large portion of that using LLMs and but I don't think we saw a case where we could entirely delegate that to LLM. I would say that at least in my experience, you have to marry these benchmarks with what I call online data. Online data is like essentially things that are happening in production. So when we integrate these agents into our internal workflows that say the DevOps agent is integrated with our ticketing system, for a lot of teams, we actually get to see their feedback. They provide us that feedback and we do have a person who's looking at that feedback, but we also have an automated system that's scanning through that feedback and it's identifying the things that we don't get right. Our accuracy is not 100% and very systematically we go and then figure out which ones of those are improvements that are broad brush improvements that we need to get back into the product, which ones are just misconfiguration so that's a different kind of improvement which we need in our product so that people don't misconfigure the product. So that flywheel, again, you asked the question of how do we evaluate. It's a combination. It's a combination of automated evaluation with humans.

[B]

And that feedback capture mechanism, is that like thumbs up, thumbs down, just plain text from people? Is it something else?

[A]

It could be thumbs up, thumbs down. It could be like a dropdown things that we are giving them. Sometimes we ask them much more specific questions and if we have access to those teams directly, we just actually sit down and interview them. So that's very.

[B]

Yeah, can't beat it. Yes, that you can't beat.

[A]

Yeah.

[B]

Okay, cool. That's nice. So one of my questions was, are you using it in production? It sounds like you absolutely are. Do you have any particularly interesting wins and you know, the interesting losses that you may have hit along the way? Like any particularly juicy things that caught or missed?

[A]

I would say the interesting things that we are catching. Some examples have been we found issues with lambda functions. There was a particular case when we were able to detect port exhaustion on a particular hose. We had a singleton running and just basically creating a bunch of connections and then we ran out of them. That was detected by our agent. That's an example that comes to mind with a few teams. They actually benchmarked against their homegrown agents and found that what they did with DevOps agent was they would get a base level accuracy of 85% but then they customized it. So I added their own runbooks as an example example and they were getting into like high 90s with that. So that was pretty remarkable to see. Just in terms of accuracy, I would say that is sort of a larger theme of things we have learned internally. So if you're asking me for the challenges I would say that we have seen as we have scaled internally are mostly around this customization bit, which teams have had to do some last mile configuration. But I call it like it is very easy to do that. It's not very heavyweight. But I feel like if you want to really ratchet up the accuracy as a team, you add your runbooks, you add some of these basic things which are very bespoke to your teams and the agent does much better. So that was one thing. The other thing which we learned, and honestly it was a challenge that we knew about, but I wish we had a better solution was we have so many bespoke systems in Amazon for even logging or metrics. We have teams that are using different version than vanilla CloudWatch. And the learning was obviously, the more information the agent has, the better it will be at giving you the root cause of what went wrong. And access to the information was one of the bigger challenges that we ran into because we said we support MCP integrations and we do support a lot of them, but internally, the number of MCP tools that teams have built and integrating with each one of them, that was a very insightful journey. We had to build a lot of what I call out of box integrations, working with our central platform team, so that every team did not have to go through that pain of, oh, now I have to integrate with my this system. We looked at what are the most popular MCP things that we should integrate with and we did that centrally. And that took us time, but I feel like that is the way for big organizations to actually scale DevOps agent is to build some of those things centrally. Don't leave that for your individual teams to configure. This is a much better way to scale.

[B]

Yeah, the MCP sprawl has already begun. That's what I'm hearing. Yeah.

[A]

Yeah.

[B]

So there's a line from the GA announcement that says, on call, engineers wake up to a root cause instead of active incidents. And so I'm kind of curious what this product is like in experience. So somebody gets paged at 3am and we have the DevOps agent enabled. What does that look like?

[A]

So the journey actually starts much before the human gets paged, the engineer gets paged. When A team configures DevOps agent in at least the internal way we are doing it is we have a system for ticketing and we have a way such that we model our teams and which team owns what system. And for every system there is what we call a resolver group. So that represents the team of people that owns that system. And anytime something goes wrong, those are the people that get paged. When you configure your resolver group and your resources for DevOps agent, it does some work upfront to create a subsystem topology. And that is the intelligence that it has inherently. So when you do get paged, it will tap into that topology that it has already created for your system. And the interface for the user is really the ticket. So there is a ticket and it has different severity levels. For the highest severity levels, we page people. For the lower severity levels, we just cut a ticket and they can look at it during the regular working hours. So when assume your system's already hooked up. Your ticketing resolver group is configured to use DevOps agent. When you have an AutoCAD today, your engineer wakes up with the pager going off. The agent is kicked off as soon as the autocut is triggered. So your alarm basically what was the term you used?

[B]

Autocut.

[A]

Autocut. So as soon as an alarm breaches a certain threshold, you have an autocut ticket. And when that autocut ticket, what it does is as soon as it reaches the threshold, it also pages the engineer. And that is the time when we start the agent to get in. The agent to the agent will look at what happened. It'll look into the topology. It will give you a root cause of what I think went wrong. The engineer wakes up and looks at the ticket and the ticket now has information from the agent on this is the time and the alarm went off. This is the system involved here is the root cause. And I think what is really helpful for the on call engineer is that it gets to see the agent work where all has it looked. How did it come to this conclusion? What was the hypothesis it had? So a lot of work is done for that engineer when the engineer comes in. So what they have to do after that is audit the work. The agent also suggests mitigation steps. So audit the mitigation steps as well and then they take action on those.

[B]

And so this interface plugs in with a customer's existing ticketing system.

[A]

Yeah.

[B]

Is that kind of what. Okay, I see. And there's like a link or something like that to the agent interface that shows this analysis or does it integrate it in with the ticket system as well?

[A]

For us internally, we show a lot of information within the ticket itself, as much as we can. But we do have a separate app interface for doing more digging into the data if people need to go there.

[B]

Yeah. So if my organization uses ServiceNow, what would that integration look like and what would the experience be? Would you hubby hopping between them? Would you be integrating them together?

[A]

We provide all the details back to ServiceNow and they can then render all those details within the system itself. For the most part, we still have our app available for those users, but I think the pattern we see is most users stay within the ticketing system.

[B]

That makes a lot of sense. And so that setup step, I would imagine there's a lot it can introspect simply from the AWS setup of a customer. Right. Like that's a lot. And you also have integrations with, you know, when I'm looking at these like datadog and Splunk. So it can pull not only from something like Cloudwatch, but also from other external logging systems as well. And that's, I would imagine, a one time configuration of this stuff and it can then pull it all together. That's awesome.

[A]

Yeah. So we will take any logging systems that you have and create the topology map, as I said, and that's at the application level. So we're trying to model the entire application for the agent. And as more of these incidents happen, the agent learns from that and gets better.

[B]

What is the pricing model for this? Like how do you charge for this? Is it per incident or something else?

[A]

No, we charge by what we call active agent tasks. So we have pricing for task hours and that's how we charge for it. And we don't charge for idle time. We only charge when the agent is doing its work.

[B]

You really don't want a lot of potential incidents because otherwise you're going to get charged a lot, which is the goal. Right. So yes, just kidding.

[A]

In fact, we do have something that we call a triage agent, which has been extremely useful for us. So it will do a dedupe of events for you. So If I have 5 tickets cut for the same event, the idea is triage agent is going to tell you that there's only one root cause, although you're seeing five tickets for it. The reason why we built this was what you just said. You don't.

[B]

Yes.

[A]

Want to pay. Yes.

[B]

I've hit it so many times you just get the same alarm over and over again. Yeah, that's the same thing. That's awesome. Cool. So this is a philosophical question, I guess, but if the agent is good enough to wake up to a root cause, why does the engineer we need to wake up at all? What's the line between the agent investigates and presents to the agent investigates and acts. And how do you see that line moving in the near future?

[A]

Today I would say we need the engineer in the loop. The engineer is still the one getting paged.

[B]

That's good to hear.

[A]

Yes. And they do have the accountability to make sure that the system is up and running. And I think as the agents get more sophisticated, the audit work of this engineer would become easier. What they do right now is basically they are an auditor. They examine and audit the work that the agent has done and say, okay, this makes sense. Or they will spot errors and say, this does not make sense. And for us in DevOps agent, our job is to make, as the agent becomes smarter, how do we make the life of this Engineer easy by giving them the right things to order the work. For example, can we show them that not just that the agent has done this, this is how the agent arrived at this. We help them test those changes that the agent is suggesting to make. So that's those are the kind of problems we are looking to solve. And I feel like that is the line that will move is the engineers will be able to see not just how the agent arrived at this conclusion, but how what the agent is suggesting. How safe is it to even take that action?

[B]

Yeah, increasing the signal to noise.

[A]

Exactly, exactly.

[B]

And maybe this already exists, but do you see that line moving into making suggestions for bug fixes that may have caused like actual application bug fixes?

[A]

Yes, absolutely. I would say why not? We have again, as I said, we picked the biggest pain point for our developers. The use case that we picked as a leading use case was this whole pager use case. But the same thing applies to even your lower severity things, which can include bugs. You can have DevOps software agent also configured to look at your lower severity events, including your bugs too.

[B]

So I saw you reported that 94% root cause accuracy in the preview and so that left over 6%. What does that failure look like? I don't know what's going on. A wrong recommendation and who's accountable if that thing is misdiagnosed or there's some sort of worse incident as a result,

[A]

it is still the engineer who's responsible. So as I said so that hasn't changed. But the way it really shows up is the engineer sees the work of the agent and it'll see the agent came to this conclusion, but he correlated this data incorrectly. So that's how the engineer would find out that there is this doesn't look right. And that's why I keep saying that's the work that we will focus on the next months and even years is just to make that audit easier so that the engineer can trust the work of the agent more and more. So I feel like that the trust boundary will start to shift. But yeah, the way it looks like today, you'll just, you'll see the hypothesis and you will have to then say, okay, here is where I think the agent is going wrong.

[B]

94% right out of the gate is pretty good. You know, you can aim for 100%, you'll probably never get there, but that's pretty good. So let's move on to determinism. So DevOps is built on determinism, so you want to run it twice, get the Same results. But agents are probabilistic so they might investigate the same incident in two different ways and provide two different hypotheses. And so how do you manage that in this system? And is, is that determinism going to go away or do you think it moves somewhere else in this whole problem space?

[A]

I don't think determinism goes away at all actually. I think what is going to be fun and challenging is how do you inject determinism in the right way into an agent so that where do you inject it, what form does it take? Those are some of the more interesting problems that we are focused on. So today the way we inject determinism is, I would say it's focused a lot around safety and security boundaries. So we give this agent really trimmed down permissions. You have read only permission boundaries by default. You can't just arbitrarily expand them. It's a very intentional move if a user is giving the agent more permissions than what it comes packaged with. So I feel like that's one way we have introduced determinism. But more ways I can think of introducing determinism is doing some actual testing of your changes that is very deterministic. You spin up a test environment, run your tests, something like that. I feel like the industry will start to move in that direction even for DevOps. And the third thing again future looking for us is also automated reasoning, which it's a different technique. It's not the same as the LLM science that we have, but it's still a scientific technique that allows you to model your system and you can do some pretty remarkable stuff with that. I think you were talking to Byron also here and he should have mentioned something more around how neuro symbolic AI is a different approach that's coming into the mix. So we are also looking at taking some of those techniques and putting them in DevOps agent.

[B]

Can you briefly explain? I'm not familiar with automated reasoning. I could guess at it. But what is that and how is it different from what an LLM does?

[A]

Automated reasoning takes a more mathematical approach. So it will model your system in some sort of a mathematical representation and you can do the before and after of a system. Let's say you're modeling a change. You can see what it is that it was before and what it is that it is after and then use that to see have you created a drift that was not intended. So it's super helpful for things like policy.

[B]

And so that is built into the DevOps agent now or Is that down

[A]

the road, some of it is built in and you'll see us doing more with that.

[B]

Okay, so in Amazon, when something goes wrong and the agent analyzes it and you fix it, how do you do a postmortem? So can you actually, like, replay the decision tree? Some of it opaque. And so for the agent itself, what is the observability look like?

[A]

That's a really good question and something that comes up from our users quite a bit. And we obsess over transparency actually quite a lot. So for us, as long as we can show more of the agent's work, I think we help the auditor, or the engineer in this case, really figure out where the agent took a wrong turn. And we do give the engineer the ability to give us that feedback so that, as in the product, they can give us that feedback and we get that signal. And when I said that the agent gets better over time and learns from its mistakes, this is one of the ways in which it learns. We take that negative feedback and then we record that in the agent's history and the agent actually learns that this was not the right conclusion. And oftentimes we also see that we know what the right conclusion was because ultimately when the ticket gets resolved, at least internally, we write down this was the root cause. So the agent will take that also into account and improve its performance over time.

[B]

So I am morbidly curious about the architecture of this thing. And like, I'm not going to ask you to go into a dissertation about how it works, but I am very curious, just at a high level, how this thing is operating, if you don't mind kind of indulging me there.

[A]

Yeah, so it's a combination of the model and the tools that we have provided. The way we've constructed the agent, it's all about that. And we use state of the art models today in DevOps agent, but we also use the agent, the models which are best suited for the job we are doing. So depending on the complexity of the job, depending on the latency requirements, we would pick the right model to solve the problem that the agent has. And really a lot of our architecture has been about what are the right tools that we want to give this agent, how do we structure those problems have been around even around those tools. What is the underlying data? How do we build the topology? That has been a very interesting problem that we have solved. And what data is useful and relevant? Because as you were saying in the beginning versus what is noise? So a lot of our architecture is about the Right tool calls and picking the right tool set of data to actually take action. And we have modeled this agent in a lot of ways as our on calls actually work. So the way we have built this agent is modeling that pattern. When you do get paged, how do you go and investigate an event? So that has been the model for.

[B]

Yeah, how is that amount of data injected, you know, as context in the model? How is that managed?

[A]

We do have a pretty large context windows for a lot of models today. So I would say that that takes care of a good amount of context. Then we, with the topology that we build, that's our intelligence too. We can compress it in ways that makes it more efficient for the context window for it to fit in there. And that has been, of course, one of the techniques that we focus on as well.

[B]

I see. Very cool. Thanks. One thing that was really exciting for me to read about was that you can extend the DevOps agent by bringing MCP servers. You mentioned you had to manage and wrangle a lot of the MCP servers internally. So what's the most surprising or creative MCP integration you've seen one of your customers build?

[A]

I think some of the more interesting ones for that come to mind are related to actually making changes. Now these are not system changes, but they have integrated MCP servers which go and update tickets as an example, or they have an internal system where they're keeping track of what incident has happened and what is the next step. So go and update that. So it's sort of a write operation, it's not a system. Right. It's just still very much logging the work. But I thought that was an interesting thing in how they're using bringing in some of the MCP server. It's not just an input, it's actually like, okay, based on this agent's result, I'm going to update this other system.

[B]

That is really cool. I know that one of the pain points I always used to have was, you know, just providing updates. It's just, just let me fix the problem. I don't want to make an update. If an agent can do that for me, that's wonderful.

[A]

Great.

[B]

So I'm going to shift the conversation a little bit away from DevOps agent. Well, let's just talk about the future of SRE or, you know, in a dev team supporting something on call. And so a lot of engineers, especially SREs, got senior by doing incident response. And developers, you know, feel that pain of I want to make a solid system so I don't get woken up in the middle of the night. But if the agent's doing that work first, how is the next generation of SREs or on call engineers going to get trained?

[A]

That's a pretty deep question. I can share what I see happening. What I see happening is that I don't think developers or SREs completely go away. I don't see that happening. The question is what about how will they train? In my view, the previous SREs were trained when systems. When you know, something would go wrong with the system and they would see a problem over and over again. And there was a lot of pattern matching that starts to happen. Oh, I've seen this problem before. And with AI, what is happening is some of this pattern matching is being done by the agent itself. So fewer of these events will come to sres, but they'll come. So I anticipate that sres will have to get much better at doing this pattern matching when the number of incidents that they are doing pattern matching over is fewer. The other thing I feel like will change for SREs or developers in general is they will deal with more complex scenarios where the agent just fails and so the agent does half the work, maybe it's incomplete. Those are the places where the SRES and developers will get engaged and, and I think that's actually a good thing because some of those trivial cases, no SRE engineer enjoyed working on those. But the real hard ones, I think that is how they will train and I think that is how they will learn.

[B]

And also something you said earlier just came to mind too, which is like, you got to get this thing off the ground. If it's a new system, there's going to be new problems and you're going to get trained along with the agent too. So the humans still have hope.

[A]

Yes.

[B]

So you lead agentic DevOps at AWS. So are you guys hiring more or fewer of the people like SRE types that you hired say three years ago for this work?

[A]

I can't share very super specific numbers or even a yes or no for this question. What I can say is we do need a lot of developers and people with the SRE experience on our team. You know, as we build these products and we are going to be building more of them. So I would say the need for developers and the right folks with the SRE experience is still very much there. I don't see that going away.

[B]

That is good to hear from Amazon. And so pivoting a little bit into the types of role, is there a particular role that's on your team that didn't exist a couple of years ago because of all these shifts.

[A]

I think I would change the question a little bit. I wouldn't say that there is a role that didn't exist, but what I expect from the roles that do exist now and then that have existed for all these years is I think we are expecting a lot more and maybe different things from them now. For example, a product manager is now able to create prototypes using one of these coding agents themselves. They don't always have to rely on an engineering team for that. So that's a change in an existing role that I see. I also see more managers are coding than what I've seen in the past. It's just become really easy to do that. And then we have, on the flip side, we have a lot of engineers who are much closer to the product. So they are, they have a lot of good product ideas that they can now put into implementation without involving the product manager. So it's, it's same roles, but the expectation from each of those roles has changed because we are seeing them do a lot more beyond their, you know, traditional specialization.

[B]

The lines are blurring. That's what I'm seeing in my role too. Yeah. So a lot of engineers listening. If they're like me, we're worried about our jobs. Not in a doomer way, but more of a like, okay, this is coming. What do I do on Monday morning? So if you're talking to like a mid career DevOps or SRE engineer, what's your advice for them? What are the next five years look for? Like, how can they best prepare themselves?

[A]

My first point is if you haven't tried using any of these agentic tools and models yourself, start there. You have to, you absolutely need to know how powerful these tools are. And I would say use them to make your grunt work go away. That's precisely what we're doing with DevOps engine. We're making the toil go away. And the more you see these trivial cases handled by the agents, and as the agents get more sophisticated, these cases may not be trivial, they will start to get more complex. But the wonderful thing that comes out of using these agents is you will find places where you are still needed to do more complex problem solving and the agent doesn't do it themselves. And what I have found is people who are continuously playing with these tools and using them for their daily workflows are getting way more creative about, you know, getting to the next level of complexity. Oh, now I can do this. Because this other thing is taken care of me for me by the agent. So that's my hope and wish for all the SREs and the DevOps folks is just move up the stack to hire more complex problems.

[B]

Yeah, I couldn't agree more. And you know, I mostly comes from a software engineering background and that's what I tell people. Just start playing and you'll be shocked at how useful this thing can be. And I tell them that the only way that you should be afraid is if you're afraid of the tools, like just embrace them. Like it's going to be part of our, our lives. So for better or worse. Hopefully for better. So let's end it on a positive note. What's the thing that you're most excited about in this whole space? Doesn't need to be DevOps, anything like that. What doesn't get enough airtime? That's something that you just want to tell the audience that you're excited about.

[A]

I think if I were to pick what I am personally most excited about it is the agents are doing what I call a lot of information synthesis work today. And I am really excited about what actions they can take as they become more powerful. And of course it will come with challenges as safe as safety, their security and all of that. But I still remain very positively hopeful that these agents will be taking a lot of actions on our behalf. And some of those would be actions that are completely boring, things that I don't want to invest my time in. So I'm excited about those. And others would be things for which I don't have to wake up at 2am I would be excited about those two, but I would truly be excited about. As I was just saying, because the agents are taking so many actions, I have more time to think of other complex, other creative things that I may not have had the time to do.

[B]

Yeah, there's always going to be grunt work too, but hopefully less of it, right?

[A]

Hopefully less, yes.

[B]

Yeah. Well, thank you so much for being here. This was super interesting. I finally got to ask somebody at AWS about some of the interactions, workings of what goes on there. So I'm very, very happy and thank you so much for being here. I really appreciate it.

[A]

Thank you, Matt. Happy to.