
a16z's Joel De La Garza is joined by Emilio Escobar, Chief Information Security Officer at Datadog, to discuss what it takes to secure a company where nearly every employee is using AI and more than 4,000 engineers are working with coding agents. Rather than trying to block new tools, Emilio explains why Datadog chose to embrace AI early and build the security infrastructure needed to use it safely. They unpack how AI changes traditional assumptions around data permissions, credentials, developer access, and software supply chains. Emilio shares how Datadog uses role-based MCP servers and ephemeral credentials, as well as an AI "judge" built by his security team to evaluate the intent behind code and agent skills before they enter the environment. They also discuss why security teams can't afford to wait for commercial solutions to every new AI threat, how the relationship between developers and security teams needs to change, and why Emilio is less concerned about an AI "escaping"...
Loading summary
Martin Casado
The number one story on Bloomberg right now is that AI has gone wild. You seem remarkably calm.
Emilio Escobar
The way I see it is if it's not an AI model, it's going to be somebody or something with actual malicious intent doing it. I do worry about what can the agents do, what tools can they call, what binaries can they pull, and also how do they get access to credentials. If a code is meant to solve the bug, but it gets rewarded on that, but it doesn't care if it's actually doing something else.
Martin Casado
The tree is sick. So to make it healthy, it cuts
Emilio Escobar
it down out of a necessity. My team built a judge that evaluates the intent behind a piece of code, like, is this thing meant to do harm or not? And we actually find quite a bit of malicious skills in all these marketplaces.
Martin Casado
Engineers. Now, security is very much front of mind for them.
Emilio Escobar
Developers have always cared about security. The problem has been that the version of security that we want them to do is just crappy.
Martin Casado
My thesis 10 years ago was that security engineers will become real engineers. So maybe now is the time.
Joel de la Garza
AI is changing the security landscape on both sides. Attackers have more capable tools, but security teams are also figuring out how to use those same capabilities to their advantage. In this episode, A16Z's Joel de la Garza sits down with Datadog CISO Emilio Escobar to talk about what he's seeing as AI adoption spreads across the enterprise, including thousands of engineers working with coding agents. They discuss how AI changes assumptions around permissions, credentials and software supply chains, why simply blocking new tools doesn't work, and how security teams can adapt without becoming the department of no. And as models get better at finding vulnerabilities, Emilio explains why he's less worried about AI hackers themselves than a more practical problem. What happens when the number of vulnerabilities we can find suddenly explodes?
Martin Casado
Thank you so much for joining us. I know you're not here to see us. You're here to go to Black Hat, and it's good to catch up. And it's been a really crazy week. So you are a CISO at a public company tech company, one of the house of innovation. Driven a lot of really cool technology out of it. And we've been talking to everyone about these models escaping, these models, hacking. And I know you've been central to kind of the adoption of AI at datadog and would love to maybe talk a little bit about how are you thinking through the risks, how are you deploying this stuff? And I know you're very close to the product team and you're kind of enabling it. So would maybe start off there and stick around.
Emilio Escobar
For us it was like we have to do it like any other software company. If we not we're going to be in the medieval ages real quick. And I like being as easel, I don't like having my own goat farm. And we started real small. I remember our first install, we started with Cursor and I remember getting like 50 licenses and it was more of putting out there and see who bit. And then not surprising, it bloomed from there. So now we use every other coding agent that's out there and some of the things that we're worried about as we deploy. Well, first you start with the data. What are we worried about?
Joel de la Garza
The data.
Emilio Escobar
And I never subscribed to the idea of, well, if I block these things, nobody's going to use them.
Martin Casado
That's never worked.
Emilio Escobar
So on the non developer side, like we just got ChatGPT licenses for everyone and it was like whoever wants it, gets it, I don't care. And you sure use it to find a recipe for pot roast. I don't give a crap. And because then I avoided the all oops moment and it was this was back. Ironically, this wasn't that long ago, this was two years ago. And it feels like things have moved 20 years since. But I remember having a lunch with a bunch of CIOs at RSA where everyone was asking me like, how am I not blocking these things? Right? And I well, A, we can because we decide the company that we are. But B, luckily we can just buy our way out of that problem and zero data retention and all those things.
Martin Casado
The inverse of hey, let's block it turned out to be correct. Which is the people leaning into it the earliest and the most are the ones that you actually want to reward enough, right?
Emilio Escobar
That's right. Yeah, yeah, yeah. So now we have over 4,000 engineers using them. I mean actually we have every employee in the company adopting some sort of AI, whether it's coding agent or Gemini Cloud Desktop, ChatGPT or what have you. And so I have two, we have two, two verticals. We have the engineering side, which Alexi is deeply involved, our CTO is deep involved in. And then I'm involved on the non engineering side for how do we get marketing, sales and everyone else to use AI. And the adoption is I think we were like 98% adoption rate, something like that. It's one version of AI or the other everyone is using on that side. On the it, corporate Side the data becomes more the paramount issue, permissioning and all those things. One thing that AI is going to do is you think you have proper controls of who get access to what Google file or what have you. AI is going to find a way to get it. All you have to do is prompt it.
Martin Casado
It flattens the organization.
Emilio Escobar
It flattens the organization quite a bit. So we actually had a moment where internally we built this business intelligence tool. It's really neat and it was starting to show some data that I was like it wasn't anything like a fire alarm but it was like should we.
Martin Casado
And the data was probably always available to everyone. They just never took the time to.
Emilio Escobar
So, so yes, so so it's data that is in a data warehouse and we thought we had the right table and row permissioning and all of that. But there was always a way to get access to the data if you knew the SQL well enough.
Martin Casado
Yeah.
Emilio Escobar
Well then what happened is that you went from only people who really know SQL are using this to now having a sales rep telling asking like what are what is? I'm a commercial sales rep. I'm just coming up with an example. I'm a commercial sales rep. How is the enterprise deal, enterprise tier team doing?
Martin Casado
Yeah.
Emilio Escobar
Which normally you want to keep those things separate and what have you and the agent just figure out the SQL to do it.
Martin Casado
Yeah, yeah, yeah. It tells them what the comp plan is.
Emilio Escobar
So but it's not always find a way around the permissioning and all that is like we had the right permissioning for the right time and then the age. So we've been obviously improving all of that. And then luckily my IT group is what we've done is now we're at a point where we have role based MCPS servers as well. So we have an NCP server for SDRs and things like that. So we control and govern that data and then we're like let them have it with whatever tool they want to use. On the engineering side, obviously in the coding agent, like when my security team is worried about similar. What you were just talking about is on what can the agents do, what tools can they call, what binaries can they pull, what dependencies can they use and also how do they get access to credentials. So for that we have a few things going on where we've contributed to this open source sandbox and expanding on that where the agent doesn't actually get access to any sort of credential file. So even if you have a file in your home directory that has your aws secret, your NPM publish secret or what have you. The agent on the sandbox can't touch it. The credentials get injected into the agent the moment it needs it. And the beauty of it is we already have CLI tool that will give you these ephemeral tokens to list things. We just have the agent not know how to call it.
Martin Casado
Yeah.
Emilio Escobar
So it's like, oh, I'm Emilio, I'm an engineer on this team. I want to authenticate to GitHub. For example, I call this tool auth GitHub and it gives me a credential that's valid for I don't know how long and it gives me my access to GitHub. But now it's the agent getting my access to GitHub, but not there's a static token written in a file.
Martin Casado
Yeah, yeah, yeah.
Emilio Escobar
The second thing is, we know the thread has expanded. Developers are now the main target for attackers.
Martin Casado
Absolutely.
Emilio Escobar
Because if I get one of these tokens, I can then build a worm that attacks packages or I can just escape or do whatever or access your production environment. The other ways that we see how these agents can do it is via skills. We track a few of these marketplaces.
Martin Casado
Yeah, how do you control the skills? That's a really hard problem.
Emilio Escobar
So we actually have built hooks into the agents. And then one of the things that which we can talk about is my team built out of a necessity, a judge that evaluates using AI LLMs, that evaluates the intent behind a piece of code. I'm not talking CVEs, vulnerabilities or anything like that. Is like, is this thing meant to do harm or not? Harm being a vague term on purpose, it actually does really well at scanning markdown files as well.
Martin Casado
Nice.
Emilio Escobar
So we built it because for the longest time, datadog, for those who you know, we have a datadog agent, we take third party code contributions to it, things like integrations and what have you. So code written by other people. We had this process for the longest time that requires a security engineer and an engineer on that agent team to reevaluate the code and then approve it and then merge it. So I'm like, well, this thing can't scale. So then we built this judge by we, I mean my team. I don't do anything that got really good at it. And then the software supply chain hijack started happening and ID extension started happening and then we threw this thing at those packages and it was actually able to identify the malicious piece of code that was injected or introduced during the hijack so we're like, huh, I wonder how it would do with markdown files. And it actually does pretty well. So then internally, what that means is we have hooks into the agents, we know what skills they're getting pulled. I've never believed in a security program that just restricts everything because the pain, innovation and all that, it's the empire of no. It's the empire of no. But also it's like, we don't want to be Free Willy either. So what we do is now we put this judge in front of every skill that wants to get introduced. We actually find quite a bit of malicious skills in all these marketplaces. We're working with a few of them partnering to see if they want to use this judge, but also like, hey, giving a heads up of, hey, we found this one, we found that one, we found that one. And they do a pretty good job at taking notes down. The other thing we worry about is what dependencies it can pull, what binaries it can pull. So I think it's plenty of it out there right now. I don't think we have to justify it, but the judge does a good job at it and then evaluating the code output of the agents against malicious intent. I think intent. And it's funny how things, how this industry works, and this is maybe where I can get a little cheeky here, if you allow me, is I have no control.
Martin Casado
You can say whatever, yeah, apparently intent
Emilio Escobar
now is one of the must have in any AI security type of solution. But the interesting thing is, like, it happened after we've had conversations about intent anyways. But apparently that's a big deal, right? It's like the fact that these agents, obviously, they're trained on existing data, on
Martin Casado
existing code, and they have a reward
Emilio Escobar
structure and they have a reward structure. So if a code is meant to solve the bug, but it gets rewarded on that, but it doesn't care if it's actually doing something else outside of that. So we were worried the tree is sick.
Martin Casado
So to make it healthy, it cuts it down. Right.
Emilio Escobar
So we were worried about like the, you know, the typical thing that we read about of an agent producing code that either would have, oh, you know, your database isn't scaling and is paging people at 4am, so the best thing I can do is just turn it off and therefore I solve the problem of paging people at 4 in the morning. Right. So you have to be careful how you prompt these things, but also how it actually interprets your prompt and executes on that. So we have this judge now evaluating the code output of the agents to then make sure that we're doing this. So the cool thing is that we do this internally because we have to. And then the product team goes like, wait a minute, that might actually become a thing. Let's explore. And then we evaluate it with customers and all of that. But I mean, I get to say I'm lucky that the team can do it. One of the things that it surprises me is that I did a roundtable last week about agentic security. And the sense that I got from a bunch of the security leaders who were in that call was a sense of helplessness, of just waiting for a commercial solution to come in and solve it all.
Martin Casado
Well, you're. Yes. I think that the profession is interesting in that there are a number of different CISO profiles. And I think, you know, I, I say this as someone who was relatively technical for a ciso, like before, before maybe just the generation above us, like there were no tech. I mean, the first CISO was Steve Katz. Right. So that's like pretty recent. Yeah. You know, and I think that there was just a lot of like, sort of like CFOs becoming CISOs or CTOs, not really security people. And then what happened, I think is that security teams got quite big and you needed a manager. Right. Not necessarily a leader or a thought leader. And so I think there's just a lot of that right now in the industry where there's very much a. You know, if you're a large industrial company and software is not your competitive advantage, you're probably going to buy most of your product. Right. Versus like Datadog. Yeah. Like, you better be the best.
Emilio Escobar
It's totally fine. Right. But I still believe even in those scenarios, you may have one or two people on that team that if you give them creative license.
Martin Casado
Yeah.
Emilio Escobar
They could do something. But the fact that that wasn't even a thought process.
Martin Casado
Yeah.
Emilio Escobar
Worries me a little bit because this thing is moving really, really fast. Naturally, like a lot of the companies that are solving this are new companies.
Martin Casado
Yeah. You know, I think it's really hard. And this is, this is. I mean, I think the talent shortage is still a problem and I'm generally bullish on security jobs because of it. But like, I think, I think for some of the bigger programs, like, it's really hard to get technologists in there because they are. But shops.
Emilio Escobar
Well, here's my counter argument. Do you really need a security person to do it?
Martin Casado
No. And I think some of the best security people didn't Start as security people.
Emilio Escobar
Yeah, you know what I mean?
Martin Casado
It's like the developer that found their way into security. Right? Yeah.
Emilio Escobar
But also in these companies I'm sure there's a developer somewhere in there that they can be like, hey, how about we have you solve this problem for us?
Martin Casado
In the meantime I think that a realignment is kind of happening. So I was talking to a family member at a big, big American corporation and they were saying that sort of like with AI tool adoption they don't need to write as much code manually any. Right. They still need engineers and like that. But like they're seeing, they're basically saying like hey look, we don't need this many tier one engineers and the security team needs them. And so I think maybe you're starting to see some of that cross pollination right. Where you'll see that talent find their way into security. Because it was always the case that if you're a developer, an engineer, like just you would just make more money on the software development side. So like I think those are equalizing in maybe, I mean My thesis 10 years ago was that security engineers will become real engineers. So maybe maybe now is the time.
Emilio Escobar
Yeah, yeah, yeah. I mean we pay them the same as software engineers now. We do, yeah, yeah.
Martin Casado
I mean in the Valley especially. Yeah, yeah.
Emilio Escobar
Like they're the same.
Martin Casado
I think it's narrowing in large corporates. I think, I think it's still software engineers have the advantage there, but that'll probably narrow over time.
Emilio Escobar
Yeah. So that's how we be thinking about identity security is like what are the things that we actually worry about and then what do we, what can we build, what can we use? And then where do we find more synergies there? And I know that's a very buzzy term but what that I mean is like our devex team is also interested to understand what's happening in the agents because they want to monitor developer experience. So why can't we just all together work on it? So that's how this thing has grown. Because it's not just a security tool for the sake of security, it's also serving other purposes.
Martin Casado
And I've noticed with engineers, especially over the last decade, like engineers now, security is very much front of mind for them, if only because they don't want to have to keep fixing things. And so I've noticed that there is this front of mind security thing for engineers which is great. I think we've made a lot of progress in that regard.
Emilio Escobar
Yeah, well I actually have a little bit more of a maybe far a take on that. I think developers have always cared about security. I think the problem has been that the version of security that we want them to do is just crappy, which is like, go fix this. Thousand of things that none of them are actually relevant to what you're building. But this scanner told me they're critical, so you have to fix it.
Martin Casado
Yeah, there is a lot of like the. You guys have sent me so much junk.
Emilio Escobar
So one of the things that I get to do at Datadog is actually I get to meet a lot of the engineering side of the equation. Talking to us about our security platform and products and how do we get their security team to use it. And, and they told me the same stories that I'm sure you've heard of. Like, I get a thousand tickets, none of them are relevant. Security doesn't know what we're doing. There's no intent to even understand the systems. And the number of times I hear a security person telling me that their developers write crappy code, it's just mind boggling because I'm like, well, then you do it.
Martin Casado
Yeah, exactly.
Emilio Escobar
See how good you can do it.
Martin Casado
Yeah, probably no better.
Emilio Escobar
Yeah, exactly. Or maybe it's good code, but they're not fixing your noise vulnerabilities and therefore you think it's crappy code.
Martin Casado
Yeah, I mean, oftentimes, Right. With code bases, this was the learning experience for me was that like you'll find these in isolation, you'll find these issues in code, but then like up there's, there's a framework, a layer up that's actually mitigating any potential fallout. And so it's like, whatever, it doesn't matter. Yeah. And I just think, I think, yeah, security people tend to have an issue, a blind spot around some of that stuff. Yeah, I guess, I guess. Sort of like as we sit here, you know, the number one story on Bloomberg right now is that AI has gone wild and the world is over and everything is getting hacked. You seem remarkably calm. So maybe, maybe before we go, like, we'd love to get your take just on sort of like, you know, supposedly there's this great white shark cruising out there, just eating people without the knowing. But in general, like, I just haven't noticed a sense of panic from CISOs. Like, what's your take on all of this?
Emilio Escobar
I mean, the way I see it is it's, it's. If it's not an AI model, it's going to be something with somebody or something with actual malicious intent doing it. So Therefore, I don't panic. I do, I do worry about our access to those same capabilities. I do worry about the lack of regulatory framework that's deciding who gets access to these capabilities.
Martin Casado
Yeah, yeah, I feel you on that. It's really interesting who's getting access to this. And it's sort of like what's at play here.
Emilio Escobar
And then it's clear that it's or communicated in a way that makes it or portrays it as clear that is outside of the Frontier Lab's hands. And so we have to put our company name on the list and then we're supposed to wait. And then when I ask what's the criteria for how these things are evaluated, I get jazz hands response.
Martin Casado
But then you come to a conference and you talk to your buddies and they've had it for nine months.
Joel de la Garza
Yeah.
Martin Casado
And they're making stuff that's way less
Emilio Escobar
critical than your product was like, hey, some of these companies are using our stuff. Wouldn't it make sense for us to get access so we can protect the stuff that you're using?
Martin Casado
Exactly. I'm sure there's a lot of government agencies running datadog software and vice versa. Right.
Emilio Escobar
We have a lot of also companies that provide services to the government. So yeah. So I'm not worried necessarily about the models escaping sandbox and finding old days. I actually think inherently it makes things better for us 100% because guess what, we're talking about it. Bloomberg is talking about it. You just said. Right. So before it was like, well, nobody's talking about it, so who cares. But am I worried about one of those things hitting datadog and something happening? It's like, well, it's either a model or somebody else. Right. If we have a gap, we have a gap.
Martin Casado
Yeah.
Emilio Escobar
At some point it's going to get discovered. So I'm not too worried about that. What I worry about is the volume of things that are going to discover and the things that are telling us how to handle that volume are not
Martin Casado
up to par 100%.
Emilio Escobar
So the you must fix every CVE that's in your environment to now you're going to get a thousand X more CVEs in your environment. Well, those things, two things don't necessarily align well. So I'm hoping that side also changes to the current day of things. That's what I worry. Sorry. And lastly, the other thing that I worry is the hypersensitivity to the security of these findings because of the fact that it was found by a model rather than a human. And I'm already seeing some of that where like, oh, Model X found this, so it must be extremely critical and true. And then you're looking at it and you're like, yeah, it's actually not, but it's kind of hard to argue. That is us arguing against this thing. Just because a Greek God found it doesn't mean it's actually a critical thing. But I think the third party risk management game is going to get really crappy even more so than it is because of this thing. So I hope eventually. So those things I do worry about, but a model escaping and hacking companies and all that is like, well, I mean they were going to get hacked no matter.
Martin Casado
There's always been hackers, right? There's two more now and whatever.
Emilio Escobar
Yeah, yeah, yeah, there are two really good ones. But hopefully they make us better rather than like making us work well.
Martin Casado
And I think, I think the lesson that I have learned throughout my entire career is that any attempt to gatekeep security fails and it only makes things worse. And that an open and free discourse on these things is the only way to get better.
Emilio Escobar
Yeah, yeah, exactly.
Martin Casado
Thank you for joining us and having that discourse. Man, that's been awesome.
Emilio Escobar
Yeah, thanks.
Martin Casado
Enjoy your time at the conference.
Emilio Escobar
Thank you.
Joel de la Garza
Thanks for listening to this episode of the A16Z podcast. If you like this episode, be sure to, like, comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes go to YouTube, Apple Podcasts and Spotify. Follow us on X16Z and subscribe to our substack@a16z.substack.com thanks again for listening and I'll see you in the next episode. Episode As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com forward slash disclosures.
Host: Andreessen Horowitz
Guest: Emilio Escobar (CISO, Datadog), Joel de la Garza (a16z), Martin Casado (a16z)
Date: August 11, 2026
This episode explores how the rapid adoption of AI agents is transforming security practices and mindsets at technology-driven companies. Martin Casado and Joel de la Garza of a16z speak with Emilio Escobar, CISO at Datadog, about practical approaches to securing organizations as thousands of employees start using coding agents and generative AI tools. The conversation addresses technical risks, organizational culture, and the evolving role of security professionals.
Leaning In, Not Blocking (02:55):
Rewards for Early Adopters:
AI Changes Data Access and Permissions:
Example (05:16):
Automated Assessment of Skills and Code:
Balancing Security and Innovation:
On AI as Just Another Capability:
“If it's not an AI model, it's going to be somebody or something with actual malicious intent doing it.”
(Emilio Escobar, 00:05/18:01)
On Blocking Tools:
“If I block these things, nobody's going to use them. That's never worked.”
(Emilio Escobar, 03:01)
On Flattening Permissions:
“AI is going to find a way to get it. All you have to do is prompt it.”
(Emilio Escobar, 04:41)
On Writing and Reviewing Code:
“The number of times I hear a security person telling me their developers write crappy code. It's just mind boggling because I'm like, well, then you do it.”
(Emilio Escobar, 16:57)
On Security and Engineering Becoming One:
“We pay them the same as software engineers now.”
(Emilio Escobar, 14:40)
On the Flood of Vulnerabilities Uncovered by AI:
“Now you're going to get a thousand X more CVEs in your environment. Well, those things don't necessarily align well.”
(Emilio Escobar, 20:14)
On Open Discourse:
“Any attempt to gatekeep security fails and it only makes things worse. An open and free discourse on these things is the only way to get better.”
(Martin Casado, 21:30)