← Safe Mode Podcast
Safe Mode Podcast
How security investigators can get the right info out of AI security tools
00:33:18
This week on Safe Mode, Greg sits down with Dov Y…
Loading summary
Transcript50 lines
- [00:00]
A
How can AI help close the incident response information gap? We'll talk about it on this episode of Safe Mode. Welcome to Safe Mode. I'm Greg Otto, editor in chief at cyberscoop. Every week we break down the most pressing security issues in technology, providing you the knowledge and the tools to stay ahead of the latest threats, while also taking you behind the scenes of the biggest stories in cybersecurity. An attack is coming. It's about keeping us safe. He's just a disgruntled hacker.
- [00:27]
B
She's a super hacker.
- [00:28]
C
Stay alert. Stay safe.
- [00:30]
B
Stay safe.
- [00:31]
A
This is Safe Mode. Welcome to this week's episode of Safe Mode. I am your host, Greg Otto. In our interview segment, we're going to be talking with Dov Yaron, the co founder of Command Zero, talking about agentic socks. And we've been talking about this a lot lately, given the AI boom, but really talking about the mindset that goes into investigations and how AI can really enhance what doesn't happen when investigations are done manually. So really interesting conversation there. But first, talking with Tim Starks. Tim's back on the program. And if you remember, a few months ago, we had previewed a Supreme Court case about geofencing and geofence warrants and talking about what that really could mean for privacy. And we had a ruling this week that when it dropped, Tim and I were very, very animated as, as we were reading it. So we were like, we should probably talk about this on the podcast. So here we are. So Chatri versus the United States. What did we get in terms of a result?
- [01:37]
B
Yeah, I think you all remember that conversation from back then where I think one of the ways we closed the conversation was it, it. There's a chance that nothing significant will come here, that they might make some kind of tiny ruling that was on the margins of things and didn't have any precedence for anybody else. That is not what we.
- [01:55]
A
No, we were dead wrong.
- [01:56]
B
We were both looking at each other going, are you reading this? Like, is this is a big ruling? Right, like, and just kept. As the reaction kept coming in, we were thinking like, this is a big deal. So what we got was a ruling that, that, you know, to hear the people on the privacy advocacy side of this say this is a major win for them on tech privacy broadly and on the side of people who didn't like it, namely some of the dissenting justices, this is something that they were talking about going to have to pick up the pieces for a long time to come. But this has seismic ramifications for Fourth Amendment Doctrine in a way that's bad. So the ruling essentially said that these kinds of geofence warrants, these kind of group collections of geographic data where phones are in a limited period of time still, if you're gathering that, it's still a search under the Fourth Amendment, which, you know, that we talked about, like, what does that even mean, really? What it means is just that, you know, you can't just go harvest this stuff from Google without anybody having any oversight of it and people not having an individualized warrant or whatever. The way this might play out as we go forward on what that means for each individual case, you know, I think we'll still find out. But this said not only that those things are searches, but that there's a very broad statement in Justice Keegan Kagan's ruling, and it was an advisory opinion, as they call it, saying just because new tech exists doesn't mean that the Fourth Amendment goes away. And that was one of the things that was fascinating about this case. Right. Is if you're thinking about this in a world before we had the kind of technology where you could harvest this kind of data, should there have been a warrant for each person and there wasn't any kind of technology that would allow you to envision what that would be like. There was no historical comparison for what these kinds of technology things mean for us. And to see such a broad ruling on tech privacy, we were both kind of like, whoa, this is. This is surprising.
- [03:55]
A
Yeah, that line from Kagan about just because it's new technology doesn't throw the law out the window, which is really surprising based on some of the ways that we have seen policy happen over the course of the past 10 to 15 years where not so much the court justices, but we've seen lawmakers not totally understand what is happening from a technological basis. So I think that's part of what went into, at least my opinion from earlier, where it was like this, this stuff is really rooted in technology, and the powers that be tend to not really understand the implications here. Oh, no. Justice is 1 Greg 0 no, we get it and we get how it applies to constitutional laws. So I was definitely wrong. But yeah, I think from a layperson's understanding of this, is it saying that even though you give your data to Google or what, whatever technology service is responsible for this particular set of data, location data, that data is still made by an individual and therefore belongs to an individual, and therefore, under the Fourth Amendment would constitute a search of an individual, which from. I tend to think I'm in lockstep with the way that that was interpreted there. I, I tend to believe it being that way. As to how that looks further going into something like chat tree happening again, I think it's still open for interpretation. Right. I think that the big takeaway was, is that there wasn't really a open and shut case necessarily. It was really more of a ruling and just saying, no, this, this stuff is construed and can be interpreted as an individual search. Yeah.
- [05:58]
B
And the lack of open and shutness applies specifically to Okello Chatri, who was the fellow who was convicted of bank robbery, that they didn't actually rule on that on his case. So there is going to be a place for courts to interpret this. But, you know, the leaving it so open ended combined with the fact that this is now the second ruling that they've had where they said this kind of collection of cell location site data in the Carpenter case from 2018. I believe everything here is saying this is, this is the way they're going to lean this current court is, is when they see a case like this set aside technology, they're going to say, you know, how are we, how are we going to protect this? These kind of things are searches and, you know, technology is going to evolve in a way we can't anticipate, but that's what makes this case. So what made this ruling so interesting is that they're saying, set aside anticipation of how technology works. These are the fundamental principles that we're going to uphold going forward. And it's a, it's what we expect all the courts to do going forward. It was a big message.
- [07:03]
A
So we said earlier, privacy advocates took this as a major win. Talk to me a little bit more about the reaction that you got from people that were like, oh, wow, this is, is a good thing for us.
- [07:16]
B
Yeah, everybody was, everybody on the privacy advocacy side was pretty happy with this. They see this as a major win. But, you know, there are also, you know, there, there's, there, I think there are people who are by nature a little skeptical of like privacy is going to stay protected indefinitely. There's, there's the ambiguity of what, how this might be interpreted a case, case by case basis. Certainly the reasonable civil warrant still will be a thing that will be discussed. There also is some concern that there might be ways around this. A couple different people talked about the way other ways police can get warrants, other ways police could get this data. Thinking of things like going to data brokers and just purchasing data. But, you know, I think that. That. I think that probably gonna be ramifications for that as well. There may be ways around it, but I think that that kind of thing is gonna get scrutiny. And then, of course, there's gonna be legislative action potentially on this. The whole big section 702 surveillance debate. There's been some effort to tie purchasing data together with that. So there will be some. I think there will be some legislative fights ahead about how much more specific they want to get on things like that. But I think that. I think that if you're. If you're in this field and you're concerned about this issue, where this is more. This is the most important part of what's going on with government surveillance. You know, setting aside things like what are the national security or law enforcement ramifications, if that's where you lean, you have to be happy with the ways that this came out toward you.
- [08:41]
C
Right.
- [08:41]
A
You know, you were talking about the different ways that this could be interpreted. Are there other ways around. It reminds me a lot of the conversations we have around policy, whether it is 702 or even going back to the fights over encryption, where there's always been a line of thinking where it's like, do you really need to break the encryption when you have all of the other ways to go about going into a phone, whether it's a warrant or some other legal means that is reminiscent here. And I can see how some skeptics could be like, okay, so they ruled on this, but there are other ways that they can go after this data, which, again, just talks about the technological age that we're in now.
- [09:19]
B
It just.
- [09:19]
A
There's never just one simple plan when it comes to how technology and policy merge together here.
- [09:28]
B
Yeah. And I think it'll be. You know, one of the things that I was a little disappointed with in the. This is not me taking sides, but in the way the dissent was written. They didn't give specific examples of how they thought this was going to backfire. They just talked about the general principle of it being bad. I mean, I certainly think if you're in law enforcement or you're in the federal government and are concerned about national security, that you could see this being. It could become a burdensome or a labor to have to get more kind of due process than you're used to or want to give. If you're thinking only about speed of breaking up a crime or stopping a crime or stopping whatever kind of thing you're worried about, it would have been nice to see. And I think my story would have been a little bit about balance in terms of like, if we could have seen them say these are the specific instances we're worried about. So on the other side of things, I don't know what they're worried about yet. I, I, I, I would like to hear from them on that side more.
- [10:21]
A
Well, I'm sure that this will continue to be a hot button issue despite the ruling. So we will definitely be keeping up on how that all unfolds. And you will keep our readers up to date as it unfolds.
- [10:34]
B
Yes, there's a government that might want to spy on you. There might be some legal challenges. Sounds like a recipe for stories.
- [10:39]
A
Just another day in dc. Thanks for hopping on, Tim.
- [10:43]
B
Thanks, Craig.
- [10:44]
A
All right, now to our interview with Dov Euron of Command Zero. And look, security operations are changing. We've talked about this a lot this year with how AI is working into the SoC. And Dove has a really interesting outlook on the way that investigations are changing inside the SoC. So really interesting conversations about the way data is handled, about the way humans are processing that data and how humans can use AI to augment their questions and augment their investigations and to get to answers that they just weren't able to get before. AI was a thing that was coming into the sock. So really interesting conversation with Dove. Check it out.
- [11:35]
B
All right.
- [11:35]
A
Joining us on this week's interview segment for Safe Mode is Dov Yaron, the CEO and co founder of Command Zero, but has been in the cybersecurity industry a long time doing all sorts of things and wanted to have him on the program. So Dov, thanks for joining us.
- [11:51]
C
No thanks. Thanks. I appreciate the time and looking forward to the conversation.
- [11:55]
A
So for those not familiar with your latest venture, talk to us about what Command Zero does.
- [12:01]
C
So what we're really doing is helping to accelerate triage, help stand up threat hunting, help security operations teams in mid to large enterprises really tackle the onslaught of alerts and investigations that are overpopulating their screens, if you will.
- [12:20]
A
All right, so with that, the security industry, you know, spent decades and billions on detection and triage. So why is the investigation side still so broken? I mean like you said, there is just an onslaught of alerts when it comes to detection and response. So especially at this point in time with the generative AI boom, how can we move to have investigations be more fluid and we can work like in concert like human and AI working together to really remediate the problems that are happening inside enterprises.
- [12:57]
C
Yeah, exactly on point. I mean alert volumes have just Exploded and continue to explode and will continue in the future to explode given all of the agentic approaches and technology modernizations that we are employing as a society and as, as enterprises are. And really compounding that faster, I mean they're compounding faster than teams can handle. And so it's just drowning in these tens of thousands of alerts and beyond. Right. Dwell times and endless burnout. Right. It just, you know, continues to be a challenge. Right. And, and so there's just a lot of areas and where you can help address that and there's a lot of problems to kind of, to pick apart and why that's happening and happy to kind of talk through those with you.
- [13:43]
A
Yeah, I would love to. Like when an alert fires, walk me through what actually happens in a typical soc and why that process falls apart and what AI can help do to remediate the issues that analysts are currently seeing when they have to do that work.
- [13:57]
C
Right. So a lot of times existing sims and soars and other platforms, right, they're aggregating and they're automating these rules, but they're not really, really reasoning through the ambiguous nature of these rules. Right. These are multi level systems and novel threats that are hard to understand how they're coming through. So these critical incidences are still unresolved sometimes for days after the teams look at them, they analyze them, they're grabbing one piece of data, they're trying to correlate with other disparate pieces of data in the environment. They're identity their edr, their collaboration platforms. Right. There's just varying stitch work throughout the environment that can and sometimes do provide evidence as to whether an incident actually happened. So, you know, the bottleneck isn't actually, you know, collecting this data. It's not the data itself, the data is out there, but it's, it's leveraging the reasoning and the expertise around it, you know, helping the more junior analysts understand what types of questions to ask and where to find this data and hopefully auto collect and push these together for them. And similarly helping the more seasoned analysts, right, to get them out of the toil and the drudgery of collecting these informations and trying to piece together the actual narrative of the attack. Right. So the things that, you know, AI can help, and we help in particular a lot is around a lot of that mundane, those repetitive tasks, collections, you know, timelining. We can actually visualize, you know, what the layout and the sequence of that attack was, you know, the report writing, the conclusions of why a certain, a certain action had happened or why this should be the remediative steps, but also the subordinate conclusions on what types of information that was presented but was not chosen as the ultimate conclusion. So it's really being able to deploy AI and agents as part of that supplemental level of effort to human analysts, if you will.
- [16:10]
A
Security environments can be messy and unique to every organization. I'm wondering how do you make AI reliable across that kind of variability and where do you see AI being injected to really speed this up and make things work better than they already do?
- [16:26]
C
So the way we approach it is it's around reasoning and intent. It's a pretty unique perspective. So when you fundamentally break it down, the unit of expertise here is the question asking the right question on the right data source and sort of building a narrative to either prove or disprove that something had happened and provide the evidence around that. So what we've done is to sort of make it a more quantifiable and deterministic universe is we've outlined and mapped the types of questions that you'd ask from our years of experience in SecOps and incident handling and malware analysis and so on and so forth. Right. We've, you know, industry's best practices, we've codified this into a knowledge base. And so this knowledge base is being shared by both agents and humans. And so when the investigation is conducted, we reach out in a federated model. So we go to where those data sources are and ask those appropriate questions in an autonomous manner. And what we do is we actually reduce that hallucination, that drift because the agents are leveraging a codified structure, body of knowledge patterns in the sequences in which they, they ask those questions. Is AI driven like the conclusions that, you know, the gathering, you know, the evidence, the, the production of it is all AI driven, but it is still sort of restricted to a certain set of questions that they can ask. So these agents aren't just running around in an unfettered manner. And those kinds of things drive a much more consistent and structured outcome. It's predictable and we get a lot of kudos from, from our clients that hey, you know, they're getting this top level pressure, you know, thou shalt deploy AI. And truthfully, many of them are already progressively and proactively doing it, but it's a safe and consistent way that we enable them to do that. There's tangible benefits to doing so. So we're pretty excited about how those outcomes are achieved.
- [18:38]
A
So how do you toe that line between having something that is predictable and reliable and making sure that it doesn't cross a line into the risk of having AI assisted investigations create overconfidence to the point that analysts are trusting an output without truly understanding the reasoning behind it.
- [18:58]
C
Yeah, that's a really critical thought that we have this conversation often the number of folks that are skeptical or, you know, it's just a natural, natural thought here fundamentally, you know, going back to the question as that unit of expertise, right, Doing this in an open, in an open and transparent way so that you're recording and every question and every answer that's pulled back and every sort of data component that's coming back, all these questions, all these answers are all drawn together, that those conclusions are drawn. But there's complete visibility into these conclusions and into the subordinate conclusions that weren't ultimately used. Right? And so what you've developed is a strong governance model, but an audit trail. And so you now have a fundamental trust that any enterprise security team can bestow on this agentic approach here on this, on our AI autonomous investigation platform to enable that level of trust, right? All these investigations are not audible, they're also replayable. So you can actually take investigation, replay it and learn from that. And so as you're building this body of knowledge, right, with every investigation an agent runs or a human, right, that feedback loop comes back in the platform, so it gets smaller. And quite honestly, it's a stepping stone for future investigations because you're learning more context about the environment and more of the actions around that. So bestowing that level of trust is, I think, a fundamental requirement that you need to do in order to deploy AI in the first place.
- [20:37]
A
So I think as AI develops and especially as SOCs and security teams are working to defend against AI powered attacks, they're obviously leveraging AI themselves. So there's sort of this AI fighting AI out there. And I'm wondering, especially with the opinion that there's been a lot said in terms of defenders being at a disadvantage right now because attackers are using AI and it's only supercharged how fast they are moving. I'm wondering, in your opinion, do you see defenders being at a disadvantage with AI? And if they are, do you think that that is structural or permanent or is there a realistic path to closing that gap?
- [21:22]
C
There's always a first mover advantage and it is almost always the attacker that has that advantage. There's just a lot of less structure, bureaucracy, the drag of compliance, and other things that we as enterprise and law abiding citizens have to follow. So it is easy, infinitely easier to Leverage AI in an attacker mode and stitch together multiple attacks and multiple scenarios and sort of unleash that. So yes, it will take time for defenders to catch up, but it is how you deploy it, right? There's the structure, there's the environment that is already working within. It's not sort of like a completely open structuralist element. Right. There's production, there's changes that happen as you move things from development into production and so on and so forth. So environments are locked down in certain regards to help protect. And so the ability to kind of weave AI into those stories or into those. Into those environments is certainly there. You know, I think there's a lot of, I would say paralysis by over analysis. I think a lot of enterprises are a little hesitant in that there's so much going on, there's AI and with every sort of release and sort of everyone sort of one upping each other, sometimes defenders are a little reluctant to say, hey, I don't want to choose the wrong horse here. I don't want to move into a scenario where I'm not, you know, where I'll be locked into a certain vendor to a certain platform. Right. So I think that, I think there's a lot of hypertension, a little bit going on in this space. But the ones that are deploying, I think there's. There's a marked difference in improvement in quality. Right. There's a marked difference in the accuracy, you know, compared to human only. Right. 60, 70%, we're hitting the 90% accuracy on our investigation prowess. There's a consistency and there's a structure so that you have investigation, investigations are coming across more consistent and predictable between investigations so that the teams can codify and structure that in a way that can really scale. And so as teams get more and more comfortable with AI models and deploying agents and leveraging that into their system, I think it'll naturally catch up and sort of be able to help defend what some of the attackers are doing in today's environments.
- [23:56]
A
Yeah, I'm wondering, you said, you mentioned scale there, and I'm wondering if that is a difference you're seeing in what is possible with AI, especially when attackers moving in are moving in seconds and wondering, what does that do to the entire model of how a sock is structured and staffed? And you mentioned there with AI, you know, you sort of see scalability being something that socks are moving towards. Do I have that right? Like, is that what you see there? Is it scale or is it something else altogether?
- [24:27]
C
It is scale and it's Honestly, it's reinventing the SOC in its entirety. Right? It's, you know, there's the tier 1 or L1, L2, L3, and the progression and escalations as things move on. There's more flavor around. Hey, you know, I don't say reducing the teams, but, but better arming the teams. So, you know, there's a lot of, there's a lot of, hey, let me just automate tier one and get rid of that altogether and get all these cost savings. The tier one doesn't go away. It re evolves and reinvents itself. It's now sort of a Cyborg Tier 1. It, it has agentix support in helping with their decisions and helping with their collections and guiding them and even putting in remediation actions that a number of enterprises are starting to deploy. So I think you're evolving from a more structured, rigid 1, 2, 3 escalation points to teams that are seeing events from cradle to grave all the way through. And I think leveraging their teammates, I not only other teammates, literally, but other teammates, agents as part of that team allows them to scale that operation. That's, I think, a dramatic change in being able to tackle more of these alerts that are happening, quite frankly.
- [25:47]
A
Okay, let's get into the philosophy of that change a little bit, because I had a guest on recently, we were also talking about autonomous socks and he was like, it's never going to really be fully autonomous. Obviously humans are always going to be in the loop. And I'm wondering, do you agree with that? And if so, where exactly is that line? Like where exactly drilling down, do you see the human being in that loop? Is it what you were just saying there with it just being the human to make sure that we're deploying the right AI or the right agent, or using the tools, pulling the right tool out of the toolbox at the right time in order to get the job done. Is that where we're always going to have that human in the loop? Or do you see a point where it's just going to be no, the agents are going to be the soc and that's that?
- [26:34]
C
Well, the agents are going to be the SOC to some extent, but the human is sort of, you know, a coordinator and an orchestrator to that. So, you know, look at these. I mean, there's no way, you know, that you're not going to have to use agents more robustly. Right. Mythos and other things where as those attacks keep progressing and propelling, you know, slowing down for that human is A luxury that we're not going to be able to afford as, as everything propels forward. So it'll be a, whether it's a gradual shift or quick shift, no one, no one really knows. But the human, to orchestrate some of those things, there's still going to be decisions and escalations to a certain point that would require some human intervention or context around that. I think a lot of it really builds around that context and how that can be pulled into that. Some of the more simple tasks and more mundane responsive actions and isolations and other things that can be deployed can happen more autonomously and as people get more comfortable with it. And it'll be a progression as I think it'll unfold over the coming years.
- [27:45]
A
I also wonder too, going back to talking about the wave of alerts that SOC analysts have to deal with, I'm wondering, is also it a more realistic or viable outcome that AI can dramatically reduce the cognitive load on human analysts and they're not just swarmed with, responding to all the alerts that they have to deal with on a day to day basis right now?
- [28:10]
C
Yes and no. I think, I think yes in a lot of ways in that, you know, the cognitive load, I mean, these are some of them maybe the more mundane and the more, you know, voluminous things that are more repetitive in nature that are better apt for agents as well. So offloading those burdens onto the agent and having them do that I think would, would obviously make jobs more interesting and they'll be dealing with more, you know, higher level or higher order magnitude types of investigations. I think that will be more, not more interesting, it'd be sort of higher job satisfaction. Right. So that's the kind of load I think you can offload. I don't, I guess I just don't know how much of, you know, cognitive, cognitive work that is. If you're kind of sort of, it's a mundane thing, you're just kind of clicking the button for slower, that can offload anyway. So I think that'll be the mix of the reaction. I think reducing some of the burnout noise will be helpful. I don't know if that, and just the repetitive tasks, if that will sort of unload or unlighten or lighten up, that cognitive dissonance, that sucking or being compelled to step through right now, Framing
- [29:20]
A
it a little bit differently then, are there categories of incidents where you would never want AI making the call without a human reviewing it first? First, the full autonomy is definitely something that I've been having Conversations about, like I just said, I've had some guests too that are like, well, we're never going to get to fully autonomous socks. But in, in that vein, well, if we're never going to get there, is there something that humans, we definitely want to rely on humans. No matter how good the technology gets,
- [29:44]
C
you know, it's always incrementalism and slow steps. And slow steps, right. You're looking at the most basic kinds of alerts, right. You know, you know, users and possible time travel and users sort of, you know, logging in from various remote elements. Right. Starting in a simple way, you know, looking at some of those restrictive processes or remediation steps that you can look into. Right. So the idea around, I'm trying to think of a good example of a. None, none are coming to my mind right now of like where you won't ultimately go to. I just think as we start layering the foundations of more basic remediative actions and steps, I think we'll slowly, gradually, you know, cover, cover that, you know, cover the full span of remediation steps. But I'm sure, you know, as you get closer and closer to the more critical actions and the more business driven implications, you just want to have a much more focused lens on whether it's a worthwhile, you know, risk reward and automating that type of, that response.
- [30:51]
A
So finally I'm wondering, as we see these products integrate themselves into sox, what does a great security investigator look like in five years? What skills matter that don't today?
- [31:02]
C
Oh, that's an interesting question. You know, part of the, the, the rush in some of the industries like hey, let's just automate tier one and do with that role or some of that ideology was that you're not necessarily building your analysts for the futures, you know, the tier two and tier three and to gain that level experience so you know, the ability to obviously leverage and understand the mechanics around AI and agents and how to prompt them to sort of collect the right information to help to help guide those discussions, but also the fundamentals and understanding when you're being served something that is inaccurate or something that is just not realistic and sort of being able to understand the sniff test. So having a strong working knowledge of networking, having a strong knowledge of, you know, tcp, IP and DNS and these other applications that are developed in the context around those within the environment, right. That are important to that specific enterprise. I think those kinds of things will never go out of fashion. I think those will be fundamental blocks that any analyst will be able to draw from to have the context to understand when the suggested results might not appear as straightforward as you're reading it, right?
- [32:21]
A
Yes, the more and more I talk to experts, the more and more I hear about as we move forward, the fundamentals really do matter more than ever. So Dov, really appreciate you hopping aboard to discuss this fascinating conversation and we'll have to have you on again soon.
- [32:36]
C
Yeah, Greg, thank you so much for the time and looking forward to it and we'll talk soon enough.
- [32:43]
A
Thanks for listening to Safe Mode, a weekly podcast on cyber security and digital privacy, brought to you by cyberscoop. If you enjoyed this episode, please leave a rating and review and share it with your friends, your co workers, your sizzos, your sysadmins, your mom, your dad. Anybody that wants to know more about cyberpunk. To find out more information or to contact me, please look for all of our social media handles or visit cyberscoop.com thanks for listening. Check us out next week.