← CyberWire Daily![Black Hat preview: "Vulnerability Research in the Agentic Age." [Special Edition] — CyberWire Daily cover](https://megaphone.imgix.net/podcasts/0aff86e6-8d17-11f1-a9f3-332767a7fade/image/9d064a62daa0817d3d0bde95f8f0f94f.jpg?ixlib=rails-4.3.1&max-w=3000&max-h=3000&fit=crop&auto=format,compress)
CyberWire Daily
Black Hat preview: "Vulnerability Research in the Agentic Age." [Special Edition]
00:24:03
Loading summary
Transcript14 lines
- [00:02]
A
You're listening to the Cyberwire Network, powered by N2K.
- [00:25]
B
From the earliest days of cybersecurity, vulnerability research has often been viewed as equal parts science and instinct. The best researchers develop a feel for where software is likely to break, combining technical expertise with creativity and persistence. But what if that process could be understood, measured, and even systematized? I'm Dave Buettner, and this is a Cyberwire special edition. Today we're joined by Jan Sochetashvili, associate professor at Arizona State University, for a conversation inspired by his Black Hat 2026 keynote, vulnerability research in Agentic Age. Jan explores the scientific foundations of vulnerability research and reflects on a career that's uncovered thousands of vulnerabilities across an extraordinary range of technologies, from IoT devices to web browsers, kernels, and bootloaders. Using dozens of different tools, techniques, and paradigms, we'll talk about how the field has evolved, what agentic systems may mean for the future of vulnerability discovery, and why understanding the science behind the practice matters for the next generation of security research. Stay with us.
- [01:45]
C
So my lab has been doing vulnerability research for a very long time. I've been in that space since my graduate studies, and I started my graduate studies in 2010. And basically ever since then, I've been analyzing different types of systems at very deep levels for vulnerabilities. During this, I created, along with my colleagues, the ANGER binary analysis framework, a lot of other techniques and approaches to find bugs in software, to understand bugs in software, to remediate bugs in software. And we've just been pushing the envelope for a long time. So along comes this new technology, which is LLM inference for bug hunting. And in about 2023, the first kind of rumbles of, hey, this is actually pretty interestingly useful come along. And it's just accelerated since then. So now both in our research capacity and an actual security competitions, ODA competitions like the Style of Ponton and so on, we are very, very much everywhere using agentic security analysis to augment, sometimes supersede, but mostly augment our existing work. So I figured a lot of experience from my vantage point that I can draw on because, you know, as a faculty researcher, I work with dozens of students and other researchers in the space. There's a lot of lessons learned that we've taken away from a lot of different projects, and it'd be cool to spread those lessons to the community.
- [03:42]
A
Well, before we dig into the details of your presentation, help me understand how vulnerability research has been considered in the past. I mean, to what degree was it thought of as a technical capability. To what degree was it a craft or even an art form?
- [03:59]
C
It's a great question because it's been considered all of these different things to different extents. When I started, it was 2010. I started as a graduate researcher, but of course I was into security before that in the CTF scene and so on to find bugs in software. When I started, you could rely on these static analyzers with high false positives, unclear usability. You could dig into the very, very early days of dynamic analysis with sending random inputs, hoping for the best, or generating file formats, hoping for the best, hoping to observe security flaws in an application. But really the majority of bug hunting back then was manual. You would stare at this software and you would understand it. And that was very much a kind of person against software. Like almost martial art. In fact, inspired by that sort of feeling of vulnerability research as a martial art in the early days, created a whole martial art and security training with my poem college platform that really is inspired by this feeling like this is an art form. And then gradually this art form became a science as new technology came up. I would say the biggest one that created in my view really a very well defined practice of this is kind of the science. The specific techniques would be the rise of very standardized fuzzers like American Fuzzy Lop. There were some before that and a lot of rows right at the same time. But around the time of DARPA's Cyber Grand Challenge competition, the world of fuzzing kind of exploded. And now you have very well defined, like, hey, if you want to analyze this type of software, well, this is what you do. You set up this fuzzer, you add these types of seeds, you configure this type of sanitization, this type of code coverage. And it became much more step by step kind of science, let's say, than more vague art. There's still space for the art. So there's this merger of both, right? As people built up understanding of the software that they were analyzing, they could really dig in much more cleverly. There's a lot of space for this human expertise and intuition. There was a huge equalization. You could really, without knowing much about reverse engineering and so on, start finding bugs in real software. And we saw a huge increase in the number of vulnerabilities identified and disclosed and so on in that time. That era, I would claim is ending now, where hacker with a fuzzer is starting to be out competed by hackers wielding AI agents that are wielding fuzzers. And now in this Newly kind of discovered area. We're in the art and invention phase of this, but it's going to resettle into its own very well regimented. This is how you analyze software in the modern day as well.
- [07:36]
A
Does the person who has the background in vulnerability research, the person who in the previous world was a gifted artist, do they still have an advantage using the agentic tools?
- [07:51]
C
Absolutely. Especially in this phase. I'm going to say something that can probably be, can certainly be criticized and I would probably call people out if they said stuff like this, but you know, in March, Anthropic released a sneak peek into Mythos right there. Super cyber capable model with a lot of fanfare. And this is how many bugs Mythos finds in this target, this how many bugs Mythos finds in this target, and so on. And we were already also, of course, as was much of the rest of the world, doing agentic driven vulnerability research. And we look at these numbers, as did a lot of other people, and say, well wait, we're getting similar numbers without Mythos now. The difference of course, is the amount of kind of human expertise. Target specific expertise really seed not seeds in the traditional fuzzing sense of inputs into a program, test cases to drive different behavior, but seeds as in insights. Different insights given to the models and the agentic pipelines that enabled capabilities that seem well beyond the base capabilities and models. I have friends that achieve, quote, Mythos like results with open models we've achieved in many different projects called Mythos, like results with frontier models and so on. The differentiator there is that human insight, human intuition. For now, it's unclear if that will remain. I don't see it fully going away anytime really soon. You're seeing a bit of a schism with hackers actually, and how they approach this. I'm very active in the competitive capture the flag community. I ran DEFCON CTF for a couple years. I've been a core member of Shellfish for like way too long Now, I think 17 years now. That's terrifying. So I've seen a lot of different trends in the CTF community. When we came out with the anchor binary analysis framework, I mentioned it minorly compared to LLMs, minorly in a minor way. Revolutionized the reverse engineering, capture the flag category. But, you know, and when decompilers came out and got good, they revolutionized the, you know, also reverse engineering and the. And the exploitation category, the binary exploitation category of ctf as well as revolutionizing reverse engineering in the real world in ctf, what you saw, what I saw, because I have been in CTF for that long with the rise of decompilers is there were hackers that refused to adopt them that really liked looking at assembly instructions instead of pseudocode. And some of the best of them could keep up for a while, but if you didn't adopt the latest tools, you were eventually kind of forced to basically announce your retirement. And we're seeing that right now in the Capture the Flag community. All of these hackers are retiring because to them what they really loved about Capture the Flag, about that sort of applied high stake security is being kind of eaten away by the agents doing the nitty gritty like that they really loved to do. We're also seeing top CTF teams with the remaining players adapt to the modern paradigm. Because if you look at the scoreboards of the top CTFs of e.g. dEFCON CTF qualifiers, there's a quick competition with bespoke software. So it's not fully represented the real world, but there is a window to the real world at it. You see the top teams remain the top teams because they really invest in understanding how to have the human value add, how to properly harness the modern technology with agents and security analysis by agents to use their human expertise to improve over the purely agent expertise. And what I see, although less of this happens in the open outside of the CTF community, I see the same thing outside of the CTF community. I see the same thing in our research labs, I see the same thing in our spinoff companies. I see the same space for human ingenuity and insights and expertise to still be very, very, very valuable.
- [12:53]
A
So what do you think this means for the future of vulnerability research? Are we going to see the same type of person attracted to this but using tools in a different way or
- [13:06]
B
might
- [13:08]
A
a different set of people attracted to this particular area?
- [13:13]
C
I think it won't be like a disjoint set of people, but it won't be the same set. We're going to see different people being attracted to the area to different extents in different ways. I was talking with a former student of mine who's now faculty elsewhere. He mentioned a mentee of his that had found a zero day vulnerability in a complex real world system without having expertise in the types of pieces of like the core security micro skills that we would expect traditionally. And this is awesome because it broadens the field in a similar way that decompilers broaden the field, but of course much more fundamental way in my opinion than the decompilers. I Don't think we've seen a shift this fundamental, but it is not completely without precedence. You're going to see a lot more people approaching the field getting interested in cybersecurity, but we're going to see them be slightly more aloof of the really, really deep details. Again, in the same way that people that reverse engineer purely through a decompiler might be relatively ignorant of the subtleties at the assembly level. And that's going to be a very interesting transition. I think the amount of people that get into cybersecurity is going to grow because this stuff is really fun, whether you're doing it through with the help of an agent or not. And realistically, again, we've always been doing it with the help of tools for decades. So this is in that sense kind of another tool. I think it'll drive a lot of the people that really love those specific micro skills. It'll drive them out. It has already driven them out. They see a lot of old timers in cybersecurity at least leave capture the flag because, you know, they feel like what they love about it is no longer relevant. I think for some of them it's good excuse to, you know, take. Put down the cape and go to a, you know, less stressful life. But, you know, will they leave? Will those people leave security in general? I have my doubts there. I think a lot of them are still leading security companies, et cetera, et cetera, exploring the new capabilities. So I think in the end the current old guard will probably stay because the current old guard at the high level doesn't look at assembly anymore anyways. A lot of new people are going to flood in because they can now. It's much more approachable. I mentioned I run a cybersecurity education platform. It's open to the world for free. And we see a lot more interest in the higher levels because people. It used to be that people might get stuck working through a problem for a month. Now they don't, which is unfortunate in terms of developing those micro skills. But in terms of getting them through, the high level concept seems to work interestingly well. You know, that population is growing despite a lot of fears that AI will eat security. This hasn't happened yet. It could potentially, especially as the models get better, it's hard to predict what happens. My personal way that I think of the definition of the AI singularity is a point past which it becomes impossible to predict anything. And so you just kind of have to go with reasonable assumptions that you can make it'll shape over time. I think we will. We won't lose the old guard that was interested in assembly, but we might lose the new guard that is interested in assembly. You know, they might go more into architecture design, things that are a little harder to identify than security if they're really in it for the specific low level concepts like the very deep foundations of computing, which is why I am into security and the introduction of my dissertation I wrote about how the magic of computing to me happens at the assembly level. Somewhere there there's a magic that ignites and these very simple instructions come together to create very complex systems that have fascinating security implications in that the security implications are like corner cases and failure modes of this emergent, complex behavior. If people are no longer working at that level, they might gradually move into other fields. I might have moved into architecture or something instead of cybersecurity if I was coming up. Nowadays, at the risk of putting words
- [18:25]
A
in your mouth, it seems as though part of what you're saying, you're kind of expressing a cautionary tale to the old timers. Despite any nostalgia we may have for the old ways, let's not be outright bitter and resentful for the new tools that are coming along. Is that accurate?
- [18:49]
C
Absolutely. The security professionals, and this again is much more public in the CTF world because they go on Twitter and announce their retirements. But I see very similar things in the professional world, or at least examples of them. The new tech just enables a reach and a capability that simply didn't exist before. There's an incredible amount, as I'll talk in my talk at Black Hat, an incredible amount of little friction areas that agents completely eliminate, allowing better use of the tools we already had. Similar to if you hired 1,000 security engineers and let them lose with tooling on a problem. Right. These are incredible to see when you actually like are observing them in action and people focus on the obsolescence of specific individual micro skills. Of course there are various concerns about the externalities of the technology, but from a purely cyber security perspective, if. If we can ignore the externalities, which of course is tricky from purely cyber security perspective, we are living in a renaissance that I haven't seen really maybe ever, but definitely not since like the rise of capable decompilers and so on.
- [20:47]
B
Our thanks to Jan Socha Tashvili, Associate professor at Arizona State University, for joining us and sharing insights from his Black Hat 2026 keynote vulnerability research in the Agentic Age. If you enjoyed this conversation, be sure to subscribe to the Cyberwire so you never miss out on our special editions, daily news, and in depth interviews with the people shaping the cybersecurity landscape. I'm Dave Buettner. Thanks for listening.