Loading summary
Transcript88 lines
- [00:00]
Jordan Schneider
Claude Mythos and national power. Has the atomic bomb just been discovered for cybersecurity? We have the two perfect guests on disgust to discuss Ben Buchanan, a big deal in the Biden White House on AI policy, who, full disclosure advises anthropic, as well as Michael Solmeier, who was a big deal in the Pentagon for cyber policy, lastly serving as Assistant Secretary of Defense for cyber policy. So how big a deal is Claude Myth?
- [00:33]
Ben Buchanan
This is a big one. I've been thinking about cybersecurity and AI for more than a decade. And I think a lot of us who were thinking about AI and cyber back then imagine that a day like this might come where you could see automated vulnerability discovery. It does feel like something that had long been imagined is actually now finally here. And it's up to all of us to figure out what that means.
- [00:56]
Jordan Schneider
So what can the model do?
- [00:57]
Ben Buchanan
What this system does at its core is it takes a general purpose capability. It is not a cyber built, cyber specific model. It takes a general purpose capability and then it applies that to the business of vulnerability discovery and exploit development. And as Michael can attest very well, these are fundamental tasks in cybersecurity. Finding a weakness in a piece of computer code and then figuring out how to exploit that weakness to do something as an attacker or as an intruder that you're not allowed to do. And the evidence is very clear that Claude Mythos is by far the best automated system in the world ever to do this, and is better than even some of the best expert humans in the world, or close to some of the absolute top tier expert humans in the world at this task of vulnerability discovery and exploit development. The proof is in the pudding here. It found vulnerabilities in code that all of our operating systems, all of our browsers are running. Those vulnerabilities in some cases had lurked there for multiple decades. In some instances, we thought that code was secure. Millions of automated tests had been run on that code, and yet Mythos found ways to exploit it. And I think there's a real raw capability there that is vital. And then I think the question is like, what's the analogy for that? That is really important question.
- [02:17]
Jordan Schneider
Let's stay on the sort of finding a 27 year old bug in a piece of open source software that the entire world uses. Like Michael, how wild is that?
- [02:28]
Michael Sulmeyer
That's pretty wild. It's pretty wild. I ended up talking, I think, to one of the original developers of some of that software and it was just silence, silence on the other end because.
- [02:43]
Ben Buchanan
Because everyone Thought this was like, this is like, you know, almost axiomatic in, in computer software development and in cybersecurity that this piece of code was secure
- [02:54]
Michael Sulmeyer
and that knowing that at some point this day would probably come where they, they'd find problems in it, but that, that today was going to be the
- [03:01]
Ben Buchanan
day and it would be a machine that did it.
- [03:03]
Jordan Schneider
The point being that like this, this type of thing the entire world has been looking for in this library for decades and like you would think that someone would have been able to find or exploit or patch it over something which is this, you know, which is this level of proliferation and is not the sort of thing where it's like oh, Apple pushed a new update three months ago. Like we gotta kind of work the kinks out.
- [03:33]
Ben Buchanan
No, this is long standing code there. For decades. The core credo of the open source software movement, which to be clear I totally support, is with enough eyeballs, all bugs are shallow. All bugs are shallow. Basically if enough smart people are looking, they will find everything that is to be found. And I think the answer probably for this moment is we need to have machines look too. Or at least a machine of this capability level can find things that a lot of good humans looking for a long time didn't find.
- [04:04]
Jordan Schneider
So now let's come back to the nuclear analogy. I started with the US of course invented the atomic bomb and then you know, had a good four year run of exclusive access to, to this power. You know, no one else has this model. Like just for the record, like where are we? Does the US Is the US government not allowed to with this at all?
- [04:35]
Ben Buchanan
I would ask the U.S. government. I, I don't know what the.
- [04:38]
Jordan Schneider
Okay, but there's like a six month whatever. So. Okay, so TBD on, on whether. Because I guess like, you know, my first thought was like this is almost like u boats like 1942. Happy times. If you're the one person in the world who can like use the offensive version of this, where on the other side, you know, we have now project Glasswing of the whole world trying to like harden their, harden their systems. I don't know Michael, how much fun would you have been able or how much fun would, would the, would a. A nation state doing offensive stuff potentially be able to have with this power and no one else having it?
- [05:18]
Michael Sulmeyer
Yeah, fund's probably not the, the word I would use in the official setting, but I, I definitely.
- [05:23]
Ben Buchanan
You're out of government now. You can say what you want.
- [05:25]
Michael Sulmeyer
That's, that's true. You know, I, I Think. When you think about what is the fundamental responsibility for the kind of role you have in government, it's to bring options to the senior, most decision makers. And what something like this allows for is a new set of options, if used right, for those kinds of offense and exploitation purposes, a new way to really scale those options for decision making, whatever the expected outcome is for better intelligence collection for other types of purposes. But it really opens up the opportunity space. What I think remains the same is that success in cyberspace generally has come down to it's a race. It's a race from when the offense or the exploiters know about a problem and how fast they can get at it compared to how fast the defenders can actually identify, fix it, and then disseminate the fix as broadly as possible. And so part of the answer to the question is, if you've got the offense, you're the only one, and defense doesn't know. It's pretty open season.
- [06:42]
Ben Buchanan
Yeah, I think I know Michael agrees with this, but Rob Joyce, who was the head of what was then called tailored Access operations, basically the pointier part of NSA, gave this talk at USENEXT in 2016, which. Which I actually used as the basis for a paper in 2019 or 2020, when I was starting the Cyber AI project and thinking about these things. And the basis of the talk is walking through the steps of offensive cyber operations. And this is the first time someone from NSA is out there saying essentially how an essay at a conceptual level goes about its business. And the conclusion that we came to in 2019 and 2020 was, at least theoretically, at each step of that offensive operation process or each step of the exploitation process, AI could help. Now, I think with something like Mythos, that conclusion is just far more robust that we saw the glimmers of it in 2019 and 2020. But, you know, Mythos is really doing it not just in the state of vulnerability discovery, though that's a key part of it, but also throughout the process. There's something, I think, in the system card for Mythos, where carried out a simulated network exploitation that would have taken a human 10 hours and the like. So there really is, I think, some evidence now that what cyber operators called the kill chain, which is what Joyce was describing in this talk in 2016, can be transformed by AI capabilities. Now, a separate question for Michael is, of course, like, will governments be able to adapt and so forth. That's a whole other thing. But as a technical matter, it seems to me we've crossed that Rubicon.
- [08:11]
Jordan Schneider
Well, let's talk a little bit about like the status quo ante and maybe Russia, Ukraine is the best analogy for that because that's the conflict that we've had like the most, presumably the most like no holds barred cyber going on between two countries in a hot conflict. And like you know, not the, on the, you know, when you're ranking like the things of what is determining battlefield progress or morale or whatever, it's like pretty low on the list. I mean, is your sense of that, that Michael, just like these two countries are equally sophisticated and like the technology leads you in, you know, 2022-20 through, you know, pre Mythos to like fight yourself to a standstill. Like is a Ukraine with mythos today performing like radically different, differently. If Russia doesn't have it, it's a really good hypothetical.
- [09:06]
Michael Sulmeyer
I, I think it could give Ukraine the advantage. I also really like how you have distinguished though cyber operations from electronic warfare, which is a common conflation. But you see a lot of battlefield use of vw which has had, you know, important battlefield effects. You've seen much less of the kind of cyber operations, cyber attacks type work. It was the opening shot in some sense of the conflict right, with the viasat compromise, but it wasn't really exploited and leveraged. I'd say you're probably still seeing a lot of cyber intrusions. You know, I don't again, I've been out of the business for a while. But that's different from then creating destructive or disruptive effects which would, as you said, then degrade and disrupt morale. But we shouldn't think that there isn't a lot of aggressive, malicious cyber activity going on between Russia and Ukraine right now.
- [10:01]
Ben Buchanan
And it does seem to me that if you had something like Claude Mythos as a state, you would probably want to use it for your intelligence operations or your pre positioning because you are almost by definition going to find vulnerabilities no one else knows of with this system or a system like this. And you don't want to make a lot of noise about that. You want to go in, you want to set up a persistent, quiet presence. My view for decades has been that the advantage of cyber is not this sort of whiz bang, skies fall and blackout though you can do that sometimes. It is the sort of slow, insidious shaping of the environment and collection of information that seemed like a capability to find vulnerabilities and exploit them autonomously would really help on that side of the ledger.
- [10:48]
Michael Sulmeyer
Can I ask Ben actually a question on something he just said? I Mean, you mentioned shaping, and you have a great piece from many years ago, which was prescient in a lot of ways about the difference between signaling and shaping. And I think the answer is pretty clear on how well Mythos would help with shaping. Does it help for the crowd that's obsessed with deterrence? Does it help the ability to signal through cyberspace at all?
- [11:15]
Ben Buchanan
Yeah, I think the backstory here, as Michael alluded to, is my pitch back when I was a cyber academic, even before the White House, was that cyber operations were suited to this kind of shaping. Stealing a card, stacking the deck, rather than changing how the other side plays its hand. I don't think Mythos changes that. I mean, I think the broadest thing you could say about a capability like this is in the abstract, this has some brandishing value or maybe even deterrent value, because it bolsters the status of the nation that has it if a government were to have it. But I imagine that a government who truly wanted to play offense would want this kept quiet so that people don't go looking for it and the like Anthropic has very clearly come out and said, look, our goal for this technology is not to play offense. Our goal here is to tilt the balance of power in cyber operations to the defender. And Michael mentioned the same Project Glasswing that Anthropic has going, where it's meant to try to give access to some critical software developers, Apple, Google, and the like, to make sure systems are secure before a Mythos like capability proliferates. But I think the bottom line for me is, no, this is incredibly important for understanding the landscape of modern cyber operations. But this does not fundamentally change their character, which I think is still one of shaping rather than signaling.
- [12:29]
Jordan Schneider
How messy is this going to get you brought up? Project Glasswing? Basically, the idea is like, let's give this to the adults first and let them play with it for an undetermined amount of time. So there were like 40 companies, a pretty awkward line in the press release saying, you know, we are open to partnering with federal, local, and state governments. You know, TBD on that one. But, I mean, at some point, the past. The past five years of. Of model development has shown that, like, this is not something that only one company, that a certain view about how this capability should be rolled out into the world is going to keep under wraps. So, like, I don't know what happens when someone else who's not only giving this to folks who just want to patch up holes gets access to this technology. And that could be you know, someone training a new model, someone releasing this in the US someone releasing this in China, or someone hacking, you know, hacking Anthropic servers.
- [13:32]
Ben Buchanan
Yeah, I buy the premise here. I do think. I mean, Anthropic says it in their press release like this is going to get out there at some point because folks will catch up. Let's not overstate that, though, which is to say it appears by all accounts that Mythos had a huge compute requirement to make it happen. And I'm sure we'll get to export controls at some point, Jordan, because you and I always do. Like this is not the kind of thing that you could just train out of the box without a lot of computing power. So things like export controls will constrain who has access to this level of capability for a while. How long, I'm not sure, honestly. But I do think there'll be a transition period and I do think Anthropic's model of the situation is right, that we want to, during that transition period, patch as much stuff as we can, find the vulnerabilities and patch as much as we can. And the consortium, the project last wing thing, I think is pretty interesting because you have, I think it's like 12 name members and then a broader group of companies, many of which are kind of fierce competitors, all coming together and saying, look, this is a systemic threat and we have to get ahead of it. And I think that's exactly the kind of response that I commend of, we need to do this in a transition period. Because to your question, Jordan, we don't know how much time we have. It's probably not a ton, even though I think it's more than some people expect.
- [14:46]
Michael Sulmeyer
I think what I also take away from it, there is a bureaucratic process which I'm sure every single listener is well aware of, Jordan. But after the Snowden disclosures, there was a study and a report done, and they recommended that the White House create what was called a vulnerabilities equities process. Right. What should the government do when it discovers a vulnerability in software? And how does it make the CyberSecurity trade off versus the exploitation or offense trade off? What you have through glasswing is, I think, at least to my knowledge, one of the first efforts by a private sector company that has developed a kind of capability that finds these kinds of vulnerabilities to figure out its own almost multinational vulnerability equities process, in a way. And it's, I think, a remarkable effort to manage those equities and do it In a responsible way. It's tough in government, and I think it's going to be real tough outside in the private sector to do it too.
- [15:51]
Jordan Schneider
Yeah, so let's, let's talk about that. I mean, you know, that, that there's, that whole project has had some up and ups and downs, and now we have the private sector version of it because I guess this White House, like, I mean, A, is fighting with anthropic and B, like maybe just didn't quite believe that this transformative capability was right around the corner. Michael? Yeah, let's, let's talk about the pluses and downsides of the, of the federal government seemingly taking a backseat for now to getting ahead of this.
- [16:23]
Michael Sulmeyer
A vulnerabilities equity process in government really works well, and I'm a big supporter of it. When there's a commitment to action and actually fix with urgency the problem that's pointed out, the vulnerability, where it doesn't work and where it feels disappointing is you, you go through the whole effort to do the right thing, make sure you warn the company or the entity that's got a problem and, and it doesn't feel like the urgency is there to fix it. So I guess that would be my
- [16:56]
Ben Buchanan
urgency on the part of the company.
- [16:57]
Jordan Schneider
Right?
- [16:57]
Ben Buchanan
Part of the vendor.
- [16:58]
Michael Sulmeyer
Yeah, the vendor. Right, exactly. You know, you've, you've handed them the understanding of here, this is what you got to do to fix, but now you got to go fix it and you got to do it quickly. You can't just sit on it. That, I think, is, you know, a great thing you've seen in Glasswing is anthropic saying, we're going to come back in 90 days with showing, you know, the ones that have been fixed. That'll be a good test to see the kind of urgency on the pickup by all the partners and everybody else, and also a way to improve the process going forward.
- [17:27]
Ben Buchanan
I do think this is an important point that, you know, historically, sometimes even once the patch was issued, people wouldn't apply it for a long time or the company would know the vulnerability and take their time to issue the patch. The whole process, from discovery of the bug to development of the patch to deployment of the patch, that's going to go have to go so much faster in a post mythos era, because stuff like this will proliferate and folks will be looking for these things and maybe they can reverse engineer patches and the like. So the IT industry or the kind of, you know, backbone of critical infrastructure is going to have to level up in speed here because of Mythos. That probably to me is a harbinger of what's going to come in AI that where we have things for societal resilience and the like, we're going to have to get more resilient and we're going to have to get more resilient faster for individual cycles, because AI is going to accelerate the offensive side of the ball.
- [18:22]
Michael Sulmeyer
I would also note on a good day, right, at a software company, where you're talking about a vulnerability that's found in a piece of software where the developers of that software are still employed or that software is actively supported, it's still difficult. Now, you also have to factor in the situation where you find vulnerability in a software where all the people who wrote it are gone. And the company said, we stopped supporting this darn thing years ago. And also, you know what, Rehire? Right. I mean, just thinking about how you're going to manage the scale of vulnerabilities that's going to come through here in the near term across software. Whoa.
- [18:57]
Ben Buchanan
And there's a flavor of that in critical infrastructure as well, where even if the critical infrastructure companies are still in business or the software providers, so those companies are still in business. You tell me this is closer to your world than mine, but my understanding is that is a messy set of systems to patch, and it is not meant, you know, to take critical infrastructure down every week, every two weeks, to apply a software update. Sometimes the uptime is, my understanding, measured in months or. Or years. So if one of the effects of this new world is that AI systems find vulnerabilities in critical infrastructure software with a much higher cadence, that's going to be its own complexity. And of course, the consequences of failure are pretty high there.
- [19:31]
Michael Sulmeyer
Yeah. We're just going to note right now, if you use Chrome, the smart people at Chrome now force Chrome to reboot, to apply the patch. Now, after a certain number of days, they warn you, I don't think that's happening at the local substation.
- [19:46]
Ben Buchanan
And Apple's figured out that if you give people new emojis, they'll update their iOS and they'll get some good security updates, security vegetables with it. There's a whole swath of software and the associated hardware that is not subject to that kind of patch cycle and vulnerability is found there. It's going to be a real problem. We thought a lot of software was secure, but then again, we thought some of the software that Mythos found vulnerabilities and were secure as well. And clearly it wasn't so what does
- [20:13]
Jordan Schneider
Mythos tell us about the offense versus defense dynamic with accelerating cyber capabilities? Because the. Or AI capabilities? Because the.
- [20:23]
Ben Buchanan
The hope.
- [20:23]
Jordan Schneider
Right. Is like, all right, well, maybe I don't have to hire these software engineers back. I can just, like, press a button
- [20:28]
Ben Buchanan
and things will be, I think, in the near term, putting Anthropic's efforts to benefit the defense aside, like, if Mythos were just dropped in the world, for anyone to use a capability like this would clearly benefit the offense. And I think Michael should talk about some of the ways in which it can benefit the offense at each step of the kill chain. My hope is that we can get through some kind of transition period in which it benefits the offense, mitigate that as much as possible by differentially privileging defenders, and then we end up in a world, I don't know how long this is going to take, where new code is secure and Mythos has found most of the vulnerabilities that are out there, and we have patched those. The counterargument, which is a pretty compelling one, which is why this is a hope and not necessarily prediction, is, as Michael said, some companies don't exist and their software is not going to get patched, even if a patch was developed, just because there won't be anyone to push it out. Critical infrastructure is harder. So I think in the long run, it's going to be messier. But you can tell yourself a good news story here. If society can use this technology to its fullest extent, I'm skeptical we're going to be able to manage it. But that's the good news story.
- [21:32]
Michael Sulmeyer
Yeah. You can't give up. Right. I mean, so this is, I think, the best way at scale. Glasswing. That is, I think, is the best way at scale to give defenders a fighting chance. And so I think it's. I cannot think of a different or better way to. To deal with it. But that doesn't mean there aren't still structural issues that are going to make uptake of it, you know, more challenging. Absolutely. Especially with critical infrastructure. I think, you know, Ben mentioned, you know, some parts on the offense. If you think about that old Rob Joyce becoming Ben Buchanan framework of what the offense kill chain is, from reconnaissance to gaining initial access into a system to then persisting in that system, lateral movement to get to where you want to go and then finally generating an effect, those are sort of the five parts of what Rob Joyce talked about and what. What Ben wrote about as well. AI, even without Mythos, probably helps you along Each one of those, if you wanted to say, well, which one does it help the most? Which one does it help the least? You know, there's an argument to be made that that AI would help you with persisting really quite a bit in novel ways because once you've broken in, you have to make sure you don't get caught. And finding ways to blend in with what normal looks like within a system and to adapt on the fly. That's pretty cool if you can do that. And that's a hard thing to do remotely for humans to do that in someone else's environment, probably the one that may not benefit the most, still benefit, but maybe not like persistence would, might be just effect. There's still ways to improve how you say extract data that you're trying to steal, but you're still extracting information that you're trying to steal. You might be able to do it in some more creative ways, maybe do it, maybe extract more in a shorter amount of time. But I'm not sure that's where you're going to really see the step change on offense from AI.
- [23:46]
Ben Buchanan
There is an interesting notion here, which is if you look at the most powerful cyber attacks historically, and here I'm saying attacks very deliberately, as opposed to espionage, there is already, before we even get to the machine learning era, there is an automated component to a lot of them and it's that automation that gives it the scale. So whoever attacked the Iranian centrifuges in Stuxnet, that clearly is an automated component, lets it spread from system to system to have the reach that it did. If you Flash forward to 2017, the WannaCry attack from North Korea, knock through was meant to be an attack, but was an attack and had an automated propagation to it. The Russian attack, not Petya in 2017, probably still on a dollar value basis, the most damaging destructive cyber attack in history, probably more than $10 billion worth of damage. Clearly very, very automated. So there is, I think, an intuition we can develop in which automation in cyber operations, even before the machine learning era, can yield the power that manual operations can't. And there have been some near misses. I mean, one of, I think the most overlooked cyber attacks in history was the Russian blackout In Ukraine in 2016, a case called Crash Override. The Russians, for context, they'd been in Ukraine 2015, December 2015, they caused a blackout, very manual. It's like this beautiful symphony of all these different pieces of the operation coming in at once, but totally manual. A year later, the Russians come back, they try it Again with crash override. But they do it in a totally automated way. Now, maybe the Russians being the Russians, they screw it up. So, like the effects of it actually are not power's out for an hour or something. The effects are not world changing. But one of the questions that came to my mind when I started seeing the cyber capabilities of Mythos is, wow, I wonder how this would work in targeting critical infrastructure. Could this actually manipulate a programmable logic controller or industrial control system in a way that has a kinetic effect in the way the Russians tried and failed to do? The answer might be yes, or if this one can't, the next one can't. And I do think there's an element here in which the automation of the kill chain, the way that Michael is describing, yields more overall power for the attacker.
- [25:54]
Michael Sulmeyer
On the flip side though, for defense, you put aside AI. There's things we've known for years that you could do to frustrate efforts like that. Air gapping an OT network from an IT network, for example, that doesn't take AI to do that on defense, and it doesn't necessarily take AI to exploit it. But the fact is that that kind of foundational step isn't done nearly enough. It isn't implemented nearly enough. So I wouldn't also want to lose the point that foundational cybersecurity measures, that is just doing what we know works is not something that goes away just because of Mythos. You should still maybe all the more reason to urgently do what you know you probably should have done a while ago, because that will help you. May not totally protect you, it was never going to totally protect you anyway, but it will make a model's life harder to jump. Right? An air gap, for example, at a critical infrastructure system.
- [26:54]
Ben Buchanan
Yeah, I totally agree with that. I mean, I think basically we're both, not that we pre planned this, but we're both landing in this spot of Mythos is a game changer for cyber operations in that, you know, this is. It's going to change how sides play the game, but it's still the same game. And the core kind of tenets of what works in cybersecurity, what doesn't work in cybersecurity. I think those are going to hold for a while. And one of the chief ones, as Michael just said, is the defender has this huge advantage which so few defenders actually realize and claim for themselves of. They set the terrain, they get to decide where the operations are going to take place within their network in terms of protecting the boundaries of the network. And air gapping and the like. And I guess I'm optimistic that Mythos can, in the right hands, aid defenders by making those networks more secure. As Michael said earlier in the conversation, by finding the configuration errors before the attackers do and remediating those. The cat announcing it is going to go on for a while here.
- [27:48]
Michael Sulmeyer
I've seen it work really well. Unfortunately, only after the organization has been had. Once you get nailed, then you find that it turns out, oh, I could just air gap these, these two networks or I could just implement these kinds of changes. And yeah, I could have done it before, but now I've got the urgency now I've got the resources right now. Now I can do it. And we just. Even though all the evidence in the world said that was the thing to do, it's like you needed to be attacked first to. To be convinced. And I hope we can get over that hump.
- [28:23]
Ben Buchanan
What was. I'm trying to think of the case. There was the. A bunch of American banks. Was it JP Morgan in 2012 got a lot of Iranian incoming and ddosing and intrusions and stuff. And that was I think one of the seven elements Financial sector. When you talk to them five years later and like, how did you guys get serious about this and the way they are like, well recognized a very clear business case with credibility to us. And also we felt the incoming in 2012 and in some sense I think they were lucky that those were not in the end extremely destructive attacks. But it was a galvanizing moment for the industry. Probably what Anthropic would say and certainly what I would say is Mythos should be a galvanizing moment for every industry. And as a reminder that now the clock is really ticking before the offensive side of the ball levels up in cyber and defense has to get there first.
- [29:05]
Michael Sulmeyer
But I think on those Iranian DDoS attacks, what I think is also really important to note is that those institutions, whether they knew it or not, at the C suite level, at the CIO level, I think knew they could have worked with and been under Akamai's protection to have resisted that kind of activity anyway. They just hadn't wanted to do that.
- [29:30]
Ben Buchanan
No, it was all totally.
- [29:31]
Michael Sulmeyer
So yeah, it was totally preventable. They just again needed to unfortunately go through it. It seemed like not to pick on. Not to pick on them because it's hard to just. I get it. But that just, I think furthers the point. They know what to do.
- [29:44]
Ben Buchanan
We can give Jordan a show back here.
- [29:47]
Jordan Schneider
No, this is amazing. Thank you guys. There were Some really interesting lines which were almost like an emotional plea in this sort of Red Team report by these outside cybersecurity experts saying, like, we're in for, we're in for it. Like, strap in. This is going to be really messy and really painful. Ben, you mentioned the $10 billion of damage by a Russian cyber attack that kind of like went awry. I've been asking ChatGPT for ransomware and cyber extortion numbers, which like, seem really low, like in the tens of millions or like a hundred, $120 million in 2025.
- [30:29]
Ben Buchanan
I'd have to imagine it's more than that. Globally, in aggregate. Yeah, yeah, I would have to imagine it's much more than that.
- [30:34]
Jordan Schneider
Yeah. That seems this is the wrong number. Okay, so, you know, cyber extortion maybe in the like single digit or like at least, or definitely low. It's kind of like $10 billion a year. Seems like that's going to change. In the meantime. I don't know how much, how much more mischievousness can you. If your goal isn't just the effect of like, I don't know, finding plans for schematics for, for fighter jets and like, you know, poking around systems like China's been reported as doing over the past few years in the US but, but really like just messing things up in a, like, I don't care what happens way or in a, I want to extract enormous amounts of money from desperate organizations perspective.
- [31:27]
Ben Buchanan
Yeah, I think there's no doubt that a capability like this in the wrong hands would allow a lot of that. I mean, it's hard to put a dollar figure on it, but you could probably do billions of dollars with damage if you were going no holds barred here if enough groups had access to it. An interesting case, I think it's from maybe January or February of 26 is a case called Void Link, where it was, I'm pretty sure it was a ransomware group. And the defenders kind of teased apart the code and they realized the code itself had been all written by, I think, Claude, but one of the AI systems. And there was a, rather than a kind of bigger ransomware operation, it was just a small number of people. Maybe in fact one person that had essentially vibe coded a ransomware operation and was going out and carrying it out. And it's pretty clear that even before Mythos, this is a capability that was coming online. In some sense, I actually think Mythos gets the play. But in some sense, I think society crossed the Rubicon with Opus 4.6 in January, February, whenever it was where Anthropic found 500 high severity vulnerabilities. Okay, they weren't as big a deal as what Mythos found, but it wouldn't surprise me if we look back with the benefit of hindsight and say that's when the exponential really started to take off. And Void Link was in the mix for that. And I think showing how non state actors could pick that up, I think
- [32:47]
Michael Sulmeyer
we've become, and rightly so, for a lot of reasons, very focused on China as the. You call it pure competitor, whatever language you know, you want to have. But you know, what happens when a Russia or China gets their hands on something like this? You know, for Ukraine, sure, the Russians are going to use it to bully them and harass them and, and attack them. But I think more broadly, the point you were making earlier is it's a really compelling espionage tool, which means you don't use it to screw things up and, and make yourself known. However, you know, in, in focusing on the great power competition, that means we've, as a result, put counterterrorism on a bit of a deprioritized basis. Right. In some sense, it was the fifth priority. China, Russia, Iran, North Korea, terrorists. And the goal was keep number five. Number five. And the challenge here is that the terrorists have all the incentive in the world to screw things up and, you know, make things be very cartels or
- [33:48]
Ben Buchanan
the Houthis or like a pretty broad group of non state actors.
- [33:51]
Michael Sulmeyer
Right, yeah. Should not.
- [33:52]
Jordan Schneider
And you're rounding down even North Korea. I mean, you know, this is their game. Right. Just like trying to make money off cyber hacks.
- [33:59]
Michael Sulmeyer
Scamming. Yes, for sure. I mean, and beating the sanctions. For sure. I would distinguish the scamming and financial expectation from more destructive and disruptive things that I think terrorists and probably cartels and others would have more reason to do than North Korea would have, you know, an incentive to do. They'd want to use it for their own purposes. Which as you say. Right. Is more about scamming the system.
- [34:27]
Jordan Schneider
What does the next step of the exponential look like in six or nine months?
- [34:32]
Ben Buchanan
I have to assume it's almost a kind of article of faith at this point. I have to assume there are vulnerabilities out there that Mythos does not find, cannot find, and that a better system would find. I doubt we're at the ceiling of AI cyber capabilities. We've saturated every benchmark Anthropic has reported. Essentially, it's very hard to measure Mythos capabilities because it aces every single test or Close to it, but I have to assume there's a little more headroom, maybe a lot more headroom to go. One of the ways that you could think about it is this is not original to me. Someone else said this. All of the vulnerabilities that Mythos finds are vulnerabilities that are immediately legible to a human once they're kind of explained. Some of them are very clever, to be sure, but there's not a lot of doubt about this. It is sometimes the case that a human, historically, could find a whole new class of vulnerabilities where it's a weakness that, you know, exploited shows up again and again and again in different places because we didn't recognize that that is, in fact, a weakness. At first, it looks alien. Then once it explains, like, wow, it makes a lot of sense. Maybe the next generation, maybe the next turn of the crank, starts finding more of these vulnerabilities that are less intuitive to us.
- [35:43]
Michael Sulmeyer
I think from a, you know, cyber and AI development standpoint, that's absolutely right. When I think about it, and you step back, you know, how do most citizens look at or relate to this technology? And I think most of the citizenry looks at it and says, why do I keep getting these preposterous phone calls scamming me, you know, for my. Asking me for my credit card number, or some prince in Krablakistan is offering me $5,000? It's very possible that at that level, they don't perceive much of a change. Yeah, because it's. It's possible that those who get their hands on Mythos and want to cause abuse have. Have other, bigger fish to fry, but that the ransomware gangs and the scammers keep at it. And that on defense, we don't. We don't look at stopping the scammers that hit, you know, vulnerable populations like senior citizens. So I think at a. At another level, it's possible that there's a big segment of the population that's fine. My life is still being severely irritated by cyber scammers one. One way or the other.
- [36:45]
Ben Buchanan
And that's where we get to the defensive side of the ball, where it's use Mythos to raise the bar for what it takes for an attacker or a scammer or a spy to achieve their objective. And frankly, I think it's an open question. I think this Glasswing thing is a really noble undertaking, and I commend everyone involved, all the companies for doing it. But the press release is not the point. The point is, if we sit here in six months, have they patched 10,000 or however many vulnerabilities it's going to be high severity vulnerabilities in the collective ecosystem. And had. That actually had the effect of meaningfully raising the bar for intruders. That's a very open question and would
- [37:22]
Michael Sulmeyer
stave off a crisis, but may not have any real impact on, again, senior citizens.
- [37:28]
Ben Buchanan
Yeah, if the phones are insecure, then it doesn't matter.
- [37:32]
Jordan Schneider
Yeah, I mean, speaking of the senior citizens, though, it's like we've had all these hospital ransomware things over the past few years, right. And if NYU Langone every two weeks is getting another pay us $500 million or we're going to delete your entire system, then like, this is a very real thing. In which I think sort of cyber extortion was like a sexy news story and a few school systems and hospitals got screwed up. But this was not like a, a society shattering trend over the past 10 or 15 years. And in the way that once you have this proliferated, it may end up being that.
- [38:11]
Ben Buchanan
That's why I want to raise the bar for defenders.
- [38:15]
Jordan Schneider
Ben, you mentioned this idea of like kind of new classes of, of exploits and things that don't, aren't like legible to human beings. You know, is there a, like, is there a theoretical limit? Like, is it possible to be sure that code is secure? I guess the answer is no.
- [38:35]
Ben Buchanan
No. The answer is actually yes. The answer is yes. So there's a branch of computer science called formal methods, which essentially gives a mathematical guarantee, a provable guarantee that a particular piece of code is secure. Now, right now we cannot do very much with formal methods. I think they're fairly limited. But I can imagine we are sitting here in five years, 10 years, who knows, maybe given AI acceleration, even less time. And all this has gone great. And we're shipping secure code because we, we have useful methods and AI can help with that. That is a possible end state here. That's a very desirable end state that I think I would love to get to. We are nowhere near that right now. But in the sense of can we use AI for defenders, one way we can do it, as I said earlier, is to shape the terrain in which the operation unfolds and we can just ship better code. And at the limit, at the mathematical limit, that leads us to formal methods.
- [39:31]
Jordan Schneider
Okay. Talking about norms, you know, it's interesting that the most dramatic AI national security application, well, I mean, I guess like targeting kind of TBD on that, but is cyber, which is A space where it's kind of no holds barred sitting here in 2026. Whereas if it was something around bioweapons or chemical weapons, then maybe because there was kind of already a global norm that like this stuff isn't cool, there might have been a sort of more rich path to have an international dialogue on, on potentially not exploring this tech tech tree. I'm curious for both of you, like, given that the big one is coming in cyber first with AI. Like what that you know, what other thoughts or implications you have on that.
- [40:27]
Ben Buchanan
I think we are very fortunate that cyber is coming first and I think we should use cyber as a lesson for what is coming next. At the intersection of AI and other fields, Bio will not be far behind. At some point we will have a Mythos moment for bio. I'm not smart enough as a biologist to know what that looks like, but I'm confident that is the direction of travel. Maybe the norms save us at that point. I kind of doubt it to be honest, especially when it comes to the non state groups and all of that. But who knows. But I do think one lesson we should take away from Mythos is not, oh wow, this means AI is really good at cyber, but AI is really good and as I said at the outset, this is a general purpose system that happens to be good at cyber. If you read the anthropic system card for Mythos, it's also really good at bio and I imagine the next version is going to be even better. So I would hope that. There's been a lot of debates for the last five years about how good our AI system is going to be. Obviously folks like me have argued for a very long time they're going to be very good faster than people think. I'm biased here, but this feels like a pretty big piece of evidence that should update us towards, wow, let's take AI risk seriously in cyber, yes, but also in things like bio, because those are not going to be far behind.
- [41:37]
Michael Sulmeyer
You know, you mentioned norms and the cyber community had a multi decade effort to try to figure out what kind of international normative commitments could be made among countries about what behavior they wouldn't do to each other during peacetime. And that was a noble effort. But I remember some Israeli colleagues telling me at one point ten years ago, you know, you miss the boat on starting a normative effort. You want to start the effort when you have enough of an advantage that the other side doesn't quite see it yet. And so you can get everybody to commit to maybe tying Half a hand behind their back because people don't quite see it as so detrimental to their own self interest. You start too late. Everybody's so invested in trying to use the technology in their own way to pursue their objectives. It's hard to get those kind of commitments. And so a question I have, and I wish Joe Nye was still here to talk, talk to him about it, is you know, have we already missed that moment in, in AI not saying that a normative regime, you know, that we should be spending a ton of, of effort on that or not just that if you're asking about norms, that would be. I guess my question is, is it already a little too late?
- [42:59]
Jordan Schneider
Yeah. And the, the thing with chemical weapons, right, is like they didn't win you World War I, so when it's still kind of an open question, then, you know, there's a lot more excitement and incentive to kind explore the possibility space.
- [43:17]
Ben Buchanan
Yeah, it's pretty clear to me the next wave to crash in terms of big societal, national security things is going to be bio. And I hope we, you know, people who are skeptics of AI look at Mythos and what it does for cyber and say this should cause me to rethink my prior views when it comes to AI and bio.
- [43:37]
Jordan Schneider
So Michael, at the very beginning of the show you, we allude, you alluded to this sort of like personal phishing type cyber stuff. The last show I did with Ben, I tried to sell him on like the, the US China AI companion race and just the, the potential implications that like AI powered case agenting and like recruiting spies and just getting people to do things around the world. AI being able to like radically improve if you, if you have a capability and sort of people aren't ready for it. Now, you know, Mythos doesn't necessarily like give you the video call with your mother that is probably like the, the, the, the, the true frontier for this. But I guess I'm curious maybe for your general thoughts on how AI is going to be able to sort of ramp up that like human relationship establishing side of the, I don't know, convincing a soldier not to fight in a war or someone to give you some secrets or a country to, I don't know, even like revolt against their leadership or something.
- [44:43]
Michael Sulmeyer
Well, it's a, it's a great way to talk about that nexus between cyber and information operations. You know, what the military world might call information operations, if cyber operations or if cyberspace is the delivery system, the way of getting to the information operations. Well, what does the Message say what is the content of the message? What does that look like for the purpose of trying to convince you to not do something that day or to do something. And that is something that I think you didn't need Mythos to see how much more convincing deepfakes were becoming. And there's an international security dimension to it. There's also a very, very at home dimension to it. I've, you know, post government have been helping K through 12 school districts look at the kinds of new security threats and challenges they're facing and deep fakes of, you know, by students against other students, by students against teachers. It's scary how this is playing out because it's so convincing. It's very, very difficult I think to have a technological solution, to have AI figure out if that's an AI generated message. And so the opportunity for then known human validation and follow up requires a level of discipline and process that I'm not sure our institutions have really developed.
- [46:09]
Ben Buchanan
I think it's deepfakes for sure and the images and the audio and the video. But one of the really surprising things to me about AI, and I'm sure Mythos is good at this too, is and frankly the whole class of systems, including From Google and OpenAI is just how convincing they can be with text alone. So this was the last academic project I did before I went to the White House at Georgetown. We had this thing where we used GPT3 before GPT3 was released for the public, like an early version of GPT3 to essentially see if it could persuade people on two political issues. One was should the US be more aggressive towards China? And the other was should the US withdraw from Afghanistan? This was the summer of 2020 and it could measurably in a statistically significant way, just write basically I think single shot text messages that would change people's minds. And that was 2020. And if you look at all that's been happened since then, I think there was a nature study on it, a Stanford study on it. It's pretty clear that AI systems have only gotten better. The one that's so striking to me, and I'm going to butcher some of the stats here, but this is pretty close to right is there's a subreddit called Change My View where people post an opinion and then it'll kind of award points. I think they call them deltas for folks who can give them compelling counterarguments. And some researchers used in 2024, 2025 an AI system to post on Change My view I think it's scored in the top 1% of earning these kind of points and changing people's minds. So we've kind of strayed a little bit from cyber operations as narrowly defined, but I think it gets to the broader point here of. In a renewed competitive information environment, an AI system can be useful for a wide range of aspects of national competition. Cyber operations on offense, cyber operations on defense, but all the. Also the adjacent category of information operations.
- [47:57]
Jordan Schneider
Yeah, Michael, you know, you brought up information operations, which folks will debate, but I think have been like, somewhere between, like, ineffective and like, national embarrassment over the past few decades. And, you know, when maybe that we can close on kind of like, like bureaucratic uptake for these tools. Because, yes, it may be hard to sort of like, have someone who can speak very well into galog that the, the Department of Defense can hire to sort of push some narrative into the, you know, Philippine political system or whatever, but if all you have to do is press a button, then it, it, it starts to be a lot easier to kind of do some of this more nefarious stuff of just like, con or nefarious or positive for whatever reason, you know, for. Depending on how you're using this power to convince people to, you know, show up or not show up a vote a certain way or give you their secrets or whatever. I don't know. Aside from just having everyone harden their systems, like, what is there anything different from the sort of slate of recommendations that you would or the types of things you were pushing during your time in the Biden administrations, or the slate of recommendations that you would want to give to the US Government around cyber sort of pre versus a post Mythos era.
- [49:22]
Ben Buchanan
You know, Jordan, you know this, but near and dear to my heart is building an American lead and a Democratic lead in AI. To me, this whole conversation we've had here, the cyber dimensions of it and everything else reaffirm, reaffirmed the importance of doing that. Obviously, that's an export control conversation, that's a domestic investment conversation, that's a infrastructure buildout conversation. But one of the things we tried very hard to do in the Biden administration was to ensure democratic preeminence in AI, in part because we had a high conviction that things like Mythos were coming, that this technology would be useful for national security and geopolitical competition. Frankly, I think a lot of folks don't take my word for it. If you look at what Dean Ball wrote, Dean Ball FORMER TRUMP administration ADVISOR when he left the government, he commented on, I think on Basically how non AGI pilled his colleagues were, how they don't believe in a world of very powerful AI systems. And I am hopeful that for them and for other people, things like Mythos and the broader development of AI capabilities can be a reminder that we really want America to lead in this technology. And it would be so much worse for the world if China had this. And I doubt if China had this. They'd be giving it to defenders first and making a lot of noise about the need to patch systems. They'd be doing exactly the kind of espionage that we've been talking about here.
- [50:42]
Jordan Schneider
You know, it's an interesting Jensen proof point, right, where he has been talking for many years about how this stuff is entirely revolutionary, but like, has basically been pretty quiet about it from a national security perspective saying like, ah, you know, they'll be able to get all the chips they need. It's not really going to sort of of it's not going to change the world when it comes to the sharp end of what governments want to do. And here's another proof point against that. I don't know. Michael, take us out. Any closing thoughts on all this?
- [51:15]
Michael Sulmeyer
I mean, you have to hope that the leaders of America's warfighters know that this is a technology they have to adopt to really make sure that our offensive cyber operators maintain and extend a competitive advantage. I mean, as Ben said, this is important to win and this is an important tool to extend that advantage for the nation in cyberspace. I think a large debate that probably has not played out enough on defense is what kind of autonomy our leaders are comfortable with for a model to run for cyber defense, for cyber security on sensitive military networks. It doesn't really work, I think, to have the model merely alert a human that there might be a problem. And yet you'd be right to have some caution about just turning the keys over to the model to say, hey, keep us safe. So I worry that between the CIOs and these kinds of bureaucracies, the instinct is to maintain that kind of human accountability and insight and not disrupt the business process, but increasingly viewing it as an operational matter, as a contested domain where you have to put more weight on autonomy to defend faster. That's where I worry. There's a really tough conversation coming and there's going to be risk that has to be taken to lean on AI to keep us safe.
- [52:55]
Jordan Schneider
That was excellent. Thank you two so much for being a part of China talk. Is this war talk? Is this model?
- [53:03]
Ben Buchanan
Jordan, I told Michael it's horrible. Hey, Jordan, I told Michael that your show used to be called China Econ Talk, and then you dropped the Econ and it became China Talk. And now you do so much stuff on AI and other things, you're just going to be talking.
- [53:16]
Jordan Schneider
Or is this talk talk or talk talk?
- [53:17]
Ben Buchanan
Yeah, talk talk. Even better. Thanks for having us, Jordan.
- [53:38]
Narrator/Poet
Born in 99 in a library down the information highway road no one knew his name or his address he lived inside a function call Every browser was his kingdom and he answered to no one at all 27 on the rock the federalis never found his trace A billion machines gave him shelter and the law could not keep pace they sent their smartest coders after him the buzzers swept the land but the outlaw slept like a baby in a line no one could understand. They called him Maximo they swore his own was clean A million audits crossed his border and not a single one had seen the way he lived inside the logic like a bandit in the hills Open source men swore they'd catch him but he's laughing, living still he took a wife inside of the kernel Baby on the way Bar in your memory Plan to always stay them from the north Not a man but a machine Mythos rodent with the coldest eyes he'd ever seen. Silence on the telephone when they told the old developer the code you wrote in 99 senor has finally met its killer the outlaw tipped his hat to Mythos smiled and said amigo, well done but don't you cry for this Bundy though for his cousin cousins ride tonight there's one who's 34 with a pension in a substation with no light.
- [56:39]
Advertisement Announcer
Your next chapter in healthcare starts at Carrington College's School of Nursing in Portland. Join us for our open house on Tuesday, January 13th from 4 to 7pm you'll tour our campus, see live demos, meet instructors and learn about our associate degree in nursing program that prepares you to become a registered nurse. Take the first step toward your nursing career. Save your spot now at Carrington Edu Events. For information on program outcomes, visit Carrington. Edu Sci. Starting a business can seem like a daunting task unless you have a partner. Like Shopify, they have the tools you need to start and grow your business. From designing a website to marketing, to selling and beyond, Shopify can help with everything you need. There's a reason millions of companies like Mattel, Heinz and Allbirds continue to trust and use them. With Shopify on your side, turn your big business idea into Sign up for your $1 per month trial@shopify.com specialoffer.