Loading summary
Transcript100 lines
- [00:04]
A
Hello everyone, this is Tom Muren. I'm here with the Grok for another between two nerds. G', day Grak, how are you?
- [00:09]
B
Good day, Tom. Fighting yourself?
- [00:11]
A
Well, this week's edition is brought to you by Corelight. Corelight makes the open NDR platform and you can find them@corelight.com so Gra, you're telling me that everyone in the world is wrong except for you?
- [00:25]
B
I mean this has never happened before except for the day before last. But yeah.
- [00:31]
A
So what we're talking about is the reaction to anthropic releasing, not releasing, model called announcement. They've sort of partially released it, haven't they? The model is the Mythos Preview is what they're calling it. And they announced that they're releasing it to I think something like 40 organizations. Apple, Google, Microsoft and some open source
- [00:58]
B
places, foundations or whatever.
- [01:00]
A
Yeah. And the idea is that it is so good at finding vulnerabilities and writing exploits that they're giving a little limited release to allow those organizations to use it for cyber security purposes. To give them a head start, I guess, on, I don't know, whatever comes next, a broader release, I suppose. And so you were telling me some of the reactions you've seen.
- [01:26]
B
Yeah, So I was just going to say it's unfortunate that there are supply side risk and the US is not allowed to use them anymore. That sounds. It would have been useful.
- [01:37]
A
Oh yeah, yeah. Perhaps useful in government too. Huh? Huh? Oh well, moving right along.
- [01:46]
B
So, yeah, one of the things I saw was that there's a lot of people freaking out and to be fair, the people who've been freaking out the most sort of like on the one side you have Volumdev people who are freaking out a bit because it's like the end of the line, where's my jobs? Right, right. Like it's, you know, finally AI is coming for their jobs. And so they've been kind of stressed about that. Whereas some of them are just excited about like this is a cool tool
- [02:18]
A
we're going to have doing something else.
- [02:20]
B
Yeah, yeah, finally, you know, I could get that goat farm I've always want them are kind of excited about it because like they either get to find more bugs or they can finally move on with their lives.
- [02:31]
A
Something other than staring at a computer screen. Yeah.
- [02:34]
B
All day. So there's that segment, but there's this sort of like technologically aware but not security people. And they are freaking out the most. Like they are. There's one guy who called it the Great Filter, which is sort of like, one of the arguments for why we can't find alien life is that there's some sort of great filter that kills all civilizations. And apparently that great filter is Anthropics
- [03:04]
A
Preview, getting very, very good at finding exploits and vulnerabilities. Yeah, that's going to destroy us all.
- [03:13]
B
Very old denial of service exploits against OpenBSD. I don't know. There haven't been nuanced takes that I've seen so far. There haven't been like. There's been a lot of, like, oh, it's marketing hype. They're doing this all like, oh, no, it's so good, we couldn't possibly release it. If you guys, if you guys could see this, you'd be so, so scared.
- [03:38]
A
So. So your reaction is to that sentiment in particular, that this is. This is going to change everything. And it's, you know, in cyber security, I mean, the capabilities of the model do sound pretty good. So they talk about having found, is it hundreds, thousands of vulnerabilities in all sorts of different programs just by giving it pretty simple prompts like, you know, find me a vulnerability, you know, with some loose pretext. And so that actually does sound good, right? Yeah, it seems it can write exploits as well with very little supervision. So, I mean, so what is my point? Right, okay, so you don't think that makes that much difference?
- [04:28]
B
Well, like, it does and it doesn't. So I don't know if you remember, but quite a few years ago, that was just the Internet that we lived in.
- [04:38]
A
Right, right, right, right, right. So I actually saw a tweet you sent. People are freaking out about an impending flood of O days. This was the norm 20 years ago. I'm not that worried. Firstly, simply having an exploit doesn't mean all that much in terms of operational capability. Secondly, I'm giving up computers and I'm moving to a farm in the hills. So you are making, I think, in the first half of that tweet you're making, I think, a reasonable point. An exploit isn't the be all and end all of everything. But then you're like that.
- [05:14]
B
I'm making a joke.
- [05:15]
A
But yeah, like, right, so you're not actually going to move to a farm is what you're saying.
- [05:19]
B
No, no, no. I really think that people are misunderstanding how exploits fit into cybersecurity, even zero days. So while they go on and on and on about how many odes they found, there's a passage where, as an aside, they say most people are hacked with N days. So n Days are known and patched vulnerabilities, and the exploits for those are generally more useful than zero day ones. So, for example, you might remember that just one month ago, everyone was freaking out about the iOS exploit kit that got released for, you know, a version that hasn't been touched in over a year.
- [06:16]
A
Right.
- [06:16]
B
Like, it's.
- [06:17]
A
Yep.
- [06:18]
B
They were very, very upset about that. And it's like, make up your mind, dude. What are you panicking about? Like, either we have to be worried because of End Day, which is. I don't think we have to be worried, but we should be more concerned about End Day than ODE in general, or we need to be terrified of this, like, ODE threat, which, again, like, I think it very much misunderstands the role of exploits in security.
- [06:41]
A
Okay, so I'm thinking that there's a. For example, what is it? Clop. So they were a ransomware group. They would go and deliberately find oday in particular types of corporate file transfer equipment. So that would sit at the edge of networks. They would get this oda, then they would run a quick, relatively quick campaign and just steal data from everyone who had those devices full stop. And then they would sell that. We use that data for extortion. And so it seems like if there's devices that are on the edge of networks, it'll be easier to find those kind of exploits that, say, a cybercriminal might be able to just rinse and repeat. Lord. Or some AI model will probably help you automate that or do it in a pretty efficient way, speed up the process. Yeah, yeah. And so that sounds like, to me, more chaos. Right, sure.
- [07:42]
B
But that assumes that CLOP scales to
- [07:45]
A
many different organization or different groups doing the same thing. You mean.
- [07:50]
B
Exactly like that. There'll either be more CLOP replicas doing the exact same thing, or that CLOP will do more of the exact same thing. And I'm not sure that the bottleneck on cybercrime has been the difficulty of breaking in.
- [08:07]
A
Right, right.
- [08:09]
B
Like, I think that the number of cybercriminals that exist is the number of people who would be cybercriminals. Like, I think that that's a. To a degree, that's a saturated market.
- [08:20]
A
Right.
- [08:20]
B
And they might not all be operating at, you know, 100% efficiency, and so they could ramp up from there. But I'm. I'm not concerned that it's going to explode. I mean, I, I admit I could be entirely wrong on this one, but it's like, it just, it seems to me like, So I was looking at Intel Broker, who was a prolific criminal. He was one of the shiny hunters. He hacked a huge number of places like caused a huge amount of chaos. And the way that he was breaking in was he would get infostealer logs. He would look for like admin at and then try and reuse that to gain access to a VPN or something. And then once he was inside a network, he would just look through all the gits that he could find to see if there was like embedded credentials for like a database or whatever. Dump the database and then extort with that. You'll notice that there are zero exploits anywhere along that chain. He did what he did without a single exploit and the infostealers were installed without exploits. So it's.
- [09:26]
A
Right.
- [09:26]
B
I don't think that's the bottleneck.
- [09:28]
A
Right. So Anthropic released a blog about Mythos Preview. Now there's one little section where it's talking about what the cybersecurity community needs to do. And I thought this was pretty interesting. So ultimately it's about to become very difficult for the security community. And so I think there's an interesting question about the security community versus geopolitics. And so I think we'll talk about that later. Does it make any difference between states to have this capability appear? But for the cybersecurity community, we have spent the last 20 years in a relatively stable security equilibrium. And I'd agree with that, actually. But they argue that things have been since the rise of the Internet, there's been this kind of balance. Ish.
- [10:18]
B
Yeah.
- [10:19]
A
Now, I saw someone saying that the current state of security is terrible and things are going to get much worse because of this. Now I actually think things probably will get much worse. But I wouldn't describe the current situation as terrible. I would just say it's acceptable.
- [10:37]
B
The status quo is good enough to live with.
- [10:43]
A
That's right.
- [10:47]
B
Exhibit A, the world. We're getting along just fine. So I think one of the things is that people sort of misunderstand the role of security. So the role of security at a company is to enable that company to do business. And if you are selling an enterprise file transfer product, you need to introduce enough friction of security that you're not the lowest hanging fruit out of all of your competitors, that you are sufficiently secure to pass whatever security audits your customers have. And that's it. That is what you have had to do. Right. You haven't had to be secure against a incredibly big AI finding vulnerabilities in every part of your code like that has not been a threat to the business. I'm not sure it will be a threat to the business even, because people don't stop buying things just because they're insecure. Like Microsoft would have been out of business long ago, but that's just. That's not the way.
- [11:53]
A
So I think that this does upset that equilibrium. And so just again, looking narrowly at the cybersecurity community, I think there will be lots of change over the next probably five years at least, but I think it'll have ripples for probably decades and.
- [12:10]
B
Yeah.
- [12:10]
A
And that that will be changing practice to adjust to a new equilibrium. So it's often. I mean, we've talked about the sort of dynamic where defenders have. I think it was. How about Flake who said all defenders problems are political problems.
- [12:25]
B
Yes. All attackers problems are technical.
- [12:28]
A
Yeah, yeah. And I think that dynamic makes it very difficult because if the technical side gets easier, it doesn't mean the politics gets any easier.
- [12:38]
B
Right.
- [12:39]
A
But I think it changes the imperative which alters how, like the political will.
- [12:45]
B
Right. So it will change the political dynamic, but not necessarily make it easier. It might make it possible in some cases, though.
- [12:53]
A
So I think this is a genuine thing to podcast about. And if you're working in that field, I think it is a genuine thing that you'll be struggling with over the next, I don't know, however many years, I think it's a good time to ask for maybe a pay rise or bigger budget or something like that.
- [13:16]
B
I mean, if they can find the bugs, they can write the patches and they can write the patches faster than we can and they can test the patches with automated stuff again, the same tools that they use to verify that there was a bug, they could verify to use that the bug has been killed and no other bugs have been introduced. So I'm. I'm cautiously optimistic in that I think that because it is like a double edged sword, it's not a purely offensive tool.
- [13:43]
A
Yes. Yep. So let's see. Language models that can identify and then exploit security vulnerabilities at large scale could upend this tenuous equilibrium. I don't think it was tenuous. I think it was. We reached a point where the amount of effort we dedicated was sufficient.
- [14:02]
B
It wasn't equilibrium.
- [14:03]
A
Yeah, yeah. Not ideal. It wasn't a perfect place, but it was acceptable. But yes, I agree, it will upset this. The vulnerabilities that Mythos Preview finds and then exploits are the kind of findings that were previously only achievable by expert professionals. That seems plausible based on what I've seen. There's no denying that this is going to be a difficult time. I say yes, sure, okay.
- [14:27]
B
I mean, I'm a little bit hesitant on that because I think it's like, it's always a difficult time. Like there's yet to be a time when companies have gone, you know what, let's just go unlimited on our cybersecurity budget. Pull out all the stops, hire who you need to hire, buy what you need to buy, blank check, go for it. And I'll be here again next year and the year after. Like you're guaranteed all the budget resources and political will that you need to get things done. It's never been the case and if anything, this might actually change that and make it easier for cybersecurity at the corporate and enterprise level where there'll now be more resources available.
- [15:11]
A
So they've got a whole list of suggestions. I won't go into them. We believe that the capabilities that future language models bring will ultimately require a much broader ground up reimagining of computer security as a field. So that's kind of speculative. I don't really have any opinion on that. Like basically anthropic saying we see a straight line up until the right and we don't know when that's going to end. So kind of agree with you that it's not, it is a dual use thing. So it's not just all bad, all offense, nothing defensive side. But because of that political versus technical dynamic, I think it will be difficult.
- [15:54]
B
Yeah. So I think, I think one of the things that people are freaking out about is, you know, we've been looking at translator jobs, copy editor, copywriter, marketing, all of these other jobs that have been getting like subsumed into AI and now they're coming for the V Def. This is serious. This could actually impact people. You know, now I could get hurt. I don't know. I, I still feel like this focus on exploits is very myopic. There was someone on Twitter who was going on about like now any company with billion dollars worth of AI, you know, just literally anyone can be. The NSA can be better than the nsa. And like, I think that fundamentally misunderstands how the NSA works and what they do for any second agency. I don't even know where to begin. But it's just absolutely not true that NSA is a bunch of exploits and if you take away the exploits, they're a hollow shell with, you know, nothing to offer.
- [17:09]
A
Like that's, no, that's Right, Yeah, yeah. They're a way of getting secret stuff from one place out of one place and putting it in a different place where you want it so that you can actually do things with it. And I guess, like. Like just military integration, for example, like, you know, signals intelligence through to, like, bombs being dropped on particular places or. Right. Claude doesn't help you at all do any of that, but I think that, like, it's. You can't replace any organization that has tens or thousands of people with a model.
- [17:43]
B
Right.
- [17:44]
A
Like, people do things that are not just processing words or whatever.
- [17:50]
B
Yeah. Like, there's a huge amount of your job that is not your job, as it's understood. Like, your job might be do spreadsheets, but actually doing spreadsheets is part of the process, which involves talking with other teams, analyzing information, predicting what is the top priority that you need to be working on right now versus something. Like, there's a lot of things that happen which are implicit and just understood, which are not part of your job description. And that people don't realize they do because it's just a normal part of what they do. And I mean, I'm still. I'm still constantly surprised at how Claude, like the opus 4. 6, like, it's. It's good, but it just misunderstands things all the time. So I asked it to go through all these Russian files and find examples of a particular phrase that I was looking for, and it found them. And one of the examples it had was actually next to a much, much more interesting anecdote, and it skipped that. It took that anecdote and said, and this is an interesting contrast because it doesn't include your phrase. And I'm like, no, I ignore the phrase. This is gold. Like, you have to pursue this thing. Right. So, like, it. It doesn't have reasoning in that sense. Right. So it's sort of.
- [19:15]
A
Yeah, Yeah. I wouldn't be surprised if these models turn out to be better at cybersecurity than anything else there. And code, I guess, because it's, you know, defined. There's. It's often yes or no answers, as
- [19:30]
B
they put it in there.
- [19:31]
A
Yeah.
- [19:31]
B
Well, like, as they put it in the. As Anthropic put it in the paper, they can verify that they've got true positives.
- [19:38]
A
Right.
- [19:38]
B
They have all these testing tools that they can run to say, is this a real thing? Yes or no. And they can get a yes or no answer. Whereas look for this phrase and find interesting anecdotes. Does not have a. Like, that doesn't have a yes or no answer at the end of it. Right. It's a judgment call. It's very subjective. And right now. Yeah, Claude, like, AI is just not good at that.
- [20:04]
A
It seems like we kind of agree that it'll be upsetting for cyber security professionals. There'll be a transition period. So now, what about the great game? What about geopolitics and the contest between nations? Does this change anything? Like you said, US government not on that list as far as we know.
- [20:26]
B
Right. Yeah, I'm not sure it does, actually. It seems to me that one of the ways it will change things is for a long time there's a sort of if you found a good bug, there was that no bus, like, nobody but us assumption that you could run with.
- [20:42]
A
Right.
- [20:42]
B
Like EternalBlue, which was the NSA Windows exploit that they had, which they kept for like seven years or something before it was patched. It was only patched because shadow brokers leaked the file.
- [20:54]
A
Yep. Right.
- [20:54]
B
So I. I think if they had some crown jewel stuff that they were relying on that might have a shorter lifespan now, like the shelf life will be going down and that could be annoying.
- [21:05]
A
One of the things Anthropic has said is that you can do better if you have specialized frameworks. So part of what has been really surprising is that they've just said the prompts have been things like, I've heard that there's a vulnerability in here. I'm working on a ctf, go find it. But they have said that if you have frameworks, you'll be more efficient, you'll do better. And so I was thinking perhaps that for places like that, they would spend a lot of time and effort coming up with a way to get the really super edge case bugs that you can't find by just going, oh, hey, tell me about it.
- [21:43]
B
Make no mistakes.
- [21:46]
A
They'll do something cleverer than that.
- [21:49]
B
Right. And they also won't be trying to evade guardrails. The guardrails have gotten really annoying, by the way. Like, I asked Claude to make a graphic of this thing I'm calling the Circle of Cyber, which was a sort of like, intrusion, collection, destruction cycle. And I just, I wanted three arrows in a circle with the words in it. And it threw up a security alert and said, like, it wasn't going to let me use OPUS anymore. And I had to use Sonnet because I was like, in violation of something.
- [22:21]
A
The end of the world is nigh. Go use an older model. It's the Great Filter. Did it ask you if it was for PowerPoint. This is too much. You've crossed a line.
- [22:38]
B
It sounds like you work for the DoD if you're making a PowerPoint slide. Absolutely not. I don't know what triggered it. That was very frustrating. And then it couldn't make it anyway, so I had to use Gemini, which damn anthropic. So, yeah, look, I'm pessimistic about the number of roles available for exploit devs within the next 10 years. I think that might be a little bit of gasoline on the fire for a while. Right. Like, like things will rage and go out of control for a little bit and then the gasoline will have been burned up and you'll get back to just the fire that you had to begin with.
- [23:20]
A
Like it will reach some new equilibrium.
- [23:23]
B
Right, right. Like you're.
- [23:25]
A
But we were talking about state competition, right?
- [23:27]
B
Well, no, like that's, that's where I think it's going to happen as well. Like, I don't, I don't think it's going to like states compete over relatively long periods of time. They have ideas of the Russia target. It doesn't stop being interesting as a target just because it's been a single 12 month cycle. They've spent decades spying on each other. I don't see it upending things particularly. And then I don't see it having a long term impact.
- [24:01]
A
Right. So I think you can make the argument that in fact it will bolster the US because the top models are US based models and that it appears that the chip restraints and other restrictions appear to be having some impact on how quickly Chinese models are developing. I think you can also make the counter argument that the Chinese are doing well at stealing the sort of secret sauce with things like distillation attacks and.
- [24:34]
B
Yeah like their diaspora and human int and stuff. So. Yeah.
- [24:38]
A
Yeah. So I guess that means it's maybe more likely that the US and allies actually benefit from this because the best case from the Chinese appears to be that they can steal to catch up. It doesn't appear that there's any likelihood in the short term at least that they'll. The leapfrog.
- [24:58]
B
Yeah, I mean I'm still going to go back to. I, I don't think oday is what makes the nsa, as someone said long ago, you cannot download your way to parity with Fort Mead. Like it's not, you're not short a few tools to get to nsa. Like that's not a. Yeah.
- [25:15]
A
So one thing. Rob Joyce, who was the former head of Tao and was President Trump's cyber Advisor in his first term he spoke to Patrick on Risky business features I think it was. And he said that the way NSA used to win was by understanding the adversaries network better than the defenders did. And he was speaking from a TAO like cyber espionage perspective. And he actually said that the models make that easier. So he didn't mention zero days and exploits at all. It was all about understanding the, the target network.
- [25:50]
B
I will buy that. I think that.
- [25:52]
A
Yeah, so that seems to be a lower bar that you could use a less capable AI to accelerate that understanding process or come up with tools that are really helpful to help understand a network. And so that would seem to be a leveler rather than a.
- [26:08]
B
So I'm just going to do a callback to. We had a Between two Nerds where we discussed the different roles inside of a cyber team. There was like six or seven different roles. None of them was you know, keeper of the oday keys. Right. Like there was you know, like one or two that were zero day or exploit adjacent in that they had to know about them in order to select which tool to use. But the vast majority of it was like you know, project management or network assessment, planning, monitoring, you know, choosing how to approach the target and how to find a path to where you wanted to get to. And that's the sort of thing where like an exploit might feed into how you choose that path because you might be able to go for certain routes thanks to an exploit. I don't think those roles change if you make exploits massively available. Like it doesn't change all the other things that they have to do. And I'm pretty sure that as like, as a state attacker, one of the first things you try and do is get credentials so that you can have like proper access without having to do things that might trigger anomaly detection or things that might crash and break. Right. Like you want to start impersonating someone very quickly. And so yeah, like that initial access might be a little bit easier, but so what? Like that's not, that's never your bottleneck. Okay. Almost never your bottleneck.
- [27:43]
A
I was thinking that there was that example of. It seems like a Chinese group was using an older version of CLAUDE to basically orchestrate an entire campaign. And I guess a hypothesis is that it's good at bomb research and exploit development because it's so defined. Right. And that seems like it would be a similar sort of problem where you have pretty defined, you've got goals and
- [28:12]
B
you can test if you've met them or not.
- [28:14]
A
Yeah. And you know I guess there's a branching multitude of ways you can achieve those goals, but it seems like perhaps this Mythos preview and later models will actually be quite good at that kind of, I guess orchestration tasking putting together all those pieces to to get together pretty well automated campaigns. And again, I suppose it would be the US models look like they're ahead. Looks like there seems to be some sort of advantage over the Chinese ones. Perhaps that plays in the US and allies favor as well. Maybe. I don't know if they can ever work with government. Who knows. So on. Seriously risky business. I speculated Anthropic. This was like all of a week or two ago.
- [29:06]
B
So back in the ancient times, the
- [29:09]
A
old, old Times of Opus 4.6 mythos preview hadn't been released. I speculated that it was so good that NSA like a political climb down from the Secretary of War. Hegseth was unlikely, but the NSA and Cyber Command were probably, you know, writing up the memo as we spoke to say, well, we need an exemption for this. Can someone sign it off on the sly?
- [29:34]
B
Maybe they could talk to Pakistan and have them intercede for a ceasefire.
- [29:42]
A
Thanks a lot, grant.
- [29:44]
B
Thanks a lot, tom.