Loading summary
Transcript103 lines
- [00:04]
A
Hello everyone, this is Tom Uren. I'm here with the Gruck for another between two nerds. G', day, Grok. How are you?
- [00:08]
B
Good day, Tom. I'm fine. And yourself?
- [00:10]
A
I'm very well. This week's edition is brought to you by Sublime Security, which is a next generation email security company. You can find them at Sublime Security. So I came across this tweet from Jerry Gamblin and it's got this chart about CVEs discovered in the first six months of each year. And it basically goes kind of exponentially in the last couple of years. So relatively stable. And then it's basically.
- [00:46]
B
It needs to be plotted on a log to have a nice line.
- [00:49]
A
Yeah, that's right. And so there's all sorts of caveats around publish CVEs, you know, are they worth anything, what can you do with them, et cetera, et cetera, et cetera. But it made me think, obviously, I suppose these, a lot of these are AI discovered.
- [01:07]
B
Yep.
- [01:08]
A
And there's really a lot of them. Where are all the hacks? So the maybe. Is it a conventional wisdom? I think it is perhaps a conventional wisdom that discovering more and more vulnerabilities will lead almost inevitably and directly to more and more hacks. But at least so far, based on my just, you know, writing a newsletter about it every week, I'm not seeing exponentially more hacks than it was a year or two ago. Right.
- [01:36]
B
Yeah. So that I'd say one of the first pushbacks you might get would be people saying, okay, there might be more like attacking, there might be more cyber weapons, but there's not more cyber warriors. Right. And I think the response to that is like, LLMs are supposed to solve that too, right?
- [01:55]
A
Yeah. Yeah.
- [01:56]
B
So it's not simply a matter of like, there's so many things available and there just aren't enough people. It's like you can be so efficient with an LLM. Like one person could be a 10 times hacker, like a 10x hacker. So that can't be the reason, I think.
- [02:17]
A
Yeah. So one theory I had was that people have just not caught up yet. So there's.
- [02:25]
B
Okay. Yep.
- [02:26]
A
You know, A delay between a goldfield being discovered.
- [02:30]
B
Right.
- [02:31]
A
And a gold rush perhaps.
- [02:36]
B
And hardware store is already out of shovels. Yeah.
- [02:40]
A
And I guess in a serious way, all those things where there's a. I don't know if you want to call it a bubble, but a whole lot of people rushing into a field because there's new opportunity, there is a supply chain behind that. Where you've got to develop the, you know, like the gold rush shovels or whatever it is.
- [03:00]
B
Yeah. Someone has to start making gold panning pans. Right. Like all stuff gets whatever it is. Right. The logistics get spun up and all of these companies are like rushing to the space and there's stuff happening, but somehow the other side isn't showing up, I guess.
- [03:18]
A
Yeah. So the twice in recent months I've written about cases where a security firm has gotten logs from a hacker that has been using mostly Claude, but also often is it Gemini, Codex, whatever, the OpenAI one and the Google one. And in each case the hacker has been able to use those machines to like do hacking. Like. Right. Able to pretty much get them to do guard execute commands. Yeah.
- [04:00]
B
They can convince the LLM that it's a legitimate red teaming exercise and it should do the hacking hack stuff.
- [04:07]
A
And in both cases, those hackers have known enough to get the machine to do what they want. They look like they've had some practice
- [04:17]
B
and they know what they want to do to a degree as well. Right. They're not saying hack this company for me. Where do we start? Okay, now what do we do?
- [04:27]
A
They've got some basic knowledge. They are able to get the machine to do what they want. They, the hacking is competent, straightforward, sufficient. Yeah, yeah. And I think you probably would be as good as a competent ransomware person who's using a play or even better.
- [04:44]
B
Right. They are sufficient enough to be criminal hackers. Like they, they have the right skill level to do any of the. So that's not what, that's not what is holding them back. Right. It's not that even with an LLM they can't do sort of like this basic contractor work that you're getting at a ransomware gang. So that's not the bottleneck.
- [05:07]
A
Right. Yeah. So it's very much like they're operating as well as a good pen tester would.
- [05:13]
B
Right.
- [05:14]
A
Which kind of makes sense because the models should push back on anything that looks like it's not. Could not possibly be a legitimate pen tester. Right. So that's some of the framing they've used. They've look like they've had practice in that they've got that they go straight away to techniques that bypass safeguards. They're not stumbling around in the dark. It's something they've done before. One of them, for example, had stolen several different CLAUDE directories from compromised boxes. And so I think my understanding is that the authentication tokens were in those directories. And so they were able to just hijack that account and get basically free access. And so they had, it seems like a library of these compromised folders.
- [06:02]
B
You know what, this got me thinking. So historically, one of the things that hackers would do is they would hack each other to steal o days. Like they'd steal exploits from each other. And I wonder if these days you would sort of steal. You'd steal skill sets for giving to Claude and access tokens. I wonder if that's supplanted it. Just idle thought. Yeah.
- [06:31]
A
So they both seem practiced. This particular case, it was a bloke in Ethiopia who had stolen someone's Claude directory out of a compromised computer, but then didn't bother wiping the conversation. So Gord kept on thinking that they were a developer from the Netherlands
- [06:54]
B
trying
- [06:54]
A
to speak to them in Dutch. Please respond in English. So there's this curious mix of both practiced and yet also like totally dumb. And for that particular person, he compromised, I think the report said, like over a dozen companies, but when it came to actually making money out of them, he spent quite a lot of time trying to get the machines, the LLMs, to tell him, how do I make money out of this? And they were obviously pushing back because that doesn't sound like a legitimate pen tester.
- [07:33]
B
Yeah. So like you could probably event the guardrails to get it to help you to break into something like to do some pen testing stuff. But yeah, I could see that the guardrails were probably much harder to evade on the now convert this into cash money for me.
- [07:52]
A
So, I mean, obviously the way for a legitimate pen test to do that is to have a contract before you break into the company.
- [08:00]
B
Typically the way that pen testing works is you break into somewhere, then you show up at their door with a contract saying, you know, it's, you know, we're going to do four days of work and you'll get a report and it's going to cost you this much. Also, you're on day three now.
- [08:18]
A
So in that particular case, it's unclear. The LLMs did give him some suggestions, but they all involved things that weren't something that you could just do sitting at a computer and typing, like typing commands and getting responses. It was, you've got to get involved in a criminal community. None of them.
- [08:35]
B
Yeah, like there's, there is no way to type a command into a computer and have it give. Like it doesn't turn into money like that. You need to, you need to mine the resource that you've captured for something valuable and then sell that valuable thing.
- [08:48]
A
Yeah, Generally speaking, I mean.
- [08:50]
B
Right.
- [08:50]
A
To be pedantic about the report, there was a bitcoin wallet that he found but he wasn't able to crack it. So that is one example of where you could type a command. But in general, I think. Yeah, that's an edge case. Right?
- [09:06]
B
Yeah, I think that's, that's sort of like, that's a bonus thing that could happen. You wouldn't be hacking just to try and steal bitcoin wallets on the off chance that you find them.
- [09:15]
A
Yes, yeah, that's right. So perhaps the theory is that there's a small number of hackers out there who are hacking away, but the sort of hacks are non consequential in that they haven't been able to convert it into either full on ransomware or haven't been able to make a lot of money. And therefore there's no huge.
- [09:38]
B
They can't chop it up for parts and sell those. Right. These are more like joyriding teenagers than professional car thieves.
- [09:46]
A
Yes, exactly. Yeah, right, yeah. So if it was professionalized, if people were easily able to make money, there would probably be a whole lot more, I'm guessing.
- [09:54]
B
Yes. So I think we could say that the thing that these guys are missing is the entire support ecosystem of the cybercrime. Not the cybercrime community, but like a cybercrime community that can convert access into money via whatever process. If it's, you know, stealing and selling data, if it's extortion with, you know, ransomware, or threatening to leak data, if it's getting credit cards and then selling those, or doing your own thing with them, and then having mule accounts that would allow you to get the money sort of out from a lot of these other things as well. So there's this ecosystem which is a social system that you need to be involved in. It's not a technical thing, it's a community that you need to be involved in in order to be able to extract value from this resource that you can gain access to. If you can only get access to the resource, it doesn't get you anywhere
- [11:00]
A
because it's the gold miner who can get the gold but doesn't know how to sell it or there's no buyers.
- [11:08]
B
Exactly. So what's funny is this reminds me of a thing that was happening a while ago where people were confusing the ability to break into something with being an apt. It was the same sort of thing of like, I know how to hack, therefore I am the same thing as a ransomware operator. I am the same thing as an apt. And what's missing is that for those things that the hacking is just step zero. Right. This actually, at the time, it pissed me off so much that I wrote about it. So I wrote a blog post called Cyber Ignore the Pen Testers. And the idea was that, you know, what would be the minimal viable apt.
- [11:56]
A
Yep. Because we don't need that.
- [11:59]
B
Yeah. Oh, yeah. So let me send you a link. Oh, look at that, man. 2016. This is a. This is a decade one decade on. But yeah, it taps into the same thing of the. Like. So having all of these CVEs means that in theory you could do more hacking, and having these LLMs means in theory, you can do more hacking. So how come there's not more things that exploit hacking? And it's. I think it's the same answer as before that I was putting forward is that the being able to hack is not actually the same thing as being a criminal hacker. Like, there's this whole professional network, this ecosystem, these like all of this other stuff. You need to do it the same thing with intelligence. Like, you don't just hack into somewhere and be like, great, you know, now
- [12:52]
A
we've hacked them, job done,
- [12:55]
B
where's the intelligence? Right?
- [12:58]
A
I mean, one thing that struck me before, before we started this conversation, we were talking, the hypothetical I presented to you was, imagine you're an Iranian hacking group. You, your headquarters, your office has just been blown up. What would it take you to restart? Because we were talking about exploits and this sort of plethora of exploits. And pretty much the second thing you said was an LLM, which I thought was interesting because I like, one way of thinking about organizations is resources, priorities and people. And so the two hackers who were using LLMs to hack, they had the resources, the technical capabilities themselves, they had the priority to hack whatever they were interested in, but they don't seem to have had the processes to convert that into money for themselves. Like, based on what the reports say, maybe they did, maybe they didn't. And that because it's got that social element seemed to be the part that was missing. But like an LLM doesn't fit neatly into that model of resources priority. Well, it sort of fits into resources, doesn't it? Because it's like you've got more capabilities to do stuff.
- [14:16]
B
Yeah. So, I mean, when you put it to me, I think the first thing I said was like, you need, like, people, I guess. And then the second thing you need is an LLM. And my thinking is you've Lost all of your tooling, you've lost all your exploits, your implants, all of your infrastructure, everything. That's a lot of dev and maintenance work. It's not a lot of hacking. It's just you're going to need to write a web shell that doesn't get caught immediately. You're going to need to write a C2 server, you're going to need to write an implant. You're going to need to write. Not exploits, because I, I don't think that that's critical. But all of the stuff that no one ever thinks about, like your C2, your implant, your web shell, your, like just the stuff that you need and that's development work. It's not, it's not hacking. Like, you don't need the hacking skills. You need someone. I mean, there's some security skills, but basically what you need is a developer who can do a lot of work on a lot of sort of just basic kind of boring stuff. And to me that's an LLM. Like you need like an LLM manager or a coder as we might call it these days. And like an LLM. And I think, I mean you would want to have a local model for this, but I don't think it's necessary. Like, I think that you could get away with just, you know, a Claude max subscription or something or.
- [15:45]
A
Right, right. You say local just for the OPSEC reasons. Right, right.
- [15:50]
B
You just want a capable model and also continuity. Like you, you don't want the adversary to, to cut off your access and you have to start off with something else.
- [15:59]
A
Right? Yeah.
- [16:00]
B
Right. So there's the, the security secrecy side, but also the availability. So yeah, like if you, if you go to the things that I was listing, so the things that you need are hackers, developers and people to do operations. Right. So your hackers are the guys breaking in, but they also have to be the guys who go in every Monday at 9am and collect the, the emails for the week and make sure that everything's okay and test to see that there's no upgrades coming down the road that are going to get you evicted. And you know, like there's just a lot of maintenance work that goes into maintaining access. That has nothing to do with the collecting of the intel itself, also collecting the data, it's just maintaining access is a job and you need people to do that as well as servicing the collection. So you need those people and you could double them up and say these roles are done by the same person. Yeah, but that's just that's going to limit the amount that you can do because now the guy breaking in is also the guy making sure that the C2 server has been registered and paid for for the next month.
- [17:20]
A
Right, right. So right up the top you've got, is it a quote, People, ideas, hardware, in that order?
- [17:26]
B
Yes.
- [17:27]
A
So yeah, so I guess, yep. In like, I like that and I guess in this kind of. Is it a paradigm? Well, at least the quote, the LLM is helping your people just do more. And it may also be a piece of hardware, but it's just an enabler. So I guess in terms of a minimal apt, you could perhaps think that you could get away with fewer people.
- [17:52]
B
Right. So I mean the, the lean approach I had is very lean. So I suggested that, you know, you've got your operators, your developers and your system administrators. Like, you just need people to keep the lights on, on your side. So I would say that 2 is the absolute minimum, but 5 is probably more realistic in terms of you can have people quit, you can hire someone
- [18:17]
A
else, you can have people go on vacation, holidays, days. Yeah.
- [18:21]
B
Like you have just a little bit of coverage.
- [18:24]
A
So yes, from a government intelligence agency perspective. Yes.
- [18:29]
B
Right.
- [18:29]
A
That's very, very small.
- [18:31]
B
Five people is not a, is not an agency.
- [18:34]
A
No.
- [18:36]
B
Right. So anyway, like let's just say four because it's, you know, easy to go with. But you're also going to need people to do like all of the support work that goes into this.
- [18:46]
A
Right.
- [18:46]
B
Like if you're going to do a phishing campaign, someone needs to register the domains and you might as well have that person also registering the VPs, like paying for VPS servers and making sure that the C2 server is paid for and making sure that maybe your shell companies are being maintained. Like all of the administrative overhead of hacking needs to be managed by someone and that doesn't need to be one of your hackers. You could just hire someone to do administration stuff. But you're also going to need analysts if you're doing operations. You can't simply break into somewhere and be like, okay, well I'm sure the important data is here somewhere. Someone will find it. You have to find it.
- [19:31]
A
Right?
- [19:32]
B
Yeah.
- [19:32]
A
So one of those examples, the Mexican hacker, he, I presume it was, he, I don't know, they broke into a government department and they'd set up, I think it was OpenAI's model and they set up different Personas, you know. You are a world class intelligence analyst.
- [19:52]
B
Right? Right, Right, Yeah. And you are Tom Uren, world class
- [19:57]
A
intelligence analyst and they fed like reconnaissance data from the hack into that LLM and got it to generate little reports. And my sort of vibe was that it was quite good at. Here's a place we can go next.
- [20:14]
B
Yes.
- [20:14]
A
And less good at the. This is really significant and is a point of leverage that we can use to extort the company, which to me is not surprising because you don't have a lot of training data on. But yeah, that's just my vibe right now. I read it a couple of months ago. So this seems like there's potential for LLMs to speed up that kind of analytic process, but only speed it up. I don't think it would really do the.
- [20:50]
B
It makes the people that you have more efficient, but it doesn't replace them. And I think that that sort of. It's a subtle point, but it's easy to overlook. Like you can't get rid of your developer and replace them with an LLM. What you can do is you can take your developer and make them five times as good in turning out code that can work because they have an LLM. Right.
- [21:17]
A
I would phrase it cautiously and say produce five times or even ten times as much code.
- [21:23]
B
Right, Right. That's probably as fair as it. Yeah, that's about as far as you can take it.
- [21:29]
A
I guess it may be better, I
- [21:32]
B
don't know, depending on how good you are at getting LLMs to produce something that you can use, that output may or may not be useful to you.
- [21:40]
A
Yeah. I could believe that in a real intelligence organization you could use LLMs for quite effective triage. You know, what other unusual things, what are the interesting things? What's the most surprising thing, I think that would get you to gold or higher value intelligence quite quickly or more quickly than now?
- [22:05]
B
Yeah. And I think there's roles for LLMs in some of the more boring stuff that has to happen. So I think we were mentioning earlier things like you need to analyze the network that you're on for scheduled maintenance Windows or there's a pen test that's scheduled to come up. So you probably want to lay low during that period to make sure that the heightened vigilance doesn't catch you out by mistake. You want to have visibility into the sysadmin's discussion groups in case one of them goes like, huh, that's funny. Has anyone else noticed that the CPU is being pegged at max on this particular box all the time? I wonder what's causing that. You need someone doing that and an LLM is quite good at that. Because it doesn't necessarily get bored like a real analyst would like you could actually feed it all of the email traffic and say read this to see what you know, read this and build a calendar schedule of their planned maintenance, windows, software upgrades, deployment plans for new security software, blah, blah, blah. Like it would probably be great at that. And then you could use that to help plan your persistence mechanisms.
- [23:21]
A
I suppose that's quite a high consequence thing to rely on though. So to me there's a question about you would need to know that it's going to work most of the time, which it probably would nowadays, I guess.
- [23:32]
B
Yeah. So like, I don't know if I would trust it blindly to do that, but I think that convincing your operators to sit there and read massive email threads is going to be a hard task. Hiring analysts, like they're kind of expensive and you don't necessarily want them to be doing this important work, but it's sort of like maybe you could have them looking at what's in the database and is it actually useful.
- [24:03]
A
So so far what I'm hearing is the reason you mentioned an LLM second is just because it's super useful, not that because it's game changing. I mean maybe it's game changing in the sense that you can do a lot more, but it's not redefining the way that you do this kind of work.
- [24:20]
B
It saves time in places that you would if you're trying to do the like my team has been blown up and I need to replace the entire thing from scratch. I would say that five years ago or even three years ago I would have said okay, so like probably the first thing that you're going to want is you're going to want to hire some developers and they're going to go on GitHub and they're going to find a web shell POC and they're going to modify it so that it's like the signatures are different, that you're going to have them go and GitHub and find a open source C2 server and they're going to modify that so it works for you and then it implant and like all of this stuff. And to me that would have been the fastest way of doing it is the go get an open source version and tweak it a bit to try and make sure it's whatever. These days I'd say you could probably do from scratch development a lot faster with an LLM than you would have been able to do searching GitHub, finding something that still works Tweaking it in your own way, testing and all this stuff. You could still do that, of course. Like you could still get it from GitHub and then use the LLM to tweak it and that would be a lot faster. It's just I think that the ability to create custom tooling is useful for an apt.
- [25:34]
A
Yep, yep, yep. So we're talking about analysis. Translation seems like another area where having yes. LLMs would be useful again. And I suppose it's part and parcel of the analyzing a whole lot of data. I would actually expect that in a professional organization you would have people who are close to native language or very good at reading anyway. And so the LLM would be just part of the triage process. I suppose in this other foreign language, it's worth giving it to Jane because she speaks, you know, whatever language. Like this is part of the triage process. I think it's interesting. For whatever reason, it's between two very important people and they're talking about something particularly interesting. So here you go, Jane.
- [26:27]
B
Yeah, so I was going to say like, there's actually, there's a bunch of, like there's a bunch of translation work that comes up that if you've ever done PED testing in a foreign language, it is so frustrating. Right. So I've done that. Like I've. I had stuff like, you know, do a web app pen test on this Dutch company and it's like, I don't know what the Dutch for about us. Contact us is. So like finding just the contact form, you know, it was extremely painful. Right. So like there was Google Translate I think, but it was very primitive. Yeah, this might have even predated Google Translate. I mean this was long ago. To give you an idea of that, there's a tin can and a piece of string that I used to connect to the Internet back then. So what this sort of reminded me of was, I guess about 10 years ago, Phineas Fisher hacked the hacking team company. And I remember at the time thinking, you know, whoever Phineas Fisher is, they speak Italian. Right. Because in their write up about it they went on a lot of things about like how they did various technical things to gain access to do lateral traversal and you know, move through the network and stuff. But then they were like reading the emails and you said, yeah, so hacking
- [27:55]
A
team was an Italian surveillance where developer that Phineas Fisher had gripe with. Like they just didn't believe that that was a thing that companies should be doing. So they, they hacked them. Yeah, I remember it being Like a very readable write up and quite entertaining.
- [28:14]
B
Yeah, it was, it was very well put together sort of. I know at the time when I was reading it, there's just, there's a lot of things that struck me as the. You have to know Italian in order to. Well, I guess not necessarily like it's. If you were Italian this would be so much easier.
- [28:31]
A
Right.
- [28:31]
B
And if you had to translate like, oh, is this a possibly interesting subject line? Like you wouldn't. The thing that you would lose by not being Italian is you couldn't skim and immediately determine what would be interesting. You'd have to painstakingly go one by one by one through every single thing because you wouldn't know just right, like this is a high priority thing that they're talking about. I should do this first. So you wouldn't be able to do triage. And I'm absolutely positive that an LLM would help with that. Even if it's not perfect.
- [29:07]
A
Yes.
- [29:08]
B
I think it's a bit like a supercharger. It doesn't replace any of the people you have, but it just makes it a lot easier for them to do their jobs. I mean. Yeah, with an apt. There's a lot of other infrastructure that you need which I started getting into things like people think about like, oh, you're collecting data and that gets you to like you're pulling in gigabytes or terabytes of just data. You need to put it somewhere, you need to sort it, you need to have it tagged, where it's come from and it needs to be processed and analyzed by people who could then figure out which parts are useful. And there's going to be people doing analysis for different reasons. So there's going to be your intelligence analysts trying to convert that data into actionable intelligence, but there's also going to be your analysts who are looking over it to find out is there scheduled maintenance, is there a scheduled Red Team event, is there an upgrade coming, Are they deprecating a part of the network that we're using? Are they adding a new system that we're going to have to be prepared for as well as just looking for any indications that they're suddenly alert to something and maybe contacting a forensics company to see if there's an issue. You need people doing that stuff that's just part of your maintaining access analysis.
- [30:34]
A
Yeah.
- [30:35]
B
And they're going to need access to data as well. And they're going to need to know that they're looking at the data for company A and Not Company C. You can't just have a file share where you dump everything and people can just sort of like, troll through it to see the right stuff.
- [30:50]
A
Right? Yeah.
- [30:51]
B
So there's a huge amount of just internal infrastructure to manage your data that you're going to need. And that's got nothing to do with hacking. I think, to a degree. You still need that if you're doing ransomware. Like, you still need a way of doing portfolio management. You need to be able to say, here's all of the places that we've got access to, here's where they are in the. We've deployed, we have not yet deployed, we have started negotiating, we have ended negotiations or negotiations failed. These need to be punished. Like, you need to manage all of that. And that is software that you need and that needs to be patched and maintained and kept up to date and debugged when it breaks. And you're going to find out that it doesn't do exactly what you need and it's going to have features out. Like there's just so much extra stuff that's got nothing to do with the pointy bit of the spear.
- [31:44]
A
Right, right. And so, in fact, I guess, although there's some evidence that there's a lot more exploits around, there's very little evidence that there's more criminal business process management software.
- [32:01]
B
Thanks a lot, tom.
- [32:02]
A
Thanks, craig.