Loading summary
Transcript120 lines
- [00:04]
A
Hello, everyone, this is Tom Uren and I'm here with the Grok for another between two nerds. G'day, Grok. How are you?
- [00:10]
B
Good day, Tom. Fighting yourself?
- [00:12]
A
I'm well. Today's episode is brought to you by RAD Security, who make a cloud native security detection platform. So, grok, one of the things that floated past me in the last couple of days is that the U.S. government, the Office of the Director of National Intelligence, actually published a little bit of information about the US's vulnerabilities equities process. First of all, I'll explain what that is. And the idea is that NSA in particular, they look for bugs, and when they find bugs, they have this process where they weigh up the different, what they call equities involved between using them as an intelligence resource, you know, using them to basically hack people. And also how, how dangerous is it to, I guess, in particular, American interests? But how bad is the bug? What kind of damage could it do? So the idea is that they weigh.
- [01:18]
B
Up those two, plus I think they factor in sort of. And how likely is it that someone else is going to find it? Yeah. Is it sort of like, you know, our. Our secret quantum computer has discovered a thing versus, you know, we were fuzzing and in 10 seconds this fell out.
- [01:36]
A
Yeah, that's right. Yeah. So a number of countries have these VEP policies, I think the uk, the us, Australia, I'm sure others. And my broad take is that they're a good idea. You don't want those kinds of agencies just hanging on to bugs, regardless of the potential downsides.
- [01:59]
B
And I think that that's important because the default for any of these agencies is just not to say anything like they would hang onto them even though they're not using them, simply because it's in their nature not to reveal any information that they have.
- [02:15]
A
So the office of DNI published, like I said, just a little bit. Kim Zeta at her substack0day, has written up examining various different angles. Her substack is what brought it to my attention. So. Thanks, Kim.
- [02:32]
B
Same.
- [02:34]
A
But we thought we'd talk about what exactly was revealed in order to pad.
- [02:40]
B
Out the length of this show, because we could literally read the entire thing and then have another 19 minutes to go.
- [02:49]
A
Yeah, that's right. Pretty much, yeah. First of all, the ODNI has basically a legal obligation to produce a classified version of this report that is sent to Congress and also an unclassified appendix, which is what we get. So basically this. It basically says two things, 39 and 29. And so in fiscal year 2023, the aggregate number of vulnerabilities disclosed to vendors or the public pursuant to the VEP was 39. So 39 bugs in the whole of a fiscal year, which I think in the States is what, September to September or something like that? One year. Anyway. Of those disclosed, 29 of them were initial submissions. So I guess there's a year's worth of bug finding. Got you 29 bugs, and 10 of them were reconsiderations that originated in previous years. So that 10 is kind of interesting. But the problem I have.
- [04:07]
B
So now, now we have three data points so we can make a line. We've got 10, 29 and 30. So you could see that it's going up very, very sharply.
- [04:19]
A
So the problem I have with this is we know that 39 popped out, jumped the classification barrier to be released because they were, I guess, like hot potatoes. We don't want to hang onto them. Out you go. We don't know how many, you know, tens or hundreds or thousands of bugs that NSA just hoarded. Right, right.
- [04:43]
B
So they've got billions of them.
- [04:47]
A
We've got the numerator, but we've got. Don't have the denominator. Is that right?
- [04:53]
B
I don't know. I don't know. Grade school math. I'm in university, not grade school. Yeah. Look, I think it brings up a number of interesting things. One of them is that it puts an actual number on how many bugs come out of the vep, which I think has been an open question. Is the VEP a political. Does it just exist as a process to say that we have a process or does it actually ever release anything? Right, so yeah, it does release things. You know, it does say make these bugs public. So that's something. It also, I think the number, like it sounds kind of high. Like this is 29 bugs discovered in one year that were deemed either too dangerous or unnecessary.
- [05:42]
A
Does that sound high? Because I look occasionally.
- [05:45]
B
No. So this is where I was going at. It sounds a bit high if you're just saying like 29. Well, maybe it doesn't, but it's.
- [05:55]
A
I was thinking of the thousands of bugs. When you look at whatever those lists are and there's endless numbers, overwhelming numbers.
- [06:06]
B
It'S basically slightly less than the output of someone in the top 10 of the Microsoft Vulnerability Reporting Club, which is pretty much 80% in Chinese. So. So it seems like it's less than the public cohort does. So they're finding and reporting fewer bugs than people in public are and to me, part of that is probably because of focus. So, like the things that would be interesting to Microsoft, like, oh, we found some sort of authentication bypass in Azure that allows someone with this level of access to exceed it and get that level of access, which wouldn't be interesting for an intelligence agency because it's just not in. It's not something they would use, so they wouldn't ever look for it. Right. So I think there's, there's partly a focus angle, but I think the other thing is like it speaks to. They just don't need that very bugs overall. Like, I don't think they've got huge arsenals. Like when we saw the shadow broker stuff coming out, there were like five bugs for a platform and that was it.
- [07:21]
A
Right. I guess you would break down what you need to do to get access and it's what, you know, getting initial access and then privilege escalation and then. Well, I'm sure there's other stuff you need as well.
- [07:34]
B
Yeah, but, yeah, like that's it. That's like you need to get access and you need to get root. And then there might be two or three bugs involved in either one of those processes, but if you've got a thing that works, that's it.
- [07:49]
A
And then once you're there, it's all just clever techniques that get you from A to B without.
- [07:54]
B
Right. Like once you're in, you get authentic credentials. Like that's what you're trying to do is to be able to start impersonating people without leaving. Like, here's the thing, right? Exploits are not ideal. They crash, they leave logs, they make programs behave in weird ways. I mean, by definition. And so they're not the things that you want to use by default. You'll use them if you don't have a choice, but it's not your first option. It's the thing that you use because other options that would work nicely just aren't applicable.
- [08:34]
A
Right.
- [08:35]
B
But once you can, you try and move away from them. Right. So you gain access and then you get, you know, like, I might break into your box, but then I'm going to steal your authentication token so I can get access to your email without having to break into your box all the time.
- [08:49]
A
Yeah. So I was looking just this week at the state of ransomware and the way that some of the more common groups nowadays get access is just by buying passwords or guessing. And so, like, you know, if it's good enough for them, like they're surviving perfectly well. Leave aside government interventions that are trying to stomp on them. But the actual getting access part is not a problem.
- [09:19]
B
Right. Yeah. Now mind you, the bottleneck is not gaining access to systems.
- [09:26]
A
Governments have a much more, much more selective about their targets. So they're not target opportunities. So there will be times when they have no other choice.
- [09:38]
B
Right. And also there are requirements of operational security which don't apply to ransomware, which do apply to governments. Right. So a government could do a complicated phishing attack, but they don't necessarily want to actually have a large email trail of interactions with someone. Like they might feel that that's too high a risk, that the 1% chance that they might spook someone is too great a risk, given the importance of this particular person, that it's much better to go with something else that's actually, if it fails, at least it won't alert them.
- [10:17]
A
Yeah. So I guess it's the right tool for the job.
- [10:20]
B
I sort of like, I think of it a bit like it's, it's burglary. Right. Like if you're breaking into a house, maybe you need like a, a, a screwdriver or a crowbar. Right. But you're not going to show up with like a chainsaw, three screwdrivers, seven hammers, you know, a range of crowbars.
- [10:38]
A
Or you'll have one of those little glass cutting machines like they have in the movies, suction cups. I think that's the standard break in procedure.
- [10:48]
B
Yeah.
- [10:50]
A
So what do we make of the. Like I said, 39 disclosed, 29 were discovered in a year. So that kind of gives you a rate of bugs we found that we don't think we can hang on to because the reasons we spoke about at the beginning, that they're too easy to find, they're too dangerous. Well, I guess the danger is a combination of easy to find and powerful or something like that.
- [11:21]
B
They might also be not powerful enough. Right, right. Like you might find something and be like, oh, this is like it's bad, like it's a medium something, but it's not the sort of thing that we can use.
- [11:33]
A
So, so that's 29. And then there were 10 that were reconsiderations. So what do you think? Those 10, why are they. So Kim Zeta writes that the way it works is there's the VP board and if you decide to hang on to a vulnerability, it gets reassessed in a year unless it's been found to be exploited by someone else.
- [12:00]
B
Right.
- [12:00]
A
And then that's my impression was it's an automatic, like you Kick it out. If it's being exploited. Yeah.
- [12:07]
B
It's treated urgently through a different pipeline would be my. From. From the way she phrased it and the way I would assume that it runs is if you're using something, you find out the Russians are using it, you remove their capability by reporting it.
- [12:24]
A
Yep.
- [12:25]
B
Even though it might damage your capability as well. The damage it does to them is more satisfying than the table does to you. The spite that you get out of it, like, at least I'm screwing them over.
- [12:40]
A
Right. Yeah, I guess it's kind of like you're running ahead of them and you're burning the bridges behind them. Is the kind of dynamic, bit of a tortured metal there.
- [12:52]
B
So 10 don't fall into that category, is the impression. Because they did go through the full VEP process a year later. Right. They don't fall into the. Like someone else found them and is exploiting them.
- [13:04]
A
I think it's right. No, I think that's unclear because it says in the fiscal year this number was released. So it. At one extreme, we have 10 of those that were found by other criminal hackers or adversaries.
- [13:21]
B
Yeah, yeah. That is, there's 10 collisions or 10 that became surplus to requirements. Okay.
- [13:32]
A
And so surplus to requirements. What are you. What's your thought process there?
- [13:36]
B
Yeah. So, like, the way I'd see it is this sort of one would be, okay, we find someone and he's using some obscure system that we will now do a special investigation on, find bugs, attack that system, do whatever we need to do, and then that ceases to be a case or a priority for us. So we no longer need that obscure bug. And so we can basically shelve that and we're not going to need it again. So we can turn it in. Like, no one's going to fight to keep it available. Right. The other one would be you've got one bug that you're using and. And then you find something that's better and so you switch to that. And now you've got this old bug which no one else is using, but you're not using either. So you look at it and you're like, well, we don't need to keep it on the shelf because we've got this thing that's like. We've got this process that's developing new stuff. We've got this thing that we're actually using and we've got this old one that's just, like. It's difficult, it crashes, whatever. Like, it's just not particularly good. That One we can turn over, particularly if it's in a component that's separate from the one that you're now exploiting. So it's unlikely to lead to a patch that kills what you're using.
- [14:53]
A
So you replaced it with a kind of novel different, navigates different path through the software or whatever to get you what you want.
- [15:02]
B
Yeah, you've got something that was exploiting Safari and then you find a thing that exploits it differently and doesn't need, you know, five bugs chained and.
- [15:12]
A
Yeah, you know, so we've got, we've got two hypotheses of where these bugs are. One is collisions, where someone else has discovered the same or very similar. And we've got your hypothesis that it's obsolescence. They've been replaced, they're surplus to requirements now. Now it occurs to me that if your VEP is working well, the whole point of retaining bugs is because you think that they'll be difficult to discover and exploit. And so if it's working well, we should be in the world where those 10 are all the ones that are surplus to requirements.
- [15:53]
B
Right?
- [15:54]
A
Do you buy that?
- [15:56]
B
That seems like a stretch. Like it just on the other hand, I gotta say, I think 10 collisions seems unlikely as well, because when we had the shadow brokers leak, like the bugs that came out, those weren't things that other people were finding. Like EternalBlue was not something that was on anyone's radar.
- [16:23]
A
I seem to remember it had been used for, was it a decade or something? It was some extremely long period of time.
- [16:30]
B
So EternalBlue was this, you know, like this SMB based exploit that basically gave remote system access without authentication, without interaction from anywhere in the Internet. Like it was a, it was a God bug. It was great. And they were using it forever, like from when they found it until when it got patched was, you know, I'm not sure if it was a decade, but it was pretty close. It was, you know, it was a thing that they just had and they were using it for year after year after year after year after year. And even when it got patched, right, like they reported it to Microsoft, Microsoft fixed it, pushed the patch out. They were silent about the urgency. I think, like I seem to remember them saying, you know, dispatch comes out, it's got, you know, like IE fixes and some SMB fixes and some this fixes. And as always, we recommend that you patch as soon as possible. Right. There was, I think, a month or two where nothing happened. And by that I mean no one came out and said, hey, that's weird. Looking at that patch, this seems like a major bug that would have affected all versions of SMB for the last 10 years that would have given anyone remote access. This was huge. So. So no one was even looking at that once it was patched. Right. Like, it was just not a thing. It only became interesting when the exploit came out. It seems to me that the number of things that were necessary to exploit it was so difficult to do that it wasn't apparent even from the patching what had been fixed.
- [18:07]
A
Right, right, right. If that makes sense.
- [18:09]
B
It wouldn't have been found by anyone else. Like, they could have used it for another 10 years.
- [18:13]
A
Yeah, yeah, yeah. So that's an argument. Like, it sounds to me like that's an endorsement of the VP process, you know, if it existed back then, in that the bug they kept was, you know, it sort of fits the criteria of hard to exploit. Very, very useful for them, but unlikely to be discovered. I mean. Well, the evidence is it was never discovered by anyone else.
- [18:39]
B
It was. Yeah, as I recall. I think there was some. There was some notice of its use after the patch came out from somewhere, but that's very likely. It was just. They were given it by shadow brokers beforehand.
- [18:55]
A
Right. I think that was Chinese or something, wasn't it?
- [18:57]
B
Yeah, that's. That was my impression. That is, it seems very likely that before these things were actually leaked publicly by shadow brokers, they were probably traded by intelligence partners because they would just be valuable like that.
- [19:11]
A
And I guess they knew the patch was coming, so you may as well get a few brownie points for a short time.
- [19:19]
B
Exactly.
- [19:20]
A
Yeah. So I guess we've talked about how Eternal Blue was a wonderful bug from an intelligence agency's point of view, and I guess that that's what they're looking for. Right? They're looking for those one in a three thousand or one in a hundred thousand or one in a million bugs that are that perfect combination of really useful but also really hard to discover. Now, thinking about how we've talked about the VEP and what we know about the bugs that were inadvertently released or leaked, that makes me kind of think that maybe that 10 is. Here are 10 bugs we find every year that are just super duper good and better than the ones we've got before.
- [20:08]
B
Yeah, it could be. It could be that, you know, the version two comes out now, it's got chrome and tail fins, and you need to get rid of your old ones. Yeah, I mean, I wouldn't be that surprised, but the other thing is maybe those 10 don't represent one. Like maybe, maybe that 10 isn't all from last year. Right. Maybe it's the first one that was submitted several years earlier and has gone through repeatedly and now this year it was selected.
- [20:38]
A
Yeah, it just says originated in prior years. So yeah, years could be 15 year old bugs that are.
- [20:46]
B
We're finally killing all of our Sunos 2.1 back catalog.
- [20:54]
A
I guess that's the other possibility. Right. Perhaps Windows XP is now old enough.
- [21:01]
B
Windows for work groups has finally been retired from. Yeah, I mean that's possible. So I think that it's also kind of funny that they say like how many of these bugs were patched? We don't know because we don't track it.
- [21:21]
A
You just throw it over the fence and eh, whatever.
- [21:24]
B
Yeah, it's your problem now.
- [21:29]
A
So in principle I'm a fan of transparency. So like just because I think in.
- [21:35]
B
Theory this is a good thing.
- [21:36]
A
There's too much secrecy becomes a default and so there's too much of it.
- [21:41]
B
Right.
- [21:41]
A
I like the idea of having these kinds of reports that are unclassified and that we can podcast about and you know, try and read the tea leaves.
- [21:52]
B
Yeah, we like getting that glimpse behind the curtain, even if it's just sort of slightly underneath the curtain and we can see someone's toes.
- [22:00]
A
But does this even tell us anything? Because we've come up with I guess, speculation upon speculation based on what we know.
- [22:09]
B
We've got one number and then it's sort of, it's subdivided into two other numbers and somehow this is supposed to tell us something useful.
- [22:18]
A
Yeah, so I was actually thinking that the real proof of the pudding is how that agency operates. So it's nice to know that it's got a vulnerabilities equities process. I think that's good that we know that. But does the number make any difference at all? Because what we really want is not to.
- [22:40]
B
Of our 200,000 bugs that we found this year, we are turning over 39 or sorry, we're turning over 29.
- [22:49]
A
Well, I mean not even that. I think it's the. We don't want to discover that there have been planet melting bugs that NSA held onto. We don't want to learn that it's gone, that they're holding onto too much. Do you know what I mean?
- [23:06]
B
So I disagree actually in that I think it's fair for them to. So if they find something that sort of meets their criteria and that it's hard to discover, it's super Useful, you know, all of these things. I think if they hold onto it because no one else has it right. Like they don't see it's being used anywhere else and they have that visibility, I think it's fair that they don't report it because it's not a vulnerability until it becomes known to someone who's abusing it. So I personally think, for example, that with EternalBlue for the, let's say, 10 years that it was kept by NSA during that 10 years, the industry was not mature enough and organizations were not mature enough to be able to push out a patch, adopt a patch and get it widely deployed. I think that if they had released it earlier, it would have been wannacry every year for a decade. I seriously do think that that's what would have happened. It would have been a very bad bug released into an environment that was unable to defend itself because it simply lacked the tooling and the maturity and the like, the practitioners to deal with it. It would have been something very bad for a long time. That's my, like, that's my takeaway on that because even in 2017 when it was released, there was still, you know, NHS hadn't patched months after this patch came out.
- [24:46]
A
Right.
- [24:46]
B
Yeah, right. You know, it still did huge damage to a lot of places, even at that later stage where there was automatic patch deployment, where there was, you know, this, you know, you had your patch Tuesday, you had like all of these processes in place of dealing with these things and it was still a huge deal. If it had happened any earlier, it would have been even worse. That's my. Yeah, that's my opinion on it.
- [25:15]
A
What I was trying to say is that you want to know that they're making those decisions correctly with the knowledge that they have.
- [25:24]
B
Right.
- [25:24]
A
So but they're not doing as a.
- [25:27]
B
Knee jerk reaction of just keep it because we, we found it, it's ours, no need to tell anyone.
- [25:34]
A
Yeah. So that the VEP is actually working, like that's the proof of the pudding. And so I mean we haven't had those sorts of planet melting bugs since then, have we?
- [25:46]
B
No, I think that was the last, like that was the last worm or the last public worm that destroyed the Internet.
- [25:53]
A
Yeah. So I guess my point is the proof is in the pudding rather than the recipe or the numbers that make up the pudding. And so I guess necessarily with intelligence services there's this level of trust where you have to trust that the oversight processes are working.
- [26:13]
B
Right.
- [26:13]
A
And shrug. I guess.
- [26:16]
B
Yeah.
- [26:16]
A
After having spoken about it for 20 odd minutes, it's like, ah, it's pointless.
- [26:22]
B
So, I mean, I. Here's the thing is, I think it would be interesting if we had 10 years of data that we could look at. You know, this year it was 39, the year before it was 20, the year before that it was 52. You could start seeing some sort of trends or patterns, but not confident that we're going to get any more, at least for the next four years of these reports.
- [26:45]
A
Yeah, so when Kim wrote about it, she pointed out that there should have been several reports. I think if I remember correctly, she says we should have had these unclassified appendixes, appendices available since 2018. I want to say, anyway, several years and they've just basically never turned up. So. So what you're saying is that we should absolutely force the DNI to go back and give us the data for 2018, 29, and then we can podcast it about it again.
- [27:17]
B
That's. That's another month of content right there.
- [27:20]
A
Come on.
- [27:23]
B
So my feeling is that the VEP works for some definition of works. Right? Like, is it perfect in that, you know, the only bugs that they're retained are these super cool ones that no one else finds that, etc. I don't think that's true, but I.
- [27:42]
A
I do think that can't be true, can it?
- [27:45]
B
Right, right.
- [27:46]
A
Because, like, you know, if some person found it, some other person can find.
- [27:50]
B
It, like, yeah, they have a 100% success rate that obviously not.
- [27:56]
A
I mean, I think there's a very strong selection effect in that you're going to look for bugs that are important for what you do. You're retreading the same ground as. This is a terrible analogy. As the landscape shifts underneath you as things change.
- [28:14]
B
Right, right.
- [28:16]
A
Like you want to get to the same destination, you want access to.
- [28:19]
B
You've got. Here we go.
- [28:21]
A
Android Science.
- [28:22]
B
It's a whole. The map is not the territory. So you've got a map that tells you where you need to go, but the territory keeps changing in its particulars. And so you're going to keep going.
- [28:33]
A
Over and over, which is, yeah, so you're following the same routes on the map, but the. You need new capabilities over time as things change, you know, whatever.
- [28:43]
B
Right. As you know, one of your vendors gets busted because they sold to people who do bad things. And so now that trick that you used successfully for two years just doesn't work. And you can either. You're going to be in a situation where you either say, okay, well, until we get another one, we're just not going to be able to collect and do our jobs. Or you can say, well, here's one I prepared earlier and swap over to that. And it's sort of. How many of those that you prepared earlier do you need to have in reserve?
- [29:15]
A
Right, yes. So that's, that's kind of what I'm thinking, is that because you're retreading the same path or getting to the same destination, there's quite a lot of. Is it duplicate discovery? That's not quite the right word, but there's sort of redundancy, redundant discoveries or.
- [29:33]
B
Yeah, there we go. It's. You're going to have a lot of ways of skinning the cat, but they're mostly going to be the same cat. And it's. Thanks a lot.
- [29:49]
A
Thanks, Rock.
- [29:50]
B
There we go.
- [29:51]
A
Tortured analogies and cats.
- [29:55]
B
Cats.