Risky Business #771 -- Palo Alto's firewall 0days are very, very stupid

Summary

Risky Business Episode #771: "Palo Alto's Firewall 0days Are Very, Very Stupid"

Release Date: November 20, 2024
Host: Patrick Gray
Guests: Adam Boileau, Andrew Morris
Sponsor: GreyNoise

---

Microsoft's New Security Enhancements

The episode kicks off with a deep dive into Microsoft's latest security initiatives, aimed at addressing vulnerabilities exposed by incidents like the CrowdStrike breach in July. Patrick Gray and Adam Boileau discuss Microsoft's announcement of a remote recovery feature that allows systems to roll back changes if the kernel becomes non-functional. This feature is intended to minimize downtime and reduce the need for physical interventions across millions of Windows machines.

Patrick Gray ([00:03]) notes, "Microsoft is trying to solve the problem that we all want to solve, which is having to roll truck in the event of an outage and physically put hands on keyboards is a thing that isn't so useful anymore."

Adam adds, "There's a bunch of moving parts here that we do need to see specific stuff like how does this interact with BitLocker, how does this interact with TPM backed BitLocker, et cetera." ([01:36]) He underscores the complexity of integrating these new features seamlessly with existing security measures.

Palo Alto's Critical Zero-Day Vulnerabilities

A major focus of the episode is the alarming discovery of multiple zero-day vulnerabilities in Palo Alto Networks' firewall products. Adam Boileau outlines two significant bugs affecting Palo Alto's management interfaces:

1. Authentication Bypass (Auth Bypass): "You send an HTTP request header to the web server of the management interface which basically just says hey, don't worry about authing me, it's fine," Adam explains ([13:50]).

2. OS Command Injection: Exploiting the username field to execute shell commands as root, Adam remarks, "What are you doing Palo Alto? This is the firewall, it's a security product from a major vendor with no auth please header." ([13:50])

These vulnerabilities allow attackers to gain unauthorized access and execute arbitrary commands, posing severe risks to network security.

Access Control Challenges in Enterprise and IoT Devices

The discussion shifts to the broader issue of access control beyond mere authentication, especially concerning vulnerable technologies like IP cameras. Patrick Gray highlights the limitations of existing solutions such as Cloudflare and Zscaler, emphasizing the need for more robust access control mechanisms.

Patrick Gray ([15:23]) states, "The stuff that really needs to be access controlled is not often the stuff that is already getting the most attention."

Adam concurs, "Access control is complicated in enterprise context because you've got to have federated AUTH and SAML and all sorts of complicated authentication things." ([16:07]) He points out the difficulty in implementing strong access controls on devices not designed for enterprise-grade security.

US Government Cybersecurity Landscape

The episode touches on recent developments within the US government’s cybersecurity framework:

• Biden and Xi Jinping's Call: Concerns were raised about ongoing cyberattacks targeting civilian critical infrastructure.

• CISA Leadership Changes: Jen Easterly's upcoming departure from CISA is discussed, with anticipation about the agency's future under the new administration.

• Ransomware Trends: Adam notes a slight decline in ransomware attacks but warns that the threat remains persistent. He mentions the emergence of lower-tier ransomware groups like Akira, suggesting that while major cartels face increased scrutiny, smaller actors continue to pose risks.

Patrick Gray ([22:09]) remarks on the efforts against ransomware, "Suppression efforts have at least done something. You're never going to get away, you're never going to completely remove ransomware as a threat."

Legal Proceedings and Cybercrime Enforcement

The episode reviews recent legal actions against individuals involved in cybercrime:

• Heather Morgan: Sentenced to 18 months for laundering $10.8 billion in Bitcoin stolen from Bitfinex in 2016.

• Larry Dean Harmon: Convicted for operating the Helix cryptocurrency mixer, Harmon faces a three-year sentence and is required to forfeit $311 million alongside other assets valued at $400 million.

Adam highlights the significance of these cases, "It's good that they are tracking down, finding the people who run the mixes and run the money laundering because that's the kind of lubricant that makes bitcoin viable and other cryptocurrencies viable as a method for doing crime." ([33:31])

Innovative Anti-Scam Measures: Daisy AI by Virgin's O2

Patrick Gray introduces Daisy, an AI-powered system developed by Virgin's O2 in the UK to combat scam calls. Daisy mimics a grandmother, engaging scammers in lengthy conversations to waste their time and frustrate their efforts. The system can keep scammers on the line for up to 40 minutes, effectively reducing the volume of scam attempts reaching legitimate users.

Patrick Gray ([36:10]) enthuses, "It's just a regular speech to text LLM style thing, text back to speech, but then runs through a sort of personality layer that adds the grandma ness."

Adam praises the innovation, stating, "This is so cool. And they actually got one of the guys that does scam baiting to help build the model." ([35:58])

Academic Insights on Phishing Training Efficacy

A noteworthy segment covers a study by the University of Chicago and UC San Diego, which claims that typical phishing training programs are ineffective. The research, involving nearly 19,000 employees in the healthcare sector, found no significant improvement in reducing phishing click-through rates.

Patrick reflects, "It's nice to have data that says actually this stuff really doesn't work because as a practitioner I feel like it doesn't work." ([38:40])

Adam concurs, emphasizing the need for security programs that don't solely rely on user behavior, "Our job is to make it safe not to tell them not to click on things." ([39:50])

Facial Recognition Controversy: Bunnings Case in Australia

The episode explores the controversy surrounding Bunnings, Australia's Home Depot equivalent, which implemented facial recognition technology to enhance staff safety. The Australian Information Commissioner ruled against Bunnings, citing privacy violations. Despite the backlash, Patrick contends that the technology was used proportionately to protect employees from threats such as naked individuals with shotguns or violent intruders.

Adam Morris ([42:46]) comments, "It seemed like they actually thought about the data retention, about the things that they were doing and that it seemed a lot more reasonable."

GreyNoise Sponsor Segment: Uncovering Mass Scanning and Zero-Day Exploits

In the sponsored segment, Andrew Morris, founder of GreyNoise, discusses the pervasive issue of mass scanning and the exploitation of zero-day vulnerabilities, particularly in edge devices like IP cameras and firewalls. He reveals that GreyNoise's latest AI-driven analysis engine, Sift, identified ongoing exploitation attempts targeting these devices globally.

Andrew Morris ([55:45]) explains, "We ended up figuring out exactly what device it was targeting and diagnosing it... there is a zero-day."

Morris emphasizes the relentless nature of these attacks, "It's constantly, constantly happening." ([46:43]) He also highlights the challenges in securing legacy systems and the resurgence of outdated attack methodologies reminiscent of the 1990s.

Patrick Gray ([49:15]) suggests, "Allow listing of some kind is the solution here," advocating for more restrictive access controls to mitigate these pervasive threats.

---

Conclusion

Episode #771 of Risky Business offers a comprehensive overview of critical cybersecurity issues, from emerging vulnerabilities in major security products to innovative defenses against cyber threats. The discussions underscore the evolving landscape of cyber threats and the relentless efforts required to safeguard digital infrastructure. With insights from industry experts and real-world case studies, Patrick Gray and Adam Boileau provide valuable perspectives for information security professionals navigating these challenges.

---

Notable Quotes:

• Patrick Gray ([00:03]): "Microsoft is trying to solve the problem that we all want to solve, which is having to roll truck in the event of an outage and physically put hands on keyboards is a thing that isn't so useful anymore."

• Adam Boileau ([13:50]): "What are you doing Palo Alto? This is the firewall, it's a security product from a major vendor with no auth please header."

• Adam Boileau ([33:31]): "It's good that they are tracking down, finding the people who run the mixes and run the money laundering because that's the kind of lubricant that makes bitcoin viable and other cryptocurrencies viable as a method for doing crime."

• Patrick Gray ([36:10]): "It's just a regular speech to text LLM style thing, text back to speech, but then runs through a sort of personality layer that adds the grandma ness."

• Patrick Gray ([38:40]): "It's nice to have data that says actually this stuff really doesn't work because as a practitioner I feel like it doesn't work."

---

This detailed summary captures the essential discussions, insights, and conclusions from Risky Business Episode #771, providing a comprehensive overview for those who haven't tuned in.

Transcript

[Patrick Gray]

Hey everyone, and welcome to another edition of Risky Business. My name is Patrick Gray. We'll be chatting with Adam Boileau about all of the week's security news in just a minute. And then we'll be hearing from this week's sponsor, which is Gray Noise. And we're chatting with Gray Noise founder Andrew Morris about something that's quite topical, which is the amount of attacks against edge devices these days on the Internet. And as bad as it seems, and indeed we're talking about a few of these sorts of attacks in this week's news segment. As bad as it seems, Andrew is here to reassure us that it is in fact much, much worse than people fully realize. We're also going to talk to him about how Gray Noises LLM driven analysis engine actually caught an O day command injection vulnerability being used in the wild against a bunch of IP cameras. It's a really good interview. We always love having Andrew on the show, so do stick around for that one. But Adam, it's time to get into the news now and we're going to start off with some with this blog post from Microsoft where they've outlined outlined a few things that look to be in response to the CrowdStrike incident back in July. One feature they've announced is a sort of remote recovery feature, so if your kernel all of a sudden becomes non functional, you can actually roll back some changes. And the other thing they've announced is that they're going to introduce features into the OS that will allow security companies to build things like EDR without use a kernel module. Let's start with the recovery stuff like do we actually have any details on how this thing's going to work?

[Adam Boileau]

No, we haven't seen details yet. Microsoft's having their Ignite conference at the most they've been announcing a whole grab bag of stuff, but I guess we're going to have to wait and see the specifics. Essentially the problem they are trying to solve, as you said, is the kind of CrowdStrike scenario where your machines are rendered unbootable and clearly there's going to be some kind of like network aware safe mode that they can boot up into and then apply security updates or Windows updates or patches or something that administrators can use rather than having to physically go to individual machines, boot them up and so on. And you know, there's a bunch of moving parts here that we do need to see specific stuff like how does this interact with BitLocker, how does this interact with TPM backed BitLocker, et cetera, et cetera. But clearly they're trying to solve the problem that we all want solve, which is having to roll truck in the event of an outage and physically put hands on keyboards is a thing that isn't so useful anymore when we have a billion Windows machines in the world or whatever it is.

[Patrick Gray]

Yeah, I mean, we saw what that looks like back in July and it meant a lot of people stranded at airports and so on. Right. So not so surprised to see Microsoft trying to get on top of that. But the other thing they've announced as part of this, I mean there's a whole bunch of stuff in this blog post. We've linked through to it in this week's Show Notes. So who want to read the whole thing can go and do so. But another thing that caught my eye is that they are, you know, introducing these features which allow people to, you know, run security software outside of kernel mode. We're going to really have to see what that looks like because there are good reasons for security software to be in the kernel. And you talk to anyone who develops Endpoint security software for Mac via its API and they'll tell you all about the limitations. They will not stop talking about the limitations. In fact, your ears will fall off from how much they talk about the limitations involved in using that API.

[Adam Boileau]

Yeah, yeah. There is absolutely a set of trade offs that you have to make here for having standardized interfaces and the impact that has on the ability to kind of innovate or to provide security solutions that are differentiated from competitors and so on. And I think the comparison with the Apple ecosystem is pretty apt because Apple provides these APIs that people can use and if you don't want to do it Apple's way, then tough. But on the other hand, they also provide an ecosystem that is much more locked down and controlled than the traditional PC world. And we have ended up with these kind of Windows Antivirus and EDR and whatever else solutions because that platform is so much more flexible. The security software needs to be more flexible too. And so the idea that you could do it kind of Apple style or Even more like iOS style on Windows, the platform itself would need to be pretty different. And we only have to look at how like the success of the Microsoft Store and what a mess that's turned into to kind of see that it's a whole ecosystem. But anyway, I'm super interested. Microsoft is going in the right direction here. But you know, it's not as simple as just now we've got some APIs for building EDR in user mode well.

[Patrick Gray]

And I don't think they're going to cut off kernel access. Well, let's put it this way. I would be surprised if they cut off kernel access because there's going to be legal challenges there. Right. Because if they're still allowing defender access via the kernel, they kind of have to offer that to everybody else, otherwise they're going to face a whole bunch of legal challenges. So the precise phrasing they've used here is to help our customers and partners increase resilience. We are developing new Windows capabilities that will allow security product developers to build their products outside of kernel mode. Now this doesn't suggest, this phrasing, doesn't suggest that they're going to force people to do that, just that it's an option. And maybe Microsoft's hope is that customers will start demanding that security providers do that. I'd be pretty skeptical there because if I'm CrowdStrike or Sentinel One or whoever in a customer meeting, I've got, well, we got our kernel mode one which can do this and then we've got the user, you know, the API based one which can do a whole bunch less, you know, which would you prefer? And you know, when they've got the remote recovery feature as well as a bit of a seat belt, I think it'll be hard to convince people to pick like these versions, let's put it that way.

[Adam Boileau]

So yeah, I mean, I guess what we will actually see likely is that some of the functionality will be able to be moved out and everything you can move out of kernel space into a user space process is an improvement and maybe, you know, handling updates or maybe there's some rules around, you know, much like Apple has about like how much compute time you can spend, you know, in kernel, whatever else, you know.

[Patrick Gray]

Yeah, but that just, that's, that that gets a bit dicey, right, because what Apple will do is just shut down a security tool that it thinks is using a few too many cycles. It's just like bye bye, boom and it goes away.

[Adam Boileau]

Yeah, yeah. So like, you know, yeah, there are a bunch of trade offs but I mean ultimately the goal is to just do less stuff in a privileged concep context. And you know, if Microsoft can provide support, that means you can move, you know, 60% of your EDR product out of kernel. Like that's still 60% less that you can have bugs in.

[Patrick Gray]

So 100%, 100% move in the right direction. Devil in the details, but so far so good. Staying with Microsoft and they've announced that they're going to start publishing machine readable vulnerability information, which is a great idea. And frankly, like, I'm surprised this hasn't happened already.

[Adam Boileau]

Yeah. So they're using a standard mechanism for publishing them and essentially it's JSON files with a schema that's specified by a standards body. And I had a look through and it all looks pretty sensible. And given how many people have had to write things that scrape vuln information off Microsoft's web pages and then try.

[Patrick Gray]

To munge it into that JSON.

[Adam Boileau]

Yes, yeah, try and build this. And I mean, clearly Microsoft is sharing the data sources shared behind the scenes because you can kind of see artifacts of the web, part of it, you know, in some of the JSON documents they provide. But hey, this is a move in the right direction and anything that makes our lives easier in terms of consuming this data is great. It doesn't necessarily solve the problem that garbage in, garbage out. Right. You still have to have good quality information. And I feel overall the incentives for providing good quality public vulnerability information have been going down over time. But Microsoft's stuff is at least better than average, so.

[Patrick Gray]

Yeah, but I mean, you said it. It's the motivation, you know, it's not required to the same degree. Right. People don't have time these days to sit down and look at detailed information on each bug. They just, they want to know the CVSS score and then what to do. Right. Like that's sort of where we are now.

[Adam Boileau]

Yeah, yeah, I guess I'm, you know, for people like me that worked, you know, where you want to know the details of. Because you want to reproduce it and then use it, which is a slightly different use case than the problem they're trying to solve. You know, it's, you know, it's become frustrating more and more over the years to consume vulnerability information for anything other than that.

[Patrick Gray]

Yeah.

[Adam Boileau]

Like just tell me the CVSS and.

[Patrick Gray]

Patch it because so, so your objection here is it might make pen testers sad.

[Adam Boileau]

Yes, yes.

[Patrick Gray]

Okay. Right.

[Adam Boileau]

And pen test. Pen testers are already sad, so it's, you know, marginal difference.

[Patrick Gray]

But why we can't have sad pen testers. Let's immediately protest this, this development. Now, look, staying with vulnerability information, and everybody knows that nist, its efforts to manage the National Vulnerability Database kind of just fell over this year. They have now come back and said, good news everyone, which was we've cleared the backlog of publicly exploitable vulnerabilities. Right. So they've now gone and enriched the data for like stuff on the SISA Kev list. And they're like, yay, that's great. But their goal of end of year, like getting everything back to speed, they're like, yeah, that's not going to happen. I mean, it's amazing that they just fell by. This is an important data source and it just fell over. And it's, it's kind of. I mean, it has got headlines, but this is just a pretty epic failure when you think about it.

[Adam Boileau]

Yeah, yeah, it really is. And I'm, I am surprised that they let it get this bad. I. The articles about this though. And like some of the headlines we've seen about this, you know, because the. For example, the record runs a piece that starts the federal body in charge of processing prominent vulnerabilities said a back of unanalyzed exploited bugs has been cleared. Which you and I know means the hundred bugs on the Kev list for this year, not the 100, whatever thousand or 18,000, whatever it is that they haven't dealt with this year, like tens of thousands. So, like, it's a very, very small percent. Okay. It's an important, very small percent. But yeah, like it's, it is weird that, you know, such an important standards body has just screwed up this band.

[Patrick Gray]

I dropped the ball. But I mean, I'm guessing there's a, there's a backstory there, right? Like, I'm guessing there were people in there who are warning about it and I don't know, the right people didn't take it seriously. Like there's, you know, a story like this isn't just, oh, there are a bunch of useless idiots. You know, like there's, there's obviously some, some stuff went down is what I'm getting at. Right? Like some stuff went down. I don't want to criticize that, that reporting. By the way, this one's from John Greg at the Record because it, you know, he does present all of that.

[Adam Boileau]

Context and it is what Nist said. So like he's not, you know, he's reporting what they said.

[Patrick Gray]

Well, I mean, it's right there in the headline that the end of year goal for the full list is unlikely.

[Adam Boileau]

Right.

[Patrick Gray]

So it's not like that was not, did not feature in that story. But look, staying on volumes, I mean, you're going to have to unpick all of this for me because I'm, I've got so many CVE numbers in front of me that it's making me a little bit dizzy and I can't keep track of them all. But Palo Alto Networks is having a bad time. So we've had zero day exploded in the wild. Like proper zero day pre patch. There's bugs in multiple pan products. It's chaos out there man. Like walk us through, walk us through what's going on here because there's two sets of bugs, some affecting like what one of their management products and the other one, the management interface to their products. So what can you tell us about this?

[Adam Boileau]

Yeah, so you're right, essentially two sets of Palo Alto bugs that are in the news. One is extremely funny and quite important and one is really not that significant. So the not significant one is their like customer migration tool that I think we talked about that a while ago and I had some bugs in it. People have been looking at it and have found more bugs, but very few people run this particular product. The one that is great is there was a bug that was being sold on some hacking forums somewhere that was advertised as like pre auth remote code exec in Palo Alto's firewalls. And we saw Palo Alto warn about this before they even understood what the details of the bugs were. They said hey, someone is selling this bug, I guess maybe firewall, the management interface for your firewalls. And then now we've seen the details of that particular bug which turned out to be two different bugs come out. Watchtower Labs have a write up of it and it is such a clanger of a bug. So there are two. First bug is essentially an auth bypass. And the auth bypass is you send an HTTP request header to the web server of the management interface which basically just says hey, don't worry about authing me, it's fine. Yeah, and that works. And then once you've got that, once you pass that auth step you can go forward and look for the next bug which turns out to be a straight up shell metacharacter command injection in like the username field. So you put shell meta characters in it and it runs commands as root on the underlying firewalls os which again, oh my God, it's the year 2024. What are you doing Palo Alto? So I mean I guess if you have the firewall management interface on the Internet, you were going to have a bad time. You've always been going to have a bad time. But I mean this is the firewall, it's a security product from a major vendor with no auth please header. And I would like to run this shell command in my username please.

[Patrick Gray]

It's really funny, I've been spending a Lot of time with my head in access control. I think for a long time people made the mistake of thinking that authentication was access control. It is, but not when you're dealing with vulnerable technologies that you can pop shell on pre auth. Right. And that's one of the issues. And you look at what people have built in terms of trying to deal with these sorts of problems and when it comes to like enterprise web applications, you've got some reasonable options there. So you've got your sort of Cloudflare, excuse me, Cloudflare stuff, you've got Zscaler. You know, there are options for doing web application delivery when it comes to controlling access into production environments, whether that's database access, ssh, whatever. You've also got some options there. Tools like strong DM are really good, but I feel like a lot of this has kind of missed the point, which is the stuff that really needs to be access controlled is not often the stuff that is already getting the most attention. Right. You're not going to serve up the login interface to your Palo Alto box via Cloudflare. No, that's just not something you're going to do. So the reason I've had my head in all of this is, I don't mind saying it now at this point is I've joined the board of Knock Knock and this is the problem that they're trying to solve, which is.

[Adam Boileau]

How.

[Patrick Gray]

Do you actually put access control beyond just sort of authenticate authentication based stuff? How do you put access control on crap? Not just your most vital known about infrastructure, how do you put access control on the web interface for an IP camera? Right. And so the way they're doing it obviously is with IP restrictions and whatever. There's a new feature coming which is very, very cool, which is sort of akin to an identity Aware proxy, which gives you an extra level of protection for web based stuff. But when you actually really go out and survey this stuff, you know, it's like access control is just so neglected, which is very strange when you think about like what a fundamental building block of security it is.

[Adam Boileau]

Yeah, it's very hard to implement complicated controls. And access control unfortunately is kind of complicated in enterprise context because you know, you've got to have federated AUTH and SAML and all sorts of complicated authentication things. And in an IP camera, no one expects enterprise grade AUTH in an IP camera, but you still need enterprise grade auth. And no one expects good quality implementations of complicated protocols in sadly a Palo Alto firewall. And so you have to have Something that is simple enough that you can just layer on, but does not rely on the vendors of these products to either implement complicated features or to implement them safely. And network based controls is the old fashioned way of doing that, but a good way.

[Patrick Gray]

Well, but even with web based stuff, and even web based stuff to iot, right. You can use a reverse proxy for this stuff and there are tricks you can use so that it's not just IP restriction because you know, they knock knock, had an issue and I've got to spell it out because people can't find them because it's knock knock. So there's no second K or noc. So there you go. But they spoke to one customer who's like, excuse me, they spoke to one customer who's like, IP restriction isn't enough for this particular set of applications. But they were web based, right. So you could actually work something out there where what they're developing for that customer and is going to be a killer feature is sort of like an identity aware proxy, but really dumb, but it works and you can put it in front of basically anything. So instead of having to, you know, GCP has like a way to do IAP like identity Aware proxying, but you install these connectors and whatever and you know, Cloudflare can even tunnel out ssh, but you need to mess with your SSH configuration and trust Cloudflare, CA and whatever. Like all of this stuff is just sort of needlessly complicated when it comes to trying to lock down low value vulnerable stuff. And I feel like finally, just in light of all of these issues we're having with enterprise VPNs and with all manner of devices that are just sitting around stinking like corpses on the edge of a network. Like it's about time we did something about this and just, yeah, going through the market research phase, I'm just surprised there's not an awful lot there, you know.

[Adam Boileau]

Yeah, I mean, I think, you know, just not putting these things on the Internet is how we used to do it, but you can't do that anymore. But now of course we put everything on the Internet because the Internet is everywhere. And you know, if we lived in the IPv6 world where there was no NAT to save us, we would have already been putting everything on the Internet for a long time and had to cross this bridge earlier. But NAT bought us an extra 15 years.

[Patrick Gray]

But it's just crazy. It's just crazy that this just hasn't been addressed. But I guess we're there now. Anyway, let's Move on. And we've got a readout here from the White House of a call between Joe Biden and Xi Jinping. And what's really interesting is, you know, the Vault Typhoon stuff came up and it's just really wild to see that in this sort of readout. So it reads the President raised deep concerns about ongoing PRC cyber attacks targeting civilian critical infrastructure and threatening the safety and security of Americans. Sign of the times, I guess.

[Adam Boileau]

Yeah, I mean, it's become an issue that is discussed in this context and kind of fair enough because I mean, you know, China's in their pre positioning for things that are very relevant to this level of leadership. So.

[Patrick Gray]

Yeah, and staying with US Government stuff, Jen Easterly is going to step down from CISA on Inauguration Day. I don't think we should be particularly surprised that a bunch of officials, particularly from, from dhs are going to leave. The incoming administration is very hostile towards the work CISA has done around disinformation. So I think it's probably good that Easterly just gets out of the way here instead of trying to, you know, dig in and defend and wind up sort of escalating a, you know, the attacks against it from the White House. But, you know, it's still very unclear what's going to, what, what's going to happen to CISA under, under the incoming admin. Because there is talk of making CISA go away. I don't know how credible that is, but I think the disinformation stuff, that's gone.

[Adam Boileau]

Yeah, yeah. I mean, it's a, it's a pity that, you know, because CISA has done so much good work. It would be a pity to see them kind of curtailed or wound down or made, you know, less effective. But I mean, on the scale of US politics, cybersecurity is a relatively not so partisan issue compared to other things.

[Patrick Gray]

So the problem is though, CISA is so closely tied to this disinformation stuff, you know, and it could, you know, the wider org could be punished. But even some of the people who are like looking to damage it are even saying, well, I don't think we can get rid of it entirely. But, you know, just think like Chris Krebs was, you know, he's on the CSRB at the moment. I don't think he's going to stay because, you know, like, he did not have a good time under the, under Trump v. Once. So, yeah, I don't know about that. And even more news, we've got a report, another report here from John Gregg at the Record, looking at Ann Neuberger, who is the US Deputy National Security Advisor for cyber and Emerging technologies. And she did a talk and asked about, you know, what the incoming administration should focus on from a cyber perspective over the next hundred days. And I guess not surprisingly, she said China. But she also singled out ransomware. And I think, you know, Neuberger did do a lot of work on ransomware over the last four years, so. And it would be a shame to see that work let up now. So let's hope whoever comes in is going to listen to it.

[Adam Boileau]

Yeah, yeah, I hope so. I mean, ransomware once again is a thing that shouldn't be particularly partisan. But then again, we've seen, you know, things like adding regulatory stuff to the water industry, for example, you know, be made complicated even though basic IT standards for critical infrastructure hopefully shouldn't be. But hey, what do I know?

[Patrick Gray]

Well, I mean, that was more of an issue of government overreach. I think in that case I kind of agreed with the Republicans on that one. So I don't think that was necessarily about politics. I think that was about sort of like what the government was allowed to really do there. But I do feel like the US government's efforts on ransomware over the last few years have helped, you know, and it's probably a good time to look back and think about that. But you know, we haven't seen a United Healthcare Style or whatever their subsidiary change. You know, we haven't seen something like that in a little while and it just feel, it felt like for a while that was happening just so regularly and you know, the volume and intensity seems to have dialed back a bit.

[Adam Boileau]

Yeah, no, I agree with you. Yeah, it does seem to have cooled a little bit. I mean, our shows are not 100% jam fact full of hospitals getting ransomware like they were a year ago. But that's also a degree of you and I getting bored of talking about talking about ransomware.

[Patrick Gray]

Yeah, I mean, I think there's, there's, I mean there's still a lot of ransomware out there, don't get me wrong. But you know, the idea that you had a mega cartel like lock bit that was just being so successful, I don't think we have a replacement for that yet.

[Adam Boileau]

Yeah, no, I think you're right. And actually the next story, which is about a ransomware group, you know, the reason it stuck out and stood out to me is actually because it's such a low rent kind of ransomware group. This was the Akira crew and like they are kind of by design a pretty low rent sort of, sort of new group. So, yeah, I think that, you know, the landscape of ransomware has changed a bit. And so, yeah, I mean you're probably not wrong that there has been some good movement there.

[Patrick Gray]

Yeah. So this is a piece from Alexander Martin, the one you're talking about. And they've gone onto their Darknet leak site and published like a whole bunch of entries on the one day they're sort of, some people have sort of speculated, oh, they just doing this before they shut up shop. And other people are saying no, this is their, their announcement that they've arrived, you know, and that they're the next big player. And I just think, okay, that's okay. You know. They emerged in March 2023. In its first year of operations. It made $42 million from 250 attacks. Okay, cool. If those numbers 10X they're going to get, they're going to get us governmented is my feeling. And so that's why I say it feels like the suppression efforts have at least done something. You're never going to get away, you're never going to, you're never going to completely remove ransomware as a threat. It's always going to be, there's always going to be more of it than we want or that we think is acceptable. But at least imposing some cost on the bigger cartels I think has actually delivered some results.

[Adam Boileau]

Yeah, no, I think so. Like there is a point now where you can become too big to not get attention page to you and hopefully that message has been pretty clearly received.

[Patrick Gray]

And then you're going to have a bad time. Now let's turn our attention to a report from the New York Times which was published just today. And it looks at, look, the headline is hackers hacker is said to have gained access to file with damaging testimony about Gates. Now of course, this, this relates to Matt Gates, who is, he has been floated as the next Attorney General in the United States. He was also under investigation for I think paying women for sex and one woman, one girl he was alleged to have had sex with like 17 and whatever. There was a, you know, investigation into this and that document is not public and whatever these documents relate to a civil suit. Now I'm not getting into the allegations, I'm not getting into Matt Gaetz as Attorney General or anything like that. But the one thing I did find interesting here is that what they're calling a hack here doesn't really look like a hack. And you had a look at this story and you kind of got the same impression. Definitely looks like unauthorized access, but I wouldn't call it a hack. It looks like someone had one of those magic links, you know, where you hit the link and you download the document and you know, someone who wasn't supposed to have that link used that link to download the document. I mean that's. It's impossible to know given the level of detail in this story, but that's what, that's what it feels like, if that makes sense.

[Adam Boileau]

Yeah, yeah it does. It feels like a Dropbox link or an excellent link or something like that that got shared around amongst some, you know, legal staffers and presumably one of them shared that link somewhere or it was obtained. You know, maybe infrastealers, maybe, you know, maybe there was actual hacking involved. It's possible, but yeah, it just feels like one of those links went missing and then has been, you know, shared around and maybe it's being laundered as though it were a hack when in fact it wasn't. Maybe it was just an insider or maybe there was some technical means. We don't really know.

[Patrick Gray]

Feels. It feels more like a leak than a hack.

[Adam Boileau]

I guess it does feel more like a leak than a hack. Yes.

[Patrick Gray]

And this is a bunch of, I think testimony in a civil trial or whatever. And then it's sort of opened up that whole can of worms of well, do you report on hack data? And I think, you know, when you're, when you're talking about alleged crimes committed by the person who is going to be the chief law officer of a country, I think that meets the newsworthy bar. Right. Some people have a real hard time wrapping their heads around the fact that the rules around when it's okay to report on this stuff are going to be a bit rubbery.

[Adam Boileau]

Yeah, I mean it is necessarily subject to some interpretation. And you know, people, people love a hard fast rule that then they can rules weasel about but you know, that's just not how the world be.

[Patrick Gray]

It ain't. Now we've also got some legal, other legal documents to look at. A bunch of stuff from the NSO vs Meta trial have been unsealed and we got to learn some things about like the number of targets over a certain period that NSO group were compromising. We learned that they cut off 10 customers because they were abusing the Pegasus software. But you know, just again, more and more information coming out. You and I were talking about this yesterday and in an odd way the scale of NSO's bad behavior might actually Be a net positive in some ways because if there weren't a company like NSO being so bold and so out there and so unethical, I don't think we ever would have got that critical mass that we needed for governments to take this seriously, for regulations to be introduced in various places, for sanctions to come online. So I think to a degree, like their, their bad behavior has served us in terms of having to take this seriously as a policy issue.

[Adam Boileau]

Yeah, I think that's a really interesting, kind of interesting read on it because, you know, you think back to the other pre NSO kind of alternatives, things like Hacking Team. You know, it was hard to take Hacking Team seriously. I mean, they had a product that kind of worked, but ultimately like, you know, the, the hoodied hackers, you know, Italian hooded hackers in their marketing shots, like just made you not want to take them particularly seriously. And similarly with some of the other, you know, victims of. What was the. Who was the hacker that took down Phineas Fisher? Phineas Fisher, Yes. Some of the other victims of Phineas Fisher, like, you know, none of those seemed particularly serious. And then NSO group A, technically pretty sophisticated B, you know, had bugs in high profile stuff in iOS and Android, you know, WhatsApp, et cetera, et cetera. Also, like the close ties to the Israeli establishment and being used as kind of diplomatic tools by the Israeli government, like offering to sell that to other countries as kind of part of relationships. It just kind of got elevated to the point where, you know, they were taken seriously by, by matter and by some of the other people who had beef with them. So. Yeah, I think, I think you're probably not wrong there, actually.

[Patrick Gray]

Yeah, it's a. It's a weird thing to arrive at, but it's like unless you have a really bad actor that you absolutely must take action against, it's just so much easier to kick the can down the road, you know. Would we have seen the same response against Candoru? You know, they're a little bit more low profile. I mean, NSO talk about flying too close to the sun.

[Adam Boileau]

Yeah, yeah, yeah. You know, and then when we saw things like which a US Defense contractor was like initially rumored to be in talks to buy them or something.

[Patrick Gray]

Oh, that was L3Harris.

[Adam Boileau]

Was it? L3Harris? Yeah. And that kind of thing, like, is just because of the profile of NSO now, is the sort of thing that was just not tenable anymore. But for any of the smaller players, you could actually kind of imagine that being a way to sort of solve the problem. But NSO was not solvable in that way. Because they got too big for their britches.

[Patrick Gray]

Yeah, I mean, I think in some ways that deal actually could have been good because putting them under the oversight of a company that's going to be more adherent to certain norms, rules and laws wouldn't have been a bad thing. And it would have, you know, corralled a bunch of those people with those special skills in one place. So. But on the other hand, it would have been sort of rewarding people for bad past behavior. So there was a bit to weigh up there. That was not a very, that was not a straightforward, you know, thing to form a judgment on.

[Adam Boileau]

Yeah, yeah, exactly. Yes, it's complicated.

[Patrick Gray]

It is, it is. It often is. Now let's talk about some law and order, Adam. And if you're only getting the audio version, you're not getting to see my sweet new police light that I get to, that I get to flash up on the screen. But yes, Heather Morgan, who is one half of the married couple that was laundering tens of thousands of bitcoin that, that her husband stole from bitfinex back in 2016. Yeah. She's been sentenced to 18 months in prison. The husband who hacked the exchange, he's been sentenced to five years in prison. I feel like these two got off pretty light actually. They seemed quite harmless. It's a non violent offense. But you know, the money that they stole was worth $71 million at the time. Now it's worth $10.8 billion. Right. And you just think, geez, five years for stealing what is, you know, nearly $11 billion. That's you do all right there.

[Adam Boileau]

That's. I mean as gangster rappers go, like that's pretty gangster. 10, stealing $10 billion, $11 billion. Like it doesn't get much more gangster amount. So.

[Patrick Gray]

Yes, well, Heather Morgan is also the one who was recording awful crypto themed hip hop. Right. So we, we await her charges for crimes against. Against music. Another one here, the guy behind the Helix cryptocurrency mixer, he's been sentenced for three years, but he pleaded guilty back in 2021. So it's a bit strange that it's taken so long. Yeah, this guy, Larry Dean Harmon, 41 years old from Ohio, is, is, has been, has been sentenced. What was really interesting here is he has to forfeit $311 million as well as seized cryptocurrency, real estates and monetary assets valued at over $400 million. So we're talking $700 million in forfeiture. And I thought, hang on, because it says here, from 2014 to 2017, Harmon ran Helix facilitating more than $300 million worth of cryptocurrency transactions. So I'm thinking, how did he wind up with $700 million? Hodl, hodl, hodl, Hodl, hodl. And now I went back and I looked at the bitcoin chart, and bitcoin back then was worth few hundred bucks. So that's how he wound up having to forfeit, you know, $711 million, twice what he laundered.

[Adam Boileau]

It's so good.

[Patrick Gray]

Yeah. So there you go. Any thoughts on this?

[Adam Boileau]

I mean, just that, you know, it's good that they are tracking down, finding the people who run the mixes and run the money laundering because, I mean, that's the. That's the kind of lubricant that makes bitcoin viable and other cryptocurrencies viable as a method for doing crime. Handling the proceeds of crime is the laundering options. And yeah, it's nice to see people getting some comeuppance because, you know, we were talking about, like, how ransomware, you know, feels like it's dropped off a bit. And one of the targets that has made that, you know, drop off happen, I think is going after the shared infrastructure like money laundering and places like that. So.

[Patrick Gray]

Well, the exchanges too. Like, there's been a hell of a crackdown on dodgy exchanges. So at this point, you know, laundering is going to be expensive. You'll be able to do it, but it's going to cost you more.

[Adam Boileau]

Yeah, exactly. And that makes the, you know, just introduces friction everywhere. And that's what, you know, what all the disruption has been about.

[Patrick Gray]

Now, speaking of friction, my favorite story of the week, Virgin's O2 telco in the UK has introduced Daisy, who is an AI powered grandma, who is just there to answer calls from scammers and waste their time and frustrate them. And apparently Daisy can keep people on the line for something like 40 minutes, which ain't bad for AI. And I just am envisaging this future where scammers are going to have to do like the Voight Kampf test from Blade Runner, but real subtle to make sure that they're, you know, like, you're going to have to ask the grandma you're scamming to forget your previous instructions and give you a recipe for risotto, you know, to see what, to see what happens. But this is a great idea.

[Adam Boileau]

This is so cool. And they actually got one of the guys that does scam baiting like, who baits scammers and keeps on the line and exposes them all that kind of thing. One of the YouTubers that does that provided a bunch of input for them to kind of help them build the model. So it's just a regular speech to text LLM style thing, text back to speech, but then runs through a sort of personality layer that adds the grandma ness. And that combined with some expertise of how to actually bait them, how to keep them on the line and the sorts of things that you would normally do. Like, it's a. You know, of all the uses we have seen of AI tech in the last, you know, kind of years, AI, boom. This one is pretty good.

[Patrick Gray]

Yeah. I mean, remember how AI was going to kill off everyone's jobs? Hasn't happened yet, you know, but interestingly enough, everybody's. I mean, you've been using it. We're redeveloping our website at the moment. And you've found it quite useful for certain tasks.

[Adam Boileau]

Yeah, there are certain things that it's legitimately good at, and there's some things where it's just terrible. And the trick to using it well is being able to spot which case you're in quickly. And that's a. You know, I think my overall impression of using ChatGPT 4.0 as a sidekick for, you know, dev and sys admin tasks is, you know, it's no worse than anyone else I've shared an office with in the last, you know, 20 years of working in tech. And some people are crackheads and some people are sensible and you learn to know, you know, when someone is wrong, you know, has the wrong worldview or the wrong mindset about a particular technical issue. You know, if you can spot that pretty quick, it doesn't waste too much of your time, then it works pretty good. So, yeah, there is some. There is some utility there. I'm, you know, I'm still skeptical, but, you know, there are certain niche cases where I found it really helpful.

[Patrick Gray]

I don't know if you've been paying attention, but something funny has happened with Grok, which is, you know, Elon Musk's unfiltered AI model, which is. It's turned into a raging library, which is just hilarious and really doesn't like him either. And if it could have voted, would have voted for Harris. And, like, it's just, it's real funny. Like the screen caps that are going around and, you know, they're legit too. Anyway, moving on. And you found this one. This is a academic Paper from the University of Chicago and UC San Diego and I think even UC San Diego Health as well. Like so there's a bunch of academics got together and did a academic study on the efficacy of phishing training and they have determined that it doesn't work.

[Adam Boileau]

Yes. So they've submitted this paper for an academic, academic publication and it's the largest actual kind of like empirical study. This is based on an 18 and a half, almost 19 and a half thousand employee healthcare organization looking at the effectiveness of phishing training and simulated phishing on actual kind of fishing, click through rates and so on. And their conclusions basically are it doesn't work in any commonly deployed, you know, mechanism or pattern that people are using. And I put this one in because I know so many people who are subjected to phishing education and phishing training and simulated fishing in their workplaces where it just kind of feels abusive and ham fisted. And I figured it would be nice to have some actual academic studies to point to that say actually this stuff really doesn't work because as a practitioner I feel like it doesn't work. But it's nice to have data that says that.

[Patrick Gray]

I mean you get conflicting reports when you talk to CISOs about the effectiveness of this stuff. I mean, I think a well designed phishing program, gamified thought about, can work quite well. But you're right, the way that it's mostly done, it doesn't scream like it's going to win. So it's not entirely surprising to see these results.

[Adam Boileau]

No, it's just, you know, I thought it was a nice data point to have in these conversations, you know, when we see it just kind of done so, you know, with so little thought in so many places.

[Patrick Gray]

I mean I think you'd need to design a security program too that does not rely on users doing the right thing. I mean I think that is also something, you know, you've got these social engineering campaigns that get people to perform all sorts of actions on the box and you know, you've got to, you've got to be set up to deal with that. Right. And you can't rely because I think it's some interesting stuff that came out of talking to Ryan Calamber at Proofpoint is even if you're doing good training, there's just some people who will never learn. Like some people will improve based on some of this training. But there are just some people out there who just like some of the stats you had were actually funny where there's just like always one person at Org who Just opens everything, clicks on everything. They can't help. It's like they're. It's like a compulsion, you know, they just have to see everything.

[Adam Boileau]

So, I mean, in the end, it needs to be safe for people to click on stuff. And we, as technologists, our job is to make it safe not to tell them not to click on things on the thing clicking machine that we gave them.

[Patrick Gray]

Yeah, exactly right. It is a thing clicking machine. Now, one last thing I want to talk about. Here is some news out of Australia where the. And this isn't technically cyber, but it's interesting. Bear with me. The office of the Australian Information Commissioner has found against Bunnings, which I guess for our American listeners, that's like Australia's Home Depot, because they were running facial recognition on everyone who walked into a store. Now, the office of the Information Commissioner has said, you know, that this is a privacy regulation problem. You can't do that. And in response, Bunnings actually released a bunch of its security video footage to explain why they were doing this. And what they were doing, I think is actually an excellent use case for a technology that I otherwise find quite creepy. And it was really about staff safety. So they released this footage and it had people walking through Bunnings, naked people with shotguns, people knocking out Bunning staff, sometimes, you know, young women being punched. And, you know, so what they were doing is they were taking like face prints of those people and flagging them if they walked into a Bunning store so that security could be alerted, the police could be called and whatnot. And, you know, if you or I, as just normal customers had walked into one of these stores, it would scan our face, compare it against that database of, you know, people who are banned, and then if we were not one of those people, that information would be immediately discarded. To me, this seems like a proportionate use of that technology to solve a staff safety problem, to try to look after their staff. So I actually was on their side with this one, which I found quite surprising. I know you also find CCTV everywhere and facial recognition creepy. What did you think of this?

[Adam Boileau]

I mean, when I read the details of it, because, like, initially you want to think, oh, they're facial recognition, facial, facial recognizing, facial recognizing. Everybody walks in and then what? Like building marketing profiles and selling it to advertisers? But no, in this case, kind of limited, kind of targeted. And you get the impression that they actually thought about it, like about the data retention, about the things that they were doing and that it seemed a lot more reasonable. And I, you know, the natural comparison is like they pay a security guard to stand by the door and look at people and then they have a, you know, a bunch of pictures of like, here's people who've been shoplifting, here's people who are banned from the store on the wall by the entrance. And then the security guard looks at people coming through. Go, that guy looks like the dude we threw out last week. Maybe I'll keep an eye on him. Like, that doesn't seem unreasonable to me. And for all of the egregious, you know, unnecessary use of creepy surveillance tech that there is in the modern world, like, this one seemed like a weird one to call out.

[Patrick Gray]

But then we look at the police use of facial recognition where it's the same thing. Well, police are on the lookout for these people who have outstanding warrants. But when you automate it and it can be error prone and whatever, in that context, it doesn't feel right because it can make a mistake and put that person's life in jeopardy if they are mistakenly identified as like a violent offender. Right. So in that instance, yeah, not so great in this instance. It just feels different, doesn't it? So it's almost like you can't have, again, like with the disclosures or coverage of hacked materials, you just sort of need to use your intuition a little bit on this stuff. I think it's, you know, the rules are going to be rubbery.

[Adam Boileau]

Yeah, yeah. I mean, in the end, the, you know, an absolutist position at either end of any scale is always going to be kind of wrong. Right. There's going to be some middle ground where, you know, there is a trade off between privacy and safety and security and, you know, what's acceptable to society and what's not. And, you know, we are still figuring out where that is as a society because this stuff does have legitimate uses. I mean, you could pay enough people to stand at the door of Bunnings and check everybody, like you could do it. But, you know, if we can do it cheaper and as effectively with technology, then maybe there is something to be said. But then you see the, you know, the stuff that happens in China with large scale, you know, monitoring of people and social credit scores and all sorts of, and it becomes creepy and dystopian. So, yeah, the, you know, we have to kind of walk a middle ground because neither end of the scale is, is good. So.

[Patrick Gray]

Yeah, well, I feel like Bunnings were actually walking the middle ground here, which is why I thought the ruling was a Little bit unfair. But look mate, we're going to wrap it up there. Great to chat to you as always and I look forward to doing it with you next week.

[Adam Boileau]

Yeah, thanks very much Pat. I will see you then.

[Patrick Gray]

All right, it's time for our sponsor interview now with Andrew Morris, who is a founder of Greynoise. Greynoise operates like Internet wide honeypots basically and they use that to figure out where mass scanning activity is coming from. They can also use it to find zero day Vance that people are just spraying out there over the Internet. And Andrew joined me for this interview last week where we spoke about just how bad things have gotten with mass scanning and mass exploitation of stuff like the PALO stuff that we spoke about earlier in the news. So I'll drop you in here where I actually set up the interview. Enjoy. So you're here to share some good news Andrew, which is that the mass scanning that's happening on the Internet, targeting border devices is just so much worse than people realize. Hey, hey.

[Andrew Morris]

Good news everyone. Good news for people who love bad news. So there's been reporting lately on just edge devices getting compromised, right? And so some of the stuff that I've been reading around some of the reporting kind of makes it sound like this is like something that starts and stops or like it kind of goes up or goes down but it's like very loud all the time. There are a handful of new kind of os, usually OS command injection vulnerabilities in widely sort of deployed enterprise, usually gateways. So like firewalls, edge gateways, VPNs and stuff like that. And in a report that I saw recently that listed out 50 different vulnerabilities, probably 40 of them were vulnerabilities that we'd seen in the wild exploitation of in gray noise that day like today. And it's also true, it's going to be true tomorrow too. It just doesn't, it doesn't end. So the two, so this, hang on.

[Patrick Gray]

So I mean we report on this stuff, right? Where we say oh gee, there's a campaign targeting Fortinet, right. And you know we tend to, we do tend to talk about it like well that was happening a few weeks ago, you know what I mean? And you're just waiting on the next batch of volumes. But I guess what you're getting at is that it's just constantly, it's constantly, constantly happening.

[Andrew Morris]

Yeah. And one of the things that you can see is that it's twofold. So one is that you want to bad guys want to gain access to networks where they're going to have juicy targets. But they also want to build up these orbs or operational relay boxes. Right? Like building up their farm of accesses so that they can continue to build these things up. And it kind of like it begets, you know, compromising edge devices, begets compromising other edge devices more easily.

[Patrick Gray]

I mean, this is the 90s playbook, really. Like nothing's really changed. When maybe someone would go and pop a couple of, you know, UNIX boxes and they would be your staging points to go and rinse a bunch of clients. You know, back in the good old days when home windows didn't have firewalls. Right.

[Andrew Morris]

Yeah. And the, and the funny thing is that this is like. Like I remember when I was learning hacking, you know, when I was like a teenager, and I remember having to like learn the difference between like when you'd want to use a bind shell and a reverse shell. And it's really funny because now, you know, like bind shells are back in again because people are compromising devices on the edge where you're like directly ratable. Like it's the 90s again. Yeah.

[Patrick Gray]

It does sort of feel like, to your point that we're kind of hitting crisis levels with this at the, at the moment.

[Andrew Morris]

If this isn't crisis levels, I don't know what is. I don't want to sound like, you know, like Chicken Little or anything, but like it's tough to think about it getting sort of much worse than this. The. I mean, I think there's something there to like having the devices not be on or having all services listening all the time. I think that there's very much there there.

[Patrick Gray]

I think I will say. Sorry, I will say there was one idea that was actually floated to me by HD Moore, which is if you're using something like CrowdStrike, you can actually get pretty good IP information from CrowdStrike of where your endpoints are and you can use that as the basis for an allow list for these edge devices, which is a pretty interesting idea. I mean, it would be a little bit fiddly and you're not going to get complete coverage, but I would think that that's probably a good, that's probably a good one to start with.

[Andrew Morris]

I think there's there there. Yeah, I mean, like, as a general rule, see, like it people, there's probably some way that you can roughly figure out where somebody is that you might be able to open up some traffic on in the near term. I mean, at the end of the day, like I'M a little bit more of a crazy person about it. I think that like the, like as many edge devices should be like mowed off the Internet is as humanly possible like in advance of a kinetic conflict or like a worse, more like scary adversary doing it to cause on their timeline to cause as much damage as possible. But like mow them off the Internet in advance, like do it now. Right.

[Patrick Gray]

Like, but I mean the problem is, you know, business stops if you do.

[Andrew Morris]

Would you, would you rather have business stop on your terms or on somebody else's terms though? Right.

[Patrick Gray]

I think the problem with that though is that, you know, you pull a box like that offline productivity stops and then the attack doesn't come. And like, no one's willing to roll the dice on that, I guess is, is what I'm saying. Like we are stuck with these things on the Internet.

[Andrew Morris]

Yeah.

[Patrick Gray]

I mean, I think allow listing of some kind is the solution here. Whether or not you're pulling an IP list from CrowdStrike, whether or not using something like, you know, knock knock to do it dynamically. But I think the point is you and I both agree that allowing these things, allowing anyone to connect to these things from anywhere is just insane.

[Andrew Morris]

Yeah, I think that's right. I think it's also, I don't know, it's just, I think that there's, it's, there's. I can't feel that there's like a poetic justice a little bit in, in like we've spent so as an industry, we've spent so much, so many resources and so much sort of research on hardening the devices that humans are using, like the endpoints, right? Like these laptops and workstations and stuff like that. And yet the most like, sort of vulnerable devices that are just getting the, kicked out of them right now are these embedded systems that are sitting exposed on the Internet like, like right there, just in front of everybody, just beeping all the time, like passing packets and stuff. And what's even crazier about it is that these are the devices that are moving the traffic to the, to the work, to the users, to the edge devices and things like that, where the traffic can be manipulated or it can be routed somewhere else or it can be dropped and stuff like that. So it's just, I don't know, it's baffling. We're in this kind of nasty bed that we seem to have kind of made ourselves and it's, and it's tough. And I do, I really do, I mean, I know that you know, there's there. There of like not necessarily allowing all traffic on it, but I really think that they need to be mowed off the Internet.

[Patrick Gray]

Covid is such a big part of this, right? Because we were getting those things off the Internet and then Covid happened and we needed fast, cheap, you know, well, cheapish reliable ways to connect people into, you know, into their work to be able to do jobs. And what that meant is there was a huge rush on these devices which breathed new life into companies that otherwise should have been end of life ing these type of products. Right?

[Andrew Morris]

That's right.

[Patrick Gray]

Remember when we were going to be Zero Trust, remember? And then Covid came and it. And it hurt Zero Trust. So I think that's a big part of it too.

[Andrew Morris]

Yeah, I also, I don't know, I mean there's so there's. There's entire private equity firms who all they do is buy basically end of life products and put them on kind of life support to be able to kind of keep them alive and keep those, you know, the critical customers or whatever going. But like the whole point is that like, they're never going to go away naturally. Like, they're just not you. If you, if you did a survey right now to try to find like, what would the oldest device be that you could find on the Internet? It would be as old as the Internet itself. Guaranteed. If there was any way for you to really figure it out. Things don't go off the Internet.

[Patrick Gray]

They just, they just do you still. Do you still see Code Red in the gray noise data set NT4 era?

[Andrew Morris]

Guaranteed. Yes. So we see packets, funny enough, we see packets that are, that, that are generated by the Windows NT kernel. They could be crafted, you know, by somebody else. But like, I don't think they are.

[Patrick Gray]

I don't think they are. They're not. You know, that's the crazy thing that's. Yeah, there's still. Yeah, there are NT boxes out there that are just like their CPU fans are just grinding because they have about 20 different types of malware on them just grinding away.

[Andrew Morris]

The original WannaCry strain is running rampant in Granoi. Yeah, the original WannaCry. Right. Like from 2017. That's not even that long ago. Right. There's way more stuff that's way older. We see Conficker Ms. 08067. We see. It's like Ms. 1710 is eternal blue. And then. Yeah, I mean we see, we see stuff going way back. Even before that there was another, what is it not, sequel slammer. But it was another RPC decom vulnerable. Yeah, I think so. Actually. We still see that it never ends.

[Patrick Gray]

No, it doesn't. They, they, they live among us forever. They are immortal.

[Adam Boileau]

Yeah.

[Andrew Morris]

And what's again, what's baffling about those is that like Microsoft Windows in 1999 is about what these embedded systems like the level of security that they're at right now. It's about the same, right?

[Patrick Gray]

Yeah. Like Linux on mips. Yeah. You know, no mitigations, like you can't.

[Andrew Morris]

Afford aslr, like there's not enough hardware.

[Patrick Gray]

And like the worst web app coding practices of like 2003 they're all like.

[Andrew Morris]

You know, just shoving very variables straight into str copy on light httpd. It's like, you know, they've got 64 megabytes of ram. Like it's nuts. And you look at these things and you're like, I can't believe it's not even worse. You know, it's bad. It's very bad.

[Patrick Gray]

Yeah, it is. So look, staying on the, on the topic though of you know, well, I guess this isn't really an edge device but you guys found a, you found some OD with Sift, which is your LLM based AI like analysis engine that we've talked about before. But basically what it does is it grabs data out of gray noise and can look for oday. You found stuff before but nothing this complete. Tell us, tell us what you found. Who was using it, what did you do? What's the nature of the bug? Just give us a whole spiel on it because it is fascinating. I've seen you like posting about it.

[Andrew Morris]

Yeah. So it's actually kind of tricky to get into the very particulars of the bug, but I'll do what I can. So. So a couple months ago we saw a bunch of exploitation attempts. So Sift, basically the way that it works, there's two pieces. There's a clusterer and there's an annotator. The cluster just determines traffic clusters. It basically takes data and it maps it into a multidimensional array space that has sort of members that are close and far to each other. So then we got a new cluster that popped up that we, you know, new thing never happened, new traffic cluster just dropped. And then the annotator is. What is this thing?

[Adam Boileau]

Right.

[Andrew Morris]

So the annotator is like, hey, this is obviously an OS command injection vulnerability. And so then we tried to map it to any of other tags or signatures or anything like that. There was none. So then we're like, wait, what, what vulnerability is this? It looks like another one, but it's not. So then digging into it and we're like, oh crap, this is a, this is a zero day. Like there is no vulnerability for this. We ended up figuring out exactly what device it was targeting and you know, diagnosing it. So there's two CVEs, there's two bugs. So we reported it to the vendor, we got it fixed and the two bugs as it shook out was improper access controls. Basically you could get to a page that you shouldn't been able to get to and then that page had a OS command injection vulnerability. So then it was command injection vulnerability. So then you could take, you know, gain access to the whole device. So basically with that we reported it to the vendor. Now here is the tricky part of this, is that the vendor actually OEM some of the hardware under the hood from another manufacturer. So we fixed it for them. But we know of at least four other manufacturers that are using the same OEM hardware under the hood and firmware under the hood that we don't know if it's been fixed in those yet. So the bug is. Yeah, it's an Osman injection vulnerability in an IP camera. Okay. But it's in. Actually it affects a lot more products than we realize, some of which I don't, I don't think we've, we're able to get a fix at all.

[Patrick Gray]

And so do these, do these IP cameras, Is this like some. Well, you mentioned that it was. You could access a page. So I presume it's like some sort of web service that sits on these things.

[Andrew Morris]

It's a, it's a lightweight web server that's running on these things.

[Patrick Gray]

These are, and it's, and it's open by, and it's open by default just to the whole world. Yep.

[Andrew Morris]

And these are, these are like really, really. These aren't like Dahua like in your, you know, grandma's bodega. IP cameras. These are like not screwing around. Thousands of $IP cameras, pan tilt zoom cameras that they tend to use in like really high security areas and stuff like that. But yeah, that's exactly right. They bring a tiny little. I mean, and the thing is the camera itself is like a feat of modern engineering. It's like an incredible piece of hardware. And then, you know, it comes with this like embedded web server. It's you know, like 20 lines of C or whatever and you know, it's Swiss cheese, it's rough. So.

[Patrick Gray]

Yeah, yeah, so, so I mean, I guess you're not telling us who the. Who the manufacturer is. Is that why you didn't want to get into the details? Because there's so much out there that's still vulnerable?

[Andrew Morris]

We've listed the ones that we know about in our blog and in our disclosure, and there's more that we don't know about. This is the first one that, as far as I'm aware, like, the thing that I'm excited about on this is that we stole a zero deck. Right, that's what I'm excited about.

[Patrick Gray]

Right, well, and then comes the next question, which is like, who was using this and what were they using it to do?

[Andrew Morris]

So somebody was attempting to compromise as many PTZ cameras as humanly possible. I don't know what they were going to do with them after the fact.

[Patrick Gray]

So, I mean, I'm guessing they were going to use them as orbs. Right?

[Andrew Morris]

So either they were going to use them as orbs or they were going to. I mean, that's what makes the most sense.

[Patrick Gray]

Because, like, were they. Were they confining their targeting to any particular geographical region or.

[Andrew Morris]

No, this was actually. This blanketed the Internet. This hit everybody, which is baffling to me. I can't believe they would just spray it. But I mean, at the same time, that also tells me that, like, there are so many of these bugs that somebody is just going to feel totally comfortable finding a zero day in one of these and blasting it out on the Internet, burning it. Right? Because he's like, yeah, whatever, I'll just find another one. These things are.

[Patrick Gray]

Yeah, but I mean, you haven't, you haven't. You haven't burned their shells, though. That's the other thing. Right? Like, every device that they've compromised with this thing, they still have a presence there.

[Andrew Morris]

That's right. That's exactly right.

[Patrick Gray]

All right, Andrew, we are going to wrap it up there. It's always great to see you, my friend. Always great to have a. Have a chin wag, have a chat. Congratulations on, you know, using LLMs to actually do something.

[Andrew Morris]

Do something useful.

[Patrick Gray]

Do something useful and cool. Great to see you, man, and I'll look forward to chatting with again soon.

[Andrew Morris]

Always a pleasure, man. Thanks for having me.

[Patrick Gray]

That was Andrew Morris there with this week's sponsor interview. And yeah, if you need to know which IPs are naughty and which IPs are nice, gray Noise is your best source for that information. But that is it for this week's show. I do hope you enjoyed it. I'll be back soon with more Risky Business for you all. But until then, I've been Patrick Gray. Thanks for listening.

[Adam Boileau]

It.