Loading summary
Transcript28 lines
- [00:04]
A
Hello everyone, this is Tom Uren. I'm here with another Risky Business News sponsor interview. Today I have with me Craig Rowland of Sandfly Security, CEO and founder of Sandfly Security. G' day Craig, how are you?
- [00:16]
B
I'm doing well, thank you.
- [00:17]
A
Craig, you make a agentless Linux endpoint detection product. And one of the mysteries for me is that Linux is both everywhere. And I write a newsletter about security issues every week and I almost never write about Linux. And it seems like it's the dark matter of the security universe in that it's everywhere but no one sees it and no one touches it. And so I know for sure that there are states and people attack Linux systems. How come we don't hear about it more often?
- [00:50]
B
The main problem Linux has is it's kind of the background operating system. So it's basically the operating system that runs the Internet. But most people in their day to day lives, they see a Windows machine or a Macintosh in front of them and they think that's what the Internet is. But 90, 95% of all cloud workloads are going to be Linux. A lot of embedded systems, IoT network devices, all sorts of stuff. They're almost always going to be a Linux based system, including a lot of critical infrastructure, telecommunications, power, things like that. So it's all over the place. So a lot of the infosec budget tends to be focused on Windows because that's what see. But behind the scenes it's actually the Linux that's really keeping the data moving in. Most organizations and a lot of these security teams tend to have a lot fewer people who understand Linux and understand Linux forensics, let alone how to investigate a host at scale. And that's what our product is coming in to do. Basically an agentless mechanism to go in and investigate and keep an eye on Linux systems and help really augment security teams that frequently don't have the Linux security personnel to go out and actually find problems as you're discussing. So people might think the Linux doesn't have any issues with security, but really most of the time what it is is they're just not looking. And that makes it a very attractive target, especially to nation states that are looking to persist for a very long time.
- [02:09]
A
Right. So it seems that you're describing a dynamic where defenders know desktop operating systems and I guess there's a whole ecosystem of security training and I guess Playbooks really for defenders, but also for people like ransomware actors who've got Playbooks. And it's just that you're describing a System where everyone is just comfortable operating on Windows and to some extent macOS. And so that's where the spotlight is, that's where most of the attention from ransomware actors goes. And so the Linux is just left behind and I guess, does that make it even a more attractive place for the, I guess slightly more sophisticated actors?
- [02:52]
B
Yeah, it is. You know, it's human nature to do what you're good at doing. Right. So if you're already trained up to do Windows and Mac, you tend to focus there. That's where you're comfortable doing it. I'll tell you what, I got my life started as a, as a red team back then we called it pen testing and one of the first systems I broke into was a Unix system. It's actually AI Xbox, but it happened on Linux too. Same thing, but it always left an impression on me because I knew nobody in that organization was looking at that system. Had an uptime of four years, it hadn't been rebooted. Right. And this was common. Four years is not the longest you've ever heard Linux. We've heard systems almost 10 years old have been up for 10 years. Right. So it's one of these things where for whatever reason they're not looked at. A lot of nation state adversaries know this and they are targeting these systems and not just targeting them, they also target them because they know the endpoint. Workstations are very closely watched by AV and antivirus products through the wazoo. So if I could get onto an edge device that, and nobody's looking at it because Linux based that, that's very profitable. So we had one customer, they reported having a command and control server on their Synology nas. So Synology is a Linux based system, there's no security on it. Yeah, they have basic antivirus, but it's Windows based, scanning files. But the thing is if I'm an attacker and I could get on that Linux system and it's concentrating all the data in the company, why do I want to mess around with an endpoint, right. I go to where people aren't looking, where it's going to have valuable data. And it could be here, Synology, but it could be a VPN router, could be another edge device, it could be IoT camera, things like that.
- [04:26]
A
Yeah, I guess what you're describing is a situation where I guess it's like don't ask, don't tell, like you're blissfully ignorant. What would drive people or organizations to look at their Linux fleet?
- [04:37]
B
Well, one of our biggest Competitors as a company is actually apathy. So people are not doing anything or they don't want to look. Right. That's another thing too. Some people don't want to look because they don't want to find a problem. You know, the drivers of the organizations that tend to adopt us tend to be large, critical infrastructure companies, people who are responsible for telecommunications, big piece of the Internet, stuff like that. People that they know, if they go down, it's like national news. So they have a very high driving interest to make sure that their systems, which are by and large Linux running the show, get monitored. They tend to have more serious security teams that are aware of the problem. Smaller organizations, they still rely on Linux quite a bit. I guarantee you almost all their servers and stuff, running stuff are running Linux. But again, it just depends on what their priorities are. Larger companies tend to know. And it's one of these things where if I get the security teams in a room and I ask them what you're doing about watching Linux, they'll kind of look over their shoulder. If the manager's not around, they'll be like, well, you know, kind of nothing. We're looking at log files or something. And I tend to tell them, you know, we deal with Linux malware all the time and it goes to extensive lengths not to show up in a log. So if you're just showing up in a Linux log file, that's all you're looking at. I promise you. Promise, promise, promise you. You are missing malware that's operating on those systems potentially, and you really should go in there and take a look. So kind of get back to your question about what drives an organization to want to look at Linux. I think part of it comes down to they're looking at their gaps. The systems look at their gaps. They say, holy cow, 90% of our infrastructure is Linux. What are we doing there to watch? And the answer is nothing. Well, you know, the CISOs that really want to keep their jobs start asking a few more questions about why we're not watching it. And then that's when it gets into our approach. By not deploying the endpoint agents, we could do it very safely with high compatibility. And that tends to be very attractive to these organizations as well.
- [06:29]
A
Right, Right. I guess you said before that in the conversation we were having before we started recording. Part of the problem is that because Linux runs on critical infrastructure, people are reluctant to deploy extra things because it's critical. Right. So you run into that paradox of the malicious actors are happy to deploy onto your Critical infrastructure because. And I guess the competent ones would do a lot of testing. Right? But in terms of defending it, you're in this sort of catch 22 where you're a bit worried about doing the same.
- [07:01]
B
Yeah, but I always describe it as Bob and accounting on his Windows machine. And the way it is, if I have an antivirus product or an EDR that goes crazy on Bob's machine, I crash his box. I got one person mad at me, I got Bob. But if I go in and I take out the entire database cluster that's running on Linux on Black Friday or something, people get fired. So it's a completely different level of risk. Or I take out a telecommunications infrastructure, your mobile phone's not working or something. Right. So these people who run the Linux systems tend to be far more aware of the risks. And that's where the endpoint agents run into big trouble. Because they frequently have compatibility and performance issues. The system operators can't often update the systems because the EDR agent might break the system or the system might break the EDR agent. So it's kind of always this yin and yang thing going on. So we come in with this message of basically look, we're not going to deploy anything that ties into that kernel. We could work on systems up to a decade old, all the way to modern cloud infrastructure. We could also work on Intel AMD, ARM, MIPS and IBM Power CPUs, which means not only are traditional servers being watched, but we're watching embedded devices, IoT cameras, edge routers, things like that. So essentially what we're saying is we want to give you this visibility across all these boxes without the traditional risks that you typically face with the Endpoint agent. And that's why a lot of the, like I said, mission critical type organizations tend to look at us because we could provide that visibility that don't have today in a very, very safe and compatible way.
- [08:31]
A
Is there something about Linux that makes it both hard to defend and attack? I'm wondering if part of the appeal of Windows and Mac is that they're pretty consistent operating systems. Like if you're an attacker or a defender, there is a playbook that you could have that would work across an entire fleet.
- [08:53]
B
Yeah, there is. One of the advantages of Windows is I could take an executable file from 20 plus years ago, it would probably run on a Windows system. Today, Linux probably not. Plus again, Windows, you have the Windows shares and things like that. It's very easy to get access to data. Linux systems, they tend to have different Distributions, different CPU types, different configurations. So every time you get onto a new Linux system it's almost like you need to do almost like custom exploitation to access and maintain it. A lot of times more advanced malware, stealth rootkits and stuff like that, they don't work between kernel versions, so you can't just load that stuff up the way you can on Windows systems. So it does tend to take a bit more time and effort and each organization is going to be set up differently in terms of authentication and monitoring and all that other stuff happening. So with the Linux it tends to be more hands on would be my experience. But also because of that, that puts a defender at a disadvantage because each system and group of systems you go into, it can actually be different. So again it's not like investigating a Windows system which is fairly consistent. Each a security team, they might go into an org with 10,000 systems and each group has something completely different from the other one. So it makes their lives much harder too. Which again gets down to why again nin, having a way to scale and automate this type of oversight and investigation on Linux is very, very important compared to Windows.
- [10:09]
A
Right? Yeah. So I guess from a state point of view, if you've got a particular target that is using a bizarre flavor of Linux, well if that's the price of entry, you'll just pay it. Using a bizarre flavor of Linux is no defense at all.
- [10:23]
B
Yeah, yeah, and we've run into that, I mean like real live, like for instance loadable kernel module rootkits on Linux, you know, there are some that were clearly compiled just for the telecommunications industry specifically and for specific kernel versions that just happen to be on their type of switching fabric underneath that they're using. I mean we've seen that type of stuff. But again, as a nation state, you might be willing to put in that time and energy to do it. Where a ransomware operator, they're looking for a quick hit, they might not really be willing to put that time and effort in.
- [10:51]
A
Right, right. A number of people say that the air quotes best thing to happen to security has been ransomware in that it's given a lot of companies the incentive to invest in security because the impacts are real and immediate and you can measure them, I guess with espionage, which is what we're mostly talking about when it comes to Linux, is that fair.
- [11:13]
B
To say it could be espionage? Yep, clear espionage, IP theft, maybe pre positioning tools and access to disrupt things in the event of a war, these we run into and it just depends what it is so for instance, we deal some universities, you know, IP theft is a major, major problem. And then you deal with telcos, they're concerned more about nation state disrupting things. So it just depends on what the industry is.
- [11:35]
A
Right, right. And then I guess in all those cases you don't pay the piper immediately. Like the risk is of a devastating event in the future or the loss of intellectual property which adds up over time. There's no, this is going to cost us $10 million next week if we don't get our security right. So there isn't that same sort of driver.
- [11:57]
B
Yeah. We have seen recently for instance, though some crossover. So past couple months there was an interesting case where someone broke into an IP camera that was running Linux. An IP camera will almost always have a full version of Linux on it, ready to go, and you could cause a lot of mischief on it. And what they decided to do to cause their mischief is they use the SMB tools built into that camera to mount the Windows shares and then they ran the ransomware encryption from that IP camera. Right. So. Right. Even though it was a Linux box, it was able to talk to those windows and that was it. So they still were able to pull it off. But I mean, to your point, it's a mix of things. I think maybe ransomware operators might be pivoting their tactics a little bit to maybe start targeting Linux systems, especially if they have Windows access to it from them. But yeah, but what we do see a lot of, like you said, is going to be these low, slow players who are coming in that could cause a significant amount of problems if they get on the network.
- [12:51]
A
Yeah, I guess for ransomware and cameras there's a lot of security cameras, but there's a lot of them from the same manufacturers. So I guess you could have a small number of techniques that, you know, work and that would get you like, I don't know, maybe 80% of the market or something like that.
- [13:07]
B
Yep.
- [13:07]
A
I was interested to see that in Israel just recently is warning about Iranian hackers accessing Israeli security cameras. So it seems like this is a technique that states are definitely using in wartime.
- [13:20]
B
So there's no question, I mean, in fact, I think there's three Israeli companies that specialize in accessing cameras around the world and you can buy access to whatever geolocation you want. So it's funny that they're complaining about it when I know I'm pretty sure there are at least three that do this. So you could say, hey, I want all the cameras in Paris. And they'll sell you access to those cameras, right?
- [13:41]
A
Oh wow, I hadn't heard of that.
- [13:43]
B
Yeah, it's just one of these things with Linux. I think a lot of times people think there aren't any problems on it because they're just not looking. So it's like, it's like a house of termites in it. And you know, you don't know until one day like hey, there's sawdust in my kitchen, like, well, how to get there, you know, and then you find out under, you look under the foundation, you're like holy smokes, it's, it's a big mess under here. So with the Linux in particular, again, the security teams for Linux, I'd say, I would estimate they're outnumbered 10 to 1. There are probably 10 Windows people for every one Linux person on a typical security team you might run into. So that's why it tends to get a lot of the attention. The Linux teams tend to be woefully underfunded in terms of monitoring the amount of systems and the importance of the systems across your enterprise.
- [14:25]
A
Right, right. I wonder if it's partly that so many of the endpoint detection companies run on mostly on Windows I guess. And so the sort of phenomena has been the talking about security incidents is a form of marketing. And so then you get this snowball effect I guess where the companies that do endpoint detection they work on Windows and so you talk a lot about Windows malware, a lot about Windows nation state activity because that's what's exciting and gets press coverage. And so I guess part of the difficulty is that Linux is so diverse that there's no ecosystem of endpoint detection companies writing blogs about the dangers of Linux malware.
- [15:10]
B
Yeah, we do. Well, we do, we have a blog, we write about it frequently. But I think the other problem with Linux is certainly with an agent based system they want to tie into the kernel, kernel hooks, ebpf, things like that. And that by its very nature drastically limits the amount of coverage they can get. Right. So the Windows EDR products have very, very broad coverage. You know, kudos to them. But on the Linux side, if you're tying into the kernel, I guarantee you, look, if you tried to cover every version of Linux that was out there, your entire budget would be involved with qa. It's impossible. So these companies have to by default lower the number of systems they watch. They also have to lower the amount of things they can actually do in the system because they don't want to impact the performance. They will when you tie into the kernel, but that they know that, you know, that they'll get in trouble if they start doing too much. So I, I think these all kind of mixes together to give security teams really pause. So security team might want to put an agent on, but then the ops team's like, yeah, you know, we've heard some costing 5, 10, 20, 30% plus CPU loads. And the ops team's like, we're not going to do it. I mean, we talked to one customer, they're like, yeah, we loaded the agent and it doubled our development build times. So now the whole dev team's mad at them. Right. So you know that that's where I think a lot of this also plays into it. So that's kind of. We focus on a solution that just says, what if we just got rid of the agent? Right. And at that point eliminate all the performance stability bottlenecks. Now at this point, you go back to the ops teams and back to these developers and other people who are running the business and they say, okay, I could get security. And it's not interrupting us the way these other approaches were. That tends to get their attention a little bit more. I think a lot of these companies know they're not watching their Linux. I just think they don't have a lot of good solutions. That's why we're focusing on it.
- [16:56]
A
Greg Rowland, CEO and founder of Sanfire Security. Thank you very much.
- [17:00]
B
Great. Thank you for having me. All right.