Loading summary
Transcript34 lines
- [00:00]
A
Foreign hey, this is Casey Ellis for Risky Business and today we're talking to Firos, a booker DJ from Socket. For those who don't know, Socket is a software supply chain security company that protects over a million repositories and counts Anthropic, OpenAI and Figma amongst its customers. Frost built WebTorrent, maintains over 100 npm packages to get downloaded a billion times a month, and teaches web security at Stanford. So we're talking to the right guy when it comes to open source dependencies and the security problem overall. So today we're going to talk about the impact of AI and kind of the agentic world that we're living in at this point in time we're transitioning into and how they're seeing some of the effects of that in the open source security problem space. A couple of other things that have been happening in and around the Internet when it comes to open source over the past little while. And we'll land on a new product launch that for us is very keen to talk about. So good to see you, mate. It's been a while.
- [00:54]
B
Yeah, good to see you, Casey.
- [00:55]
A
This is going to be fun because I think this is a latent trash fire that the Internet's kind of collectively waking up to at the moment. Right?
- [01:04]
B
Yeah, I mean, this has been a smoldering trash fire for a little while now, just with kind of how developers deal with open source software and how they know what's safe and what's not safe. And I think AI is just pouring fuel onto that fire. So excited to talk about it with you today.
- [01:18]
A
Cool, let's get into it. So you're seeing inside of some of the leading AI labs and a whole bunch of large enterprises, I guess. What does agents writing 90% of the code actually look like in practice when it comes to particularly open source, but just in general as well?
- [01:32]
B
Yeah, I think we're seeing a really big structural shift. Agents now write the majority of production code at the leading AI labs and I think this is only kind of spreading downstream to kind of the later adopter companies as well. In some of the leading AI labs we're seeing 90% plus of new code written by agents. And of course, like you said, we have a lot of the leading labs as customers, anthropic cursor, OpenAI scale, AI XAI, et cetera. And so this is obviously, I think, probably one of the biggest shifts we've seen in the way that software gets written ever. And if you think about it, just how many shifts like this do you see in a lifetime, you see the Internet, you see mobile, you see cloud, and I think AI. So we're really dealing with something here. And what's interesting is it wasn't like human developers did a great job with avoiding vulnerabilities and especially dealing with dependencies beforehand. And so we're seeing dependency counts at these customers really, really high. Like they're in the mid hundred thousand range of dependencies. And we're seeing agents pulling in packages automatically and dependency graphs only getting larger and larger over time. So I think the core question is how do you enforce safety when neither humans nor agents can realistically reason about dependency risk at the time they're installing that code?
- [02:52]
A
Yeah, absolutely. And I think you call out a really important point there as well. I mean, partly. One of my favorite quotes around AI that I heard a little while back was AI in 2023 was kind of like having a website in 1997. And this is from a guy who went through the dot com thing kind of firsthand. So this idea of it being pretty much a transformative across the board technology that's going to have this massive hype cycle, but does go on to pretty much adjust how we do a lot of things from a technology and even just a straight up living as a human standpoint, it's definitely in the process of starting to do that now. I guess the bit I want to index on just a little bit there. When you're talking about the frontier labs and 90% of their code being generated by AI, what are you seeing in the enterprise? And I'll load this up a little bit by saying I do think that a lot of the development work and kind of the different kind of noble practices for enterprise software development that are going on inside the Frontier labs are actually pretty predictive of where we're going to eventually end up. Having worked very closely with them myself over the years. But where are we up to in terms of adoption on the enterprise side?
- [04:04]
B
I mean, so I've been at dinners with CTOs and CISOs and we've gone around and folks have said the percentage of code that's written by AI inside their organizations. And I hear even in curmudgeonly companies that you don't think would be up there, you hear numbers like 40%, 50%. So they are only trailing the labs by maybe a year or two, maybe even less. So I'm actually surprised at kind of how much the typical conservatism has been,
- [04:33]
A
I guess maybe forced, kind of gone out the window, right?
- [04:35]
B
Yeah. It's like CEOs and boards are saying, what are we doing with AI? How are we going to adopt AI faster? And I think that folks have just been forced to kind of say, all right, we're going to not worry about security for a little while here. We don't want to get left behind. We need to figure out what this is about. And so that's where I think the challenge comes in, is you want to adopt this stuff, people want to adopt this stuff, there's clear demand, but then how do you do it safely?
- [04:55]
A
And there's probably a degree of competitive pressure in the mix there as well. Right. As you're saying that, I'm thinking about all of the very rapid changes enterprises made when Covid hit and everyone shifted to work from home. It was like a bunch of fairly deeply unnatural things in terms of giving access out to remote employees and all that kind of stuff. But then the quick follow up to that was a lot of pretty quickly built customer access systems. And it's partly environmental like Covid and the pandemic is a thing that was affecting kind of everyone at the same time, to me has some similarities to AI. And this idea of not getting left behind, do you see that as driving some of the adoption on the enterprise side, I guess is the question there?
- [05:38]
B
Absolutely, yeah. No one wants to get left behind. And so folks are just letting loose. And I think we're seeing kind of the predictable impact of this because even before, before AI and agents and AI coding tools, humans weren't very good at, I mean humans weren't the best at evaluating open source dependency risk, which is the area that we focus at socket, not to mention avoiding writing vulnerabilities into code. And I think there's always been this pressure on developers to optimize for velocity and functionality and not necessarily think through adversarial scenarios and especially reviewing third party code. I mean no one, I mean nobody does that. Like really it's a hard problem. And so, and then you see kind of like this focus on the, on, you know, the legacy SCA tools are really focusing on CVEs and licenses and they don't do anything to address malicious code risk, supply chain attacks and that kind of more holistic risk. And so I mean many organizations, I think they have tools that can help in theory. I've seen folks that have, you know, this or that tool that supposedly helps with these kinds of problems. But rollout is super fragmented across different source control systems, build systems, and so practice, I think we see there's really no single control point that guarantees that your humans and your agents aren't going to cause you to have a really bad day.
- [06:57]
A
So really what I'm hearing you say there is that we've taken something that we're already not very good at and given it a bunch of caffeine and a puppy.
- [07:04]
B
Exactly.
- [07:05]
A
And said go at it before AI. I guess how this kind of preempts one of the questions I was going to ask, but leading into that, because you've had obviously a bird dog seat on all of this over the years. As I mentioned in the intro before AI kind of came in and started to accelerate all of this. How are we doing from a dependency hygiene and a dependency risk management standpoint, realistically?
- [07:27]
B
Well, there's been a few trends that have kind of really affected the way that companies are using open source software. I think the first one is there's way more open source than before and those are in tinier and tinier components of functionality, so tinier packages. And what that means is you're depending on a lot more individual people, individual teams or maintainers across the world. And so that's just created this huge vector into the organization so you can take over any one of these packages and suddenly be in all the top companies. And so over the past few months this is really what we've seen. And actually, really through all of 2025, there's been this sustained wave of high impact supply chain compromises affecting foundational packages with millions or in some cases billions of monthly downloads. We're seeing all kinds of vectors for the way these are taken over, whether it's phishing or GitHub Actions flaws or credential theft to hijack the maintainer accounts. And then really what they do is they do all that to publish a malicious update. And so then anybody downstream who's depending on that code, they're pulling in that malicious code. And in multiple cases we've seen kind of the effects of this be anything from harvesting tokens, API keys, cloud credentials, SSH keys, environment variables, crypto theft, CI secrets, and then of course the most interesting stuff I think has been the worm variants that we've seen. So this is something we've known about for a really long time that this could happen. But to actually see kind of the first few instances of open source worms where you install a malicious package and then suddenly all of the code that you have access to, all of your packages get compromised. And then we've seen literally thousands of packages get taken over within the period of Hours and it's just the scale is really unprecedented. So anyway, we're not doing a good job, I'd say today. Not because it's an easy problem to solve, but it's definitely a growing issue.
- [09:22]
A
I kind of wonder if in some ways we'll come back to the recent attacks and the worm stuff in a second because I'd like to unpack that a little bit. But thinking about it in terms of a lot of the focus around the use of AI and the kind of tooling that exists around AI developed code, it's very much emphasizing the whole shift left thing. It's like let's help developers in the IDE or let's do vulnerability assessment in code or dynamically more effectively, which is great. But those are the first kind of five turtles of a stack of 30 turtles and open source is kind of below those five.
- [10:00]
B
Yeah, I agree. I mean, I think that the shift left thing, it's still the right idea. I think that we just have to shift left even further than we thought before because for a while it was considered good enough to be in the developer workflow as far as in the IDE or in the GitHub pull request or workflow where you're giving feedback to developers inside the pr. And that's great, but that just isn't early enough anymore. I mean, if you think about the ways that that fails. You have agents that are being tasked with building entire websites, entire subsystems. Those agents are installing open source dependencies and they're making decisions about what code to bring in. And that's going to affect the developer laptop, it's going to affect the developer machine, the build servers. It's going to go into prod. You also have data scientists and AI developers who are working in jupyter notebooks and they're working on code that may never get pushed to GitHub, it may never get scanned therefore. And so you just have this whole kind of almost invisible supply chain of your data scientists and it's all on the local machine. And so that's where you really actually have to think about how do we protect all the machines and how do we protect all the third party code that even makes it into the organization? And that's why I'm excited toward the end of this to talk about the socket firewall and share that with folks.
- [11:15]
A
Yeah, definitely. We'll come to that in a moment. So I guess you've got all these different folk that are moving faster, they're focused and prioritizing, just making the damp thing Work and not necessarily ignoring or being kind of ignorant when it comes to security. But obviously if there's a priority focus at that point in time, that outcome is not necessarily one that's difficult to anticipate. So I guess the task at that point becomes to make insecure as obvious as possible and secure as easy as possible. I guess talking about going back to the attacks piece where that's failed and kind of evidence of the fact that this is actually an important problem to solve. Do you want to, do you want to double click on that a little bit?
- [11:53]
B
Yeah. Well, first of all, I mean AI agents don't have, and AI coding tools don't have an idea of like what the latest security state of an open source package is. And if you think about the like, one way I'd like to describe it is, you know, there was old school antivirus that had signature based scanning and then you know, we had the crowdstrikes and the kind of modern EDR come out and they look more at the capabilities, the behavior, like what is this file going to do when it executes? And they observe kind of the behavioral characteristics. And I think that's kind of the shift that we are doing with sca, like old school SCA is just literally looking down a list of dependencies and saying These are the CVEs that affect it. And what we've done with Socket is kind of looking at the behavior of the package. What is it going to do when you run it? And that's really what's been missing. And when you start to look at packages that way you start to find a lot of stuff. Part of it is there's been an acceleration, but then part of it is just no one's been looking for this stuff. And so we're finding now around 10,000 malicious packages per week across the ecosystems that we track. And this varies. Sometimes it's 40,000, sometimes it's 8,000. It really, it can be quite spiky. But the way to kind of model this is open source code is really just, it's like a big wiki and anyone can put anything up there. And yes, there's some benefit to it being open and folks can audit that code and ultimately maybe find vulnerabilities and find issues. But the reality is everyone is expecting others to do it and they assume someone else must be taking a look at this code. And, and it's really interesting because when everyone makes that assumption and then you don't really get any kind of herd immunity from others doing that work and you're really oftentimes the first person to be installing that code or everyone is getting hit at the same time. I'm happy to walk through maybe one of the interesting attacks if it's helpful to get an idea of what this looks like when it happens.
- [13:42]
A
Yeah, what happens when it goes bad? And I think just tapping back real quick, fully agree on the whole mini eyes doesn't necessarily make all bugs shallow thing because we saw that in early bug crate days. It was around the same time the heartbleed and shell shock and some of those kind of early kind of Internet trash fire vulns hit and they'd been sitting around the code base for a long, long time. And in the meantime bugs, when it came to functionality and features and kind of, you know, increasing the utility of these libraries, they'd been pretty actively worked on and pretty actively solved. It's just that, you know, security issues and some of these kind of anti patent things weren't necessarily being incentivized or looked at in the same way. So. Yeah, but yeah, you want to come back to. Yeah, so Shayalud, I always pronounce that wrong, but the worm.
- [14:31]
B
Yes. Yeah, happy to walk through it. But just on your last point, I think it's interesting that there's this trade off between fixing vulnerabilities and being affected by supply chain attacks. So we've been as an industry trying to get folks to update their software for so long and now folks are actually doing that. But the faster you update and the faster you resolve those CVEs in your open source dependencies, the more you're pulling in code that has not been vetted by anybody. Right. It could be that you're pulling in code that was published yesterday and no one has looked at it. Like no one has audited it, no one on your team has looked at it. So this is where you start to really see that it's a direct trade off between volumes and supply chain risk. And so I think, you know, that's where we try to come in and be like, you can actually have the best of both worlds. You can, you can be super up to date and you can just make sure that Socket has said that, you know, this package is safe and we've done our vetting and, and, and, and, and so, but yeah, let me tell you about the worm because I think it's a super cool story.
- [15:24]
A
And yeah, and I'm, I'm working, I'm working back through the queue a little bit here. Let's, let's just touch on the worm real quick, quickly. But then, you know, pretty much what you described there, I understand, is exactly what you've sought out to go off and solve with your new product. So we'll touch on the worm a little bit and then we'll loop around to that.
- [15:40]
B
So Shaihalud was a, it's probably the biggest supply chain attack we've seen in open source in terms of scale and reach and number of organizations affected. It was an NPM worm and we had thousands of packages that were taken over in a matter of hours. The way it worked was anyone who installed one of these malicious dependencies during that window was infected and it harvested environment variables, AWS secrets, GitHub tokens, and then the key thing was it propagated by compromising additional packages. So you saw a lot of companies affected just because somebody on the team was unfortunate and unlucky enough to run an install command during that window when those compromised malicious packages were live. And this is kind of like, kind of what we created the company to solve is that even when that code is live, you should not install it. You should know that it's malicious and you should know that within seconds of that artifact being first published. But this is what happened. So it was something we've known could happen for a while, but to actually see someone go and implement it and succeed at this scale and really take over so many companies packages, it was, it was really the case that during that window, anyone who installed anything, you were going to be affected because these supply chains are so deep and these packages have such broad reach and they're so deep in your supply chain. It's almost like it's foundational Internet infrastructure. Just to give you an idea of what I mean by this, there's one person on the socket team, one guy who we recently hired, who alone is personally responsible for 15% of all NPM download traffic just with the packages that he controls. So obviously that's an extreme example, but this is the reach. If you compromise a single developer, this is the kind of scale, it's not like a matter of when is your organization going to be hit, it's just you will be hit. Sorry, it's not if, it's when you're going to be affected if one of these widely used packages is taken over. And one anecdote I want to share, because I think we were talking about it before the show is before we started recording, is I just spoke to a Fortune 5 company and I'm not going to name them because I wouldn't want to do that to them, but they were telling me kind of what their response was to the Shai Hulud worm. And I just think it's very illuminating because I think a lot of other organizations are in the same shape, in the same position. So when this was happening, they just sent an email to their entire engineering team and mind you, this is a fortune fit. Five companies. So this is like tens of thousands of engineers. They did an org wide engineering email blast and told everybody, do not install anything today. Nobody install anything today. That was their mitigation to this. They didn't have any kind of centralized control that they were confident could actually block the malicious packages. And obviously this was also a developing story with more and more stuff being discovered and our blog was covering it. And the lists of packages kept expanding as we learned more and more. And then the next thing they did was after things had settled down and the incident had passed, they sent another email and told everybody, hey, if you think your team was affected by this, email us and we'll help you to investigate whether you may or may not have been affected. Which again is like you'd hope there would be some kind of visibility, some kind of tooling that could just tell you, did anybody install this? It shouldn't be this hard. And so there was no instant visibility and there was no ability to enforce a global block on these packages. And this is exactly what we solve.
- [19:12]
A
So tell us how you solve that.
- [19:13]
B
Yeah, yeah, sure thing. So Socket, we have a new product called Socket Firewall and this is really exciting because it basically brings the kind of core value and intelligence of Socket to a different part of the stack. And so the way it works is it's kind of a guardrail, a default guardrail for your entire software supply chain. And it works really well for AI built software and agentic software. And what it does is it blocks malicious packages at install time. And so every dependency, whether it's human or AI installed, is going to flow through the firewall and malicious artifacts will never reach laptops, CI systems or production. There's a couple ways to deploy it. You can do kind of a lightweight CLI endpoint integration so that whenever a developer or an agent goes to run NPM install or PIP install or any of these package manager commands, the firewall spins up and it basically proxies all the traffic and makes sure that nothing that is against your policy. And that would obviously include things like malicious packages, untrusted maintainers, it could be vulns, it could be licenses, whatever you Care about nothing gets onto the system. And then the other place we do it is kind of network level enforcement as well. So we can kind of redirect the registry URLs. So any system on your network that is trying to access NPM or the Python registry or Maven Central, you just kind of DNS redirect that to the Socket firewall and everything works exactly the same, except no malware gets in. So it's really simple to roll it out. Really, really straightforward. And we've seen a lot of success rolling this out.
- [20:46]
A
That's awesome. Definitely, in terms of the ability to just sit in the middle of the process and intercept in that sort of way. Going back to what I was saying before about make insecure obvious insecure easy seems like a fairly elegant way to solve that. In terms of the actual blocking and the intelligence itself, are you actually passing the code as it goes through and doing that on the fly or is that kind of threat intelligence led outside of the team? Is it a combination of the two? I'm curious kind of how that part's working?
- [21:16]
B
Excellent question. So this is one of the main advantages of Socket and why we are able to find so much malware and so much bad code is we're not doing this with one off threat research. A team is just sort of out there looking for things and they might find something, they might not find something. At Socket, our threat intelligence is our product. It is like weaved throughout the entire platform. And so what I mean by that is Socket has an index of every piece of open source code that's ever been written. And as new code is published, we also pull that in within seconds. So you can kind of think of it like how Google has a full copy of the web. We have a full copy of all open source code and whenever new code is published, we immediately start analyzing it, scanning it, we look for about 70 signals of supply chain risk. That's everything from maintainer history behavior to the characteristics of the code, the capabilities of the code, what is it going to do when it runs, Even things like metadata around the code. Maybe the name is similar to another package name, but the download counts are really different. So you have a typo squat risk. And then we take all that and we do that across all the code. It's actually quite a lot of code to analyze. I can imagine. Yeah, yeah. And then we put it into kind of this AI system we've created that is really good at surfacing kind of the interesting threats for our human security team to look at. And so we do a really good job at Socket, this AI human hybrid. So nothing gets out to customers unless a human security threat analyst has signed off on that finding. And despite that fact, we're still finding we have hundreds of thousands of threats identified with the system. It's really, really good at finding risks. So what kind of all boils down to, Casey, is when somebody goes to install something, we're not blocking that installer, delaying it. The data is already there and we already kind of have the answer of whether or not it's safe.
- [23:07]
A
Yeah. You shared before we started a fairly extraordinary story in terms of the, I guess the proof of pain that exists in the market that this is sort of stepping into. And I think in terms of this being a problem that is a smoldering trash fire that is starting to become kind of real and kind of the market response to that. I think it's a pretty good anecdote of the fact that that's actually now very much on the move. Do you want to share that one? We'll close it off there.
- [23:32]
B
Yeah, sure thing. I mean, so the anecdote is pretty wild. We basically recently rolled Socket firewall out at a top four US bank. And we did this in a period of two months, which is unheard of, you know, from, from, from kind of beginning to end. They skipped a POC like the pain. The pain was so real for them that they were willing to kind of do anything possible to accelerate this. And at the end of that, you know, they were congratulating us because they'd never seen a vendor able to go through their process in less than nine months. So it was pretty special.
- [24:06]
A
That is, that is one heck of a sales pitch, like, you know, both for the problem that you're out there solving and the way that you guys have approached solving it as a product as well. So congratulations on that. It's going to be really interesting to see how this plays out over the next year because I'm sure there's going to be a lot of action in this space.
- [24:22]
B
Yeah, 100% agree. I think it's a really interesting time and, you know, just seeing, seeing kind of, especially as AI is finding more vulnerabilities. You know, we didn't, we didn't touch on the, The Claude Opus 4.6 announcement earlier this week, but it's just such an interesting time with all the, all the developments and. Yeah, what a great time to be alive.
- [24:40]
A
All right, so for us @booker dj, thank you so much for joining. This has been Casey Ellis for the Risky Business podcast. This is a sponsored interview with socket.dev check them out at their website.
- [24:50]
B
Cheers. Thanks, Casey. This was super fun. Thanks for having me.