← CyberWire Daily![CyberWire Daily at 10: The vulnerabilities, zero‑days, and hardware flaws over the last decade. [Special Edition] — CyberWire Daily cover](https://megaphone.imgix.net/podcasts/dfa84512-73eb-11f1-aa94-e7e09e93018a/image/9d064a62daa0817d3d0bde95f8f0f94f.jpg?ixlib=rails-4.3.1&max-w=3000&max-h=3000&fit=crop&auto=format,compress)
CyberWire Daily
CyberWire Daily at 10: The vulnerabilities, zero‑days, and hardware flaws over the last decade. [Special Edition]
00:35:20
Loading summary
Transcript94 lines
- [00:02]
A
You're listening to the Cyberwire Network, powered by N2K.
- [00:25]
B
Hi, everybody. I'm Dave Buettner. Over the past decade, we've watched the cybersecurity landscape transform at an astonishing crashing pace. Attackers found new ways in. Defenders adapted. And vulnerabilities, from software bugs to hardware flaws became defining moments that reshaped how we think about security. In this special edition of our CyberWire Daily 10th anniversary series, Maria Vermazes joins me for a look back at 10 years of vulnerabilities. Zero days and the lessons they've taught us. From watershed events like WannaCry and log4shell to the growing impact of hardware security and artificial intelligence. We'll explore the flaws that change the industry and what they tell us about the road ahead. Stay with us.
- [01:16]
A
It is my singular joy and pleasure to welcome once again the host of the Cyberwire Daily, Dave Buettner. Thank you so much for joining me today.
- [01:24]
B
It's always my pleasure. Happy to be back with you here today.
- [01:28]
A
Oh, I'm so glad we get to talk yet again in this year of the 10 year anniversary of the Cyber Wire Daily. We're getting to, I think, the meatiest of the meat and potatoes conversation. I know. I think people have been sort of wondering when we would get to this one. Well, it's here, everybody. We're talking about.
- [01:44]
B
Hold on.
- [01:45]
A
Buckle up, buckle up, buckaroos. We're doing it. Vulnerabilities. Truly the world that so many of us live in, we breathe this. The volumes of the last 10 years. All right? So to set the stage, first and foremost, neither Dave nor I are security researchers by trade. So do we have every single voln of the last 10 years and then some memorized? No, we don't. However, Dave, I know that you have a good sense of sort of the overarching narrative arc of a lot of the biggies, given that you've looked at these and been covering these the last 10 years. So we're gonna stick to sort of the big stories of the last 10 years of Volns. Yeah. So I guess let's start with maybe a scene setting first. Where were we 10 years ago? When you think about the state of
- [02:36]
B
vulnerabilities then, I think 10 years ago, I think most people thought of vulnerabilities as being an IT problem rather than a company wide problem. I don't think companies were thinking of vulnerabilities as being business risk the way that they do today. So some vulnerability would be found and it was up to the IT team to figure out how Quickly it needed to be taken care of, how serious it was and what kind of schedule we could patch that relative to our business needs. And it probably, you know, it was probably a line item in the quarterly presentation that the, the IT folks made to their bosses or the board or whatever and said, oh, we patched this many bugs this quarter and it's great, but I don't know that security was top of mind because we hadn't really seen any of the really bad things that were coming on the horizon.
- [03:39]
A
Yeah. I would imagine adding to that, there was also an element, and this is me editorializing a little bit, but when it would go to someone up the executive chain and go, so this specific situation, this specific vuln that we got to patch, it's going to result in some downtime. I imagine back then they would get a lot more pushback than they might now. And of course, I imagine that may depend a lot, but I imagine making the business case for, why do you have to take this system down to just do a patch? Like, why is that so important? Can't this wait?
- [04:10]
B
Yeah, I think the IT folks were probably considered to be lower on the, the chain in the pecking order, however you want to describe it within the business, because again, we hadn't really seen this possibility that the cyber issues could kill your business. So the stakes weren't as high.
- [04:32]
A
Yeah. And I'm thinking also about timelines for vulnerability, exploitation. I mean, not a secret. In the last 10 years, those timelines have accelerated dramatically. Do you have any recollection of what we were typically looking at 10 years ago?
- [04:48]
B
I think it was much. Well, let's see. Again, not a practitioner, but my sense is it was probably more often than not. We'll get to that when we get to it. In other words, we'll get to that when it will have the least impact on the business. And so I think that meant a lot of things got put off, but. But there probably weren't serious consequences because of that. That was a reasonable plan back then.
- [05:16]
A
Yeah, yeah. So inevitable question here. What changed? Something changed. Can we point to a singular vuln or situation, or was it a cascade thing? What changed?
- [05:31]
B
I think the emergence of ransomware was a big part of it, that becoming its own business. The attackers being able to take these vulnerabilities and use them in ways against companies that were truly potentially devastating to the organizations. The whole notion of reputational damage, which again, is connected to ransomware, it's just much. And also, I guess, awareness among the general public that this is Now a thing that a cyber attack is a thing, because now, as opposed to even 10 years ago, everybody's online, everybody has some connection to something online. There's no escaping it. And 10 years ago, I think it was a little easier to escape it if you wanted to.
- [06:24]
A
Yeah. And going back a little before 10 years ago, going back further than 10 years ago, Heartbleed and was, I think, 2014. And that made a whole lot of people who had no idea what open SSL was. Suddenly everybody had to become an expert on that pretty quickly.
- [06:39]
B
Right.
- [06:40]
A
Especially if you had a business that did anything online, which is most businesses. So, not that I want to say, hey, thanks everybody for naming vulnerabilities, because it made it easier, but maybe that actually did help a little bit. As much as we make fun of that kind of thing. Yeah, it might have actually helped a little bit with awareness. Who knows? Hmm. I'm just throwing that one out there. I'm gonna get hate mail for that one. But anyway, looking back specifically within the last 10 years, if you had to choose maybe one Voln that was the most seismic with its arrival on the scene, is there one that you could point to?
- [07:18]
B
Well, I think the most seismic was probably Log for Shell in terms of how broad and serious it was and how people responded to it. So log 4 shell was, I would say, late 2021, I believe. And it took advantage of what was a vulnerability in log 4J, which is this open source logging framework. Again, not to get too deep in the weeds, but it was a very serious thing. And what made it so serious was how many devices it affected. It was just kind of this thing that was in everything. And so it meant that all these devices were vulnerable to it. I think Jenny Sterley, when she was running CISA at the time, and she said it was one of the most serious that she'd ever seen in her career, if not the most serious. You mean the Federal Trade Commission got involved to. To get companies to update so that they weren't vulnerable to this? It was kind of an all hands on deck kind of moment. In terms of seriousness, I think if you want to rewind the clock to when did a lot of this stuff start to become more broadly known throughout the non IT world? I think you have to look at EternalBlue, which was what, 2017 or so? So Eternal Blue was an exploit that allegedly came out of the US National Security Agency.
- [09:00]
A
Allegedly.
- [09:02]
B
I think that's pretty settled at this point.
- [09:05]
A
Who made stuxnet? Okay, yeah, I think we're good.
- [09:08]
B
Who Knows we'll never know. It's a mystery. And so this was a zero day vulnerability affecting Windows systems. But it got acquired, found whatever, by the shadow brokers who then, you know, famously used it in WannaCry.
- [09:27]
A
Yeah.
- [09:28]
B
Which used the EternalBlue exploit.
- [09:29]
A
WannaCry was a worm and it made you WannaCry. Yeah, it sure did, yeah. Yeah.
- [09:35]
B
And it was also used in the NotPetya attack. So if you want to talk about turning points, I think that certainly was one. And then also the notion that a tool that had been developed by our own intelligence community. Air quotes. The good guys got turned against us, turned against the world.
- [09:58]
A
Yeah. And I remember before WannaCry there was a lot of work being done, I think, within the IT world to try and help practitioners and, you know, IT execs, even message the importance of this is why we need a patch. And, you know, sometimes it fell on deaf ears to the higher up, so to speak. I feel like WannaCry kind of did a lot of that heavy lifting for people from that point. It's like, you know, no security awareness campaign was ever gonna be as effective as touching your hand on the hot stove like WannaCry was.
- [10:30]
B
That's right, that's right, that's right. Yes. I'm laughing because as the father of a teenage goes through life touching hot stoves and being the only way he can learn anything, you know, that resonates with me.
- [10:48]
A
I feel like, truly, it's a very human thing. Very, very human. Yes. Yeah, absolutely. We warned you, we warned you, warned you. Oh, now you gotta learn the hard way. Oh, dang it. Now we've all learned the hard way. Yeah, But I mean, WannaCry and truly, if we bubble that up to EternalBlue in general, I remember there was also the geopolitical aspect. You touched on this a bit as well, about governments having zero day stockpiles and what does that mean? What's the danger there? That's an ongoing conversation too.
- [11:19]
B
Yeah, it was eye opening. You mentioned stuxnet before. And every now and then one of these makes its way out. It's a question that I've asked folks. I interview some of the researchers, particularly for our research Saturday show. And it's usually a question I ask off the air, but it's.
- [11:43]
A
Yeah, some of the interesting stuff comes out.
- [11:45]
B
It's for my own curiosity and people feel more like they can share more, but say, how often do you come across something that has all the signs of being a piece of United States offensive trade wear or.
- [12:02]
A
Yeah, yeah, yeah.
- [12:03]
B
And they'll say, yeah, it happens. You know, there are. Yeah, of course it happens.
- [12:11]
A
That's why espionage exists. Right?
- [12:13]
B
Yeah, right, exactly. So, you know, we joke about what should we call it, you know, Eternal Eagle or something like that. Some very stereotypically American name, you know, Righteous Eagle or something like that. But anyway.
- [12:30]
A
Yeah, well, going back to touching the hot stove for a second also, there was what the sort of overall business community learned, I think from WannaCry. But what would we say, sort of the takeaways for the security industry were from that?
- [12:44]
B
For WannaCry?
- [12:46]
A
Yeah,
- [12:48]
B
I think it was a message of responsibility. Who's responsible for these things? Because in this case, this was affecting Windows systems. So to what degree is this Microsoft's responsibility to get patches out there to remediate this? But then to what degree is it the responsibility of agencies or researchers who stockpile these things, who know they exist, and rather than disclosing them, either sell them or keep them or save them for whatever use that they might be useful for in the future? What's the moral and ethical thing to do in a case like that?
- [13:36]
A
Hmm. That is a question for much smarter people than us.
- [13:40]
B
Yes, I concur.
- [13:41]
A
It's a good question, though. Yeah. So we've been talking about these paradigm shifts and vulnerabilities and typically when we talk about vulns, it's a software problem. But another huge paradigm shift in the last 10 years was when we saw vulns that were hardware problems. And I'm thinking of Spectre. That was massive. It's not just because I'm married to a hardware guy who works at one of these companies. Although, full disclosure, I am, but the. I mean, that was. That was a really. That was. I remember felt actually kind of scary on a level of, oh, gosh, I didn't. Were we really thinking about that? I thought, you know, hardware issues were the realm of the extraordinarily nerdy security through obscurity. Why do we have to even think about that? We've got enough problems with software. And then came Spectre and meltdown.
- [14:26]
B
Yeah, I remember when Spectre came out, my first thought was about a bug that affected Pentium processors. Back in like 1994 or 1995, there was some issue baked into certain intel processors, Pentiums, that would return floating point results that were wrong. And we kind of count on processors to get math right. Right.
- [14:57]
A
A little bit important.
- [14:58]
B
Yeah. And you know, it was one of those things where it was a very rare bug. It probably wouldn't happen very Often. But there was also a way that if you fed it this exact very simple math problem, it would give you the wrong answer. So it was very easy to illustrate, which made it easy to understand, which made it easy for people to be nervous about it. And intel did a recall and we all lived to tell the tale and live another day. So that's what I was thinking of. Are all of these chips going to have to be recalled and figure the difference between 2020, or, I'm sorry, 1994 and 2016, wherever Spectre and Meltdown were. That's a lot more CPUs out there. That's a lot of hardware.
- [15:50]
A
What's going to happen and how do we fix it? How do we easily patch that at scale?
- [15:55]
B
Right? Because hardware's hardware. Unless talking about, I don't know, field programmable gate arrays or something really down in the weeds, your processor's. Your processor. And so over time, all these assumptions had been baked into modern processor design and were tried and true, and suddenly you discover a vulnerability. What does that mean? And what it meant in the short term was that the patches made the processors slower.
- [16:28]
A
Cause, well, you're adding something to it, right? I mean, yeah, you're adding something to
- [16:31]
B
it, but you're also taking something away. Like there was this, my recollection, and I'm sure this is imperfect, but my recollection is that it had something to do with the predictive nature of how modern processors flow information through them. And some of the assumptions of what you could and couldn't do, that's where the vulnerability was. And so you had to basically disable some of those predictive presumptions. And that slowed things down. So now your server rack that yesterday you were selling that had these capabilities, had those capabilities minus 20%. Right. And so what do you do?
- [17:15]
A
And it also zooming out a little bit really changed the conversation around what we sort of consider the arena of fair play for what needs to be actively monitored. I guess. I mean, maybe that's the wrong phraseology there, but I just, I don't remember the average bear talking about, you know, hardware vulns as much. And now it's like, yep, that's part of what we gotta be thinking about more regularly as opposed to, you know, that's a weird anomaly. And just don't worry about that. Don't look in that corner. Yeah, it's just part of the field of play now, I suppose, I guess,
- [17:51]
B
and I wonder too, how much of it is just that, like, we're at the point where so much of the low hanging fruit has been picked. So you're getting into more of these edge cases now, I guess. And I know they'll take away our broadcasting licenses if we don't mention AI in this somewhere.
- [18:12]
A
Everybody drink.
- [18:13]
B
Right. But I think AI has replenished that fruit.
- [18:24]
A
We'll be right back. We probably don't want to leave AI to the very end.
- [18:40]
B
Okay.
- [18:40]
A
Yeah. No. Although I kind of want to because there's so much that happened in the last 10 years before AI really shook things up. But yeah, I mean, it's, it's. AI is, is really adding. I don't want to just say it's changing the paradigm. It's adding so much into the pipeline that it's just kind of hard to know what to do anymore. It's hard to know what needs to get prioritized. It's hard to know how to make sense of just the pure volume of it without saying, okay, AI's finding all these problems, let's let AI fix it. But where's the human in the loop on that? And it's just, that's. I mean, I think that's where we're all at right now.
- [19:17]
B
I think so. To me, a big part of it for people who are fighting the good fight out there is it's a velocity issue. That stuff's just coming at you so much faster. And for me, like, you know, I, back in the day, I used to enjoy playing first person shooters like Marathon or Doom or you know, that first generation and the first version of Halo. Right. I can't play them anymore. I can't watch my kids play them because they're too twitchy, they're too fast. And I feel as though I was wired up right. My first exposure to these was at a certain speed. And that speed is way faster than it used to be. And so it's hard for me to adapt. And the next generation coming up, that's all they know. So they can handle the speed. They don't think twice about it. And I wonder if that's something that we're experiencing right now where you have generations of researchers, people who came up with a certain cadence of patching, of remediation. All that's been thrown out the window because it's like what's the thing in Mario Kart where you get the speed boost? Right. It's like that.
- [20:34]
A
The mushroom.
- [20:35]
B
Yeah. And so how is it just the old timers who have to catch up and the next generation is going to be fine with this new velocity because it's all they know. Or as many people are saying, the only way to do this is machine versus machine, where the AI is the only thing that can be fast enough to keep up with the AI.
- [20:58]
A
Yeah, it almost makes me wonder if someone listening to this conversation a few years from now might go, oh, how quaint. They're talking about singular named vulnerabilities. Who has time for that anymore?
- [21:10]
B
They're talking about humans patching, you know. Oh, what was that like? You people actually, you had keyboards, you touched your computers. Ew, yuck.
- [21:19]
A
And also that you had the time things were slow enough that you were looking at one or a group of things at a time as opposed to just a torrent. I mean, I know Voln's and patching, I mean, the report lists are hundreds of pages long and nobody knows every single one that gets patched. But at the same time, okay, that's not true. But the idea of like we're thinking about one specific Voln as opposed to, you know, tens of thousands at that, the volume is just a total different scale. We're going to be talking about things in scientific notation instead of, you know, just in dozens.
- [21:54]
B
Yeah, like we thought we were in an age of automation. Like, hold onto the bar because here comes a new age of automation that is driven by necessity.
- [22:05]
A
So, okay, so as we're talking about all of these huge paradigm shifts over the last 10 years as we reflect on vulnerabilities, I would be remiss if I didn't also mention supply chain attacks. That was another huge shift in the last 10 years. And the biggie was solar winds. And that's the one that, you know is synonymous with supply chain issues. What's your recollection of how that went down when that started out?
- [22:30]
B
Well, I think people were whistling past the graveyard. Right. For a long time before that one landed. And it was proof that it wasn't a matter of if, it was a matter of when. So solar winds hit and it was a massive supply chain impact all over the world and affected real people. Couldn't use their gas pumps. I remember in the southeast of the U.S. here, people were hoarding gas. So real world, regular people, repercussions of this, and it got everyone's attention and served as, I guess, the proof of concept that supply chain vulnerabilities are a thing and they need to be taken seriously. And so I think it kicked off a whole era of supply chain research of things like SBoM initiatives, software, bills and materials, triggered a bunch of scrutiny of open source Ecosystems.
- [23:38]
A
Oh gosh.
- [23:39]
B
Everybody started thinking about what's in my system that I don't know is in my system. Right. It's the. What was it for? The unknown. Unknowns.
- [23:49]
A
Thank you, Rumsfeld.
- [23:50]
B
Yeah, because nobody's, you know, nobody is. I'm sorry, I shouldn't use absolutes. Most people are not creating software from whole cloth. They're using open source packages. They're using because why not? Who has time to write everything that's just efficient?
- [24:08]
A
I mean, that's how things are built now. I mean, it's how it's always been, really is my understanding. I mean, you don't have to recreate everything from scratch. There's a perfectly good library or something else that someone has done. You build on that. That's how we make progress.
- [24:21]
B
Yeah. So if you have these dependencies and you've been using this little nugget of open source software for the past five years and every time a new version comes, you wait a few days to see if there are any major issues and then you just slide it into your own production environment, no problem, business as usual. And then all of a sudden somebody can take that, put something bad in it, and without even realizing it, it slides into your environment and now you've got a problem.
- [24:51]
A
And going back to AI, we're seeing such an influx of people vibe coding or trying to contribute to open source projects using AI. What we're talking about right here is part of that conversation for why that is so dangerous. You know, we have the idea of people making contributions to these extremely important open source projects that a lot of our modern world is built on. But who's actually able to keep track of what's being contributed if it's being done through AI? And understandably, some open source project maintainers are saying no AI contributions whatsoever. But this is part of the reason why there's a lot of paranoia understandably about that. Because who knows what's being introduced that way. It's wild seeing these things converge in real time. Wild, scary fun. Interesting. I suppose speaking of people in business truly and speaking of wild fun. Interesting and all those things. A story that honestly, before we started doing research for this, I had actually a little bit forgotten about and I cannot believe I forgot about this. Bloomberg had this huge bombshell story about Super Micro and that set off this sort of feeding frenzy to chase that one down and it ends up that that one wasn't real. And if you look for it though, it's still there on bloomberg.com so it's I. What the heck happened here?
- [26:16]
B
Well, it was 2018, I believe, that Bloomberg published this story. And the allegation was that Chinese operatives had inserted these tiny little hardware devices, little tiny chips, into super micro motherboards.
- [26:35]
A
Right? Yep.
- [26:36]
B
And so this was a supply chain compromise and a degree of sophistication we had not seen before.
- [26:48]
A
And a nightmare situation that, you know, all this physical hardware has been tacked onto all this incredibly important stuff on the motherboard, like that's a nightmare.
- [26:57]
B
Right. And these super micro servers are everywhere in the government and, you know, very popular, good brand, you know, all that. They got dragged into this. And so everybody started looking for what's going on here. Because if this is truly a hardware bug. Well, the hardware exists. It's a real thing. It must be on the motherboard. And nobody found anything. There was never, never any public evidence that supported the central claims.
- [27:32]
A
Yeah. And I remember the initial reaction from a lot of folks in sort of the federal sector going was just how the heck did we miss this? Like, this is the kind of thing we're looking for. How on earth did we miss this? And how did Bloomberg find it first? And the answer was that you guys didn't miss it. But that was a crazy cautionary tale. And you know, that, you know, everybody kind of, especially in the media sector, we all want to be the ones to report on the. The new. The new, you know, hot Vaughn, because you want to be the place that breaks that story. That's a great way to be in front of stuff. But this one didn't exist, so it's horrifying.
- [28:15]
B
And do we think Bloomberg was acting in bad faith? I don't think so, but they got it very wrong. Whatever internal checks and balances, they must have had failed them, in my opinion. And remarkably, what's been eight or nine years now and there's never been a serious retraction from Bloomberg like you said, the article is. You can still go read it.
- [28:47]
A
So, Dave, this is. We've only taken a very top line view of the many, many Volns over the last 10 years. Certainly this is not an all inclusive look. We could not possibly do that in even just a few hours. It would take us days. You know, it's been a good 10 years of you hearing these stories come and go. Some of them, some of these Volns have a pretty short life cycle. Some are ongoing still, like log four shell, for example. Like we're still dealing with these. What sticks with you as you look back on these?
- [29:17]
B
Well, I think the thread that runs through all of this is this notion about having our assumptions challenged. Right. Like we talked about at the beginning, people assumed, rightly assumed that they had time to patch. People rightly assumed that software was trustworthy. People rightly assumed that hardware was trustworthy. They thought they had time to take care of things. And each of these events have challenged those assumptions. Who do you trust? To what degree do we have to scrutinize software or rely on other people to do that for us? To what degree could something be lurking in our hardware? We have all these assumptions and ultimately, at some point, you have to trust things. You have to trust people. That's how the world works.
- [30:18]
A
I can hear a few people going, no, no, I won't.
- [30:21]
B
Right.
- [30:21]
A
Well, there are a few folks that feel that way. That's true.
- [30:24]
B
Yeah, yeah. And you know, trust but verify. Right. All that stuff. I mean, there's only so much you can do, but at some point you have to believe something. Right?
- [30:37]
A
Yeah, no, no, you're right. You're right.
- [30:39]
B
It can be as simple as when I flick the light switch, the light comes on. Right. Like you have. There are. And so I think it's. Over the past decade, we've seen many of our assumptions fall away. And we've learned that things that we used to assume were true now require an additional level of scrutiny, additional level of care. Double checking all of those things, assumptions
- [31:03]
A
we didn't even know we had, in some cases just. They were so baseline that we just. It was like breathing. We didn't even think about it. And now it's like, oh, we have to examine that as well. Oh, my God.
- [31:12]
B
Right? And things have gotten so much more complex. Things are so much more interconnected than they used to be. And now we've thrown AI into the mix, which has put us all on turbo speed and. Yeah, what a world.
- [31:29]
A
What a world. Well, we'll be here for the next 10 years too, I'm sure, Dave. And thank.
- [31:35]
B
I don't see it slowing down anytime soon. And it's always something interesting. That's the thing about this business. Right. There's always something to learn. There's always something new coming and surprises. And so it never gets old.
- [31:53]
A
That's an awesome way to end it. So. Dave Buettner, host of the Cyberwire Daily, thank you very much for joining me today.
- [32:07]
B
My thanks to Maria Vermazes for joining me as we continue looking back on a decade of cybersecurity, stories that shape the world we defend today. Thanks for listening to this special edition of the CyberWire Daily's 10th anniversary series. We'll be back with more conversations exploring the people, the moments and the milestones that have defined the past 10 years of cybersecurity. I'm Dave Buettner. Thanks for listening.