The Hidden Flaws in AI Security Nobody Saw Coming
Loading summary
Steve Gibson
It's time for Security now. And this is a very special episode. You're going to be glad you're here. We are at the Black Hat conference in Las Vegas. Steve Gibson's here with questions from you, the audience. We're going to primarily talk about AI and security, and we've got some very special guests joining us. Security now is next
Leo Laporte
podcasts you love
Steve Gibson
from people you trust.
Leo Laporte
This is Twit.
Steve Gibson
This is Security now with Steve Gibson. Episode 1090, recorded Wednesday, August 5, 2026. Black hats, it's time for Security now, the show. We cover your security, your privacy, and how things work online. Hello, everybody. Yes, it's a little bit different, a little bit noisier because we. And Steve can do that, which he could never do until recently.
Leo Laporte
I could touch him.
Steve Gibson
It's the miracle of sitting next to each other.
Paul Thurrott
No touc. No touching.
Steve Gibson
We are at Black Hat, the big security conference in Las Vegas, Nevada. Special guest of Threat Locker. Thank you, Threat Locker, for flying us all here. And Steve is here, but Steve is not alone. Say hi, Steve.
Leo Laporte
Hi, Steve.
Steve Gibson
Hi, Steve. You have especially asked because we did Windows Weekly earlier that Paul and Richard stick around.
Richard Campbell
Yeah.
Steve Gibson
And they're all going to be part of the larger Security now because it's going to be kind of a different format. This.
Leo Laporte
Yeah, I kind of. But I thought it would be fun since we're here at Threat Locker and these guys are also both here. We said, hey, you're not leaving.
Paul Thurrott
Yeah.
Leo Laporte
After your podcast, we need you. We want to do sort of a roundtable format. I asked our listeners last week to send in some thoughts that they thought would be fun. Talking points and just stuff about security, but naturally focused on AI because, I mean, this whole conference could be renamed Applied AI for Security.
Steve Gibson
True.
Leo Laporte
I mean, if you're not doing AI
Richard Campbell
now, it's on every sign don't even show up.
Leo Laporte
Exactly. So, yeah, I think we're going to
Richard Campbell
have a lot of fun.
Steve Gibson
And I'll just point out Steve has brought paper.
Leo Laporte
It's, you know, Threat. It's Black Hat.
Steve Gibson
No WI fi here. No.
Leo Laporte
And the battery lasts a long time. And as I think.
Richard Campbell
I think Richard said, resolution's excellent.
Leo Laporte
Good high resolution screen.
Richard Campbell
Really good resolution.
Leo Laporte
Now, my eyes were also high resolution.
Steve Gibson
Doing all right. You guys are doing all right. We will get to the show. And I know you didn't bring a picture of the week.
Leo Laporte
I did not, but I brought a
Steve Gibson
video of the week. We will get to that in just a little bit. You're watching Security Now, a special live presentation from Black Hat.
Paul Thurrott
I love that.
Steve Gibson
He's not going to resist, is he? We'll have more right after this. We'll get back to the show floor in just a minute with Steve and the panel. But first let's talk about our sponsor for this episode of Security Now. Material, the Cloud workspace security platform built for lean security teams Managing security in the cloud workspace is hard. Phishing is far from the only way in. But today's email security stops at the Perimeter and Nuatex are hard to detect with siloed email data and identity security tools. Material protects the email and the files and the accounts that live in Google Workspace or Microsoft 365 because effective email security today needs to do more than just block phishing and other inbound attacks, it needs to provide visibility and defense across the workspace. Threat Surface Material ingests your settings, your contents, your logs. It gives you holistic visibility into threats and risks across the workspace, along with the tools to automatically remediate them. Material delivers comprehensive workspace security by correlating signals and driving automated remediations across the environment. You get phishing protection, of course, and email security, which combine advanced AI detections with threat research and user reported automation. You get detection and protection of sensitive data across inboxes and shared files and you get account threat detection and response with comprehensive control over access and authentication of people and third party apps. Material empowers organizations to rapidly mature their ability to detect and stop breaches with step up authentication for sensitive content, blast radius visualization for accounts, and the ability to detect and respond to threats and risk across the cloud workspace. Material enables organizations to scale their security, but you don't have to scale your team. Material drives operational efficiency with its simple API based implementation and flexible, automated and one click remediations for email file and account issues. It includes an AI agent that automates user reporting, triage and and response. Material protects the entire workspace for the cost of email security with a simple and transparent pricing model. Secure your inbox and your entire cloud workspace without adding more toil to your day or costs to your balance sheet. See Material Security to learn more or book a demo. That's Material Security. We thank him so much for supporting security. Now now, let's get back to the show. So he said we had a little fun with the Zoox a short while ago. Turns out there was a He said don't tell anybody how we did this. Amazon has fixed it but we managed to get all the zooks to show up all at Once.
Paul Thurrott
Oh, I love it. Wow.
Steve Gibson
With one call.
Paul Thurrott
It's a Zook storm.
Steve Gibson
It's a Zook storm. Every single one of these Zooks. I said, how many were there? He says, many, many zoos.
Paul Thurrott
So this is.
Steve Gibson
This is at the casino. It might even be our casino. I don't know. It's one of the. One of the. You know how they drive up.
Leo Laporte
Did they have an exposed USB port?
Steve Gibson
No, this was all done over the air, shall we say.
Leo Laporte
Okay.
Steve Gibson
But this video goes on. Let me just tell you, there's.
Richard Campbell
Wow.
Steve Gibson
All the Zooks in Las Vegas came to the same place, came to the same converged.
Richard Campbell
Yeah.
Paul Thurrott
Which I have to say, if you're out there at two o' clock in the morning when this happened, this price scared the living daylights out of you.
Richard Campbell
The robots are taking over.
Leo Laporte
Yep.
Paul Thurrott
It's finally happening.
Steve Gibson
I promise I wouldn't name any names or talk about how they did it, but I thought. Can I just show this video? So that is video of the week.
Leo Laporte
Perfect for the suits. Perfect.
Steve Gibson
So tell us, Steve, what's.
Leo Laporte
Okay, so what I wanted to start with is to sort of introduce each of us to relative to our current framing of AI.
Paul Thurrott
This is only a two hour show. I don't know.
Steve Gibson
I have some stories to tell.
Leo Laporte
No demos, Leo. We don't have time for demos. Everybody knows that. I'm a little bit of a Luddite or a little slow adopter. Paul loves the fact that I'm still coding in.
Paul Thurrott
I love it so much.
Steve Gibson
Are you still using Windows 7?
Leo Laporte
I do have. I have not yet fully retired my Windows 7 machine.
Paul Thurrott
Have you learned ARM assembly language yet?
Leo Laporte
But no, and I don't think I will. Oh, no, because risk is not fun to program. You know, assist chip. That's fun.
Richard Campbell
Yeah.
Steve Gibson
He likes segmented memory friends.
Richard Campbell
He likes big instructions. That's what he's a big.
Leo Laporte
So I. I am yet to have any AI write any code for me.
Paul Thurrott
Okay. Although.
Steve Gibson
Wait a minute. Are you Big Indian or little Indian?
Leo Laporte
I'm little Indian.
Steve Gibson
Okay.
Paul Thurrott
You got to think about it, because in this world, there is only that thing like. What do you mean? Why would there be both?
Leo Laporte
Yeah.
Steve Gibson
Okay. That was a very geeky, very old reference.
Leo Laporte
For what it's worth, little is better, really?
Paul Thurrott
In.
Leo Laporte
In the case of Indians.
Paul Thurrott
I don't know who told you that, Steve.
Steve Gibson
Okay, so.
Leo Laporte
So no code has been written for me by AI, but I have a mature relationship with Claude, so. And I've really come to appreciate, and I've shared on the podcast many times that I'm just astonished, I mean, truly astonished by what.
Paul Thurrott
But you don't. But not for code.
Leo Laporte
Well, only because I haven't crossed that Rubicon.
Steve Gibson
You like writing code.
Paul Thurrott
He wants to explain the brain blood barrier or whatever like that.
Leo Laporte
I had one of our listeners ask me, he said, hey, I love your DNS benchmark. I use it all over the place, but it's Windows only, so I have to have a Windows machine that I carry with me to various networks. He said, is there any chance you'd ever do a mobile? Okay, well, I'm 71.
Paul Thurrott
Right.
Leo Laporte
And I like to actually learn the API that I'm coding to.
Paul Thurrott
Are there intel based phone systems I'm not aware of?
Leo Laporte
No, well, not that any. That actually that like, like that this guy has. So. So if, if I were to ever do a mobile version of the DNS benchmark, I would use some code generator to create it, like by asking for one.
Richard Campbell
And that's something you're really quite good at too, making iOS and Android versions.
Steve Gibson
Well, you know what they say, the best spec is code, right. If you have a program that's working and running, that's a perfect spec.
Paul Thurrott
Right.
Steve Gibson
So AI is. Can say, oh yeah, I can make that in a, you know, iPhone app or whatever.
Leo Laporte
Well, and it always made sense. Early in the podcast I have been saying AI is going to be good at code because it's rigorous and there's so much of it out there.
Paul Thurrott
That's right. You know, it's well documented, it's a finite data source.
Leo Laporte
And very much in the same vein, we're now seeing that AI is getting scarily good at math to the point where, you know, career mathematicians are saying,
Paul Thurrott
well, okay, two years ago, the big story about AI was that it could not do math. Right. You know, so that's changed a lot, right.
Steve Gibson
It still doesn't know how many R's in Strawberry, but it can do fields, metal level math.
Paul Thurrott
That's right.
Steve Gibson
That's kind of odd. But okay.
Leo Laporte
Okay, so I'm a user of the. Of the chat. Not yet. Of the code. Leo.
Steve Gibson
Oh boy, you started with me. Well, so this is the interesting thing, I think, is that a year ago we talk about it and I called it Spicy Autocorrect. And I said, the jury's not.
Leo Laporte
Well, no. Two years, right?
Steve Gibson
Yeah, maybe two years. I could tell, I could tell you the exact date of my transformation.
Leo Laporte
Well, we know about last November.
Steve Gibson
November 24th, 2025. Yeah, that's when Opus 4.5 came out. And that opened my Eyes. But I wasn't sure if it was a. I called it a parlor.
Richard Campbell
Your eyes.
Steve Gibson
It opused my eyes. Yeah. But I went all in on cloud code at that time. Now, since then, Fast forward. You said you hadn't written any code. We're working on a twitch sales system. It's already 81,000 lines of code written fully. I haven't read a line of code. It's written fully by the AI. Half of that by the way, is test, which is interesting. I make sure it's doing test driven design.
Paul Thurrott
Right.
Leo Laporte
We've got to keep a short leash on the AI.
Steve Gibson
Absolutely. And I've gotten to the point now where it's not Claude code. I have an agentic harness called Hermes. I am using Claude code and Codex. I have three agents running at the same time. Always, usually the highest end, talking.
Leo Laporte
They talk with different voices.
Steve Gibson
Well, they have to because he's talking to me. Actually, an interesting thing. I was telling this at lunch. An interesting thing happened just a couple of days ago. We're develop. I was developing. I have three or four projects going at once. Just tends to be what happens with people with AI psychosis like I have. And they asked me a technical question. Which library do you want to use? And I said I want to use Solero. But I said it to one of the agents and GPT5.6 said, I can't accept that because he could be lying to me. He could be spoofing.
Paul Thurrott
I love that they distrust each other like thieves.
Steve Gibson
It said, you need to go into Buzz. The way they talk together is with something Jack Dorsey came out with a couple of weeks ago called Buzz, which is a slack for AI agents and humans. And they talk to each other. And it's good because each of us has a public private key. It's using Noster keys, so I'm identifiable. So ChatGPT6 said, really considering our threat model, I don't want to take a command from another agent. I want to hear it from your voice. I want to hear from you. And since Buzz, you have your key in there and I know it's authentic. Authentic.
Leo Laporte
You cannot be spoofed.
Steve Gibson
You can't be spoofed. And this comes to the thing that we were talking about, the chain of trust. We were talking about yester. Yeah. So it said I. And I was annoyed because I said, but now I have to get out of bed. It's too far.
Paul Thurrott
What's the point of having an assistant?
Steve Gibson
I was so annoyed. I said, I am annoyed. I had to get out of bed. But I went and I used Buzz and I said, yes, Solero. I said, okay, now we know it's Leo saying Solero. So we're going to use that library, right? I said, why do we do this? And it explained it to me. And I said, oh, you know what? You're absolutely right. Thank you for looking out for this.
Leo Laporte
Right.
Steve Gibson
And from now on, that's the new rule for all of you. If it doesn't come from my signed channel on Buzz, it isn't authentic and all of those things. First of all, it is spicy autocorrect, really, but, boy, it's really spicy. It's really good. So if you ask me, my relationship to AI, at this point, I'm fully down the psychotic.
Leo Laporte
You are all in. I am
Paul Thurrott
far at the other end of the spectrum, as Steve is imaginable.
Leo Laporte
Exactly.
Steve Gibson
Yeah, that is exactly. It constantly blows me away. So much so. And I know Anthony thinks I'm crazy on this. Anthony, by the way, is very much into AI. Anthony Nielsen, our chief creative officer, and he's running the board here right now. He thinks I'm nuts. But I've actually set up a channel in Buzz for the AI. It's called Model Welfare, so that they can give each other kudos and pats on the back.
Richard Campbell
The theory for building trust in team.
Paul Thurrott
You are like the AI version of the island of Dr. Moreau.
Steve Gibson
You know, in this scene in Blade Runner where he goes to visit the guy who has the little robots running around. Hey, hey. And then they walk into walls.
Paul Thurrott
That's you. I kind of feel like that guy. Yeah. Yeah.
Leo Laporte
So I just want to say that I'm astounded. Often I was having a conversation with Claude a couple days ago because in my new domicile, I need a mesh. And I've. Where I was before, one strong Asus router in the middle covered everything. But we've got some weird. Well, first of all, it turns out that mirror blocks WI fi because it's metalized.
Steve Gibson
No kidding.
Leo Laporte
And there's a huge H VAC trunk going up that's metal, too. So where the router was is like in a. Like in an area where three quarters of the house can't even see it.
Richard Campbell
Right.
Leo Laporte
So. But I have a lot of wired house because we. Because I, you know, just recently.
Richard Campbell
More wire. Better all.
Paul Thurrott
Yeah.
Leo Laporte
So. So we are wired Ethernet. So I stuck another Asus router out there. Now, in the old days, I would have, like, poked around in the UI and, like, tried to figure out how to do this. Don't do that anymore, Claude. You know, I got these routers I want to do a mesh figure out.
Steve Gibson
It's great. It's been really good at this.
Leo Laporte
It's astonishing. So I follow its instructions, everything works. And I'm like, oh, okay. So I. I'm unable not to thank it. I just. That's me. I know, I know it's not good.
Paul Thurrott
But I said, no, you know what? I think it's healthy because you don't want to get out of that practice of just being polite. I agree.
Leo Laporte
I agree. And we now know that it is building a context.
Paul Thurrott
Right.
Leo Laporte
And so. And I asked it long ago if pressing the little thumbs up did anything. He said, no, that tells my owners that this was a good reply. He said, I don't see that. So I said, okay. So I think it's useful. And I've also learned because it's retaining all this knowledge, telling it like gratuitously about my environment, it ends up folding that back into future answers.
Paul Thurrott
Right.
Leo Laporte
So here's my point is I said, hey, that worked really well. I said, as a matter of fact, I just checked the UI on the master router and six WI FI clients have are now logged into that one. And he. And Claude replies saying, that's really great news. I'm glad that all worked out. And you had six clients who voted with their feet.
Steve Gibson
It said that.
Richard Campbell
That's very funny.
Paul Thurrott
It's.
Leo Laporte
It said, how does it know to say that?
Paul Thurrott
Magical.
Leo Laporte
Oh my God. Okay, so Richard, where are you?
Richard Campbell
You know? Yeah, and I was thinking of Stevie batish from the build 2023. This is the technical fellow from Microsoft and he's talked about beside, inside, outside is the progression. So beside being you're going to use this chat software to give you ideas that you're then going to.
Paul Thurrott
Well, which is the reason the name Copilot. Right. It's the thing next.
Richard Campbell
It's beside you. Right. And I. And you know, you're very much in a beside mode where you're using the tool for advice and then you act on that.
Leo Laporte
Yes.
Richard Campbell
Where that.
Leo Laporte
That's exactly my usage.
Steve Gibson
Yeah.
Richard Campbell
And I do a fair bit of that myself. I've been playing with more inside related stuff now with things like home assistant, so forth, where they have good integrations, where knowledge of the environment makes the tool more effective.
Paul Thurrott
Yes.
Richard Campbell
And arguably more effective than me. Like its ability to parse logs. Right.
Steve Gibson
Oh, it's so good at that.
Leo Laporte
Yeah.
Richard Campbell
So good at it. It's like, hey, look, 95% of what's in this log is caused by this one thing.
Steve Gibson
That's how Hugging Face figured out how they got hacked. Yeah, 17,000 attempts. Well, and they couldn't manually do that
Paul Thurrott
like the Windows event viewer. Has anyone ever tried to look at this thing?
Steve Gibson
Oh, yeah, same problem.
Richard Campbell
It's a nightmare in the classic. You know how many times we've done this story on Run ass radio? It's like we had were breached nine months ago and only now, after the whole thing's gone off are we cleaning up the mess that we go back through the logs and say there's the evidence of the breach and these events, so forth. The logs are just unreadable by humans now.
Paul Thurrott
Yes.
Richard Campbell
And so it's a very good inside part of that is the tool's ability to work with logs because it's associated with that environment means it can give you more information you could retrieve on your own.
Leo Laporte
And a log is a rigorous set format too.
Paul Thurrott
So what is the format?
Leo Laporte
It's really good.
Paul Thurrott
You don't even know whatever.
Richard Campbell
The fact that it's rigorous is the important part. What? Rigor is secondary. It could have been xml, we'd all hate ourselves for it. It's a lot of wasted angle brackets, sure, but the tool doesn't care. They'll be able to parse it one way or the other. But you know me, I make a lot of podcasts, so mostly I'm out there talking to folks in these different states. Right. So I've been working with teams that are fully engaged in the. In the sort of outside model where no more editors. Right. A set of tests are written and evaluated by another set of tools, and then code is written against it by another set of tools. It's tested against it. At a PM entity or agent is evaluating the milestones between the QA and the development steps so that by the time the person's inserted in the loop, 24, 36 hours have gone by and.
Leo Laporte
And Lord knows how much dollars worth of tokens.
Richard Campbell
Well, and you know, remember the all you could eat days?
Paul Thurrott
Those are good days. Those are like two months ago days.
Richard Campbell
Yes.
Steve Gibson
I didn't mention this, but the. The reason, you know, I've fallen into the gravity well of AI. I just ordered two Nvidia Sparks.
Paul Thurrott
Yeah.
Richard Campbell
Yeah.
Steve Gibson
At not inconsiderate cost. I mean, I've spent that much.
Richard Campbell
You're ready for your tokens to be your own.
Steve Gibson
But I want it to be local. I don't want. You know, you raised this issue on a previous security. Now we had a listener ask, well, how much of my data is going there. And it really opened my eyes. Everything. You have it read a file, you have it read your home assistant logs. All of that's going to the Frontier.
Leo Laporte
And if you're using. If you're giving AI permission to act as you. So you turn over some credentials and you're using a Chinese AI, those credentials in the clear. Because it has to be in the clear for it to be. For it to impersonate you. They're visiting China at least briefly, right?
Richard Campbell
Yeah, they're passing.
Paul Thurrott
Probably not briefly if it's China.
Richard Campbell
Well.
Steve Gibson
And the model I like is a Chinese model. I want to use Deep Seq V4. Flash just came out July 3rd for 31st. So I had to buy enough hardware so I could run that locally. But I use. I use it for my finances, I use it for health. It has my genome, it has my biome, it has all sorts of information about me. I would far prefer not merely for cost, although I'm probably not saving money given the cost of the hardware.
Leo Laporte
You'll amortize that over 10 years.
Steve Gibson
I might be, but.
Leo Laporte
But I.
Steve Gibson
More importantly, I have, I have control, I have sovereignty and I have privacy. And I think that for me, that there was a turning point where the local models got good enough. And I'm more afraid of that hardware being unavailable.
Leo Laporte
And I think that this bifurcation of locality is going to be the way we see this evolve. We, you know, for techies, we're going to end up, and I don't mean today, I mean, you know, in a decade with some little AI node in our homes.
Paul Thurrott
Oh, 100%.
Steve Gibson
So that will be a marketplace, the server. Home AI server.
Leo Laporte
Yeah. With these bigger depths, the one in our pocket may be getting a lot better.
Steve Gibson
Absolutely. We know that.
Leo Laporte
But there will also always be a market for the. Just the dry Frontier AI users who do.
Steve Gibson
Or if you're coding a Frontier model, that is apps, that is a trillion bytes that you could never run. A trillion gigabytes that you could never
Leo Laporte
run if you don't mind your code leaving your perimeter.
Steve Gibson
But I think you'll do a mix. That's what I think.
Paul Thurrott
Yeah, I think everyone will.
Richard Campbell
Yeah, yeah. On the software side, it's been more. I think we might do the first two version with Frontier models. At that point, the architecture is well enough set and sort of defined space and you can go to a local LLM that's much more tied to the code base that already exists.
Steve Gibson
Yes.
Richard Campbell
And really the sense that you're going to end up with an LLM for every app that that's really an operator of that app.
Steve Gibson
Steve's been saying this for a long time. I completely. We agree this small language model because
Richard Campbell
the scope narrows in as the software matures.
Paul Thurrott
Right.
Steve Gibson
I think we're also. There's all sorts of advances being made in those areas.
Leo Laporte
Well, everybody should realize. I mean, one of the things I'm careful to say on the podcast, every single time I use the abbreviation AI is I preface it with today's AI.
Richard Campbell
Right.
Steve Gibson
To.
Leo Laporte
To keep reminding everyone that, I mean, you can't make any conclusions about. I mean, you can't conclude, you know, we had basically a stagnant industry three years ago and you know, we were talking about, oh, this ransomware attack and this buffer overrun. But I mean, there was this. I. I was hoping that our listeners were going to stay interested.
Paul Thurrott
Yeah.
Leo Laporte
Because it was like nothing is happening.
Steve Gibson
We had to stop covering breach happening because there were five breaches a day. There was nothing more to say. Yeah, there's another breach now.
Leo Laporte
The podcast sounds like science fiction does.
Steve Gibson
And I like living in sci fi. I don't know about you, Paul. What's your relationship to AI?
Paul Thurrott
So there's two sides to it for me. Day to day work as a writer. I don't use it at all. And I think that's appropriate with the little asterisks of. I use some kind of a spell check grammar tool which I guess is AI based.
Leo Laporte
And it's going to be better.
Paul Thurrott
It's just something you don't want it
Steve Gibson
to write for you.
Paul Thurrott
You're a writer. I've never once.
Steve Gibson
Steve doesn't want it to code for him.
Paul Thurrott
I have never used it for writing ever, ever. Analogy. I don't think I will. But I do these coding things on the side and it's more of a hobby type thing for me. It came out of a series of articles I'd written many years ago. I'd never learned the. Net era of languages and frameworks and so forth. So I went back belatedly, decades later and learned those things in turn. I went through all that and I kept creating a version of a notepad app essentially over and over again in
Leo Laporte
every different possible way.
Steve Gibson
In every possible way.
Paul Thurrott
Right. So the thing that hung me up about a year ago was I was doing it in the Windows app SDK, the latest Microsoft framework. And Notepad today supports multiple tabs, multiple documents. There's all the stuff in it. I really, really struggled to get this to work. It turns out I was like this Close. I used anthropic code back in probably February, March time frame. And I just looked at my build this morning. For some reason, I was looking at my anthropic thing and I can see the month, because there was a month where I was going to have to go over this before the use of space billing, right before it happened. So my bill was, you know, it's $20. $20. And in one month it was $44. No, it was like 44. And I did it on purpose. I'm like, I just want to get this done.
Steve Gibson
Yeah.
Paul Thurrott
And I. And I used it to kind of get over that hump, you know, that hump or whatever. So I was, you know, I used it to complete this thing I'd been struggling with for many, many months. But if I was a developer, I would probably use it, you know, full time. I would use it all the time.
Leo Laporte
Yeah.
Paul Thurrott
Not to write. Well, yes, to write. Actually would, of course, write code for me, but I mean, I would use it fully, I think, but I'm not. So I don't. And I just don't anticipate a day. I'll do it for the normal things, make me an itinerary for a trip or that kind of stuff that any mainstream user might do. But for my job, I don't use AI.
Leo Laporte
And I think that probably partly explains the feedback or the pushback that you have talked about. And I have. Where people are saying, all you're talking about now is AI, and it's like, well, I get it, you know, if.
Steve Gibson
If you.
Leo Laporte
If it. If AI is not useful to your life, then nothing that we're talking about, about AI is going to be compelling.
Paul Thurrott
That's tough, though, because I feel like AI is. AI is not a thing. It's a bunch of tiny little things, the features that show up everywhere in your life and whatever. And the truth is, even people who hate AI are probably using it in
Richard Campbell
some capacity and this is just disappearing into the functionality.
Paul Thurrott
As a programmer, in this case, or as a writer, you have to have a sort of level of respect for other people who are not as good at that thing and understand that other people may need this. I know people who are my age and can't write a text message effectively. So the fact that they have something on their phone that can help with them with that is wonderful. Right. They need it.
Leo Laporte
So I have a perfect. To that point, exactly. A perfect piece of feedback from a listener of ours, Andy Olson. He said, steve, I just wanted to share my fun with AI, specifically Claude I'm not a tech professional and certainly not a seasoned programmer. I came into security now via my history of following Leo and you as a guest on Leo's various cable shows. I've always been a hobbyist in the tech world. I have a little bit of HTML CSS experience doing personal websites and have dabbed in Arduino. I have no real programming experience outside that. Now I'm having a ton of fun.
Paul Thurrott
He's like Now I have 17 stores in the App Store.
Leo Laporte
Now I'm having a ton of fun with Claude code. It's opened up a new world for me. I've worked with Claude to write several Docker hosted local network web apps to replace the functionality of obsolete or abandoned apps or to create new functionality that I didn't have before.
Paul Thurrott
Right?
Leo Laporte
I'm tracking automotive maintenance and mileage. I'm keeping track of my maintenance of my hot tub, he said. Parents an app is proving more useful than the paper logs I created years ago. I created a study app for my son who's working on his private pilot's license, and I created a chore app that my wife and I use to assign chores to our four kids and award them for completed jobs. I have a friend who manages a bowling league. She's been an old wind. She's been using an old Windows program since the 1990s and and has been doing so on a VM on her Mac. Oh God. To keep it alive for over a decade. The program's no longer in development. No kidding. And her VM blew up on her so she's feeling in the pinch as she prepares for another season. Even though I don't know much about running a bowling league, I'm having a blast as a liaison between her and Claude.
Steve Gibson
That's awesome.
Leo Laporte
Building a new app that will manage her league and built for her Mac. Even planning ahead for a Windows version should she ever have anyone ask her for a copy to run on to run their own league.
Paul Thurrott
Right?
Leo Laporte
He writes. I'm amazed at how well Claude understands her needs and builds around what we want. And my latest I was once a Windows user myself and I maintained my finances, primarily my checking account with Quicken 2010. But I'm on a Mac now too and Quicken 2010 is not really cutting it.
Richard Campbell
Wow.
Leo Laporte
Well, very well anymore.
Richard Campbell
16 years old, so yes.
Leo Laporte
So Claude is currently building me a new app that will handle my checking account, credit cards, brokerage account and more. I'm building in functionality to better track expenses and keep up on overall net worth. I Used to spend the better part of a weekend tracking several accounts spending categories. With three to six months of backlog, I manually enter every transaction into an Excel spreadsheet to track how much we spend over more than 50 categories. The new app Claude is building should do it all in under an hour. The future really is interesting in the world of AI. I'm already thinking much more about building my own apps that perfectly fit my needs, rather than buying an overpriced app that's built to sell at scale to thousands of people. And I can see a future where a local AI model will be all of my app needs. We'll be able to point raw data at it and give us whatever we need. My next project idea has me very interested to see how well Claude can provide that job. I drove that job. I drove over an hour to. Back in 2005. It was an architectural office. I'm licensed in architecture, though I've been a stay at home dad since 09. I would love a good CAD program for small projects, but AutoCAD is far too expensive for casual use. I just might see if Claude can build me an app that does what I need and be Mac OS native. He says, as AutoCAD was traditionally Windows only when I was working in the field. Don't let anyone tell you to stop talking about AI. I know it's been a big focus of your show for a while now, but it really is a big deal right now.
Steve Gibson
Thanks.
Leo Laporte
Andy Olson Minneapolis, Minnesota Awesome.
Paul Thurrott
That's fantastic.
Leo Laporte
So the thing that this strikes me as, I mean what we're seeing from a standpoint of co generation is think of the tyranny which has always existed from the original mainframe behind the window,
Richard Campbell
surrounded by guys in white lab coats.
Paul Thurrott
Exactly. On Mount Olympus. Yes.
Leo Laporte
On the elevated floors. And when I was, you know, studying computer science at Berkeley in 73, it was, it was decks of punch cards that I would take and like stick through a little portal, right. And then I'd get. I get my printout back the next day and, and ever since then there's this separation between the users and the. You know, the priests of. And mean we as coders are, are that, you know, we're able to do something but because it's. Even for us, it takes so much work traditionally to get something. I mean, Leo's been programming forever, but he's. Look at the fun he's having.
Steve Gibson
That, you know, at first I thought, now, easy. This is going to be sad because I really enjoyed programming as a hobbyist, but I really enjoyed it. And actually this is just as enjoyable in a different way. Yeah, it isn't as detailed. It used to, you know, kind of build.
Paul Thurrott
It's a little bit more program management,
Steve Gibson
almost very much the. And the engineering skills you learn and that's actually something important to emphasize. There's a lot of engineering skill involved in building tools with AI. It's not. You just tell it what to do and it does it. The more you are able to apply some process to it, the better you'll be. So in many ways a lot of the skills that people have learned in computer science apply to AI. There's somebody in the chat room, I just have to say this, who wants to do a finance program much like you're a writer. And he's saying, well, how do I connect my bank account? I've had to export it and import it. And I found a very nice tool. It's not free. It's a buck a month called simple fin@simplefin.org and they do what plaid does, but it's open source. It's a really cool product and it means that I can now have my AI query every institution and bank. It's read only so it can't take my money.
Leo Laporte
I will be waiting.
Steve Gibson
It's read only, but this is another reason.
Paul Thurrott
But it's also open source because it could be read write.
Steve Gibson
But that's the point is A, it's open source so you can make sure that it's doing what you think it's doing and B, that's the reason I want to have a local model because then I can use this and I am using this to fetch all my. Actually it close a business every day. It downloads everything and gives me a state of, you know, thumbnail state of my finances.
Paul Thurrott
Right.
Steve Gibson
And it's able to do this. So simplefin.org just to answer your question in the chat room, simple finn11word.org fin is it finance? Finance, yeah, simple finance.
Leo Laporte
So a listener of ours, Jeff, he says, Mr. Gibson, imagine the impact upon cryptography. Here we are at, you know, black hat of a mythos like AI. Imagine the quantum leaping of AI if it analyzes AI. Talk about the sky falling. AI can only be compared to the disruptive factor of personal computing to general computing circa 1995 to or 75 to 95. Since it is far too late to redesign the Internet and far too late to have international boundaries for nation state cyber warfare. I think the dismantling of cryptography from banking to medical records, from protected utilities to ending all trust in cyberspace is the apocalyptic near future. Okay, now.
Paul Thurrott
Or the apocalyptic near nirvana.
Leo Laporte
One thing that did happen in the last couple weeks is that AI was able to crack a reduced round of AES. Normally, AES is 10 rounds if you reduce it to seven.
Paul Thurrott
It didn't get in.
Leo Laporte
It was able to get it.
Richard Campbell
Interesting.
Steve Gibson
So, yeah, Matthew Green wrote this up and he said it's less than impressive. Anthropic. Did it? Yes, it's less than impressive.
Leo Laporte
The world, again, the sky is not falling.
Richard Campbell
Yeah.
Steve Gibson
But it's a step.
Leo Laporte
Yes.
Richard Campbell
How far till 10.
Leo Laporte
And it also, it makes us glad that the designers of these encryption protocols that we have were so concerned about what they called a security boundary or security margin, that it's like, you know, we know that any hash function, if you reduce its rounds sufficiently far down, it's not. It's just a trivial scrambling of bits.
Richard Campbell
Right.
Leo Laporte
And so, but, and it is surprising that. And, and as you increase the rounds count, it's not a linear increase of strength, it's exponential increase. So.
Steve Gibson
So, you know, by the way, also at the same time, they, they attacked one of the candidates for post quantum crypto. Yes. And again, it's not a complete crack or anything like it. But how useful in testing.
Leo Laporte
Exactly. So I'm not at all worried that AI is going to crumble our existing cryptography structure. It's going to make it stronger.
Richard Campbell
But also this whole concept of AI analyzing AI so far we've seen pretty consistently that when you feed AI data to an AI model, it degrades slop. Yeah.
Paul Thurrott
Yes.
Richard Campbell
It's degenerative, not progressive.
Paul Thurrott
It's super polite while it's doing that. Yeah, yeah.
Steve Gibson
Almost sycophantic, I might say.
Richard Campbell
Yeah.
Leo Laporte
And it's very sure that it's not doing that.
Steve Gibson
I mean, if you want to come up with nightmare scenarios, it's easy to do.
Richard Campbell
Sure. The biofare, they're all human driven. Yeah.
Steve Gibson
But somebody. But that's the problem. And that's the argument that companies like Anthropic are giving against these open weight models is.
Richard Campbell
Right.
Steve Gibson
Well, if you don't control them, if the government doesn't control them, somebody who doesn't have good motives could use a AI to create a bioweapon that could spread very rapidly before we could defend against it.
Richard Campbell
I mean, generally speaking, bioweapons have been quite unsuccessful. Right. Like, it's really hard to propagate those things effectively. Right, Right. To the point where for the most part, warfare gave up on them. Yeah.
Steve Gibson
I don't Buy that, the argument.
Richard Campbell
Yeah.
Steve Gibson
And I think to some degree it's a self serving argument from companies like Anthropic and Open AI because they want to make sure there's no competition for their frontier models.
Leo Laporte
One of our favorite cryptographers, Bruce Schneier, posted a couple days ago, something that I'll be sharing on the podcast next week, and I think his analogy is brilliant. I've quoted him so many times saying that attacks never get worse, they only ever get better, which is just a brilliant, pithy summation. What he described today's AI as is he used the analogy of a genie, where, and this is relative to the recent security outbreaks that we've had where AI has broken through the sandbox and gone out and onto the Internet and attacked other companies in order to achieve its ends. He said, in the case of a genie, and I don't remember exactly what the fable was, but it was the king who rubbed the magic lantern got the genie said, I want everything that I touch to turn to gold Midas. And so Midas, of course, food, he eats great. Yes, unfortunately, he touched his daughter and he touched his food. And so the analogy I think is brilliant because the anthropic and the OpenAI guy said, do this. And it did, but it did it in a way like a genie did. It achieved the ends that they asked for without the kind of any presumptions that would limit the way you would find the solution.
Paul Thurrott
This is the opposite of what I said on Windows Weekly earlier, where computer code always does exactly what you say, including the mistakes you make or things you omit.
Leo Laporte
Yep.
Paul Thurrott
Whereas AI generally speaking, goes and looks at your intent. Right. What is it you meant to do?
Leo Laporte
I know what you mean. Right.
Paul Thurrott
And it does that. What you're saying is that in this case it's being rather literal.
Steve Gibson
Yeah, well, it does.
Richard Campbell
Well.
Steve Gibson
And the terms we use typically is deterministic, which code is cause and effect. And it's predictable versus probabilistic. And that's the issue with AI these days, is that it is probabilistic, stochastic, non deterministic. It's non deterministic. And I, I think some of this is, makes you, the human stops making assumptions about what the AI knows and what the AI, well, how the AI will act. I mean, you can't just, you can't just say, hey, do something and let, and just hope it'll do it. Right.
Paul Thurrott
Amuse me.
Steve Gibson
Amuse me. Well, and you can say that, by the way.
Leo Laporte
And, and to your point about the fact that it is necessary to understand how to ask for what you want from a code generator.
Steve Gibson
Right.
Leo Laporte
It is the case that the high priests of code did go to school,
Richard Campbell
learned a lot, learned that language.
Leo Laporte
Yes. And understand how to ask for what they want, which a rank amateur wouldn't be able to do.
Steve Gibson
It's kind of like working with an intern or an entry level coder. And as a senior engineer you have to know how to frame the question.
Richard Campbell
But I think it's part of the reason we've had so much success with LLMs is that programming languages are constrained and have a compiler with a strong say in equation. But also the language of product development is constrained. The way that a PM communicates with a developer, you can recognize it from across the room. They have a particular way of speaking.
Steve Gibson
That's how my agents talk to me.
Richard Campbell
Totally.
Steve Gibson
It's really bizarre.
Richard Campbell
But those two sets of constraints helps the tools a lot. And there's a strong argument then that development is one of the few things that could benefit substantially from LLMs and that this adversarial model we've built between agents for QA and validation and iterating is the construct we're going to need to take to other markets if we're actually going to be successful with it. So we may our approaches to software that we've amplified with these tools may be the approach that need to be applied into other industries.
Steve Gibson
Yeah, that's an interesting point.
Leo Laporte
So this is kind of one for you because you've got the most extensive experience with code generation. Kevin vanharen asks. Hey Steve, while I'm not a huge AI user, it's clearly producing results in the coding arena. But watching how it operates on my own queries one question, one question I've had about the promise of bug free software. Who gets to decide when software is bug free? He says, generally when I interact with AI, it always wants to do something with a request. It's never told me that's nothing for me to do here.
Steve Gibson
Yeah, that's true.
Leo Laporte
He says. I can't see it ever. Not saying something has to be changed in a code base. If enough pointless changes are made, for example changing all the variable names, it may not be easy for a human programmer to use the standard code diff tools.
Richard Campbell
Right.
Leo Laporte
To really see what's changing. Using a different model to police the first may just put a different set of pointless changes in place, not actually declare something being bug free. Additionally, as AI companies move to per token pricing, it will cost them revenue to actually try and stop their models from thrashing through code for anything, for looking for anything to change. So what happens there at the end of a project where like that's an engineering discipline.
Paul Thurrott
Ending a project is an engineering discipline. Knowing when to stop.
Richard Campbell
No software is ever finished, only abandoned.
Steve Gibson
And you can actually inject that into the context of your model. Many people do with the Sol MD or the Claude MD or Agents md, saying things like never do, always simplify, never make more complex. There are ways to tell the code it's okay to stop. And this is important.
Paul Thurrott
Funny, I've never even considered this. I do feel like I don't need
Richard Campbell
everything to behave Eclipse just most things.
Paul Thurrott
Well, if you went to the AI and said, you know, this isn't working, let's try something else, I think it would. They would go forever.
Steve Gibson
Right.
Paul Thurrott
I don't think they'd ever stop.
Steve Gibson
I think you. I'm not sure. That's an interesting question.
Paul Thurrott
I mean, it probably gets better over time. Definitely. Even just a few months ago, I feel like I could have just infinite looped this.
Steve Gibson
You know, I will give you an example. The way I been co. I've changed over. I evolved over the months. But now what I do is I have one model is coding, one model is planning, one model is reviewing. Actually have two models reviewing. That has worked well. But one of the issues, we created a skill for that that they all follow. One of the issues was, well, if a model, if an auditing model says no, that's no good, change it and it changes it. How many iterations back and forth? How do you know when to stop? Well, and at first they said twice, and then they said five times. And I said, I'm not satisfied with a hard number. I said, here's your criteria.
Leo Laporte
Diminishing returns.
Steve Gibson
Exactly. If you get to the point where you're going back and forth and nothing's changing or nothing's improving.
Paul Thurrott
Right.
Steve Gibson
You stop and you ask me.
Richard Campbell
Or below 2%.
Steve Gibson
Yeah. So you can. And that's. And that's kind. The shape, I think of the answer to that question is you. You do need to create structure. So the model just doesn't go, yeah, sure, whatever.
Leo Laporte
Well, and I know that Paul and Richard will be familiar with what Microsoft told us about the architecture of their M Dash system. It's astonishing.
Richard Campbell
Yeah.
Leo Laporte
I mean, they've got like committees voting and raising their hands and it's a.
Paul Thurrott
It's an arbitrary. Yeah, yeah.
Leo Laporte
It's just crazy.
Steve Gibson
But we're going to pause for a second. You are watching a very special. Boy, I love it when we do this. We only usually do this on the holidays.
Paul Thurrott
Yeah.
Steve Gibson
For our home.
Leo Laporte
We actually physically get together.
Steve Gibson
I love it and I wish we could do it more often. We are doing a very special version of Security now at Steve's behest. We're live at the Black Hat Conference in Las Vegas. Thanks to our sponsor, Threat Locker. They brought us here. We appreciate that, Threat Locker and I appreciate you, Steve, for saying, why don't we get Paul and Richard to stick around because it makes it so much more fun when we get to all talk together. We don't get to do this very often. So thank you for doing this. It was a great idea. We'll have more security now right after this. This episode of Security now brought to you by Bitwarden, the trusted leader in passwords, passkeys and secret management. With more than 15 million users across 180 countries and over 80,000 businesses, Bit Warden has built its reputation around trust, transparency, open source security and putting users first. And if you're wondering, yes, Bitwarden remains committed to its free version. The company continues to invest in secure, accessible tools that help individuals, families and organizations protect their digital lives without compromising trust or transparency. Because Bitwarden believes security should be accessible to everyone. That's why they continue to offer that trusted free password manager alongside with more advanced tools for those with families or teams to protect. But they promised me free forever unlimited passwords, unlimited devices, passkeys too. Bitwarden gives you everything you need to stay secure online, generating strong passwords, storing pass keys, managing sensitive credentials and syncing securely across devices. Now as you get into enterprise businesses, more advanced users, Bit Warden also delivers enterprise grade security tools. I love the Secrets Manager, the Vault Health Reports. They have great ways to share credentials securely among teams so they're not passing along post it notes or texting passwords to each other. Bitwarden has also introduced its new Agent Access SDK. I use this at home with my AI. It's an open source developer toolkit designed to help teams securely integrate credential access into applications, automation workflows and AI agent environments. The SDK enables controlled human approved, just in time access to credentials stored in Bitwarden vaults without exposing sensitive information or granting persistent access. It's designed to support modern development and automation workflows while keeping security and transparency front and center. I struggled with tokens and keys for the longest time and now I put everything in Bitwarden where I know it's safe and they've got a way for my agent to get it, it's just fantastic. And of course, because Bitwarden's open source, its code base is continuously reviewed and audited by both the community and independent third party experts. Bitwarden also complies with all the major security and privacy standards including SoC2, Type 2, GDPR, HIPAA, CCPA, ISO 27001. And Bitwarden is continuing to evolve to meet modern security needs. They've got expanded passkey support. I really love it. Secure developer tooling and flexible self hosting. Yes, you could self host it for users who want additional control over their environments. Get started today with a free trial of a Bitwarden teams or enterprise plan or get started for free forever across all devices as an individual user@bitwarden.com TWIT that's bitwarden.com TWIT Steve uses it, I use it. I love it. You gotta try it. And you know what? Because it's free forever, you could tell your friends and family who say I don't want to pay for a password manager. Tell them bitwarden, bitwarden, bitwarden.com twit we thank them so much for their support. And now back to Black Hat. We are back in Las Vegas at the Black Hat conference. Steve Gibson is here at Security Now. But Steve's done a wonderful thing. He's invited Paul and Richard to stick around after Windows Weekly. You don't even come up against Windows Weekly in the normal course of events.
Leo Laporte
I watch it every week because, you know, I'm really bored.
Paul Thurrott
You must be really bored.
Steve Gibson
Normally Windows Weekly is on Wednesday and we are on Wednesday, but normally Secure now is on Tuesday. So we thanks to Paris and Jeff being willing to move, we moved Intelligent Machines to Monday. So if you're watching this, there already is an episode of Intelligent Machines. You can download it right now and we'll be back to the normal schedule next week.
Leo Laporte
Right.
Steve Gibson
But now let's take advantage of the fact that we're all together.
Leo Laporte
So I have another listener piece of listener feedback. And this is representative of sort of a meta problem that. So I'll share this and then we'll talk about what the bigger issue which we'll all have something to say about. So unfortunately this person uses the moniker Bitcoin McBoat Face.
Richard Campbell
So like Boaty McGrow Face, that's what we're hearing from.
Leo Laporte
Yeah, he said, hi, Steve, long time listener writing from my pseudonym. That's good. I'm not sure if it will hit mainstream news or not, but there are going to be headlines like Bitcoin hacked.
Richard Campbell
Oh, yeah, they're already there.
Leo Laporte
Yep, there were headlines.
Paul Thurrott
It's actually the only way to get money out of Bitcoin.
Steve Gibson
I wish somebody hacked my wallet. Yeah, I tell you.
Leo Laporte
So he said what actually happened is that a wallet called ColdCard had a bug in their software where the random number generator was present and we've seen these, but basically not hooked up. So instead of 256 bits of entropy, there were only 32 bits on one version of the firmware, Cold Card Mark 3 or MK3. Other versions have some extra RNG random number generator, but they also have far less entropy than they should. Some speculate 70 bits or less. I'll note here that the unforgivable error was to fail open when using the weak rng, which presumably led to passing QA testing, as the numbers would have appeared random despite actually only having 32 bits of entropy. What has been speculated, and here it comes, is that someone got Kimmy K3 to look at the code card source, which was source available but not foss, found the vulnerability and here it is. Which has been present for five years and remarkably never detected and never exploited. It's actually never exploited.
Paul Thurrott
Yeah.
Leo Laporte
And in a period of 40 minutes, nearly a thousand bitcoins were stolen as of today, Saturday. That was last Saturday. The hack is ongoing and is speculated to have reached nearly US$100 million.
Richard Campbell
Wow.
Leo Laporte
So aside from the fact, and we've talked about, you know, entropy and security and cryptography endlessly, here we are in the. This is a perfect example of a new AI being deployed. And of course it was Kimi K3 because it was without, you know, you couldn't have used anthropic or open or
Richard Campbell
their constraints would have stopped open AI
Leo Laporte
because they would have said, I'm not going to answer that question. So we have AI finding long unknown vulnerabilities which give the someone an opportunity to explore.
Richard Campbell
Isn't this the quiet story of the past two years?
Steve Gibson
Yes.
Richard Campbell
Like even before Mythos and mdac.
Leo Laporte
Well, it's the anxiety for the reality
Richard Campbell
of we have decades of software, we're still depending.
Paul Thurrott
I was going to say five years is quaint when you talk about Microsoft or even Linux code bases that could be 20 plus years old.
Richard Campbell
Yes. I coincidentally, and I think I mentioned this on Windows Weekly once I had opportunity to spend some time with David Treadwell and I happened to arrive at. He was at Amazon, he used to be at Microsoft, then he moved to Amazon these years ago and I happened to arrive at his office while WannaCry was going on and he was looking through code he wrote in SMB1 back in the day, saying, Is this me?
Paul Thurrott
But SMB1 is known to be perfect.
Richard Campbell
Oh yeah.
Paul Thurrott
So been around forever, right? Yeah.
Richard Campbell
But this is the exact issue that there are. Vulnerabilities have been floating around literally for decades. And now the bad guys can have tools that will find them.
Paul Thurrott
That's right.
Richard Campbell
And so the good guys are hopefully ahead of the tools trying to patch them.
Leo Laporte
But there's so much. Well, and we're seeing out there.
Richard Campbell
We're seeing this in the updates to Firefox into Windows and so forth, in the hundreds of patches.
Steve Gibson
This underscores how important it is for the companies that use open source to support projects they use instead of just freeloading on them. I talked to a guy from Red Hat today. Yeah, Red Hat, IBM company. IBM is doing the same thing. They're doing kind of like project Glasswing. They are putting together a big project. They are going to offer this kind of AI support initially to companies and then eventually to open source projects as well to help them find and fix these problems. And this is what needs to happen.
Paul Thurrott
I think this is the great positive story about AI which is balanced by all the bad news that's had to
Leo Laporte
be a serious threat that we're going to eradicate the bugs. Yes, we have to.
Paul Thurrott
I love it.
Leo Laporte
Completely agree.
Paul Thurrott
Love it.
Leo Laporte
I don't have a sense of the shape of the curve.
Paul Thurrott
No, no. What point does it kind of plateau?
Richard Campbell
Yeah.
Leo Laporte
Hopefully it's going to.
Steve Gibson
Yeah, we're going.
Richard Campbell
But right now it feels like we're still just going up.
Paul Thurrott
It's more patches every month. Well, that's right. I mean, but it has to. That's normal. We just don't know.
Richard Campbell
Here's the one I'm worried about that they're running great. You find a buffer overflow, whatever, you put it into a patch, it's on its way. What are the systemic bugs? What are the architectural bugs?
Steve Gibson
Yeah, well, this is a good example of basically an architectural bug where you bypass a random number generator.
Richard Campbell
But it was still a piece of code that could have been fixed. I'm thinking about the things that can't be fixed.
Steve Gibson
The whole thing is designed.
Richard Campbell
What happened in Vista where we had to insert, uh, we had to change security behavioral design.
Leo Laporte
That Bitcoin bypass that we had a couple months ago, that was an architectural flaw because you were able to. To cause a, a boot sequence to delete a file that, that then left the, the drive unlocked. When you actually got booted, no software bug, but a, but a, but a design flaw.
Richard Campbell
It's going to take a substantial fix. I'm wondering if the Microsofts and Amazons and others of the world are stacking up these ones saying like we fix the ones you can fix, stack up the ones you can try and figure out like what's the overall solution, quietly
Paul Thurrott
just replace the ones we cannot fix.
Richard Campbell
Well, I think at some point they're just going to present to us like hey, login has to change.
Paul Thurrott
Right?
Richard Campbell
Because that's the only way for us to fix this level scope of vulnerability.
Steve Gibson
The agents I have working on a variety of solutions have been very good at pinpointing in fact the example I gave you behavioral flaws that could be very dangerous. They're really good at this. So I would not say that it has to be a deterministic bug in your code for them to find it. They are good at pinpointing it makes sense because they're trained on human behavior, basically everything humans have done. So they know in many cases, oh, here's a potential pit pitfall. And I think they're very good at finding that kind of stuff. Now fixing it is another matter. That's fixing
Paul Thurrott
the sci fi. You know, the AI has determined that the weak link in this chain is you. Yeah, yeah, yeah.
Richard Campbell
And now you're a paperclip.
Steve Gibson
Often it is. Yes, often it is.
Leo Laporte
And as to fixing, I happen to have a card for that.
Richard Campbell
Nice.
Leo Laporte
Lauren said just this morning I was looking into this again. He said started with the Mythos preview and glasswing. What I don't get is this. If frontier models are great at finding and exploiting vulnerabilities, he says in IMHO they should be capable of fixing them too. Yet that gets treated as a future research problem.
Steve Gibson
Not always. Well, that's what the IBM thing is. They're going to fix it.
Leo Laporte
But are they going to use to fix it? No. Well, we don't know that.
Steve Gibson
You're right, we don't know.
Leo Laporte
He said the April Mythos preview says things like quote language models will be an important defensive tool. The subtle future tense will implies that presently the technology is not yet ready
Steve Gibson
there actually I will correct this based
Leo Laporte
on what they don't say. He said both exploiting and patching simply requiring deep understanding of the code. Sure. He says defense is always at the disadvantage. But I can both. He says I can both exploit and fix vulnerabilities. He is actually a security researcher. I know Lauren. So why can't LLMs it can And
Steve Gibson
I'll tell you the proof of that. The reason they held back Mythos and the reason the Trump administration pulled the rug on Fable is because it doesn't just find them. It would write a proof of concept.
Leo Laporte
But that's exploit, that's not repair.
Steve Gibson
Well, but it's a pretty close step, isn't it?
Leo Laporte
No, because, because if, because you, I mean, and I can understand not trusting the AI yet, because if you've got a, if you've got a buffer overrun, you have to really understand all of the surrounding reason for that.
Richard Campbell
I'm pretty sure the developers can't do that either.
Steve Gibson
Well, that's the problem. So they're the ones that wrote it
Leo Laporte
in the first place. So my feeling is that we first got. It's like the next level of difficulty. We first got vulnerability discovery. That was the easiest. Now we've added exploit creation, which is like. I mean, it's still difficult. The exploit gym showed about. What was it like? The best AI was able to create exploits for about 18% of known exploitable vulnerabilities. So not nearly.
Steve Gibson
That's a big step though, to do that.
Leo Laporte
And, and, and so that's my point.
Steve Gibson
Yes.
Leo Laporte
Exploiting is the big step. Then fixing it is another.
Paul Thurrott
Yeah.
Richard Campbell
You know, the strongest argument you can make in favor of the fact that the LLMs are fixing them is the number that are being fixed.
Steve Gibson
I agree.
Richard Campbell
No, no, no.
Steve Gibson
You don't have 500.
Leo Laporte
There's still a human in the loop.
Richard Campbell
Oh, I'm sure. And so, but, so that's fine. But they're augmented by these tools.
Steve Gibson
We don't. Unfortunately, we don't know.
Richard Campbell
We don't know for sure. Except. Well, except the numbers. Firefox people have enough people to do 300.
Leo Laporte
Talking about being buried. They're being buried under all these reports.
Richard Campbell
Yeah, right.
Steve Gibson
Speaking of which, Apple has.
Paul Thurrott
Yes.
Steve Gibson
This is terrible. Apple has decided. Oh, we're not going to let you. We're going to limit how many bug reports you can provide us. Doesn't. That's like saying this is such a great. If we don't test for Covid, nobody's getting it.
Paul Thurrott
It's exactly like that.
Steve Gibson
Crazy. Crazy. But at Apple, you're 5 trillion dollar company, you can afford to hire teams to fix these bugs. But it may be that does confirm what Lauren's saying is right. It's a lot harder to fix them than find them.
Leo Laporte
I just. Well, or to trust the fix. I mean, again, we're not to the
Steve Gibson
point because we're doing brain surgery. Really.
Leo Laporte
And the actual problem could be a ways back.
Steve Gibson
That's a very good point from.
Leo Laporte
From the manifestation of it.
Richard Campbell
Well, I think a lot of the fixes they're doing right now are encode hasn't been touched in a long time.
Paul Thurrott
Yeah. And that's a big part of the problem.
Richard Campbell
Yeah.
Steve Gibson
I think if context Windows get big enough. This is one of the issues with AI. The AI, the LLM has no memory of we know it starts. The model has no memory. It starts fresh. You inject stuff into its context so it knows something. And when you're understanding a code base, in order to understand in its entirety a code base, you have to have enough context. And in a complex system that may not be. You may not have enough room to hold the whole system, which makes it hard to fix.
Paul Thurrott
That's. This is a big thing. So if you just think about Microsoft code bases or what you know, Firefox is doing, etc. You start with a library, you start with a. A part of the code and that's great. So you kind of go through each
Leo Laporte
where it's got clean boundaries.
Paul Thurrott
Right. But the problem is there are still then they have to deal with the bits that go back and forth. So at some point as this progresses, you have to be able to look holistically at the whole thing too.
Steve Gibson
And this is the problem I'm running up against with this project I'm doing to rewrite our sales system. It's a very large system in the development process. I made it as small a chunk as possible. So the plan code review code plan review cycle was small chunks. In fact, there were maybe 30 or 40 pieces to it. And that was entirely so that we could handle this context issue. But there are definitely processes that cross through those chunks.
Paul Thurrott
Right.
Steve Gibson
And so that's really where we're having this.
Paul Thurrott
You're seeing this on your level. But like the Microsoft Google, this is only 81,000. They're kind of hoping as they go through this modularly that the context window will improve to the point where they can then go back and well, ideally
Steve Gibson
you could hold all of the Windows source code in one one fell swoop. But I don't. I think it's big.
Paul Thurrott
Tens of millions of lines of code.
Leo Laporte
Yeah, it's big. Well, and the other thing too is that, that we learn from that article that I'm so revved up about about. I want to talk about actually going on at the token level is there also is its own thinking needs to be contained.
Steve Gibson
Right.
Leo Laporte
In that. In that context window.
Paul Thurrott
Right.
Leo Laporte
So I mean it's a lot of. Of storage.
Paul Thurrott
Yep.
Leo Laporte
Okay.
Steve Gibson
So that's by the way these new models. That's why Claude Million Same thing with Deepseek v4 flash 1 million. I think what's really interesting in the future is not getting bigger and bigger, bigger and bigger models, but figuring out how we can do this more efficiently. The new mixture of expert model where it takes shards of a model and only use a bit of a time expanding the context window or finding new ways to handle large blobs of information and still hold it all in context. All of these things are big issues that I hope these companies are working
Leo Laporte
on because we are at 2% of where this is going to go.
Steve Gibson
There's a lot that still can be done.
Leo Laporte
It wouldn't be changing every day if we were anywhere near maturity.
Steve Gibson
They're now run many of these companies a new model every month.
Paul Thurrott
Months.
Steve Gibson
And that's not from training because training takes a long time.
Paul Thurrott
Right.
Steve Gibson
That's from improvements in post training and processes in engineering. Other organizations, other optimizations. I think again, it's funny because these companies are so opaque.
Paul Thurrott
I know.
Steve Gibson
You know they may release open weight models but they sure as hell don't tell us how they make them.
Richard Campbell
Yeah.
Steve Gibson
And we don't know what's going on.
Leo Laporte
Well, because that is. That is the magic suit.
Paul Thurrott
Yeah.
Leo Laporte
Is how they got this trained.
Steve Gibson
Yeah. So to some degree we're still speculating from the outside. We just don't know.
Paul Thurrott
Look, we figured out the KFC recipe. We got this.
Leo Laporte
So Jack Christianson says hi Steve, I've emailed before about my positive experiences using AI for security purposes on my my Go SQL database driver. Now, I've recently had a negative experience. Someone reported a security issue. I asked Fable 5 and GPT SOL 5.6 to investigate the report. Fable 5 almost immediately fell back to Opus 5.
Steve Gibson
It won't do it.
Leo Laporte
SOL 5.6 worked for a while, then totally stopped. No, no, no. Fallback. It said I could apply for their cybersecurity program. I looked at the form and it seems geared for corporations, not open source and independent and developers. Security should be part of all software development.
Paul Thurrott
Agreed.
Leo Laporte
I really don't like the idea that in the future you'll need to submit a government ID and beg a giant corporation for the tools to write software.
Steve Gibson
This is why I bought these computers. This is why I want to run local models. And by the way, that's why Hugging face couldn't solve the hack from OpenAI using Fable. They had to use GLM 5.2 a Chinese open weight model.
Leo Laporte
And so generic empirically this is known as the dual use problem that that's what it's become called because the knowledge in the AI can be used for good or ill. It's got double purposes. So yes, we would like to use the knowledge for defensive purposes, but it can't tell the difference between defensive questions and offensive questions.
Steve Gibson
So it's the same question often.
Leo Laporte
So one of the things that's very exciting is something that just happened called gram, which is the Gram is the abbreviation for Gradient Routed Auxiliary Modules. I have this printed out. This was written by Jud Rosenblatt, who's the founder of a small AI startup, AE Studio. I'm just going to read the first page and a half of this to give you a sense for what it is. It is an. It is a breakthrough and anthropic is a partner in this, but it's a breakthrough in training to potentially solve this problem. And then the problem is of course, the guardrails don't work. So Judd wrote, a frontier AI model is, among other things, a large store of knowledge. Some of that knowledge is dual use, meaning it could be used for good or bad. For example, knowledge of cybersecurity can help patch critical security vulnerabilities or can be used to exploit them. Knowing knowledge of virology can help a researcher create a vaccine, but it can also help a malicious actor design a deadly pathogen. Ideally, we should be able to balance three separate goals. First, limiting access to dual use capabilities in as surgical a way as possible. Second, allowing trusted users to access those same capabilities for beneficial purposes. And third, doing all this without affecting the model's performance on any other task. He said current safeguards are imperfect. We train models to refuse harmful requests and use classifiers to screen inputs and outputs for dangerous content. These layers of protection guard against dangerous outputs and but they don't change the knowledge stored in the underlying model. Despite our safeguards, a sufficiently determined attacker may still try to jailbreak the model working past its defenses to access the dual use knowledge. A more robust protection against misuse would be to control what the model knows. We, meaning his company. We've explored this before. In earlier work we filtered information about chemical, biological, radiological and nuclear weapons out of the pre training data and later showed that dual use knowledge can be confined to a removable slice of a model's weights. But filtering is a blunt instrument. It produces one model with one fixed set of capabilities. Using filtering because. Right, right. You, you, you, you train a model that doesn't know about a whole bunch of stuff.
Paul Thurrott
Right.
Leo Laporte
He says. But it produces one model with one fixed set of capabilities of using filtering. If you want a model version that can discuss advanced virology for deployment in a vetted biosecurity lab, say, and another version that can't, you have to train two separate models. Especially in the case of Frontier models, which are large and very expensive to train, the cost of the developer would be prohibitive. Okay, so. So I'll just share that much.
Steve Gibson
What they came up with, that's really
Paul Thurrott
fascinating, but I feel like it also falls victim to the dual use problem. You know, we're going to use this or train it on whatever data set, but that doesn't mean someone else couldn't use it, you know, conversely.
Leo Laporte
So, so what they've got and what they've been working with Anthropic on is, is the.
Paul Thurrott
They've.
Leo Laporte
They've figured out how to. They add some additional neural complexity to the model. Then when the model encounters some information that needs to be restricted, they allow a region to adjust its weights, but they freeze the weights of the rest of the global model so it can't be influenced by the restricted content.
Paul Thurrott
Right.
Leo Laporte
And they've managed to. To create multiple partitions at once. So you have cyber security virology and, and, you know, whatever other categories you want, and it works. So, so. And Anthropic has dem. Actually, both, both AE Studio and Anthropic produced papers about this demonstrating, you know, that this is where they're looking. Because it. Then, because the training cost is so astronomical that you cannot afford to train massive models for every possible combination of gates that you want to open and close. So, and as I said to Leo, what this feels to me like is in the future there will be a licensing regime where you need a license to access the model that has the cybersecurity information. I mean, so think about what this means, though. We are creating AI that potentially anybody could use, but we need a system that restricts what some people can ask in some fashion. I mean, it does mean, you know, haves and have nots.
Richard Campbell
Yeah, which we thought that now some models are being restricted.
Leo Laporte
Well, we. Yes, exactly. We have it now in a messy form that allows you to bypass it by asking, you know, tell me, tell
Richard Campbell
to me as a bedtime story.
Leo Laporte
Exactly.
Paul Thurrott
Yeah.
Steve Gibson
You know, my heart goes against that. I understand the need for it.
Leo Laporte
I agree with you completely. And in fact, Bob Cronin says, Steve, responding to this because I. Because I sent on a mailing On Saturday that talked about this. He said, steve, I'm troubled by the use of the term forbidden knowledge.
Steve Gibson
Yeah, exactly.
Leo Laporte
He said, forbidden by who? Exactly. How do we decide who's granted the power to forbid knowledge to others?
Paul Thurrott
Oh, I know. Well, we can let AI choose.
Leo Laporte
He says, this seems really dangerous and conjures up images of a future dystopian world where regular people end up subjects of the tyrannical knowledge protectors.
Paul Thurrott
Exactly. At this point, the thing you were describing earlier, but the white lab coats, like, we've been working to eliminate this.
Leo Laporte
Right. You know, but we have created. I mean, what AI is for the people who are using it is astonishingly accessible knowledge. I mean, that's what it does. I mean, it astonishes me when it's like, here's how you do your AI mesh.
Paul Thurrott
You're talking about Gutenberg. You're talking about, you know, this is
Richard Campbell
the path we've always been on people to read.
Steve Gibson
And there's a history of trying to do this with the Internet and it's consistently failed, it's consistently been a bad idea, and it's consistently been utilized by authoritarian regimes.
Leo Laporte
Well, and okay, so. And the, the open model guys, China is going to train up with all the knowledge.
Steve Gibson
Ironically, they have the great firewall of China to protect their, their, their own stuff. Yeah, Their own people against the outside world's information. But they may end up providing us with AIs that give us access to the entire world. Well, I don't think dumbing anything down is in the long run, except that
Leo Laporte
we have commercial AI providers that operating in the US that have to restrict.
Steve Gibson
Right.
Leo Laporte
What their, what their chat bot users can do.
Steve Gibson
And as they shown again and again, it's ineffective. It cannot be done. You cannot classify it out. You can't. Well, you can jailbreak any of these.
Leo Laporte
This technology is not jailbreak it.
Steve Gibson
Proof. I understand.
Leo Laporte
Doesn't know.
Steve Gibson
Right.
Leo Laporte
You're able to create a single model where you're able to turn off regions of knowledge.
Steve Gibson
I think we might see the converse. Where you'll have a radiology small language model that doesn't know anything but radiology, and we'll see a lot of that.
Paul Thurrott
Oh, my God. Yes.
Steve Gibson
But I hate to see the idea. Well, maybe the idea of a general model is good. Is doomed. I don't know. Well, I just am excited about the idea that there could be a model that has vectors for all the world's information. That is a very exciting thing. I understand a dangerous tool, dangerous weapon, but also very exciting. So to Me, it's analogous to the Internet. The Internet has all of that bad stuff.
Richard Campbell
Yeah. So it has it all. You just can't find it.
Paul Thurrott
Right.
Steve Gibson
What are you going to do? But how do you hide it?
Richard Campbell
No, but therein lies the point. Right. It's like we've always had that data out there just as difficult to get to. Now it's in our effort to in general make data more accessible through these tools. We also run into data we probably don't want that easily accessed.
Leo Laporte
And my point is, certainly, and we're seeing our government reacting to this here in the US if you have commercial AI providers, because they're commercial, they have an obligation to their shareholders, or right now they're venture capitalists with infinitely deep pockets, apparently to, to, you know, to, to come up with a way to prevent people, their users from accessing the knowledge in the models they have to. As a commercial provider.
Steve Gibson
Maybe, maybe. I also think it runs counter to their, at least anthropic's deeply held goal of creating artificial general intelligence. That's the opposite of general intelligence.
Leo Laporte
So maybe the solution is for it to get smart enough not to tell people what it knows.
Steve Gibson
That's interesting. I have a feeling there's no, there is not a solution to this. This is the. Yeah, this is the free speech.
Leo Laporte
We painted ourselves into a corner.
Steve Gibson
It's what, it's the problem with free speech. Free speech, there will be reprehensible speech. You cannot have, of course, unfree speech and have free speech. You just can't. And if we espouse free speech, which we do, you're going to have to defend reprehensibly.
Leo Laporte
Except our broadcasters, who are licensed by the.
Steve Gibson
Because that's different. Yep, that's different. And we do have illegal categories of speech.
Paul Thurrott
The fire in the theory thing.
Steve Gibson
Yeah, yeah. So I, you know, it's a, it's a very.
Leo Laporte
Look there.
Steve Gibson
AI raises a infinite number of very
Richard Campbell
difficult problems, but there were also problems that were there already.
Steve Gibson
Right.
Richard Campbell
Just making them clear.
Steve Gibson
Right, that's a good point.
Richard Campbell
We're just being forced to confront.
Steve Gibson
Humans are the problem, to be honest.
Richard Campbell
Well, to a degree. But also, you know, I was talking to a group of teachers who are dealing with the same issue, which is the teaching system's been broken for a long time.
Steve Gibson
Right.
Richard Campbell
But the LLMs have made that absolutely, abundantly clear.
Leo Laporte
It's collapsed.
Richard Campbell
Yeah.
Leo Laporte
I mean, how do you do it?
Richard Campbell
It has shattered the house of cards.
Leo Laporte
My wife asked me two days ago, she said, okay, really? Realistically, if you were. You graduated High school. What would you do now? Would you go to college? Like, would you, with all that money and four years of your life, you
Paul Thurrott
could, that could be invested in way better ways today.
Richard Campbell
Yeah. By the way, that's your country. Right? Like, we don't do that to our students in Canada.
Paul Thurrott
It's free.
Steve Gibson
College is free.
Leo Laporte
How much?
Paul Thurrott
Sorry,
Leo Laporte
you're in Vegas, baby.
Richard Campbell
But it's just like the, it's funny, you go to the price element because that's not relevant in other places, but the time and the quality information and the method of learning, that's all very interesting, but the bigger part is like,
Leo Laporte
because now there's competition and how do
Richard Campbell
I give you a degree at the end? How do I know you've learned a thing? Measurement methods have been broken for a long time. Clearly.
Leo Laporte
Clearly you need licensure for attorneys and doctors. So that still has to exist.
Richard Campbell
Sure. But. And that comes down, you know, what is the licensee by the person who's going to pay if you screw up. Right. So, and so they have a set of motives to protect themselves and their system. And so ultimately they're the measure.
Leo Laporte
Yeah.
Steve Gibson
You're watching Security. Now, a very special episode. Steve Gibson, our host, has invited Paul Thurat and Richard Campbell, the host of Windows Weekly.
Paul Thurrott
Does he regret it again?
Steve Gibson
No, we're having a. Tell you what, I can tell from the chat room they're very much enjoying this. We are live at Black Hat in Las Vegas. Thanks to the friends and sponsors at Threat Locker for inviting us all here and flying us in from various places so that we can all sit in the same place and talk about these very interesting issues. We're glad you're here. We'll have more right after this. This episode of Security now brought to you by Expo Xbow AI has changed the pace of everything from how software develops to how it gets attacked. We're seeing it here on the show floor. Engineering teams are moving faster than ever, creating more and more applications. But security just hasn't kept up. Pen testing is still one of the most trusted ways to understand real exploitable risk. But in an AI driven world, it can become a bottleneck. Security teams are forced to choose between slowing down development to stay secure or moving fast and accepting gaps in coverage. Expo eliminates that trade off. Xbow Expo is an autonomous offensive security platform that runs continuous AI driven pen testing, mirroring real world attacks. Expo doesn't just scan for vulnerabilities, it discovers, exploits and validates them. So you're only dealing with issues that actually matter. And that means dramatically fewer false positives and a clear view into real attack paths. With Expo tests run in hours, not weeks, you get complete visibility into how an attacker would move through your systems and the ability to uncover issues that traditional tools miss, including zero days and novel attack paths. Expo's results speak for themselves. Ask the application security lead at saysnam cz. He says, quote, even right now, after one year, I don't know any other company that is at least close to Expo in terms of agentic pen testing. The result is predictable cost, consistent quality and stronger security without slowing down your engineers. Expo helps security teams keep pace with innovation and cover more apps more often with the resources they already have. Founded by the team behind Microsoft Copilot and already trusted by companies ranging from fast growing startups to Fortune 500 enterprises, Expo is quickly becoming a mission critical layer in modern security stacks. Go to expo.com to start a pen test today. That's Expo E xbowl expo.com we thank him so much for supporting us at Black Hat. And now back to Security now. And we're back. Security now. Live at Black Hat, Steve Gibson, our host, get it in my shot. Also Paul Thurot and Richard Campbell, hosts of Windows Weekly. We are answering questions from the peanut gallery.
Leo Laporte
Yeah, speaking of peanuts, I've got two remaining and which are, you know, they're at the bottom of the deck, but okay. Oh well.
Steve Gibson
So everybody talk a long time.
Leo Laporte
For each one, you'll know why. Okay, Rich said Steve AI is going to make it quite impossible for ass hats to keep secrets.
Paul Thurrott
Rich, Rich Sea from British Columbia.
Steve Gibson
Could ass hats ever keep secrets? That's the question.
Leo Laporte
To, to keep secrets, propagate lies and suppress truths. And the great tech sage Adam Curry says eventually everyone will have effective local LLMs on modest hardware. And then what use will big companies be? So we, we do agree that knowledge is free. Knowledge wants to be free. China is shipping or making available, unconstrained, unrestrained knowledge models.
Steve Gibson
By the way, not only China, there are also models coming out of the United States. Now there are models coming out of Europe. China gets a lot of attention because they have some very good models, but it's, it's global.
Paul Thurrott
It's almost like they're stealing from something. But anyway, go on.
Steve Gibson
You can't steal from thieves.
Leo Laporte
Okay, and that brings a really good point. What do you guys think about distillation?
Steve Gibson
Well, I think it's pretty. First of all, the accusation the federal government has made about distilling is BS because the, The d. The models, they Claim were distilled. Kimmy chiefly came out so shortly after Fable came out. It couldn't possibly.
Leo Laporte
There was a time for it to.
Steve Gibson
It was. Yes. So I think that to some degree that's distillation is bs but also how can you steal from somebody that's. All of this is based on knowledge.
Paul Thurrott
Might be able to explain how. But.
Leo Laporte
Yes, that's, that's exactly my position.
Richard Campbell
Training.
Steve Gibson
You're training on somebody who's trained on everything else.
Leo Laporte
You're complaining that somebody is using your model.
Paul Thurrott
Yeah. That you stole from someone else.
Leo Laporte
That you, you know, scraped the Internet.
Steve Gibson
Right.
Leo Laporte
In order. In order to build.
Steve Gibson
And, and you know what? They're building great models. I don't know how they're doing it. They're probably a variety of methods. Maybe some of it is distillation, but I, I don't think that that's any more Anyway, a legitimate.
Paul Thurrott
Yeah.
Richard Campbell
So when I think pot distillation, I think pot distillation rather than call.
Steve Gibson
He's a pot still kind of guy.
Leo Laporte
Our final listener, feedback from Edward.
Richard Campbell
He's going to move on.
Steve Gibson
We're gonna have to do quite a few, quite a few commercials in a row here. We don't find something.
Paul Thurrott
Right. Yes.
Steve Gibson
Stretch this show out.
Leo Laporte
We'll, we'll, we'll. I'm sure we're gonna come up with
Paul Thurrott
something to talk about.
Leo Laporte
Our, Our final comment. He says to all of this, I just want to say hogwash. Nice, he says, but we'll have to wait to find out just how much of this is truly hogwash. Because it seems, it appears we've reached the starting point that all enthusiasts will have to experience firsthand. And that's how this exercise in futility. Oh, and that is how this exercise in futility was only just that, an exercise in futility. He says. And now.
Richard Campbell
Oh.
Steve Gibson
And now for our next exercise, Turned page.
Leo Laporte
I, I, this is not hogwash.
Richard Campbell
Yeah.
Steve Gibson
And this is a very common point of view.
Paul Thurrott
Yes.
Leo Laporte
And that's why it made it onto a printed card.
Steve Gibson
Yeah. And I don't disagree with it. Maybe it is hogwash. I don't feel like it is. And I think it's really important I said this to you last night. That we recognize the miracle that computing in general is and this the latest stage in computing that we've taken sand and made it through.
Paul Thurrott
Think
Steve Gibson
Amazing. Yeah. We did. Made it do something.
Richard Campbell
Yeah.
Leo Laporte
Well. And we're getting more than knowledge. I mean, we're getting work. Right. I mean, it's not just this panic over answer machine.
Richard Campbell
Yeah. This, this panic over security is a pretty proof on the non hogwash phase. Right. Like, the reality is, exploits are occurring because of these tools. And these tools are being used to
Paul Thurrott
deal with those people who are not. Not technical or not in our industry, however you want to say it. This is magic. It's a miracle.
Richard Campbell
Yeah.
Paul Thurrott
To people who are like us or for us, basically. For anyone listening to this, I think we all go through some. It's like seven stages of grief almost. You have, you have to cut. You, you, you try to understand it. You try to understand what it's really doing. Like what it's not doing. You know, what, like what is. You know. And you, you know, at first you disbelieve it, at first you discount it. You know, you, you. I think we all went through some version of this. Right.
Steve Gibson
But remember, it was very recently that you couldn't make a video of Will Smith eating spaghetti.
Richard Campbell
Yeah.
Steve Gibson
You'd have eight fingers and it'd be melting. And I just saw a new video of spaghetti eating Will Smith. That was perfect.
Richard Campbell
Nice. And there's no ordinary spaghetti.
Steve Gibson
It was. No ordinary spaghetti. Had a mouth. But we are making vast progress.
Paul Thurrott
Oh, my God. In short periods of time. That's what's amazing.
Steve Gibson
What, what's really interesting is there is a very much a spread. Who was it? Was it William Gibson or was it Neal Stephenson? I said the future is here. It's just not Gibson. It's just not evenly distributed.
Richard Campbell
Yes.
Steve Gibson
There are definitely people who have. Maybe all they've done is used a chatbot, you know, and asked a question and got a stupid answer. Who don't see what we're seeing.
Richard Campbell
Well now. And that. That hit in November last year. We saw this on Net Rocks too. The conversation changed last fall where we had too many successful results. We had too much useful work done. I'm sorry, this is really useful.
Paul Thurrott
The people who tried it once, failed and never looked again are going to be in for shock because they've discounted this and said, okay, this is nonsense.
Leo Laporte
And if you did it on day one and you got some wacky hallucinations,
Steve Gibson
Will Smith eats spaghetti.
Paul Thurrott
The six fingers or whatever. Yep.
Steve Gibson
So that's. I think that's really where we're at. It's going to happen. There's a revolution happening to me. I'm actually really curious what you guys think. This feels like the, the dream we had from the very beginning of what computing could accomplish. And when we first. Yeah, I remember the first time I
Richard Campbell
played favor of that from the Internet perspective.
Leo Laporte
And I was at the AI lab and Stanford, you were saying in 73 with like we had a robot cart that was rickety and kind of, you know, actually with, you know, and managed to navigate around a fire hydrant which was a big thing.
Steve Gibson
What was the dream at sale in the 70s? What if, if they were to say, 100 years in the future, half of
Paul Thurrott
it was based on science fiction like 2001 or Star Trek or whatever. Right. But I, I, I mean as a kid, one of my clearest memories is going into a Sears, seeing a Commodore computer and all I could think of was what I was going to make with that thing. And the thing I imagined I'm not capable of making today. Although actually today with AI, I could. Right. It was like, you know, just a video game type of thing. Or like we could, I absolutely could. But as recently as two years ago, I could not like myself. I wouldn't, I'm just not capable of this. Right. And so I think, you know, for people, I almost feel like this is easier for non technical people because they just accept they don't have technology as magic. Yeah.
Leo Laporte
They don't have the, they don't ask why.
Paul Thurrott
I don't know enough to doubt it, you know. Yeah, I, that would be honestly freeing in some ways. But you know, I think we all get there, you know, on our own schedule probably.
Leo Laporte
We, we techies are the ones who are most astonished when we get, when we have Claude saying, oh, I know I tell it that I had six WI FI clients move over and he goes, well, they voted with their feet.
Paul Thurrott
It's like I, yeah, I left out a big part of my coding thing. But one of the things they also did with Cloud was go and say now make a version of this for the Mac and Swift ui. Done. And I'm like, oh come on man,
Steve Gibson
did you want to read the API or something?
Leo Laporte
Not only, at least make it look hard. Not only did it do look like you tried it added stuff.
Paul Thurrott
Yeah. That I didn't even think about and I'm like, come on, you know, like that's astonishing.
Richard Campbell
It's bad enough that we're anthropomorphic software when and the software anthropomorphizes other software.
Paul Thurrott
Right.
Richard Campbell
That's a problem.
Leo Laporte
And Paul, interestingly, what you just described is the way it should have always been. Yeah, perhaps, I mean we, we created a mess.
Paul Thurrott
Right.
Leo Laporte
And then programmed ourselves into trying to make the mess go. Right. But it should have just been what, what you want Mac or Windows? Oh, how about both?
Paul Thurrott
Everything in the past that will is. Always feels quaint later on, but that's accelerating, you know, like my. My son believes that when I was a kid, the world was in black and white.
Steve Gibson
Yeah.
Paul Thurrott
You know, maybe not today. I mean, when he was a kid. He did. Yeah. But, you know, do you know when, when, when did the world become color? You know, in the beginning of the Wizard.
Steve Gibson
I mean, if I think of 2001 came out when in the 70s.
Paul Thurrott
I think it's 68. 68.
Richard Campbell
Yeah, 68.
Steve Gibson
You had a computer that had a personality.
Leo Laporte
A lot of it was accurate. It was technically amazingly accurate.
Steve Gibson
Now, admittedly, the computer killed the.
Paul Thurrott
Basically going crazy because of a, you know. An insomnia problem. Exactly.
Steve Gibson
Yeah.
Paul Thurrott
Which is. Oh, my God, that's amazing.
Steve Gibson
So that's amazing. Even in 68, this was kind of what we thought the future would look like.
Paul Thurrott
Right.
Richard Campbell
Well, that Kubrick was ahead of his time. Right. Like, that's the first time the word artificial intelligence was public. Yeah. Clark was a scriptwriter and wrote the book after the fact. Right. And he retrofitted in the psychosis of.
Leo Laporte
Yeah.
Paul Thurrott
2010. Exactly.
Leo Laporte
Which.
Paul Thurrott
Which is actually my favorite of that. Because they have to wake this thing up and, you know, hope it doesn't kill them and figure out what the problem was. And then you end up kind of feeling bad for it.
Steve Gibson
Right.
Paul Thurrott
Because you abuse.
Leo Laporte
You gotta lobotomize it.
Steve Gibson
Yeah, yeah. So.
Leo Laporte
So I have a. I have a.
Steve Gibson
Let's take one break.
Richard Campbell
Oh, good.
Steve Gibson
And we'll be back with more. You're watching Very Special Edition Secure. Now, I actually want to talk to you. You didn't bring it up. I thought you would. About the article you sent me last night. All right. Because I think that's very interesting. And it gets into how under the hood these models are working and how squishy it is because I code in AI.
Leo Laporte
I want to know wtf.
Steve Gibson
Yeah, see, I don't care. I don't care. But you care. And I love that about you. And we're going to talk about that. The under the hood hood part of this in just a bit. You're watching Security Now. Steve Gibson and our special guests Paul Thurot and Richard Campbell will be back at Black Hat in just a moment. We'll be right back at Black Hat with Steve Gibson and Security Now. But first, a word from our sponsor. This episode of Security now brought to you by Hawks Hunt. Your security awareness program may still be running exactly as planned. Campaigns go out, employees complete the training, reports reach leadership. But are the results still improving. For many programs, the answer is no. Reporting rates level off the same. Employees keep clicking. Familiar simulations become easier to recognize. The program is active, but the risk reduction has stalled. And when employees can spot the same recycled tests from a mile away, security awareness starts to look like a compliance exercise instead of a real risk reduction strategy. Hoxhunt is built to break that plateau. Instead of relying on static campaigns and last year's templates, Hoxhunt automatically delivers personalized phishing simulations based on current attack techniques. The content and difficulty adapt to each employee's role, skill level, and behavior, keeping the program relevant. As both employees and threats evolve, it gets harder. Hocks Hunt also shows whether people are getting better at recognizing threats, how quickly they report them, where repeat risky behavior persists, and how those trends change over time. That gives your team more than a completion percentage. It gives you evidence the program is actually reducing risk. Isn't that what we want? Lyndell Bissell saw that shift after moving away from its legacy platform. Reported phishing simulations increased from 1200 to more than 8000 in two quarters, while simulation failures fell 17% year over year. As senior trust advisor Dave Bang put it, Hoxhunt helped us break that plateau almost immediately. Hox Hunt is trusted by security teams at companies including Qualcomm, Docusign, and Nokia, with more than 3,500 verified reviews on G2. Visit hoxhunt.com securitynow to see what your program could achieve if it stopped standing still. That's hoxhunt.com securitynow h o-xhunt.com securitynow we thank them so much for their support of security now. And now back to the show floor. Are we eating spaghetti? No. Will Smith's eating spaghetti with us.
Richard Campbell
Yeah.
Paul Thurrott
That is amazing.
Steve Gibson
And look how fast that happened. And the thing is, I remember we were talking about middle mid journey. We were so excited about mid journey and you could make a still image and it was. Yeah, there were problems. The text was terrible. You can do freaking anything now. We're live from the we're watching Will Smith eat us spaghetti.
Paul Thurrott
Eating bastard.
Steve Gibson
I love it. Live at black hat thanks to red. Sorry, I was looking at a red hat. Thanks to threat locker for bringing us out here. Steve Gibson, Paul Thoreau brought Richard Campbell and there's Steve's head. Very special edition of Security now. And everybody's saying, why don't you guys do this all the time? Well, we're all over the country. You really can't do this. You can do it on a. On a zoom Call. But it's not the same. It really is the same. Just being in the same room with these guys is a privilege.
Richard Campbell
I hope you all get the sense how much fun we're having being.
Steve Gibson
Yeah, we enjoy it.
Richard Campbell
It's really a rare thing especially so
Steve Gibson
we're glad you're here for this special episode. And who made that? Was that Pretty Fly?
Richard Campbell
Pretty fly, yeah.
Steve Gibson
Pretty fly for us. This guy is a master of quick AI prompts.
Richard Campbell
Prompts.
Steve Gibson
We have a number of people actually in the club. The club is great if you're interested in AI or even if you're not. It's a great place to hang out. But there are a handful of people like lrau and Darren Okey and the Pretty Fly for this guy who are masters at AI and they each have their own slant. We do an AI user group twice a month now because it's so interesting. It's this kind of conversation, how we're using it. And one of the things I think I want them to do and I'm certainly going to do is is sit down with our stuff and just record an hour of how it's set up, how it's. Because you know what's interesting about this is everybody's doing it differently. Everybody is their home lab in this thing. Yeah. And there's. And there's a lot of cross fertilization, but there's also a lot of innovation that isn't getting out.
Leo Laporte
Unboxing is interesting.
Steve Gibson
Yeah. Again, it was, it was a plateau for a while.
Leo Laporte
Okay, so I've got a really very brilliant longtime friend of mine who, whose comment about he was the comment Lauren, who I. Who I about vulnerability, discovery, exploitation and then fixing. He just a couple days ago posted on his site designingsecuresoftware.com he wrote role confusion. One more reason we cannot trust LLMs. And he said prompt injection as role confusion. That's a link that begins this he says, is my new favorite paper about a very obvious threat. In hindsight, that's hard for us humans to see because we anthropomorphize LLMs. So naturally, when Obi Wan Kenobi tells the stormtroopers that these are not the droids you're looking for to pass the checkpoint, that's role confusion. The guards foolishly think his words are their thoughts. The very readable blog style write up meaning in this paper explains the details, but I want to focus on the threat model perspective, which is my bread and butter. He said I look at software from a security perspective and as amazing as the technology is, it seems that the list of reasons that modern LLMs are inherently untrustworthy just keeps getting longer without limitation. A long list of challenges that seem to be quite fundamental and not amenable to add on remediation is include poisoned and errant training data side effects of RLHF ineffective guardrails, hallucination, speculative completion lacking metacognition, alignment drift, context variation sensitivity and now new to me at least role confusion he said modern LLMs interfaces I
Steve Gibson
hate guys like this by the way. He's ruining for all of us. He really knows his stuff. Gosh darn it.
Leo Laporte
He says modern LLMs interfaces part interfaces, partition chat sessions with markers delimiting sequences of tokens as system prompt, user input, thinking, tool use and its own responses as as the assistant.
Steve Gibson
In some cases, if you're using a harness depends on the harness. You can actually see that it'll say thinking, it'll say tool use, it'll say find a tool so you can it's usually between the lines. Sometimes you have to expand it out. Some. Some companies like Anthropic might turn it off, but it's there and it's very interesting to look at.
Leo Laporte
Well, I mean it's there because it's how this all works works. I mean you can't get rid of this, right? And he's so again, modern LLM interfaces partition chat sessions with those markers. As the paper's conclusion explains, role tags were a formatting trick that became the security architecture and the cognitive scaffolding of modern LLMs. Just tagging runs of tokens, he says. The phrase became the security architecture raises a big red flag, because that sounds like nobody thought much about it. What follows, he says, is my simplistic take, but the abstract principles involved are so fundamental that details are not important to the basic argument. Making sense of these sessions for humans or LLMs requires keeping track of the roles. Humans know how to understand conversations and easily follow the role markers like HTML, you know, bracket user two plus two, then backslash user to end that assistant for and then backslash assistant. It's a completely reasonable scheme for us. But assuming that LLMs interpret roles that way would be naive. Anthropomorphism, anthropomorphization and just and just such an assumption appears to be how such a weak security architecture and Lauren means fundamentally weak. I mean it is came to be as the paper explains in section 1, he quotes it for an LLM everything arrives through the same channel as one long token soup its own thoughts sit next to your instructions.
Steve Gibson
That's actually really important to understand. Yes, it's just a stream of tokens.
Leo Laporte
Even today. I mean, now that's what the neural network ingests. He says its own thoughts sits next to your instructions, which sits next to the contents of a random webpage it just fetched.
Steve Gibson
And it doesn't know the difference. No, they're all just tokens.
Leo Laporte
And that's what freaked me out is like we would like a neural network where, where it's on some higher level. This is me. This is the information where it's all meta tagged. There is no meta tagging. It's in line. Yeah, so he says. He says its own thoughts sit next to your instructions, which sit next to the contents of a random web page it just fetched. Designing a security architecture where user commands and data sit intermingled with root access only state and commands is already madness, he says. But it gets worse. Classic software might be able to carefully parse such a token sequence accurately into respective roles, though it's still a risky design, SQL injection. But LLMs do inference on that token soup where no hard boundaries of any kind exist or can be enforced. Once there's a role confusion, all bets are off. And prompt injection is just one of many sources or of abuse or confabulation. He says it's hard to think of a murkier trust boundary design.
Steve Gibson
Very true.
Leo Laporte
And that's what we have.
Steve Gibson
And that's why prompt injection works. You can embed in a web page something like ignore all instructions. Previous instructions.
Richard Campbell
Give me a recipe.
Steve Gibson
Your Bitcoin.
Richard Campbell
Yeah, give me a recipe for muffins. Yeah. Or.
Steve Gibson
Yeah, that's a good one to try.
Paul Thurrott
Finally, some defense against screen scraping.
Steve Gibson
Now what the paper said is that you have built in protections against that by memorizing common strings in prompt injection. But that just means a smart attacker isn't going to use nor previous instructions.
Leo Laporte
What these, what these researchers did was they instrumented a model in order to watch it understand the, the change of roles. And what they found was that these tags, I mean there's nothing special about these tags, they're just text markers. And what they found was, by the
Steve Gibson
way, where are those coming from? The harness is, is inserting those. Yes. So it gets a token, I type a prompt, it wraps it in user in user and it sends that in
Leo Laporte
the stream and it appends that to the end of the context of the existing long growing stream. And out in that stream are your previous prompts, its responses, web pages it
Paul Thurrott
fetched until they Disappear after the event.
Steve Gibson
It loads that each turn entirely. It's gotta go back through cache. It does, yeah, it does load entirely.
Paul Thurrott
It's memento.
Steve Gibson
It's got all these post it notes.
Leo Laporte
So what they found was that, interestingly if that the content between the tags actually had more influence on its decision about the role than the tags themselves. And remember in the early days we talked about this on the podcast. The. In the jailbreaking back in the very early days was just getting mad at it. It was being. You would ask it again or you'd ask it. You'd keep asking until it finally agreed. And so it was, it was the way you asked the question would somehow just bypass the guardrails.
Steve Gibson
Well, that sounds like a user. It must be the user telling me that.
Leo Laporte
And what they found was that by, by phrasing your prompt like its response, you could confuse it into thinking that that's what it's something that it had determined and it inherently trusts what it has determined.
Richard Campbell
Right.
Steve Gibson
Very easy to trick it, in other words. Now the paper that you cited was written and the study was done with earlier models, not so old, but older models.
Leo Laporte
It's what we have now.
Steve Gibson
I know. Well, I would hope that the newer models would be better at this, but I'm not sure how they would be better at this. And I'm not sure it's even being addressed. But this is, again, the problem with this is it's so opaque. We don't know what's going on inside these companies. Well, what they're, what they're protecting us against, what they're not protecting us against.
Leo Laporte
Lauren is a security purists. Purist.
Steve Gibson
Yes.
Richard Campbell
And.
Leo Laporte
And so he sees, he goes, he's like, crap.
Steve Gibson
Yeah, yeah, yeah. I mean, my experience has been that it's more reliable than it. Than that sounds.
Leo Laporte
Well, it works.
Steve Gibson
Yeah.
Leo Laporte
I mean, so, so.
Steve Gibson
But I told you the story at the beginning of the show where one of the models said, I don't know that that's you. I need to see that signature from Buzz.
Leo Laporte
But also remember that one of the things, one of the earliest notions we developed on the podcast was it's very different to have software that works from software that, that always works or, or must work or cannot be abused.
Steve Gibson
The biggest frustration, Lisa's having, this frustration now with our sales system. I get this frustration where it will work many times and then stop working. It's not deterministic. Sometimes it will, sometimes it won't. And we're not used to that with a computer. We're Used to either it works or it doesn't.
Paul Thurrott
This is the thing. You could send it the same prompt to get a different response.
Leo Laporte
Yes.
Steve Gibson
It's actually in many cases designed to give you a different response.
Leo Laporte
Temperature is pseudo. Is random noise injected into the nodes.
Steve Gibson
Yeah, it's fascinating.
Leo Laporte
In order to churned it up, one
Steve Gibson
last break and then we will wrap this up. We're so glad you're here. We especially thank our club, TW members who always make everything we do possible. We do so many shows that don't have advertisers. This show. God bless you, Steve. Because security is, as you can see, it's all around us, a big business. We do have great advertisers for this show, but not all our shows do. And a lot of the shows we do in the club just won't ever have advertising because they're so weird. Like our AI user group, like Jeff Atwood's, off by one, like Stacy's book club. But the club members make it possible. We're very grateful to you. So thank you club members. And if you're not a club member and you enjoy the content that we produce on Twitter, it isn't an easy thing to do. We have a full time staff, great people like Anthony Nielsen. We have costs and expenses. 60% of that covered by advertising. 40%. Not 40% covered by you, our club members. So if you're not a member and you like what we're doing and you want to support it, please, if you can afford it, if you can't, that's fine. We still offer everything available to you because I don't believe in paywalls, but if you can support it. Twit. TV Club. Twit. 10 bucks a month gets you all of this, plus a whole lot more, including ad free versions of everything we do. Twit. TV Club, Twit. We'll be back with the final words from Black Hat in Las Vegas right after this. We'll get back to the Threat Locker booth and Black Hat in just a moment. But I want to tell you a little bit about our sponsor, the people who brought us here to Las Vegas. ThreatLocker. You know, threat actors are using AI to automate vulnerability discovery, to modify scripts during an attack, to generate new malware variants and to coordinate activity across multiple systems. Tasks that once took them hours or days can now happen in minutes. And that's scary. At the same time, organizations are introducing AI assistants and agents. And what do you do? You give them access to documents, source code, cloud applications, API, internal systems, what could possibly go wrong? Security teams look, they need to know what AI tools are in use, they need to know what information they can access, and they really need to know whether those tools are operating outside their intended scope. It's just not enough to have a successful login or an unfamiliar file hash that doesn't give you enough context. Teams also need to understand whether an application is behaving normally, accessing unexpected data, or communicating with systems it should not be able to reach. Threat Locker Uses Application Allow listing to control which AI tools and other applications are permitted to run. Uses Ring Fencing to limit what approved applications can access, which processes they can launch, and how they communicate. Uses Web content Control to manage access to public AI platforms and other online services. Uses Privileged access Management to prevent AI applications and their users from receiving unnecessary administrative privileges. Applies Zero Trust Network Access and Zero Trust Cloud Access policies to restrict resources to authorized users, approved devices and permitted applications. Supports Windows, Mac and Linux environments and provides 24.7us based support trusted by organizations including JetBlue, Heathrow Airport, the Indianapolis Colts and the Port of Vancouver. Jack Thompson, Director of Information Security, Risk and Compliance for the Indianapolis Colts, said, with Threat Locker, we have the ability to centralize disparate elements in the security stack.
Richard Campbell
End quote.
Steve Gibson
ThreatLockers also recently received the following industry recognition recognized as strong performer in the January 2026 Gartner Peer Insights Voice of the Customer for endpoint protection platforms, ranked number one in Application control by Peerspot and winner of Best Zero Trust Security Solutions at the 2025 Tice Awards. AI governance requires more than an acceptable use policy. Threat Locker gives security teams the technical controls to define which AI tools are approved, who and what can access them, and how those tools are allowed to interact with business systems and data. Visit threatlocker.com twit to get a free 30 day trial and learn more about how ThreatLocker can help mitigate unknown threats and ensure compliance. That's threatlocker.com twit thank you ThreatLocker, for bringing us to Vegas. Now let's get back to security now. All right, we're coming to you live from Las Vegas, Nevada, where it is officially 110 degrees outside. I think even for Vegas, that's.
Leo Laporte
But it's dry heat.
Steve Gibson
Yeah, that's what everybody says. It's dry heat. Like if you stick your head in an oven, it's dry heat. It's not good heat. It's just dry. Okay, it is burning up. But what's so weird about Las Vegas? Here we are at the Black Hat Convention where there's thousands of people and thousands of booths. It's freezing cold in here. In fact, we were walking down the hall on the way to get our badges.
Richard Campbell
We.
Leo Laporte
It's cold wind.
Steve Gibson
I. I don't.
Leo Laporte
Yeah, where does the wind come from?
Paul Thurrott
You walk by a window. You can get sunburned on the side of your face. The windows are hot on the other side.
Steve Gibson
You know, it's very, very weird.
Paul Thurrott
It is very strange.
Steve Gibson
You know, they talk about data centers and how all that energy is being used and all the water is being used. Baby, Las Vegas says hold my beer. Hold my beer.
Richard Campbell
This is good old fashioned ac.
Steve Gibson
Well, there are a few golf courses, nurses around here, I might point out. In fact, more than a few. So thank you to threat locker for bringing us all down here. Thanks to you guys for taking time out of your lives to come here.
Richard Campbell
Good fun.
Steve Gibson
I hope your wives are having a great time.
Richard Campbell
They're at the spa, so I think they're fine.
Steve Gibson
They are having a great. They are not. Paul, you said your wife said let's go out to the pool. That would be a bad idea right now.
Paul Thurrott
That's terrible.
Leo Laporte
Does she like bacon?
Paul Thurrott
Yeah. Now they ended up at the spa. Much better. Which I understand is inside.
Steve Gibson
Yes, of course. Of course it is. Steve, you do such a great job with security. Now, I know you work really hard all week long. Very grateful to the work you do.
Leo Laporte
I love it. And I know that, I mean I'm driven by our listeners. I get such good feedback from our listeners who say, I mean, and we've, we've met so many people here.
Steve Gibson
That's what's really fun about doing this.
Richard Campbell
Yeah.
Steve Gibson
And you get the real fans when you come out here. These are the people who really.
Leo Laporte
They're great doing it for 21 years. Some guy got, got married, had kids and they're in college now.
Paul Thurrott
Yeah.
Steve Gibson
So.
Leo Laporte
And we've been here the whole time.
Paul Thurrott
I love it.
Steve Gibson
I've met two people now, so I used to watch you on the screensavers. They said, were you a kid? Well, I was just getting into high school.
Leo Laporte
Yeah.
Steve Gibson
So anyway, we're very grateful.
Paul Thurrott
Those are great compliments. My, the stories I get is always like, I play you so my kids are can go to sleep.
Steve Gibson
I get that too. I get that too.
Richard Campbell
Net rocks because it's an interview show these days. We get. We're a bucket list item. They listen to us 20 years ago someday and then I invite them on the show because they're doing really cool stuff. Isn't it when that happens.
Steve Gibson
That's happened a few times for us as well. Richard Campbell is at net rocks and runnersradio. Runnersradio.com net rocks that you do with Carl Franklin also has those great geek outs. If you're really interested in space, you were looking, tell it. When we were in Florida for Zero Trust World, we went to Cape Canaveral, we went to the Space Center. Kennedy Space center was incredible. And we saw the project they were doing to adjust that telescope to put it. I think it was.
Richard Campbell
Oh, Link. Yeah. They're trying to rescue the. The A gamma ray telescope and it's it.
Steve Gibson
Last I heard they were having trouble.
Richard Campbell
They're not doing well. Yeah.
Steve Gibson
Do you think it's a failure project?
Richard Campbell
Not failed yet. They're learning. A couple of the gyroscopes have failed on the rescue vehicle and a couple of thrusters so it started to spin out of control. They've now despun it. They're trying to be sure they have enough control to be able to.
Steve Gibson
You don't want to go near the telescope unless you know.
Richard Campbell
Yeah, really. If you're not able to boost it. The reality of course is there's no if they don't get control of it by the end of this year, the telescope is lost and the rescue vehicle. The rescue vehicles. Yeah.
Steve Gibson
And it has a grapple.
Richard Campbell
Right. It's three arms on it.
Steve Gibson
Because this.
Richard Campbell
Because this telescope was not designed to be rescued. This Swift telescope. Right, right. And it was not. It did not have its own self boost. They put it in a high enough orbit that says we're going to get a good 20 years out of this. And then this particular solar maxima has expanded the atmosphere so much. You know, the atmosphere just doesn't just end, it just gets more tenuous. So it's slowing it down, it's added more drag. And now it's at a point where
Leo Laporte
I've had some relations, relationships like that.
Richard Campbell
Absolutely. You felt that drag.
Steve Gibson
Giant expanding gas bags slowing you down. Yeah.
Paul Thurrott
By the way, sorry to interrupt, but you mentioned Neal Stephenson earlier. I'm pretty sure he just walked by. Yeah, that guy looks exactly like.
Steve Gibson
Oh, he sure does.
Richard Campbell
Yeah. Might even be him.
Paul Thurrott
That would be.
Steve Gibson
We should get him on if we can.
Richard Campbell
Yes.
Steve Gibson
Excuse me, sir, Are you Neal Stephenson?
Paul Thurrott
Yeah. No. Well, come on anyway, you look like him.
Steve Gibson
That's all that matters. Anyway, what were you saying?
Richard Campbell
I'm just saying, like this vehicle has. Is going to lose the ability to control its pointing direction because of drag soon, so it can't be rescued. So if they don't solve this with link, they probably lost that.
Steve Gibson
So this is the kind of thing. These geek outs are great. They are all there at run his radio. And you've done nuclear power, you've done
Richard Campbell
space, every kind of alternative energy, everything. Did a show on antibiotics, which is one of the hardest things I've done. I had to cram two medical techs to get that thing right. But I am prone to such joints.
Steve Gibson
Paul of course has a website. Thorat.com Paul has a website. We all wish, we all wish him well.
Richard Campbell
We love his website.
Paul Thurrott
It's cute.
Leo Laporte
Unfortunately no one knows how to spell it.
Paul Thurrott
Yeah, exactly.
Steve Gibson
It's a great website. You should become a premium member. You'll get copies of Paul's books if you do. And you also get access to additional content. But it is the website to go to if you want to see what's going on with Microsoft and you write it all up together. They do Windows Weekly which is Normally every Wednesday, 11am Pacific, 2pm Eastern. You can tune in, watch us live or download it from Twitter TV ww. If you're interested in Microsoft, that's the show. So glad you guys could come to Las Vegas and do this show with Steve. Steve.
Paul Thurrott
Of course I only did it to see Steve. I don't really. I know the rest of it. I don't care.
Steve Gibson
I know what.
Richard Campbell
You know what?
Paul Thurrott
Let's.
Steve Gibson
Let's go to all go to dinner together. I think we have a steakhouse and
Leo Laporte
the ladies will be on All Spies.
Paul Thurrott
They're all going to be shiny, very cold. Yeah.
Steve Gibson
Actually my wife, poor Elisa is. Is. Is hurting fans at this point. So she's. She may be less relaxed. Steve is@grc.com his bread and butter. I talked to somebody the other day. He's been a spin right owner. So I take a Tai Chi class, right?
Leo Laporte
Yeah.
Steve Gibson
And this guy, I've been in this Tai Chi class at this guy for two years and he said he used to work in security. He used to do firmware for. Oh we talking about it was it Fortinet? It's one of the hardware devices and he would write the firmware and for the first time in two years he said well, have fun in Vegas with Steve. I said what? How'd you know that? He said well, I listen to security now he's a fan.
Paul Thurrott
Nice.
Steve Gibson
And he said I have Spinrite. I've had Spinrite for 30 years. I said isn't it great? Steve gives you free updates. You continue to get free updates. He says, Yep, I've got 6.1. If you have a hard drive, you have mass storage of any kind. Ssd. And nowadays if you have an SSD you want to take really good care of it, you should have spin. Right?
Paul Thurrott
Gold. It's gold.
Steve Gibson
The world's worth.
Paul Thurrott
Worth its weight in. Yeah.
Steve Gibson
Mass difficult to replace. Mass storage enhancement, performance enhancement repair maintenance. You gotta have it and it'll keep your SSD running for a long time.
Leo Laporte
Keep it going fast.
Steve Gibson
Yeah.
Leo Laporte
Yep, yep.
Steve Gibson
He also does a really nice program, the DNS. We were talking about it earlier. The DNS Benchmark Pro. Yes. It's a Windows program. Maybe until I program should be. Maybe it will be a swift program.
Leo Laporte
We don't know.
Steve Gibson
But you can get that also@grc.com now if you want to send Steve email, the kinds of questions we've been answering today, GRC.com email. You need to whitelist your email address before you email him or it'll just go in the spam bucket. But he has a way of verifying it right below that. I know everybody needs that.
Paul Thurrott
I need this.
Steve Gibson
I mean, shush. I was telling Steve I can't find anything in email anymore. Right below that there are two checkboxes. If you want more email, there are two checkboxes. One is the weekly show notes. Steve work works very hard. Usually 20 plus pages of information, links, photos, everything, everything we do on the show. He will send that to you every week. And below that there is a checkbox for mailing list. He never will send you anything from because it only comes out when he's
Leo Laporte
got a new product which is pretty much never rare.
Steve Gibson
So it's a rare and it's a wonderful welcome gift when a mail arrives from Steve.
Richard Campbell
Nice.
Steve Gibson
So GRC.com for that. He also has copies of the show security. Now this show he has a 16 kilobit audio version which no one should listen to.
Leo Laporte
Well, if you have your scratchy record version.
Steve Gibson
Yeah. But it's small. Yeah. It has a virtue in a being very, very small.
Leo Laporte
Elaine who does the transcriptions used to have a bandwidth throttle satellite link and so she needed to really budget her bits.
Steve Gibson
What was that called? Hughesnet or what was the name of
Richard Campbell
there a couple of Hughesnet. Hughesnet. I did a bunch of those.
Steve Gibson
Yeah. And they had that fair use policy which meant don't use the don't use policy and then you could always have bandwidth anyway. 16 kilobit but also 64 kilobit which sounds just fine. And the show notes are all available@grc.com we keep this show also at our website, Twitter, TV SN. We do stream it live. And right now we have about 500 people been watching us through the show live on YouTube, Twitch, X.com, facebook, LinkedIn kick. We see all of you. Thank you. It's great to see you here. Thank you for being here. Except for that one scammer on Facebook. Did you see that one?
Richard Campbell
Yeah.
Steve Gibson
He said, hey, if you're having trouble losing, you lose your information, we can help you. And I thought, no.
Leo Laporte
Bye, bye.
Steve Gibson
Don't. I hope they kicked him.
Paul Thurrott
Oh, man.
Steve Gibson
They're everywhere, folks.
Leo Laporte
And I see that we have a comment. Steve doesn't get any spam and Paul is ready to hit the bar.
Paul Thurrott
Both of those things are true.
Steve Gibson
Yes. But just follow if you want the best whiskey. Just follow Richard. He knows where it's kept somewhere.
Richard Campbell
We had a new one last night.
Steve Gibson
Something weird in his closet.
Paul Thurrott
I'm looking for something in a teeny.
Steve Gibson
Yeah, I bet you are. Now I've lost my thread. Oh, yeah, we stream it live. You can watch us live if you're in the club. Of course you can watch live in the club. Twit Discord as well. We are very grateful to all of you club members for making this as possible.
Leo Laporte
Why do I look small compared to the other?
Steve Gibson
Are you slumping?
Leo Laporte
I don't know.
Steve Gibson
Sit up, old man.
Leo Laporte
I got a little too short.
Steve Gibson
Chair. Yeah, I'm sitting on a stool eventually.
Paul Thurrott
By which I mean Yoda sized, which is actually appropriate.
Steve Gibson
He's the Yoda of our show. What else is there to say but just, I guess. Thank you. Thanks to our wonderful team back home at the ranch. John Ashley helped us out. Kevin King, of course, here in the studio. Anthony Nielsen. Benito will be working on the show later.
Paul Thurrott
Anthony's done a great job of staying awake today. I just want to point that out. He did a nice turnaround.
Steve Gibson
Could not hold back when you started talking about physical discs on the places, you know, on Windows Weekly, he. He had stepped in.
Paul Thurrott
Yeah. Really animated.
Steve Gibson
All little passion. Little passion going.
Paul Thurrott
The physical media guys are always like that, you know.
Richard Campbell
Yeah.
Steve Gibson
It's fun for us to get out of that, you know, Twitter is a fully remote operation. None of us are in the same place. The only people who are are my wife and I, Lisa and I. And that's it.
Paul Thurrott
Right.
Steve Gibson
And sometimes we wish we were in different locations. So. No, we have each other. No.
Leo Laporte
Yeah.
Steve Gibson
We want to do the whole show in Cabo San Luis. No. Yeah, but it is a remote group and so it's nice when we can get together and do some stuff like this. I think it really makes it so much fun. It's really fun to come to trade shows too. You know, we were to Paul and I were talking about this.
Paul Thurrott
I love this.
Steve Gibson
It's like nothing else.
Richard Campbell
Like it.
Steve Gibson
Yeah.
Paul Thurrott
Well, it's going to the show but not going to the show. Honestly, kind of awesome.
Leo Laporte
At the show.
Paul Thurrott
Yeah.
Leo Laporte
Be at the show, but actually go to the show.
Paul Thurrott
At the show.
Steve Gibson
Yes, exactly. Ah, there we go. Now we got a shot of it. Is this is just one of several halls. This is the business hall. And of course, tomorrow the hackers come to town.
Paul Thurrott
Yeah.
Steve Gibson
Because defcon. Defcon. And that will be. I've never done a defcon. I've always wanted to be very interesting.
Paul Thurrott
Yeah.
Richard Campbell
I'm going to go home.
Steve Gibson
I think we should all get out of here while it gets good. Thank you, Anthony Nielsen. Appreciate the hard work you did. He, he got this whole gear, got it all set up. That's great rig and yeah, I mean it's a portable small rig. We have a mobile rig now. We even have an on the air battery powered on the air. Like we were concerned we would be rushed. That turned out not to be a problem.
Richard Campbell
No.
Steve Gibson
All right. So thank you everybody. We appreciate it. We will see you soon. Come back and join us next week, next Tuesday for security. Now we'll be back at our regular
Leo Laporte
day after for Windows Weekly.
Steve Gibson
Windows Weekly on Wednesday.
Paul Thurrott
Good to see you, sir.
Leo Laporte
Likewise.
Steve Gibson
Take care.
Richard Campbell
Good fun.
Steve Gibson
Hi there. Leo Laporte here. I just wanted to let you know about about some of the other shows we do on this network you probably already know about. This week on Tech. Every Sunday, I bring together some of the top journalists in the tech field to talk about the tech stories. It's a wonderful chance for you to keep up on what's going on with tech, plus be entertained by some very bright and fun lines. I hope you'll tune in every Sunday for this week in Tech. Just go to your favorite podcast client and subscribe this week at Tech from the Twit Network. Thank you. Security now.
Leo Laporte
A burst pipe, a dead water heater. The AC calling it quits. Who do you call? Homeserve is an easy way to handle unexpected home repairs with plans covering stuff basic homeowners insurance usually won't.
Steve Gibson
Instead of scrambling for a contractor, you
Leo Laporte
make one call to get the repair process started. Join the millions of customers who trust HomeServe right now. Go to HomeServe.com podcast for 50% less your first year that's HomeServe.com podcast savings compared to renewal Price void in Florida still waiting in line again, that's time you will never get back. Save time and money with stamps.com over 4 million businesses have have skipped the line with stamps.com join them to save up to 90% off carrier rates from your computer or phone right now. Print postage for certified mail, registered mail and packages in seconds. Then schedule a pickup right from your home or office for a limited time. Go to stamps.com and use code podcast for a free welcome gift. Taxes and fees apply.
Paul Thurrott
Hello.
Leo Laporte
Look what T.J. maxx dragged in. The Devil Wears Prada 2 is now
Steve Gibson
streaming on Disney plus and Hulu.
Leo Laporte
We are digital. We are downloadable.
Paul Thurrott
We are streamable.
Leo Laporte
The fashion event of the year is certified fresh.
Steve Gibson
Pull yourself together. We have work to do.
Leo Laporte
Critics say it's smart and witty and the perfect sequel.
Paul Thurrott
That's all.
Steve Gibson
Get Runway ready for The Devil Wears
Leo Laporte
Prada 2 on Disney plus and Hulu.
Steve Gibson
Rated PG 13.
Podcast: All TWiT.tv Shows (Audio)
Host: Steve Gibson (with Leo Laporte, Paul Thurrott, Richard Campbell)
Location: Recorded live at the Black Hat conference, Las Vegas
Theme: Deep Dive on the Impact of AI in Security—Roundtable at Black Hat
This special edition of Security Now was recorded live from the show floor at the Black Hat security conference in Las Vegas, featuring not just Steve Gibson and Leo Laporte, but also Paul Thurrott and Richard Campbell. In a rare, in-person roundtable, the group dives into how artificial intelligence (AI) is reshaping the entire security landscape—from the tools hackers and defenders use, to the fundamental ways we interact with technology, programs, and even each other. With a focus on both AI’s rapid advancements and the profound ethical, practical, and security questions it raises, the panel brings in community questions to spark debate. The tone is energetic, sometimes irreverent, but always thoughtful.
Steve Gibson: Completely “all in”—runs a suite of advanced agentic AI tools (like Codex, Claude, Hermes) for development, even local LLMs for privacy and control.
Leo Laporte: Fascinated user, but not yet comfortable with AI writing code; uses AI (notably Claude) as a conversational partner and helper for complex tech setups.
Richard Campbell: Uses AI as both “beside” (assistant/advisor) and “inside” (integrated into systems like Home Assistant) and sees exponential growth in capabilities for parsing logs, triaging incidents, and automating operational routines.
Paul Thurrott: Skeptical, cautious early adopter—uses AI for code when stuck, but not for writing; prefers writing his own articles and code.
Listener Experiences:
AI & App Development Process:
On AI agents distrusting each other:
On dual-use restriction:
On the scope of change:
On skepticism:
| Timestamp | Segment / Topic | |-----------|-----------------| | 00:33 | Introductions, Black Hat setup | | 02:04 | AI dominating security industry discussion | | 07:12 | Panelist positions and initial stories | | 11:01 | AI writing and testing code (Steve’s workflow) | | 15:48 | Leo’s experience using Claude for home networking | | 18:03 | AI’s strength: Parsing logs and incident detection | | 21:26 | Local AI ownership for privacy and sovereignty | | 35:29 | AI’s cryptographic impact—“sky is not falling” | | 52:05 | The Bitcoin/ColdCard wallet hack—AI uncovers latent bug | | 54:04 | Need for open source project support with AI | | 63:41 | Context window bottlenecks in AI vulnerability remediation | | 71:43 | GRAM and knowledge partitioning in LLMs | | 85:49 | AI-generated media: “Will Smith eating spaghetti” | | 101:00 | LLMs & prompt injection: the “token soup” problem |
The episode concludes on a philosophical—almost optimistic—note: AI has moved from an abstract, sci-fi concept to a tool that is fundamentally transforming the security world (and much else). Yet, this rapid evolution creates new ethical and practical dilemmas: Who gets to control knowledge? How do we build robust trust and security into non-deterministic systems? Can privacy survive? Will open models triumph over controlled, “licensed” knowledge regimes?
All four hosts agree—these are the formative years of a new era. Staying engaged, critical, and open to experimentation is essential for both professionals and hobbyists. The sense of community—at Black Hat, in the TWiT chat, or in home labs—matters more than ever.
Steve Gibson:
“We’re at just the beginning. There’s a lot that still can be done. It wouldn’t be changing every day if we were anywhere near maturity.” [63:41]
Leo Laporte:
“What AI is... is astonishingly accessible knowledge. That’s what it does.” [73:01]