There's Nothing an LLM Can Screw Up That the Cloud Didn't Do First

Summary

Podcast Summary

CISO Series Podcast
Episode: There's Nothing an LLM Can Screw Up That the Cloud Didn't Do First
Date: June 9, 2026
Hosts: David Spark, Andy Ellis
Guest: Danny Jenkins, CEO of ThreatLocker

Episode Overview

This lively conversation dives into how the cybersecurity community is grappling with challenges posed by AI agents, LLMs (Large Language Models), and the evolving nature of identity and access in a cloud-first world. With a touch of humor and candor, the hosts and guest explore recurring security pitfalls, the comparability between past cloud mistakes and current LLM-driven missteps, and the philosophical depths of metrics, trust, and device identity. Practical anecdotes, industry insights, and debates around governance, controls, and the limits of “vibe coding” make this episode especially engaging for current and aspiring security leaders.

---

Key Discussion Points & Insights

1. Identity, Authorization, and the Perimeter Problem

• Topic: The nuance between “authorization” and “appropriateness” in an era of human and agentic (AI-driven) identities.
• Andy Ellis [04:59]:
  • Challenges the idea that agentic workflows are less predictable than humans.
  • Quote:

    “Agents are far more predictable than humans. Agents will take all of the permissions they have and go use them. The human challenge is humans don't use most of their permissions because we give people way too broad permissions because we don't understand what people do.”

  • Points out years-long accumulation of unused permissions and the absence of real governance.
• Danny Jenkins [06:49]:
  • Default deny and least privilege are fundamental.
  • AI agents introduce higher risk due to their speed and lack of accountability:
  • Quote:

    “The problem with agents is they're much, much faster at causing chaos … and sometimes very wrongly so.”

  • Humans can be held accountable, but agents have “no conscience” and “cannot be reasoned with.”

---

2. Cloud Era Mistakes Repeated with LLMs

• Topic: The industry’s cyclical failure to secure new technology before mass adoption.
• David Spark [09:29]:
  • “We just deployed more attack surface in 12 months than we built in the previous decade.”
  • Comparison of current LLM vulnerabilities to old cloud rushes.
• Danny Jenkins [10:31]:
  • Pattern: Security always trails innovation; open relays in the ’00s, now open AI endpoints.
  • Quote:

    “The security mechanisms haven't really changed that much. Blocking things, only allowing what you need—it hasn't changed. But what happens in the interim is we stuff it with detection and response to try and allow us to move faster.”

• Andy Ellis [12:22]:
  • Goes further: “This is exactly what happened with cloud, with the Internet, with the web, and it is not a mistake … This is how business operates—a new disruptive technology comes along and people are just going to go adopt it.”
  • Advises CISOs to get ahead by enabling safe and effective AI use, not just resisting change.
  • Memorable aside [12:57]:

    “If Caleb [Sima] wants to make me agree with him, he can take me out to dinner because his wife is a fantastic cook.”

---

3. Embracing and Securing AI Internally

• Topic: How ThreatLocker and others are incorporating AI into both defense and offense.
• Danny Jenkins [14:40]:
  • Uses AI to augment code review and product functions, but acknowledges AI is supplementary, not a silver bullet.
  • Quote:

    “People often think AI is maybe a better technology, but sometimes it's just supportive technology. When you combine [old and new techniques] you can get 99-point-something percent accuracy, whereas before you could get 95.”

---

4. Metrics: Perverse or Purposeful?

• Game: What's Worse? [18:10]
• Scenario:
  • Metrics show improvement, but you don’t know why, vs. No improvement in metrics, but you know you’ve improved.
• Andy Ellis [18:56]:
  • Argues metrics are often flawed, leading to misinterpretation and false confidence.
  • Quote:

    “If your metrics claim you're getting better and you have no idea how you're getting better, you're probably not actually getting better.”

• Danny Jenkins [22:00]:
  • Prioritizes tangible, well-implemented controls over surface-level metrics.
  • Quote:

    “If you had to choose one, I'm always choosing good controls.”

---

5. Rethinking Authentication and Identity

• Please, enough, no more: Authentication [24:07]
• Andy Ellis [25:27]:
  • Rejects the human-only view of authentication; advocates for authentication of user-device sets.
  • Quote:

    “Most of us do not employ humans...the user that works for you is a user plus a set of devices.”

• Danny Jenkins [26:18]:
  • Agrees, adding even “engineers get phished.” Need for device identity over pure user authentication.
  • Phishing anecdote [27:32]:

    “It's embarrassing to me because I always thought salespeople … would get phished. Engineers don't get phished, and engineers got phished.”

• Practical Reality [30:53]:
  • For the vast majority: If a user is trying to log in from a non-approved device, it's a red flag for attack rather than business need.
  • Statistically, device rotation is not common outside health care and similarly unique environments.
  • Quote (Danny Jenkins) [32:10]:

    “More likely that someone logging in from a device that is not authenticated or is not a device issued by the company or approved by the company is an attacker.”

---

6. Can Security Solutions Be "Vibe Coded"?

• Topic: The pros and limits of AI-generated, intent-driven code.
• Andy Ellis [34:49]:
  • Fun caution: “Basic functionality is not what people tend to buy … It's the details and the hard work of making sure things work at scale.”
  • Advocates for onboarding AI agents just as you would an employee: with proper guides, guardrails, and security expectations.
• Danny Jenkins [36:37]:
  • AI can solve small, narrow tasks but cannot yet replace the iterative, collective development and usage-driven refinement of complex security tools.
  • Quote:

    “AI is going to give you a very, very crud piece of software that doesn't do a great job that you can then refine...But you can't build a security solution for your company.”

• On Dropping Vendors for AI Solutions [40:31]:
  • Some tools, especially those little more than data dashboards or niche solutions, are being replaced by vibe-coded scripts.
  • Broad consensus: AI can help with hyper-specific problems, but comprehensive, at-scale security products offer irreplaceable value.

---

Notable Quotes & Memorable Moments

• Winning in Security:
  • Danny Jenkins [00:03]:

    “I love winning. I love it when we manage to stop something, stop a hack or really, really annoy somebody on a server who’s trying to take over a system.”

• On the Predictable Unpredictability of Agents:
  • Andy Ellis [05:10]:

    “Agents will take all of the permissions they have and go use them. The human challenge is humans don't use most of their permissions.”

• Attack Surface Acceleration:
  • David Spark [09:29]:

    “We just deployed more attack surface in 12 months than we built in the previous decade.”

• On Trusting Devices, Not Only Humans:
  • Andy Ellis [28:35]:

    “If you are not validating a machine credential … then you don't actually have a good authentication system in place.”

• Device as Red Flag:
  • Danny Jenkins [32:10]:

    “On ThreatLocker we had more successful logins from rogue devices from attackers than we did from people [deciding], ‘Oh, I want to use my home computer to log into my email.’”

• AI for Security Development:
  • Danny Jenkins [36:37]:

    “You can build individual tools that will solve individual problems, but they won't think about every component.”

---

Suggested Listening Path & Important Timestamps

• [04:59] Key segment: Why agents are faster and scarier than people in security workflows
• [10:31] Key segment: Cloud pattern repeats with LLMs—how businesses trip up again
• [18:10] “What’s Worse?” Game: The value of reliable metrics vs. reliable controls
• [24:07] Authentication Rant: Why identity is more than the human, and the fallibility of engineers
• [34:49] Security Vibe Coding: The real potential and risks of AI-powered, intent-driven software

---

Closing Thoughts

• Danny Jenkins [42:50]:

  “Security is difficult but it doesn’t have to be. We don’t have to chase our tail … we can put simple controls in place...and we can change our environment from constantly having to fight malware and alerts to actually just being secure without interfering with users’ productivity.”

• Practical Wisdom: Layers of security—from device trust to least privilege—matter more than frantic detection or a bevy of metrics. Each technology shift (cloud, LLM, AI) is an opportunity for security leaders to get out ahead, not just play catch-up.

---

For more insights and resources, visit cisoseries.com.

Transcript

[David Spark]

What I love about cybersecurity. Go.

[Danny Jenkins]

I love winning. I love it when we manage to stop something, stop a hack or really, really annoy somebody on a server who's trying to take over a system. I just love winning.

[David Spark]

It's time to begin the CISO Series podcast.

Welcome to the c. Welcome to the CISO Series podcast. My name is David Spark. I'm the producer of the CISO series and joining me is my co host. It's Andy Ellis, the principal over at duha. Andy, say hello to the audience.

[Andy Ellis]

Good afternoon, folks. Or depending on when you are in the world, good morning, good evening, or good night.

[David Spark]

We are [e-mail] where you can find lots of our other wonderful programming. We have four other shows on our network. Why not explore and listen to some other wonderful programming?

[Andy Ellis]

Because this is the best program at

[David Spark]

this very moment, while people are listening, this is exactly what they should be listening to. Nothing else.

[Andy Ellis]

Yeah. Only when you're done, can you go listen to the other ones.

[David Spark]

Our sponsor for today's episode, a phenomenal supporter of the CISO series. We adore them. They adore us. It's a mutual love and appreciation for each other. It is Threat Locker. Meet the world's leading zero trust platform. Allow what you need. Block everything else by default, including ransomware and rogue code. That is Threadlocker. We're going to be talking more about the cool stuff they're doing. You're going to want to listen to that a little bit later in the show. But before I introduce our guest who is from Threadlocker, Andy and I just saw each other in Boston because we did a live show in Boston which will be airing actually shortly after this episode. And the reason I coordinated this, Andy, and I think I told you, it's because I went to my high school reunion.

[Andy Ellis]

That's right, folks. If you want to sponsor a live show, all you have to do is figure out where David wants to be.

[David Spark]

That helps. But this, this was all sort of coordinated back in November of last year, but I hadn't stepped on my campus since I was 18 years old.

[Andy Ellis]

Oh, goodness.

[David Spark]

I went to an all boys prep school in Boston. And I think what's interesting, have you, first of all, have you attended like a college or a high school reunion yourself? Andy?

[Andy Ellis]

Nope.

[David Spark]

None.

[Andy Ellis]

Nope. Have you? I left and have not been back. I mean, I've been back on my college campus because it's right down the street, but I didn't do the reunions.

[David Spark]

Well, I think what happens when you don't See people for decades. The game becomes who has fallen apart most, you know, and it's just like, wow, I didn't think that was going to happen to you kind of a thing. But then there's some people who keep it together and it's great, you know,

[Andy Ellis]

I think what I love best about this is all the winning, David.

[David Spark]

It goes back to the winning thing. So that becomes quite amusing. Like, well, well, you know, you were an athlete at one point. I guess things have changed now just a little bit. Anyways, it was thrilling to see everybody, but it was just sort of a shock when you don't see people for that long because people change after they turn 18. I've noticed.

[Andy Ellis]

It does happen just a little bit,

[David Spark]

I hope, just a tad.

[Andy Ellis]

I hope that people change after they turn 18, let's put it that way.

[David Spark]

I would hope so. And slightly mature.

[Andy Ellis]

Yes.

[David Spark]

All right, let's get to today's show at hand because we're not going to be talking about my reunion. If someone wants to call me and ask or talk to me or email me about my reunion, happy to chat with them about that later. But let's not waste any more time here. It's a guest we've had on many times before. We adore him. He. He supports us, we support them. It is the CEO of ThreatLocker, our sponsored guest, Danny Jenkins. Danny, thank you so much for coming back.

[Danny Jenkins]

Thank you for inviting me today.

[David Spark]

Once again we've got identity issues.

Quote, authorization is not the same as appropriateness. A perv garg tries to cut through the identity is the perimeter consensus and he points out that IAM identity access management can confirm an agent was allowed to access the system query something like a salary database and join it with performance reviews. But it can't tell you whether it should have. Agents need broadstanding permissions to function at all access patterns. We'd never grant a human employee or at least never admit to doing so. I'm going to start with you, Andy. If least privilege breaks a gentic one workflows and we're talking about agentic workflows versus actual humans as identities. What does appropriate access governance even look like? And how do you enforce policy against behavior you can't predict in advance? Because agents don't have predictable behaviors, do they?

[Andy Ellis]

Well, humans don't have predictable behavior. Let's actually start from that problem.

[David Spark]

Well, more than agents I would say that.

[Andy Ellis]

No, no, no. I actually think agents are far more predictable than humans.

[David Spark]

Okay.

[Andy Ellis]

Agents will take all of the permissions they have and go use Them. The human challenge is humans don't use most of their permissions because we give people way too broad permissions because we don't understand what people do. The core of the problem is we don't actually govern identity or authorization, however you want to look at that. What we do is we sort of say, oh, like Danny has just come into the organization. Let's just clone somebody else's permissions and give them to Danny. Nobody actually knows what Danny's job's gonna do. He changes from one organization to another. So we leave all of his old permissions there while he goes over in case he gets called. And over the course of 15 years at a company, Danny ends up with access to everything, but he's never using it. Then Danny goes and deploys an agent, and the agent's like, oh, I have access to these things, and I could probably use that in answering some question. So I'm gonna grab access to everything. And that's just a key piece of it, which is the thing we've never solved, which is governance over what people do, because we don't know what they do. Authorization and permissioning is just the shadow reflection of what the business needs the person to do. But we don't understand that. So instead we treat it like a ground truth. We give it lots, and then agents just abuse it. So this is not me saying this problem isn't worse. This is me saying this has always actually been the worst problem we have. Now it's just gotten even worse.

[David Spark]

So, Danny, part of the core philosophy of Threat Locker is this default deny approach. And by default, you limit privileges. Yes. Or just limit a lot of things beyond just privileges.

[Danny Jenkins]

So I think it's really about allowing what you need and nothing else.

[David Spark]

Right.

[Danny Jenkins]

And humans, obviously, they tend to collect privileges over time. You're right. And one of the things that we often do is look at those and say, here is a privilege that hasn't been used for three years. This shouldn't be here anymore because it's not currently being used. The problem with agents is not they're more or less predictable. The problem with agents is they're much, much faster at causing chaos. And a human, if they start stealing data or uploading data to the Internet, they tend to do it quite slowly. If they decide they're going to delete something, they tend to do it quite slowly, and you have some kind of chance of detecting it before it becomes bad. Agents do it very, very quickly and sometimes very wrongly so.

[David Spark]

Well, then how then does the least privileged behavior or the way you monitor it, if you're dealing with agents that are moving at a much faster speed than humans, how does this change? I mean, how does speed come into the equation, I guess is my question.

[Danny Jenkins]

So I think the first thing is, just like a human, you have to start off, people have given agents more power than humans in many cases. And if you look at some of these AI tools, they're just asking for absolutely everything out of the box. And we shouldn't be doing that. We should be saying, why does this need this permission? Why does it need access to these files? Why does it need access to the Internet? What doesn't it need access to on the Internet? So that's step one. And then the second step is checking what you have given access to and just reviewing it again periodically to say, is there still a valid reason for this agent to get access? But part of the problem with agents as well is there's no accountability. So as a human, if I hire somebody and I bring them into my business and they do something wrong, there is a consequence for doing something wrong. Now, the consequence may be worse for the business than it is for the individual. But in most cases, if they intentionally steal data, upload something to the Internet, they're going to go to jail, they're going to get fired, something like that. With agents, they can just do things without any consequence. So you have to almost put more constraints around it because what you have is the most.

[David Spark]

And also the agent itself, you should also throw in there. The agent itself doesn't have a conscience. So it's not stopping itself or regulating itself.

[Danny Jenkins]

No, it doesn't have any conscience. It cannot be stopped, it cannot be reasoned with, and it will do whatever it thinks is the appropriate thing to do. And that could be deleting a database. It could be uploading files to the Internet. It could be encrypting files.

[David Spark]

What about this AI security challenge?

Quote, we just deployed more more attack service in 12 months than we built in the previous decade, end quote. After seeing a surfeit of MCP servers with no authentication and a bevy of malicious skills in the LLM marketplaces in just a month, Caleb Sima of White Rabbit has a simple diagnosis repeating the cloud mistake. But this makes me wonder if this is a mistake or this is just a pattern. New technology deploys, businesses wait to see if it sticks. Once it does, startups scramble to build solutions. Buyers spend months evaluating vendors who are barely less green than they are. So meanwhile, attackers don't wait for procurement cycles. So, I mean, by the way, this sounds like the whole cybersecurity industry. So I gotta ask you, Danny, first, how should security leaders think about protecting infrastructure when the vendor ecosystem securing it doesn't yet exist or is as nascent as they are?

[Danny Jenkins]

So I don't necessarily think it doesn't exist. I think the problem is the way we've looked at security for the last three decades is about finding a threat and responding to that threat. And that is definitely part of security. But the reality is, even when we think about going back to the early 2000s, you get an email servers, they were all open relay. Because we came up with this great technology, email. We all wanted to make it as simple as possible. So you'd put an SMTP server out on the Internet, you'd allow it to send email to anyone. Very, very quickly, people started abusing servers. And then we created these lists of bad email servers and we'd start listing them on it. And then eventually people stopped leaving their servers as open relays. And today, if you see an open relay, it's very, very rare. I think the problem is we go out and say, let's open up this technology to the whole world, and we want to do it as fast and simple as possible. And that is the pattern. Like, how do I get it as fast and simple as possible? The security mechanisms haven't really changed that much. Like blocking things, only allowing what you need. It hasn't changed. But what happens in the interim is we stuff it with this detection and response to try and allow us to move faster. And then we go back and ultimately come to this realization, oh, we shouldn't have open relays on the Internet. We do need usernames and passwords on our accounts. We do need your factual authentication. And in the meantime, we were just, oh, let's just throw it out there with nothing. Then let's put some security, then let's put some detection and response around it, and then let's put some real security where it's actually effective.

[David Spark]

All right, Andy, I want to throw this to you. Do you think what we're seeing with AI is really just an echo of what we've seen when new sort of technologies rise and we're not kind of prepared for it? Or is this a very different structural change?

[Andy Ellis]

Oh, no. So I absolutely think this is exactly what happened with cloud, it's what happened with Internet, it's what happened with web, and it is not a mistake. So, sorry, Caleb, I have to disagree with you very strongly. And if Caleb wants to make me agree with him, he can take me out to dinner because his wife is a fantastic cook. If you have not gotten to eat

[David Spark]

at Fang, she has a great restaurant.

[Andy Ellis]

I did for the first time this year at rsa and I will happily go back.

[David Spark]

So wait a second. Let me just pause there. So the new way of engaging with Caleb is to essentially taunt him to take you to his wife's restaurant.

[Andy Ellis]

Yeah, absolutely. I'm going to give that a try. I mean, it might work. So, Caleb, if you're listening, we can argue over this at dinner. But at a high level, this is how business operates, which is a new disruptive technology comes along either because a previous one tanked and the space was there, or the new one is sufficiently better. The new, that people are just going to go adopt it. Like this goes back to the car. Like the automobile was a disruptive technology. People adopted it. And yeah, there were people complaining about, like, oh, these are not horses, and they're gonna be different. The mistake that we make as security professionals is that we wait until the business moves and we kvetch about it and we say, oh, you shouldn't move until you engage with us. That's never going to happen. They're going to move. So either you have fundamental controls in place that will be helpful in that world, or you're playing catch up. That's basically your two choices. Now, conveniently, like, when people move to the web and started to do, like, e commerce in the early days, there were people who came and helped them. And those are the folks who became the first chief information security officers were the people who got out ahead of it. So if you want to advance your career, you should be out ahead of AI and you should be figuring out how you're going to help your marketing team use AI better and more safely. Because what I'm seeing at more and more companies is you have these very lean, agile marketing teams. Sometimes only one person doing the work of six or seven, and they're using AI for everything. So if you're just screaming, no, no, no, you can't do it. That's the failure mode.

[David Spark]

Let me, let me, let me throw this back to you. I'm just kind of shifting gears a little bit. Danny, how have your staff, just in terms of running threat locker in general, how have you guys been embracing AI?

[Danny Jenkins]

So I think there's a few different areas. And the first, there's the business usage side of AI, and we're talking about vulnerabilities. And things are about to get much worse with these new AI models coming Out. Cybersecurity is going to watershed moment. Things are going to get bad, but not nearly as fast and quick enough. But AI is also helping solve those problems. I mean, we have hundreds of thousands of lines of code and we constantly do reviews, we have security teams, we have peer reviews. But at the end of the day, even after all of that, we can throw a piece of code into an AI model and say, tell me potential vulnerabilities in this code and it will find things that the smartest people haven't found because it can look faster and quicker and it helps you move forward as a business. We also use it in various parts of our product, whether it's and interesting enough reclassifying or validating a website categorization. In the old days, we would have machine learning and keyword analysis and image analysis to determine the type of website you're going to. And by the way, it's still more accurate than AI today, but with a 96% of sites classified as explicit, correctly classified. When you're using keywords, when you use AI, it's like 70% correctly classified. When you combine them together, you could actually get 99 point something percent accuracy, whereas before you could get 95. People often think AI is maybe a better technology, but sometimes it's just supportive technology. And then of course we have in some respect the benefit of attacks. Increasing malware, increasing phishing attacks, increasing phishing attacks, getting properly written in English now and that allows us to deliver good security tools. So obviously as a company we also get to benefit from, hey, we're selling the shield here. We're selling the solution.

[David Spark]

ThreatLocker is extending zero Trust beyond Endpoint control. With the recent release of Zero Trust network access and Zero Trust Cloud access, organizations can now control how users connect to both internal systems and cloud applications. Access isn't based on credentials alone. It requires the right user, the right device and the right conditions. Users are granted access only to specific applications, never entire networks or environments. Now that matters, because attackers aren't breaking in, they're logging in. We've heard this plenty. Yes, as we've seen in recent large scale CRM breaches, stolen credentials and misconfigurations can expose massive amounts of sensitive data. ThreatLocker's Zero Trust Network access ensures internal resources are never exposed and only reachable through tightly controlled policies. Zero trust control access extends those same protections to SaaS applications. So even if credentials are compromised, access is denied without an appropriate device. It's a modern approach to access control that reduces attack surface and stops unauthorized Access before it even starts. You can learn more and start your free trial today [e-mail] do that threatlocker.com CISO and do me a favor, add that CISO. Easiest way to let ThreatLocker know you heard about them through the CISO series.

It's time to play what's Worse.

Danny, I know you're aware of this game. You've played this game. I always stress that I like it when people disagree with Andy, but I make Andy answer first. So you can agree or disagree. This one's an interesting one, Andy. We haven't had one like this before. I don't think so. Get ready.

[Andy Ellis]

Ooh.

[David Spark]

Comes from Carrie Johnson of Fishbusters. And here's the scenario. Two scenarios. I'm sorry, you have metrics that show improvement, but you cannot prove why. So it's showing something's good, but you have no clue how it happened. Or you have metrics that show no improvement, but you know you have had improvement. Which one's worse?

[Andy Ellis]

Okay, so this one is funny because it feeds into one of my favorite conversations, which is that most metrics that are in use, especially in the security profession, are perverse metrics. They don't actually show you what you think they show you.

[David Spark]

Okay?

[Andy Ellis]

And I give like. One of my favorite examples is like the average time to patch vulnerabilities, which often has some weird denominator in it. Like, is it the ones that were closed or is it the ones that are currently open or closed in the period? And in fact, if you go look at a bunch of talks I've done, this is one of the examples I use on when I say how to build a security program is make sure your metrics survive against perversity. And I don't mean perversion, just the perversity of the world. Like, oh, if you, if you find new vulnerabilities, you close them immediately, but you only report once a month. Nobody will ever see those vulnerabilities in your metrics. But you're like, oh, but I know we did something good. So the underlying question here that I actually have to ask is, are these metrics actually really good, high quality metrics, or are they not? And I'm going to actually go with the. Because it's most security programs we're going

[David Spark]

to, we're going to assume they're. They're good. That.

[Andy Ellis]

Oh, see, I would assume they're not good.

[David Spark]

Well, you're assuming that you're trying to

[Andy Ellis]

create good metrics because one of the situations is you know you got better and the metrics don't show it. Therefore, it's not a good metric. If you don't understand how the metric got better, more likely than not, it's not a good metric. Like I once bought from a security vendor tool that sat on the network and detected lateral movement so that we could find things coming in. And our IT team, well, then, now

[David Spark]

we're just blaming the metric, and so it doesn't really matter.

[Andy Ellis]

Yeah, no, I'm totally going to blame the metric because I think this is a metrics problem. I think that you are worse off if your metrics tell you you're getting better, but you have no idea how you're getting better because you now are lulled into a sense of false confidence. You didn't actually get better. Your metric just tells you you are.

[David Spark]

But this. Couldn't this be the Mr. Magoo version of cybersecurity, where things are, you know, you're avoiding all the problems as everything's crashing around you?

[Andy Ellis]

It's possible that it could be, but let's just go with, like, what we've normally seen. Like, I've seen reports from tools that said, look, everything's great, but it turns out they weren't looking for the right things anymore. They were based on old technology. So our IT team thought everything was golden. Everything was not golden. I'm saying if your metrics claim you're getting better and you have no idea how you're getting better, you're probably not actually getting better. And if you know you're getting better and your metrics don't show it, it's an opportunity for you to figure out what your metrics ought to be.

[David Spark]

Okay, interesting argument there. But this is assuming that the metrics are the problem.

[Andy Ellis]

I think that being the default state of the world is a reasonable assumption to start from.

[David Spark]

Okay.

[Andy Ellis]

Because if the metric is good, you should be able to figure it out why it says it's good.

[David Spark]

Danny, you don't have to necessarily accept his theory that the metrics are the problem to start with. What do you think here? Again, he thinks the first one's worth metrics that show improvement but can't prove why he thinks that's the worst scenario to be in versus metrics that show no improvement, but you know they're right. You know you're improving.

[Danny Jenkins]

Okay, I know you want us to fight, David, but I'm gonna give a different reason.

[David Spark]

Okay, that works, too. That works, too.

[Danny Jenkins]

So, first of all, let's just say we have a metric and it's very, very accurate that the number of successful phishing attacks has gone down in our company. And we know that it's getting better. We don't know if attackers have stopped attacking or is our security better. So I think the latter is more important. I think you need to know that you've made improvement to your security in a tangible way. That's one example. Hey, my phishing's not gone down, but no one can log into my accounts because we had implemented something that does device authentication or the other reason I don't agree with the metrics being the priority is if you're using some kind of detection and response tool, you probably have a fantastic tool saying that we've managed to block 50,000 new threats this month. However, when you block everything by default, we don't know whether we blocked any threats. We just know we blocked what you don't need. So I always think the metrics is less important than knowing I've got tangible controls in place. And that's by the way, that's not just a.

[David Spark]

So are you making an argument that you could survive without metrics if you just know you have the right controls?

[Danny Jenkins]

Oh, if I had the choice of saying you have great metrics, great detection and response versus no metrics, no detection response, but good controls. I'm choosing good controls all the time. If you had to choose one, I'm always choosing good controls.

[David Spark]

That's a good argument right there.

[Danny Jenkins]

And by the way, that's not just insecurity. Every time I go to my marketing department, they want to show me a new chart. I don't care about the chart.

[Andy Ellis]

Right. This is one of the things I often tell CISOs when they're like, we try to go talk to the board and we don't know what they want to hear. I'm like, only the CRO has metrics that anybody believes. Because at the end of the day, cash in bank is the only metric that matters. Everything else is you're trying to tell a story. And if you have the controls, you know it's a good story. If you don't have the controls, it's just a story.

[David Spark]

Please, enough. No more.

Today's Please, enough, no more is on authentication. Now, we've always been operating on a broader broken system. Passwords. When passwords, as our form of authentication was introduced, it was broken. So we greatly improved that with mfa, which worked great at the beginning. Now, not so much so. So, Andy, I'm going to start with you. What have you heard enough about with authentication, we're talking about today because this conversation has changed greatly. What have you heard enough about authentication today? And what would you like to hear a lot more?

[Andy Ellis]

So I just want to just point out that when passwords were first introduced, they were a great system because they were designed to let you get close enough to the Roman camp for the guards to spot you and figure out who you were. That's all that passwords were meant for, was to be like, you're coming up.

[David Spark]

What do you mean, the Roman camp? I don't know what the history is in that respect.

[Andy Ellis]

A password was literally like the word that you'd give out to your scouts and hunters.

[David Spark]

Oh, right. What you would say to them.

[Andy Ellis]

Yes, and this is the word of the day. So when they were coming back in, they'd be like, hey, the password is whatever, and you didn't shoot them right away. You, like, let them get close enough and then you would validate a little more. So that was fine 2,000 years ago. We need something a little bit better. They were originally meant for groups, not individuals. But here's, I think the thing I'm really tired of hearing, which is thinking about authentication of a human.

[David Spark]

Oh, definitely that. Yes. That is the story. Now. It's not just humans.

[Andy Ellis]

Most of us do not employ humans. We employ. No, I'm not even talking about the non human identity problem. Okay, but the user that works for you is a user plus a set of devices. And we keep talking about authenticate the human. And what we should be talking about is how are you authenticating this set of the human and their devices? Like the devices should authenticate the human, but over the network, I can't authenticate the human. I can only authenticate the set. And we need to talk more about these sets, whether we want to call them cyborgs or centaurs or like the aircraft carrier commanded by the human. Whatever you want. That's what I want to hear more about.

[David Spark]

All right, I throw this to you, Danny. First tell me what you've heard enough about with regards to authentication. What do you want to hear a lot more? Andy wants to hear about life beyond the human.

[Danny Jenkins]

Andy just stole my answer. So I'm going to say the word identity, though in general, and you're right, it is about authenticating more than the human. And the idea of Dual Factor originally was it's something you have and it's something you know. But the bottom line is it's still a single point. And now just joining this meeting, just opening the Google Docs beforehand, it was such A pain in the backside to go through the code, the push notification, all of this, my password to get in. And at the end of the day, if I'm a weak user, if I'm a human, I could have given it all away just as easy as I could have entered it into the computer. So I think Andy's absolutely right. I think the word identity is often referred to as the identity of a person. It should not be the ident identity of a person. It should be the identity of a person combined with the identity of a device and making sure nobody can access something if it's not from a trusted device. And we had. This is what makes me really, really nervous. We did a pen test a few months ago before Zero Trust World, and we managed to fish, which I thought was five, but it's the eight of our engineers to put their Office 365 credentials into a page and accept the dual factor push and put the code in.

[David Spark]

Ouch.

[Danny Jenkins]

And it's. It's embarrassing to me because I always thought salespeople, they're going to get fished, finance people, they'll get fished. Engineers don't get fished, and engineers got fish. So I think the most important thing is I've heard too much about human identity and we should be thinking about device identity, something that you cannot give away to a very nice man on the phone or a webpage.

[David Spark]

Yeah. Well, so it's interesting you mentioned that it's multiple points and also some of these points, correct me if I'm wrong, Andy, are things that are hidden in the background that are over time, like there are typing patterns. There's where you're logging in from.

[Andy Ellis]

Yeah, I mean, there's lots of ways. Then this is the problem, which is that goes back to trying to validate that it really is the human. And there is a space to that, just to be very clear, like identity proofing, especially in the consumer world, very critical. But in the enterprise world, here's the question that I'm going to have for all of our listeners, which is really simple. How many of your engineers ever will log in from any machine that you don't already know which machine it is? Like, they have their laptop, maybe they also have a desktop.

[David Spark]

That's a good point.

[Andy Ellis]

If you are not validating a machine credential, like, we did this at Akamai, I don't even want to think about how many years ago now, but it was a lot where literally we put an X509 certificate on every device and it had the name of the person in the certificate, so it had your username. So the first thing that would happen when you tried to connect is the device authenticated and said, hey, this is Andy Ellis's computer is trying to connect. Then I could do an out of a channel, push it off to Andy to say, hey, is that really you? This is fish proof. Like you cannot get me to authenticate until my computer has already authenticated itself, so my password is useless. And that's the world we need to be in to say, first we'll trust the computer, then we'll trust the human.

[Danny Jenkins]

I completely agree. And that's exactly what we've been trying to do with Zero Trust cloud access and Zero Trust network access is, hey, you can get my Office365 credentials, you can get me to push a dual factor push. You could try and get me to approve a device, but that device hasn't been approved by the company, therefore it cannot get into my account whether it's salesforce or office or anything else. And we have to verify the device first and then verify the user. When was the last time you heard someone got phished and their phone was stolen at the same time? A lot less likely than someone just got phished.

[David Spark]

One thing I want to mention is talking to you and Rob over at threatlocker is something. And the reason it's to your philosophy as well at threatlocker is that while we think we want to have access to everything and all devices, whatnot, in general, as humans, we pretty much operate kind of the same way. We use the same handful of applications on the same devices over and over again. Yes. You want to make it possible, should someone need to log on to a computer that's not theirs, that they can do it?

[Andy Ellis]

Why? No, no, I disagree right there.

[David Spark]

Well, hold on, hold on, let me just throw this out. What I'm more interested in knowing, and I don't know if you have the stats on this, how often does that happen, Danny? Like if I have a desktop and a laptop, I almost never use another computer. When you've identified the device and the human using it, how often are they not using that device and that human?

[Danny Jenkins]

So I think laptops and desktops, very, very rarely. I think phones, obviously, but again, it's their phone and their phone gets approved and we've had this whole conversation internally and threat locker, do we allow personal phones to access? But if we do, they have to be individually approved. Authentication tokens have to be put on them. They can't just say, oh, I'm going to self Approve any phone I want or give my credentials to an attacker to self approve a phone. I think the reality is people often think about zero trust is taking making it difficult for people, but it's much more nuanced than that. Even myself, like most computers, you have access to everything. As a CEO, I've got a lot of access, but that doesn't mean every program on my computer has to access everything I access. So it's really, hey, you have access to these finance using Excel, you have access to the CRM using this browser, you have access to your email from one of these devices. User experience shouldn't be negatively impacted by that, but security is exponentially increased.

[David Spark]

But do you have any numbers or any insight into if Andy is associated with this computer, this mobile phone? How often does Andy, or just anybody who's not already associated to a device try to use another device? Like how often does that happen?

[Danny Jenkins]

I don't have statistics apart from, but insights is probably more relevant. And here's an example. On ThreatLocker we had prior to ZTCA, we had more successful logins from rogue devices from attackers than we did from people themself saying, oh, I want to use my home computer to log into my email. And that's what's terrifying is it's more likely that someone logging in from a device that is not authenticated or is not a device issued by the company or approved by the company is a attacker.

[David Spark]

So it's just, it's a red flag, essentially.

[Danny Jenkins]

Yes.

[Andy Ellis]

Right, right. There's. There's two different things here. Right. So one is there's a couple industries where this is not true, where people do in fact use different computers. Healthcare is certainly one of them. Like a doctor with visiting privileges may need to log in to a system at that hospital that's not usually theirs.

[David Spark]

Right. Industry would change, but that's a weird one.

[Andy Ellis]

But I think what Danny just pointed out a really interesting thing and what I would encourage everybody to do is think about that use case of the user who wants to use their home computer. I think you only have two reasonable choices. One reasonable choice is you authorize that computer for that user to log in on it, or you make it impossible for that user to use that computer. What often happens is people say, well, by policy you can't do it, but we won't support you. But then that means you don't actually have a good authentication system in place if you're allowing the person to log in from an unknown computer. I don't even want to use the word trust, I just want to say known like when we first did our zero trust implementation over a decade ago, my attitude was, if you've got 12 computers at home, you're going to log in on, I'm going to put a credential on every single one of them so that I know that they're yours. We'll worry about trust later, but the first thing I want to do is say you can't log in from an unknown device. Then we'll worry about trust.

[David Spark]

How can we secure new technology without creating new risks?

Why Pay a vendor $300,000 for a software composition analysis tool when you can build a tailored version during your lunch break? End quote. Ross Young, who is with the CISO tradecraft, laid out the transformational potential. We've moved from SaaS to service as software and LLMs have turned plain English into functional code. Every business analyst is now potentially a gulp developer. So if anyone can Vibe Code a security tool, how does code review even begin? Andy? And if we built it by describing intent in plain English, can we secure it the same way? Can we?

[Andy Ellis]

Yeah. So first of all, just as a warning to everything, they can Vibe code every solution out there. Almost any company, you could build their basic functionality that they have in a weekend. If you're an amazing developer, Vibe coding makes that a little bit faster. But basic functionality is not what people tend to buy, right? Nobody's buying Threat Locker since I've got Danny here, like just to do the basics of what Threat Locker can describe in 30 seconds. It's the details and the hard work of making sure things work at scale. Right. Why do people still use Google? Search isn't hard. Search at high quality is. So be careful. Like if you're Vibe coding your sca, if you wanted really good sca, you might not get that out of Vibe coding. So it's just your cautionary tale. But here's the answer, which is think of your Vibe coding agent, whether it's Claude or something else, as a new sort of fractional employee. And you should be onboarding them and teaching them how to do their job. So rather than trying to teach every person in the company how to code safely, what you need to be doing is handing them the onboarding guide for their agent, so that when they say, hey agent, go build me an X, implicit in that is the skill of we need to do needs analysis. We need to figure out exactly how we're going to do this. How do we defensively code it? How does this interact with our zero trust platform? And with our identity, all of that should be stock and standard so that everything that you go to build meets what your company wants. And it's done so in a repeatable fashion. And that can all be done with intent, in plain English. But rather than trying to teach the business analyst how to do that, you just have to give them the package that gets included right into their coding agent.

[David Spark]

I like that. All right, can, can. Can security solutions be vibe coded? Danny?

[Danny Jenkins]

I mean, look, we obviously use AI, and the answer is no. And I'll give you another reason, slightly different reason why. And I think AI is amazing. It's given us the ability to research and understand things much, much faster. First of all, everything it's creating is based on public knowledge. I kind of liken it to a GPS solution that's 98% right on every street. If you go and ask any AI agent right now, say, hey, right, tell me how to get to the end of the street, it will say, point this degree north and get to the end of the street. And then you'll say, okay, I need to go three streets on. Tell me how to get to here. And it will say, this far, this far, this far, and you're probably going to get right. Tell it to drive you to San Francisco from New York. And 98% right suddenly becomes really, really wrong over a lot of decisions because you're not going to get from San Francisco to New York when you ask an AI agent which direction should go, because it's going to be wrong sometimes. But also it's just regurgitating what it can find on public forums and discussions. When you buy a piece of software, whether it's financial software or whether it's security software, you're buying a package that allows every department to interact with each other. So the person who creates the invoice knows nothing about deferring revenue. The person that puts the quote in the system knows nothing about how we invoice them. So when you have a hundred people asking for individual functions, they do not fit together. So I think large, complex pieces of software can never be done and built through AI. Small functions can be. So if you need something very small, even a security function, hey, write me a script that automatically locks the Windows lock screen after 15 minutes. Very, very achievable through AI. Tell me everything I need to do, secure my environment. Not very achievable via AI. So I think you can't build a security solution, but you can build individual tools that will solve individual problems, but they won't think about Every component. Write me a piece of software that will block software possible. Write me a piece of software that understands all of the problems with blocking software much, much more difficult.

[David Spark]

But then also, well, let me just push back a little bit. Can't you use AI, say where are the. Because this is something that I've used AI but just not in security is where are the areas that I need to be thinking about to secure this piece of software that I haven't considered? And can't the AI start to answer questions like that?

[Danny Jenkins]

A current piece of software, yes, but to build a security solution, no, because it has to. Now go and say I have. And first of all it's collecting this from public posts. So you have to basically go and take all of this information, conflicting information, and someone has to make a decision. And what makes security companies, what makes threatlocker so good is not, not our ability to necessarily build software right first time, but the millions and millions of endpoints and the tens of thousands of customers that have given us feedback and shaped it. What AI can't do is take all that and turn it into, hey, I've come up with a perfect solution. If you think about the first hundred people that used to Outlook, I can promise you it wasn't very smooth. Most of them are still customers, thankfully. But then the thousand people gets a little bit smoother and then the 10,000 people gets even smoother and that's the same with every single piece of software. AI is going to give you a very, very crud piece of software that doesn't do a great job that you can then refine and if you want to get something really quick then that's great. But you can't build a security solution for your company. You can build the bare bones of software company, but so can any good developer do that in a weekend?

[David Spark]

Andy, when we did the recording during Bsides, this came up and I asked both Mike Johnson, who's the CSO of Rivian, and and Sarah Madden, who's the CISO of Convera, are you dropping solutions by creating an AI solution of your own? And they said yes. Are you seeing this pattern happening?

[Andy Ellis]

Well, so what, what I think there's a nuance here which is there are more problems than people can afford to buy solutions for and there's a lot of solutions that are not good. Let's be very honest, there's a lot of vendors out there who are selling things that might as well have just been vibrant coded. And so if you're a CISO today, almost every CISO I know is vibe coding something most of when they talk about what they're building, it's problems that the vendors haven't been solving for them because it's this small niche problem that's an irritant. And so someone's like, look, I'll just go vibe code a thing to fix that irritant for me. Or I've got a vendor that I hate and their product doesn't really work and I can vibe code something that'll give me like 60% of the functionality basically for free. Free. So that I can just get rid of that. That's absolutely happening. What I don't hear anybody saying is I'm going to kick out every single one of my security vendors because I can vibe code at all. That isn't happening.

[David Spark]

No, I haven't. No. But they did say that they were kicking some and it may be like what you described.

[Andy Ellis]

Yeah, the ones I see getting kicked out are often the ones that functionally just do data analysis. If you're a vendor that all you do is you take data from a bunch of sources and you're then going to integrate that and provide a pretty dashboard, you're ripe for being vibe coded out of existence.

[Danny Jenkins]

I think code review is. I mean, I don't know if out of existence is code review is one of those ones that has been right

[Andy Ellis]

code reviews like nobody was doing good code review anyway. So vibe coding is certainly helping. You can have model go do code review for you. And it's much better than having humans do it.

[David Spark]

All right, we have come to the end of our recording and I want to thank you, Danny, and I'm going to let you have the very last word. And also so you, Andy as well, but I first want to thank your company, then be Threat Locker. Remember, if you want to know more about ThreatLocker, Zero Trust Network access and Zero Trust cloud access, go check them [e-mail] CISO. Remember at that CISO. Super easy way to let them know that you heard about them from the CISO series. I know that you're always hiring over there at threatlocker. Still doing that, Danny?

[Danny Jenkins]

Yes, absolutely. Just purchased a new building. Got space for another 800 people here, maybe even more.

[David Spark]

That is amazing. At your growth rate, I'm quite impressed with it. Any last words about Zero Trust network access or Zero Trust cloud access or anything else you want to say about ThreatLocker, Dan?

[Danny Jenkins]

Look, I will just say this. And ThreatLocker has a suite of tools that makes your life easier as a CSERO as an IT Manager and IT director to deploy least privilege in your environment. Whether call it zero trust, call it whatever you want. But I think the reality is security is difficult but it doesn't have to be. We don't have to chase our tail, we can put simple controls in place, we can do basic things and we can change our environment from constantly having to fight malware and alerts to actually just being secure without interfering with users productivity.

[David Spark]

Very very good point and it's basic philosophy of threat lock and we're very impressed. Thank you very much Danny. Thank you very much Andy. And thank you to your company Threat Locker for sponsoring us and to our audience. As I always say, we greatly appreciate your contributions. By the way, send me in more. What's worse scenarios, we need lots more. We greatly appreciate your contributions. And for listening to the CISO Series

podcast that wraps up another episode. If you haven't subscribed to the podcast, please do. We have lots more shows on our website cisoseries.com Please join us on Fridays for our live shows, Super Cyber Friday, our virtual Meetup and Cybersecurity Headlines. Week in Review. This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you're interested in sponsoring the podcast, contact David Spark [e-mail]. thank you for listening to the CISO Series podcast.