It’s a Little Hard to Evaluate New Solutions When You’re Screaming “AI” at Me All the Time (Live in Houston)

Summary

Podcast Summary

Podcast: CISO Series Podcast

Episode: It’s a Little Hard to Evaluate New Solutions When You’re Screaming “AI” at Me All the Time (Live in Houston)
Hosts: David Spark, Mike Johnson, Andy Ellis
Guests: Jarek Beeson (CISO, WM), Jack Lydecker (CISO, Gong)
Date: October 28, 2025

---

Episode Overview

This live episode, recorded in Houston, brings together security leaders to debate real-world collaboration between CISOs and vendors. The focus is on practical advice, the pros and cons of new technologies like AI, the ongoing struggles with cybersecurity basics, and ways for both practitioners and solution providers to work better together. Lively audience participation, games, and candid dialogue fuel a fast-paced, highly engaging session.

---

Key Discussion Points and Insights

1. Security Advice and Culture: Learning from Mistakes (00:03)

• Jack Lydecker shares his “worst-best” security advice: firing users for repeated phishing failures. The well-intentioned but misguided approach prompted reconsideration. The lesson: focus on changing behavior, not punishing mistakes. Fear of reprisal reduces reporting, making everyone less secure.
  • Quote: “If someone thinks they're gonna get fired, they're not gonna tell you. They're not gonna report.” – Jack Lydecker (00:18)

2. Games to Develop Security Team Judgment (02:11)

• Jarek Beeson details his team's “Pick Your Poison” game, modeling it after the podcast's “What’s Worse.” Team members choose scenarios and attempt to select the response Jarek himself would pick in front of the board. It helps staff preemptively understand leadership thinking, improving autonomy, and risk-based decision-making.
  • Quote: “A lot of decisions are made now without me having to be in the room because we play that game.” – Jarek Beeson (02:32)
  • Diverse perspectives are preserved because the majority is often wrong, driving learning.

---

3. Open Source: Reciprocity and Responsible Support (03:32–07:17)

• Prompt: Should maintainers of open-source software go on strike to protest exploitative reliance by enterprises?
  • Jarek: Agrees there’s disproportionate consumption over contribution. While large firms do contribute (code, tests, funds), most benefit passively. Calls for strike are dramatic and could be catastrophic (cf. Log4j, MITRE CVE funding scare). He advocates ecosystem-strengthening—“fix the ecosystem, not weaponize it.”
    • Quote: “The solution should fix the ecosystem, not weaponize it.” – Jarek Beeson (05:16)
  • Jack: Recognizes the desire for balance and points to fundraising or hybrid models (e.g., Wikipedia approach), but warns that going full commercial undercuts open source’s spirit.
    • Quote: “If you move straight into commercial, it kind of defeats the whole core of what Open Source was.” – Jack Lydecker (05:44)

---

4. The Geopolitics of AI-Enabled Threats (07:23–12:39)

• Jarek (citing the 2025 Armist cyber warfare report): Rapid AI advancements mean intent and access, not just resources, are sufficient for nation-state-level cyber threats. The bar to entry is lower, broadening the landscape.
  • Risk Assessment Essentials: Firms must ask about critical operations, supply chain dependencies, hostile nation states, and any business activity that could attract adversarial interest.
  • Quote: “It's not should we include this information? It's how do we operationalize it as fast as we can?” – Jarek Beeson (09:29)
• Jack: Emphasizes that smaller organizations can still become targets, sometimes due to attackers just wanting attention. Even those with limited resources should maintain at least a base level of threat awareness.

---

5. The "What's Worse" Game: People vs. Technology Burnout (14:44–19:21)

Scenario:

• Spend $1M/year on an LLM-based tool that only reduces analyst burnout (but not incidents), OR
• Spend nothing and rely on overwhelmed human analysts who churn every 9–12 months

Both guests prefer investing for staff well-being, despite the lack of direct security ROI.

• Burnout and turnover kill effectiveness; long-term, human damage hurts more.
  • Quote: “If my SOC is unhappy, I guarantee you that's permeating to other parts of my team.” – Jarek Beeson (17:47)
• The audience largely agrees: preventing churn is less damaging.

---

6. Cybersecurity Family Feud: Security Culture in Action (19:29–26:13)

A lighthearted segment reveals common security pitfalls:

• Things you should never share online: passwords, social security number, PII, credit card info, nudes (!).
• Mistakes everyone makes but won’t admit: password reuse, clicking phishing emails, risky behaviors like using public Wi-Fi, sharing credentials.

---

7. Distinguishing AI Hype from Reality (26:19–30:26)

• How do you separate real AI value from gimmickry?
  • Jack: First, ask vendors to quantify impact—can they show evidence? Second, probe their understanding: are they using real models, or just swapping buzzwords?
    • Quote: “If all they're doing is taking your data, throwing it in a giant LLM and spitting it back out, you can do that yourself, probably a lot cheaper.” – Jack Lydecker (27:44)
  • Jarek: Asks vendors what their tool offers that humans can’t, how their models fail, and whether they understand data governance. If they can’t answer, the “AI” is probably just hypercharged automation.
    • Quote: “Vendors with the best failure analysis actually have the best products.” – Jarek Beeson (28:11)

---

8. The Ongoing Struggle: Basics, Fundamentals, and Accountability (30:32–36:35)

• Why can’t we solve the easy problems, like patching and asset management?
  • Jack: The issue isn’t finding vulnerabilities—it’s managing ownership, patch maintenance, and business/testing processes.
  • Jarek: Security never actually patches systems; IT or dev teams do. When CISOs are blamed for breaches, it ignores that they often only provide context, not the “action.” The root is a business problem, not a purely technical one.
    • Quote: “This is not a security problem. This is a business problem... If a doctor diagnoses you... and that person decides not to do those things... Do you blame the doctor? No. But in security, you blame the CISO every single time.” – Jarek Beeson (34:54)
• Both stress the need to align patching and remediation with business priorities and establish accountability.

---

9. Audience Speed Round: Career Hacks, Anonymity, AI, and Negotiation (36:41–42:16)

• Hacking the Hiring Process:
  • Find out what hiring managers want and demonstrate impact/conference presentations.
  • Outstanding example: an applicant presenting OSINT about the interviewer.
• Anonymous Forums:
  • CISOs would anonymously discuss “bad actors” in the industry and internal political roadblocks; but full anonymity can limit constructive dialogue.
• Making Space for AI:
  • Encourage teams to use AI to improve efficiency, but always set clear objectives to avoid “flailing with tech for tech’s sake.”
• AI Data Integrity:
  • Start by auditing where AI models and data are used, then apply strong governance and guardrails.
• Negotiating with Stakeholders:
  • Use analogies/metaphors to make risk real and relatable; tie impact to business outcomes, especially customer and sales impact.

---

Notable Quotes & Timestamps

• On open source “strikes”:
  “The solution should fix the ecosystem, not weaponize it.” – Jarek Beeson (05:16)
• On AI in threat landscape:
  “It's not should we include this information? It's how do we operationalize it as fast as we can?” – Jarek Beeson (09:29)
• On AI vendor hype:
  “If all they're doing is taking your data, throwing it in a giant LLM and spitting it back out, you can do that yourself, probably a lot cheaper and more effective.” – Jack Lydecker (27:44) “Vendors with the best failure analysis actually have the best products.” – Jarek Beeson (28:11)
• On accountability for security basics:
  “This is not a security problem. This is a business problem... But in security, you blame the CISO every single time.” – Jarek Beeson (34:54)
• On supporting burned-out SOCs:
  “If my SOC is unhappy, I guarantee you that's permeating to other parts of my team.” – Jarek Beeson (17:47)
• On stakeholder negotiation:
  “Usually when people understand a risk, they're willing to take an action. But no one's willing to open up their paybook for something that they don't truly understand.” – Jarek Beeson (41:44)

---

Timestamps for Critical Segments

• Changing Security Culture advice: 00:03–02:11
• Games and CISO team decision-making: 02:11–03:07
• Open Source: Reciprocity & Strikes: 03:32–07:17
• AI & Geopolitics: 07:23–12:39
• What’s Worse Game: 14:44–19:21
• Cybersecurity Family Feud/Mistakes: 19:29–26:13
• AI Hype vs. Reality: 26:19–30:26
• The Basics Problem: 30:32–36:35
• Audience Speed Round: 36:41–42:16

---

Overall Tone & Takeaways

The conversation is witty, candid, and practical, focusing on real-life dilemmas CISOs and their teams face daily. There is a strong emphasis on empathy (for both staff and vendors), healthy skepticism (especially regarding AI tools), and the need to un-silo security as merely a technical or CISO concern—making it clear that business alignment and cultural shifts are at the root of most persistent problems.

For practitioners and vendors alike: Listen more, question the hype, sweat the basics, and always bridge gaps between technical reality and business urgency.

Transcript

[Podcast Announcer]

Best advice I ever got in security.

[David Spark]

Go.

[Jack Lydecker]

The best advice to me initially started as the worst. So I had a previous boss who wanted us to fix our phishing issues, and we're gonna fire everyone after they get phished. Three times I had to say, actually, we would end up firing you. But more importantly, it helped us reevaluate what we wanted to do for phishing, which is changing behavior. And if someone thinks they're gonna get fired, they're not gonna tell you. They're not gonna report.

[Podcast Announcer]

You're listening to CISO Series Podcast, recorded in front of a live audience in Houston.

[David Spark]

Welcome to the CISO Series Podcast. My name is David Spark. I'm the producer of the CISO series. And to my immediate left is my guest who was here last year when we did the show. It's Jerik Beeson, the CISO for wm. Let's hear it for Jarek. Say hello to the audience, Jarek.

[Jarek Beeson]

Hello, audience.

[David Spark]

That's what my other co host, Steve Zaluski, says it just like that. In fact, he's been. People have come up to him and said, hello, Steve. Just like that. He introduced it. By the way, we are recording live at Whose Second, which, by the way, I pronounced it incorrectly. I did like howsetcon or something stupid like that. I got definitely admonished for that. But we're at Whose Econ. Let's hear it from the whose second audience. And we are [e-mail], where all our wonderful programming is our sponsor for today's episode. The reason we are here in Houston, Vorlan Security Enterprise, SaaS Security. That's light years beyond the legacy SSPM Tools. More about that later in the show. Let's hear it for Vorlan for making us come here to Houston. All right, before we begin, one of the staples of the CISO series is a game we play called what's Worse, which we will play today. And as I understand, you play a version of it at your office. Some of your staff members are here. Jarek, how do you play the game differently?

[Jarek Beeson]

All right, so we call it Pick youk Poison, because, you know, copyright. And what we do is we have a scenario. Different people on our team provide that scenario. And the team is divided on if they go with option A or option B, but their goal is to pick the option they think that I would want to take in front of the board. That is the mindset that they have, and it helps them learn who I am, how I am, And a lot of decisions are made now without me having to be in the room because we play that game.

[David Spark]

That is awesome. The fact that they can essentially think like you but now, but hold it. How do you then if they're all thinking like you, how do you get that very diversity of sort of understanding of cybersecurity incidents.

[Jarek Beeson]

So one usually the majority is wrong. For me that's actually best because it's a learning opportunity and at the end of the day incidents are a shared responsibility. So I'm a still finding about everything. But when we have to make risk based decisions, they can say, well, what would Jerrick do? It's another version of wwjd.

[David Spark]

I like it. All right, let's bring on our guest. He's to our left. You heard him at the very beginning of the show. He's a CISO for gong. Let's hear it for Jack Lydecker. Say hello to the audience, Jack.

[Jack Lydecker]

Hello.

[David Spark]

That works for me.

[Podcast Announcer]

What's the motivation to do this?

[David Spark]

Open Source Devs of the world unite. Quote it is long past time that maintainers stop letting them being the organizations take advantage of their the maintainer's good nature. Now that's Justin Warren of Pivot9 arguing that organizations have developed a parasocial relationship with open source software. It's seen as naturally recurring resource rather than human labor requiring support. His solution. Listen to this. Maintainers should go on strike, stop rushing to fix things for those freeloaders and patch security flaws at a leisurely pace unless someone's paying. All right, Jarek, do you agree organizations are taking advantage of open source's goodwill and if so, can this continue because it has been or should something be done? I mean Warren's suggestion is pretty dramatic.

[Jarek Beeson]

Yeah, I would agree that most IT shops with some type of development function are definitely heavily dependent upon open source and it is a disproportionate level of consumption versus contribution. That being said, I would look at that parasocial concept a little bit differently. It's not completely one sided. Large organizations are contributing, code testing, in some cases even funding. But by and large the majority of organizations are more consumption than contribution. This whole concept of striking though, I think that could be catastrophic.

[David Spark]

It sounds pretty rough.

[Jarek Beeson]

Log4jexe YouTube maybe a year ago showed us that a small set of maintainers can have impact over millions of systems. And what we learned when we thought Mitre was going to get defunded is that companies are willing to pay if they're going to lose their CVE database. I think that's the blueprint that we follow. I would agree ultimately that the status quo is not working, but the solution should fix the ecosystem, not weaponize it.

[David Spark]

That is a good closing line there. All right, Jack, I throw it to you. What's your feeling on this?

[Jack Lydecker]

Yeah, no, I mean, I think it's a little messy, right? Because on one hand, people want open source so they can have transparency, see what they can do. It is hard when you're not being paid for. But I also think a lot of the volunteers want to be able to be part of a project. But I think there does need to be a bit more of a balance, Right. How do people contribute to it? Are companies being able to sponsor it? I think if you move straight into commercial, it kind of defeats the whole core of what Open Source was.

[David Spark]

Right?

[Jack Lydecker]

Which is we want to share and.

[David Spark]

We want to commercial software. It's something different.

[Jack Lydecker]

Exactly. And even a lot of the people that are part of the project really don't want it to go commercial. Right. Because they got frustrated with them vendors not being able to actually update it correctly, being able to change it. So it's being able to find what that balance is. But I agree, like, striking isn't the right answer. But I do think whether it's fundraising or pushing a bit more, or saying, hey, if we can't get this, like you said with mitre, you can get something without having to go full commercial, which to me, kind of defeats the purpose of open source too.

[David Spark]

All right, so what is the happy balance here? And my feeling is, a lot of times people do this for the recognition and the love of it. So is there a better or a different recognition we can offer? What do you think? Jerk.

[Jarek Beeson]

I think the Wikipedia model might work, right? Like, we consume Wikipedia and we know that when we go there, they say, hey, if you want to help us stay on, you can pay. I think people even having an opportunity and a path to contributing would be helpful. But if you create a more commercial model, SLAs come along, structure comes along and expectations come along, and now we've lost the spirit of what we're trying to accomplish.

[David Spark]

Have either of you hired someone who supported open source software?

[Jarek Beeson]

If you have a developer, you have someone that supports Open Source software.

[David Spark]

Okay, that answered that question. All right.

[Jack Lydecker]

And I would say I've actually had some that were very involved in some projects, and it was actually interesting because they supported some security onion work and other things in the past. So there's a benefit from that perspective. It helped them get experience, but it is a bit of A different model. Right. As Jared mentioned, I think if you go commercial, SLAs is going to kill it.

[Podcast Announcer]

Is AI going to help us or hurt us?

[David Spark]

Quote, Almost 2/3 of IT leaders agree that generative AI challenges the geopolitical status quo, allowing smaller nations and non state actors to emerge as near peer cyber threats. I'm quoting you Jarek, as you were referencing the 2025 Armist cyber warfare report. With AI in the mix, threat actors no longer need deep expertise or vast resources, just intent and access. Now something we saw initially with ransomware as a service, some regional conflicts that wouldn't normally be on many companies radars now have global implications. I'm going to start with you, Jarek, on this. Since you brought it to our attention. Does geopolitical intelligence need to be part of a more of a company's risk assessments? Because some small companies, it doesn't even hit their radar at all. And if so, how do you decide what geopolitical events fall into your risk profile? And how are you even tracking this once you define it?

[Jarek Beeson]

Man, I wrote that.

[David Spark]

You didn't write all of that. You wrote the beginning quote that I quoted you on.

[Jarek Beeson]

Yeah, so historically we would look at a small set of nation states that were well funded and that was the adversary we had to focus on. But now it's not about capability, it's about desire. Because with $100, a couple thousand dollars, we've now lowered the bar for how you can actually enter into some type of cyber attack. And so now we don't just have a few countries, we have the entire world that could potentially be targeting us. And AI is making that so much easier as a service, as you pointed out, is making this so much easier. So I look at it a few different ways. There are about three questions you need to ask in your risk assessment process. One, what are the services and critical operations we have and what are the supply chain dependencies? The second one is, are there any specific nation states that are targeting our industry in particular? And then lastly, are there any business endeavors, M and A activity, sponsorships, you name it, that could be viewed as opposing to one side of a conflict? And those questions will help you determine likelihood and impact. And we can no longer look at cyber risk in a silo. It has to be viewed collectively and holistically. And at the end of the day it's not should we include this information? It's how do we operationalize it as fast as we we can? Because an attack that's happening on one side of the world can end up at your doorstep if you're considered to be on the opposite side of a conflict.

[David Spark]

All right, I throw this to you, Jack.

[Jack Lydecker]

So, I mean, I think we've been seeing this change for a while anyhow, right? I mean, as cybersecurity and especially on the other side, it's been commercialized, right? I can go by passwords, I can move things quickly. AI accelerated some of this. I think it is important to understand what's my risk, what's going on. I think the aspect of, hey, we may not be a target because of something is something you really want to reevaluate. Because a lot of times we've seen some of the smaller groups just try and make a name for themselves, right? They may not even be going after what you expect. They want to create an impact. If you're vulnerable, they're able to attack you a lot faster. So I think here it's just more you need to be aware of what your risk is, how can you respond to it? And really, regardless of what industry you're in, someone's going to be interested in being able to take you down. You need to be able to understand what that is and how you might be able to mitigate it.

[David Spark]

So the thing is just going to the last things you were saying there is that I'm a small company with a small security team. I can't be dealing with everything. Is it enough to just build up my security program and not be paying attention to the threats? Because I don't know if I have even the bandwidth to even think that.

[Jack Lydecker]

It'S hard when you're on some of the smaller companies, right. I think there, this is where you see lots of partners and other ones to try and help with it. But honestly, if you're not aware of what's going on, on, especially from a threat intelligence perspective and everything else, you're going to miss what some of your vulnerabilities are like.

[David Spark]

What would be a bare minimum. Like we talk about this quoting Wendy Nather. She talks about the security poverty line. What would be the equivalent of the security poverty line for threat intelligence?

[Jack Lydecker]

I mean, there's a lot of open source that's out there, right? And I would say most of your SIM vendors, even your edr, have things that you can add in. It's not super fancy. You may not be able to get, hey, I'm scanning dark web just for me. But at least being aware of what's going on out there, right? Because especially they see big campaigns, they usually start off smaller so if there's a big ransomware campaign that's targeting retail, I'm a small retail business. I need to understand if the attacks that they're using are something that I'm more vulnerable to or not.

[David Spark]

All right, same question for you. What do you think that sort of minimum line of security awareness, threat intelligence is?

[Jarek Beeson]

Well, you kind of alluded to the small medium business. Most of them don't have large security teams because they've outsourced it. So instead of you doing that work, you should make sure your outsourcer is doing that work. Threat intel is something that is not really understood, but at the end of the day, it's all about understanding your adversary and what they're after. You want to know if your data, your systems would be a target and you want to know what tactics that the bad actors are using so that you can protect yourself. If you see a geopolitical conflict in place and X, Y and Z group is involved, you know that they're after financial data. So let's look at our financial data, you know, that they use. Calling to your help desk. Let's make sure our help desk is ready, whatever it may be. Understanding your adversary is step one in understanding where your attack is going to come from.

[Podcast Announcer]

Who's our sponsor this week?

[David Spark]

Before we go on any further, let me tell you a little bit about Vorlan, our phenomenal sponsor. They sponsored us here last year at WhoWhatCon. All right, big update for security teams. Thinking beyond basic SaaS security posture management. Vorlan just redefined SaaS security with a platform that brings context and control to your entire SaaS ecosystem. Here's what's new. Vorlan's patent pending data matrix technology gives you near real time visibility into every connection. Secret and sensitive data flow across your SaaS and connected apps and services. Now you can actually see what's talking to what. Spot risky data sharing and catch abnormal activity as it happens. Now, worried about dormant oauth tokens over permissioned service accounts or shadow integrations sneaking in, Vorlan alerts you and helps you respond in minutes. Revoke risky secrets, investigate suspicious events and even automate remediation with your SIEM SOAR or ITSM. And let's talk compliance. Whether you're wrangling PCI, HIPAA, SOC2 or just prepping for the next audit, Vorilon gives you audit ready reports and evidence at your fingertips. No more screenshot marathons. Now the best part. Vorlan's agentless proxy free setup means you're up and running in under an hour without disruption for instant value. And of course you get world class support, trusted by Fortune 500 companies and high growth teams alike. You can see detect and secure your SaaS and AI ecosystem. You can get started. You got to go to the website though. Voron Vorlon I.O. go to Voronl I.O. and when you go there, let them know you heard about them from the CISO series.

[Podcast Announcer]

It's time to play what's worse.

[David Spark]

All right, we are going to play again. You've played this game many times, Jarek, and you play with your team many times?

[Jarek Beeson]

I have.

[David Spark]

All right, so you are familiar with this game, Jack?

[Jack Lydecker]

Yes, a little bit.

[David Spark]

All right, you get two crappy scenarios. One of them you have to deem is worse, so you don't choose the one that's better, the one that's worse and explain why. And you can disagree or agree on this, but here we go. Comes from Eric Block of Allumio. And here are the two scenarios. Scenario number one, you spend a million dollars a year on an LLM powered detection and response tool that reduces your analyst workload and their burnout. But there is no measurable reduction in risk or incident volume. So the value you get out of it is your team is not getting burnt out or overloaded. Okay, that's the value, but that's what you're spending a million bucks on. Or you spend no money on. LLM powered tools rely on human analysts who are constantly overwhelmed and they turn over every nine to 12 months. Which one is worse?

[Jarek Beeson]

Man, that sucks. Because LLM powered is a problem in itself. It's not agent based. Well, I'm always going to err on the side of caring for the human.

[David Spark]

And so you'll drop the million dollars a year to keep them alive and healthy.

[Jarek Beeson]

There might be some AI SOC vendors in here. None of them are a million dollars. So the fact that an LLM based one is a million dollars is a problem. So I'm gonna eat the fact that I'm not making a good financial decision for my company, but the humans that I care for are cared for and so I'm gonna go with that option.

[David Spark]

All right, so the business is gonna be furious with you.

[Jarek Beeson]

They're not gonna know because I'm the security guy and they're secure.

[David Spark]

Well, you're spending well. Supposedly. You got a million dollars to spend on this. We'll see.

[Jack Lydecker]

Yeah, it depends on what your budget is, right? If I can hide that, then that makes it easier. But turning over people every nine months that sucks. Like, your SOC is ineffective at that juncture.

[David Spark]

If that's happening, you're probably not. Well, I don't know. Will that equal a million dollars? Nine to 12 months?

[Jack Lydecker]

Oh, easily, yeah. You're turning over that many analysts.

[David Spark]

Yeah. You'll be running through that pretty fast.

[Jarek Beeson]

That's seven analysts fully loaded. Maybe.

[David Spark]

No, but you're still spending the same amount of money. But I'm talking about the cost of the turnover each time. Are you spending a million dollars on the cost of the.

[Jack Lydecker]

Yeah, but it takes a while for someone to become effective. Right. So if I'm turning those people over every nine months, you're also not going to get people that are actually evolving and like my.

[David Spark]

All right, Are you. Are you agreeing with Jarek on this?

[Jack Lydecker]

I would be, sadly, yeah.

[David Spark]

All right.

[Jack Lydecker]

I really don't want to, but I.

[David Spark]

Think I'll think you'd be burning a million dollars in effectiveness and turnover if you chose the other option.

[Jack Lydecker]

Yeah, well, not even just effectiveness and turnover risk.

[Jarek Beeson]

Right.

[Jack Lydecker]

Because you're not going to be able to detect what's going on if your people are turning over that quickly. They're not.

[David Spark]

But the thing is that in both cases, your security is, like, the same. Your effectiveness is the same. One case, you're spending a million dollars to keep people not burnt out. The other case is you're not spending the money and they're burning out every 90 days.

[Jarek Beeson]

So they say one bad apple spoils a bunch. If my sock is unhappy, I guarantee you that's permeating to other parts of my team. So this is a bigger impact than just the stock.

[David Spark]

Okay, so you see this trickling over. All right, let's go to the audience now. All right, so the two scenarios are you spend a million bucks and you keep your audience, your stock from burning out, or you don't spend the money and they burn out nine to 12 months. In both cases, your. Your quality of security is an incident response is the same. So again, you want to applaud for the worst scenario, the one that you think is worth. So the first one is the one that they both went with by applause. How many people think that's the worst scenario? People raising hands. They can't hear you. Hold it. Wait a second. I think the whole audience is going to go against you on this one.

[Jarek Beeson]

I think they agree with this. They misunderstood.

[David Spark]

Yeah. Okay. Well, no, you think they agree. Again, the worst scenario. Let me see. You think the worst scenario. Oh, no, I'm sorry. I take that.

[Jack Lydecker]

Yeah, you had it backwards. That's why no one screwed it up.

[David Spark]

That's why no one replied. Nobody agrees with you. One person raised their hand. Okay, you guys are following. I'm not saying the right thing. The second one they said is the worst scenario. Turnover, 9 to 12 months by applause. How many thinks that's the worst? All right, and let me go to the first scenario. How many people by applause. Not raising your hand because I can't record raising your hand on this microphone by applause. Did anyone think spending the million dollars to get essentially no results is worth.

[Jarek Beeson]

Way to be brave.

[David Spark]

Way to go, Rich. Way to go.

[Jack Lydecker]

Got one.

[David Spark]

We got one person with a sarcastic clap. I appreciate it.

[Podcast Announcer]

It's time to play a brand new game.

[David Spark]

All right, you guys are all familiar with the Family Feud, right? You know how that game goes.

[Jarek Beeson]

I am. I yell at my TV all the time.

[David Spark]

All right. It's a fun game. So we kind of did this with our audience as well. We. We put up a survey on our site. We have a participate page. We're going to put up another one. Hopefully soon we'll more questions up and we ask five questions and we'll see how many of these we can get through here. We ask five questions of our audience. So I'm going to ask you these questions and I want to see if you can get the most popular answer. Shout it out if you know it. And if you can't get it, we'll go to the audience and then we'll also see if the audience can get some of the other answers as well. All right, the first question is, remember we. And we got 70 responses here. Name something you should never share online.

[Jack Lydecker]

Your password.

[David Spark]

Password. That is number one response. 27 responses on that. All right, very good for you, Jack. Jack gets a point for that. All right, let's see. Can you get any of the other responses, Passwords being. Number one.

[Jarek Beeson]

Passport.

[David Spark]

Passport. I will put that under personal identifiable information. That's number three. With 11 responses. We have a number two more.

[Jack Lydecker]

So it's social. Probably would fall under that too, right?

[David Spark]

What'd you say?

[Jack Lydecker]

Social Security.

[David Spark]

Social Security. Number two. All right, you got first. Second. All right. All right, we have three more popular responses. What you think you can get the last three? Either one of you.

[Jarek Beeson]

Your vulnerabilities.

[David Spark]

Vulnerability. No individual personal things. No user ID that would fall under pii. Your address, contact or location information. That was number five. We have four and six. What do you think?

[Jarek Beeson]

Kids names?

[David Spark]

Kids names? No. Well, that would fall under contact and location information. If you get this one wrong I'm going to the audience, see if they can get the last two.

[Jack Lydecker]

Your date of birth.

[David Spark]

No, no, no. All right, two more. By the way, 3 responses for the content. We have 2 more. Number 4 and 6. And we just shouted out. What do you think? Credit card. I will say financial information. Yes. And then the very last one. Health. That would be under pii. No. Anything else? Oh, this one's a good one. No one's getting it. Company information. No. Nudes don't share. Nudes. All right.

[Jack Lydecker]

I don't know that. I would have guessed that.

[David Spark]

All right, good job. Good job. Good job, everybody. Nobody got the. All right, here we go. Question number two. Name something in cybersecurity that gets harder the longer you wait. Jump in when you know it.

[Jarek Beeson]

Asset management.

[David Spark]

Asset management. Oh, number six. Asset inventory. Number six.

[Jack Lydecker]

Patch management.

[David Spark]

Patching updates. Number one with 11. All right, two for two here. Excellent. Good job. All right, keep going. I got a total of eight here. What do you think?

[Jarek Beeson]

Identity and access management.

[David Spark]

Identity response or. Well, hold it. Identity. No, by the way. By the way, patching and updates and vulnerability management is the same thing. I have that as one. Yeah.

[Jack Lydecker]

Patching and vulnerability, that's the same thing.

[David Spark]

That's the same thing. All right.

[Jack Lydecker]

Secret management. Although that could be password.

[David Spark]

Responding to incidents into incident response and breach management. Yes, that's number two.

[Jack Lydecker]

Phishing.

[David Spark]

Phishing. No, that's not on our list at all. I'm going to give you one. But he's striked out.

[Jarek Beeson]

Regulatory compliance.

[David Spark]

Compliance and regulations. Yes, it's number three. I have. All right, again, name something in cyber security that gets harder the longer you wait. And I have three more. I have number four, five, and seven here. Hiring a ciso, that's career and professional development. I'll give that to you. That's number five. Number four is a little Vegas. Well, as fixing the organization. All right, you know what the Latin, Number seven. I'm going to the audience on this one. This is a tough one, but it is something that definitely I'm going to stress. The longer you wait, the more painful it becomes. What is it? No, no, no. Think I'm gonna. I'm gonna give you a big hint. You work for a very big company. Like maybe your company or a health care company or an energy company. What did you say? No, not vendor. What'd you say over here? No, not Shadow it. No. Legacy tools. Legacy tools. All right, let's keep going on this.

[Jarek Beeson]

The audience agrees.

[David Spark]

All right, here we go. Question three. We'll just do it here. We go. We're going to finish on number three here. This is a good one. Name a cybersecurity mistake everyone makes but will not admit to. This one's easy. You should be able to get this one.

[Jack Lydecker]

Reusing a password.

[David Spark]

Yes. Number one. Good job. Good job. All right, can you get any of the other? We've got a total of seven here. 33 Responses, by the way, on that one.

[Jarek Beeson]

Using unapproved software.

[David Spark]

No, that is actually not on our list.

[Jarek Beeson]

Oh, now we bring those out.

[David Spark]

All right.

[Jack Lydecker]

Bypassing your security controls.

[David Spark]

No, not really. It's not on our list here. I was trying to see if we could fall into for that.

[Jack Lydecker]

Clicking on a phishing email.

[David Spark]

Clicking phishing email. Yes, sir. Good job. That's the second one. 7 Responses to that. Hold on. You know what? Risky behaviors was number three. And that's using public wi fi, accepting cookies on websites, spam calls. It could be a lot of different things. All right, so we get another one with that. Okay, so I have number 437. What do you think?

[Jarek Beeson]

I gave you my good stuff, man.

[David Spark]

Give you good stuff. Let's go to the ice.

[Jack Lydecker]

Unloading. Oh, okay.

[David Spark]

What do you got?

[Jack Lydecker]

I was going to put downloading pirated software using Tor.

[David Spark]

No. Okay, we got four more here. Let's see if the audience can get them. Shout them out if you know them. Sharing. Sharing credentials. Sharing access issues. Yes. Number six. Four responses for that. Using their computer. What? Leaving their computer open. That would be risky behavior. Someone we. That would fall under that category. Would someone yell out here? Not following change. Well, neglecting updates and patching. Yeah, that would fall under that. And then there's one good one, actually. Only came in seventh that I thought. But also that would also fall under security assumptions and overlooking basis. There's one more. That's a good one. Anyone want to guess it? Default settings. You all do it. You know it.

[Podcast Announcer]

How is AI going to solve this problem?

[David Spark]

All right. What specific pain does your AI solve that simpler methods cannot? That's number one question Caleb Sima of White Rabbit says you should ask any vendor patching AI security solutions. If you ask, be ready for a dearth of answers. Other AI hype red flags include vendors claiming 98 to 100% detection rates, promising to solve most security problems with a single platform, or being evasive about implementing timelines and concrete metrics. It's good to have healthy skepticism of a new buzzy AI tool. But the reality is we're all under pressure to embrace AI solutions At the same time, we're securing Them. So I'm going to start with you, Jack. How do you separate a legitimate AI innovation? And by the way, this is a good tip for as we look at the vendor hall here, legitimate AI innovation for marketing hype when you're evaluating vendors.

[Jack Lydecker]

Yeah. So this one's kind of near and dear to me, especially since I'm also at an AI company in this perspective. But at the same time, I think because there's so much hype, there's two things that I usually focus on. One is can you quantify impact? Because a lot of times we'll throw fancy things, but if I can't quantify it, it's meaningless. And then also, what do they actually even mean by AI? Is it just a wrapper? Are they actually using their own models? Do they fine tune it, really understand what are they doing that's different? Because if all they're doing is taking your data, throwing it in a giant LLM and spitting it back out, you can do that yourself, probably a lot cheaper and more effective. And there's just so much stuff that you can't quantify. And there's too much hype right now with this.

[David Spark]

All right, how do you verify. And by the way, you must have deal with this all the time because everyone's got AI in and everyone's making claims.

[Jarek Beeson]

Yeah. And I'm a consumer of AI as well. My team is in the room. They know I've kind of set out a path with AI heavily involved. When I ask vendors, I ask a few different questions. First I ask, what is it that your tool is going to bring me that hiring a few people won't? Number one. The second one is I ask them around what makes their models fail. I find that vendors with the best failure analysis actually have the best products. And then I also look to see if they ask me a question. If they don't ask me about data governance or data quality and the AI tool is based off of my data, then I realize they're just giving me a hyper powered automation tool. And not true AI, because AI is based off of clean data.

[David Spark]

I love that. I love that they should ask you a question. I love the fail one because by the way, this is something that we ask on our show. Security, you should know. Tell me what your product does and what it does not do. But the fail question is really a good one. Specifically around AI, what are some of the answers you've heard on that?

[Jarek Beeson]

Most vendors don't have that answer because the salespeople haven't been given that level of understanding. Most of them understand the overall objective, but behind the scenes they usually say, well, I have to get my CTO on the call. Or something along those lines.

[David Spark]

Have you heard any answers to that question?

[Jack Lydecker]

Yeah, no. I mean, I think there's some good answers on it. Like, this is what we do. We have an AI governance team. We've evaluated. Are they 42001? But it did bring up kind of a different thought of another thing to ask too. That kind of separates what is their data team inside? Do they have data scientists? Are they building their own stuff? Because I've had some vendors where they actually didn't have any data scientists or anyone really that understood AI on their technical team. Which I'm like, how are you selling me a solution you don't even understand?

[David Spark]

Chris Hoff had a post about this on LinkedIn and he was arguing you should ask them some really basic questions about understanding AI. They don't have basic understandings of AI. It's like, you know, run away. Like, you don't want to be dealing with something like that.

[Jarek Beeson]

That's a great point. Most of the vendors are not building their own models. They're heavily dependent upon one of the three or four population. So I also like to ask them, how has your product evolved as this model evolved? Because you're basically at, you're depending on those models to improve, for your product to improve. So if you can't show me the difference between those two, then you're probably not doing an AI as well.

[Jack Lydecker]

And I think even with that, you can push like, is it fine tuned or not? How is my data being utilized? But like, to that point, you don't even need to go down that aspect because if you're going like, hey, what's your governance? Do you even have data scientists? If the answer is no, you're kind of already done.

[Podcast Announcer]

It comes down to the basics.

[David Spark]

All right? The hardest part of vulnerability management isn't discovery, it's everything after the scan. And that's Rinky Sethi, who's the CISO at Upwind Security, cutting right to the heart of a problem that's been plaguing security teams for decades. We still struggle with the basics. Jerome Levy, CISO over Adobe. He would say the fundamentals because if they were basic, we would have figured them out. Security teams are held back not by a lack of intent or tooling, but by fragmented asset inventories, lack of business context, and mountains of unprioritized CVEs. For Sethi, security teams need real time asset intelligence, AI driven contextualization and empowerment to remediate continuously without slowing down the business. AI tools are promising real time asset intelligence and contextualization. Isn't that what we all want? I'm going to ask you, Jack, shouldn't we be solving this problem already? I mean, I hear this again and again. What is stopping us from figuring out the basics, the fundamentals, whatever the heck you want to call it.

[Jack Lydecker]

I think sometimes we're looking at the problem wrong, quite frankly, because security isn't the one that typically owns it unless you're in an org where maybe you're blending your IT and development with your security team. The problem is giving us better data around CVEs and other things I don't think is a problem. It's if we're going to deploy a package, we're deploying software, do we have a way of how that's going to be maintained? If you don't have that in place, you're going to be chasing contextualization and everything all day long. But the biggest delay isn't the fact of updating the package. It's really easy to update a package. It's the testing and the business that needs to maintain that and make sure it still operates. Because I've had some where it's like, hey, we just updated all our containers and then it doesn't work, so it doesn't help anyone, right? So I think the bigger issue is fundamentally we want to skip the hard basic things, which is what is our asset management lifecycle, what's our patch management? And if we want to bring in a container, a package, et cetera, who is going to be responsible for owning and maintaining that? Because it's real hard to do that after a couple years. And if you add tech debt on top of it, that's where everyone's having the pain. It's not validating that it's an issue. It's who's going to be responsible for testing and actually getting that functional.

[David Spark]

All right, so the problem is it's just making sure the thing you bought actually works and doing the thing it's supposed to.

[Jack Lydecker]

No, no, I would actually, it's who is going to maintain it.

[David Spark]

Well, but that's part of the job of doing that.

[Jack Lydecker]

But like people skip that, right? Like, hey, finance wants this new tool, who's going to be maintaining it, right? Who's updating it when it needs to get updated? Do they know how to test it? Do we need to do things? Who's managing the vulnerabilities from Those vendors to make sure, like, hey, I need to update my packages.

[David Spark]

I will. By the way, I'll tell you something I've heard from CISOs before, is they only now look at platform plays because of this fear of if I bring on another tool, I need to train somebody on this other tool. But if I get on a platform, well, hopefully it's going to work alongside everything else I have. Hopefully. All right, I throw this to you, Jarek. Why is the basics of fundamentals still a giant pain in the butt? Like, we've been doing this for years. This comes up all the time.

[Jarek Beeson]

Yeah, I agree with Jack. I have a little bit of a soapbox, so apologize if I take a little longer here. Security doesn't patch. Like, most of the time, security doesn't patch DevSecOps, something like that. But I've consulted for 50 companies and in the Fortune 500 and multiple letter agents, it does the patching and they're at the behest of the business that says, yes, the system can go down. Yes, it's okay if the system goes slower for a little while because you didn't test the patch, or yes, it's okay that we're going to have three weeks of exposure while we test this patch before we move it to production. All security does is provide context. Security provides asset intelligence. Security provides threat intelligence. And so this is not a security problem. This is a business problem. And so when you ask me what's stopping us, what's stopping us is that we still look at security. CISOs are always raked over the coals when there is a breach and they say they had X, Y and Z vulnerability, the CISO never can patch that vulnerability. And so I look at it like a doctor. If a doctor diagnoses you and says you have this issue, here's this medication, work out, eat healthy, do all these things. If the person decides not to do those things and they get sick or God forbid, they die, do you blame the doctor? No. But in security, you blame the CISO every single time. When people don't follow the prescription that was handed to them, it's a business problem.

[David Spark]

As a ciso, do you feel this pressure?

[Jarek Beeson]

Could you hear the passion in my voice?

[David Spark]

Yes, I did.

[Jarek Beeson]

Absolutely. I feel this pressure.

[David Spark]

What's your advice to other CISOs to alleviate this then?

[Jarek Beeson]

Well, number one, communicate to the leaders and the SLT and the board and so forth. Hey, we're going to have a bad day one day, no matter what, just expect it. My goal is to help you understand where that Bad day can occur. And, and my vulnerability reporting, my timelines for how quickly we report, we address those vulnerabilities. Those are all indications as to how well we're actually performing. And I always say we because it is not me. It's a. We always say security is a team sport. This is one of the areas where they say, no, it's a, it's a, it's a sport of one when it comes to vulnerability management.

[David Spark]

But that's a lot of weight on your shoulders. 100%. Were you cringing or did you feel some of Jarek's pain as he was on his soapbox there, Jack?

[Jack Lydecker]

Yeah, no, I mean, that's even why I mentioned before. I don't think think contextualization is our issue. Right. It's an alignment issue. It's a business priority issue. Do we agree that we're going to be maintaining stuff and what's going to be our schedule? Because otherwise, to Jared's point, we're the doctors that someone's still smoking and eating junk food every day and okay, they gained weight, surprise, surprise, or worse, you.

[David Spark]

Have a heart attack.

[Jarek Beeson]

Right?

[Jack Lydecker]

We need to be able to get into the point of how we're aligning with the business and it needs to be from the top executive down. We agree this is a priority because it's really easy. Because it's like, oh, that's your metric, not mine in some cases, which is part of the issue. Right. Do you hold the business accountable if that doesn't happen? Or are you being held accountable for something you don't control? Because that dynamic is where you can tell that you have a big problem. You need to make sure that they're being held accountable for the right metrics, otherwise you're going to fail.

[Podcast Announcer]

It's time for the audience question. Speed round.

[David Spark]

All right. We have a good amount of time to get through a bunch of these questions. These are questions I got from last night. We had a fun meetup of fans of the CISO series. Now, this comes from Aaron Hipley and I just want to just answer these as quick as you can. And I've run into people who are looking for jobs here in cybersecurity and Aaron is looking self so positive. And when I say hacking, I'm talking about like sort of working around the system, not literally breaking in.

[Jack Lydecker]

Hacking should not always be a negative term.

[David Spark]

No, it's a positive. We're using it in a very positive way. So either one of you jump in first on this. What are the best examples of hacking? The hiring process that you have seen that you were impressed by.

[Jack Lydecker]

I think if you understand what someone's looking for and you can show quantifiably how you can do that, or even presenting at a conference, I've seen people actually get things from that. Contributing to open source we talked about. I think there's lots of different ways to be able to do that. And then also a lot of it's just networking, getting to know people. Right. Like that's going to help you, at least. May not get you the job, but at least it'll get you through the front door in a lot of cases.

[David Spark]

What about you? What's the best example you've seen?

[Jarek Beeson]

This hasn't happened to me, but Adesiso tell me a story where he was hiring a threat intel analyst, and the analyst showed up with all this OSINT on. The person interviewing him said, here are all the things that are out there in the wild. Here are all the things that we could have done that I have seen. And by the way, this used to be your password. I can do that for the company. Hire me.

[David Spark]

Yes, that's good. And the OSINT is out there. It's pretty, pretty visible. All right, that's a good one. All right. This comes from Bill Brenner of Cyber Risk Alliance. By the way, one of the things we were talking about is like, that CISOs and security professionals need a space where they can just complain about issues. Maybe the pressure the business is giving them, like you've done right here on this show. But the problem is there's some issues that you would like to speak about anonymously, like you don't want your name attached to. So what do you think people would talk about if there was an anonymous space? Reddit can sometimes be that to talk about cyber and cyber stress. What do you think that we're not talking about that people would talk about more if they spoke about it anonymously? What do you think?

[Jarek Beeson]

Excuse my French, but that in the industry, we can't always talk about them. But if you knew that you could share who was and who is, I think that that would be something that we share a little bit more frequently.

[David Spark]

Okay.

[Jack Lydecker]

I think also some of the internal roadblocks you get sometimes, right. On a closed form, maybe you feel open on it. But honestly, I think if it's more anonymous, people might even talk more about, hey, I tried to do this and it got shut down. Or we wanted a patch and we're just told no. And then something bad goes up and it's like, oh, not Necessarily surprised, sadly.

[David Spark]

Would you see benefit from either reading or participating in an anonymous cybersecurity forum? And again, Reddit could be that. What do you think?

[Jack Lydecker]

I think the problem you have sometimes is when it gets anonymous, it's hard to make sure that they're adding value and it doesn't. Just trolling.

[David Spark]

Good point. What do you think, Jarek?

[Jarek Beeson]

I mean, if you just want to vent, sure. But if you want to have a dialogue, then the anonymous aspect of it provides a disservice at that point.

[David Spark]

Okay, I like this one from Josh Dray over at San Jacinto College says, quote, for your own security environment, how are you making space for AI innovation? And can you make space? Do you do that?

[Jarek Beeson]

Yeah, I mean I was just, I challenged my team. I mean they're in the room, right. To find ways to do what you're doing more efficiently, harnessing and leveraging AI. If you succeed, great. And if you fail, nothing has changed.

[David Spark]

Nothing's changed.

[Jack Lydecker]

I think you just need to be conscious about it. Right. What are you hoping to accomplish? What are you doing? Do you have a way to kind of measure yourselves with it? And also automation is key here. Right? Some of the AI really helps extend some of the automation. Even. So back in the day that you had. So being able to actually see what you're doing and replacing manual work.

[Jarek Beeson]

Yeah. Set the goal and objective. Don't say go use AI. You got to set an objective, whatever that objective may be. If it's be more efficient, if it's move at scale, if it's move faster, whatever it is, set the objective and then see what happens.

[David Spark]

All right, Try not to take this as too global a question. We're going to get through two more really quickly. What are ways you're managing the integrity of AI data? Because this is a big issue. What do you look for? Like, what's the first step when you're dealing with that?

[Jack Lydecker]

So I think for that, do you even understand where your data is and do you have a governance model? I would say for us when we went through that, being able to understand how many models we have in house, what's fine tuned, what do we use that's outside? What's open source? Where do we have AI models inside of our vendors? Just getting that inventory to me is step one and then two, defining what controls you want on it. Hey, if this is something we're doing fine tuning ourselves, do we do bias testing? What guard wheels do we implement? Having a strategy on that to me is key.

[David Spark]

All right, last question. And comes from Brian Zabetti of Ply nc. What's your best negotiating technique with a stakeholder? You got to get them to do something. How do you get them? What's your best technique?

[Jarek Beeson]

My go to are always analogies and metaphors. Hopefully one that puts up a mirror and helps them understand the decision that they're making. Usually when people understand a risk, they're willing to take an action. But no one's willing to open up their paybook for something that they don't truly understand. So it's my job to help them understand it. And then from there the negotiation's a lot easier.

[Jack Lydecker]

Yeah, so mine is pretty simple with it. Especially since we're a B2B company. It's impact on customers with it. If we don't do this, this is the impact or we're not going to be able to get this business. Relating it back to sales to me has always been the easiest way to make it a bit easier to get them on board.

[David Spark]

Well, that brings us to the very end of our episode. Let's hear it for our guest today, Jack Lydecker, who's the ciso of gong. Also Jarek Beeson who's the ciso over at WM and our audience. And for whose.com, let's hear it for yourselves. And lastly, Vorilon Security Enterprise SaaS Security. That's light years beyond legacy SSPM tools. Remember, go to Vorlon IO that's Vorlon IO. Let them know you heard about them from the CISO series. My very last question for you gentlemen. Are you hiring?

[Jarek Beeson]

Yes.

[David Spark]

Are you hiring, Jack?

[Jack Lydecker]

Yes, I am. You can always look at the job board.

[David Spark]

They have job boards and I asked them earlier. You can contact them. We will have their LinkedIn profiles linked on the podcast episode for this very show. Thank you very much. Thank you to whose second for making this possible and and thank you everybody. We greatly appreciate it. Thank you for listening and contributing to the CISO series.

[Podcast Announcer]

That wraps up another episode. If you haven't subscribed to the podcast, please do. We have lots more shows on our website cisoseries.com please join us on Fridays for our live shows, Super Cyber Friday, our virtual meetup and cybersecurity headlines. Week in Review. This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you're interested in sponsoring the podcast, contact David Spark [e-mail] thank you for listening. To the CISO Series podcast.