← CISO Series Podcast
CISO Series Podcast
Managing Risk Has Been a Priority Ever Since You Asked About It (LIVE in NYC)
00:40:37
Loading summary
Transcript179 lines
- [00:00]
Announcer
Biggest mistake I ever made in security.
- [00:03]
Saket Modi
Go. Thinking security is a technical problem and not a business problem.
- [00:09]
Announcer
It's time to begin the CISO Series Podcast, recorded in front of a live audience in New York City.
- [00:26]
David Spark
Welcome to the CISO Series Podcast. My name is David Spark. I am the host and and producer of the CISO Series podcast. Sitting to my immediate left is the guest co host for today's episode. He's actually done this show as a guest co host before. Please warm round of applause for Matt Southworth, CISO Priceline. Thank you. All right, Matt, just quickly. We are at the Faircon 25 conference. You come to many of these conferences. Here's my quick question for you. When you come to an event like this, you talk to your colleagues. Everyone talks about like the conversations being the most important part of the show. My question to you is what are the questions you ask your colleagues when you come to an event like this?
- [01:10]
Matt Southworth
Few things. So everyone likes to talk about the new vendors, but I like to talk about what are you not doing, what vendor, what service, what process are you dropping? I think that's interesting. And sometimes people aren't even aware of of what they're dropping. It's always good to find out what everybody else's board is asking about. Somebody mentioned quantum crypto to us and that's a project for next year. And what are the open source tools your engineers love to play with?
- [01:36]
David Spark
That's very good. All right, let me also introduce our sponsor guest for today's episode. Very thrilled they're on board. Many of the reasons why many of you are here today. To my far left, it is the CEO of Safe Security, Saket Modi. Let's hear it for Saket.
- [01:52]
Saket Modi
Thank you, David.
- [01:58]
Announcer
It's time to measure the risk.
- [02:03]
David Spark
Quote, what's actually driving risk in your environment right now? This is Lisa Bagando of Health Catalyst. She argues that too many security teams never answer that very critical question. Instead, they're consumed with busy work, filling out third party questionnaires and chasing compliance scores. She proposes a popular trend, modernizing your GRC system to automatically pull real time control data from your environment and feed it into a risk engine that quantifies how control failures impact risk as they happen. Now, if you throw a rock, you'll hit any number of AI powered tools, promising continuous visibility into your control health. But if AI gives us a genuinely better understanding of our security posture, and how should that change your legacy GRC program? I'm throwing to you, Matt. First, what's becoming antiquated? This references what you just said at the beginning of the show, what becomes antiquated that we should simply stop doing of our traditional grc?
- [03:07]
Matt Southworth
I feel like this is somewhere that I've always struggled. Is what we do here measure risk and how do we talk about that? What I think we can stop doing is questionnaires, right? Everything that our vendors and that our partners are looking for from us. Because the truth is those are going to be filled out by an LLM whether we tell our vendors that or not. For that to work and for us to stop doing the busy work, we need to make sure that we are doing a better job documenting incidents, documenting controls, and just keeping a corpus. It doesn't have to be well organized of information about our program so that we can steal new answers and new information.
- [03:49]
David Spark
All right, shocked. Same question to you. I want to know, just high level, what should you stop doing with your GRC program?
- [03:57]
Saket Modi
I think the first piece is GRC has three letters. Governance, risk and compliance. I think let's start focusing in all three, if not equally, at least giving them that degree of priority to understand that there is a risk, not just compliance. So to what Matt was mentioning very rightly, that when you do questionnaire and when you're very, very compliance led to say, hey, is this something compliant? Is it? Check in the box. That is not really risk management. How do you actually get to the bottom of real data, real telemetry? And there's enough sources today where you can pull telemetry from and then compile that together. Not just to see how compliant you are, not just seeing what is the control maturity in your environment, but that compliance and controls can lead to what are your business risks and going ahead and tracking that. So as we say start with the why. The why has to be we have to do risk burn down, not just achieve compliance. So that is what we have to stop doing. Being more very compliance focused. We have to go and change the conversation to be very risk focused.
- [05:01]
David Spark
Have you had to do that shift? Because we've heard this line many times before. Nobody throws stones at me. Compliance does not equal security. But at the same time, most businesses know that if we don't do this, we're gonna be automatically fined. I mean, can you make that shift? Or is it just. I mean, it sounds like it's just like it's a cost of doing business.
- [05:23]
Matt Southworth
It is, but it's also a little bit of an opportunity sometimes, right? If you've got a compliance program and you've got strategic goals, you can connect them. However Loosely and drive where you actually want the program to go with a little veneer of compliance on top of it.
- [05:36]
Saket Modi
Yep. And I think compliance is a subset. Nobody's saying don't do compliance, but think of it like this. 99.9% of financial services companies which have been hacked in the last 10 years have been compliant to PCI, to the popular stuff. They still get hacked. What does that show? It means that you need something beyond compliance. So you're not saying don't do compliance, but that's the floor, not the ceiling, of how you manage risk.
- [06:04]
Announcer
Do you trust this LLM?
- [06:10]
David Spark
Who is responsible when an autonomous agent makes a mistake? I love this. Ritu Giot argued in a piece on CIO.com that because AI agents lack legal personhood, responsibility must fall on the humans who deploy them. So we need clear ownership, revocable credentials and audit trails for AI. Now, we don't always blame humans for making mistakes, we expect it. But agentic AI will make mistakes too, probably at scale. So should we treat AI errors the same way we treat human errors, or fundamentally different? What does accountability even look like when an autonomous agent screws up? So I'm going to start with you, Saket. The blame game could be endless here. We know this. And more practically, how are you building governance today that can actually trace responsibility when things inevitably go wrong?
- [07:07]
Saket Modi
I think the way you look at that is, firstly, David, this depends on how you define a mistake. Okay, there are different degrees of mistakes.
- [07:14]
David Spark
That you know, but the more the fear is, is the accountability that these mistakes could start happening.
- [07:21]
Saket Modi
That's right. And that can be a difference between somebody very smart in your team doing things very quickly versus somebody who takes two weeks to do the same thing. What I'm trying to get towards is if you start thinking of Genai agents as autonomous human beings and treat them like identities. I know this is like a Pandora's box, but the reality is that given the capabilities of these AI agents that we are talking about, they have to be treated in a similar way, if not exactly the same way of how we look at human identity. And the moment you look at that, you're talking about the guardrails, the identity and access controls, you're talking about the privileges that they have access. From a data access perspective, what can they see, what can they act upon? So again, the first principles remain the same. Because it's human intelligence or artificial intelligence. Intelligence is still intelligence, and that needs to be guarded with the same degree of governance. It'll be slightly different, but not entirely different in my view.
- [08:18]
David Spark
We got, we got applause on that. Someone really appreciates that.
- [08:21]
Saket Modi
I'm glad somebody liked my answer.
- [08:23]
David Spark
All right, hold it. Before I toss to you, Matt, one of the things that we have heard often is maybe treat these AI agents as like a brand new employee. You can't assume a new employee is going to know all your systems. And not only that, you gotta create guardrails around them. You can't have them running wild in your environment. AI has a little bit of a semblance of that of treat them like a new employee. But a new employee that could be crazy efficient or cause a lot of damage really quickly.
- [08:55]
Matt Southworth
Yeah, I think new employee and intern's a good perspective. Things we've learned in working with these is you can't use an agent where you can never accept a wrong result. They are going to make mistakes, or if that's assigning too much intelligence to them, they're going to give you unexpected outcome. The other thing we've learned the hard way is bugs don't stay fixed. We think we resolved something. We updated the prompt. It's repeating the same mistakes. So just like a new employee or a dumb intern, sometimes they get set in their ways and you need a human to review and take action upon what the results look to be. I love your point around the identity or the non human identity of these agents. It's a very important place for control. But that comes after the human thinks about the process and thinks about the business goal they're trying to solve.
- [09:47]
David Spark
Let me just skip to a big concern a lot of people have. What is your number one? Just please isolate it to one thing. What is your number one concern around AI?
- [09:57]
Saket Modi
The known unknown ways that AI can act. And just if I double just the.
- [10:04]
David Spark
Known unknown, I would think the unknown unknowns.
- [10:06]
Saket Modi
So the reason I say unknown unknowns is because I'm saying the known unknown is very large. That's a subjective debate. But the reason I say that, if you go back to Alphago, that's the movie very popularly made where for the first time an AI system defeated the world's greatest GO player. What was scary, and the most scary part of that was something called Move 37, which was the fourth game where AI makes a move. And that move not only by his opponent, which was a human being, the world champion in go, but pretty much everybody who was giving the commentary said this is the most stupid move somebody can ever make. Like, why would somebody do that? And everybody thought that AI has lost what ended up in that game? AI won. So what scares me is not that, oh, there'll be new ransomware attacks or there'll be new phishing attacks, or that's the known side of the problem. The unknown ways by which AI can reason and almost say this is the way to infiltrate systems organizations, governments, identities.
- [11:16]
David Spark
That, in my opinion, that's a really good point because one of the jokes that I used to always make about the RSA conference, I used to call it the Scare the Crap out of me festival. Because every year I'd start talking to people and I'd be one of those, like, what, they can do that now? No, really. And so what you just described is it's that at a pace we've never seen before.
- [11:40]
Saket Modi
Totally.
- [11:43]
Announcer
Who's our sponsor this week?
- [11:48]
David Spark
Our sponsor this week? It's Safe Safe Security Sockets Company. They are the autonomous cyber risk management company. Now, today's security leaders face more risk, more vendors and more expectations and have outpaced the tools we use to manage it. Security leaders are buried in dashboards, heat maps and disconnected data across business units and vendors. The board wants financial clarity. The business needs speed. Security teams need to focus on the risks that truly matter. That's where SAFE comes in. SAFE is the category leader in cyber risk quantification and the first platform to unify and automate cyber risk management across your enterprise, third parties and AI. Built on the fair framework, SAFE uses a fleet of specialized AI agents to continuously assess exposure, quantify risk in dollars, and automate everything from onboarding to board ready reporting. Now, if you're ready to scale with less manual effort, stronger board and business alignment, and real time visibility, you gotta go visit their site. It's SAFE security for the future of cyber risk.
- [13:09]
Announcer
It's time to play what's Worse.
- [13:14]
David Spark
All right, for those of you who are fans of the CISO series and have heard the show before, this is a game we've been playing since we first started the show, many, many years ago. In fact, well over seven years ago. All right, so the way it works is we have two scenarios. They're sent in by listeners. This comes from Eric Block of Illumio. And they're two crappy scenarios. You won't like either one of them, but it is a risk management exercise. You will not be able to pull out any quantification here. To do it, you're going to have to do it on the fly. All right, so let's figure this all out. You're going to answer first though, Matt, and you're going to agree or disagree. Socket here we go, what's worse? And by the way, audience, we're going to want your feedback too on this as well, so come up with your answer as well. I'll ask for applause on that. Onto what you think is what's worse. And remember, it's what's worse, not the one you prefer. Scenario number one, you pay a million dollars a year for a custom LLM copilot for your SOC analysts, but it is a black box and no one trusts it. So they still escalate 90% of the alerts. So it's kind of useless. You're just flushing a million bucks down the drain. Or you don't invest in AI at all. And onboarding new analysts takes 9 months and 100k in training and turnover costs. Which one is worse, Matt?
- [14:34]
Matt Southworth
Both terrible, right?
- [14:35]
David Spark
They're both awful. Yeah. You have to tell me which one's worse.
- [14:38]
Matt Southworth
Which one's worse is the second option, right? No AI, just onboarding humans. Because at least with a broken million dollar budget, I can do something with that. Right. I think there's an opportunity there. The driver, existing vendor, it's going to.
- [14:51]
David Spark
Probably burn out your staff because you're still escalating 90% of the alerts that's.
- [14:55]
Matt Southworth
Going to happen in either scenario. Right. So at least I've got a million dollars to play with here.
- [14:59]
David Spark
Well, you may not have that many alerts and in fact actually in the other scenario, because you don't have AI escalating anything.
- [15:05]
Matt Southworth
Sure, sure. So at least if I'm hiring analysts that are willingly walking to an environment with zero AI tooling, I'm probably hiring the wrong people.
- [15:16]
David Spark
All right, I'll throw this one to you. Saket. Which scenario is worse? Here you pay a million bucks a year for custom LLM block, boss. No one trusts it. Escalate 90% of the alerts or you don't invest in AI at all. And onboarding new analyst takes nine months at 100K a year. And that's by the way, the nine months of onboarding. That means they're essentially ineffectual for most of the year.
- [15:38]
Saket Modi
Yeah, well, the moment you said you don't invest in AI, my natural option was number what Matt selected. So I agree with what Matt said, but for a different reason.
- [15:49]
David Spark
Okay.
- [15:50]
Saket Modi
The reason is very simple, that it's like any product in the world. Do you know this? When iPhone was launched, it did not have cut copy paste.
- [15:59]
David Spark
Really?
- [16:00]
Saket Modi
Yeah, the first two years, iPhone did not. You could not cut copy paste on your iPhone. And look today, what is the iPhone.
- [16:07]
David Spark
Right.
- [16:07]
Saket Modi
The same way. Every single product, even if today it has 90% false positives, it is bound to be better. How much better depends on the execution. But I'm very confident if it's 90 today and if it's any company worth their Salt, it'll be 90 to 85 to 80, etc, etc. So the accuracy, even if it's a black box, will keep increasing. And that's a way better place to be than investing in humans, which again, can be turnover and they churn.
- [16:35]
David Spark
So that is a really good point. So even though the first scenario stinks, it's awful. At least you're starting in the right direction, where the second scenario you're not even beginning.
- [16:45]
Saket Modi
Totally.
- [16:46]
David Spark
What do you think of that, Matt?
- [16:48]
Matt Southworth
I think he's a very bright guy. I agree with him agreeing with me.
- [16:53]
David Spark
All right, by applause. How many people agree with them that the second scenario is far, far worse? To just invest in the people and not do AI at all? By applause. I mostly agree with you on that. Buy applause how many people think the first scenario where you're spending a million a year and the AI is just causing more havoc by giving you a ton of lurched for you to go chase by applause how many people think that is worse? No one does. So everyone agrees with. Oh, no, wait, one person. One person. The outlier. We have to have an outlier. Thank you so much. I appreciate it.
- [17:31]
Announcer
What are these security pros talking about?
- [17:37]
David Spark
All right. Okay, so we have this new game for you and you're going to like this. We have interviewed a lot of people recently at the Husetcon conference and the RSA conference, and we asked them a series of questions. And what we have here is three to four, maybe even five people answering the same exact question. I'm only playing the answers. You have to figure out what the question was. Okay. All right, here you go, audience. If they can't get in, I'm going to toss to you to see if you can figure it out.
- [18:06]
Saket Modi
Zero incidents, protecting your data everywhere it goes.
- [18:10]
Matt Southworth
Not having to deal with the product anymore.
- [18:12]
Saket Modi
Being able to help security teams show that what they're doing has impact.
- [18:19]
David Spark
All right, either you can guess what the question was. They were all being asked. Those are all the answers. Any idea?
- [18:25]
Saket Modi
Spoke about the ROI of security controls. I'm biased. I think it's crq. Cyber Risk quantification.
- [18:31]
Matt Southworth
I think you're asking people to come up with the slogan for their new company.
- [18:35]
David Spark
No, definitely not that. Let me play one more time.
- [18:38]
Saket Modi
Zero incidents, protecting your data everywhere.
- [18:41]
Matt Southworth
It goes not having to deal with the product anymore.
- [18:44]
Saket Modi
Being able to help security teams show that what they're doing has impact.
- [18:50]
David Spark
I will admit this is the toughest of the four questions. We're starting out hard. Any idea what do you think this is? Okay, I throw this to the audience. Just literally yell it out what you think the answer. What was the question? That these were the answers. Anybody? What keeps you up at night? No, no, not what keeps you up at night.
- [19:08]
Saket Modi
What do you wish you could have in cyber?
- [19:10]
David Spark
What do you wish you could have in cyber? That is close.
- [19:12]
Saket Modi
The promise of AI.
- [19:14]
David Spark
No, not the promise of AI. I'll give one more and then I'll tell you what it is. Board reporting. The question was what is the cybersecurity equivalent of a home run? All right, I'm going to give you another one. That was the toughest. I'll be honest. Those users that may fall into the.
- [19:34]
Matt Southworth
Same trap again and again. People asking for local admin access, pushing.
- [19:39]
Saket Modi
From the vendors to get deals done.
- [19:41]
Matt Southworth
When it's not important to the customer.
- [19:44]
David Spark
People skip over learning networking and go.
- [19:48]
Saket Modi
Straight to buying certs.
- [19:51]
David Spark
Any idea what that question was?
- [19:54]
Matt Southworth
Sounds like what are the biggest mistakes that your co workers make?
- [19:58]
David Spark
That is extremely close, but no. Saka you want to try on this one? Remember that he was close, but no.
- [20:04]
Saket Modi
No. I think. Yeah. Like what's been the biggest learnings in your career of cybersecurity professional?
- [20:11]
David Spark
Not exactly. Here, I'll play one more time for you. Those users that may fall into the.
- [20:16]
Matt Southworth
Same trap again and again. People asking for local admin access, pushing.
- [20:20]
Saket Modi
From the vendors to get deals done.
- [20:23]
Matt Southworth
When it's not important to the customer.
- [20:25]
David Spark
People skip over learning networking and go.
- [20:29]
Saket Modi
Straight to to buying certs.
- [20:32]
Matt Southworth
Biggest time wasters.
- [20:34]
David Spark
Want to try again?
- [20:35]
Saket Modi
No.
- [20:36]
David Spark
Anyone want to guess on this one? What was the question? Pet peeves. Yes, Correct. It is pet peeves. All right, we got two more. Let's try this. I turn off wifi and bluetooth on my device.
- [20:52]
Matt Southworth
Sometimes I turn my device off.
- [20:54]
Saket Modi
Leave my devices at home is the first thing. And don't go online.
- [21:00]
Matt Southworth
Other than hiding my badge so that vendors don't scan. Stay conscious from an electronic perspective.
- [21:06]
Saket Modi
Hold onto your stuff and put your phone in flight mode.
- [21:10]
David Spark
What do you think that is? Socket. What's the question?
- [21:13]
Saket Modi
Yeah, well. Or how do you stay protected while you are with a cell phone?
- [21:21]
David Spark
Close. Not really.
- [21:23]
Matt Southworth
Sounds like. What's your conference? Operational security.
- [21:26]
David Spark
Yes, that is 100% correct. Good job. All right. Very close. Very good.
- [21:31]
Matt Southworth
I do none of those Things, by the way.
- [21:32]
David Spark
Last one, last one. This one I think you'll get.
- [21:35]
Matt Southworth
Shirts, apparel in general, knickknacks, a backpack.
- [21:39]
David Spark
Whiskey cubes, a good multi plug, or especially one that goes into an AC.
- [21:45]
Saket Modi
Plug with all the different multi plugs. The cell phone holder of all things.
- [21:49]
David Spark
All right, what was the question?
- [21:51]
Matt Southworth
What's the swag you leave behind in your hotel room when you're going home?
- [21:54]
David Spark
Actually, no, no, no, that's not the answer. You're close, but you have the opposite answer. Socket. Take it away.
- [22:02]
Saket Modi
What's the swag that you get at an RSA conference, shop floor?
- [22:06]
David Spark
Well, it's essentially what swag do you still have? What swag do you still have? So it was the opposite of what you were saying.
- [22:15]
Announcer
What's the starting point for a C?
- [22:20]
David Spark
All right, quote. Likelihood is 50 50. Either it happens or it doesn't. Impact is super critical. My asset is the most important thing on God's green earth. Response is accept, since I still don't give a damn. End quote. That's how one frustrated security manager on the cybersecurity subreddit described their staff's approach to risk management. They're starting from near square one. No risk culture in place, stakeholders making wild guesses on risk. The advice they got ranged from, quote, build a simple impact likelihood table in Excel. To quote, you need process before tools to just use historical incident data. I'll start with you, Matt. When you're starting a risk management program from scratch or inheriting what amounts to security theater, what's your first move? Do you start with the spreadsheet and workshops, invest in a tool to force structure or focus entirely on building risk culture before touching any frameworks? What do you think?
- [23:21]
Matt Southworth
I doubt I have the latitude to build a risk culture over five years before showing any results, but definitely process over tools. And I don't hate the idea of workshops and brainstorming. Finding the people who understand the business and grilling them on what's important. Everyone thinks their world is best place on God's green earth. But narrow it down and then talk about it from a business perspective, not even a risk perspective. Focus on what they already understand.
- [23:55]
David Spark
I like it. All right, this is right up your alley. Socket. Where do you start? You want to do risk quantification management, but I mean, so many places start. Where do you begin?
- [24:06]
Saket Modi
You know, I'm going to add to what Matt said because I totally agree with him and I'm going to add one more variable. It's actually people process and then technology. You start with people. Because if you don't have so building the culture first, you don't build the culture. I actually feel that there are companies which are very risk driven and then there are companies who doesn't care about the risk at all. You can't go as a CISO or a CIO or a risk manager and change the mindset of the leaders of the company. So it's just the reality. That's where you start with people. And if there is appetite to understand that there needs to be risk, that needs to be managed, you start with processes, basic ones, and then you go ahead and invest into tools and technology. Technology is not the answer.
- [24:42]
Matt Southworth
No.
- [24:43]
David Spark
And as we know, like we talked about that if you try to implement technology to fix a problem, all you're doing is going to take whatever problems you have and make them go a lot faster. Correct. That's assuming you don't have the process in place like you pointed out.
- [24:57]
Saket Modi
I agree.
- [24:58]
Matt Southworth
Second, how long do you think you should ask for to show some results?
- [25:04]
Saket Modi
So a couple of points even before you show the results, the way you calculate the results is important. So you want to start with that to say, look, this is the first phase where even before I show, oh, I'm burning down the risk. The way I calculate the measurement of risk is important. And that is the reason why we see that we come in very early stages on the maturity cycle where company wants to start their journey in understanding and make more risk based decisions and getting the instrumentation right. And once that is in place, again depends on the speed of the company. When we are working with technology companies which are software developers, they move very fast in burning down the risk when they see there's something critical on the other side. There are companies which are very, very comfortable saying, hey, we've never got hacked, we don't think we'll get hacked. And we're okay with that. So as I said, right. It's the mindset of the people where you that is really where rubber meets the road. It's not the technology, it's not the process.
- [26:00]
David Spark
So question about process, like what are some kind of key processes you need to have in place before a tool enters the picture.
- [26:09]
Saket Modi
I think the big thing is again, process is a very heavy and a loaded word.
- [26:14]
David Spark
Yes.
- [26:14]
Saket Modi
I think something like what they say about culture, the culture of the company is the behavior of the founder and the CEO and that really gets reflected across the company on what is considered as normal. The same thing is applicable for risk. The way the CEO and founder looks at risk and how do they manage risk? There's some people who will just wing it and say, hey, we're good and we'll do that. No matter what process you have in place. Doesn't work on the other side. There'll be CEOs. I know. For example, there was a big town hall meeting with bank of America and one of their staff told their CEO that because of this crazy multi factor authentication that we have, we're actually losing business. And this was on a town hall. And Brian, their CEO, actually said, guys, I want to make this on record. I don't give 2 cents about the business. Even if we lose business, it cannot come at the cost of CyberSecurity. When the CEO of your company makes that statement in the town hall, Big, big. These were the exact words. And this was actually published on a few threads. So that's why I read it and I found it out. But the reason I'm saying this to you is that that is the moment, you know, that that is a company ready to be managed by risk. And no doubt bank of America is the gold standard in a lot of ways.
- [27:30]
David Spark
And I'm sure you've had situations where a lot of sort of a sales process of this is just education. The platform, you know, none of the.
- [27:40]
Saket Modi
Platform of, of the risk that they're actually sitting on. Because a lot of times, yeah, because people don't care about platforms or tools. What they care about a lot of times is that, hey, I didn't even know if something goes wrong, it can cost me $50 million. The moment they understand that, they don't know that they're worse.
- [27:59]
David Spark
Because it does look like a black box.
- [28:00]
Saket Modi
It looks like a black box and like, hey, it's not going to happen.
- [28:03]
David Spark
To me till, well, until it happens. And it's one of those. Yeah, it's because it comes out of nowhere.
- [28:08]
Saket Modi
Totally.
- [28:09]
Matt Southworth
So before process or tools, we have people, right? How do you identify within the organization the people who substantially own the risk, whether it's in their JD or not?
- [28:22]
Saket Modi
The owners of the risk are the owners of businesses. So if you've got five businesses, the CEO or the business unit head is the one who owns the risk. This is generally misconception. The GRC team, the CISO does not own the risk. The job of the CISO is like the job of the cfo. The CFO doesn't own the business of each business unit. The job of the CFO is to show the mirror to each business of how well, slash, not well are they doing. That's exactly the job of the CISO to say, hey, this is the risk you're sitting on. It's fine if you want to be okay with that, but this is how you can burn it down in the most effective way possible. And that's the core job of the csa. So to your answer Matt, who owns the risk? It's always the business owner and that.
- [29:04]
Matt Southworth
Those people is who you need to invest your time with. Yeah.
- [29:10]
Announcer
What would you advise?
- [29:16]
David Spark
We spend millions on EDR and firewalls, but our real weak point is a 10 minute phone call to a Tier 1 agent a debate on the cybersecurity subreddit had some arguing for eliminating human touch points entirely, for example for self service password reset with MFA or in person resets only. Others pointed out that help desk agents are underpaid, outsourced, graded on ticket velocity and incentivized to be helpful above all else. And some pointed to a middle ground, callback verification with managers and identify assurance tools. One commentator tried to find a silver lining saying quote, if most user requests are automated and go through some nasty servicenow UX then most requests are protected against social engineering. End quote. So where's the line? And I'll start with you Matt. When does removing human judgment from the help desk actually improve security and when does it just create a worse user experience that people will find a root around?
- [30:18]
Matt Southworth
I think most people love self service and would rather click a couple buttons and get their problem solved and not pick up a phone. At least as a basement dwelling indoor cat, that's me. Ten minutes before we started my boss called me to ask me about a phishing email. Like you're the cfo, you don't need to worry about this. But some people need that human conversation because they're going to have follow up questions. So answer your question. As much as we can give guardrails to a bot to do things automatically, that reduces the attack surface. As long as you've tested and you know it's constraints, humans are always going to want to escalate, ask a question, have a follow up. And I think the solution there is have those frontline help desk folks feel like they're part of the security team, have them a little bit empowered but heavily drilled and tabletopped about what could go wrong. Send them the worst case scenarios and let them express human judgment.
- [31:14]
David Spark
That's actually a good point that we talk about Tabletop, but Tabletop for the Tier one as well.
- [31:20]
Matt Southworth
Oh absolutely. That's probably more important than me, right? They need to see it all the time and it's good cross team relationship building. Right. If the head of security engineering sits with the help desk and walks them through scenarios, that's great for both sides.
- [31:36]
David Spark
Socket.
- [31:37]
Saket Modi
Yeah. The challenge there, Matt, let me challenge you there is that when you talk about a few hundred people, that's still okay. But when you're talking about a help desk for an insurance company which has probably 5,000 people and doing that for every single person in a scalable way, generally you only do it with videos and these video awareness. But that clearly doesn't work. Right. So that can be a challenge.
- [31:57]
Matt Southworth
Absolutely. If it's an organization at scale, if it's a customer service org where you can only give one answer, then yeah, you're stuck. But what happens when the CFO calls that customer service org?
- [32:09]
David Spark
Right.
- [32:09]
Matt Southworth
You've got to have some sort of executive care escalation for them.
- [32:13]
David Spark
Can't you do some kind of like periodic phishing test for your service?
- [32:19]
Saket Modi
I actually think periodic phishing tests are actually now more expensive than just giving a simple AI bot. Because if you think about it, right again, replacing the agent with a AI bot, which a lot of companies are trying to do, will still take time. If it wouldn't be installed, there would be no contact center agents today itself. Right. So it'll still take time. But recording a call, which is already happening, looking at what is being told and actually flagging that, oh my God, this looks like a scammer. Oh my God, this looks like somebody's trying to fool us, right. And giving like a big red sign on the screen. And that's, that's easy, that's not very difficult. I think that those are the measures again, AI to the rescue. I actually think that would be the thing which will change that industry and that problem.
- [33:04]
David Spark
Is this line clear or is it very blurry? I'm assuming blurry, where you sort of cross the line of making things too difficult for people or making it too permissible, where Tier 1 agents can fall victim. What do you think? Do you know where it is?
- [33:22]
Matt Southworth
No, I don't know where it is, but. But it's. I'd actually step back and look at the incentives on both sides, right. The people who are calling out, do they have incentive to game the system? Are they trying to get something from nothing? Are they trying to, you know, get some free product or something? Right. Then you're going to tighten things down. On the other side, how are you incentivizing the agents? Right. Are they comped because of the number of tickets they Close. Are they comped because the person calling got the outcome they wanted?
- [33:52]
Announcer
It's time for the audience question. Speed round.
- [33:57]
David Spark
All right, we are coming to the end of the show here, but before we close up, I have questions from you, the audience, right here on these index cards. And with what little time we have left, I'm going to get through as many questions here as possible. So quick answers to all of these. You have not seen these at all. They don't know what these are, so they're going to be surprises for both of them. All right, either one of you jump in here when you see data or a dashboard. This, by the way, comes from Greg Wham of Trend Micro. When you see data or a dashboard, what questions do you ask of the data to see if it is actionable? Meaning, should I act on this?
- [34:31]
Saket Modi
Yeah, simple thing. You know, I like the data behind the data. So you're asking a simple question of so what? And who cares? Is a very, very simple litmus test to say, is this even useful?
- [34:42]
Matt Southworth
Yeah, I think this morning I asked, am I going to do anything different with this data? And if the answer is no, I'm going to stop looking at it.
- [34:49]
David Spark
All right, good, quick answers. All right, from Henry Stanley of Fabric, who asks, what incentives can we provide to get suppliers to share their security and assurance data? Is there something we can do to make them go, oh, yes, I want to give you this data.
- [35:06]
Matt Southworth
So we tell a story about how we don't compete on security, right? How we're all in this together and then we hug at the end of the day and we buy them drinks.
- [35:13]
David Spark
At Black Hat, but we don't share our data.
- [35:15]
Matt Southworth
Exactly.
- [35:16]
David Spark
So what's the incentive? That's the question.
- [35:18]
Saket Modi
There are only two dimensions, time or money. You give back time by saying, hey, I'll make it easy. Just drag and drop this and you've got it. No more wasting time on that. Or you give discounts, or you say, hey, I'll pay you more money or time.
- [35:32]
David Spark
Yep, that's a good way of putting it. All right, I like that. All right, From Paul Love of Delta Airlines asks, I like this for both of you. I'll start with you, Matt. Give me just one. One storytelling technique you found successful. When you're talking to the board, I.
- [35:49]
Matt Southworth
Fall back on using I statements. Right. I try not to talk about anyone but me. And what did I do? What did I feel in this situation?
- [35:57]
Saket Modi
That's a simple one. Use safe.
- [36:02]
David Spark
That is good. I like that. All right, from Ayub Thandi Grc with the GRC Engineer podcast. How do you build a product when everyone thinks they're a snowflake? This is definitely for you, Saket.
- [36:16]
Saket Modi
Well, you know the fun part of the company, Snowflake, that's the reason why you've got a hundred billion dollar company. And I think, you know, again, thanks to the flexibility of the world of agentic AI, you actually can. Because the old way of doing things when every company is a snowflake is you need a 200 clicks. And that's why the user experience in one of your questions says shitty user experience of almost every B2B product that is changing thanks to AI. So I actually think it'll be much easier in the world of AI to build a great user experience while having something very personalized.
- [36:50]
David Spark
And so the answer for all the snowflakes, More AI.
- [36:53]
Saket Modi
Yes.
- [36:53]
David Spark
All right. In fact, I'm going to give you a follow up question sake, from, from your colleague over at Safe Security, Megan Monaval. She wants to know what was your aha moment to start Safe. Did you have an aha moment?
- [37:06]
Saket Modi
I was in college and I didn't know if I would be employed. So I started this in my third year. It's been a good ride.
- [37:12]
David Spark
Okay, so it's more of a fear.
- [37:15]
Matt Southworth
It's like, is anyone going to hire me?
- [37:17]
Saket Modi
Well, I never applied anywhere, to be honest. But then I was like, yeah, I guess people do something in their third year so why don't do a startup? And that's what happened.
- [37:26]
David Spark
All right, this last question from Jennifer Chu of Alliant Credit Union for both of you. Start with you, Matt. What are the pitfalls of a CRQ program? Cyber Risk quantification program. What are the pitfalls?
- [37:39]
Matt Southworth
I think it's just garbage in, garbage out.
- [37:41]
David Spark
Right?
- [37:42]
Matt Southworth
It's where is your data coming from? And the biggest problem I've seen is with completeness of data. Right? You have some data, it's accurate, but does it tell the whole story?
- [37:52]
Saket Modi
Well, Matt, you know I shared three learnings of the problems of CRQ programs. You took the exact center one and the right one. I literally had on this screen garbage in, garbage out, which is one of the biggest problems. So I totally agree with him. I think the big problem in my view, a bigger problem in my view because you're talking about when the program is being implemented. But I'll tell you why programs are not even starting. That's because the biggest problem with CRQ is the name itself. If you ask anybody, do you need quantification? No, I have dashboards I have numbers. No, but the moment you understand the value that that delivers, do you want to know if something got hacked? Are you vulnerable to that? How exposed are you to that? Something crowdstrike detected. How does that change my risk posture? How do you compare with your peers? How did your risk burn down with your budget? These are value. Everybody wants to know that. So CRQ itself, if you just say crq, it's not useful is what most people think, especially the smart people, because they don't want to double click there.
- [38:47]
David Spark
So the pitfall is, is missing.
- [38:49]
Saket Modi
The value is the name itself.
- [38:52]
David Spark
Oh, so you got to rebrand it. It's your fault now.
- [38:54]
Saket Modi
Decision Intelligence, Sierra Key was branded before I was born, probably. So that's the problem. So we can take the blame. We go to Decision Intelligence. We like that better.
- [39:05]
David Spark
All right, well, that brings us to the very end of the show. I want to thank our guests here. Saket Modi, CEO of SAFE Security. Let's hear it for Saket. Also, Matt Stouthorse, who's the CISO of Priceline. Let's hear it from Matt. And huge thanks to our audience. Thanks to safe. Thanks to the Fair Institute for bringing us here. And also remember, SAFE Security, autonomous cyber risk management reinvented with agentic AI. We are thrilled that SAFE sponsored us for this. Remember, go to their website SAFE Security. As always to our audience, those here and those listening to us right now, we greatly appreciate your contributions. And for listening to the CISO series podcast.
- [39:59]
Announcer
That wraps up another episode. If you haven't subscribed to the podcast, please do. We have lots more shows on our website cisoseries.com please join us on Fridays for our live shows, Super Cyber Friday, our virtual meetup and cybersecurity headlines. Week in review. This show thrives on your input. Go to the participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you're interested in sponsoring the podcast, contact David Spark directly@Davidisoseries.com thank you for listening to the CISO Series podcast.