Loading summary

Transcript138 lines

[00:00]
David Spark
10 second security tip. Go.
[00:03]
Kush Kashyap
I would say stop being the tollbooth. When people reach out to you and they ask you questions about a certain control, what is the best practice? How was it designed? How does it need to be operated? Own the decision and share the decision because you'll lose credibility if you don't move fast with guardrails.
[00:24]
David Spark
It's time to begin the CISO Series podcast.
[00:27]
Foreign. Welcome to the CISO Series podcast. I am David Spark. I'm the producer of the CISO series and joining me since episode one, we're recording this actually just a few weeks after our eight year anniversary. It is Mike Johnson, the CISO of Rivian. Mike, say hello to the audience.
[00:54]
Mike Johnson
Hello, audience. It's amazing to believe it's been eight years.
[00:57]
David Spark
David, it's been eight years and you had no idea when I took you out to lunch this was going to happen at all. Neither did I, for that matter.
[01:09]
Mike Johnson
Who knew what would happen?
[01:10]
David Spark
By the way, that was money well spent on a lunch. I will say that we are available@cisoseries.com where we have four other programs that launched not too long after that first episode, which was eight years ago. Go check them all out. Our sponsor for today's episode, which has been a phenomenal sponsor of the CISO series responsible for us to continue these eight years, it is none other than Vanta. Trust is everything. Earn and prove it with Vanta. And we're going to be talking a little bit more about that today. And our guests, courtesy of Vanta. But before I do that, now, one thing that's been sort of a chronic joke in Cyber Mike is how everything is XYZ with AI. This with AI, Right, sure. Yep. Well, we're like operating our own bubble. And it isn't just security that gets XYZ with AI. So just a quick story. So I live in San Diego. I like to go to these entrepreneur meetups and meet other entrepreneurs and hear what they're doing. Now, as you might imagine, I'm in Southern California. It's San Diego. There's a lot of sort of beach people, a lot of sort of earthy characters. And, you know, there are many stores with psychics and healing and crystals and things like that. Okay. And people giving you advice on how to lead and run your life in general. Sure. Now you get that with AI. Oh, no. Yes. The number of entrepreneurs I've met who have essentially that same shtick, if you will, or however you want to describe it or however you feel about it. But now, say, for example, I met this one woman who is starting a business hasn't launched it, but the idea is she will. Look at the way you lead your life, Mike. So how you sleep, how much coffee you drink, anything else behaviorally in your life and tell you what's the best time you should schedule a meeting so you perform that at your best.
[03:17]
Mike Johnson
Wow.
[03:18]
David Spark
Yeah.
[03:19]
Mike Johnson
You know, is that a service you
[03:21]
David Spark
find yourself paying for?
[03:23]
Mike Johnson
Yes. But what's, what's interesting about that and the kind of the correlation with that is there are entire tech companies who are essentially front ends to AI. Like they're just a UI on top of ChatGPT or Claude or Gemini or what have you. And all these are going to go away.
[03:42]
David Spark
I know. And I was, I was also, I was talking to these other women who, you know, said, oh, you know, we'll help you design your business model. Well, we did it ourself. We asked AI and just type it, did it. And I go, and so what would stop anyone from doing exactly the same thing you do?
[03:58]
Mike Johnson
Exactly. That is the problem every one of these companies has.
[04:02]
David Spark
So we are not the only ones that are getting fooled by, by businesses slapping on AI. It's trickling everywhere.
[04:14]
Mike Johnson
It's good to know that there are bad ideas everywhere.
[04:16]
David Spark
Yes, there are. But you know what, here's the thing about bad ideas, I am sure, like the ones I just mentioned, I will be proven so wrong, it'll be a colossal hit. And I go, look, what a moron I was.
[04:30]
Mike Johnson
Good for them, good for them.
[04:31]
David Spark
I don't begrudge anyone from their success. But I, you know, I heard that I'm like, really? And then, and also, you know, you, you know, this like something becomes a smash success like a movie or an artist and you're like, really? Someone wants to watch that, listen to it. Guess so.
[04:47]
Mike Johnson
Yeah, there, there, there is an audience for every show.
[04:51]
David Spark
Exactly. And with that being said, this show is not infused with AI. It is actually my voice recording it and Mike's voice recording it. And my guest, who we're going to introduce in just a moment, our return guest. Thrilled to have her back with Vanta, the senior director of grc. Our sponsor guest, Kush Kashyap. Kush, thank you so much for coming on back.
[05:13]
Kush Kashyap
Oh, thank you so much, David. It's always a pleasure.
[05:18]
David Spark
Is this the best use of my money?
[05:23]
The user sees six words. The system processes a small novel. Barbara Roos of Starboard collectives argued at AI Field Day 8 that employees are burning tokens with no awareness of what they're consuming. Now we took this up on our BSIDES SF live show. You were there, Mike. And the frame shifted pretty fast. So Jensen Huang, who is the CEO of Nvidia, reportedly wants his $400k a year engineers spending 200k a year on tokens. That's the goal, which is understandable from his viewpoint, as you pointed out. So the premise is that organizations should steer employees toward token literacy before cost spiral assumes the wrong problem. Roos complains that ROI measurement and token awareness are the same. Conversation falls apart if experimentation is the ROI during early adoption. So is watching how employees use AI just micromanagement with an infrastructure budget attached, which is the way I saw it originally? Or do people actually need to know what a token is? Or do they just need access and room to run? Mike, what does this say about your AI strategy if you're managing prompt efficiency instead of outcomes? What do you think?
[06:40]
Mike Johnson
I do wonder. That quote from Jensen was quite a while ago, right?
[06:45]
David Spark
So this was February, I think.
[06:47]
Mike Johnson
It's amazing how quickly these things move these days. And I don't know, like, AI. My guess is he's changed his mind since then. You've got the case of someone at Uber saying that they had blown through their entire year's budget in a few months.
[07:03]
David Spark
There was some story that some company had like a $500 million token expense.
[07:08]
Mike Johnson
It was. I don't know, there was 500 million, but Uber's. Uber's was. Was way up there, and that's what they were talking about. But I think right now, now is probably not the right time to be concentrating on roi. We're very much in the experimentation phase.
[07:26]
David Spark
And do you say that to your staff? Because I've said it to my staff. I go, oh, yes, please feel free to experiment.
[07:32]
Mike Johnson
Yes, yes, absolutely. And we have quotas attached with our AI usage, and we manage that. And my way of looking at it is if someone on my team comes to me and says, hey, I want to spend more, I just approve it. Like, let's go, let's experiment and let's see where this is going to take us. And I think that's one of the things that companies need to think about, is the costs right now are not going to be the same runtime costs in the future.
[08:01]
David Spark
That's a good point.
[08:02]
Mike Johnson
We're going to settle down on how we use these tools. And in this day and age, how we use it today is not how we're going to use it in the future. And so I think the idea of teaching people what tokens is that doesn't make sense. They don't need to know what a token is.
[08:19]
David Spark
It's also kind of confusing.
[08:21]
Mike Johnson
It's confusing and it can also change, right?
[08:25]
David Spark
I mean it's a made up thing. It's like cryptocurrency, like who frigging knows?
[08:29]
Mike Johnson
Yes. I mean it has an underlying technical meaning, but it doesn't matter. What matters is how much you're spending and what value are you getting out of it. And right now that value is more in the R and D phase than it is in actual hard business outcomes
[08:49]
David Spark
token uses is your R and D budget. All right, Kush, I like that. What are your thoughts on this? And by the way, have you seen this trend? Because I think these stories of these ludicrous budgets being blown is shocking people. Because people like yes, we want to be aggressive, like, oh, I didn't realize how aggressive we can be. Your thoughts?
[09:10]
Kush Kashyap
Yes, totally. I think I have seen it, personally seen it in several companies I've been at as well. But one thing that I do want to mention here, which is slightly different take, is that I think more than just understanding what our usage is in terms of tokens, we are also at a point now where we can start developing taste and judgment. We should start asking ourselves if we are building something, is it going to be used by various people? Is it going to add value for the company, for the business, for the security function, is it efficient? Do we know what good looks like, which model to reach for, when to push and when to use something else? For example, you may not want to use your most expensive model on like a Slack conversation framing. You don't want to use it for drafting a Google, you know, a Gmail message, but you may want to use it when you're thinking about like a gnarly architecture or when you're thinking about creating a PRD for the next set of AI driven vulnerability management task tracker. So there are different things around taste and judgment which we should also I feel like we are at a place where we can start experimenting with quality as much as we hone in on usage. Because I don't think fixating on token efficiency right now is a good thing. We should focus on what good output looks like, what good adoption looks like. How will we make sure that what we are building is going to be useful for our company, for our business, for our security teams? Instead of wasting a lot of money and just trying to see what sticks on the wall,
[11:01]
David Spark
it's time to measure the risk.
[11:06]
Not every risk can or should be Reduced to a dollar amount or outage scenario. Now that's William McBorough, CISO at MC Global Tech, pushing back on the playbook CISOs have been handed for executive conversations. Tie everything to money downtime or data loss and you'll get the room. But he's pointing at something the standard framework glosses over the risks that shape a business for years. Eroded customer confidence, a compliance lapse that delays market entry, an insider event that exposes proprietary designs. Those do not always register on next quarter's balance sheet. So the question this piece opens up is the harder one. What exactly are those risks and how do you make them land with a CEO who hears risk and thinks cost? So I'll start with you kush on this one. How do you build a case for something you can describe but can't price? And at what point does the push to quantify everything make your risk conversations less honest because you find that you're trying to push a round peg into a square hole kind of a thing?
[12:16]
Kush Kashyap
Absolutely. I mean, I also want to throw in a caveat here. Back in 2016, I spent a whole two years doing risk quantification and selling risk quantification models to some of the largest tech companies in the world and some of the largest financial services companies in the world. But those were the same questions that were being asked then is the same question that are being asked now. How do we know it's accurate? How do we know financially it will hold up? So, I mean, having struggled with it in a very close basis, what I can say is I have realized over time that the most important risks are not the ones which are quantified in dollar terms, but they are around eroding customer trust or a compliance gate that locks you out of a market potential or a proprietary asset might be getting lost.
[13:12]
David Spark
Well, essentially, business brand damaging.
[13:15]
Kush Kashyap
Absolutely.
[13:16]
David Spark
And I would assume every business understands that their brand is their most valuable asset.
[13:22]
Kush Kashyap
Exactly. And I also think that the way you make them land with the CEO isn't that this risk is this amount of dollar. It's about the decision. If we ship something without security controls, here is the deal we can lose. Here is the market we won't be able to enter to. Here is the customer commitment and the obligation we promise to respect. We are breaking that if we anchor it to the promises and the controls the company has it made. Now we are talking about tracking those things. And a risk is tied to a specific commitment the CEO personally signed off on, and it lands harder and better than a quantified number that People are always going to question, and you don't want your discussion to be around, where did the number come from? How do we know it's accurate? What's the model behind it? What's the formula behind it?
[14:17]
David Spark
Then you're defending something completely different.
[14:19]
Kush Kashyap
Exactly. And you'll be spending your time on making your decisions better, making the people in the room accountable for the risks that they need to drive to mitigation, and you'll be spending your time on better places instead of analysis paralysis.
[14:34]
David Spark
And speaking of spending your time on something, at the beginning, I mentioned the metaphor of the round peg in the square hole, which I did it in reverse, because you can actually fit a round peg into a square hole. You can't fit a square peg into a round hole. So I don't want any letters about that, because I do know what I said, which was the complete opposite of the appropriate. This is like when people say, oh, I did a complete360, when they meant a180. So I pulled one of those. Is what I did just there, Mike.
[15:04]
Mike Johnson
Well, actually. David. Well, actually, you just need a smaller peg, and it'll fit just fine.
[15:09]
David Spark
It'll fit fine. You know what? Yeah, if you're. No, but if I have a very large round peg trying to fit into a small square hole, I can't do that.
[15:18]
Mike Johnson
You just need a bigger hammer is what it comes down to.
[15:20]
David Spark
All right, But I'm sure, by the way, Mike, you have been a CISO for a long time. You've had to describe risk God knows how many different ways. I'm sure you've beaten your head over. How am I going to describe this? How are we going to prove this? How are we going to explain this? I mean, let me ask this. Look at your first year as a ciso, and now, what is the thing that you know about risk now and sort of quantifying and explaining it that you did not know that first year?
[15:51]
Mike Johnson
I think a lot of it is a realization that so much of our job is about storytelling. And I have long wanted to put numbers to everything. I'm an engineer by training, and so I always wanted to put numbers on things. And risk quantification seemed like this amazing opportunity to do that. And what I've come to realize is it is but one tool in our quiver, and it's not necessarily going to be the best one to tell the story to convince folks that you need to go in a direction. And one of the things that I was, you know, as I was listening to Kush, there's the quote that says if you're explaining, you're losing and if you're spending all of your time defending why this is going to cost this amount, explaining the model you've already lost and the story is not going the way that you want it to. So I really think the biggest thing that I've learned is so much of it is about being able to tell a story that the point that you're trying to get across is what people come away with, not well. Why did you name the character that you don't want them going down off script in a direction that is totally tangential from the point that you're trying to make?
[17:18]
David Spark
What's the one thing in business that's spreading as fast as AI? AI risk. Every new tool your team signs up for, every vendor that turns on AI features, every new integration, each one is an opportunity for something to go wrong. And most security programs weren't built for AI's pace of growth. I mean, geez, nothing was. But this is where Vanta enters. Vanta is the number one agentic trust platform used by over 16,000 fast moving companies like Ramp, Cursor and Harvey to ensure they're always audit ready. And now Vanta is helping companies like yours watch for the risks that show up between audits across your vendors, your AI tools and your whole environment. Now how do they do this? Well, the Vanta agent works like a 24.7grc engineer in the background finding issues, drafting fixes for you and cutting vendor assessment time by up to 50%. Now whether you're a fast growing startup or a global enterprise, Vanta is here to help you automate your security and compliance and earn improve trust. That is important. So get started today@vanta.com CISO and it's spelled V A N T A.com CISO so vanta.com CISO do me a favor, add that. CISO. Easiest way to let Vanta know that you learned about them from the CISO series.
[18:51]
It's time to play what's worse
[18:56]
Kush. You've played last time. Now do you remember if you agreed or disagreed with our guest last time we played?
[19:02]
Kush Kashyap
I think I agreed on a couple and I disagreed on at least two.
[19:07]
David Spark
Okay, well, there'd be issues I guess, because there's only issues.
[19:10]
Kush Kashyap
Yes.
[19:11]
David Spark
We're only going to play one round of this, but I'm going to make Mike answer first.
[19:14]
Kush Kashyap
Yes.
[19:14]
David Spark
And since we've been talking about AI here, guess what this one has to do with AI.
[19:19]
Mike Johnson
Great. Maybe I'll Ask AI the answer.
[19:22]
David Spark
You know, we could. I never thought about that. I mean, I get my two guests and go, by the way, this is what AI Said.
[19:27]
Mike Johnson
Yeah.
[19:28]
David Spark
This comes from one of our listeners who sent. Who has sent us many wonderful what's worse scenarios. It is Dustin Sachs of Cybercog Labs. And here you go, scenario number one. Your agentic AI starts taking all actions without your permission. Okay, everything. Or the complete opposite. Your agentic AI asks for permission on every single action. We're talking thousands upon thousands of actions. So which one is worse? Mike?
[20:01]
Mike Johnson
It's funny that as we're recording this, literally yesterday I was doing something with AI and I had to just keep hitting the inner key over and over and over again.
[20:11]
David Spark
You know what this reminds me of? Did you ever see the TV series Lost where like X number of hours, they had to just press a button. They didn't have a clue what the button was doing, but they had to do it because if it didn't happen, something horrible would happen.
[20:24]
Mike Johnson
That's a different perspective than what I remember is when Homer Simpson had the dunking duck that he put over the button at work. So I like your callback better than mine. It adds more urgency to it. So as I think about these two scenarios, it's like you've got one which is like default open versus default closed. Right. Like you've got. It'll just do anything, everything. You can't stop it.
[20:52]
David Spark
Essentially, there are problems on both sides of this.
[20:55]
Mike Johnson
Exactly.
[20:55]
David Spark
And let me qualify what Dustin says, which I think he sets it up. Nice. One is the goal. I mean, the goal is the first scenario. It is kind of. But it's problematic today. And the second is safer, but it completely defeats the purpose of a gentic AI. It actually makes your life far, far more miserable.
[21:14]
Mike Johnson
Well, I mean, he just answered with that. Right.
[21:17]
David Spark
But which one's worse?
[21:18]
Mike Johnson
Well, I think the point is today, which, again, the scenario, we don't get to change things. The point is today, the one that hasn't really gained the trust to operate fully autonomously, that is the one that's the bigger risk. The second one is a user experience nightmare. Like, that's terrible, but that's going to.
[21:45]
David Spark
Okay. And then back up a step here, Mike. Both are very damaging to the business.
[21:52]
Mike Johnson
Again, the point is they're both terrible.
[21:54]
David Spark
Yes.
[21:55]
Mike Johnson
Just in different ways. And that's usually the point of these is. Yeah, because if it was easy, if, like, one of them wasn't terrible, then, well, problem solved.
[22:05]
David Spark
That sometimes happens that I'm not aware of it.
[22:09]
Mike Johnson
But I really think today maybe we dust this one off again in a year and see what the answer is. But I don't think that we've quite reached a comfort level with agents that we can just let them just run completely unsupervised and just do whatever the heck that they want. So I think today the first one is the worst.
[22:30]
David Spark
But your answer may change a year or two years from now.
[22:32]
Mike Johnson
Frankly, I hope it changes in the future.
[22:36]
David Spark
Well, again, like Dustin said, the first one is the goal.
[22:41]
Mike Johnson
Absolutely.
[22:42]
David Spark
It's totally the goal. All right, Kush, do you agree or disagree with Mike here?
[22:46]
Kush Kashyap
Gosh, it's so hard to disagree on it. Like there is a portion of my team who is in charge of making sure that the agents we ship have human on the loop. Quality assessment, everything done. Accuracy is high. You can't get there. When you let your agent run even with guardrails, run loose and have no human in the loop to check what it's doing, you don't want to make any material changes on what's going to impact your audit later when your auditor will ask you who made this change? Why was this change made? Show me the history. Walk me through it. What are we going to say? So that's a nightmare. Yes, it's inconvenient to keep clicking. Yes, allow. Yes, allow. But I would rather have the inconvenience than like a poorly drafted design control.
[23:35]
David Spark
Also, it's going to bring your employees. First of all, it's going to bring your employees productivity to a screeching halt. It will let me throw some other things. I'm playing devil's advocate here. How many people are going to be enjoying their job, doing this all day, not getting access to anything else? You may lose staff. I'm just. There's a scenario that go wider than this, but then they throw this else out is we know that if you let AI run rampant, it's going to be a matter of very little time before you know what starts hitting the wall. Give you example, Mike. When I was working at zdtv, the television cable network which later became known as tech tv for one of our shows called the Screensavers, we had a, a computer called the virus computer that was purposely set up with no protection on it whatsoever. And the, the joke was how quickly could we get a virus? How quickly could this be? And it would sometimes be minutes. You know, it'd be like the speed was. So I think the same thing would happen here, Kush as well. It's like, well, let's let's give this a whirl. How. What's the speed? We're gonna have problems because we know it's not like it's gonna run for months. And it's. It's gonna be. We're gonna be talking minutes or a day at max.
[24:49]
Kush Kashyap
Yes, I think so.
[24:52]
David Spark
It's gonna be pretty fast. But also, who knows, it couldn't. Maybe it couldn't be that damaging.
[24:58]
Mike Johnson
Maybe it'll fix itself.
[25:00]
David Spark
I like that idea. Oh, Mike, you are very, very optimistic.
[25:07]
Mike Johnson
Aren't you ever the optimist, David? Ever the optimist.
[25:10]
David Spark
Very. I'm sorry. So I cut you off because. Any last thoughts on this?
[25:14]
Kush Kashyap
No, I actually do agree with Mike on this one. Sorry, David. But I have to say I hope we can revisit this answer in a year and hopefully our answer will change and we will be able to trust the models to make decisions and to carry out actions without losing sleep over them and without losing the trust of our stakeholders.
[25:35]
David Spark
I think. I argue that the second one could be worse because you're going to. The employees are going to be so annoyed, so irritated, they're going to learn nothing, and they're going to be very disturbed, and they may leave the company. Although I think in the first scenario, a company will fall apart before that could happen.
[25:57]
Kush Kashyap
Exactly.
[26:02]
David Spark
Please, enough. No more.
[26:07]
All right, today's topic, Kush, Mike, is security team resilience. I'm gonna start with you, Mike, on this one. What have you heard enough about with security team resilience? And by the way, this whole discussion of security shifted to the term resilience a long time ago, like, a few years ago, we started talking more about resilience than just, quote, security, because we all realized that 100% was impossible. So what have you heard enough about with security team resilience? And what would you like to hear a lot more about?
[26:37]
Mike Johnson
So when I hear the term security team resilience, I mean, like the humans, I hear the resilience of the team. And so when I hear that, I go kind of the opposite side of this. Like, what is the opposite of security team resilience? And to me, that's burnout. That is where you have folks who are just working too hard. They will just eventually crack and they'll leave, to our last discussion, deliver subpar work. It just. It's a bad outcome. And I think we've been trying to solve that with either, oh, well, we'll just give them more vacation or we'll hire more people. And I think that's really the challenge that I see and what I've heard too much of is those are the wrong ways to manage burnout and drive towards a resilient team. So what I'd like to hear more is frankly, how do you do more with the team that you have and actually make them more resilient rather than just looking at it as oh well, I'll just hire my way out of this.
[27:45]
David Spark
Very few people can pull that off, by the way. I don't think I've ever heard success on that. All right Kush, I will ask you the same question. What have you heard enough about and what would you like to hear a lot more about with regards to security team resilience?
[28:01]
Kush Kashyap
Yeah, I have a spicy take on this.
[28:03]
David Spark
Let's hear it.
[28:04]
Kush Kashyap
So in terms of what I've heard enough about is the whole resilience as like a wellness culture vibe in companies, right? Especially when it comes to this.
[28:17]
David Spark
Welcome to San Diego by the way. Come here. We can help you with all of that and put little AI on it too while you're at it.
[28:26]
Kush Kashyap
I can't wait to visit. But that being said, I do see resilience as nothing but delayed attrition with better branding. Because what are we doing to the folks on our team by saying that absorb more with the same headcount we are essentially saying double the workload, triple the workload thanks to AI. Which is not. Which is still. We are still figuring it out what does good look like, what tools can we build where we can really depend upon it, solely where we can't. What I would like to see more of is creating better structures, creating better processes and yes, scaling, but scaling in a manner that's not going to burn out people and actually taking work away from their plate when they are doing these projects, when they are trying to scale and trying to put the processes in taking away some of the work so that the team is not always running on like 100 mile per hour with creating single point of failure and creating a risk for the business, for the security of the organization and the way everyone is operating. So I think that's really important because the honest conversation here is the workload has exploded even when we are adding more to our plate thanks to AI we have so many more non human identities to manage. There is an agentic sprawl. CISOs cannot say no as much as they could do before to vendors and to especially to AI models and AI vendors. The complexity curve is real as well. So we can't always keep matching that entire Curve and all that work with just efficiency. We need to have a better, more moderate conversation about it.
[30:12]
David Spark
We at the CISO series have found that we find these single projects at a time and are looking for ways to get AI to help us make it more efficient. I'll take a perfect example. If you've seen on our CISO series page, we always post these quotes, like from this episode, we'll pull three quotes from you and from Mike and we'll create this little meme image and we'll post it. This was done very manually and it took upwards. We would do six to eight episodes and creating all those quotes that could take us days to do. Days. We're now down to a few hours.
[30:56]
Kush Kashyap
Wow.
[30:57]
David Spark
But it did not happen overnight by any stretch. And it didn't go from days to a couple hours. It went from days down to a day and a half, down to a day, then to a number of hours. We kept finding these things. And I think that's where we have to sort of accept that's how the discovery efficiency will happen, is that it will not happen overnight, but you'll see incremental steps. And can you give me an example, either one of you, of something that you've done incremental steps and gotten from a thing that took days or a week down to maybe even hours? I don't know. Can you give me an example?
[31:38]
Mike Johnson
I think the best example is we have over time built all of our incident response playbooks and they were designed to be run by humans and they starts off with very manual. You figure it out, you eventually write down the process, you improve the process, and now we're able to turn some of those over to agents to actually operate. And so that, I mean, that's a journey of years in order to get there. So that's the first one that comes into my mind.
[32:05]
David Spark
And ours took years too, for that matter. Kush. I mean, this is kind of the bailiwick for Vanta. I mean, this is what you've been doing. And by the way, there has been AI infused in Vanta for a few years now. I'm sure what you guys had two years ago, a year ago, a few months ago, and today, completely different.
[32:25]
Kush Kashyap
Yes, yes, completely. A good example that I'll say is when we first released our MCP server capability, we used to take a look at how can we use our MCP server to build on top of anta, because there were certain nuances or certain reports that we wanted that don't come out of the box. Like maybe Our CISO wants to see something which is very risk driven in certain area of the business and very contractual obligation driven in another area of the business. How can we create these nuanced reports? It used to take days and sometimes weeks of review and making sure the messaging gets correct. But now the first draft, thanks to the MCP server and getting all that data in and drawing that analysis from all the different types of data elements that we have, from trust, from risk, from governance, from policies, from how our automated tests are doing, all these different signals come together and create the first draft of the report. So then we are thinking about is the story right? Who is the audience? Will it sit right with them instead of just going through days of identifying the data and making sense out of it and creating visuals for it. So, so that's just like a good example of every quarter or every month when we go into our risk meetings and our risk decisions, how much work has been taken away from some of my team members plate and what do they do instead with it? Which is like getting in the meetings and having active conversations about why is your roadmap this and does it include more things around security? How can we identify more areas of work around security and add it to engineering and it's roadmap, which are better discussions to have than spending time on just analyzing the data.
[34:12]
David Spark
Are we creating more problems?
[34:17]
Quote, security outcomes follow incentives, not intent. Joshua Copeland of Crescendo frames this as a security problem, but it starts upstream the compliance first. Spending he flags isn't what executives choose over resilience. It's what keeps the business running at all. Whatever survives that baseline is what goes towards actually securing the environment. He's right that it's an alignment problem, but the incentives start with regulators and insurers, not executives. So the harder question is whether you could flip the sequence and build genuine resilience first. Earning compliance as a byproduct. This would be nice. Could you get your SoC2 through actually securing the environment rather than documenting it? And I'll start with you, Kush, on this. And as long as regulators set the starting conditions, is there any realistic path to reordering where those incentives point? What do you think, Kush?
[35:14]
Kush Kashyap
I think for your first half of your question, if done right, the compliance artifact should be a byproduct of security you're already doing.
[35:23]
David Spark
I mean, that is the ideal.
[35:27]
Kush Kashyap
Yes, yes, absolutely. You have designed your amazing controls. They're operating really well. They should be producing their own evidence. So the team should not be optimizing for the audit artifact they should be identifying. Have you scoped your control to the most critical areas of your organization and are you monitoring it continuously? Are you getting the trends and the data to reflect where stuff needs to be worked upon further? But why? While I'm saying that, the fix is not to abandon audits and compliance altogether, but it's about how do you make your security posture really better and how do you make it the source of the truth for your audits and for your compliance activities? I think that's the most important thing. It shouldn't be about checklist. It shouldn't be about going through every single audit for every single geolocation. It should be about how your controls are operating that you have designed to be operating in your environment and whether that's meeting the intent of your audits or not. I think it's the other way around. But I love the point that you made about the restructuring of the order and you said that the incentives start with regulators and insurers. I think that even further, more important than that, the incentives actually start with the market. They start with the customers and the trust. Enterprise buyers and auditors gate deals on trust continuously. That's moving faster than any regulator. And it rewards the companies that are actually secure rather than the ones with the best audit workbook and the best binder is all I can say there.
[37:12]
David Spark
All right, Mike, good takes. I'm assuming you like to have a security program first that is also compliant, rather than building a compliance program and go, okay, now let's build the security program. Am I right on that thinking?
[37:29]
Mike Johnson
For sure. And I think if you were to rewind and really look through, like, move like a slider through the history of a security program, almost always they start with, hey, we need a security program. And then the need for compliance comes later. I have a hard time putting myself in a place where you start with, hey, let's go do compliance, and then let's go do security. I genuinely think that it starts with security. I know it can, to some people, feel like the opposite after the program is already up and running. And I think that's where people get frustrated is it feels like compliance is leading the program by the nose rather than the other way around. And I think that really just comes back to what Kush was describing of if you do it right, it doesn't feel that way. It feels like the compliance is a byproduct. To be fair, there's always some additional work. You always have to do some documentation. You always have to write things down. In order to have a compliant program,
[38:46]
David Spark
well, AI can start writing a lot of this stuff down for you.
[38:49]
Mike Johnson
Well, it's being written down. Maybe it is an agent that is writing it down. But it does have to be documented because the whole point of compliance is to actually have something that somebody else, a third party can review and say, yep, looks good, you're doing security. Right. And they can't get into your mind. So there has to be something, some evidence that you're showing to them. So that's always going to be there. But the magic is how do you make it feel like that is no additional work and that's what changes things.
[39:28]
David Spark
Very good point, by the way. Soon AI tool coming to you, getting into your mind. Oh, gosh, it'll happen. Count on it.
[39:39]
Mike Johnson
I feel sorry for that agent. It's going to have its own issues when it does that.
[39:44]
David Spark
This brings us to the very end of the show. So now that we realized that, it's going to be kind of a Men in Black moment where your brains are going to be taken over. I want to thank our guest, Kush Kaship with Vanta. I'm going to let you have the very last word here, Kush, but I also want to thank your company Vanta. Remember, go to vanta.com for all your compliance needs and they have iteratively figured out and they continuously iterate figure out how to speed up your entire process. Go along the journey with Vanta. Go check them out. Vanta.comciso V-A-N-T a.comciso do me a favor, add the/ciso when you go check them out. Easiest way to let them know that you heard about them from the CISO series. Mike, any last words about today's show?
[40:32]
Mike Johnson
Kush, genuinely thank you for joining us. I really appreciated you kind of walking through some of your history of your career and talking about how risk quantification 10 years ago seemed amazing. And you know, here you are still in risk management even after going that deep. So thank you for bringing that experience to the conversation, sharing your perspectives and really let us think about risk in a more holistic way. So thank you very much. It was great chatting you, awesome.
[41:01]
David Spark
And Kush, I'm tossing this to you first. We always like to ask, are you hiring at Vanta and B, anything like you would like to offer to our audience or tell us about what's new with Vanta?
[41:12]
Yes.
[41:12]
Kush Kashyap
So selfishly, I don't want to get the hiring plug in.
[41:15]
David Spark
Okay.
[41:16]
Kush Kashyap
I am building out a GRC Experts team right now at Vanta. So if you're a practitioner who thinks of GRC as being engineered instead of audit and checklist, please come find me on LinkedIn and apply to the roles.
[41:30]
David Spark
We will have a link to Khush's LinkedIn profile on the blog post for this episode. Go on. Anything else?
[41:37]
Kush Kashyap
Yeah, I mean I just want to say first of all thank you so much for having me here. I always learn so much from the CISO series and having an opportunity to have a real time connection with people like you, it's just so much thrilling and some of the best parts of my month when I get to do this and learn so much in like one sitting.
[41:57]
David Spark
We love hearing that.
[41:58]
Kush Kashyap
And the other thing that I'm also very excited about is the future of grc. I love that we are going agentic. I love that we have MCP server, we have all this data in front of us and we can analyze it in record breaking speed like we have never been able to do before. So I think my only thing to all the GRC practitioners out there is engineered the data more build tooling on top of it. We are not here to see the next thing that's getting released or the next feature that's getting released. We are here to build our unicorn, valuable GRC program reports and everything the CISO and the board deserves and do everything ourselves in an engineering way. So GRC engineering is here and hope you all make the best use of all this technology.
[42:47]
David Spark
Awesome. Thank you Kush, thank you Mike and thank you to our audience. As I always say and truly mean, we greatly appreciate your contributions. In fact, send me a lot more what's Worse Scenarios and listening to the
[43:02]
CISO Series podcast that wraps up another episode. If you haven't subscribed to the podcast, please do. We have lots more shows on our website cisoseries.com Please join us on Fridays for our live shows, Super Cyber Friday, our virtual meetup and cybersecurity Headlines Week in Review. This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved including recording a question or a comment for the show. If you're interested in sponsoring the podcast, contact David Spark directly@Davidisoseries.com thank you for listening to the CISO Series podcast.

Summary5 min read

Podcast Summary: CISO Series Podcast

Episode: See, Our Compliance Framework Includes a Checkbox for Resilience

Release Date: August 4, 2026
Host(s): David Spark, Mike Johnson (CISO of Rivian)
Guest: Kush Kashyap (Sr. Director of GRC, Vanta)
Sponsored by: Vanta

Episode Overview

The hosts and guest explore the real-world tensions between resilience, compliance, and risk in cybersecurity programs. They discuss the evolution from checklists to performance-driven security, the economic realities of AI experimentation, and how true resilience and risk management demand more than just keeping up with regulatory paperwork. They candidly dissect the limits of quantification, the burnout challenge in security teams, and whether compliance artifacts can (or should) be natural byproducts of real operational excellence.

Key Discussion Points and Insights

1. The Economics and Experimentation With AI Usage

Token Awareness and Experimentation:
Initiated with a discussion around organizations overspending on AI tokens due to a lack of “token literacy.”
- Mike emphasizes that at this stage, AI experimentation is more valuable than controlling costs (“right now is probably not the right time to be concentrating on ROI. We're very much in the experimentation phase.” – Mike, 07:25).
- Both Mike and Kush agree that users shouldn’t worry about "tokens" but should focus on AI output quality and where its use provides the most business value (09:10–09:45).
“I think more than just understanding what our usage is in terms of tokens, we are also at a point now where we can start developing taste and judgment.”
— Kush Kashyap (09:10)
Judgment Over Blanket Approaches:
AI resources should be applied sensibly—reserve expensive, powerful models for tasks with significant business or security impact, not routine work (09:10–10:40).

2. The Honest Limits of Risk Quantification

Not All Risks Are Measurable in Dollars:
The panel agrees on the inadequacy of reducing every risk to dollars/outages—brand erosion, compliance blocks, or exposure of proprietary designs may have massive impacts not immediately quantifiable (11:06–13:21).

“The most important risks are not the ones ... quantified in dollar terms, but ... around eroding customer trust, or a compliance gate that locks you out of a market potential, or a proprietary asset might be getting lost.”
— Kush Kashyap (12:46)
Storytelling Over Model Defending:
Mike notes storytelling’s importance—focusing on commitments and impacts rather than dragging executives into numerical model debates (15:50–16:15).

“If you're explaining, you're losing ... If you're spending all your time defending why this is going to cost this amount, explaining the model, you've already lost.”
— Mike Johnson (16:00)

3. The "What's Worse?" AI Scenario Game

(Segment starts ~18:50)

Scenario #1: Agentic AI acts autonomously, never seeking permission.
Scenario #2: Agentic AI asks for permission for every action ("thousands upon thousands").
Which is worse?
- Both agree: today, autonomous AI (Scenario 1) is worse due to risk, compliance, and audit trails.
- But the burden of excessive permissions (Scenario 2) could crush productivity and morale.
- This answer might change in the future as AI trust and accuracy mature.
“You can't get there [to resilient automation] when you let your agent run—even with guardrails—run loose and have no human in the loop to check what it's doing.”
— Kush Kashyap (22:46)

4. Security Team Resilience—Beyond Wellness and Burnout

Resilience ≠ Wellness Propaganda:
Kush’s “spicy take”: Framing resilience as a wellness goal is just “delayed attrition with better branding,” especially as workloads surge with AI and complexity (28:03–29:28).
- Real resilience requires genuine reduction of team burdens and intelligent process/systems design, not just wellness initiatives.
Burnout is the Opposite of Resilience:
Mike stresses that burnout threatens team resilience—throwing more vacation or more headcount at the problem isn’t a sustainable solution (26:37–27:31).
Incremental, Engineering-Led Improvements:
Both share stories of using automation and AI to move from tedious, manual security/reporting to smarter, faster processes—but warn it takes years and happens step by incremental step (30:12–32:25).

5. Compliance as a Byproduct—not Just a Checkbox

Can Resilience-Leading Programs Satisfy Compliance?
- Both agree: the gold standard is for compliance artifacts to be natural outputs of robust security controls, not the other way around (35:13–37:11).
- However, regulatory and market incentives often force compliance-first thinking, which risks relegating true resilience.
“If done right, the compliance artifact should be a byproduct of security you're already doing.”
— Kush Kashyap (35:13)

“Almost always [security programs] start with, 'Hey, we need a security program.' And then the need for compliance comes later.”
— Mike Johnson (37:29)
- The true driver of incentives? Not just regulators, but customers and the trust of enterprise buyers: “That's moving faster than any regulator.” (36:54 – Kush)
The Role of Documentation and AI:
Compliance will always require some explicit documentation (“...it does have to be documented because the whole point of compliance is ... a third party can review and say, 'Yep, looks good, you're doing security.'” – Mike, 38:49).
But AI can increasingly automate this.

Notable Quotes & Memorable Moments

On Overuse of “AI”:
“We are not the only ones that are getting fooled by businesses slapping on AI. It's trickling everywhere.”
— David Spark (04:02)
On Risk Modeling:
“If we anchor it to the commitments the CEO personally signed off on, it lands harder and better than a quantified number.”
— Kush Kashyap (13:21)
On Security Team Fatigue:
“Resilience is nothing but delayed attrition with better branding.”
— Kush Kashyap (28:03)
On Incremental Efficiency Gains:
“It did not happen overnight...you'll see incremental steps.”
— David Spark (30:56)
On Automation in Incident Response:
“That's a journey of years in order to get there.”
— Mike Johnson (31:37)

Important Segment Timestamps

| Topic | Timestamp | |---------------------------------------------|---------------| | AI Token Usage & Efficiency | 05:12–11:00 | | Risk: Beyond Dollars & Outages | 11:06–17:17 | | "What's Worse?" Agentic AI Scenarios | 18:50–25:34 | | Security Team Resilience—Traps & Solutions | 26:06–34:11 | | Compliance as a Byproduct, Not the Goal | 34:17–39:38 |

Final Takeaways & Calls to Action

For GRC and Security Leaders:
- Focus on outcome-driven security; compliance and artifact generation should flow from your actual risk/reliability investments.
- Team resilience is about reducing actual burdens, not just branding over existing churn.
- Documenting your processes and controls will always matter; make automation and tooling work for you, but don’t expect AI to be a magic fix overnight.
Guest Invitation:
- Kush is hiring GRC engineering-minded practitioners at Vanta—interested listeners are encouraged to connect on LinkedIn (41:12–41:30).
- For more insights into efficient, agent-driven compliance and risk management, Vanta remains a recommended resource.

For more engaging discussions and live shows, visit cisoseries.com. Listeners are also invited to contribute “What’s Worse?” scenarios for future episodes.

Loading summary

Transcript138 lines

[00:00]
David Spark
10 second security tip. Go.
[00:03]
Kush Kashyap
I would say stop being the tollbooth. When people reach out to you and they ask you questions about a certain control, what is the best practice? How was it designed? How does it need to be operated? Own the decision and share the decision because you'll lose credibility if you don't move fast with guardrails.
[00:24]
David Spark
It's time to begin the CISO Series podcast.
[00:27]
Foreign. Welcome to the CISO Series podcast. I am David Spark. I'm the producer of the CISO series and joining me since episode one, we're recording this actually just a few weeks after our eight year anniversary. It is Mike Johnson, the CISO of Rivian. Mike, say hello to the audience.
[00:54]
Mike Johnson
Hello, audience. It's amazing to believe it's been eight years.
[00:57]
David Spark
David, it's been eight years and you had no idea when I took you out to lunch this was going to happen at all. Neither did I, for that matter.
[01:09]
Mike Johnson
Who knew what would happen?
[01:10]
David Spark
By the way, that was money well spent on a lunch. I will say that we are available@cisoseries.com where we have four other programs that launched not too long after that first episode, which was eight years ago. Go check them all out. Our sponsor for today's episode, which has been a phenomenal sponsor of the CISO series responsible for us to continue these eight years, it is none other than Vanta. Trust is everything. Earn and prove it with Vanta. And we're going to be talking a little bit more about that today. And our guests, courtesy of Vanta. But before I do that, now, one thing that's been sort of a chronic joke in Cyber Mike is how everything is XYZ with AI. This with AI, Right, sure. Yep. Well, we're like operating our own bubble. And it isn't just security that gets XYZ with AI. So just a quick story. So I live in San Diego. I like to go to these entrepreneur meetups and meet other entrepreneurs and hear what they're doing. Now, as you might imagine, I'm in Southern California. It's San Diego. There's a lot of sort of beach people, a lot of sort of earthy characters. And, you know, there are many stores with psychics and healing and crystals and things like that. Okay. And people giving you advice on how to lead and run your life in general. Sure. Now you get that with AI. Oh, no. Yes. The number of entrepreneurs I've met who have essentially that same shtick, if you will, or however you want to describe it or however you feel about it. But now, say, for example, I met this one woman who is starting a business hasn't launched it, but the idea is she will. Look at the way you lead your life, Mike. So how you sleep, how much coffee you drink, anything else behaviorally in your life and tell you what's the best time you should schedule a meeting so you perform that at your best.
[03:17]
Mike Johnson
Wow.
[03:18]
David Spark
Yeah.
[03:19]
Mike Johnson
You know, is that a service you
[03:21]
David Spark
find yourself paying for?
[03:23]
Mike Johnson
Yes. But what's, what's interesting about that and the kind of the correlation with that is there are entire tech companies who are essentially front ends to AI. Like they're just a UI on top of ChatGPT or Claude or Gemini or what have you. And all these are going to go away.
[03:42]
David Spark
I know. And I was, I was also, I was talking to these other women who, you know, said, oh, you know, we'll help you design your business model. Well, we did it ourself. We asked AI and just type it, did it. And I go, and so what would stop anyone from doing exactly the same thing you do?
[03:58]
Mike Johnson
Exactly. That is the problem every one of these companies has.
[04:02]
David Spark
So we are not the only ones that are getting fooled by, by businesses slapping on AI. It's trickling everywhere.
[04:14]
Mike Johnson
It's good to know that there are bad ideas everywhere.
[04:16]
David Spark
Yes, there are. But you know what, here's the thing about bad ideas, I am sure, like the ones I just mentioned, I will be proven so wrong, it'll be a colossal hit. And I go, look, what a moron I was.
[04:30]
Mike Johnson
Good for them, good for them.
[04:31]
David Spark
I don't begrudge anyone from their success. But I, you know, I heard that I'm like, really? And then, and also, you know, you, you know, this like something becomes a smash success like a movie or an artist and you're like, really? Someone wants to watch that, listen to it. Guess so.
[04:47]
Mike Johnson
Yeah, there, there, there is an audience for every show.
[04:51]
David Spark
Exactly. And with that being said, this show is not infused with AI. It is actually my voice recording it and Mike's voice recording it. And my guest, who we're going to introduce in just a moment, our return guest. Thrilled to have her back with Vanta, the senior director of grc. Our sponsor guest, Kush Kashyap. Kush, thank you so much for coming on back.
[05:13]
Kush Kashyap
Oh, thank you so much, David. It's always a pleasure.
[05:18]
David Spark
Is this the best use of my money?
[05:23]
The user sees six words. The system processes a small novel. Barbara Roos of Starboard collectives argued at AI Field Day 8 that employees are burning tokens with no awareness of what they're consuming. Now we took this up on our BSIDES SF live show. You were there, Mike. And the frame shifted pretty fast. So Jensen Huang, who is the CEO of Nvidia, reportedly wants his $400k a year engineers spending 200k a year on tokens. That's the goal, which is understandable from his viewpoint, as you pointed out. So the premise is that organizations should steer employees toward token literacy before cost spiral assumes the wrong problem. Roos complains that ROI measurement and token awareness are the same. Conversation falls apart if experimentation is the ROI during early adoption. So is watching how employees use AI just micromanagement with an infrastructure budget attached, which is the way I saw it originally? Or do people actually need to know what a token is? Or do they just need access and room to run? Mike, what does this say about your AI strategy if you're managing prompt efficiency instead of outcomes? What do you think?
[06:40]
Mike Johnson
I do wonder. That quote from Jensen was quite a while ago, right?
[06:45]
David Spark
So this was February, I think.
[06:47]
Mike Johnson
It's amazing how quickly these things move these days. And I don't know, like, AI. My guess is he's changed his mind since then. You've got the case of someone at Uber saying that they had blown through their entire year's budget in a few months.
[07:03]
David Spark
There was some story that some company had like a $500 million token expense.
[07:08]
Mike Johnson
It was. I don't know, there was 500 million, but Uber's. Uber's was. Was way up there, and that's what they were talking about. But I think right now, now is probably not the right time to be concentrating on roi. We're very much in the experimentation phase.
[07:26]
David Spark
And do you say that to your staff? Because I've said it to my staff. I go, oh, yes, please feel free to experiment.
[07:32]
Mike Johnson
Yes, yes, absolutely. And we have quotas attached with our AI usage, and we manage that. And my way of looking at it is if someone on my team comes to me and says, hey, I want to spend more, I just approve it. Like, let's go, let's experiment and let's see where this is going to take us. And I think that's one of the things that companies need to think about, is the costs right now are not going to be the same runtime costs in the future.
[08:01]
David Spark
That's a good point.
[08:02]
Mike Johnson
We're going to settle down on how we use these tools. And in this day and age, how we use it today is not how we're going to use it in the future. And so I think the idea of teaching people what tokens is that doesn't make sense. They don't need to know what a token is.
[08:19]
David Spark
It's also kind of confusing.
[08:21]
Mike Johnson
It's confusing and it can also change, right?
[08:25]
David Spark
I mean it's a made up thing. It's like cryptocurrency, like who frigging knows?
[08:29]
Mike Johnson
Yes. I mean it has an underlying technical meaning, but it doesn't matter. What matters is how much you're spending and what value are you getting out of it. And right now that value is more in the R and D phase than it is in actual hard business outcomes
[08:49]
David Spark
token uses is your R and D budget. All right, Kush, I like that. What are your thoughts on this? And by the way, have you seen this trend? Because I think these stories of these ludicrous budgets being blown is shocking people. Because people like yes, we want to be aggressive, like, oh, I didn't realize how aggressive we can be. Your thoughts?
[09:10]
Kush Kashyap
Yes, totally. I think I have seen it, personally seen it in several companies I've been at as well. But one thing that I do want to mention here, which is slightly different take, is that I think more than just understanding what our usage is in terms of tokens, we are also at a point now where we can start developing taste and judgment. We should start asking ourselves if we are building something, is it going to be used by various people? Is it going to add value for the company, for the business, for the security function, is it efficient? Do we know what good looks like, which model to reach for, when to push and when to use something else? For example, you may not want to use your most expensive model on like a Slack conversation framing. You don't want to use it for drafting a Google, you know, a Gmail message, but you may want to use it when you're thinking about like a gnarly architecture or when you're thinking about creating a PRD for the next set of AI driven vulnerability management task tracker. So there are different things around taste and judgment which we should also I feel like we are at a place where we can start experimenting with quality as much as we hone in on usage. Because I don't think fixating on token efficiency right now is a good thing. We should focus on what good output looks like, what good adoption looks like. How will we make sure that what we are building is going to be useful for our company, for our business, for our security teams? Instead of wasting a lot of money and just trying to see what sticks on the wall,
[11:01]
David Spark
it's time to measure the risk.
[11:06]
Not every risk can or should be Reduced to a dollar amount or outage scenario. Now that's William McBorough, CISO at MC Global Tech, pushing back on the playbook CISOs have been handed for executive conversations. Tie everything to money downtime or data loss and you'll get the room. But he's pointing at something the standard framework glosses over the risks that shape a business for years. Eroded customer confidence, a compliance lapse that delays market entry, an insider event that exposes proprietary designs. Those do not always register on next quarter's balance sheet. So the question this piece opens up is the harder one. What exactly are those risks and how do you make them land with a CEO who hears risk and thinks cost? So I'll start with you kush on this one. How do you build a case for something you can describe but can't price? And at what point does the push to quantify everything make your risk conversations less honest because you find that you're trying to push a round peg into a square hole kind of a thing?
[12:16]
Kush Kashyap
Absolutely. I mean, I also want to throw in a caveat here. Back in 2016, I spent a whole two years doing risk quantification and selling risk quantification models to some of the largest tech companies in the world and some of the largest financial services companies in the world. But those were the same questions that were being asked then is the same question that are being asked now. How do we know it's accurate? How do we know financially it will hold up? So, I mean, having struggled with it in a very close basis, what I can say is I have realized over time that the most important risks are not the ones which are quantified in dollar terms, but they are around eroding customer trust or a compliance gate that locks you out of a market potential or a proprietary asset might be getting lost.
[13:12]
David Spark
Well, essentially, business brand damaging.
[13:15]
Kush Kashyap
Absolutely.
[13:16]
David Spark
And I would assume every business understands that their brand is their most valuable asset.
[13:22]
Kush Kashyap
Exactly. And I also think that the way you make them land with the CEO isn't that this risk is this amount of dollar. It's about the decision. If we ship something without security controls, here is the deal we can lose. Here is the market we won't be able to enter to. Here is the customer commitment and the obligation we promise to respect. We are breaking that if we anchor it to the promises and the controls the company has it made. Now we are talking about tracking those things. And a risk is tied to a specific commitment the CEO personally signed off on, and it lands harder and better than a quantified number that People are always going to question, and you don't want your discussion to be around, where did the number come from? How do we know it's accurate? What's the model behind it? What's the formula behind it?
[14:17]
David Spark
Then you're defending something completely different.
[14:19]
Kush Kashyap
Exactly. And you'll be spending your time on making your decisions better, making the people in the room accountable for the risks that they need to drive to mitigation, and you'll be spending your time on better places instead of analysis paralysis.
[14:34]
David Spark
And speaking of spending your time on something, at the beginning, I mentioned the metaphor of the round peg in the square hole, which I did it in reverse, because you can actually fit a round peg into a square hole. You can't fit a square peg into a round hole. So I don't want any letters about that, because I do know what I said, which was the complete opposite of the appropriate. This is like when people say, oh, I did a complete360, when they meant a180. So I pulled one of those. Is what I did just there, Mike.
[15:04]
Mike Johnson
Well, actually. David. Well, actually, you just need a smaller peg, and it'll fit just fine.
[15:09]
David Spark
It'll fit fine. You know what? Yeah, if you're. No, but if I have a very large round peg trying to fit into a small square hole, I can't do that.
[15:18]
Mike Johnson
You just need a bigger hammer is what it comes down to.
[15:20]
David Spark
All right, But I'm sure, by the way, Mike, you have been a CISO for a long time. You've had to describe risk God knows how many different ways. I'm sure you've beaten your head over. How am I going to describe this? How are we going to prove this? How are we going to explain this? I mean, let me ask this. Look at your first year as a ciso, and now, what is the thing that you know about risk now and sort of quantifying and explaining it that you did not know that first year?
[15:51]
Mike Johnson
I think a lot of it is a realization that so much of our job is about storytelling. And I have long wanted to put numbers to everything. I'm an engineer by training, and so I always wanted to put numbers on things. And risk quantification seemed like this amazing opportunity to do that. And what I've come to realize is it is but one tool in our quiver, and it's not necessarily going to be the best one to tell the story to convince folks that you need to go in a direction. And one of the things that I was, you know, as I was listening to Kush, there's the quote that says if you're explaining, you're losing and if you're spending all of your time defending why this is going to cost this amount, explaining the model you've already lost and the story is not going the way that you want it to. So I really think the biggest thing that I've learned is so much of it is about being able to tell a story that the point that you're trying to get across is what people come away with, not well. Why did you name the character that you don't want them going down off script in a direction that is totally tangential from the point that you're trying to make?
[17:18]
David Spark
What's the one thing in business that's spreading as fast as AI? AI risk. Every new tool your team signs up for, every vendor that turns on AI features, every new integration, each one is an opportunity for something to go wrong. And most security programs weren't built for AI's pace of growth. I mean, geez, nothing was. But this is where Vanta enters. Vanta is the number one agentic trust platform used by over 16,000 fast moving companies like Ramp, Cursor and Harvey to ensure they're always audit ready. And now Vanta is helping companies like yours watch for the risks that show up between audits across your vendors, your AI tools and your whole environment. Now how do they do this? Well, the Vanta agent works like a 24.7grc engineer in the background finding issues, drafting fixes for you and cutting vendor assessment time by up to 50%. Now whether you're a fast growing startup or a global enterprise, Vanta is here to help you automate your security and compliance and earn improve trust. That is important. So get started today@vanta.com CISO and it's spelled V A N T A.com CISO so vanta.com CISO do me a favor, add that. CISO. Easiest way to let Vanta know that you learned about them from the CISO series.
[18:51]
It's time to play what's worse
[18:56]
Kush. You've played last time. Now do you remember if you agreed or disagreed with our guest last time we played?
[19:02]
Kush Kashyap
I think I agreed on a couple and I disagreed on at least two.
[19:07]
David Spark
Okay, well, there'd be issues I guess, because there's only issues.
[19:10]
Kush Kashyap
Yes.
[19:11]
David Spark
We're only going to play one round of this, but I'm going to make Mike answer first.
[19:14]
Kush Kashyap
Yes.
[19:14]
David Spark
And since we've been talking about AI here, guess what this one has to do with AI.
[19:19]
Mike Johnson
Great. Maybe I'll Ask AI the answer.
[19:22]
David Spark
You know, we could. I never thought about that. I mean, I get my two guests and go, by the way, this is what AI Said.
[19:27]
Mike Johnson
Yeah.
[19:28]
David Spark
This comes from one of our listeners who sent. Who has sent us many wonderful what's worse scenarios. It is Dustin Sachs of Cybercog Labs. And here you go, scenario number one. Your agentic AI starts taking all actions without your permission. Okay, everything. Or the complete opposite. Your agentic AI asks for permission on every single action. We're talking thousands upon thousands of actions. So which one is worse? Mike?
[20:01]
Mike Johnson
It's funny that as we're recording this, literally yesterday I was doing something with AI and I had to just keep hitting the inner key over and over and over again.
[20:11]
David Spark
You know what this reminds me of? Did you ever see the TV series Lost where like X number of hours, they had to just press a button. They didn't have a clue what the button was doing, but they had to do it because if it didn't happen, something horrible would happen.
[20:24]
Mike Johnson
That's a different perspective than what I remember is when Homer Simpson had the dunking duck that he put over the button at work. So I like your callback better than mine. It adds more urgency to it. So as I think about these two scenarios, it's like you've got one which is like default open versus default closed. Right. Like you've got. It'll just do anything, everything. You can't stop it.
[20:52]
David Spark
Essentially, there are problems on both sides of this.
[20:55]
Mike Johnson
Exactly.
[20:55]
David Spark
And let me qualify what Dustin says, which I think he sets it up. Nice. One is the goal. I mean, the goal is the first scenario. It is kind of. But it's problematic today. And the second is safer, but it completely defeats the purpose of a gentic AI. It actually makes your life far, far more miserable.
[21:14]
Mike Johnson
Well, I mean, he just answered with that. Right.
[21:17]
David Spark
But which one's worse?
[21:18]
Mike Johnson
Well, I think the point is today, which, again, the scenario, we don't get to change things. The point is today, the one that hasn't really gained the trust to operate fully autonomously, that is the one that's the bigger risk. The second one is a user experience nightmare. Like, that's terrible, but that's going to.
[21:45]
David Spark
Okay. And then back up a step here, Mike. Both are very damaging to the business.
[21:52]
Mike Johnson
Again, the point is they're both terrible.
[21:54]
David Spark
Yes.
[21:55]
Mike Johnson
Just in different ways. And that's usually the point of these is. Yeah, because if it was easy, if, like, one of them wasn't terrible, then, well, problem solved.
[22:05]
David Spark
That sometimes happens that I'm not aware of it.
[22:09]
Mike Johnson
But I really think today maybe we dust this one off again in a year and see what the answer is. But I don't think that we've quite reached a comfort level with agents that we can just let them just run completely unsupervised and just do whatever the heck that they want. So I think today the first one is the worst.
[22:30]
David Spark
But your answer may change a year or two years from now.
[22:32]
Mike Johnson
Frankly, I hope it changes in the future.
[22:36]
David Spark
Well, again, like Dustin said, the first one is the goal.
[22:41]
Mike Johnson
Absolutely.
[22:42]
David Spark
It's totally the goal. All right, Kush, do you agree or disagree with Mike here?
[22:46]
Kush Kashyap
Gosh, it's so hard to disagree on it. Like there is a portion of my team who is in charge of making sure that the agents we ship have human on the loop. Quality assessment, everything done. Accuracy is high. You can't get there. When you let your agent run even with guardrails, run loose and have no human in the loop to check what it's doing, you don't want to make any material changes on what's going to impact your audit later when your auditor will ask you who made this change? Why was this change made? Show me the history. Walk me through it. What are we going to say? So that's a nightmare. Yes, it's inconvenient to keep clicking. Yes, allow. Yes, allow. But I would rather have the inconvenience than like a poorly drafted design control.
[23:35]
David Spark
Also, it's going to bring your employees. First of all, it's going to bring your employees productivity to a screeching halt. It will let me throw some other things. I'm playing devil's advocate here. How many people are going to be enjoying their job, doing this all day, not getting access to anything else? You may lose staff. I'm just. There's a scenario that go wider than this, but then they throw this else out is we know that if you let AI run rampant, it's going to be a matter of very little time before you know what starts hitting the wall. Give you example, Mike. When I was working at zdtv, the television cable network which later became known as tech tv for one of our shows called the Screensavers, we had a, a computer called the virus computer that was purposely set up with no protection on it whatsoever. And the, the joke was how quickly could we get a virus? How quickly could this be? And it would sometimes be minutes. You know, it'd be like the speed was. So I think the same thing would happen here, Kush as well. It's like, well, let's let's give this a whirl. How. What's the speed? We're gonna have problems because we know it's not like it's gonna run for months. And it's. It's gonna be. We're gonna be talking minutes or a day at max.
[24:49]
Kush Kashyap
Yes, I think so.
[24:52]
David Spark
It's gonna be pretty fast. But also, who knows, it couldn't. Maybe it couldn't be that damaging.
[24:58]
Mike Johnson
Maybe it'll fix itself.
[25:00]
David Spark
I like that idea. Oh, Mike, you are very, very optimistic.
[25:07]
Mike Johnson
Aren't you ever the optimist, David? Ever the optimist.
[25:10]
David Spark
Very. I'm sorry. So I cut you off because. Any last thoughts on this?
[25:14]
Kush Kashyap
No, I actually do agree with Mike on this one. Sorry, David. But I have to say I hope we can revisit this answer in a year and hopefully our answer will change and we will be able to trust the models to make decisions and to carry out actions without losing sleep over them and without losing the trust of our stakeholders.
[25:35]
David Spark
I think. I argue that the second one could be worse because you're going to. The employees are going to be so annoyed, so irritated, they're going to learn nothing, and they're going to be very disturbed, and they may leave the company. Although I think in the first scenario, a company will fall apart before that could happen.
[25:57]
Kush Kashyap
Exactly.
[26:02]
David Spark
Please, enough. No more.
[26:07]
All right, today's topic, Kush, Mike, is security team resilience. I'm gonna start with you, Mike, on this one. What have you heard enough about with security team resilience? And by the way, this whole discussion of security shifted to the term resilience a long time ago, like, a few years ago, we started talking more about resilience than just, quote, security, because we all realized that 100% was impossible. So what have you heard enough about with security team resilience? And what would you like to hear a lot more about?
[26:37]
Mike Johnson
So when I hear the term security team resilience, I mean, like the humans, I hear the resilience of the team. And so when I hear that, I go kind of the opposite side of this. Like, what is the opposite of security team resilience? And to me, that's burnout. That is where you have folks who are just working too hard. They will just eventually crack and they'll leave, to our last discussion, deliver subpar work. It just. It's a bad outcome. And I think we've been trying to solve that with either, oh, well, we'll just give them more vacation or we'll hire more people. And I think that's really the challenge that I see and what I've heard too much of is those are the wrong ways to manage burnout and drive towards a resilient team. So what I'd like to hear more is frankly, how do you do more with the team that you have and actually make them more resilient rather than just looking at it as oh well, I'll just hire my way out of this.
[27:45]
David Spark
Very few people can pull that off, by the way. I don't think I've ever heard success on that. All right Kush, I will ask you the same question. What have you heard enough about and what would you like to hear a lot more about with regards to security team resilience?
[28:01]
Kush Kashyap
Yeah, I have a spicy take on this.
[28:03]
David Spark
Let's hear it.
[28:04]
Kush Kashyap
So in terms of what I've heard enough about is the whole resilience as like a wellness culture vibe in companies, right? Especially when it comes to this.
[28:17]
David Spark
Welcome to San Diego by the way. Come here. We can help you with all of that and put little AI on it too while you're at it.
[28:26]
Kush Kashyap
I can't wait to visit. But that being said, I do see resilience as nothing but delayed attrition with better branding. Because what are we doing to the folks on our team by saying that absorb more with the same headcount we are essentially saying double the workload, triple the workload thanks to AI. Which is not. Which is still. We are still figuring it out what does good look like, what tools can we build where we can really depend upon it, solely where we can't. What I would like to see more of is creating better structures, creating better processes and yes, scaling, but scaling in a manner that's not going to burn out people and actually taking work away from their plate when they are doing these projects, when they are trying to scale and trying to put the processes in taking away some of the work so that the team is not always running on like 100 mile per hour with creating single point of failure and creating a risk for the business, for the security of the organization and the way everyone is operating. So I think that's really important because the honest conversation here is the workload has exploded even when we are adding more to our plate thanks to AI we have so many more non human identities to manage. There is an agentic sprawl. CISOs cannot say no as much as they could do before to vendors and to especially to AI models and AI vendors. The complexity curve is real as well. So we can't always keep matching that entire Curve and all that work with just efficiency. We need to have a better, more moderate conversation about it.
[30:12]
David Spark
We at the CISO series have found that we find these single projects at a time and are looking for ways to get AI to help us make it more efficient. I'll take a perfect example. If you've seen on our CISO series page, we always post these quotes, like from this episode, we'll pull three quotes from you and from Mike and we'll create this little meme image and we'll post it. This was done very manually and it took upwards. We would do six to eight episodes and creating all those quotes that could take us days to do. Days. We're now down to a few hours.
[30:56]
Kush Kashyap
Wow.
[30:57]
David Spark
But it did not happen overnight by any stretch. And it didn't go from days to a couple hours. It went from days down to a day and a half, down to a day, then to a number of hours. We kept finding these things. And I think that's where we have to sort of accept that's how the discovery efficiency will happen, is that it will not happen overnight, but you'll see incremental steps. And can you give me an example, either one of you, of something that you've done incremental steps and gotten from a thing that took days or a week down to maybe even hours? I don't know. Can you give me an example?
[31:38]
Mike Johnson
I think the best example is we have over time built all of our incident response playbooks and they were designed to be run by humans and they starts off with very manual. You figure it out, you eventually write down the process, you improve the process, and now we're able to turn some of those over to agents to actually operate. And so that, I mean, that's a journey of years in order to get there. So that's the first one that comes into my mind.
[32:05]
David Spark
And ours took years too, for that matter. Kush. I mean, this is kind of the bailiwick for Vanta. I mean, this is what you've been doing. And by the way, there has been AI infused in Vanta for a few years now. I'm sure what you guys had two years ago, a year ago, a few months ago, and today, completely different.
[32:25]
Kush Kashyap
Yes, yes, completely. A good example that I'll say is when we first released our MCP server capability, we used to take a look at how can we use our MCP server to build on top of anta, because there were certain nuances or certain reports that we wanted that don't come out of the box. Like maybe Our CISO wants to see something which is very risk driven in certain area of the business and very contractual obligation driven in another area of the business. How can we create these nuanced reports? It used to take days and sometimes weeks of review and making sure the messaging gets correct. But now the first draft, thanks to the MCP server and getting all that data in and drawing that analysis from all the different types of data elements that we have, from trust, from risk, from governance, from policies, from how our automated tests are doing, all these different signals come together and create the first draft of the report. So then we are thinking about is the story right? Who is the audience? Will it sit right with them instead of just going through days of identifying the data and making sense out of it and creating visuals for it. So, so that's just like a good example of every quarter or every month when we go into our risk meetings and our risk decisions, how much work has been taken away from some of my team members plate and what do they do instead with it? Which is like getting in the meetings and having active conversations about why is your roadmap this and does it include more things around security? How can we identify more areas of work around security and add it to engineering and it's roadmap, which are better discussions to have than spending time on just analyzing the data.
[34:12]
David Spark
Are we creating more problems?
[34:17]
Quote, security outcomes follow incentives, not intent. Joshua Copeland of Crescendo frames this as a security problem, but it starts upstream the compliance first. Spending he flags isn't what executives choose over resilience. It's what keeps the business running at all. Whatever survives that baseline is what goes towards actually securing the environment. He's right that it's an alignment problem, but the incentives start with regulators and insurers, not executives. So the harder question is whether you could flip the sequence and build genuine resilience first. Earning compliance as a byproduct. This would be nice. Could you get your SoC2 through actually securing the environment rather than documenting it? And I'll start with you, Kush, on this. And as long as regulators set the starting conditions, is there any realistic path to reordering where those incentives point? What do you think, Kush?
[35:14]
Kush Kashyap
I think for your first half of your question, if done right, the compliance artifact should be a byproduct of security you're already doing.
[35:23]
David Spark
I mean, that is the ideal.
[35:27]
Kush Kashyap
Yes, yes, absolutely. You have designed your amazing controls. They're operating really well. They should be producing their own evidence. So the team should not be optimizing for the audit artifact they should be identifying. Have you scoped your control to the most critical areas of your organization and are you monitoring it continuously? Are you getting the trends and the data to reflect where stuff needs to be worked upon further? But why? While I'm saying that, the fix is not to abandon audits and compliance altogether, but it's about how do you make your security posture really better and how do you make it the source of the truth for your audits and for your compliance activities? I think that's the most important thing. It shouldn't be about checklist. It shouldn't be about going through every single audit for every single geolocation. It should be about how your controls are operating that you have designed to be operating in your environment and whether that's meeting the intent of your audits or not. I think it's the other way around. But I love the point that you made about the restructuring of the order and you said that the incentives start with regulators and insurers. I think that even further, more important than that, the incentives actually start with the market. They start with the customers and the trust. Enterprise buyers and auditors gate deals on trust continuously. That's moving faster than any regulator. And it rewards the companies that are actually secure rather than the ones with the best audit workbook and the best binder is all I can say there.
[37:12]
David Spark
All right, Mike, good takes. I'm assuming you like to have a security program first that is also compliant, rather than building a compliance program and go, okay, now let's build the security program. Am I right on that thinking?
[37:29]
Mike Johnson
For sure. And I think if you were to rewind and really look through, like, move like a slider through the history of a security program, almost always they start with, hey, we need a security program. And then the need for compliance comes later. I have a hard time putting myself in a place where you start with, hey, let's go do compliance, and then let's go do security. I genuinely think that it starts with security. I know it can, to some people, feel like the opposite after the program is already up and running. And I think that's where people get frustrated is it feels like compliance is leading the program by the nose rather than the other way around. And I think that really just comes back to what Kush was describing of if you do it right, it doesn't feel that way. It feels like the compliance is a byproduct. To be fair, there's always some additional work. You always have to do some documentation. You always have to write things down. In order to have a compliant program,
[38:46]
David Spark
well, AI can start writing a lot of this stuff down for you.
[38:49]
Mike Johnson
Well, it's being written down. Maybe it is an agent that is writing it down. But it does have to be documented because the whole point of compliance is to actually have something that somebody else, a third party can review and say, yep, looks good, you're doing security. Right. And they can't get into your mind. So there has to be something, some evidence that you're showing to them. So that's always going to be there. But the magic is how do you make it feel like that is no additional work and that's what changes things.
[39:28]
David Spark
Very good point, by the way. Soon AI tool coming to you, getting into your mind. Oh, gosh, it'll happen. Count on it.
[39:39]
Mike Johnson
I feel sorry for that agent. It's going to have its own issues when it does that.
[39:44]
David Spark
This brings us to the very end of the show. So now that we realized that, it's going to be kind of a Men in Black moment where your brains are going to be taken over. I want to thank our guest, Kush Kaship with Vanta. I'm going to let you have the very last word here, Kush, but I also want to thank your company Vanta. Remember, go to vanta.com for all your compliance needs and they have iteratively figured out and they continuously iterate figure out how to speed up your entire process. Go along the journey with Vanta. Go check them out. Vanta.comciso V-A-N-T a.comciso do me a favor, add the/ciso when you go check them out. Easiest way to let them know that you heard about them from the CISO series. Mike, any last words about today's show?
[40:32]
Mike Johnson
Kush, genuinely thank you for joining us. I really appreciated you kind of walking through some of your history of your career and talking about how risk quantification 10 years ago seemed amazing. And you know, here you are still in risk management even after going that deep. So thank you for bringing that experience to the conversation, sharing your perspectives and really let us think about risk in a more holistic way. So thank you very much. It was great chatting you, awesome.
[41:01]
David Spark
And Kush, I'm tossing this to you first. We always like to ask, are you hiring at Vanta and B, anything like you would like to offer to our audience or tell us about what's new with Vanta?
[41:12]
Yes.
[41:12]
Kush Kashyap
So selfishly, I don't want to get the hiring plug in.
[41:15]
David Spark
Okay.
[41:16]
Kush Kashyap
I am building out a GRC Experts team right now at Vanta. So if you're a practitioner who thinks of GRC as being engineered instead of audit and checklist, please come find me on LinkedIn and apply to the roles.
[41:30]
David Spark
We will have a link to Khush's LinkedIn profile on the blog post for this episode. Go on. Anything else?
[41:37]
Kush Kashyap
Yeah, I mean I just want to say first of all thank you so much for having me here. I always learn so much from the CISO series and having an opportunity to have a real time connection with people like you, it's just so much thrilling and some of the best parts of my month when I get to do this and learn so much in like one sitting.
[41:57]
David Spark
We love hearing that.
[41:58]
Kush Kashyap
And the other thing that I'm also very excited about is the future of grc. I love that we are going agentic. I love that we have MCP server, we have all this data in front of us and we can analyze it in record breaking speed like we have never been able to do before. So I think my only thing to all the GRC practitioners out there is engineered the data more build tooling on top of it. We are not here to see the next thing that's getting released or the next feature that's getting released. We are here to build our unicorn, valuable GRC program reports and everything the CISO and the board deserves and do everything ourselves in an engineering way. So GRC engineering is here and hope you all make the best use of all this technology.
[42:47]
David Spark
Awesome. Thank you Kush, thank you Mike and thank you to our audience. As I always say and truly mean, we greatly appreciate your contributions. In fact, send me a lot more what's Worse Scenarios and listening to the
[43:02]
CISO Series podcast that wraps up another episode. If you haven't subscribed to the podcast, please do. We have lots more shows on our website cisoseries.com Please join us on Fridays for our live shows, Super Cyber Friday, our virtual meetup and cybersecurity Headlines Week in Review. This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved including recording a question or a comment for the show. If you're interested in sponsoring the podcast, contact David Spark directly@Davidisoseries.com thank you for listening to the CISO Series podcast.