If We Can't Do Better, at Least Do It Faster

Summary

CISO Series Podcast

Episode: If We Can't Do Better, at Least Do It Faster

Hosts: David Spark, Andy Ellis
Guest: Vikas Mahajan (VP and CISO, American Red Cross)
Date: February 24, 2026

---

Episode Overview

This episode explores the tension between compliance and real security outcomes, especially around third-party risk management, the reliability (or theater) of security policies, and the realities of security operations. The hosts and guest candidly discuss how the industry often prioritizes speed and box-checking over lasting improvements, and debate when it makes sense to build security functions in-house vs. outsourcing. The conversations are lively, honest, and peppered with memorable analogies and frank career advice for CISOs.

---

Key Discussion Points & Insights

1. Top Advice for CISOs: Build Relationships

[00:02]

• Vikas Mahajan: "My best advice for a CISO is make friends, not enemies. You must learn to get along with your peers… and with key executives. They are the ones who manage risk and need to understand cyber risk."

---

2. Dealing with Basic Privacy & Security Failures

[01:24-04:26]

• Discussion on encountering casual mishandling of sensitive data (e.g., publishing birthdates in public fields).
• The hosts highlight how many organizations lack in-house security talent, leading to "ew" moments for practitioners.
  • Andy Ellis: "They didn't even name the field. Come on. At least name the field 'birth date.'" ([02:29])
• The issue is often not malice but ignorance or carelessness.
• A core point: distinguishing between accidental exposure to a community vs. outright public posting — and the importance of understanding the audience and purpose of data sharing.
• David Spark: "Exposing them will make a problem for me... I just want to fix the problem." ([04:16])

---

3. Third-Party Risk Management: Compliance Theater or Real Security?

[04:53-10:43]

• Ross Young's Quote: "Being busy is the new stupid. No one cares how many hours you logged... they care about outcomes." ([04:53])
• Discussion on how most organizations treat third-party risk management (TPRM) as compliance theater, focusing on forms and reports (e.g., SoC2, CAIQ) rather than risk reduction.
• Andy Ellis:
  • Critiques the proposal of outcome-driven vendor contracts (e.g., paying bonuses for meeting patching SLAs) as unworkable.
  • "Absolutely not… This proposal doesn't reduce what you pay me, it's going to increase what you pay me." ([05:57], [08:50])
  • Vendor-side reality: filling out questionnaires is cheap and automated; meeting custom SLAs is costly and complex.
• Vikas Mahajan:
  • Also disagrees with incentivizing SLA compliance.
  • "We do not manage our third party vendors' environments. We will never know the truth of what's in there… I care more about them being able to get back online… than anything else." ([09:02])
  • Suggests a better path is focusing on vendor resiliency, and actively involving critical suppliers in tabletop exercises to build joint response plans for incidents.

---

4. Are We Just Pretending Security Is Better?

[10:49-17:43]

• Joshua Copeland's Unpopular Opinion: "Cybersecurity didn't get better. We just got better at pretending… We built an industry around looking secure instead of being secure."
• Vikas Mahajan:
  • Agrees that focusing on fundamentals is crucial (patching, vulnerability management) but defends progress: "A lot of security these days is things that our users don't see… I think overall it is [making things better]."
• Andy Ellis:
  • Strongly disagrees with Copeland:
    • "Security has gotten way better since 1999… our environments are functionally secure and we have built a trillion dollar economy… and it's mostly fine." ([14:17]/[16:57])
  • Provides a historical perspective, describing how much more primitive and insecure everything was in the 1990s. Routine secure updates and encryption were rare.
  • Vikas Mahajan: "My EV gets its updates from Tesla automatically. My car, right, I'm driving on this thing… how different a world it is today." ([17:34])

---

5. Policy vs. Control: What’s Worse— Toothless Policy or Minimal Control?

[19:11-28:41]

• Game: What's Worse?
• Two scenarios debated:
  1. No enterprise AI strategy; only a basic, likely-ignored acceptable use policy (AUP); organic, uncontrolled shadow AI usage.
  2. No broad AI strategy or AUP, but strong technical controls for a single AI tool (Claude); users can enter anything, but usage is monitored.
• Andy Ellis:
  • Scenario 1 is worse: "All you have is a policy nobody reads. That's the wild west."
  • Legal protection is less than people think: "If the person who signed the policy violates it, he didn't mean anybody to follow it…"
  • Advocates for implementing controls and practices first, writing policy last: “Too many security practitioners think [writing policy] is the first thing you should do. It’s actually the last.” ([25:38])
• Vikas Mahajan:
  • Agrees: "From a risk perspective… you have a controllable environment [in scenario 2]… you know where it all is."
  • Shares that showing users responsible use of tools is more effective than blocking everything.

---

6. Policy Writing Philosophy

[26:32-27:16]

• Policies should be the last layer, reflecting already-established good practices, not aspirational or academic standards nobody follows.
• Andy Ellis:
  • "When most people write policies, it's an exercise in writing the dogma of where we believe we ought to be…"

---

7. Building a SOC: In-House or Outsourced?

[28:46-33:58]

• Starting a Security Operations Center (SOC): Most subreddit commenters say outsource unless you’re a very large enterprise.
• Vikas Mahajan:
  • Agrees: "It's better to procure the service… They've already built all the rules."
  • Recommends a hybrid approach: keep data and investigation access internal, outsource the monitoring function.
• Andy Ellis:
  • Highlights the need for organizational maturity: "Who is your tier 4 engineer? If you don’t have one… you’re not ready.”
  • Outsourcing works only if you have someone senior enough internally to manage the MSSP relationship.
  • SOC and IT helpdesk should be tightly integrated; don't build an entirely parallel operation.
• Vikas Mahajan: "You do have to manage the relationship and that’s the key. If you want good service, you have to put in the time and effort."

---

8. Shift Left: Is It Still Valuable or Just Buzzword?

[34:04-39:01]

• Derek Fisher's Quote: "'Shift left' has become a cringe-inducing or eye-rolling phrase… but the goal is to be proactive."
• Fisher suggests using Adam Shostack’s lightweight four-question threat modeling at the user story level.
• Andy Ellis:
  • Shift left is overused, but the core practice—engaging in dialogue early, understanding system goals and threats—remains powerful.
  • "Instead [of coming in with demands], walk in and say: what are you doing? How could it go wrong?"
  • Developers are more receptive when respected as owners and experts, not as policy targets.
• Vikas Mahajan:
  • Agrees context is crucial: "Bring the water to the horse — bring capabilities to the people, embed security in their tools and train them."
  • Security should be a partner, not "the OWASP top 10 thrown at you."
• Consensus: Lightweight, conversational threat modeling is better than box-checking or prescriptive policies.

---

Notable Quotes & Moments

• Vikas Mahajan, on CISOs:
  • “Make friends, not enemies. You must learn to get along with your peers... They are critical for you to be able to get the security things you want done.” ([00:02])
• Andy Ellis, on TPRM:
  • “Absolutely not... This proposal [pay-for-patching SLA] doesn't reduce what you pay me, it's going to increase what you pay me. That's why it's dead in the water.” ([05:57]/[08:50])
• Andy Ellis, on policy writing:
  • “The last thing you should do is write a policy… the first thing that you do is you go and you secure a system… when you finally write a policy, it’s already being followed." ([25:38])
• Andy Ellis, on history:
  • “Security has gotten way better since 1999. If I go back to where I was… nothing was encrypted… the amount of just insecure by default… software was through the roof.” ([14:17])
• Vikas Mahajan:
  • “We will never know the truth of what’s in [vendors’] environments… I would rather focus on resiliency.” ([09:02])
• Andy Ellis, on shift left:
  • “When a security team walks in and says, 'you need to do X'… and developers don't see the context, they’ll stop listening to you for everything.” ([36:25])
• Vikas Mahajan, on shift left:
  • “Bring the water to the horse, bring the security capabilities to the people, embed it in the tools they’re working with… security as a trusted advisor.” ([37:55])

---

Timestamps for Important Segments

• [00:02] CISO advice: "Make friends, not enemies"
• [04:53] Compliance Theater in Third-Party Risk (Ross Young quote)
• [05:57]/[08:50] Andy Ellis: Why outcome-based vendor contracts won't work
• [09:02] Vikas Mahajan: Focus on resiliency, not patch-based incentives
• [10:49] Joshua Copeland: "We just got better at pretending"
• [14:17] Andy Ellis: IT security is vastly better than in 1999
• [19:11] Game: What's Worse? (AI policy vs control)
• [25:38] Policy writing philosophy: last not first
• [28:46] Building a SOC: in-house vs outsource
• [34:04] Shift left: Diluted buzzword or valuable practice?
• [36:25] Andy Ellis: Why developer-first threat modeling works
• [37:55] Vikas Mahajan: Embed security in dev workflows

---

Takeaways

• Real Security Demands Relationships and Context: Technology and policies provide little value unless grounded in organizational context and sustained by trusted relationships.
• TPRM and Compliance Are Ripe for Reform: Box-checking doesn’t protect anyone; focus on resilience and active collaboration with vendors.
• Security is Better, But Fundamentally Hard: The basics still matter, but core technologies and hygiene have improved, even as attack surfaces have grown.
• Don’t Lead With Policy: Build working practices first, write policies to codify, not mandate, them.
• When in Doubt, Outsource (But Don’t Abdicate): For SOCs and other key operations, buy expertise, but keep internal ownership and oversight.
• Security Engagement Should Be Developer-Centric: Avoid “checklist” security; embed guidance in developer tools, and respect their expertise.

---

Episode tone: Candid, practical, often irreverent and sprinkled with industry war stories. The conversation focuses on real-world experience, genuine skepticism, and constructive advice for security leaders.

---

For more CISO Series content, visit CISOseries.com

Transcript

[Host]

Best advice for a ciso, go.

[Vikas Mahajan]

My best advice for a CISO is make friends, not enemies. You must learn to get along with your peers. They are critical for you to be able to get the security things you want done. And it's also important for you to make friends of your key executives. They are the ones who manage risk and they're the ones who need to understand cyber risk.

[Host]

It's time to begin the CISO Series podcast Foreign.

[David Spark]

Welcome to the CISO Series podcast. My name is David Spark. I'm the producer of the Seeso series and joining me as my co host, it's one of your favorites. It's Andy Ellis, principal over at Duha. Andy, say hello to the audience.

[Andy Ellis]

Shub do peher ya apduniya menke hen hen iske adharpar sub prapat Shub sandhya ya shub ratri.

[David Spark]

What language was that?

[Andy Ellis]

That was Hindi.

[David Spark]

Well, you see how little Hindi I know, which is zero for those of you just tuning in every now and then. Andy likes to say a welcome to wherever you are listening to us in whatever time zone you are in. But the problem is only certain people can understand it. Although you could take that, throw it into AI and say, translate that for me in English and you could find out.

[Andy Ellis]

You could.

[David Spark]

Or I could just tell you what he said. It's not that exciting.

[Andy Ellis]

It's very exciting.

[David Spark]

We're available [e-mail] if you haven't checked out our other programming. We have four other shows. Why not go check them out? Lots of great shows that deliver wonderful things for your cybersecurity knowledge and entertainment. Our sponsor for today's episode is Adaptive Security. Security Awareness. Employees love more about that a little bit later in the show. But first, Andy, I want to talk to you about something. And I'm sure you've had this happen either with some organizations that you joined, schools that your kids are with, that you see privacy or security violations that make you go ew. And you realize they don't have the talent in house to deal with that. And so I saw yet another one, one where essentially birth dates of all participants were published very publicly to the day and year I just sent. And by the way, it was in. The information was in custom field number two, which could have easily been removed.

[Andy Ellis]

They didn't even name the field. Come on. They didn't even name the field. At least name the field. Birth date.

[David Spark]

No, custom field number two. I just sent to say, hey, you have information about our age. You don't need to publish Our birth dates. Could you please remove this?

[Andy Ellis]

So is it published just within. To the whole community, or is it public?

[David Spark]

Oh, it's easy to find publicly. Yeah. I mean, it was to the community, but it's pretty easy to find publicly.

[Andy Ellis]

Yeah.

[David Spark]

I just said, can you remove this, please? And I think this is just a case of they didn't do it maliciously. It's like, well, we just did it. We didn't know what was going on.

[Andy Ellis]

Well, I think there's two different issues then playing here. Right. So one is like, did they have other data in this document that was also privacy?

[David Spark]

Yes. I think what they just said is publish everything. Most of it is perfectly fine to publish, but it was one of the things like, oh, these two fields. Maybe we shouldn't have published these kind of a thing. Yeah.

[Andy Ellis]

Within a community, you have this challenge, which is you kind of want to publish everybody's birthdays to the community so that people know and can wish each other happy birthday.

[David Spark]

But also, you don't need to put the year down either.

[Andy Ellis]

Well, the years are kind of obvious if it's school kids.

[David Spark]

Well, school kids. No, but I'm talking. Oh, this was all ages, kids to adults, the whole nine yards.

[Andy Ellis]

Yeah. I mean, so I try to try to figure out what could they use this for positively.

[David Spark]

And.

[Andy Ellis]

And then is the problem that they made it public or is the problem that they let the community see it?

[David Spark]

It's public is the issue.

[Andy Ellis]

And often we combine those two and say, oh, you shouldn't have published that at all. When it's, no, you shouldn't have published this publicly, even just the members of the community. You shouldn't be publishing our names.

[David Spark]

Well, I have another issue with another organization, and they're at the point of ghosting me, which I'm like. Just like, oh, God. But their offices are a few blocks from my house, and I think I just have to knock on the door is what I need to do.

[Andy Ellis]

That would be funny. You should bring a camera.

[David Spark]

I'm not gonna do that. Exposing them will make a problem for me. So I don't. I just wanna fix the problem is what I wanna do.

[Andy Ellis]

There you go.

[David Spark]

All right, let's get to our discussion at hand. And actually, a new guest we've yet to have on the show, thrilled we're getting him on it, is the VP and CISO over at the American Red Cross, Vikas Mahajan Vikas. Thank you so much for joining us,

[Vikas Mahajan]

David, Andy, it's an honor to be here. Thank you. Namaste. By the Way and pleasure to be here.

[Host]

Are we creating more problems?

[David Spark]

Quote, being busy is the new stupid. No one cares how many hours you logged or how many documents you churned out. They care about outcomes, end quote. That's CISO tradecraft host Ross Young's take on why 33rd party risk management has become compliance theater. Most breach companies could have produced clean SoC2 reports and completed consensus assessment initiative questionnaires. Or is it pronounced cake? C A I Q, C A I Q.

[Andy Ellis]

But yeah, I think you can go with cake.

[David Spark]

Cake. Which means the entire enterprise was box checking, not risk reduction. He advocated for outcome driven vendor contracts with actual incentives, but pay vendors a base rate plus monthly bonuses if they meet your patching SLAs. It turns vulnerability management into a market tested system where vendors are motivated to actually fix issues fast instead of just filling out questionnaires. It's an interesting theory that I like that Ross proposed. Andy, could this work or is this dead on the water? What do you think?

[Andy Ellis]

Absolutely not.

[David Spark]

Okay.

[Andy Ellis]

And I don't usually say that. I love Ross.

[David Spark]

Ross is great.

[Andy Ellis]

Shout out to Ross.

[David Spark]

And by the way, I like him poking the bear here on this one.

[Andy Ellis]

He has correctly put his finger on a problem which has been since day one. Like TPRM is this. Are you big enough to fill out my questionnaire test? And that's all that it does. And I actually have had this before. I had a team that worked under the CIO that did vendor assessments. The team was complaining that half of their people were now doing assessments and reviewing all of these. And I said, okay, so you've been doing this for. It's one and a half people. It's a three person team. So one and a half of you for the last two years have been doing these vendor assessments. How are we safer? Because you've done them. And they started to say, well, we verified their sphere. I'm like, no, no, no. Have we ever not signed a contract because of your findings? Have we ever gotten a vendor to change what they do? And the answer was no. And I'm like, so why are we doing this? They filled out the questionnaire that apparently was good enough. So just stop. And now nobody's gonna just be like, oh, if you fill out a questionnaire, we don't even read it. But that's kind of the state of the art. And so I think Ross is onto something, which is we have a problem. Now why did I just reject Ross's suggestion right out of the gate?

[David Spark]

Why?

[Andy Ellis]

Which is having been a vendor before, I can Tell you that this would be laughable if somebody came to me and said, well, look, you're proposing 100k a year. Deal. We're going to instead pay you 80k and if you can meet our compliance SLAs and demonstrate them, then we'll give you the extra 20k. I'd be like, okay, by the way, I'll take the 20K. But that's now on top of 125K. Deal, deal. Because it's going to cost me money to interact with you and your vendor procurement team.

[David Spark]

No, but he argues. But the thing is, it still takes you time to fill out these darn questionnaires. So like, let's not do that, let's do this.

[Andy Ellis]

Oh, do you know how cheap it is to fill out the questionnaires now?

[David Spark]

Well, now with the AI tools, yes.

[Andy Ellis]

Well, even before we had the AI tools, like, yeah, I had one person at Akama, I think we ended up with like one and a half who just did questionnaire filling out as a side piece of their job and then manage customer audits because we knew what all the answers were. We had seen all these. If you sent us, oh, this is the one derived from the bit sig, we could almost, we could just like copy and paste a column of answers. And now there's startups that do this for you. It's like you go, and I'm not going to name anybody since you know, we're careful not to name vendors who aren't currently sponsors. But there are startups that will like, you fill out one questionnaire, hand it to them, and they'll fill out the rest of your questionnaires for you. So this scales just fine from a vendor perspective. Prove to you that I met whatever crazy SLA you have that you don't define right in my environment. That's expensive and will probably take me more time than any of the services we're providing to you. So this proposal doesn't reduce what you pay me, it's going to increase what you pay me. That's why it's dead in the water.

[David Spark]

All right, Vikas, what's your thoughts? Do you agree with Andy or could this or some version of it work? What Ross proposes? Again, this is financial incentives to meet SLAs.

[Vikas Mahajan]

Yeah, I agree with Andy in that Ross has an interesting idea. It's a problem, it's a challenge. The simple fact is we do not manage our third party vendors environments. We will never know the truth of what's in there. You can certainly try to enforce Something like a reverse SLA is basically what we're proposing, right? We're saying, oh, you have to meet our requirements and pay us back or we'll pay you more the other way if you meet these requirements. But again, you have to define some criteria. No one patches at 100% all the time, every day of the week. It's impossible. So then where do you set the bar? Right. And then we have legacy systems. You have all kinds of things in an environment. Which systems are you looking at? I think that would be difficult. My personal opinion is the better approach these days. The business cares about not only reducing risk, but they care about operations. If you're. That is a critical vendor and they're a key supplier who's producing something for you. I care more about them being able to get back online and producing whatever it is they're producing than I do about anything else if I'm the business person. So I would rather focus on resiliency. I would rather focus the questions I have with them on what are you doing to survive from an attack or survive from an outage of some kind of disruption. What have you done in your infrastructure there? I think the other option also that we should really be considering is inviting these critical suppliers into our tabletop exercises and saying, okay, if you want to do business with us, you need to work with us through this exercise so that we have a plan together of how we can survive a cyber incident of some kind. And I think that's going to be more productive in the end. No one's environment is perfect. It's impossible. Right? We all have to be prepared for the worst case.

[Host]

Didn't we solve this already?

[David Spark]

Quote, cybersecurity didn't get get better. We just got better at pretending. That's Joshua Copeland of Crescendo's unpopular opinion about where security stands today. Despite all our new tools, every breach still starts with the same five failures we documented in the 1990s. Missing patches, credential reuse, misconfigurations, flat networks, phishing. Security teams don't usually battle state based actors every day. They're too busy not doing their homework. Copeland says, we built an industry around looking secure instead of being secure. Now, are we losing our focus on security? I'm going to ask you Vikas on this. Are the many market forces such as analysts, research reports, VCs and media distracting us from doing what we say all the time on this show, which is focus on the fundamentals? What do you think?

[Vikas Mahajan]

Well, I can't argue that focusing on the fundamentals is absolutely the number one thing you can do in any organization. But blocking and tackling, as we say, right? In a football analogy, you've got to be able to patch, you've got to be able to identify your vulnerabilities, you've got to be able to keep up with those things. But that doesn't necessarily mean everything else we're doing is just theater either. I sort of look at the tsa, right? You walk through the lines, we all hate it, we all go through it and you're feeling like it's all theater. But there's things going on behind the scenes that none of us see that are also out there protecting us. Looking at things, if you go to Israel, you'll know they do full background checks on people. There's a lot more in depth scanning. And it's the same thing with our tools. A lot of the security these days is things that our users don't see. We're looking at behavior analytics, we're looking at the way they're logging in, where they're logging in from what they're doing. And we're trying to get some sense of whether that person is who they claim to be. And we're looking at a lot more data. Is that making things better? I think overall it is. We're also doing a better job educating our users about this and making it more of a priority. But there will always be the human element, the human mistake, the human error. And we have to have tools that can help us detect and respond and, and recover from those very quickly. But yes, absolutely, you have to do the basics, you have to do them well. But I don't think what we're doing is weak.

[David Spark]

But we get this complaint all the time. All these sort of entities I mentioned, vendors, research reports, VCs, media, are they distracting us from doing everything you said we should be doing?

[Vikas Mahajan]

Good question. There's definitely a lot of noise out there and you'll get the same information over and over again. And I think the problem is there isn't always. Context is something critical, is something out there that you have to worry about. It's applying that context to your environment that's important. Log 4J. Great example. Huge news for people who have no idea what in the world that thing is. And suddenly everybody's worried about it. But do I need to worry about it? Do I have healthcare manufacturing equipment for a blood environment that's using something that. Log 4J. How likely is it that a hacker can get into that equipment? Because it's on an isolated network, Do I need to drop everything and prioritize that? And then how do I explain that to my executives or my leaders who may be asking, I heard about this thing. What does it mean to me again? Overall, I'm glad there's more talk about cybersecurity. I'm glad there's more out there, that it's on people's minds. But we as CISOs, as cybersecurity professionals, have to be able to translate that and explain what that means in our organizations so that the priorities are in the right places.

[David Spark]

All right, Andy, I'm throwing this to you.

[Andy Ellis]

I'm going to say, and I love Joshua when he does these unpopular opinions, but I think he's just flat out wrong here.

[David Spark]

Okay.

[Andy Ellis]

Like, I just want to start with that. Security has gotten way better since 1999. If I go back to where I was in 1999, which was doing security for the US Air Force, but also

[David Spark]

it's far more complex than it was back in 1999.

[Andy Ellis]

Right. First of all, the IT environments are amazingly complex. But just to be clear, like we had. In fact, I've got one of these boxes right behind me. Network Systems group Border Guard 2000 was our line rate packet filter. And literally we were looking for plain text strings because nothing was encrypted that were like, oh, people are sending admin commands to SMTP servers. That's the level of vulnerability things we were dealing with. This is not like basic. We wouldn't even call this basic patches anymore. The amount of just insecure by default, and you're stuck there. Software was through the roof. In fact, to connect into this, our security system, we had to use telnet.

[David Spark]

That's pretty old.

[Andy Ellis]

We didn't have ssh. We're in the clear logging in across international boundaries because we're deploying these in theater. So we had to run an IP tunnel. Was it IP type 40 encrypted tunnel to then telnet across, like doing functionally. GRE encapsulation, very flaky just to get to our devices. Like, this is the sort of stuff in 1999. Yes. At a high level. Oh, credential reuse was. Yes, it was still a problem, but

[David Spark]

everything is more today, like across the board. More attacks, more technology, more complexity.

[Andy Ellis]

Let's talk about basic patch management. And I'm going to pull up an iPhone. You can't see this if you're just listening.

[David Spark]

People know what an iPhone looks like, Andy.

[Andy Ellis]

You either have one or one of your family members does. I Don't know how many of you remember when the first iPhone came out, what it took to implement patches. You wanted to update your iPhone. You had to install special software on your computer. You had to download the pat the update. You had to then plug your phone into the computer using a cable that only worked with that phone and make the update happen. Now over there, updates, you never interact with them. Like, your phone just stays secure. And that's the default. Your web browser, like, updates every time you reboot it. And if you don't reboot it often enough, it says, hey, restart the web browser so that I can take my updates, you idiot.

[David Spark]

So this is an interesting take. In all aspects of our world, we hear people complaining, oh, it was much better back in the day kind of a thing.

[Andy Ellis]

It was not.

[David Spark]

And people say, but the reality is, no, it's better today because you don't remember. Also, our memories are short. We're just worried about right now. We didn't remember how much stunk back then, Right.

[Andy Ellis]

Thirty years ago, there was almost no software and it was all awful. Like we were basically trying to secure environments that had no security consideration anywhere in the design environment. And now we're 30 years later and yes, there are still problems at the edges, but the core of our environments are functionally secure and we have built a trillion dollar economy. I'm just pulling that number out of thin air, so please don't yell at me for making that one up. This huge economy on top of the Internet and computing, and it's mostly fine.

[Vikas Mahajan]

And all I have to say is, Andy, my EV gets its updates from Tesla automatically. My car, right, I'm driving on this thing. What in the world? How different a world it is today.

[Andy Ellis]

Yep.

[David Spark]

Hey, you remember at the top of the show I told you that today's episode is sponsored by adaptive security? And here's something else you might not know. They are the first cybersecurity company backed by OpenAI. Now, here's something you do know. All CISOs and security professionals know that the playbook has changed. I mean, attackers don't need malware to get in anymore. They can use AI to impersonate your CFO on a call, clone a vendor's voice, or send a perfectly written email that looks like it came from inside your organization. The new attack surface is actually trust. Adaptive is built for that reality. Their platform runs realistic deepfake and social engineering simulations. So your team experiences these attacks the way they happen in the wild and learns how to spot them and respond fast. And with adaptive's AI content creator. Security teams can take a breaking threat, a policy update or a compliance requirement and instantly turn it into an interactive multilingual training without designers and without delays. So time sensitive stuff, pretty darn impressive. Adaptive is actually trusted by Fortune 500s and backed by Andreessen Horowitz and OpenAI. You can learn more if you go to their website. It's adaptivesecurity.com and it is spelled just the way it sounds. Adaptivesecurity.com.

[Host]

It's time to play what's up Worse

[David Spark]

the cost. Are you familiar with this game? Two bad scenarios. You have to pick one.

[Vikas Mahajan]

Okay, okay.

[Andy Ellis]

Just to be very clear, they're both going to be very awful scenarios.

[David Spark]

You're not going to like either one.

[Andy Ellis]

We judge them as presented. We don't get to like create arbitrary things. You take them as they are, take them as is. And I'll have to go first so you can decide whether you agree with me or not. So I take the hits of going first.

[David Spark]

All right, this is kind of a long one, but I'm going to kind of read through and abbreviate some stuff. This comes from Steve Wingate of Cyberguard Advisors and he's actually dealing with a variation of this right now. Okay, okay, okay. Scenario number one. And I'll give you kind of the headline for it and then like the issues that are going under it. So scenario number one is you have no enterprise wide AI strategy. Your governance relies solely on an acceptable use policy description. That is it.

[Andy Ellis]

Okay, okay. So this is what everybody is doing right now?

[David Spark]

Pretty much. Well, they're probably not paying attention to the acceptable use policy too.

[Andy Ellis]

I think that's the point. I mean, I think governed by a clause in the acceptable use policy is his polite way of saying the lawyers are pointing and saying we have a policy, but nobody pays attention to it.

[David Spark]

All right, so employees are permitted to use AI tools based on general aup, acceptable use policy outlines behavioral exceptions, provides no architecture, no control mechanisms or organizational alignment. All the usage is organic. So what you got? You got the shadow AI issues, untracked use of every tool.

[Andy Ellis]

Are we still in scenario one or is this a new scenario?

[David Spark]

Yes. So we're still in scenario number. Inconsistent risk posture, no framework to classify operational chaos. You've got compliance exposure, and the leadership doesn't have a clue what's going on. Okay, so it's summary, it's governance by policy alone.

[Andy Ellis]

So I'm trying to figure out, do you actually have a policy or it sounds like you don't really?

[David Spark]

You do have a policy. You totally have a policy, and people are told to follow it. So the policy is sound, everything else is not.

[Andy Ellis]

Oh, okay. So you have a very sound policy that is well written and observed only in the breach.

[David Spark]

Got it. Maybe, maybe not. We don't know. We don't know.

[Andy Ellis]

No, but I think that's going to be key when I'm comparing it to the next one. And so that's why I want to land it before we get to the next one.

[David Spark]

Okay, so here's the second one. There is no enterprise wide AI strategy and there's no aup, no acceptable use policy. But here's the good news. Technical controls are implemented for a single AI provider, that being claude. Okay. So the organization implements access to monitoring controls for claude. So sso, DLP integration, prompt logging, but no enterprise strategy defines broader AI objectives, integration standards, or vendor neutrality. But there's no adherence or knowledge of an aop. So employees could be entering anything they want into claude. No perceived limits. All right, which one is worse?

[Andy Ellis]

So this one's a fascinating, because I think a lot's gonna come down to the definition of that first one. So my experience of people who point at AUPs and say like, oh, our policy is. There is somebody wrote a policy and nobody actually pays attention to it.

[David Spark]

People know it exists.

[Andy Ellis]

People know it exists because there's a checkbox in the security awareness program that says it exists.

[David Spark]

Yeah, that's the unknown. Yeah.

[Andy Ellis]

So I'm gonna posit that if I'm evaluating this scenario, I have to assume the AUP is toothless and people don't actually follow it. It's not very helpful to them. And so while we fail feel good because we covered CYA'd around policy, that scenario number one is complete wild west. Like, sure, we wrote stuff down, but there's no governance that's posited. We don't know what people are doing. So scenario number two is we don't know what people were doing, but at least we have securely integrated with claude. So given that, I think that's the reasonable way to frame these two. I will take the first one as worse because I have nothing to build on other than a policy nobody reads.

[David Spark]

But do you have. Do you have. And I don't know the answer to this at all, but do you have better legal protections in scenario A than you do scenario B?

[Andy Ellis]

I don't care about the legal protections on this one.

[David Spark]

No.

[Andy Ellis]

In fact, I would actually argue that we wrote a policy and people followed it is not actually as Legally protecting as people think it is, because all you have to do is show like, this was one of the things I had an argument I once had with our general campaign. We'd just written one of the policies that was like, you can't use your cell phone while driving. And we had an incident, and the general counsel got called onto the incident. He said, well, I'm in my car, so you know, it'll take me a minute to respond because I'm going to keep you on mute. And I said, you have to get off this call because we have a policy that says you can't be on the call. He says, that's okay. And I'm like, oh, this is fascinating. Like, exactly how effective is that policy itself? Because it's written from a liability perspective that if somebody gets in a crash while they're on the car, the company wants to say, well, they violated the policy. But if the person who signed the policy violates it, he didn't mean anybody to follow it. I don't think you get real liability protection from that. So I would much rather be in scenario two where we securely integrated the one enterprise grade AI that we're using.

[David Spark]

All right, Vikas throwing this to you. Do you agree or disagree with Andy?

[Vikas Mahajan]

From a risk perspective, a Cyber perspective, Andy's 100% right. You have a controllable environment. You have a place where you have control. Right.

[David Spark]

Well, now, hold it again. Control. And putting this in quotes, as you see. But it's like now in the first scenario, they are told to do the right thing. The second scenario, they're like, do whatever the heck you want.

[Vikas Mahajan]

Yes. The reality is that's one fire you have to fight if there's an issue. Right. One place to go, you know where it all is. So that makes it much matchable.

[Andy Ellis]

Yeah. And Claude gives you controls at least that you can go look at of what people are doing. Even if you haven't turned on the enforcement, you at least have the accessibility to a lot of data.

[David Spark]

You have a clue what's going on and you could shut things down as you see them.

[Andy Ellis]

Yep.

[Vikas Mahajan]

Now back to the first scenario. While I don't like it as much, I think that's the reality of where most people are. Right. And I think working within that is not unreasonable either. But clearly we're putting a lot of trust into our users and our folks doing the right thing. And that's a matter of trust in your business model.

[Andy Ellis]

This is one of the things I see this failure actually a lot in security teams and I've had to beat it out of a lot of people who worked for me. The last thing you should do is write a policy. And too many security practitioners think it's the first thing you should do. It's actually the last. The first thing that you do is you go and you secure a system and, and you figure out how to operate it and you provide operating guidelines. You say, here's how to do it, right? I'm not writing a policy. I'm saying, here's what you should do. Here's our key principles. Then you can write the security practices that say, here is actually what we do to secure this. Then you can write guidelines. And so you basically work your way back to a policy. And so when you finally write a policy, it's already being followed and the policy is actually weaker than the guidelines and the practice that you have implemented. So people look at the policy and they're like, oh, I'm doing better than that, I'm fine. Too many security practitioners write the policies. What I want you to do, even though you're not even on step one, the policy is perfection.

[David Spark]

But, you know, writing the policy, I understand, because it's an exercise to see where we're all at. You know what I mean?

[Andy Ellis]

No, it's not.

[David Spark]

It's not.

[Andy Ellis]

When most people write policies, it's an exercise in writing the dogma of where we believe we ought to be. And it's written from this very academic perspective of this is what perfection would look like. Ivory tower. And when a practitioner looks at it, they're like, well, I don't follow that. And nobody else in the company does. So it's clearly not the policy. It's basically like walking around and telling somebody to get off the playground. And everybody else is on the playground and they like. Because you look at you and say, you can't kick me off the playground because you haven't ever kicked anybody off the playground. So they stop listening to you.

[David Spark]

By the way, you know what this reminds me of is my wife and I used to work together. We no longer work together and luckily we're still married. So there you go. I read so many articles about spouses working together and almost every single one said, well, don't bring your work home and you know, don't bring it. That is impossible. It is not reality. Yes, that is a. That is a sky high goal vision. That's BS. You can't avoid it.

[Andy Ellis]

Well, 90% of the people who write about spouses working together are. You both work for the same company. Which is very different than you two are the company.

[David Spark]

Yeah, yeah, well, yeah, well, that was the case at the time we were at the company. Vikash, I cut you off. What were you about to say?

[Vikas Mahajan]

Oh, I was just gonna say that we're doing. Andy, kind of what you were saying, which was show the users how to use the tools responsibly.

[Andy Ellis]

Right.

[Vikas Mahajan]

When ChatGPT first came out, and the Microsoft version of it, we actually went through and showed our users, here's how you can responsibly use this type of technology, here's how it can be of assistance, and here's not what to do with it. Do not put this type of data. Do not do things. And it was, I think, more effective than just me going out and blocking everything and saying, nobody can use anything. Then users go and find work. It's so easy, right, to do that. So I think that approach is much better.

[Host]

What's the starting point for a CISO?

[David Spark]

Where do you start when you're building a SoC from scratch? Now, that came up on the cybersecurity subreddit from a new CISO who never worked in a soc but needs to launch one quickly. The top advice Quote, don't start with which SIM should I buy? Start with what are my crown jewels? What can I actually see today? And what can we realistically respond to? But the overwhelming consensus was actually don't build one at all. Outsource it. The math is simple. 24, 7 coverage means 1.5 to $2 million in salary alone, plus tools, plus at least five to six people minimum. So commenters said you need a company of 5,000 plus staff to justify it financially. An MSSP or MDR gets you better detection coverage across multiple customers than you'll build internally. Some pointed out that an outsourced SOC treats you just like quote another client and then deliver boilerplate service. So I'll start with you, Vikas. So when does building a SoC from scratch actually make some sense versus leaning on an MSSP?

[Vikas Mahajan]

I think that's an interesting question. And yes, it does vary on maybe the type of business you're in. Maybe if you're in the federal government contracting space and you need to run and watch over these things yourselves, maybe it's better to run it in house. I don't really know. My personal opinion is I agree with the latter half, which is in this day and age, it's better to procure the service. Those folks are trained in how to run and manage and monitor and alert on these things. They deal with IT all the time. And I think I'm going to get a higher quality of service based on that experience and expertise. They've already built. They already built all the rules for the sim, they've already built all the rules for everything you need to know. Why not take advantage of all those years of experience and expertise in your environment?

[David Spark]

And I mean, people are complaining about boilerplate, but I don't know how boilerplate really is. I mean, if they can get things up and running quickly for you and maintain it for you, that's better than you struggling along to get it going, I would assume.

[Vikas Mahajan]

I agree. But the best of both worlds is sort of what we've tried to implement, which is we manage our tools and we have access to them. I don't outsource the SIEM logs to someone else. I want to have those so that our teams can do investigations as well. So we found vendors and providers who work in that model. It's our data, it's our environment, but you have access to it and you're monitoring the systems. That, for me is sort of the happy medium.

[David Spark]

All right, I throw this to you, Andy. When does it make sense? You worked with a company that was quite large, so you probably had your own sock.

[Andy Ellis]

We had multiple socks.

[David Spark]

Yeah. But also was critical also to your business model too, for that matter. But you also work with startups which I'm assuming probably don't have their own soc.

[Andy Ellis]

Yeah. So a lot varies here. You have to ask, what am I actually trying to accomplish? And then you can figure out how you want to deliver it. But my core answer is, if your question is, I want to build a SoC, I say, who is your tier 4 engineer? And most people, like, look at you and they're like, what's tier four? Right. Tier one is the alert fires. They run very simple triage response playbooks. Tier two is you got to tweak the response playbook, maybe do escalation. Tier three is you get something unusual that you can brainstorm. Tier four is you get something nobody in your company's ever seen before and you have to go deal with it. We often call them threat hunters now, like that's your first hire is that person. And then you figure out who is doing all the work that escalates to this person because they're either the person who's managing your mssp, not from a contract perspective, but from a day to day. Like MSSP has issues, this is who they're going to talk to. And you have to have that person on your staff. The places where I've seen MSSPs go bad is somebody just hires the MSSP, outsources everything to them and there's nobody in the company who knows what the MSSP is doing. So yeah, you get treated like just another customer. They don't actually care about what you're doing. Cause you've demonstrated that you don't care about what you're doing.

[Vikas Mahajan]

Andy, you said it well, you've got to manage the vendors. That's so fundamental and I think something we as CISOs sometimes forget or cybersecurity professionals. You can't just hire someone and outsource it. You do have to manage the relationship and that's the key. If you want good service, you have to put in the time and effort to manage the relationship with the vendor, have those quarterly updates, meet with them regularly, have them take you out to lunch, talk to you about their program, their new features, the new capabilities. If you're showing interest in them, they're knowing that you're watching their services and they'll want to deliver. So definitely you have to have that relationship.

[Andy Ellis]

And then the second thing that I would suggest is do you already have an IT help desk? Because if you do, odds are your SOC should be tightly tied into your help desk services as well. You already have an operations team. You should not be trying to stand up one completely in parallel. You should be standing up more capabilities inside that team and maybe they're going to manage your MSSP for you. Now you've got a paired help desk or maybe you can just take your security alerts and if you've written good playbooks, you can hand them to your help desk.

[Host]

Could this possibly work?

[David Spark]

Quote shift left left has become a cringe inducing or eye rolling phrase in cybersecurity, end quote, writes Derek Fisher of Temple University. It's simply just saying, quote Be proactive with your security program. Fisher proposes lightweight threat modeling at the user story level using Adam Showstack's four what are we building? What can go wrong? What are we going to do about it? Did we do a good job? Simply a conversation on these topics before you dovetail into requirements, code or penetration test. Fisher admits he'd prefer a more thorough process, but when speed is the business priority, this might be the best option. Does this gut check conversation, water down security to make it palatable to developers? Or could this actually help us with the mystical shift left handy?

[Andy Ellis]

So this is sort of a weird one because like shift left gets a bad rap because it means a lot of different things Besides do security architecture reviews early.

[David Spark]

It was also overused for a period of time, too.

[Andy Ellis]

Oh, no, it's still overused, just to be clear.

[David Spark]

By the way, I just want to say, everything overused in cyber gets slammed, period.

[Andy Ellis]

Right. But that said, let's just take the nuts and bolts here, which is when you're the security person who's first engaging with a product, what's the great way to start the conversation? This is a really fine icebreaker because all too often what happens is security engineers show up and they say, here is my list of requirements. And they don't know what they're applying to. And all that it takes is for you to be grossly wrong on one point to completely undermine any authority you might have had if you walk in. And I'll give you a great example, I had this with the Fedramp program. Office wanted to fail us because we didn't monitor the humidity in our data centers because they had this one requirement. You must monitor humidity. And I'm like, first of all, this is not a security requirement. And second of all, you wrote that because you're going after people who only have one data center. I'm in 2000. I don't care about humidity in them. Like, I assume that the people who, like, own 95% of the space in every data center care about humidity. I don't. Have a nice day. And it completely undermined their authority on other things. When they lost that battle because they invested in that battle, I blew them off. I succeeded. When a security team walks in and says, you need to do X, and they're like, well, I see an X, you demand I do this one stupid thing. And. And they win the battle. They're gonna stop listening to you for everything. So instead walk in and say, what are you doing? How could it go wrong? Like, get me started. You've already thought about this. What are your failure modes when those happen, what's your plans? You've already done security work, right? It comes in and it respects the work of the developer and puts it onto a positive footing. They get to teach you. And now you have context. So when you say, oh, I see that you are taking user inputs. How are you managing filtering and prompt injection and dealing with that? Whereas if you walked in and said, how are you dealing with prompt injection? And they're like, I don't have an LLM, you just sound like you don't know what you're talking about, which is going to hurt the conversation.

[David Spark]

All right, your take.

[Vikas Mahajan]

I don't have A perfect answer for this. I think the best approach here from my thoughts are yes, you have to understand the application, what it is they're trying to build or buy or whatever the case may be. And that's when you have that conversation. Now usually in some environments it starts with the business. What is the business trying to do? What is it that their goal? What are they trying to accomplish? And from there we start working into the level down details. Okay, great. Like you said, you're getting input for this. Okay, now have you thought about the failure scenarios where those things happen? The reality in organizations though is you guys, lots of projects, you have lots of things getting thrown at you and it's hard to sit down and have a lot of those conversations all the time. You do have to have an answer. The executives, businesses are sometimes looking for that. Hey, just give me your gut check. Does this sound like the right thing to do or do we need to go back and look at it? And I also bring in third parties here because in some organizations, ours included, we rely on third parties to develop some apps for us. And I have to go back and make sure are the third parties doing their due diligence and writing code securely and doing things the right way? If we get something thrown over the fence to us, I don't want the business just taking it and running with it. For me, I like the idea of instead of bringing the horse to water, which is bring all of these developers and folks into the security world, I think it's going to be much more effective to bring the water to the horse, bring the security capabilities to the people, embed it in the tools they're working with in their git, whatever it is they're using. Right. Put those things in their hands and then train them on how to use it and understand what it means and then provide that added value, which is I'm your trusted advisor, I can help you understand what this thing is telling you to do if you don't necessarily know. And we're here to help you through this process. Not here to just be a sort of audit or checklist, make sure this is done. OWASP top 10 thrown at you kind of a thing.

[David Spark]

Excellent. Well, I'm glad we're putting a little bit more context around the shift left malignment that has happened. I honestly didn't think it was getting maligned. But again, also, like I said before, take the whole the zero trust philosophy. It makes sense, but people still slam it too. Andy.

[Andy Ellis]

Yeah, well, the challenge with shift left is like zero Trust. Everybody has adopted it. So every vendor at some point has said, oh, well, we're helping you shift left. Which really was just like, how do we get from being completely reactive?

[David Spark]

It didn't start that way. It didn't start that way. Like it would start. We have a zero trust solution. Or we have a. Or we're a zero trust answer. We're a shift left in rather than no, we're part of the solution to do that.

[Andy Ellis]

Right. Well, and zero. Like I remember being at a conference where after it, I got pitched, it was a grad, a student conference. And after all the students came up to sell me their ideas and every single idea mentioned zero trust and half of the mentioned blockchain, and I was just like, okay, like, did you. Did you basically pick a couple buzzwords and then try to say how this is. And that's where the malignment comes from, is everybody wants to use the buzzword because they feel the buzzword is successful, even if they have nothing to do with it.

[David Spark]

All right, let's come to an end of this show. Thank you very much, Andy. Thank you very much, Vikas. This has been a great show. Both of you just slammed way too much knowledge in way too much period of time. And I greatly appreciate. And we also greatly appreciate our sponsor. And that would be Adaptive Security. Remember, it's the security awareness employees love. You can learn more about them if you go to adaptivesecurity.com and when you go there, let them know that you heard about them through the CISO series. Thank you very much, Vikas. Thank you very much, Andy. And to our audience, as always, we say this, we greatly appreciate your contributions. And for listening to the CISO series

[Host]

podcast that wraps up another episode. If you haven't subscribed to the podcast, please do. We have lots more shows on our website, cisoseries.com Please join us on Fridays for our live shows, Super Cyber Friday, our virtual meetup and cybersecurity Headlines Week in review. This show thrives on your input. Go to the participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you're interested in sponsoring the podcast, contact David Spark [e-mail] thank you for listening to the CISO Series podcast.