Every Failed Startup Starts as a Dream for a Single Pane of Glass

Summary

---

Podcast Summary

CISO Series Podcast

Episode: Every Failed Startup Starts as a Dream for a Single Pane of Glass
Hosts: David Spark, Andy Ellis
Guest: Sean Marion, VP and Chief Security Officer at Xcel Energy
Release Date: February 25, 2025

---

Overview

This episode explores the enduring challenges and wishful thinking in cybersecurity vendors and practitioners, with a focus on why certain persistent solutions—like the mythical “single pane of glass”—remain out of reach. The discussion dives deep on organizational culture, effective policy-making, soft skills across cyber roles, evaluating crisis leadership, and the allure (and folly) of cybersecurity startup “tar pits.” The hosts are joined by Sean Marion, who brings both physical and digital security expertise from his new role at Xcel Energy.

---

Key Discussion Points & Insights

1. Policy vs. Practice: When Does Policy Matter?

[03:55–08:51]

• Reality vs. Written Word: Policies without implementation don’t just fail—they degrade overall compliance and introduce liability.
  • Andy Ellis: “Policies that don’t match the real world are harmful to your business because they degrade every other policy.” [04:36]
  • Sean Marion: “You can introduce a tremendous amount of legal liability. If you say, we’re doing one thing and you don’t actually do that one thing, that’s liability.” [05:56]
• Practical Implementation: Start with practices that work, then codify as policy rather than imposing “one-size-fits-all” directives.
  • Both agree: It's better to have a simple, fully-adopted policy than a complex, ignored one.
• Change Management: People adapt over time, but context (“the why”) and communication are usually neglected.
  • Sean Marion: “I don’t think that people come to work and say, I don’t want to do the right thing. I think sometimes they just don’t fully understand how to do the right thing.” [07:25]
• Eye-Roll Policies: Example: Badging in and out for physical security is often ignored unless tied directly to an action (like opening a door).

---

2. Cybersecurity Soft Skills: Speaking the Language of the Business

[08:57–12:38]

• Storytelling as a CISO Superpower: Sean emphasizes that communicating risk or security priorities often hinges on storytelling—not just to boards, but also to architects and other teams.
  • Sean Marion: “A lot of what I do is storytelling… not dumbing it down… just trying to make sure I can communicate that story effectively.” [09:40]
• Peer Support and Coaching: Leveraging a network of CISO peers helps improve narratives and presentations, especially for high-stakes board interactions.
  • Sean Marion: “There have been many times when I’ve reached out… Give me feedback. Am I relaying this appropriately?” [10:24]
• “Hard” and “Soft” Skills: Andy flips the terminology—soft skills are often harder and less measurable.
  • Andy Ellis: “Soft skills are actually much harder than what we call hard skills. The reason we call hard skills hard skills is because we can measure them directly.” [11:15]
  • Adapting the narrative to the audience is critical—risk for a product manager needs to be discussed in business impact, not technical exploit lingo.

---

3. What’s Worse? (Game Segment)

Scenario: Known Breach with Missing Logs vs. Unknown Breach with Missing Logs

[14:05–19:30]

• Scenario 1: Breach occurs, but logs are sporadically missing during the incident.
• Scenario 2: Weeks of logs are missing with anecdotal evidence of suspicious activity, but no confirmed breach.
• Panel Verdict: The known breach with missing logs is worse; partial information can mislead, while ongoing visibility (even when incomplete) in the second scenario offers more opportunity for real-time remediation.
  • Andy Ellis: “…the breach is what matters. So, given my choices, I’ve got a breach that I know about, or I’ve got a potential breach I don’t know about. … I’m just gonna go with the breach is worse.” [16:46]
  • Sean Marion: “Partial information can be a red herring too. It can lead you down the wrong path. … Either one is rough.” [18:33]

---

4. Hiring for Crisis Experience

[19:35–26:35]

• The Dilemma: Is it worse to hire a CISO who’s had five major crises (some handled poorly) or one who has never faced a crisis?
• Experience Diversity: Both agree context is key—industry, company size, and potential risk profile matter more than incident count alone.
  • Andy Ellis: “The real question I want to ask… are you putting your team in the roles of being able to run maybe not your worst cyber incidents, but the ones that are one step down? Because that’s where they’re going to learn the skills…” [26:35]
  • Sean Marion: “If you haven’t had a single incident, I’m more curious where you’re looking… I really want to understand the failures. Like, what did they learn from that?” [23:12]
• Success Isn’t Always a Lack of Incidents: CISOs who build resilient, preventative programs can have spotless records for the right reasons.

---

5. Tar Pit Startups: The Doomed Quests of Cybersecurity

[26:41–34:40]

• Classic Tar Pits:
  • Single Pane of Glass/Dashboards: Universally sought after, universally elusive.
  • Real-time Third Party Risk Management: Vendors resist real-time transparency; demand will never align with supplier willingness.
    • Andy Ellis: “Technically, you could go solve this. Practically, you can’t.” [27:44]
  • AI-powered SoC: Aspirational, but unclear if true value is achieved—often leadership doesn’t understand their SoC’s value, making “AI replacements” a false promise.
  • Perfect DLP: Sells well, delivers poorly—executives believe the pitch, but “Data Loss Prevention” is really just “Data Loss Notification.”
    • Sean Marion: “…they pitch it as what it is and they come back… This is going to keep all… It’s data loss prevention. It’s in the name. […] But it’s data loss notification.” [30:09]
• Hope for Improvement: Both Andy and Sean express cautious optimism that AI could eventually improve DLP and data categorization—the main barrier in past attempts.
  • Andy Ellis: “…the use of AI to do categorization. Because categorization has been the thing that has killed every DLP project… people are creating content faster than I can do anything to categorize it.” [30:35]
• Other “Escapable” Tar Pits: Non-human identity management and self-service security marketplaces may see progress with appropriate innovation.

---

Notable Quotes & Memorable Moments

• On Policy:
  • Andy Ellis (04:36): “Policies that don’t match the real world are harmful to your business because they degrade every other policy.”
• On Legal Liability:
  • Sean Marion (05:56): “If you say, we’re doing one thing and you don’t actually do that one thing, that’s liability.”
• On Storytelling & Feedback:
  • Sean Marion (10:30): “Can I share this with you? Give me feedback. Am I relaying this appropriately? It’s things like that that make me better.”
• On DLP False Promises:
  • Sean Marion (30:09): “It’s data loss notification. It’s like, oh, there it went, there it went.”
• On Startups Chasing Tar Pits:
  • Andy Ellis (27:44): “Technically you could go solve this. Practically, you can’t.”

---

Other Highlights

• Andy’s Writing: Andy runs howtocso.com, an evergreen content site focused on practical CISO guidance based on real-world scenarios. Many show topics are reflected and expanded on there.
• Networking Importance
  • Sean Marion (35:39): “It just shows how these times go around. So lean on your friends. Make friends more than just colleagues.”
• Personal Milestone: Sean celebrates his 28th anniversary during the episode. “Today is 28 years. I celebrate my anniversary today. So super happy about that.” [36:19]
• Hiring at Xcel Energy: Sean notes they’re “wrapping up pretty heavily”—interested candidates should watch his LinkedIn for job postings and appeals to attitude and aptitude over resumes.

---

Key Timestamps

• 03:55: Introduction to real policy effectiveness and liability
• 08:57: Soft skills and storytelling at the CISO level
• 14:05: “What’s Worse” Security Scenarios Game
• 19:35: Crisis experience vs. no crisis experience in hiring
• 26:41: “Tar Pit” security startup ideas (single pane of glass, DLP, TPRM, etc.)
• 35:17: Closing words, networking significance, and recruitment plug

---

Takeaways

• Culture & Context Matter: The best policies grow from the ground up—make sure policy reflects established, real practice and communicate why changes happen.
• Networking is Power: Seasoned CISOs rely on each other for practical advice, emotional support, and honest feedback; building this network is invaluable.
• Beware the Dream Solutions: The industry is replete with “just one more dashboard” or “AI will fix it all!” dreams; skepticism, realism, and understanding the actual problem remain crucial.
• Evolving Optimism: While everyone dreams of perfect DLP and real-time everything, incremental improvement is possible as tech like AI matures—if anchored in clear outcomes.

---

Further Reading

• HowToCISO.com – Andy Ellis’ practical resource for CISOs
• Venture in Security: Tar Pit Ideas – Source for the “tar pit” blog post discussed

---

Transcript

[David Spark]

Best advice for a ciso, go get.

[Sean Marion]

Out network, meet peers in the industry. Don't duplicate things. You're going to hit problems, you're going to run into situations. Good, bad. Otherwise, lean on that network you've built so you don't have to duplicate. Don't reinvent the wheel.

[David Spark]

It's time to begin the CISO Series Podcast.

Welcome to the CISO Series Podcast. My name is David Spark. I'm the producer of the CISO series and my co host for this episode. We love, love having him on. It's none other than Andy Ellis, partner over at YLVentures. Say hello, Andy.

[Andy Ellis]

Good evening. Or depending on when you are in the world, good morning, good afternoon or good night.

[David Spark]

You know, Andy has recently prepared different languages, but he got lazy this time and he's leaning on English again.

[Andy Ellis]

I did in fact get lazy. I meant to do Swedish this morning, but then I had a complete brain far.

[David Spark]

It's all right, we'll let it slide. We are [e-mail] that's where you can find many of our programs and God willing, by the time we are releasing this episode, our new show will have been released. But not to be said yet. Our sponsor for today's episode is Nommo Security. Secure your entire data and AI lifecycle. A very interesting combination. More about that later in the show. But before we begin, I want to mention a site that you have launched, Andy, that I'm impressed that you launched this and you are populating it with your wisdom and knowledge of, may I say, decades of being a CISO.

[Andy Ellis]

Yes, decades indeed.

[David Spark]

Decades of being a CISO. The web address is howtocso. So h o w t o and then ciso.com what started this? What is it? Explain.

[Andy Ellis]

So I've always been a fan of writing evergreen content, the things that you write, not to get clicks today, but that you can always just easily go reference. And what I found is there's a distinct lack of here's how to do the job aimed at either CISOs who are transitioning from one role to another or just want to get better at it, or people who want to become CISOs. And we sometimes pretend that a lot of our knowledge is aimed at CISOs when it's actually aimed at lower level practitioners. And I wanted something to say, hey, if you're a ciso, here's what you should be thinking about. So it's not going to have like tons and tons of content because everything's going to be carefully curated to be what are the things you need about. So whether it's the OP eds that are relevant, the how to CISO volumes, which are a little bit longer, so folks may remember I wrote one a couple years ago about the first 91 days on the job. Just published another one, which is the idealized CISO job description, which you might find entertaining. I've got one on risk measurement in the works.

[David Spark]

Ah, very good. Well, we are sure to lean on some of that content because it is, well, pretty much in line with the editorial of this very show.

[Andy Ellis]

Exactly.

[David Spark]

So if you don't read it directly, you're going to get it on this show whether you like it or not.

[Andy Ellis]

Yep. A lot of the things that I've written down, there are things that I first said in response to something here, and I'm like, ooh, I should write that down. And here's where it goes.

[David Spark]

Then we want full credit. All right, let's bring our guest on. The last time we had our guest on, he was working as a CISO at a different company, but now he's at a new company. So I like to think of it as a fresh, new, brand new guest because he's a CISO somewhere else. It is the vp, Chief Security Officer, CSO at Xcel Energy, none other than Sean Marion. Sean, thank you so much for joining us.

[Sean Marion]

David, it's great to be here.

[David Spark]

They didn't think that through all the way, did they?

Do we really have a policy for this or did we simply write one? Alan Williman of Kirkpatrick Price asked that question, reminding us that policies are a good starting point but mean nothing without implementation. He posted a picture of a bird sitting on top of a no birds sign. Sure, you can put up a sign, but where's the control to back up the policy? For Alan, a good policy must be paired with training, monitoring and coaching to actually be meaningful. All right, Andy, I'm going to start with you. When does a written policy become an actually implemented policy? And what causes an implemented policy to fall apart?

[Andy Ellis]

First, I just want to comment that he's an optimist. If he thinks a policy without implementation is. What did he write Meant Nothing. No, no, it means a negative. Just to be very clear, policies that don't match the real world are harmful to your business because they degrade every other policy. The moment someone says, oh, look, there's a policy, nobody follows it. That means they don't follow other policies as well. So cut that out. I think of policies actually a lot like processes, which is you have to account for all the different nuances, all the different ways things can go wrong. And so instead of writing a one size fits all, either a statement, this is what we do, or a process, do step A, B and C without noticing that sometimes you need to do D and sometimes you skip B, you have the same problem. I'm a big fan of implementing policy slowly. You want to change policy? Great. Go do an implementation as a process. Get some buy in from people who want to do better.

[David Spark]

So do it the opposite direction.

[Andy Ellis]

Do it the opposite direction. Like, start with the practice. And once the practice becomes the norm, then you just write a policy to say, hey, here's what we do, and nobody's going to object because it's actually what you already do.

[Sean Marion]

Two points here. One is, I call these woobies. I don't think that will translate well across lines. But it's Linus in his blanket, Right? It's your safety blanket.

[Andy Ellis]

Yep.

[Sean Marion]

It's a false sense of security. So you walk around, we've got this policy, we're all good. But in reality, the second point is you can introduce a tremendous amount of legal liability. If you say, we're doing one thing and you don't actually do that one thing, that's liability.

[David Spark]

Ah, good point.

[Sean Marion]

So I agree with everything Andy said. Start small, get it right. I would rather have a policy that is less effective but implemented completely than one that's just. That would be that nobody follows.

[Andy Ellis]

Yeah. My favorite wooby is the policies against being on a phone while you're in a car.

[Sean Marion]

Right?

[Andy Ellis]

Right. And you get that call and general counsel's in the car. You're like, okay, great. You have moved from being negligent to being reckless. You know better because you wrote the policy and yet you're still violating it. And I actually think you increase your risk rather than decreasing your risk.

[Sean Marion]

Couldn't agree more.

[David Spark]

Well, as we know, this also falls under the area of change management. People don't usually like to change, but at the same time. And this I know, is not policy, but I'm going to quote Mark Zuckerberg, who every time he made a massive change to Facebook. And by the way, I reported on this multiple times, everyone would comment and get upset. Oh, Facebook is making a mistake doing this. They shouldn't have done this. And what happens is that thing that they got so upset of becomes the norm and they forget completely about it. Does that ever happen with policies, Sean?

[Sean Marion]

Absolutely. Well, there's two problems. One is everything you said where they kind of get numb to it. And two is I don't think we do a good job of communicating those changes. The why behind it. We tell them what, but we don't always tell them why and they don't always understand, so it's just not a priority. I don't think that people come to work and say, I don't want to do the right thing. I think sometimes they just don't fully understand how to do the right thing. And then back to the prior point, we can make these policies so complex that sometimes I'm not going to read that back to the Apple. Like, I'm just going to click through. I'm not going to read that.

[Andy Ellis]

Yeah, I think Sean's also being pretty generous. I think that a vast majority of the policies implemented in the corporate world are not actually that helpful. And so if somebody comes in and says, I want to do the right thing, that policy isn't it.

[David Spark]

Is there an example of a policy where, you know, the people looking at it are rolling their eyes like, seriously, we're not going to do that.

[Sean Marion]

I've got a great example. So as chief Security officer, I have physical and cybersecurity. On the physical side, we ask people to badge in and to badge out. They know the policy, there's signs. And the reason we have them badge out is if there's an emergency, we know who left the building. We have data on that. But they will push back. I gotta badge out. It's such a pain. It's like it's a very simple thing and they know it, but sometimes they'll push back.

[David Spark]

That's a good point. Andy, you got a quick example?

[Andy Ellis]

Oh, I was gonna go with the same one. Like, if your badge out is not tied to actually opening a door, nobody's gonna do it.

[David Spark]

Are we having communication issues?

We're always saying cybersecurity professionals need to, quote, speak the language of the business. We have said that more than once on this show. But what that actually means varies across roles. If Jenny Karam's recent LinkedIn post and his book Architecting the Art of Soft Skills in Technical Sales highlight how soft skills like communication, curiosity, empathy and active listening can look different for vendors, engineers and executives. So I'll ask you, Sean, do certain soft skills play differently in different cyber roles for either yourself or others? What have you seen work the best? And if you can specify certain things work better in other roles, that'd be interesting, but I don't know.

[Sean Marion]

Well, I'll speak specifically for my role because I've not been doing it long as Andy, but I've been doing this for about 12, 13 years, so at least one decade under the belt. But I think in my role, a lot of what I do is storytelling. And I don't mean fake storytelling. I mean, it's connecting with others through story. So whether it's the architects and trying to relay a priority, or whether it's the board of directors or whoever it is trying to relay what's important, not dumbing it down, not watering it down, but just trying to make sure I can communicate that story effectively so they understand.

[David Spark]

Can I double down on your cold open today where you say, look for help? Have you ever had a situation where you're like, I need to explain this? I don't know. You go to your CISO peers and say, what's the best way to tell this story? And has someone helped you?

[Sean Marion]

Yeah, absolutely. I've got a friend, a confidant, a coach, a mentor. Rich Mason. I think you might know him.

[David Spark]

Yeah, sure.

[Sean Marion]

Rich has been a friend of mine for 25 years, and there have been many times when I've reached out to him to say, I had a board presentation recently. Hey, can I give this pitch? Can I share this with you? Give me feedback. Am I relaying this appropriately? It's things like that that make me better and get me more prepared.

[David Spark]

Ah, by the way, I can't stress the value of whatever you have, like a five to 15 minute presentation, getting an audience for it. I did this where I had a 15 minute presentation, and it went through, like, four or five different audiences who were willing to list to it. By the way, people will listen to a 15 minute, not an hour to give you feedback. And from the first time I said it to the last time, it drastically changed. Maybe like 70%. It completely changed, Andy.

[Andy Ellis]

So first of all, I want to start by saying soft skills are actually much harder than what we call hard skills. The reason that we call hard skills hard skills is because we can measure them directly. We know how well you can write code or write English. How well you can communicate to somebody in their jargon is much, much harder. So let's just put aside the soft skills things for a moment. But we've all known that's my soapbox. When we say, speak the language of the business, what that actually means is, can you craft that narrative like Sean talked about that's gonna resonate with the party. And I like to think about it like fairytales. Why do we tell fairy tales? Like Little Red Riding Hood is basically saying, don't talk to strangers. That's literally all it says. Stranger approaches you on the street, asks you a question, you shouldn't talk to them, but you put it in the language of a child because you're trying to teach a child. You have to do the same thing with the business. If you want to talk about risk to a product manager, you'd better be putting it in terms of product launch delays, in terms of customer dissatisfaction. If you've got this weird, arcane story about this vulnerability being exploited in lateral movement and it sounds really cool to you, but their eyes are glazing over and they're thinking, I got a product to launch, it's going out in a month. How does this affect me? And if they're like, oh, I'm just going to spend an hour listening to you and then I won't do anything, and that's how I get through this, that's what they're going to do.

[David Spark]

Before I go on any further, let me tell you about our absolutely spectacular brand new sponsor, and that's Noma Security. You know, the rush to embed AI into applications has created incredible innovation for enterprises. No question. We've been seeing the news, we've been doing it ourselves. But with that innovation comes new risks. That's where NOMA comes in. It's the first application security platform built to secure the entire data and AI lifecycle. And there's a lot involved there, from securing your AI supply chain to protecting AI applications in runtime. NOMA detects and prevents threats like misconfigured data pipelines, vulnerabilities in notebook environments, malicious models, and adversarial AI attacks like prompt injection. With a single platform that integrates seamlessly across your AI tools, code and application SDKs, Noma empowers AppSec teams to secure AI applications without disrupting data engineering and AI teams day to day workflows. Fortune 500 companies are already using NOMA to bridge the gap between security and AI teams, delivering complete visibility, security and compliance across the data and AI lifecycle. You want to learn more? You want to know how you can get this into your environment? You got to go to NOMA Security site. And it's simple. Noma N O M A Security. That's it. NOMA Security.

It's time to play. What's worse.

Sean, you've played this game before. Do you remember this game?

[Sean Marion]

I have indeed. And I do.

[David Spark]

Two crappy environments. I will make Andy answer first. I always like it when you disagree with Andy, but no pressure.

[Andy Ellis]

I don't.

[Sean Marion]

Oh, raise the bar here.

[David Spark]

All right. This comes from jaydance over at StubHub. He's given us lots and lots of good what's worse scenarios. Here you go. Scenario number one. You have a breach and your security vendor tells you that they had a problem and they're missing some of your logs sporadically and randomly throughout an eight hour window during the breach. Okay? So your ability to investigate is going to be a little messed up to say the least. Or your security vendor tells you they failed to log two to three weeks worth of security logs. Now, no mention of a breach here. Nobody has reported anything concrete. The leach you to believe that there was a breach during that time, but you were receiving anecdotal evidence from it of weird behavior in the environment. Makes you very suspicious. Andy, which one's worse?

[Andy Ellis]

Ooh. So what we're really doing here, let's ignore the security vendor. I actually think that's a red herring, but I really like you bringing that one in, Jay, because that's fascinating. So the question is, I have a breach that I can't fully investigate, which means I have a breach. You can never fully investigate a breach. I don't think anybody has an environment where they have every log they want post breach. Or maybe I've got a potential breach, and again, I can't really easily historically look for the evidence of one coming in.

[David Spark]

And by the way, there's far more that's lost in the second scenario, right?

[Andy Ellis]

And there's a bunch of stuff that's lost. But let's just be honest, how many people have security logs that they never go look at unless there's a breach? The breach is what matters here. Like, the logs are important, but the breach is what matters. So given my choices, I've got a breach that I know about, or I've got a potential breach I don't know about. Like, this is now very challenging because I might be spending a lot of time chasing down this possible breach, or I might walk away and not bother chasing it down, but I've got a breach. Ooh, this is a hard one, Jay. So I actually want to give Jay credit because I could argue both directions. So I have to figure out which way I want to argue to try to convince Sean not to disagree with me.

[David Spark]

And by the way, I want to get a little credit on this. I went back and forth with Jay a few times to make this a little more even.

[Andy Ellis]

Yeah, no, this is a really good one. I think I'm actually gonna go with the known breach is worse. Cause very clearly you have a breach and you don't have all of the logs related to the breach. I'm not saying the other one is like a better. There's not a category thing of one is way better than the other.

[David Spark]

No, but it's the what's worse. So you're saying the first scenario is worse cause it's a known breach.

[Andy Ellis]

Yeah, I don't want to have the breach like the other one. I'm gonna go investigate. I'm gonna go send my people to go look real time. Maybe it gives you an opportunity to clean stuff up, but that's. I'm not allowed to take the serendipity of. Well, this can force some cleanup because that's not part of what's worse. I'm just gonna go with the breach is worse.

[David Spark]

All right, Sean, I throw it to you. These are two very difficult choices. Which one do you choose again? Which one's the worst one?

[Sean Marion]

Yeah. Yeah. Well, so I'll go door number two. Andy's right, though. Like, you could argue both sides of this fairly easily. You gotta. Is ignorance bliss. Sometimes I think if I go with door number two, though, I can at least try to understand a little bit more because I have a little bit. This is the scenario I've got. I'm missing two weeks of data, Right. But I've got real time, recent data.

[David Spark]

Yes.

[Sean Marion]

I can at least see what's going on right now. I have no historical data to go back, but I can at least see what's going on, do a little real time analysis. So I still think in both scenarios it sucks, but in my perspective, I can at least see what's going on live, get a little bit more data with option two. Option.

[David Spark]

Okay, you're agreeing with Andy on this one.

[Andy Ellis]

So you're saying option two is worse? No, no.

[David Spark]

You said option two is better because you can look at the recent date.

[Sean Marion]

Yeah. This is like a punch in the face or a kick in the butt. Is it worse? They both suck.

[David Spark]

No, no. But you're saying the first one's worse because the second one at least you can look at the more the current data.

[Andy Ellis]

Right. So I think he agreed with me though, then.

[Sean Marion]

Yeah. So May. Oh, Andy and I are agreeing.

[Andy Ellis]

Yes, he agreed. We're not.

[David Spark]

But hold on, Wait, wait a second. There's nothing that says you can't do that in the scenario number one because it says.

[Andy Ellis]

No, but scenario number one, we know.

[David Spark]

We were breached, right?

[Sean Marion]

Yeah. And we're missing some data. So it's like.

[Andy Ellis]

What's like. That's why the log the sporadic logs doesn't really matter in scenario one. It matters in two more than in one.

[Sean Marion]

And what's worse, like having partial information or no information? Partial information can be a red herring too. It can lead you down the wrong path. So that's why, I mean, either one is rough.

[Andy Ellis]

Right. We're really judging the probability of the breach.

[David Spark]

Okay. Again, but just confirming you're saying number one is the worst because just you're agreeing with Andy here, unfortunately.

[Andy Ellis]

And so score keepers mark that one at home as a win for Andy and a loss for David. But really, actually, honestly, much as David says these are losses, it's actually, I think it's a win for David that we argued this one.

[David Spark]

I think. No, I think it's a win for Jay Dance. Cause this was a good.

[Andy Ellis]

Yeah, Jay, amazing. I would like buy you tickets for something, but you're the source.

[David Spark]

Yeah, he's a sit stubhub.

[Sean Marion]

And it's not that far fetched what he described either.

[Andy Ellis]

No, no, these are actually real scenarios. I think the reality of scenario one is that's every scenario. Like you never have all the logs you want to have.

[Sean Marion]

Yeah.

[David Spark]

All right. Great job, Jay Dance.

Would this person be a good fit for the job?

For the practical question of hiring around cyber crisis experience, the answer is counterintuitive, but ultimately simple. Hire for crisis experience. But beware the onlookers. Recent research has shown that for cybersecurity incidents in particular, decision makers that don't hold a stake in the crisis are the most at risk to latch onto real world parallels to learn potentially misleading lessons from. According to a piece by Christopher White on CSO Online, those indirect response are most likely to see the event as full of unique variables and thus pretty idiosyncratic. Now, with that in mind, we're going to play another quick round of what's Worse. So what's worse? You're hiring a CISO who's been through five major crisis incidents. Three turned out very well and two that were very, very bad. Or you hire a CISO with the same number of years of experience but has never had to face a critical incident. I throw this one to you, Andy. Which one's worse?

[Andy Ellis]

So I want to say only five.

[David Spark]

I'm just throwing that out. And I'm saying critical, like monstrous incidents.

[Andy Ellis]

Yeah, I've lost track of the number I've had to deal with that were monstrous incidents. So a lot's going to come down to. So I'm hiring somebody who has never managed a crisis. So I need to understand what's the likelihood I'm dealing with crises? Like, if I'm a technical, planetary scale business, crises are a way of life. And so I'm gonna go, probably gonna go with the person with the crisis experience. If I'm not, I'm kind of less concerned about it because, yeah, crisis will happen maybe at some point, and we're all gonna learn through it. What's really gonna matter for me is I want the narrative of those five incidents. Like, I got somebody who can come in and do they have a credible narrative about how they run crises, how they learn from crises, how they deal with what I call instant authority syndrome? A lot of people think of this as the Dunning Kruger effect. By the way, other soapbox, Dunning Kruger effect is bogus. Research at Dunning and Kruger are the only example of the Dunning Kruger effect. I can do that later if anybody wants to catch me on Twitter or LinkedIn, make me explain. But what you do see is people who have bad models, but they're experts, they're really good at one thing, but they've got models in that one thing. They come to something nearby and their model doesn't really apply, but they don't know enough about it. And that's what people often look at of, oh, these people aren't good, but they don't know how not good they are. No, they know how good they are. They just don't realize they're in the wrong world. That's what I want to try to detect. Like, were the category failures on those two incidents the CISO's fault, or was it an organizational problem? Because sometimes it is the organization and not the ciso.

[David Spark]

All right, Sean, you're nodding your head and let me just set you up again. I think Andy brought up something was very interesting for the first ciso, the one that's had five bad incidents, three that went well, two that were disastrous. Finding out the process. The thing is, in the second scenario where you haven't faced a bad incident, do you ever have a process? Can you even tell one? Or maybe you create a scenario and say, how would you handle this? I don't know. What do you do with that second person?

[Sean Marion]

So I agree. Andy started talking about 5 crisis. That's actually a fair question. Because no CISO graduates college or whatever the case is, and the next day they're a CISO. They've been working their way through various ranks. After 10, 15, 20 years, if you haven't had a single incident, I'm more.

[David Spark]

Curious of where you're looking again, single massive incident.

[Sean Marion]

Yeah. Okay, even single massive, but even then. So I take a different perspective here. Sort of. It depends on the role. So people like Andy, me, some others, we're not as common in the industry. We've been CISOs for many, many years, but the industry is extremely exploded. And so you have a lot of deputies, number twos that maybe haven't been in the role that I think are ready for that big job, but may not have had the experience to manage through that crisis. But I think it depends on the company too. Like, to Andy's point, like, if I pick on a, I don't know, Google, like something massive, I'm probably not going to put a CISO there who's not had some experience managing through a crisis. I really want to understand the failures. Like, what did they learn from that? You can learn a ton from a failure. If I'm running make all this up like some local credit union, right. Much smaller. That's a pretty extreme examples. And I've got a deputy who's never lived through that, but is maybe up for the opportunity. Well, that's how they're gonna cut their teeth. That's how they're gonna get strong. So I think it really kind of depends. I mean, yeah, I've had my fair share of major incidents. I wouldn't wish that on anybody. I think most of us have. But I did learn a ton from it. So there is value there. But I think it depends on the company and the role.

[David Spark]

Can you respect a CISO that's been in the job for a while that hasn't faced a major incident?

[Sean Marion]

Oh, yeah, yeah.

[Andy Ellis]

Oh, absolutely. What are they doing to keep the incidents at bay? Like, they might have be an amazing preventative builder who creates a program that keeps incidents at bay.

[David Spark]

Shouldn't they be more applauded if you describe it like that?

[Andy Ellis]

Yeah. No, I want the people who build things that keep us safe rather than the people who get us back to safe. But we need both of them.

[Sean Marion]

And I think you could argue too, you've got. I'm taking all the extremes here, right. Maybe they've had their back to the example earlier. Maybe they've had their head in the sand. They're just not paying attention. So that's why they don't see a breach. It could go either way and it's hard to judge that. I'm looking at the experience they've got where they worked to Andy's point, like, what have you built? You've built a really good vulnerability management program like that's not the sexiest thing. That's pretty boring, but it has a huge impact. Talk me through that. Maybe that's why you haven't had a big breach. So I don't know that for me having 1, 2, 3, 5 major breaches or 0 would be a limiting factor. It would be a talking point, but it wouldn't be a limiting factor.

[David Spark]

Well, I more want to lean on the one who's had none. And let me ask you, is this even realistic what I'm describing that does there exist a CISO who's never faced a major incident?

[Andy Ellis]

I don't think there's anybody with CISO experience in the multiple years category who's never faced an incident like they might not have been on the front page of the Wall Street Journal. I'm never happy when my incident hits that level, but it has. But there's a lot of incidents that are major for the company they're in that are not major for society. Not every company when they have a failure is going to break the world. You should be thankful for that. It's a lot less stress there. The real question I want to ask and comes to something Sean did, so it's really asking for our listeners, which is if you're a ciso, are you putting your team in the roles of being able to run maybe not your worst cyber incidents, but the ones that are one step down because that's where they're going to learn the skills to be able to handle themselves when it becomes a major incident in their next job.

[David Spark]

That might not have been the best decision.

A tar pit idea occurs when someone sees something, something that hasn't been done and remains unsolved and tries to take it on. The problem is the reason there is an opportunity. There is because everyone who has tried it before has failed. Invading Russia is the ultimate tar pit idea, with the single pane of glass coming in as a close second. Ross Hallelujk on Venture and Security put together some classic tar pit ideas that cybersecurity startups seem to keep falling for. In a recent blog post of his now, he listed the classic quote only tools CISOs will need to do their job. Dashboards that would fall under the single pane of glass too. The better detection and response tools, perfect DLP and the self serve security Marketplace as a few examples. All right, Andy, this. I'm sure you see this as being as a vc, so I want you to jump on this one first. What are some of the seemingly good ideas that always seem to become Cybersecurity tar pits. They just like they're going to. And I'm sure you've seen plenty.

[Andy Ellis]

Yes, I have so many of them and it's the question of which ones do I want to go with today? So I'm going to use two. So one is real time tprm, third party Risk management. The number of times I see people who come in and they're like, well, I'm going to build a startup that'll give you a real time view of your vendor cybersecurity. And I'm like, have you ever been a vendor? Have you ever talked to one? There's not a single one that wants to expose a real time view to their customers or literally anybody else. There's a tarpit. Technically you could go solve this. Practically, you can't. That is certainly going to be one. And then I think the second one that I'm running into a lot lately is the AI SoC. People are like, oh, we'll just replace our sock with AI agents. And I have a really hot take here, which is nobody actually knows what value the SOC provides. So providing it gooder and harder is not necessarily the solution.

[David Spark]

Well, the AI socks that I've heard of, which many are actually of our sponsors, is not a replacement of the SoC, but a reduction of the level one activities in the SoC.

[Andy Ellis]

Yeah, that's the replacement of the SoC. If when you're looking at it from what the company wants, they're all messaging it softly because you don't sell somebody on, we're going to fire your whole team. Who's going to help with that one? But the real challenge is, is that in most places, right, socks are really triaging into three categories. One category is, oh my God, we have a breach, like get humans involved and solve it. Like we have a really big problem. Category 3 at the bottom is, oh, we have a procedure for this. We should have just automation go solve this and the human bridges it. And that is a place for AI and really it's for automation. What people really want is like chatgpt on top of something like Torque or Tines. But then the middle category is the stuff that the business isn't going to fix anyway. And so it's going to get stuck in the log jam of organizational dynamics. And AI is never going to solve that problem.

[David Spark]

All right, Sean.

[Sean Marion]

I think of, and I hope AI can solve this problem by the way, because we've been looking at it for years. But my favorite acronym in the security world is dlp, primarily because we get the executives, the cio, the CEO, somebody goes and meets with Microsoft or some big provider and they walk you through what DLP could be and not what it is, but what it could be. But they pitch it as what it is and they come back and like, this is going to solve everything. This is going to keep all. It's data loss prevention. It's in the name.

[David Spark]

The name sounds great. It does sound good.

[Sean Marion]

It does, but it's data loss notification. It's like, oh, there it went, there it went. If I think of AI and some of the possible, maybe I don't think it's going to get it right, but I think it can get a lot better. So maybe that's an example of a tar pit. It's been a tar pit for what, 20, 30 years, that maybe there's an opportunity there. But I'm also. I got scars, man. I'm still trying. I don't know. I'm cautiously optimistic, but that seems like a tar pit.

[Andy Ellis]

The one place in the DLP space that I've got some optimism and full disclosure, I've got a company in that space is the use of AI to do categorization. Because categorization has been the thing that has killed every DLP project for the last 30 years. You say, oh, this is an amazing project. We'll make sure that your categorized data doesn't go anywhere. And you're like, well, who's going to categorize it? You're like, well, you're going to. And I'm like, people are creating content faster than I can do anything to categorize it. I'm going to fail.

[Sean Marion]

Yeah. I think you and I might be talking about the same company, Andy, because I met with them.

[Andy Ellis]

Oh, yeah. Awesome.

[Sean Marion]

Yeah. Yeah. And when I. When they walk, that's exactly what they're talking about, right? Is how do we do. How do we categorize at scale.

[Andy Ellis]

Yes.

[David Spark]

Which.

[Sean Marion]

Which, okay, that's why I say I'm cautiously optimistic.

[David Spark]

Well, and actually, by the way, I've used AI for categorization and for identification as well. And it is. Actually does a pretty good job, I've seen. So, I mean, like, I'll tell you just one super simple example. I took a bunch of photos of the sponsors for an event and I fed that into ChatGPT and it goes, just type these all out for me, all the logos that are on it. And it did it perfectly. It was great.

[Andy Ellis]

Oh, no. I love a whole bunch of the use cases for AI, but you have to understand what the output is of the thing you're handing to AI before you hand it to AI and like, oh, I want to use AI to replace low skilled labor. I think there's a huge opportunity for that in a lot of the workforce. The I'm going to use AI to solve my organizational problem that I don't even realize I have an organizational problem. That's where to me that becomes a tar pit.

[David Spark]

You're calling these all tar pit. Let me ask you, are you still though optimistic? I'll take the one of the real time third party risk management. Are you optimistic it could ever change? But the, but like the way you described it, it's like, no, that's kind of a human saying, no, it's never going to change.

[Andy Ellis]

No. Yeah, no, no. TPRM is a huge problem and there's going to be lots of niche solutions in a bunch of places around it. But fundamentally solving it by this some magic real time. I know all the risks of every vendor in my ecosystem. The moment the risk happens. No, like never going to happen. Like the worst thing for a vendor is having a customer who spends more time looking at your data in real time than you do. Because they're going to ask you things about that you haven't even seen. Like, why would you have somebody calling you up saying why haven't you patched the system yet? And you're like, what are you talking about? Like, vulnerability came out three days ago, but you haven't patched. That's a real conversation you would have with somebody with real time visibility into your patch status.

[David Spark]

All right, so then let's close on this question. It could be. Well, it's not going to be the real time third party risk management, but it could be. Dale Pull. Which of these Tar Pit ideas are you most optimistic are going to get out of Tar Pit?

[Sean Marion]

Sean, dop. Yeah. And I think, I know this is an anomaly given the role that we have, but I'm an optimistic person by trade. I don't know that it'll be perfect, but I do think we will get much closer to what we thought we'd have 20, 25 years ago with like Vantu and those things. I think we will get much, much closer. Perfect. We're not going to get per. Anytime you introduce the human element, you introduce a tre amount of complexity. But if we can get closer, I'm pretty good with that.

[David Spark]

Andy, what do you think what's going to come out of Tar Pit?

[Andy Ellis]

Well, definitely I'm optimistic on dlp. Obviously I made an investment there. I'M optimistic on non human identity management that has been a massive tar pit for a long time that most people don't even realize existed as one because we were so careful about trying to talk about that. I'm also actually really optimistic. The self service security marketplace depending on what people think of that as but I look at trying to do security hygiene in complex environments like AWS is such a disaster. So the ability to self serve and say, hey, I just want this set up sanely. I think there's some things coming down the road that might make that better.

[David Spark]

Awesome. Well, that brings us to the very, very end of this show. I want to thank Sean Marion who is the VP and Chief Security Officer. He's dealing with with both physical and digital over at Xcel Energy. Sean, I'll let you have the very last word here, but I want to thank our sponsor and that'd be NOMA Security. Secure your entire data and AI lifecycle. Learn more how they can do that because there's a lot involved in it. Go to their website noma. N O M A Security. Just NOMA Security and you'll end up there. Sean, any last words for today's discussion? Maybe a callback to your open and. And are you hiring over at Xcel Energy?

[Sean Marion]

Yeah. So I can't stress enough the importance of networking. In fact, I've shared this with him, but I have a picture of Andy And I about 20 years ago at an event in Florida. I think we both had a little bit more hair, but Amelia island, if you remember. I'll send it to you, andy. Amelia Island. CSO 50, I think it was.

[Andy Ellis]

Oh, yeah, I remember that one.

[Sean Marion]

You were at Acoma, I believe.

[Andy Ellis]

Yeah.

[Sean Marion]

It just shows how these times go around. So. So lean on your friends. Make friends more than just colleagues.

[Andy Ellis]

Yeah. Was that the one that I brought beer to Bob onstage?

[Sean Marion]

You did, yeah.

[Andy Ellis]

That was the fun one.

[Sean Marion]

So it's been a while. So I appreciate the friendship and the partnership over the years. You know, on the plug side, we're going to start wrapping up pretty heavily at Excel, so look for my profile on LinkedIn. You'll see we got jobs posting. Always looking to find, I'd say exciting individuals. I always hire aptitude and attitude over anything. So I would love to see and if I can make a personal plug. Today is 28 years. I celebrate my anniversary today. So super happy about that.

[David Spark]

Ah, congratulations.

[Andy Ellis]

Congratulations.

[Sean Marion]

Thank you.

[David Spark]

Well, I want to thank our audience. We greatly appreciate your contributions to the CISO series podcast and for you listening.

To the CISO Series Podcast that wraps up another episode. If you haven't subscribed to the podcast, please do. We have lots more shows on our website cisoseries.com Please join us on Fridays for our live shows Super Cyber Friday, our virtual Meetup and Cybersecurity Headlines. Week in Review this show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved, including recording a question or a comment. Comment for the show. If you're interested in sponsoring the podcast, contact David Spark [e-mail] thank you for listening to the CISO Series podcast.