CISO Series Podcast – Episode Summary

Episode Title: I Don't Just Guess About Effectiveness, I Make Educated Guesses!
Date: November 4, 2025
Hosts: David Spark, Andy Ellis
Guest: Sarah Madden, CISO of Convera

Episode Overview

This episode of the CISO Series Podcast brings together hosts David Spark, Andy Ellis, and featured guest Sarah Madden to discuss the evolving challenges and strategies in building effective security programs. The panel explores greenfield security design, the shifting landscape of GRC (Governance, Risk, and Compliance) amid AI advancements, automation’s role in pen testing and vulnerability management, and the difficulties in measuring and proving security effectiveness. The tone is practical, candid, and filled with insightful anecdotes from seasoned practitioners, with humorous interludes to keep the conversation lively.

Key Discussion Points & Insights

1. Greenfield IT and Security: Designing for the Real World

[03:54–10:40]

• Main Theme: If given a clean slate—how should one architect IT security?

• Human-Centric Security: Andy Ellis emphasized that relying on users to “do the right thing” is a blind spot; systems must be designed around natural human behavior.

  “Humans actually are the people who provide value in your business... If you say, ‘I’m going to put a security control in your way, take an extra 30 seconds every time,’ they’re going to at some point be like, ‘this is stupid.’”
  – Andy Ellis [05:14]

• Zero Trust Done Right: Andy’s take on Zero Trust involves authenticating the person and the device as a pair, simplifying the user experience.

• Sarah Madden’s Experience: She describes building a greenfield security program at Convera, which allowed her to centralize tools, avoid legacy system bloat, and create clear processes for tool requests and usage. Centralization streamlines security while letting users get what they need efficiently.

  “Control is the name of the game and it’s productivity versus control...we don’t have toolset bloat, we have clarity on what we need to manage.”
  – Sarah Madden [07:24]

• Handling Acquisitions: Andy’s approach was to replace an acquired company's IT stack immediately to avoid supporting decades of merged tech, while Sarah highlighted the visibility benefits of modern cloud tooling for managing legacy assets in new environments.

2. AI, Automation, and the New Face of GRC

[10:54–15:49]

• GRC Transformation with AI: With automation spreading into compliance roles, GRC (particularly the compliance “C”) shifts from manual monitoring/enforcement to engineering preventative, automated controls.

• Job Evolution: Sarah Madden notes moving team members from audit-focused roles into “GRC engineering,” directly automating policy enforcement through platforms like AWS.

  “We are shifting the role of somebody on my team...into the UI of AWS and the native tool sets...for automating policy in config and starting to actually click automation buttons.”
  – Sarah Madden [11:43]

• From Compliance Policing to Controls Engineering: Andy Ellis underscores that most GRC is just compliance work; he pushes for true “controls” where violations become technically impossible, not just forbidden on paper.

  “We need to move to a world that is control-systems based, where when you say you have a control, it is impossible for it to happen in a different way—that’s literally what control means.”
  – Andy Ellis [14:02]

• Practical Impact: Both agree that automating controls makes everyone happier and less reliant on retroactively slapping wrists for mistakes.

3. Automated Security Scanning vs. Human Red Teams

[24:15–29:21]

• AI Native Security Tools: New AI tools can detect business logic bugs in code that traditional scanners miss, forcing a re-examination of what is possible to automate.

• Limits of Automation: Sarah is cautiously optimistic—excited for internal automation (e.g., vulnerability management in SDLC), but notes awareness gaps in business logic might persist until AI matures.

  “I still think there’s so much of the business logic that a red team has that can’t be automated yet, but I don’t think we’re far off.”
  – Sarah Madden [25:01]

• The Practical Line: Andy suggests most pen test work (“running vuln scanners, break in, look around”) is automatable; exceptional humans are needed only for niche, complex cases.

• Business Model Idea: Andy jokes about "external washing" internal pen test results to appease compliance demands for third-party reviews.

4. Measuring Security Effectiveness (and Admitting Our Blind Spots)

[31:09–36:32]

• Hard to Prove a Negative: The insurance industry, with every incentive to find effective controls, often struggles to determine what doesn’t work.

• Why Security Metrics Lag: Andy highlights the lack of standardization in digital environments—unlike houses or cars, every IT environment is a “special snowflake,” making actuarial science difficult.

• Cyber Insurance as a Catalyst: Sarah finds cyber insurers ask better questions than regulators, driven by financial necessity, but both are years behind real-world attack tactics.

• Root Cause Analysis Gap: The panel agrees the hardest (and often overlooked) part is methodically analyzing incidents to address underlying problems—requiring strong communication as much as technical skill.

  “The change management part of security is the harder part.”
  – Sarah Madden [36:32]

Notable Quotes & Memorable Moments

• “[Zero trust]: for me, the core…is actually taking a laptop and associating it with a human and authenticating the pair of them…seamless authentication continuously.”
  – Andy Ellis [06:17]

• “Trust is not a control. This is something I say all the time in my role. Our job is to design security controls that limit user impact to create an actual massive security incident.”
  – Sarah Madden [06:56]

• “We need to move to a world that is control-systems based, where when you say you have a control, it is impossible for it to happen in a different way—that’s literally what control means.”
  – Andy Ellis [14:02]

• “Back to trust is not a control. People don’t want to fail controls…The more you can automate them out of making mistakes, the happier they are.”
  – Sarah Madden [15:34]

• “If you say, ‘I’m going to put a security control in your way…they’re going to go around the task. That’s not the human doing the wrong thing, that’s the human doing the right thing from the business perspective.”
  – Andy Ellis [05:14]

• “I will give [cyber insurers] credit—they are better at asking important questions as it pertains to the threat landscape…because there’s a vested interest.”
  – Sarah Madden [34:30]

Fun & Light Moments

Earthquakes & Cockroaches

[00:29–03:45]
The show opens with a comedic riff comparing Andy to a cockroach (tough to eliminate), which segues hilariously into stories about living through California earthquakes and using pool waves as earthquake meters.

“What’s Worse” Game: Russian Chatbot vs. Clown Security Awareness

[18:04–24:08]

• Debate: Is it worse to outsource security awareness training to a Russian chatbot or have clowns jump out every time someone clicks any link in an email?

• Consensus: Clowns are worse due to the disruptive fear and panic, with the chatbot’s errors at least mitigatable by other technical controls.

• David cheekily tries (and fails) to convince the group of the viral branding opportunity from clown-prank videos.

  “Would we all laugh at it? Yes. Is it a useful thing to do? Absolutely not.”
  – Sarah Madden [22:31]

Key Segment Timestamps

• Greenfield Security Discussion: [03:54–10:40]
• AI’s Impact on GRC: [10:54–15:49]
• Automated Vulnerability Management & Pen Testing: [24:15–29:21]
• Measuring Security Effectiveness/Cyber Insurance: [31:09–36:32]
• Memorable Game Segment (“What’s Worse”): [18:04–24:08]

Closing Thoughts

Sarah Madden encapsulates the spirit of the episode:

“I think sometimes it’s hard to step out of your bubble and have conversations with colleagues and there’s so much we can learn from each other, if not just a therapy session. I think we’re all fighting the same fight every day...I would just encourage everybody in the defender space to just keep talking to each other.”
– Sarah Madden [37:26]

Andy Ellis adds to the good-natured vibe:

“They think group therapy with the whole industry.”
– Andy Ellis [38:19]

Takeaways

• Design security for real humans, not just ideal processes.
• Automate controls wherever possible—move compliance from audits to engineering.
• AI and automation will majorly change the roles and effectiveness in both GRC and vulnerability management.
• Proving “what works” in security remains an elusive challenge—root cause and clear metrics are needed.
• Don’t underestimate the power of practitioner-to-practitioner learning and group “therapy” among defenders!

For more episodes and discussions, visit CISO Series.