CISO Series Podcast – “AI Confidence: It's a Trap!” (LIVE in San Francisco)
Podcast Hosts: David Spark, Mike Johnson (CISO, Rivian), Andy Ellis
Guest: Sarah Madden (CISO, Convera)
Event: BSides San Francisco 2026
Date: May 5, 2026
Episode Overview
This live episode, recorded at BSides San Francisco 2026, dives deep into the challenges and opportunities of artificial intelligence (AI) in cybersecurity, vendor selection pitfalls, the realities of incident response simulations, and the evolving nature of technical conferences. With a lively audience, the panelists debate the “AI confidence trap,” reflect on security leadership mistakes, dissect the difference between technical and people problems, and champion a culture of continuous learning and cross-functional collaboration.
Key Discussion Points and Insights
1. Security Leadership: Lessons from Mistakes
- Sarah Madden on “I told you so” Moments
“It’s not our jobs to say I told you so. It’s our jobs to manage risk and never have that happen.” (00:03)
- Biggest regret: bending to business pressure against better judgment.
- Focus on managing risk over proving oneself right after the fact.
2. Vendor Selection: Avoiding Long-Term Traps
-
Common Pattern:
Evaluate a few vendors, pick the best demo, sign a long contract, then swap out mid-cycle due to unforeseen changes (mergers, better tech, etc.).
-
Mike Johnson’s Strategy (04:49):
- Don’t sign 3-year deals on first engagement:
“The first step is don’t sign a three-year deal the first time that you’re working with a vendor, period. Period, full stop.” (04:49)
- Leverage peer CISO networks to shortlist vendors.
- Build clear requirements, conduct thorough proof of concept.
- Accept occasional unavoidable bad outcomes but mitigate risk with careful initial steps.
“At the end of the day, you can’t predict the future.” (06:34)
-
Sarah Madden on Negotiating and People Power (06:59):
- Got locked into three-year deals during greenfield buildout, but now shifts renewals to 1–2 years for flexibility.
- Stresses the need to air grievances and leverage renewal threats to push for better vendor quality.
- Calls out SaaS instability and vendor trend towards shipping lower-quality releases.
- Importance of matching tools with internal competency:
“It’s as much about finding the right tool as it is making sure you have the skill sets on your team to manage that.” (09:00)
“If you don’t have the talent to manage it, you can have the greatest toolset ... but it’s not going to work.” (09:16)
3. The AI Confidence Trap
- Quote:
“Gen AI is deceptively complex. It gives you the false impression it’s simple to use and delivers excellent quality ... This is the trap.” (09:32)
- Statistic: 74% see productivity gains from AI, but only 11% see clear ROI.
- Sarah Madden’s Approach:
“We don’t trust it. We use AI every day.” (10:38)
- AI’s results are useful for efficiency (especially in SecOps) but always require verification.
“We question your output every time ... It likes to be flashy and dramatic.” (10:44)
- Human-in-the-loop remains critical.
- Mike Johnson:
- Evolution in team behavior: no longer blindly trusting AI output.
“Six months ago ... [people] would just blindly believe them. But nowadays, everybody understands you need to double-check.” (11:48)
- Draws parallel to false positives in traditional security tools.
4. What’s Worse? – Security Team or Technical Debt
- Scenario 1: Security program depends on “a few very exhausted security heroes” – if two leave, everything collapses.
- Scenario 2: Huge technical debt with unmanaged accounts/tokens/identities.
- Mike Johnson’s Take:
“People problems are harder than technical problems ... if you lose your people, you can’t fix it.” (17:20)
- Sarah Madden:
“The reason you've got unmanaged service accounts is because you only have three people.” (17:25)
- Agrees: losing key people is worse, as it makes technical issues unfixable.
5. Getting Value from Security Conferences
- Take a strategic approach to conferences: don’t just network, seek out people with advice relevant to your challenges.
- Mike Johnson’s Guidance:
- Have a plan and purpose before attending.
- Return value:
“Bring something back to the team—a debrief, a writeup, links to talks.” (27:23)
- Sarah Madden:
- Send only those eager and likely to learn/share.
- Follow up with brown bag sessions to share knowledge with the team. (27:56)
6. Injecting Realism into Incident Response Tabletop Exercises
-
Central Problem:
In most exercises, real-world messiness is missing (authority, escalation, political friction).
“The breach won’t expose your controls. It will expose who is allowed to act without permission.” (28:38)
-
Sarah Madden:
- Quarterly tabletops—growth comes from repetition, trial and error.
“Every time, my feedback is: you didn’t bring me in early enough.” (29:57)
-
Mike Johnson:
- Perfect simulation isn’t possible, but frequency and external facilitators up the stakes.
“We’ve actually brought in an outside firm ... It does change the stakes because they have less familiarity ... and you spent money on this.” (32:14)
-
Cross-functional benefit:
Sarah highlights major leaps in cross-team learning over time. (32:59)
7. Audience Q&A / Speed Round
AI Adoption, Agentic AI, and Vendor Displacement
- Jensen Huang (Nvidia CEO) Quote:
“If that $500,000 engineer did not use at least $250,000 in AI code tokens, I’m going to be deeply alarmed.” (36:09)
- Panel Reaction:
- Conflict of interest (Nvidia sells the hardware).
“He wants everyone to be spending that kind of money.” (36:14)
- But agree engineers should use AI to boost productivity.
- Measuring AI Usefulness:
Sarah:
“We’re thinking about how to measure effectiveness of AI ... You have to look at goals for AI as a business.” (37:15)
- AI displacement of vendors is already happening: evaluating every renewal against in-house AI capabilities.
“Every time we renew a software license, we ask ourselves, can we replace this with AI?” (39:12)
Advances in Agentic AI (38:21):
- Sarah: Faster adoption, improved accuracy, better reinforcement learning.
- Mike: Shift towards orchestration of multiple agents (“master control agent on top”).
Concrete AI Use Cases
- Mike:
TPM with no coding background used AI to automate part of their workflow.
“Everybody can augment themselves ... It’s very empowering.” (40:30)
- Sarah:
Compliance team builds bots to automate audit responses, displacing vendor tools.
- AI experimentation is now broad across roles, not just engineering.
8. Notable Quotes and Moments
| Timestamp | Speaker | Quote |
|-----------|---------------|---------------------------------------------------------------------------------------------------------------------------|
| 00:03 | Sarah Madden | “It’s not our jobs to say I told you so. It’s our jobs to manage risk...” |
| 04:49 | Mike Johnson | “The first step is don’t sign a three-year deal the first time that you’re working with a vendor, period.” |
| 09:32 | Quote/Host | “Gen AI is deceptively complex. ... This is the trap.” |
| 10:38 | Sarah Madden | “We don’t trust it. We use AI every day.” |
| 17:25 | Sarah Madden | “The reason you’ve got unmanaged service accounts is because you only have three people.” |
| 27:23 | Mike Johnson | “Bring something back to the team—a debrief, a writeup, links to talks.” |
| 29:57 | Sarah Madden | “...my feedback 100% of the time is, you didn’t bring me in early enough.” |
| 32:14 | Mike Johnson | “An outside party does help raise the stakes [in tabletop exercises].” |
| 36:09 | Jensen Huang | “If that $500,000 engineer did not use at least $250,000 in AI code tokens, I’m going to be deeply alarmed.” |
| 39:12 | Sarah Madden | “Every time we renew a software license, we ask ourselves, can we replace this with AI?” |
| 40:30 | Mike Johnson | “Everybody can augment themselves ... It’s very empowering.” |
9. Audience Games and Moments
- “What’s Worse?”: Team fragility vs technical debt—audience overwhelmingly agrees losing core team is worse. (18:00)
- “What is Dave’s Mom Talking About?”: Audience repeatedly outsmarts panel on deciphering cybersecurity terms from “mom logic.”
- Memorable moment: “See, this is why we all need to work together.” – Sarah Madden (22:37)
Timestamps for Key Segments
- Security mistakes and “I told you so” – 00:03–00:36
- Vendor selection pitfalls & strategies – 03:50–09:16
- AI Confidence Trap & Team Strategies – 09:32–11:44
- What’s Worse Game – 14:18–18:00
- Conference strategies & ROI – 25:10–28:33
- Tabletop exercises & injecting realism – 28:38–33:53
- Audience Question Speed Round (AI-focused) – 35:16–41:57
Takeaways for Security Practitioners
- AI’s confident outputs demand skepticism and human review.
- Vendor lock-in can be toxic—prioritize flexibility and talent fit over contracts.
- Security programs are most fragile when dependent on too few, overworked heroes.
- Conferences are an investment; maximize by planning and sharing learnings.
- Regular, cross-functional tabletop exercises drive real incident response improvement.
- Agentic AI continues to accelerate—with benefits and displacements felt across roles.
For more wisdom (and some laughs), tune in to the full episode or subscribe at cisoseries.com.