AI and cyber practicum [CISOP]

Summary

CyberWire Daily – "AI and Cyber Practicum" [CISOP]

Date: December 9, 2025
Host: Kim Jones, CyberWire/N2K Networks
Guest Expert: Tony Goda, VP of Cybersecurity Architecture at Intuit

---

Episode Overview

In this unlocked episode of "CISO Perspectives," host Kim Jones and guest Tony Goda explore the profound impacts of rapid AI innovation on enterprise cybersecurity. They discuss both the benefits and operational risks of integrating AI, challenge preconceptions about automation vs. accountability, and debate what it takes—culturally and organizationally—to operationalize transformative AI in a security context. The episode delivers practical insights for CISOs facing pressure to "do AI" while managing risk, liability, and organizational culture.

---

Key Discussion Points & Insights

1. AI's Promised Benefits and Underlying Risks

(01:30 – 06:39)

• AI's Benefits (as seen by optimists):

  • Productivity: Automation of mundane tasks, freeing employees for creative work.
  • Customer Experience: Personalization at scale via generative AI.
  • Analytics & Insights: Faster, deeper pattern recognition and anomaly detection.

  “AI can automate mundane tasks, allowing employees to focus more time and energy on more complex and creative tasks, thus improving the overall productivity of an organization.” — Kim Jones (01:44)

• Risks & Challenges (protector's perspective):

  • Data integrity and normalization are persistent obstacles.
  • Data governance lapses could lead to data poisoning or leaks.
  • Difficulty recognizing bad or “BS” AI-generated results; AI can confidently generate wrong answers.
  • Accountability quandaries: Who is responsible for AI errors?
  • Infrastructure and budgeting strains.
  • Lack of realistic cost/benefit analysis as organizations rush to be seen as “AI enabled.”

  "Whereas my friend is an innovator, my nature is that of a protector... I focused on the challenges with operationalizing AI within any environment." — Kim Jones (03:05)

• Warning Against Hype-Driven Adoption:
  Many enterprises are “leaping into implementing something, anything actually, just to say that they are AI enabled,” mirroring the past Agile adoption rush.

  "I'm almost reminded of the 'I need Agile' phenomenon a decade ago." — Kim Jones (06:07)

2. Introduction to Tony Goda and Perspectives on Innovation

(07:01 – 14:02)

• Tony's Background:

  • Engineer-turned-innovator with deep roots in cybersecurity (fraud detection, cryptography, insider threat, startups, Intuit).
  • Experienced at bridging business needs with technical innovation.
  • Emphasizes people management and the challenge of motivating teams for transformative outcomes.

• Innovation vs. Invention:

  • Invention is novelty, but innovation must be useful and operationalizable without harming culture or protection.

  "The difference between innovation and invention is both are new, but innovation is actually useful." — Kim Jones, quoting Frank Kim (12:46)

3. AI is a Paradigm Shift, Not a Mere Incremental Upgrade

(14:02 – 15:49)

• Traditional Model vs. AI-Centric Model:

  • Existing systems = humans at center, tech as helper.
  • Future vision: AI at center for repeatable, automatable tasks; humans oversee, fact-check, govern.
  • Challenge: Rethink problems and solutions entirely, not just “bolt on” AI for incremental gains.

  "What I think we should be doing is fundamentally rethinking the approach... put humans into a position in which they are in some cases fact checking what the AI is doing." — Tony Goda (14:30)

4. Accountability vs. Automation – The CISOs’ Dilemma

(15:49 – 21:16)

• CISO’s Concerns:

  • Accountability doesn’t shift; CISOs remain liable (reference to SolarWinds case).
  • The conundrum: Handing over control to agentic AI raises stakes for liability if errors cause harm.

  "You're still gonna attempt to throw my large butt in jail... How do I balance that as a CISO?" — Kim Jones (17:06)

• Tony’s Rebuttal:

  • AI is not a magical infallible force; it needs checks, especially for high-stakes decisions.
  • Lessons from other fields (e.g., self-driving cars/Waymo): Only deploy full trust when technology is mature; introduce staged, controlled use as safety grows.

5. How to Operationalize AI Safely in Security Operations

(21:21 – 27:08)

• Tiered, Automated SOC:

  • Automate “Tier 1” (routine) security operations—not by dictating details but empowering teams to define and execute.
  • Give room to experiment, even to fail; psychological safety is key for learning and progress.
  • Human "double checks" remain in play as guardrails.

  "What you want is for them to have the psychological safety to actually go out to find the tools... and to just double check what [AI's] doing." — Tony Goda (21:16)

• Cultural Impediments:

  • Most orgs don’t genuinely support experimentation (fear of failure/career impact).
  • Business leadership expects near-perfection from cybersecurity, an unrealistic standard compared to expectations in physical security.

  "If the expectation for perfection in security in the IT space existed in the physical space, we would expect a murder and kidnapping and theft rate to go down to absolutely zero... which is unrealistic." — Kim Jones (26:24)

6. Who (or What) Is Accountable? Risk Management in AI-Driven Defense

(27:08 – 34:06)

• Humans are Fallible, Too:

  • Three SOC analysts might classify the same alert differently.
  • AI isn't perfect, but neither are people; both need oversight.

  "Humans themselves are some of the most indeterminate, non deterministic things that exist on the planet." — Tony Goda (27:27)

• Checks & Balances, Not Blind Trust:

  • Accountability = “checks and balances” via layered oversight—either human or AI.
  • Example: Limits on what agentic AI can do, agentic “peer review” before high-stakes action, human in-the-loop for exceptions.
  • No unrestricted destructive actions for AI (e.g., code or database deletion).

  "You should not give it the ability to delete databases or to... commit code to your repository that is unreviewed." — Tony Goda (33:09)

7. Moonshots, Not Marginal Gains: The Mindset and Organizational Challenge

(34:13 – 39:44)

• Setting Transformative Goals:

  • Don’t get stuck on incremental improvement targets.
  • Frame the AI journey as a “moonshot”—set ambitious, outcome-focused objectives and delegate tactical details to empowered teams.
  • Provide psychological safety and resources for experimentation—even at the risk of (some) failure.

  "Set the goal, don't necessarily set the tactics... empower them to make the types of decisions that need to be made." — Tony Goda (36:40)

• Empowering the Team:

  • Trust expertise at the front lines; they know where improvements are possible.
  • Expect transformative, not just incremental, change.

  "Trust your team. Empower them. It's literally all about... not 10% improvements, but 10x improvements, 100x improvements." — Tony Goda (38:57)

---

Notable Quotes & Memorable Moments

• On AI hype and risk:

  “Most organizations are whitewashing risks and costs and leaping into implementing something, anything actually, just to say that they are AI enabled." — Kim Jones (05:58)

• On the folly of AI infallibility:

  "AI is not a magical infallible force; it needs checks, especially for high-stakes decisions." — Tony Goda (21:16–22:38 summary)

• On accountability refusing to disappear:

  "So what we're talking about is a future that is inevitable... I don't think the accountability shifts at all." — Tony Goda (17:57)
  "Right now CISOs are being held liable... And if it goes sideways, you're still gonna... attempt to throw my large butt in jail." — Kim Jones (16:06)

• On fast AI, faster adversaries, and the need for speed:

  “The adversaries are using AI to traverse through our systems to find vulnerabilities at faster than human speeds.” — Tony Goda (29:57)

• On cultural change:

  "A culture that allows for experimentation is a culture that allows for the possibility of failure. Because if you experiment, not everything is going to work." — Kim Jones (24:46)

---

Important Timestamps

• AI Benefits vs. Risks Breakdown (01:30 – 06:39)
• Tony Goda’s Background and Intro to Innovation (07:01 – 14:02)
• Strategic AI Adoption: Paradigm Shift Required (14:02 – 15:49)
• Operationalizing AI – The Accountability Conundrum (15:49 – 21:16)
• Cultural Shifts and Experimentation (24:27 – 27:08)
• Checks/Balances & Guardrails for Agentic AI (27:08 – 34:06)
• Moonshots and Empowerment (34:13 – 38:57)
• The One Thing CISOs Should Do Now (38:57)

---

Episode Takeaways

• AI is both a massive opportunity and a massive challenge—approach it with eyes wide open.
• Operationalization is not about bolting on AI to current processes, but about reimagining workflows with AI at the center.
• Checks, balances, and layered guardrails—not blind trust—are essential for safe, scalable agentic AI.
• Transformative outcomes require moonshot thinking and a deep cultural shift toward experimentation and psychological safety.
• CISOs must empower expert teams to define and deliver ambitious AI-enabled solutions, balancing efficiency with accountability.

---

Final Action for CISOs (per Tony Goda at 38:57):

“Trust your team. Empower them... set the expectation that blow my mind. Like tell me where we can get... 10x improvements, 100x improvements, and you’d be surprised at the answers that you get.”

---

For further analysis, visit the CISO Perspectives blog (link in show notes).

Transcript

[A]

You're listening to the Cyberwire Network powered by N2K.

[B]

This exclusive N2K Pro Subscriber only episode of CISO Perspectives has been unlocked for all Cyberwire listeners through the generous support of Meter building full stack zero trust networks from the ground up. Trusted by security and network leaders everywhere, Meter delivers fast, secure by design and scalable connectivity without the frustration, friction, complexity and cost of managing an endless proliferation of vendors and tools. Meter gives your enterprise a complete networking stack, secure, wired, wireless and cellular in one integrated solution built for performance, resilience and scale. Go to meter.com CISOP today to learn more and book your demo. That's M-E T E R.com CISOP Foreign.

Welcome back to CISO Perspectives. I'm Kim Jones and I'm thrilled that you're here for this season's journey. Throughout this season, we will be exploring some of the most pressing problems facing our industry today and discussing with experts how we can better address them.

Today we are looking at how rapid innovation around AI can introduce unplanned risks into an enterprise.

Let's get into it I was eating dinner with a friend at the RSA conference this year. After taking the time to catch up, we began to discuss artificial intelligence and its application in most businesses. My friend is an innovator by nature. As such, his focus tends to be to look at the benefits of a new technology. First, he quickly rattled off a list of potential benefits of AI 1. Improve productivity. AI can automate mundane tasks, allowing employees to focus more time and energy on more complex and creative tasks, thus improving the overall productivity of an organization.

2. Improved customer experience with the analytic capability of generative AI, it is possible to create highly personalized and responsive customer experiences without the need for humans. At the other end of the exchange, this could potentially lead to higher customer satisfaction rates and ultimately increase sales.

And 3 data analysis and Insights Generative AI excels at data analytics, the timeframe for turning data into information, and then intelligence can therefore be shortened. Further, the depth of these insights may be potentially deeper as AI engines recognize patterns and anomalies with more efficiency.

Whereas my friend is an innovator, my nature is that of a protector. While I agreed with my friend's insights, I focused on the challenges with operationalizing AI within any environment. Environment.

1 clean, normalized data. This is a must for any AI implementation, yet is a struggle for many organizations who are leaping into the AI fray.

2. Exceptional data governance Lack of good data governance to include pristine knowledge of data pipelines can lead to inadvertent data poisoning or worse, inappropriate leakage of data. Most organizations continue to struggle with data governance, eschewing the detailed measures and approaches needed in an AI driven environment to ensure safety for fear that such measures will impede progress.

3. Recognizing bad results in 2024, an article in Scientific American described the technical term for what AI does as BSing. While respecting the author's premise, it is, I believe, a tad harsh. AI is performing predictive analysis based upon a series of inputs and is drawing conclusions. Unfortunately, just like human beings, AI is subject to error and misinterpretation.

[A]

4.

[B]

Accountability in some respects, this is an extension of the data governance concern I raised above, but I believe it merits special attention. If your AI agent, specifically in an agentic AI situation, makes an error hallucinates, who is accountable for that error? More importantly, how do we minimize the likelihood of such an error causing material issues within the enterprise?

5. Infrastructure upgrades tying AI into systems and workflows not designed for it will lead to short term and hopefully not catastrophic failures.

And six Costs IT infrastructures are accustomed to being told to do more with less. As such, more organizations will not add AI dollars into IT budgets and just expect CIOs to figure it out. This will mean trade offs. While this is nothing new, the newness of AI means that many organizations will not know or understand the nature or depth of the trade offs that need to be made until they are in midst of their AI journey.

When pressed on these concerns, my friend went into an all too familiar reiteration of the benefits. While he eventually acknowledged the potential concerns, he offered no insights as to how these risks could be mitigated.

This discussion is not atypical of some of the discussions going on within enterprises today around AI and AI usage. While all agreeing, myself included, about the benefits of AI, most organizations are whitewashing risks and costs and leaping into implementing something, anything actually, just to say that they are AI enabled. I'm almost reminded of the I need Agile phenomenon a decade ago.

Is AI here? Yes. Should we embrace it? Definitely. But do so smartly and with eyes wide open.

My two sets.

Working with a visionary thought leader like Tony Goda is one part exasperation, one part mind stretching, and three parts fun. Tony and I have had many discussions about how to take his truly insightful ideas and operationalize them in a way that doesn't break an organization or its culture. We sat down to have one of our typically spirited discussions around innovation as pertains to all things AI.

A quick note that the opinions expressed by Tony in this segment are Personal and should not be interpreted as representing the opinions of any organization that Tony has worked for, past or present.

So, you know, you and I have known each other for three years now, but my audience may not be familiar with you. So tell my audience about Tony Goda.

[A]

Oh my gosh. First of all, it feels like I've known you for 30.

[B]

Yeah, it's like that.

[A]

Yeah, I'm an engineer, been doing software development all my life. My dad had a computer consulting firm, brought me in at a pretty young age and told me, hey, I can touch every computer in this organization, but I can't play games on any of them. So what that meant is that I had to just, you know, figure out ways to entertain myself. And that meant, I guess, building things to entertain myself with. And I've been doing that pretty much ever since. At some point I ended up kind of working at. I got into kind of cybersecurity because he actually had. The computer consulting firm actually was a fraud prediction system for the cellular industry. So I was kind of indirectly involved with kind of building fraud prediction systems. And then kind of later in life I found my way to MasterCard where I kind of helped design the first generation of AI powered, actually credited debit fraud detection systems. Did that for a few years. And at some point I bought a MacBook Air, ran out of space and decided, hey, I can solve this problem, the storage problem, with software. So ended up quitting my job, moving out to the Valley on a whim and start and being a startup CEO for a company that I, that I founded on some technology that I built at the time. Built that for probably five years, left that company, started another company, was the CEO of that for another five years, spent a decade in the Valley, just kind of being an engineer, slash kind of CEO, which allowed me to better understand the intersection between business and technology actually how to do people management. Because remember, people are a lot less deterministic than software is.

[B]

Well, really they are.

Shocked.

[A]

I am the current generation of people. And we'll talk about that later actually, because, you know, if we didn't even get into the agentic people yet just.

[B]

Yet, we'll get there, right?

[A]

But, you know, really learned how to, you know, kind of, you know, motivate people to kind of do the best work of their lives. And you know, and both those startups actually were cybersecurity adjacent or cryptography, you know, kind of related. One was a storage company, the other was an insider threat detection company. So then took a bunch of time off to kind of Catch up with family, because I was basically, you know, traveling a lot, you know, had young kids at the time and, you know, just wanted to spend some time with them. And then, you know, I got a random LinkedIn request from someone at Intuit who, you know, kind of reached out and said, hey, you know, you've got a lot of, you know, kind of external startup experience. Looks like, you know, you're. You're impatient, you know, kind of with your, you know, in your expectations. And you also, you know, or, you know, it seems like you're pretty innovative, you know, you haven't been, you know, maybe I guess, indoctrinated by a larger organization just yet. So we'd love to bring you in so that you can kind of be that architect for the next generation of cybersecurity technologies that we need to kind of build to solve for the problems at Intuit Scale. So that's my current position. I'm the vice president of cybersecurity architecture at Intuit. Been there for about almost three years now. Really enjoying the job. I walk in every day and ask the question, hey, is this the right thing that we're doing? And if it's not, let's change it. Let's figure out what the right thing is and kind of help drive that strategy across the company.

[B]

Fantastic. So the nature of our relationship began, and I won't talk about Austin, because that's a different story, but.

We don't talk about Austin. We were talking about Bruno. We don't talk about Austin. But.

The nature of our relationship has been. And that's really what I wanted to bring here is when we met, you coming in as an architect, and I'm probably using the term slightly incorrectly, but, yeah, I'm a simple guy. So you have a futurist outlook and look at what can be and how to project truly from a strategic vision as to what we ought to be thinking about. And I've been an operational CISO for 17 of my 38 years in cyber and was running the SOC for Intuit when we met. So a lot of our relationship is a. Yeah, great, wonderful fan, fantastic. That's going to break all this stuff now. How do you want me to get there without breaking stuff? And this led to more than a few very, very engaging conversations about, yeah, I want to go forward. I don't want to just invent, I want to innovate. And I had a dear friend, Frank Kim, tell me that the difference between innovation and invention is both are new, but innovation is actually useful. So I want to innovate out in the market and I want to help support the innovation. But how do we get there in a way that doesn't blow up the culture, blow up our protective protection posture, et cetera? And you and I regularly have conversations about this and did for three years. And as I started thinking about the theme for this season and thinking about the dreaded AI, and yes, we're going to use, if you're playing AI bingo, trust me, we're going to use it a lot here. But what should the cyber professional be thinking about in terms of where his or her business is probably looking at AI? But more importantly, how do I, as a cyber professional, utilize AI to create efficiencies, to create excellence, to optimize what I'm doing within the environment and to better keep people safe? So, Tony, what I'm going to do in front of a thousand or so people is have the conversations you and I regularly have had for the past three years and talk about, tell me what your vision is and then let's talk about how we get there in a way that makes sense, yet doesn't needlessly slow things down. So just talk to me, what you know. I am a ciso. What should I be thinking about as I look at AI in the future and utilizing it within my organization? Floor is yours.

[A]

Yeah, I think the challenge is, actually, I think the challenge is, is, is, is massive. But also think the opportunity is, is just as massive. I mean, we're, we're at a, we're at a, we're basically at a crossroads not only within cybersecurity, but across every industry in which we traditionally have thought about things incrementally. So it's like, hey, we've got this existing kind of process, we've got this existing kind of technology. We're going to upgrade it a bit. It's going to get a little bit faster, it's going to get a little bit more accurate, it's going to kind of help us be a little bit, maybe 5, 10% more operationally efficient across an organization? And I think that there's a lot of organizations that are thinking about AI in a very similar capacity. And I think that actually is a mistake. What I think we should be doing is fundamentally rethinking the approach that we have for all of these types of problems that we're actually solving for. Because if you think about it today, the problems, the solutions that we have in place are basically, we've got humans at the center that are being augmented with technology that can help them, you know, kind of get to some outcome within some organization. But if you were to fundamentally rethink the problem, to have AI at the center to do a lot of the redundant, you know, kind of repetitive tasks, the things that we know are automatable, or the things that have some level of automation to them and then put humans into a position in which they are in some cases fact checking, fact checking with the AI is doing or giving the ability to, you know, to, to help govern an AI centered system versus augmenting a human based system with AI, I think that opens up the possibilities to much more autonomous kind of systems which could solve for the types of problems that I think we're going to face in the future, which are much faster than human speed.

[B]

So I've got to push a little bit, and I know that's shocking to you, but I've got to push a little bit. You've said a couple of things. Well, the smaller portion regarding creating efficiencies through automation is something that every good business leader, that includes every good cyber business leader, because every CISO runs the business of security within their organization, is always attempting to do. But reflecting back, you're talking about flipping the model and making it automation centric for a lot, or specifically AI centric within the environment. The challenge that I have with that, and first, not that I disagree with you because I think there are plenty of opportunities there, but the challenge that I end up having there right now is that's great until you run into that nasty a word accountability. And right now CISOs are being held liable. The CISO, SolarWinds is still under indictment right now having spent millions of dollars due to something in a human based system. Now you're going to tell me to turn this over to agentic AI, basically, for lack of a better term, you know, in the environment. And if it goes sideways, you're still gonna, you know, you're still gonna attempt to throw my large butt in jail. So how do we balance that? You know, agreeing that we could do more. How do I balance that as a CISO in terms of, okay, you want me to divest a level of positive control within the environment over to the automation to drive certain decisions, not analysis, but decisions yet what does that do for our liability engine within the environment? Because you know as well as I do, your boss currently and their boss currently aren't going to let you off the hook for that. So how do we create an ecosystem that allows a CISO to do that in this highly litigious society where two CISOs have been placed on trial and one has been found guilty.

[A]

Yeah, I don't think the accountability shifts at all. I think. So what we're talking about is a future that is inevitable. Like, I mean, AI will power a lot of the systems that today are being powered by.

[B]

Okay, so I'm going to cut you off and I'm going to push back on you on that. Telling me it's inevitable is not something I disagree with. Okay, but telling me how to do it is. And again, this is the conversation. We had this over at a steakhouse in rsa.

[A]

I was going to recount that story. I literally was.

[B]

And I agree with you. I'm not talking about the inevitability. Let's assume that we're heading there. The issue is the how, which is the same conversation we had. You know, I am sitting here pointing out that we have to create a how that enables the person who is that responsible charge to take that step. That requires an organizational structure change potentially. It requires issues regarding liability and accountability. It requires potentially decisioning regarding how AI impacts that within the environment. So my push for you. And again, this is nothing new. You and I do this all the time is okay. Telling me it's inevitable is telling me the sky is blue. I'm not a Luddite, and I agree with you because I want to innovate. What steps do I need to get there that are going to allow me to protect the company as well as myself? That's what I'm asking.

[A]

So, Kim, listen, totally get it. And I think one of the issues is that people think that AI is some magical kind of genius that's 100% right all the time. And they're putting it in positions where, you know, if it's wrong, then it can be catastrophic to the organization. Now, you know, I'm a huge fan of, you know, kind of Waymo and the technologies that they've got and kind of how they've taken this extremely difficult problem of navigating, you know, a city street with a life inside of it, you know, safely across, you know, across, you know, even some of the most busiest streets in America. Now, what I'm never going to do is take a Waymo up some cliffside California highway. Like, that's not a thing I'm going to do until this thing becomes way more stable. But at the end of the day, there have been checks and balances that have been put in place to make sure this thing actually does, you know, react in such a way that can't be that it's not as that is not a catastrophic to a human life.

[B]

Should we be prepared to change organizationally to enable that? Those are the questions that I want to push you towards.

[A]

Yeah. So if you think about, you know, kind of what happens today, you know, you've got a security operations center, you've got some person sitting in front of a, you know, from a, you know, PC or some terminal, they get an alert, they respond to that alert. You know, I mean, in all, you know, frankness, a lot of these things today are either outsourced to, you know, to other organizations or we're doing a lot of the tier one things with, with automation. So if you think about it, setting a goal within your organization where you want to automate a lot of your Tier one operations, so not determining kind of what those are, but letting your discrete teams decide what Tier 1 actually means, and then empowering them to be able to go out and make the technological decisions to put these things in place, but also giving them the room to experiment and to actually fail in some cases. Because at the end of the day, what you want is for them to have.

The psychological safety to actually go out to find the tools to train the users, to expect.

The first responders, to actually to take the systems that are, or to take the cases that are being generated by the automated system and to just double check what it's doing. So it doesn't necessarily have to be this person is making a decision about a particular incident that's happening. It's double checking what the system recommends as a course of action. And that in itself allows you to automate, I mean, which I think is an actual standard operating procedure and a lot of more advanced kind of cybersecurity organizations today.

[B]

Have you ever imagined how you'd redesign and secure your network infrastructure if you could start from scratch? What if you could build the hardware, firmware and software with a vision of frictionless integration, resilience and scalability? What if you could turn complexity into simplicity? Forget about constant patching, streamline the number of vendors you use, reduce those ever expanding costs and instead spend your time focusing on helping your business and customers thrive. Meet Meter, the company building full stack zero trust networks from the ground up, with security at the core, at the edge, and everywhere in between. Meter Designs deploys and manages everything an enterprise needs for fast, reliable and secure connectivity. They eliminate the hidden costs and maintenance burdens, patching risks and reduce the inefficiencies of traditional infrastructure, from wired, wireless and cellular to routing Switching firewalls, DNS security, and vpn. Every layer is integrated, segmented, and continuously protected through a single unified platform. And because METER provides networking as a service, enterprises avoid heavy capital expenses and unpredictable upgrade cycles. METER even buys back your old infrastructure to make switching that much easier. Go to meter.com CISOP today to learn more about the future of secure networking. And book your demo. That's M-E-T-E-R.com CISOP.

You know, as we think about this automated soc, so, you know, reflecting back, it requires culturally, an environment.

A culture that allows for lack of a better term for failure and experimentation.

[A]

That's exactly right, yeah.

[B]

So to experiment, we have, and I'll use that term, a culture that allows for experimentation is a culture that allows for the possibility of failure. Because if you experiment, not everything is going to work.

1000% agree. You and I have had this conversation on more than one occasion. I absolutely agree with you.

[A]

And that's actually extremely important because otherwise you, because us as literally risk owners, the first thing we're going to say is, I don't want to trust this thing, like my job is on the line.

[B]

And by the way, I own no risk. The business owns the risk. I just see. Yeah, yeah. And you know how I feel about that one. But.

So.

As we sit here and create that culture, I guess the question that I would ask is culturally. And we're going to get. We're not going to get metaphysical, but we're going to get beyond just soc issue here. Tony, Culturally, security has become. There has become an expectation of security, of lack of failure within most environments. You know, I have said in other venues that if the expectation for perfection in security in the IT space existed in the physical space, we would expect a murder and kidnapping and theft rate to go down to absolutely zero across the country, which is unrealistic. Yet our business customers, our CEOs, our COOs, our CTOs, et cetera, expect that nothing is going to go wrong within an environment and want us to drive to zero. And anything less than zero is considered problematic. How do we.

Change that mentality if we need to create a culture of experimentation? I would say that I am not familiar with a lot of Fortune 500 organizations that do more than talk a game regarding experimentation within the operational arena.

[A]

I think what we're talking about is a reframing of the risk of the risk that we're already accepting. Because if you give three different SOC analysts the same alert at different times of day, it will get classified three different ways. We Talk about humans as if they are infallible, but in their current environment they're extremely fallible. Humans themselves are some of the most indeterminate, non deterministic things that exist on the planet.

[B]

Absolutely, absolutely true. But I can hold the human accountable, and based upon conversations we've had, it can't necessarily do that with a generated AI. No, but I, at least not yet.

[A]

So I think if the question is if we expect to hold the AI accountable, I think that's not, I think that's not the argument that I'm making.

[B]

What I'm saying is that you want me to accept a level of error. I mean, you and I were just talking about this earlier from an operational standpoint. There was a recent report regarding a AI coding tool that wiped the production database, fabricated 4,000 users, and then actually lied in terms to cover extracts within the environment. I would argue that the code review may exist, but the code review, even if done by an agentic system, is, it is not perfect. Humans are not perfect, machines are not perfect. So the challenge here gets to be there's a point where we create systems of accountability as much as a method of check and balance. So as I roll to more agentic AI systems here within the environment and turn more of these processes over, it's not that humans themselves aren't infallible because they are absolutely fallible, but they can also be held accountable. So you want me to turn over more systems to make more decisions where I can't create that level of accountability?

[A]

No, I think so. It's not. So, yeah, we hold humans accountable by performance reviews, by looking at kind of how they, how well they work with each other. But with agentic systems, what it should be are checks and balances that could be either humans checking and make sure that the agentic system is making the right decision, or other agentic systems that are put in place to validate the decisions that are being made.

[B]

Okay, so you're talking risk mitigation. So not mitigation, risk reduction, risk management management, which will hopefully create some level of reduction. So using the NIST AI risk management framework, you would want to create systems that create human in the middle type interactions. So which makes sense, right?

[A]

So it's not necessarily. And what we're not looking for is something that is infallible, but we're looking for something that gives us a lift in efficiency, something that allows us to have much faster response times. Because remember, the adversaries are using AI to traverse through our systems to find vulnerabilities at Faster than human speeds. So if the expectation is that we get an alert, we take a look at it. In the meantime, between detection is now, you know, 30 minutes or whatever it may be. Let's say it's five, let's say we even get it down to five. Imagine the amount of damage that an agentix system or an AI powered adversary can do in five minutes to an organization that could be catastrophic.

[B]

Take that same argument regarding an agentic system that is responding inappropriately within an organization. Imagine the amount of damage they can do. So you're absolutely right. But you know what, I'm. And I guess the question here that I'm having is yeah, putting a human in the middle is great, but if an agentic System is reviewing 15,000 different actions within the space of 10, 10 minutes and taking 15,000 different actions within that time frame, I can't keep up in terms of that review other than sampling. So we're still in the environment where I'm depending upon folks to make those decisions with a high level of autonomy, which again and again, Tony, just for the sake of our audience, I want to reiterate where I started. You or I are in violent agreement that this is going to occur. And you know this is happening. But what I'm trying to get to is, you know, we're still in an environment where I'm looking down and staring down the barrel of that gun and I don't, you know, and I don't have a solution other than to continue to do what I'm doing at a pace that can't keep up within the environment and still accept the liability associated with that. And then I'm told, not as a security professional, I shouldn't be resisting this because it's coming. Yet you don't have an answer for me.

[A]

It's not even coming, it's already here.

[B]

But again, you don't have an answer for me. Yeah, you don't have an answer for me if I actually, at the pace you tell me is coming, tell a system to make automated decisions and it deletes tons of code within the environment.

[A]

Yeah, I think not having the right checks and balances within any system, even a human powered system, is probably a very risky decision to make.

[B]

Agreed. In your sidestepping because you still haven't told me what the right checks and balances are at speed. How do I do this at speed?

[A]

It is a standby agentic system that is double checking the decisions that are being made. So think of it as a consensus based system where it has the ability to determine if this agentic system that is given a limited set of functionality, so it shouldn't be, hey, you can do whatever you want, which is what we typically give people the ability to do, is to make whatever decision they want and to affect lots of systems. We can set guardrails for an agentix system to say, hey, you can kind of do these three things and then we'll put a secondary system in place to validate that this is the appropriate action to take. And if it doesn't fit within a certain requirement or doesn't fit within a certain set of parameters, there is an exception flow in which a human in the loop can actually decide if this is the right thing to do. But none of these should be destructive. You should not give it the ability to delete databases or to, or to put or commit code to your repository that is unreviewed, that's not reviewed.

[B]

The challenge that I have here as I go to operationalize these within the environment is not just checks and balances, but how do we put appropriate checks and balances on the system that's moving faster. You did answer that in terms of a sort of check and balance AI system within the environment, as well as limiting the access to what these agentic systems can do, at least initially. Until you can build up the knowledge, you can build up the trust, you experiment within that environment and say, oh, that went wrong. Let me actually open this up within the environment. That makes sense to me. I am still struggling, hey, how do.

[A]

I as an organization, like how do I actually operationalize this? How do I tell a team to actually achieve this goal?

[B]

Less that from the, from the technical standpoint, but the things that you're talking about, which you did bring up, Yes, I was actually trying to poke at you to get Tony to come out.

It's always more fun when you're here, brother.

I don't want to gloss over the piece that is necessary for this as you have laid out. And that is not just a mindset shift, but a cultural shift within the environment. It is a mindset shift that says we have to experiment within the environment and that we have to be able to experiment without failure being a significant career impacting event within the organization. So I guess where I'm trying to get to is as someone who has been in cyber.

For not a short time, probably not as long as I am, because I'm an old guy.

How do we empower our constituents, meaning the businesses we support, to allow us to create that mentality? Because gotta tell you.

It'S not there as deeply as it ought to be for this level of transformative approach.

So that's. What are your thoughts there?

[A]

For me it is. So I think about all of this in the same way that a startup would, because if you think about it from the traditional kind of sense, which is again, incremental improvement, I think you're going to get 5, 10% kind of efficiencies. I think what you do is that you set an expectation that is well beyond what is possible with our current systems. You say, hey, within six months, between six, eight months, maybe 12 months, let's give it the old Enterprise 500, let's say in an FYI, right, let's say an FY26, we're going to achieve a level of autonomy in Tier X responses, period. You don't talk about the tactical, you don't define what Tier 1 is, what Tier X is. You allow the team to make that decision, but you give them the resources that they need to actually execute against that. So set the goal, don't necessarily set the tactics for how to achieve it, but then empower them to make the types of decisions that need to be made. Maybe that's bringing in an external, you know, a system that we haven't had in the past. Maybe that's bringing in, you know, a, which could be a startup, it could be a well established, you know, kind of providers, you know, kind of soccer sim tool. But whatever the case may be, empower that organization to actually make that decision and give them the budget and the psychological safety to achieve what I like to call a moonshot. Give, like, encourage them to deliver on this. And if, let's say they don't hit the moon, let's say they speak, you know, they get halfway there, that is a, that is a tremendous increase in the capacity of the organization and something that you can absolutely talk out to your executive team. Like at the end of the day, there has to be, you have to set the tone that a transformative, you know, kind of result is what is expected, which means they're going to rethink how they're doing everything.

[B]

Yeah. So it takes from reflecting back appropriately, it takes both a mindset and a willingness on the part of the business to take that moonshot to achieve that.

[A]

Level of transformative success and characterize it in that way. And that will galvanize the team. So it's not just, hey, let's do this thing that's a little bit better. Hey. If you literally go to the team, say, hey, if you had a budget and you wanted to achieve some massive transformational change within your organization, what would that look like? And what support would you need from me to allow that change to happen?

[B]

So let me ask that one question, then. Let me give you a final word. Let's answer that question for me. Okay? You're a ciso. The business wants AI. The business wants you to adopt AI. The business doesn't know necessarily what adopting AI means for you, et cetera, within the environment. If there's one thing, one thing that I could do today as a ciso, who's starting that journey, what would it be?

[A]

Trust your team. Empower them. It's literally all about. You've got experts that have that know exactly where the areas of improvement exist within the organization and if given the opportunity, they will absolutely blow your mind away. Just set the expectation that blow my mind. Like tell me where we can get some so, you know, not 10% improvements, but 10x improvements, 100x improvements, and you'd be surprised at the answers that you get.

[B]

And that's a wrap for today's episode. Thanks so much for tuning in and for your support. As N2K Pro subscribers, your continued support enables us to keep making shows like this one, and we couldn't do it without you. If you enjoyed today's conversation and are interested in learning more, please visit the CISO Perspectives page to read our accompanying blog post, which provides you with additional resources and analysis on today's topic. There's a link in the show Notes.

This episode was edited by Ethan Cook with content strategy provided by Mayon Plout, produced by Liz Stokes, executive produced by Jennifer Ibin, and mixing sound design and original music by Elliot Peltzman. I'm Kim Jones. See you next time. Episode.

Securing and managing enterprise networks shouldn't mean juggling vendors, patching hardware, or managing endless complexity. Meter builds full stack zero trust networks from the ground up, secure by design and automatically kept up to date. Every layer from wired and wireless to firewalls, DNS security and VPN is integrated, segmented and continuously protected through one unified platform. With Meter, security is built in, not bolted on. Learn more and book your [e-mail] CISOP that's meter.com CISOP and we thank Meter for their support in unlocking this N2K Pro episode. For all Cyberwire listeners.