← GRC & Me
GRC & Me
CISO to CISO—Let's Get Real About AI
00:44:54
The hype is real, but what’s the reality? In this episode of GRC & Me, we’re getting real about AI with Anecdotes CISO, Jake Bernardes, and LogicGate CISO, Nick Kathmann. They discuss the implications of the key differentiators between AI usage/gov...
Loading summary
Transcript47 lines
- [00:00]
A
Foreign.
- [00:09]
B
Hi, everyone, and welcome to this episode of GRC and Me, a podcast designed to break down the complexities of GRC and turn them into practical strategies that we can use every day. I'm Jane Tataro, and today we're tackling a topic that's top of mind for for organizations of all sizes and industries AI here with me today. I am excited to have not one, but two CISOs anecdotes, Jake Bernardis, and Logigate's very own Nick Kathman. Jake and Nick are here to discuss all things AI, from the EU AI act to the rise of agentic AI, shadow AI, the risks of siloed governance, and so much more. So CISO to ciso, let's get into it. Jake and Nick, welcome to the show. Thanks so much for joining today. To kick us off, let's start with an easy question. What is one thing that is not on your LinkedIn that we should know about you? We'll start with you, Jake.
- [01:15]
C
That's actually not that easy. I live very publicly on LinkedIn, so it's quite hard to say what's not on there. I'm actually. So I found out I was claustrophobic a few years ago by going in Ezekiel's tunnel in Jerusalem. When I stepped into the tunnel and realized that I am actually quite claustrophobic and I'd never done this before in my life. That is one of the rare things that is not on my LinkedIn
- [01:38]
A
might be the exact opposite. I don't post on LinkedIn practically ever at all. So, I mean, maybe more of a. More of a secret to people. But I think one of the things about me was, I don't know, I'm a geek, and I've been a geek at heart since I was a kid. So if you go into my basement now, there's a whole din rail full of raspberry PI clusters and Raspberry and Arduinos and let's call it the beagle bones and rabbit cores and a whole bunch like MCP430s and ESP32s. So I have a whole bunch of, you know, just hobbies that are techie and geeky outside of cybersecurity. But then it always, cybersecurity somehow always finds its way into that. Like that Raspberry PI cluster right now is running a Kubernetes node so that I can mess around with Policy Agent and like, experiment with that and do some learning with that. So cybersecurity always finds its way into it. But, you know, in general, I like embedded Programming and a lot of other things that have nothing to do with security or the tech that we're in right now.
- [02:38]
C
So very cool.
- [02:40]
B
Two different sides of the spectrum for that question. So that's fun. Especially Jake, hearing, hearing that tidbit. I mean, that's a crazy way to find out that you're claustrophobic. So love to hear what's not on the LinkedIn.
- [02:57]
C
Cool.
- [02:57]
B
Well, thanks for that. Let's jump into our first segment here, which is called GRC mythbusters, where guests try to debunk common myths and misconceptions in the GRC space. So, Nick, we'll start with you. Myth or fact? All AI is the same.
- [03:17]
A
Absolutely a myth. If you look just across all of the different models, whether they be proprietary or whether they be open source, the differences in them is so extreme. I'll give you some examples. If you try to put into Gemini and ask it for candidate, let's say you're looking for a certain candidate and you put in a question like, here are the criteria I'm looking for. And in these regions with these skills that has gone through this, you know, level of a business phase, things like that, it'll actually come back to you and say, this is against our acceptable use policies. Whereas ChatGPT will just go straight through and give you a whole bunch of recommendations. But outside of that, if you look at just in a lot of cases, how they operate together, like the proprietary versus or the, I would say the SaaS versions, the proprietary versions versus the open source versions, you get a lot of crazy consistencies and in terms of what they were made for. So for instance, there are a lot of models that are built specifically for coding and they do a really good job at generating code. There are other general purpose ones that are really good at parsing, you know, just like having chats back and forth with you. There are others that are really good at parsing large data sets for scientific modeling, for et cetera. So like, I like to play and look at different. So I, I'm, I like to play with LangChain and Langsmith a lot on the side, like experiment with what can I build with agentic AI with Langsmith? And one of the things that you can do with LangChain easily is abstract out the model and run it through a bunch of different models and see the results you get. And it's pretty extreme. The differences you'll get by running Gwen vs Gemini vs ChatGPT vs Olamo vs, you know, Gemma in the different versions of Gamma or Deepseek the answers and the responses you'll get and the reasoning steps they'll take are wildly, wildly different. So definitely a myth.
- [05:08]
B
Yeah, that's, that's very true. I've actually tested and with ChatGPT and other AI tools, just writing the same question from multiple different devices to see what it comes up with in multiple different people and you never get the same answer. So that's a great point there, Jake. Anything to add on here?
- [05:32]
C
I think the only thing is that I agree entirely, but I think the only real thing is how much that's going to grow in terms of a chasm, particularly from an ethical standpoint. Like, I think it was only today or yesterday that OpenAI announced they're now going to come up for profit company. They're moving away from this concept of being like for the people, by the people, and they're already on the real periphery of what is ethical, quote, unquote, in terms of you can get data from it that you couldn't get from Claud or from anything the EU or kind of more regulated. So I think the variance in terms of what a model can tell you is going to change much more dramatically than it even is now. Go back to the days of like you used to be able to. This is a weird side bit, but you still have to like take a topless picture and ask OpenAI what your body fat percentage was, which they then ethically slowly step backwards to stop being able to tell you that because of the risk of encouraging eating disorders or abnormal or dangerous behaviors. Whereas we're already seeing that like actually it's quite easy to socially engineer a model and start to push it in direction it's not supposed to go into. So I think what you'll find is that when is AI all equal? I think it's going to be a case of how protected it is to protect its regulatory stance, its ethical stance, or how malleable it is for people to play with it, prompt it and move and get through things it's not supposed to do.
- [06:42]
B
Yeah, that's a great point. And on that kind of. I know you touched on the EU for a minute there. So jumping into the next round of questions, let's talk about AI regulation and strategy specifically. Jake, this first question is for you around the EU AI act or EU AI act now that it's in full effect. So this is a multi part question. But Jake, what is the biggest challenge that CISOs face in operationalizing these new regulations and what are the first steps that they should take and Will this and how will this be different in the US versus the eu?
- [07:28]
C
Wow. It's probably easier to reverse that than is to start at the top. But like in a reverse statement, I think this is like privacy or gdpr. The US doesn't really give a damn proof be told, like they don't really care about your personal data in the us. Like the lobbies will always maintain that position inside of Congress, inside the federal government, they don't really care about personal data. I can't see a world in which the US really cares about personal data apart from it being surface level. So I think the first is it will never be the same. I think that in terms of what does these do and what how do you start. The problem with every regulation is they're super vague, right? You go GDPR is a great comparison for AI. It was released and it's things around technical security based like be adequate and sufficient. The hell does that mean? Like that's really hard to interpret to logical controls to actually apply and maintain the act. The, the AI act in the EU as well as ISO 4, 2001 and various other standards trying to regulate space are similarly super vague. They're really hard to work out what they're actually asking for. They're very much subjective, they're very much risk based. They're very much a thing where you can take an audience down a path and show them what they want to see. So they're really hard to work out. What you actually need to do second is they're really hard to monetize. I love the GDPR comparison. Like how many companies actually got fined and how many are still breaching that regulation? This is the problem. If I go to my board and say I want to spend all this money on AI security, they're going to say what's the tldr? And I'm like, well we probably in breach, but I don't know if we'll actually ever get fined or penalized for it. Then the question is, well, is it worth spending or do we take the risk? I think inside that actually also how do you show the data? Like all compliance is shown through data if you have no data. And it's very hard to trust AI as a source of data in terms of it like basically hallucinates on its own data sources. So that one's really hard. And I'd say beyond that also as well is it changes so fast. Like ISO40:2001 came out, it doesn't even reference agentic technologies. Like they came out and Suddenly it was irrelevant. MCP is not talked about in there. Like there's so much stuff in that. In AI, which is changing so quickly, it's almost impossible for the regulators, particularly those from a legal and non technical background, to write, maintain and enforce regulation around technology where people are changing it on a daily basis. I think this is probably the hardest space right now to do anything in and I'm not seeing anyone doing anything meaningful and effective from a regulation stance.
- [09:50]
B
Yeah, and kind of following on that, Nick, can you talk about the implications of implementing AI at scale and what this means for risk? So I mean, that's a great point that Jake said. I mean things are just changing so quickly. So what would, what do you have to say about that?
- [10:10]
A
Yeah, I'll just add on to what Jake said. You know, that's what I'm really, really hoping doesn't happen here in at least in the US is AI regulations become the same as privacy regulations where every state has their own privacy regulation. And next thing you know you're casing down 50 different AI regulations for different states that you're doing operations with, as well as the EU as well as the UK as well as other countries. So you know, I'd say one of the biggest risks you have, and there's a whole bunch of risks that you're going to have running at scale and especially large scale. But one of the biggest risks, at least on the regulatory, just to build on what Jake said, is if you're running at scale in multinational companies, there's keeping track of the regulations and how they're going to change and everything that's coming down the path. It's a full time job and it's a full time job for at least one lawyer, probably two or three lawyers, plus a bunch of tech people to figure out how do we stay ahead of this, how do we make sure that we aren't getting ourselves in trouble. But it's also, I 100% agree. The change is so rapid in AI right now and you always have this curve where you know, you can create new policies and standards and then it takes a while for the company to adopt them before, you know, just, just to adopt them before they actually show any type of value. With AI, it's so like, and this is usually for most companies, about a year. So you implement a new policy, you know, you do all of your, your it's evangelization, you train everybody on it and then it takes about a year to get fully ramped up and that's about as fast as most companies can move like some smart startups move faster, really large Fortune 50s move a lot slower, might take three years, but you're generally like you're around a year. AI is changing like on a monthly basis. You can't rewrite your policy and rewrite your standards every month and re educate your users every month and expect them to constantly be ahead of the game on this. It's just impossible. Um, so I think some of the biggest, you know, outside of just the technical threat vectors, you know like you know, prompt injection and tool attacks and things like that and MCP types attacks, there's a whole bunch of, there's a whole bunch of regulatory risk and just change risk that I think companies are going to deal with when trying to implement this at scale across their entire company. And another big part of this is I almost compare it to the dot com, the dot com boom when it came up. AI changes everything. So it's not just a technology that changes just engineering or just finance or just you know, sales. It has broad implications across everything. And not only that, but it has broad implications across. Not just like how what are you doing with AI? Am I just an end user of an AI model? Am I developing my own AI model? Am I developing agentic AI on models or with my own models? So there's a lot of different security avenues you have to go down based on what it is. But it's, it's fundamentally going to change every single function of the business. It's going to change finance, marketing, sales, how the executives run their business. It's going to change, you know, already has changed cyber security, engineering. So it's just so pervasive and so widespread that it's, it's going to be really hard to stay on top of.
- [13:07]
C
I think also there's like, there's a fundamental naivety in most of the CISO community. They just say that AI is just another risk. Like AI is not a risk, it's a whole new category of risks. Like it permeates revenantly. It says now thanks. That's the real risk is that people don't actually understand the risk associated to it. Nick, to your point, like, I mean that's my question. Can you see a world in which there is federated AI regulation? Because I just can't see it happening. I think it's going to be state based.
- [13:31]
A
No, I think they'll probably put something at the federal level and some states might adopt it. So think kind of like fedramp state ramp and some states, but I See, most states are going to try to implement their own types of regulation. And as we've seen from Congress and others writing anything to do with tech that they don't understand even the smallest inkling of it. So it'll be very vague, very naive, it'll be very restrictive in some cases to make absolutely zero sense. And there'll be a lot of lobbying to make sure that it's watered down is my guess on it.
- [14:07]
C
And even the state ones, it is now like the Texas one. Their definitions of AI compared to things in Salt Lake and Utah and Colorado are completely different. It's not even like the regulation barriers. They define things differently. I don't know how you even start with that in terms of trying to do that as a US entity, let alone a global entity. It just seems like we're so far from being able to actually tackle this problem, and yet it's very much here.
- [14:31]
A
Even the AI experts in the field, the subject matter experts and the guys writing the research papers and the chief research scientist for the different companies can't agree on a definition for agent and agentic. And now we're going to write regulations that are brought across all of them. And you're trying to take this into account and we're going to put the regulatory bodies in charge of that. It's, it's going to be a, it's going to be a long, bumpy road before we get anywhere meaningful.
- [14:56]
C
And I mean, not to put the tinfoil hat on, but the other problem is that the people, as you point out, when the government lacks context and knowledge, it turns to industry experts. The industry experts are on the payroll of the four largest AI companies in the US right now. Like, they're not, they're not, they're not agnostic experts. They are being paid by these companies to represent their views and values. So that's the other problem. When you have regulation being influenced by external sources that are incentivized to make one or another thing happen.
- [15:22]
A
Exactly. Yep.
- [15:25]
B
Within the same subject, let's talk about AI literacy. So, Jake, how can CISOs effectively equip their teams to understand and govern AI? And why is this so crucial for the future of risk management? I know you talk about being at the state level between the Colorado act and the Utah and Texas. So talk a little bit about how can companies equip their teams.
- [15:50]
C
Like I, I've said quite a lot publicly recently that when the CISO took it and basically, in my opinion, made the modern CIO sort of irrelevant, this is the next chance like every CISO should be saying, I want to own AI, like this is a massive risk for me, so I'm going to step or just take ownership of it full stop. So I think that's the first thing, is that CISO should be taking ownership of it as a corporate and as an entity level so they can own both the risk approach to it and how it implemented and dealt with and try and remove some of the shadow risk of it too. Then the second is being very risk aggressive. I think in AI you have to be risk aggressive inherently as a ciso because if you take the other path, people will subvert you. Like if you decide not to try and be forward thinking and try and embrace and encourage this stuff, they'll do it anyway. So you need to make sure that you can kind of wrap something around it by saying, you know what, I'm going to embrace it all, let me enable you, let me facilitate you. I think the last point, which I think is fundamental for me, but both Nick and I are nerdy tech geeks at heart, is you have to be doing this stuff. Like you have no right to talk about AI regulation in your company. If you, if you're the extent of AI goes to ChatGPT and Claude, like if you're not building your own suites with brains like notion and building zaps out and like creating workflows and playing with agent kit from OpenAI, you have no right to tell people what they should or shouldn't do. So I think firstly it's get on the front foot, get involved, embrace it, use it, understand it, own it in the business and take an aggressive attitude for risk stands to enable and facilitate because that's the only way you'll get people to follow you to any degree.
- [17:19]
A
I've seen the sidestepping in person sitting at a conference, a banking conference of all things, where banking regulators have not put any type of regulation yet around AI. So banks are very slow to adopt it because the regulators haven't given them the guidance on how they can adopt it yet. And it's still coming out. And here I am watching this, you know, bank, high level bank risk. I don't know if he was an officer or just one of the risk analysts. Literally take pictures with his personal phone of his emails into ChatGPT, have it, write the email, he would slack it back to himself and then copy and paste it back into Outlook and send it. He did this for every single email that came through. And it's like, how are you? You know, there are technologies to Detect that. But they are so intrusive to the end user and to privacy and like so. And they require huge teams to look for that. It's the employees are just going to subvert you any, any which way they possibly can. And I've even seen it at the executive and board level where you know, they'll just use whatever tool they've decided they want to use regardless of, you know, what the security implications of it are. Most of them never put it through a security review.
- [18:25]
B
We do have shadow AI on the, on the horizon to talk through. So I think some of these points are really relevant and I'd love to, to cover in a minute here. Next we're going to jump into agentic AI. So Nick, how are you seeing the impact of agentic AI in the GRC space today?
- [18:46]
A
Yeah, so I think it's, I think it's going to make, I think it's, we're still early on in the, I'd say the journey of agency AI in GRC and in security in general or even in business in general. And I'll, I'll step back a little bit and you know, kind of define agency and the different, and what I'm going to call the different levels of agency. So think about, I like to draw a lot of analogies for people who don't work in the space. But think about self driving car. You've got a whole bunch of different levels of self driving cars. You've got everything from, you know, what we've had forever, which is cruise control where you set the speed and it just goes to adaptive cruise where it automatically keeps the lanes to adaptive cruise with lane keeping assistance. And then you've got the next level ones which will actually make turns and negotiate stop signs and speed limits, stuff like that all the way up to there is no steering wheel in the driver and you have to figure out, so you think like level zero being cruise control, level five being you're getting into a waymo. There's no driver, no steering wheel, no anything like there's, there's going to be some level inside of there and some risk tolerance within there of the different processes you have within GRC and security where you say I trust this process within GRC to be at a level one. I want it to be, you know, human in the loop all the time for everything. And then I want this one to be completely level five and then there'll be some, you know, based on criteria within there as well. You know, this is a third party risk review and this Vendor is non critical to this, doesn't have any sensitive data and it's non critical to the business. I'm okay with a level 5 and it might make mistakes and declassify, you know, some business continuity risks as a medium instead of a high. It's not critical to the business operations. It doesn't matter. So there's going to be a whole bunch of like I think what we're going to start to see a bunch of agentic action starts automate out a lot of the workflows within GRC and then what we're going to, the interesting part is going to be see companies as they go in there, what level they're going to choose and what levels they're going to accept as their risk tolerances come in and go. And then you know, adding on to that, there's just a bunch of, as we start to build out these things, there's just a bunch of, let's say sprawl that's going to come in as well from it. So things like mcp, things like agentic and tools. There's going to be a lot of, a lot of the unknown unknowns. Like I don't know if anybody's ever played with agentic AI now, but I'll give you an example of one that I've been building. So I stepped back and said why do I need to, why do I need to have a policy process standard and I need to go out and I need to build something in a process automation flow. Why can't I have one source of truth that builds the other? Can I just build the process into some tool and then it writes my standard operating procedure? Or can I have my standard operating procedure write my process and then when I change one, the other modifically changes. Why do I need to have duplicate efforts here? So what I did was I trained a agentic AI to essentially give it reasoning steps to say okay, go read the policy, go read the standard, go read the SOPs and then you know, non deterministically figure out how you're going to make that process. Then go read all of the data about whatever it is that you're evaluating and then go collect all the supporting data you need from there and then make a risk decision or whatever the standard operating procedure was the process was. And I would say about one out of every five times using the same model, the same prompts, the same data, the same everything, one out of every five times it will, you know, skip reading the policy. Then one out of every three times it'll skip reading the operating procedure and then sometimes it'll just make up what all of the data about the risk is and does not do that. So it's interesting to see if you run it 100 times, you'll get 50 different outcomes of how not deterministically it's decided to come in there. So there's still a lot that we're going to get there, but a lot. I would say we're still in the the art of possible phase right now with the Gentek AI. It will disrupt things like crazy, but it's going to be where on that risk tolerance of, you know, self driving, car autonomy, we're going to allow it to operate.
- [22:58]
C
I think there's already some super exciting stuff you can do. Like this is where, well, some of the stuff weird in anecdotes I get really excited about in this space because we spent a long time building a tool where we ingest compliance and GRC data and now we're sitting like, actually we can do a hell of a lot of crap with that data. Like you look at an MCP on top of it and you start to automate your KSI for ISO. You look at being able to do automated internal order. You look at like working alongside logic gates. We do. And you look at how can you actually dynamically monitor risk. Here's a risk is the control I say mitigates that risk. Let's go agentically and actually look at the efficacy of that control currently to assess whether that's actually functioning, whether we're actually failing to mitigate our risk. You start to look at how you can make things dynamic and real. I think when you look at what GRC has been and where it's gone up to now, where we've gone very much from like spreadsheets to what I call home, the tick box security theater of green ticks. Red cross is actually now you're like, well, if you've got the data and you've got agentic capabilities, you can start to do really meaningful workflows to automate problems like VRM and the T4M space. Whether it's internal audit, whether it's the KPIs around your compliance boss, there's loads of things you can now start to do which you just couldn't do before. I think it's like, it's an exciting thing to do. But I agree with Nick. We equally see that every time you build one of these things, it it like you keep asking it, what pizza do you want? And it says I want pepperoni pizza. Like you don't want a pepperoni pizza. You're AI. You don't know what a damn pizza is. Right. You see, it takes a while to get things to actually stop losing and do what they're supposed to do, but when they do, like, there's endless possibilities in our world, in this space.
- [24:28]
B
So with that being said, Jake, how should CISOs be preparing for this new frontier of agentic AI that's still kind of developing?
- [24:37]
C
I think it starts with mcp, but I think right now it starts with model context protocol. But if you have a data set, you can put that on top and start to query the data that say, well actually on the data I've got, what can you tell me about this? Or if I want to know a what would be the result? And then on top of that you can start to build base agents to say, well if this is true, then I want this to happen, then let's make that happen. Let's monitor and see it. So I think it's right now you've got to start tinkering and start playing. There are going to be enterprise products. I'm sure we will both and lots of the people in the space will release agentic workflows of some capability or form every few months, every quarter. But I think right now you need to start looking at what you actually have because the key to success in agentic is the raw data. If your data is wrong or insufficient or incomplete or defunct, the whole thing doesn't work. I think that's for me is like right now. I see. So it's go back and understand what data you have, what data you want to have, how you're going to get the data that you want to have, how clean that data is, how many hallucinations are coming off of it, why it's hallucinating. How can you limit those hallucinations we always about right now to understanding your data and starting to play.
- [25:38]
A
Yeah, I'll add to that. It's, it's data is king, especially when it comes to AI. And I've said this time and time again, if you go to a chatgpt and say, build me a business model, it'll build you a business model. It might be a taco shack, it might be a lawn care service, might be the next Google search engine. It's, it's going to build you a business model. When you give it a, when you give it more information, say I want to build me a business model around a SaaS company trying to get to this, in this space with this, these customers and these types of products. It was, it's going to build you a much better business model. So the more data you can give it and the more accurate and sanitized that data is, the better results you're going to get. But if you don't have that data to give to it, you're just going to get really general results or you're going to get completely wrong results. Because if you think about what a lot of the LLMs do, they're really fancy autocorrects or not autocompletes. So it's just picking statistically what the next best word is going to be and then putting that in there and then using statistics to find what the next best word is from there. So the more data it has, the better it can pick the next words. If you just give it no data or bad data, it's, it's going to order you a pizza, as Jake said. But now, now that you've put an McP in a Gen 2, it's going to order you a pizza and it's going to arrive at your house instead of just giving you, you know, hallucinating that it wants to eat pizza. Um, so there's a lot, a lot that's going to come out of this.
- [26:57]
B
Yeah, very cool. All right, let's move on to shadow AI. So shadow AI is everywhere. Everyone's talking about it. Nick, why should this unsanctioned and unknown use of AI be one of the top priorities or biggest blind spots on a CISO's radar right now?
- [27:17]
A
Yeah, I would say it's, it's, it's a very hard problem solved. Every company is running as fast as they possibly can to integrate AI features and capabilities in a lot of cases without understanding AI at all. And if you think about the space of AI security experts right now, it's very small. So we don't have a lot of AI security experts. A lot of the AppSec community is still trying to catch up to what AI security is going to look like. And it's ever changing. It changes even once you learn it and you feel like you've become a subject matter expert next month, you're completely behind. There's new technologies this newer that have come along. So I think a lot of these companies are building these AI features in that it's very hard to figure out are they doing it safely and then even worse, they're all doing it opt out. So if you've got a hundred vendors that you're managing within your environment. All of a sudden 90 of them turn on AI inside the product. And you didn't go through a risk assessment of that. You didn't look at what their sub processors are, you didn't look at what their policies are related to it. And in a lot of cases like the policies will actually conflict. And we've seen AI vendors where in the state like they'll have two different agreements for their AI. One of them says they own all of the data you put in and they can train their models and use it for marketing purposes and whatever. And then another one saying that the, you know, they don't use your data, your data is yours and that it'll never be used or never be sold. And we couldn't get the company, a really large, well known company, to give us a clarification on which one was actually the stance legally. So that's just, you're constantly chasing your tail trying to figure out, you know, where what, what new, you know, what of our existing vendors has turned on AI for what use cases did they build it properly, did they actually scope limit it down so the user's permissions and auth z vols or did they just leave it wide open. And we've seen this a lot like you know, Copilot was pretty famous in this where people are going out and querying, you know, board reports that they didn't have access to because Copilot did and things like that. And we see this a lot within all the different products that are there. So I think it's shadow AI like discounting just everybody adding new, like going out and signing up for a bunch of services and using their own personal chat GPTs and their own, you know, groks or whatever that they're getting comfortable with. You know, take that, that's a problem within itself. But then those, the vendor space adding everything without telling you. And then you have to, you're always behind the curve. And then constantly, even once you get ahead, you're still behind the curve because they add more is a problem with itself. But then there's, there's just limited enforcement around this as well. So the biggest way you can be able enforce most of it is at the browser at the Internet level. But if you're which and it's easy if the product makes direct calls to the LLMs because then you can say okay, this product, my, you know, CRM is making a direct call to OpenAI, the API call for it to OpenAI. I can see that that's, that's happening and I know that, you know, they've now, our CRM company has now incorporated AI, but a lot of cases they just pass the data back to the CRM web servers or app servers and it makes the call out to OpenAI. You're blind to that. And you know, in the browser space, in the Internet security space. So there's very, in a lot of cases, it's very hard to even detect and track across the board.
- [30:32]
C
Yeah. The only thing I would add is that we all thought shadow it was the worst problem in the world. I think shadow it was a surfable wave compared to like the tsunami that is shadow AI and like tsunami, like you can't stop this, it's too late. Like the reality is now we have to work out that whatever the hell is going to destroy in our orgs, we're going to have to look at how we rebuild better, more tsunami resistant structures for the future. That's that this is not a solvable problem. It's a how do you rebuild afterwards? It's too late already.
- [31:02]
A
And I'll say a lot of, a lot of executives have zero appetite to restrict it. So it's a, I think there's a lot of pressure within the industry and within business leaders to fully embrace and I would call it run with scissors when it comes to AI. So I think it's really hard to get the tone from the top unless you're in extremely regulated industry like financial services. Be very, very hard to get that tone from the top, to even try to control it.
- [31:30]
B
All right, let's move into the mind of the ciso. So going to ask you both a couple questions here. Nick, beyond the daily AI headlines that we're seeing day and night, what is keeping you up specifically in regards to AI at night?
- [31:49]
A
I think it's the, I referenced the lack of AI security experts. I think it's, you know, every company is building them as fast as possible and if you look at some of the capabilities, so like, you know, AI, so like if you're attacking an API, you control everything, every flow about the API and say, you know, you can only come into these endpoints but you have to be authenticated this way. You have to be controlled this way. With AI, it's so deterministic and then you add things like protocols like MCP on top of it where you can then like do tool injection or like do tool poisoning and say, you know, so thinking of like some of the attack vectors that come in and that are going to be very hard to mitigate and that we don't have a lot of security knowledge around and staff around is, you know, I'm going to call, I'm going to come in for something and where it was supposed to go out to a tool to check the weather. I'm now going to command prompted to go to and run a command for remote code execution on my server. And I'm going to use one of the MCP tools that's available there. But I'm just going to tool poison to get into there. And there's a whole bunch of ways to do this. Like I was reading one the other day where it was an internal red team was testing some stuff and you know, they had put in a bunch of protections around prompt injection. But what they was once they got into the multimodal mode, I don't remember how many people remember from, you know, their CISSP or whatever they took way back in the day of steganography where you were like, so if anybody doesn't know, steganography is. It's like hiding like messages inside of images. So you're like changing certain pixels in the image and from a person you can't see that anything was actually changed because the pixel is so small. But computers can actually still read that message. Well, what we learned is some of the AI models actually read that steganography and you can put prompt injection into there. And since a different tool, a different processor manages the images and multimodal versus text, the controls didn't apply. So they were able to do tool injection, do prompt injection, completely get around every control that was there just by putting in a gif. And then when they found ways to get around that, they could do the same thing by putting in a PDF and they were able to get around that by putting in another image, a JPG that was white text on a white background. You can't see it as a person. And we saw this as one of the attacks against the. I forgot which one. It was one of the IDE attacks with a VS code VS code attacks where it added a whole bunch of black text on black background or white text on white background. So the person code reviewing it would never see this attack coming in, but would never see the text that was there. But AI sees it, the computers see it and actually evaluate it as code. So I think there's a lot of really novel attacks. There's a lot of. There's a lot of the changing attacks and there's just not a lot of experts. And then, you know, once you start incorporating things like MCP where you know you can add new functions without ever knowing like the third party provider that's providing the side can incorporate a whole bunch of new functions that are now tools that are available in your tool state stack. It's going to make it very very interesting and very very hard to build meaningful security around AI for a good while.
- [34:50]
B
Jake, do you have anything to add here in terms of what's keeping you up at night? Are you experiencing the same thoughts as Nick or where's your some.
- [35:01]
C
So yeah I'm eternally an optimist so more what keeps me up at night is I keep thinking gosh like when the techies is think of a thing I could do that or like could we do that or can we like enable this? I keep thinking like what is actually possible when this stuff really works is reality is a lot of agent self building like natural language code doesn't work quite as well as you hope. It was an ideal world as a lot of it's marketing. So when it really starts to function like the capabilities here to change how we do things are so significant that I just think there are transformative opportunities particularly in our space in the GLC world. I think I'm kept awake a lot by what's possible. I say the opposite side is I think less of the technical attack vector, more of the career vector. I worry and say well actually we're looking at replacing tier one everything with agentic, right? Whether it's soc, whether it's like entry level grc, whether it's entry level, like cyber analyst roles. And I think what does that mean for our career future? If we're looking at saying like all of us, Nick, me, we all cut our teeth in some entry level. I was in Pentecost, I don't know when Nick was. We all cut our teeth somewhere technically, right? And if those roles disappear, what's the future of cyber mid management and CISO look like? Like are we going to get rid of this whole knowledge gap of people having played with Raspberry PIs, is that going away because people now just play with agent kits and actually know how to physically deal with infrastructure or actual networking? I don't know. And that that concerns me more is like the changes we're making now are great from an ROI and finances perspective but we don't know in the long run what implication they have from a knowledge based perspective.
- [36:28]
A
I got to ask that question so much about what keeps me up at night and that there is a lot that actually on the optimistic side that Actually does. Like I hate to say it, but Gwen woke me up last night because I was messing with Gwen right before I went to bed. One of the open source models. So from that perspective, I think the art of possible keeps me up. It's kind of what Jake was saying, seeing what agency AI can do things like the REACT framework can do or what they're capable of. There's not a huge gap between where we are now and where it will completely automate out and be level five as we're talking about agentic for the things. So it's, I think a lot of it's going to be to what level can we get where, you know, right now we're just sending prompts to say here's as much data as possible, give us back information whenever we get to the point where the prompt is going to be run by business. So think about the anthropic was an experiment they did where they took and they had AI run a vending machine in the office, completely soup the nuts and just said you have, you can make all business decisions. Your objective is to make as much money as possible. When are we going to get to that point and what is that going to look like for the future of business?
- [37:39]
B
Right. And on that note of optimism and kind of how can AI be looked at as positive, Jake, how can GRC teams proactively use AI for more of the risk detection and mitigation that we talk about?
- [37:54]
C
I think when you really look at AI and you talk about risk, risk has been static for so long. It's been like a finger in the air kind of hope. Like, here's the risks I've got, here's the category I've got his priority, my remediation plan. Here's the efficacy of that remediation. Here's the owner. That's a traditional risk model. When you look at a line in a risk register, I think here you've got the opportunity to make it truly dynamic. Like let's create risks out of what's going on inside of our environment, seeing at the data. If more people are suddenly failing phishing tests or if more people are fainting to their training on time, or if more users are suddenly logging in with their Google or through all kinds of tools that aren't in the vendor stack. Let's create and populate risks based on that in a dynamic and automated fashion. Then let's check the controls and the efficacy of those controls that we say are either mitigating or reducing the risk that we have and show they actually are doing that or not. And if they are, let's remove the risk dynamically. Let's make risk a living, breathing thing where actually it's supplied by, endorsed by and proven by data. Because AI allows those workflows and it becomes thing which is real and tangible. That's for me and the risk world where this really goes.
- [39:02]
A
Yeah, it's almost, I think Jake, Jake pretty much summed up my dynamic risk or my dynamic GRC talk track pretty well. I mean, the only things I'll add there, I consider risk management a data sciences problem. Now we have the data. If you look at security teams, security teams are overwhelmed with data like how many people are going back and a SIM and saying, I'm paying too much for this, I need to cut back this data. Or the telemetry they're getting out of their EVRs or the telemetry they're getting out of all these different tools. There's so much data on the security side. The GRC side is still, the risk side is still starving for data and bridging that gap and getting that in there. But I think the one other big thing that's going to come out of this is we can use AI. So much of risk right now is, I call it. As you put your finger in the air, somebody comes and says, what is your risk tolerance towards this risk? And it really is a guess in most cases or the way the industry's been doing it to now is a guess. Or they go back to, you know, the rise and breach report and they say, you know, finishing happens on this, this cadence. So that's what we should base it on. And they base it on really broad industry. You know, in our service, with our size company and our, you know, tenure, this is what it might look like. But every company builds their security stack differently. Every company builds their defense in depth differently. Every company is in a different level of maturity. And I think AI now has the ability to come back and say, okay, based on the attacks, based on our controls and our control effectiveness and our risks and the incidents and the telemetry that we're getting, our risk tolerance should be this. And here's why. Because every time our Kris get to this level for this risk, we have an incidence. So we want to stay just below that line or we want to stay below that line at all times because we don't want to be constantly responding to incidents. And the AI is going to give us the ability to do this with as much data and really good data as Jake put it to figure this out where we can now say, you know, this is what our risk tolerance really is for our company with our tech stack, with our maturity levels and stop guessing and flapping in the wind to say, you know, the industry standard says that I should be here. Well, the industry, the rest of the industry is getting compromised left and right. I don't want to be at their standard. I want to be at the standard that we don't have to deal with incidents left and right. And that's going to be a really big game changer with AI and risk management.
- [41:19]
B
Thanks, Nick. And thank you, Jake. So as we wrap up the episode, we typically like to leave our listeners with some strategies for success we like to call it. So as a final note, Nick, let's start with you. Looking beyond the technology itself. What's the most critical human element that must evolve within organizations for AI to be both successful and positive?
- [41:50]
A
Yeah, I mean I think the biggest thing is AI is here to stay. It's not going away and it will continue to change business and change business in really, really fundamental, in large ways. So I think it's the biggest thing is when you're looking at, when you're looking at people, especially when you're hiring, I like to look at and say, oh, there's kind of three classifications of people. There's AI deniers. I've never used AI, hate AI. AI is all hyped, nothing will ever happen with this, I'm not going to use it. And then you've got the next level, people who are AI acceptors who use AI, whether it's just even if it's as simple as instead of using Google to use ChatGPT or Claude or Grok or whatever their favorite LLM is to do research. But then you're looking for people a little bit beyond that that then try to use it for more advanced use cases and try to explore like you know, what is deep reasoning, what is different things like that. So you know, people who will actually embrace and use AI, maybe not be extremely technical on it and understand the difference between a GPT and a rag and a gentic and agents and also that but at least somebody who's who uses it and embraces it and is willing to use it. And then you've got the next step which is the AI builders or the people who do have understanding of exactly what it is and are using things like aging kit to build out AI and understand exactly how we can build it to make the AI acceptors more powerful in their jobs. And this is so when you're looking at staff and when you're looking at different people and you're looking and you're looking at yourself and reflecting, ask yourself, am I an AI denier? Am I an AI acceptor or am I an AI builder? And then figure out where you want to be. I would say the AI builders are going to be the most valuable in companies going forward, but everybody should be at least an AI acceptor.
- [43:33]
B
Jake, do you have anything to add here? What is most critical human element that you think must evolve within organizations for AI to be successful?
- [43:42]
C
No, I would agree with Nick and in one sentence just say it's risk maturity and risk aggression. You'd have to understand this isn't going away. So we have to find a way to adopt, embrace and enable it.
- [43:55]
B
Well, thank you. Thanks Jake and Nick for these incredible AI CISO insights. It was so awesome to have you on the podcast today and listeners, you've heard what we have to say here. Now tell us in the comments. What is the most critical human element that must evolve within organizations for AI to be successful and positive? Jake and Nick, thank you so much for being here today. This has been an incredibly insightful conversation. You've covered the AI topics and trends that I know our listeners are thinking about and definitely reading about on a daily basis. For everyone listening, thank you for tuning in to this episode of GRC and Me. And if you found this conversation valuable, please be sure to like and subscribe wherever you get your podcasts so you don't miss the next episode. Until next time, thanks for listening.