B (14:04)

Comes to AI extreme? In one word, that's all you need to think about. It's such a rapidly changing industry that it's hard to keep up with the technology first and then to be able to assess that. I think people are totally clueless and and just floundering in trying to figure out what's the best way to do that, not only in our own organizations, but with the third parties. And that's kind of what, what sparked the idea of this and started me doing some research about a year and a half ago.

A (14:39)

Well, why have traditional risk frameworks struggled to keep up with this rapid adoption of AI?

B (14:45)

Well, because AI changes so rapidly.

A (14:49)

Fair enough.

B (14:50)

And it's interesting. So a lot of people think that AI is a completely separate entity, that you have to do completely new assessments on AI. And that is a total misconception. A lot of the underpinning infrastructure that AI runs on, you're already assessing, but there are obviously new components about it. And that's been the challenge. And you watch the industry every week, every two weeks, a new framework, some new organization, new government comes out with. Here's what we say is the best way to assess AI.

A (15:29)

When I think of the current landscape of AI risk assessments, it strikes me that it's very fragmented. Is this a matter of there being just so many frameworks or even so much regulation, or is it a lack of consistency? What's going on here?

B (15:46)

Yes, all the above. And. Well, here let me make an analogy. So AI is a technology that has been advancing faster than anything we've ever experienced before. You know, if you go back far enough and, and we remember about, in the early days of personal computers, every two to three years they would double in capabilities. There was something called Moore's Law and, and, and that held true for a long time. But now it seems like the AI competition every week a particular LLM is coming out with something new and better than the competition. They've enhanced it. The, the cost of compute is down. And you know, so as a risk assessment methodology, you know, that's a, that's a very sprawling landscape that we have to take into. And imagine a city where every city block, they've developed their own slightly different building code. And that's because it might be the industry. They're in the, you know, financial industry looks at things one way might be geography. If you look at, in, in Europe, their initial frameworks that they were bringing out were, weren't really focused so much on, on technical, they were focused on other things. But you know, look at that, that, you know, the city keeps growing, there's a new framework, there's somebody new bringing it in. And ultimately what you've got is you, you have a city that is in total chaos about what the best way to do it. You know, we looked at, and this again, you know, we give the number 50 plus. But I can predict, and I usually don't do predictions that in the next month that's going to increase, there'll be new ones. But essentially, you know, going through that and synthesizing all that variety and seeing, okay, underneath what's the basic non negotiable security standards in a building code. You know the analogy, it's like, you know, fire exits, load bearing walls, plumbing, things like that. Why couldn't we do something like that for AI? That, that's kind of what, what really was the genesis of all this?

A (18:13)

Well, let's dig into the BKGA3 framework. I mean what, what problem were you and your colleagues trying to solve when you began developing it?

B (18:22)

Well, I started on my own because I, as a vendor, we undergo assessments and I look at the history of how third party risk assessments have been done and they're very static, very non agile. And you know, it's always been challenging to be able to look at that and understand, you know, where the risk is or how I like to say it is. You know, the reason why I look at a third party and I'm assessing them for risk is I want to understand and reduce the amount of surprise in that relationship. Surprise is the unknown uncertainty and you know, I don't want to be surprised. So that's what we're trying to do. And so when AI, you know, AI has been around a long time, it's not like it just came out. But two and a half years ago, ChatGPT really hit the market and it just captured people's attention, it captured investors, and it's been expanding ever since. So I started playing around with looking at using AI to analyze the frameworks in about a year and a half, almost two years now, it was about 15 and to see what commonalities there were. And that was the genesis. And as we saw it get worse and worse because new frameworks came out, new things happened. Well, the folks that run our research department, they thought that that initial look that I looked at, that they found it to be very interesting and they expanded it. And that's kind of, you know, I'm not the genius behind it. I was the curious one that was asking questions and smart people then took it and run with it and develop this.

A (20:11)

Well, you've described this as an open standard for assessing AI risk. Why is that openness important to you?

B (20:19)

Well, it's openness is in the DNA of our company. So one of our co founders, John and bullockbush, has always been focused on giving back to the community. And from day one, you know, with Black Kite, there were tools, there were things that he would make available that people could use to help, as I say it, reduce the surprise as. As he would say it, you know, assess the risk. But that, that's always been important to the company. And, you know, one of those challenges is if you look at, in, in the AI world, some of the frameworks that are maybe de facto standards, they're not free. You have to pay, you have to subscribe, there's a tool you have to buy. And that really goes against that, you know, that whole concept of, you know, making things better for the global community. So we looked at that and we looked at there really wouldn't be any value to us developing something that was proprietary. Everything that Black Kite does is based on open standards. And since there was no common open standard for AI, why not create one and put it out there and make it open to whoever wants to use it. So that's what, what brought that to the table.

A (21:40)

Can you take us behind the scenes? I mean, you mentioned you evaluated hundreds of requirements from over 50 frameworks. What was that undertaking like?

B (21:51)

Well, for me, it was using the power of AI to, to read them, to assess them, to categorize them, to use semantic similarities, to see which ones had commonalities, which ones could be boiled down to maybe, yeah, eight or nine frameworks that are looking at the same control, the language is a little different, but ultimately they're trying to get to the same point to understand about that particular thing. So why not make it simpler and be able to, you know, be that Rosetta Sones, so to speak, of all those. Those different frameworks. So that's how I got started. And then the research folks, they put a lot more intelligence into it. And I mean, they're some very smart people. And using tools like that, the company's been built on nine years ago, using AI to be able to look at documents like SOC2S and information security policies and assess those and analyze what controls they speak to. So the technology was there, so they were able to use that as well.

A (23:06)

While you all were on this journey, were there any surprises and anything that popped up along the way that was unexpected?

B (23:15)

The only thing unexpected is it's growing faster than I even imagined. The capabilities of the various LLMs, I, I was doing some research. I was in a meeting, and somebody mentioned that a particular version, an LLM, was really biased and you should never use it. It's horrible. And statements like that are always interesting to me because I want to find out, you know, is that based on fact? So I used one version of AI to do a deep dive and research that very question. And essentially the AI came back a little snarky and said, yeah, that, that particular AI, it's only good for writing fiction. And it said I could do it better. And I go, oh well this sounds interesting. So I said, okay, well here's a concept, go ahead, write something for me. And what developed from that was it's a thing I publish on, on LinkedIn, on LinkedIn articles. It's called the Clause of Compliance. The Rise of the Raccoon Red Team. 100% written by AI. Really crazy. But then after the first one gave me a couple episodes, it started to, I noticed that it was getting repetitive. So I thought, oh well, let's give its evil competitor an opportunity and let it write it. And so I've been doing that. My plan is to have four or five different LLMs complete the entire series and then do a vote and see which one was better and then that'll be my best AI. That's how I'm looking at it.

A (25:00)

Wow. You know, you mentioned how quickly all of this is evolving. How do you plan on keeping BKGA3 up to date, keeping it current?

B (25:11)

Well, obviously, use of AI, use of automation, constantly looking at changes in the compliance landscape. You know, foremost are there changes to any of the existing frameworks, Are there new frameworks? That's one component of it. And the other component is, is looking at the risks that are being identified with AI. MIT has done some really great research on that. They've identified a large number of risks. So that's part of again the DNA of our company looking at potential risks in third parties. And this just leads right into that very well. And it, it will grow. It, it's, that's one of the A's, adaptability to be able to adapt to that ever changing world, whether it's from the compliance or whether it's from bad actors and updating it on a, on a much more frequent basis than most frameworks get done. Most frameworks get changed at most once a year. Very, very low agile capability. And it doesn't really work today that the bad actors are very agile. They're using AI. So being able to produce something that help an assessment process become more agile and keep up.

A (26:31)

You know, there's this phrase that folks are using these days, responsible AI risk management. I'm curious what that means to you and how you feel like the industry is adapting to that.

B (26:46)

Well, that's a challenge when you have to identify what does that mean? And your question is what does it mean to me because it means something different to everybody. That example I used about a particular LLM being biased. Well, responsible AI is to reduce the biases as much as possible. I don't think that we'll ever be able to completely remove all bias because it's a fundamental human frailty is that we tend to have biases and whether we know it or not, they get built into things that we do. But responsibility is looking and trying to ensure that the through the development that you've done the best possible work to keep that that bias minimal.

A (27:42)

For folks who are curious about this and want to dig deeper or even check out bkga3, what's the best way for them to find out more?

B (27:51)

I'd have to check in with our folks who are going live with it. There's an entire announcement. There's resources that'll be available that they'll publish. It isn't live today. It will be very shortly. Check the blackkite.com website for the announcements there and the links to the free resources will become available there.

A (28:17)

You know, Black Kite has a history of releasing tools and research openly to the community. It seems to me like this really aligns with that fundamental part of your mission.

B (28:27)

Absolutely. One of the things we bring out every week, it's called Focus Friday. It's the research that our team does to enhance our platform. And it's done, it's used for that. But the research has a far greater value if you put it out there for everybody else so they can take advantage of it as well. So it's that concept. So it's something that's not new. It's something we've been doing for a long time.

A (28:56)

That's Bob Maley, Chief security officer officer at Black Kite. At Talas, they know cybersecurity can be tough and you can't protect everything. But with Thales, you can secure what matters most. With Thales industry leading platforms, you can protect critical applications, data and identities anywhere and at scale with the highest roi. That's why the most trusted brands and largest banks, retailers and healthcare companies in the world rely on Thales to protect what matters most. Applications, data and identity. That's Thales. T H A L E S learn more@thalesgroup.com cyber and now a word from our sponsor, Threat Locker, the powerful zero Trust enterprise solution that stops ransomware in its tracks. Allowlisting is a deny by default software that makes application control simple and fast. Ring Fencing is an application containment strategy ensuring apps can only access the files, registry keys, network resources and other applications they truly need to function. Shut out cybercriminals with world class endpoint protection from Threat Locker and finally, London's Southwark Crown Court has officially dethroned the bitcoin queen. Jimin Quan, also known as Yadi Zhang, was sentenced to 11 years and eight months in prison after laundering a staggering $7.3 billion from a Chinese crypto scam that fleeced more than 128,000 victims. Quan, who fled China under a false identity, tried to reinvent herself in London's luxury property market, apparently forgetting that the blockchain remembers everything. Police eventually seized 61,000 bitcoin worth 5.5 billion pounds, the largest cryptocurrency haul ever recorded. Her accomplices didn't fare much better, one serving nearly seven years, another five. British officials called it a landmark case in tracking digital crime, proving that while money may talk in crypto, it also leaves a paper trail just with fewer trees. And that's the Cyberwire. For links to all of today's stories, check out our daily briefing@the cyberwire.com we'd love to know what you think of this podcast. Your feedback ensures we deliver the insights that keep you a step ahead in the rapidly changing world of cybersecurity. If you like our show, please share a rating and review in your favorite podcast app. Please also fill out the survey in the show notes or send an email to cyberwire2k.com N2K senior producer is Alice Carruth. Our Cyberwire producer is Liz Stokes. Were mixed by Trey Hester with original music by Elliot Peltzman. Our executive producer is Jennifer Ibin. Peter Kilpe is our publisher and I'm Dave Bittner. Thanks for listening. We'll see you back here tomorrow.