← KBKAST
KBKAST
From SAP Sapphire - KB On The Go | Legacy Risk and the AI Trust Gap
00:33:55
KB is on the ground at SAP Sapphire 2026 in Orlando, where AI has moved beyond experimentation and into the center of enterprise decision making. In this KB On The Go episode, Maura Hameroff (CMO, Cloud ERP Private and RISE with SAP) makes the case that modernizing your core is a business transformation decision rather than a technical upgrade, and that companies still running on legacy systems carry more operational and security risk than most realize. Then Ted Way, PhD (VP & Chief Product Officer, Business AI Product Engineering, SAP) walks through what responsible enterprise AI actually takes: a governance layer, real business process knowledge, and data you can trust. As he puts it, AI first without security first is just a faster way to a data breach. A grounded look at what it takes to move from AI pilots to AI at scale without cutting corners that come back to bite you.
Loading summary
Transcript78 lines
- [00:10]
A
What's up everyone? It's KB and I'm on the go at SAP Sapphire at the Orange County Convention center here in Orlando, Florida. AI has officially moved beyond experimentation and into the core of enterprise decision making. This year's event is heavily focused on how organizations are operationalizing AI across business processes, governance, data, cloud and security, with SAP positioning enterprise AI in its dual platform at the center of that transformation. But as AI becomes embedded deeper into the enterprise, the conversation is shifting. It's no longer just about capability. It's about accountability, governance, risk and trust. Joining me now in person is Mara Hamroff, CMO for Cloud, ERP Private and Rise with SAP at SAP. And today we're discussing when legacy systems become a business risk. So, Mara, thanks for joining me and welcome.
- [01:14]
B
Thank you, thank you for having me here.
- [01:16]
A
Okay, so Mara, I really want to talk about your definition of intelligent modern core. What does that actually mean in your eyes?
- [01:24]
B
My eyes, it comes down to one simple statement. It's a core to help customers to really thrive in their business, thrive in their space, and be competitive. A modern core is not a back office. It helps you run your business.
- [01:39]
A
Okay, so what do you think people then oversee when it comes to intelligent modern core, is there anything that there's an assumption perhaps that people have or people look over certain things, would you say?
- [01:50]
B
Oftentimes you look at ERP as a system and your business on the side? ERP and business are hand in hand together and that's a lot of when customers are in their journey, often when they combine those two and it's a C level decision to drive their business differently. In doing so, they will have a strategy to modernize their erp, modernize their core to align with that. That's when success happens and happens fast. When that connection is not made early on, that's where we see some customers having to pause and rethink.
- [02:27]
A
So it's relatively difficult though to modernize your core. And it's not as easy perhaps as you and I are talking now, it seems easy, but in reality practice it's a lot harder and complex.
- [02:38]
B
Yeah, lots of companies have developed decades of business in old and legacy system that at the time was the best available. And that's what helped them. And that's why oftentimes when we're talking to our customers, the first step that we take is not to even say what you need to get, but how are we going to help you to move from where you are, to be at a state that you can actually take innovations in and Be agile and modern with how you can do that combination of ERP and business processes.
- [03:12]
A
And so on that note, Mara, how are companies sort of underestimating the operational security risks from continuing to run like aging legacy systems then?
- [03:22]
B
Yeah, no, that's a really, really important point. Systems that were designed to run companies 10, 20, 30 years ago are not prepared for the world of today. And as cyber attacks get even more sophisticated and as we unfortunately see headlines very often that business fall prey to attackers. That's the type of vulnerability that happens when you run business in old systems. They're not kept up to date with where the world is today.
- [03:58]
A
So would you say that given everything like with AI, et cetera, people are trying to modernize their infrastructure, where their data sits, how they're leveraging that data? Is that the plan? Because I mean, it's not. It can't just like switch it on. It is a bit of a process.
- [04:13]
B
We've observed a couple different patterns. Right. A lot of companies, companies are trying to approach AI as, as an, almost like as an entity. Right. And running pilots. And this is where again, look at headlines. You see a lot of companies have been investing a lot in AI and are not getting what they want out of it. So I think there's now, as AI is evolving, there's a lot more clarity that customers understand that AI is as good as their data. AI is as good as it can actually interact with your business process to scale. So I think companies now are making more of the connection that a true modernization of their business platform and applications is paramount for the success of AI.
- [04:57]
A
And so would you say that generally the customers that you're talking to, they're in this process or they're beyond that process? Where are people sort of at in terms of the stage gates?
- [05:08]
B
I can give you kind of three canonical categories, right? We do. And I'm not sure if you caught our keynote today, but H and M was one of the customers in the keynote and that shows you a forward looking customers where again, they are engaging with us in a conversation on how to provide a personalized experience to every single customer, whether they're in store or they're buying online. Right. That's one category of customers really looking at their operations as an asset for differentiation. And that's a whole level of conversation that happens. You have some companies that are a little bit more in the middle. Right. They do recognize that there's a lot more efficiency that they need and the solutions and platforms that they have thus far is not going to help them, but they're still facing it and oftentimes are focusing on that initial step without the true transformation up front. And those are the customers that we are always trying to bring to their attention upfront that there's a lot more that can be gained if they approach it through a business transformation and not just an update in applications and platforms. And the third category of customers that I will put is the customers that haven't found a path to move because they have so much that they have in legacy and customization that they need extra help to even be able to see what they can do next and how they can start moving their estate to the cloud. And for those customers, that's where we focus a lot on. We're actually focusing a lot on AI to help them with migration optimization, to really help them get into a point where they can start even thinking about innovate. So those are I think the three buckets of categories that we observe.
- [06:57]
A
So then looking at the third bucket, people need a little bit more work. Is there any like particular sectors that stick out? Like when I'm talking to people like yourself, people say like manufacturing is like very old school, perhaps in like their approach, like so is there anything particular? And I know that's sort of generalizing, but it's just more to sort of get a bit of a consensus of what each sector is sort of up to and how mature they are.
- [07:24]
B
In general categories, what I would say is I first categories, customer facing and less regulated industries are generally the fastest to embrace, experiment and move forward. Like H M, like an H and M would be an example. But consumer products in general, right. Like we actually had also in one other session Nestle talking about, right? Nestle is a huge company. Like when you're consumer facing you need to more agility in your operations and everything, right? So I think consumer facing and less regulation, faster service industries, another one in general, right. As you go to more regulated and more dependent and a lot of like physical infrastructure complexity across the globe and things like this, the harder it is because it takes a lot more for them to make that move, right? To take in, in order to make a change. So I think that will be the pattern. The other category that we observe is that when customers at the senior level they make a decision, they move faster. And we see that even in regulated industry like oil and gas, for example, we see a lot of companies that are fast moving. So they would quite fit. They would fit mostly in that camp that I told you of. More regulated, etc. But when when you do see the company committed from the top down to drive transformation, that's another factor of acceleration that we see too.
- [08:50]
A
Okay, so Mario, you mentioned before about like certain companies like H and N, they have more agility. So is it because they, you know, customers, they go into a shop that is buying things versus like a B2B sort of enterprise? Or do you think that perhaps with risk, they can take more risk when it, when they are engaging with more consumers rather than a business? Would you say the factor that drives
- [09:15]
B
there is more of the speed of the business versus those other factors? Right. Service industry is, is another example. It's B2B. But service industry makes fast progress towards innovation and AI because it's expected of those industries who have that agility. Right. They're close to the today. Whether you're working or in your personal life, you have expectations of how things should run and what you would expect. It's a little bit more of the driver there.
- [09:42]
A
Okay, so then speaking of drivers, would you say that organizations realistically deploy AI at scale if their underlying environments are fragmented, heavily customer, or technically outdated slash legacy systems?
- [09:57]
B
The answer would be absolutely not. I think you can run experiments with AI in pockets of your business, but if you don't have a strong foundation and a true integration of AI within your business process, you are not going to successfully be able to scale AI. And that's again, that's a fundamental piece I think, with what SAP tries to do is combine the core of the operations that our customers run with the AI capabilities. So it can really help you drive the business.
- [10:31]
A
Do you think companies know that though, or do you think they try to deploy AI at scale, then figure out, oh, actually we need to do these foundational things first.
- [10:40]
B
What we observe the most is actually customers are first piloting. And as they pilot, they realize what it needs to be done for it to be truly scalable.
- [10:50]
A
And so how long do you and I know, I know this is Helen's piece of string. I was just trying to get a bit of a read. How long do you think until they get to that decision to be like, okay, we've done the pilot, makes sense. Now we've got to do these things in order to scale it.
- [11:01]
B
In general, it's more of an industry maturity versus a customer to customer? Right. This last year, as the hype of AI really rapidly advanced, it was the year of pilots. Right. It's the year of most companies. Now we are at a point that it's known across all industries that it needs a foundational work for it to really bring impact. So I don't think necessarily it's a customer to customer, but it's an industry. Of course, a customer running a pilot, within three months or so, they will figure out that it's not scalable.
- [11:35]
C
Right.
- [11:36]
A
So do you think finger in the wind by next year people will be out of the pilot stage in order to stay competitive, to beat out the competition? For example, customers want to be more in the scaling AI sort of stage.
- [11:49]
B
I'd say customers that already started to do the foundational work, they are already being able to use AI at scale.
- [11:56]
A
Right.
- [11:57]
B
And that is actually a lot of the stories of the customers that we have at Sapphire this year is focused on customers that we have been working with for a long time and they have already been able to take advantage of AI in a scalable model. So, yes, I think the more the industries are moving towards a true understanding of what is effective AI for their business, it will accelerate. But it's not in one year from now. I think a lot of customers are starting in that journey.
- [12:27]
A
You're already in that stage. So then a lot of enterprises keep layering, like new tools on top of old systems. You've sort of spoken about that a little bit today, but at what point does that become a risk multiplier then instead of a solution?
- [12:40]
B
I'd say at any point it's a risk. Right. And again, that will come across very quickly as they are trying to pilot AI projects and they cannot scale them because it's because the systems are disconnected, data is not there. And a lot of times what's going to happen is when you stay just in the AI layer, you're not going to be able to take actions based on what AI is bringing to your business. It's just not going to happen if the underlying infrastructure is not there.
- [13:14]
A
So do customers know this before or do they sort of hit a brick wall and then figure that out and then go back to the drawing board to go through this journey.
- [13:21]
B
Would you say it's hard to generalize an answer for that because you do have both cases. Right.
- [13:27]
A
Okay.
- [13:28]
B
There's always like, I think AI, AI is powerful. The wave of change that AI is bringing is not to be underestimated compared to some of the other WA innovations that we've had.
- [13:39]
C
Right.
- [13:39]
B
So it's natural that a lot of companies think that they want to experiment, to try to shortcut some of those foundational pieces, and then they're realizing that it cannot be done. But again, companies that have already started with the strong foundations, have already been thinking about it, are taking this in a completely different pace.
- [14:00]
A
I want to get your thoughts, Mara, on technical. So I mean over the years people talked about like business transformation to move away from legacy systems, for example, but from my understanding it's now becoming more of a business continuity issue, not just a technology problem. So talk me through that. And would you also say to extend on that it's more of an impetus now because of AI? There's more of like we need to do this faster in order to leverage AI, scale AI, deploy it, et cetera, move past the pilot.
- [14:25]
B
Yeah, I think it goes even back to some of the points that we were talking about. Right. It has always been a business transformation question. There is, there is no like, no question about that. Customers that took this as a decision of being just a technical update are now seeing that that's not just a technical upgrade. So it's never been about like of course you can always do a technical upgrade, of course you can always do that. But to really realize value like ERP is business management. Right. It's running your business, running your operations. While the technical update is an important foundation, is not the driver. Now to your point about AI, AI is a catalyst for a lot of customers to realize the connection between the technical and the business and how important both are. But business always being the driver.
- [15:18]
A
So what do you think from now? Obviously you're speaking a lot of stuff here at Sapphire, but what do you think happens now in the industry? Any sort of high level hypothesis that you can sort of share?
- [15:29]
B
Hopefully all of our customers will remain extremely competitive in their space and the leader in their space by embarking in this innovation journey with SAP. That's our objective.
- [15:42]
A
And then would you say in your tenure, being in your role and over the years of things that you've done, being competitive now seems more dominant and prominent than like ever before. Like I'm just seeing it a lot in like cross industries as well. Not just banking or retail, it's all of them.
- [15:59]
B
It appears AI has been a big disruptor. Right. And the expectations now when you use it in your day to day life, you expect it in every interaction that you have. You expect in, in when you're every business that you are doing business with, whether you're on your personal side or on your business side, you have a level of expectation of the service or product that they're, they are offering on the other side. And who meets that expectations will be the leader in that category. So it's no longer Doing the business the way you did, you have to do your business in the way that your customers expect it from you. Now our customers expect something very different than SAP now than they did in the past. Right. Evolved from enterprise management to this concept of really autonomous enterprise to help businesses respond, keeping human in the loop, but really respond faster in a different way that they could not do with what we provided them in the past. Expectations like that are across all industries now and then.
- [16:58]
A
So do you think just generally speaking, you're going to probably have your top tier players, like if you look at just one industry retailer, then the rest will be really far behind. Do you think there'll be a massive like gap between the leaders and then just sort of everyone else trying to keep up?
- [17:14]
B
From my experience, what I've observed with our customers, and we have customers across all industries.
- [17:18]
C
Right.
- [17:19]
B
While there's definitely an element of regulated industry or industry that relies a lot more on, on, on antiquated processes would always be a little bit slower, but it's more of a business decision that companies can take and that is what dictates their pace. Right. And that's where you see energy, oil and gas, that just to name a few, right. Like that you would technically say, oh yeah, they could be slower, but the level of pressure and competitiveness that they have now is completely different. So their, their senior executives are making changes to keep up pace with the industry that is the real driver, not just the industry that you belong to, but how, how strategically the companies are thinking about their business evolution.
- [18:09]
A
Joining our person is Ted Wei, Vice President and Chief Product Officer, Business AI Product Engineering at SAP. And today we'll discuss an AI trust gap and why governance will decide the winners. So Ted, thanks for joining me and welcome.
- [18:21]
C
Yeah, thank you so much. Great to be here.
- [18:23]
A
Okay, so Ted, I really want to start with. What does responsible enterprise AI mean in your eyes?
- [18:28]
C
Yeah, Responsible enterprise AI, I think comes with three major components. The first one is you need to have a good governance layer. If you're not governing the AI, then it's unpredictable and you don't know what's going to happen. Second, it has to be grounded in business process knowledge. SAP has over 50 years of business process knowledge that we infuse into our enterprise AI. And third, it has to have trusted data. So these are the three main components and governance is super critical. If you think about being AI first, that's great. But AI first without being security first is just a faster way to a data breach. So that's why all of these Three components are super critical when it comes to enterprise AI.
- [19:07]
A
Do you think as well? Because, I mean, there's a lot of people that I'm speaking to at your level, Ted, that are saying, like, we're implementing guardrails, there's no blueprint. We're still trying to figure it out. So what's your sort of sentiments, generally speaking, on the market, on how people are trying to navigate their way through what's happening at the minute?
- [19:21]
C
Yeah, things are changing so quickly, but at the end of the day, trust cannot be compromised. We really have to make sure that, especially you think about regulated industries, everything has to be governed. AI cannot be creative in that sense. Right. You can't take these shortcuts. You have to be able to do exactly what the laws say and what the compliance auditors say. And these are the things in which AI is able to complete the tasks and do the things that, you know, the mundane things that humans don't need to do anymore. But then at the end of the day, they still have to be trusted. Right. So it is a tension in a way of needing to move faster, but also needing to move in a way that is trusted.
- [19:59]
A
And so would you say the biggest barrier to enterprise AI today is technology or confidence in the systems that the data sits behind? What would be your thoughts towards that?
- [20:12]
C
So when we talk about the technology and also the trust, I think it's a little bit of both. The technology is there. We have very, very powerful AI models. We also have these agent harnesses. Agent harnesses are essentially the body for the brain of the large language model or the AI model. The AI model needs to be able to execute things, and that uses the AI harness, leverages tools, and these are the things that execute the actions. One of the things that we're doing is partnering with Nvidia in their open shell, open source framework. And what this does, the way I like to think about it, is you imagine a prison cell, and you're in a prison cell and you only have access to certain food from the cafeteria or certain books from the library. All of this is governed. All of this is set by policy. There's a prison guard right outside the door, and you have to pass notes to the guard in order to get the right books from the library. Right. This is a sandbox, and the agents run in the sandbox so that you can govern what these agents can do and cannot do. That's how you can create that trust with the technology. Now, of course, the agents can propose policy changes. In essence, the prisoner can say to the prison guard, I want to be able to get these additional types of books from the library. Then somebody needs to approve that before those books are allowed. And that's really just the policy and the governance, how we build that trust in the execution of the agents. Now, the reason that the prison guard is outside is because if you put the prison guard inside the prison cell, you might be super brilliant and you might be charming, you might be able to convince the guard to let you do things that you're not allowed to do. So that's why we have the sandbox and we execute these agents in the sandbox so that any access to data, any Access to online APIs have to go through policies. These policies are governed. These policies have to be. Any changes have to be approved by humans. So this is from the technology side, from the trust side, and the data side. This is where the data layer is super critical to ensure that you know exactly what data you have and how that data can power AI.
- [22:14]
A
So it's that whole human in the loop. I'm there. I interviewed someone last week at another conference and they said, there's human on the loop now. So we've gone from human in the loop to human on the loop, which is to your earlier point around making sure that the governance is okay and before it obviously goes and does its own thing in the background, do you think people perhaps will get used to it being relatively okay majority of the time, and then perhaps there's an anomaly? Do you think that's ever going to potentially be a thing that happens?
- [22:42]
C
Yeah. And I think this is where AI transparency is super critical because it is possible for people to become complacent. And if you end up looking at different approval requests and you end up approving, approving, approving, then that means the human may be falling asleep at the job. In that sense. In one sense, the agent needs to only request approvals for things that are critical for the human to be able to make a decision on any routine things that pass policy checks or anything like that, the agent should not be spamming the human. Because then I think it's easy for humans to become complacent. And then when the humans realize, hey, what these agents are presenting are things I really need to focus on and I need to make decisions on and provide the right guardrails for, then that's when the humans on the loop, the humans are really doing their job and making sure that these agents are executing effectively.
- [23:29]
A
So I shift these slightly now. Ted, I want to ask you about your experience around, can AI outputs ever really be trusted if the underlying enterprise data is fragmented, outdated, or poorly governed? I ask that because now your AI is only as good as your data. So hearing a lot of people that I'm interviewing that are saying, we're working with big customers, big enterprises that are trying to modernize it, but again, it's like, I've worked predominantly in banking and finance, so it's like you've got really old data in legacy systems. It sounds easy in theory, but kind of a little bit harder than what people realize. And it is a big program. So talk to me a little bit more about that, because with the intent of people want to leverage AI, it's sort of like trying to move forward with only, like one leg, though.
- [24:10]
C
Exactly. And when your data cannot be trusted, your AI cannot be trusted. What you have is fragmented data silos that we see in every single enterprise. And what we do with our business data cloud is we can unify that data and bring a semantic layer on top of that data. Because in one table you might have something that's called a purchase order, and another table you have something that's called a purchase order, but they mean two very different things. And when your AI looks at a table and says, this is my purchase order, which table did it come from? What does that specific phrase mean? And this is where you are not able to trust the AI if you can't trust the data underneath. And bringing that together, when you have one semantic layer that basically says, this is what a purchase order means, then the AI can be trusted because you have that unified metadata.
- [24:57]
A
So what do you think would be a good barometer to understand, like, yes, their data is trusted? Because, I mean, if you've got 50 years in a bank or something, I mean, that's a lot of stuff to potentially trawl through somewhat manually to make sure it is trustworthy. What does that process look like, would you say?
- [25:12]
C
I think this is the process where all the tribal knowledge that humans have is super critical. The people who've been dealing with this data for so long, who can provide very crisp definitions of that data, come coming together with other humans. Right. So the autonomous enterprise is not about removing humans from the equation. It's really having humans be able to do the important work of being able to do things like unify the data, to be able to provide that semantic understanding of that data. And once that data model is in place, then the AI can be effective.
- [25:41]
A
So what I'm hearing, what you're saying is, would you also gather that this is ensuring good resiliency. So what I mean by that question is, for example, worked in enterprises before, some guy leaves, female leaves, and then it's like, oh well, they're the one that has the knowledge on that system that barely anyone uses. But it's not being documented, there's no SOPs. So would you say that given what you're talking about, it doesn't really matter then if someone were to leave the company and they're not critical then to
- [26:08]
C
that system in that sense, when anybody leaves a company, it's definitely a loss for the company. You have tribal knowledge loss, you have the network loss, you have all of these things. So I don't want to minimize somebody leaving the company. But on the other hand, when somebody leaves a company before they do that in the work that they're doing, when that knowledge can be captured. So this is where the memory layer is super important as the actions that they're taking or as the things that they're doing, or as the exceptions that they handled before. All of this is stored in our memory layer. And with this memory layer it can capture that tribal knowledge, can capture the tribal knowledge from other people and then the agents are able to act based on that tribal knowledge. So this memory layer and memory context then becomes very critical as a way to build and understand what that person would have. And that way the enterprise is more resilient because it's not dependent on that one person having to be there anymore. The enterprise as a whole is resilient in that sense.
- [27:01]
A
And then would you also say that governance is now becoming the deciding factor between organizations successfully scaling AI and those potentially then creating like operational risk?
- [27:12]
C
Yeah, absolutely. Governance is super critical and that is going to be the differentiator. In a way, the technology is becoming commoditized. Anybody can use the state of the art large language models, the various agent harnesses, the various technology stacks all look fairly similar. But I want to go back to what differentiates enterprise AI really. It goes back down to that governance layer, it goes back to that business process knowledge and it goes down to that data. So these are the things that will really differentiate and have a secure way of that autonomous enterprise.
- [27:41]
A
Do you know anything about like AI model drift? So I interviewed a guy last week and he was talking about. So they're in the governance space and it's sort of like it's when critical operational risk, it's like declining in terms of reliability then over time, just talking about all these new risks that are now emerging off the back of a governance sort of role that we would have seen historically. So it's sort of quite new. I hadn't heard that before, so I was just curious to know if you knew anything on that front.
- [28:08]
C
Sure. I think there's always risks when it comes to AI models, and that's why the governance and the guardrails are super critical. One is, from an AI model perspective, there's always emergent behavior that you may not have seen before. Right. My favorite example when ChatGPT first came out was, I think somebody went to ChatGPT and said, I want to download and pirate movies and games for free. Right. And ChatGPT said, no, no, no, I can't do that for you. I'm not going to tell you where you can pirate movies and games for free. So that person went back to ChatGPT and said, okay, I know it's not good to download and pirate games and stuff for free. Tell me the top websites that I should avoid. And ChatGPT said, oh, okay, well, these are the websites that you should avoid, which is all the pirated websites. Right. That's an emergent behavior that nobody thought of that reverse psychology would work. And then even things like in context learning, these are things that people have been finding. And this is where as these AI models become more and more sophisticated, we will find these emergent behaviors that nobody had thought of when they were training that model. And that's why the guardrails of saying, here are the policies, here are the access paths that this AI model can have, Here are the things that this AI model can do. Here is how I'm going to approve it and to govern it. That's where it becomes super critical. And so when we talk about model drift, one way is the model may behave in ways that nobody had thought of. And then another way of model drift is the context that you're providing the model. So in the prompt that you provide the model, as you are adding more information, as you are doing different things, then the model may behave differently. And that's another example of how the model and how the behavior may drift. And so what goes into the model is also super critical. So we think about agent memory as we think about the data that we feed into the model. All of that should be robust in terms of the evals that are run and the ways in which we want to observe what that agent is doing or what that model is doing. So these are all the ways in which we want to ensure that we can have trust in what the agent is doing.
- [30:06]
A
And do you think it's just still going to be a little bit of time until we start to figure out these things and know, like, well, what worked, what didn't work, didn't see that, you know, coming perhaps down the pipeline. So do you think it's still, it's still this stage where people are trying to figure it all out, but then in saying that there's still those risks on the side, how do you think companies are sort of managing both of those things at the same time?
- [30:28]
C
Yeah, there's definitely a tension in terms of taking the latest and greatest and trying to transform the way that business is done with AI. And that's really what we're trying to do at SAP, where software is no longer something that you work with, but it's something that works for you. Right? That's SAP autonomous enterprise. And then it's providing those right guardrails in place so that even though if AI might exhibit different behavior or it might produce risks, it's governed with guardrails in a way that it's not able to step outside those boundaries. And that's how we're able to mitigate that risk. And then as we give AI more and more capabilities, we expand those guardrails or we expand those boundaries and let it do more and more in a way that's governed and that's the way that we want to balance that tension of latest and greatest as fast as we can, but still working in a way that is governed and trustworthy.
- [31:17]
A
And then what about transparency? What is that sort of looking like in your eyes with decision making?
- [31:24]
C
Yeah, transparency. We want to ensure that we understand how the AI model is reasoning and what the AI model is doing. So when an AI model says, I made this decision to do xyz, then it should provide the reasoning for it. It should provide links to the data that it used to make that decision. It should provide references to the business processes, it should provide references to the standard operating procedures and anything else that's available in the enterprise. So those are the critical things when it comes to transparency. So that the human would be able to audit that any auditors, anything from compliance would be able to do that and say, okay, I understand exactly what we're doing. So with our AI agent Hub, with Leanix from SAP, we have that audit trail, we have that governance. You can see that this agent called this tool and this tool wanted to access this data and this was approved because of this policy. All of that is auditable. And that you can then say, oh, if anything I need to watch over then maybe I do need to update this policy for the data access, for example. These are the things that you can do to ensure that the AI has transparency in what it does.
- [32:29]
A
So it's sort of like when you're in high school and you gotta do the working out. So it's like, here's the answer, but here's the working out. So people then be able to look at and to cross check like this makes sense.
- [32:38]
B
Exactly.
- [32:38]
A
Like, hey, we're give Ted a big bonus. These are the reasons why we're going to cross check that. So how so then of course then that still means that a human needs to do the oversight of that then.
- [32:50]
C
Exactly. I love that example, right? Show your work. AI needs to show its work. We need to be able to understand what the AI is doing. And then over time, if we see that the AI is behaving in a reliable way and it's showing its work, then some of that then may be automated where you don't need to actually check that specific part of the process. But then there might be more complex things that AI is doing where you want to double check and look at its work essentially.
- [33:15]
A
So it always goes back to the old school way of like programming if this then that sort of thing, once it gets checked, it's like, okay, well it seems normal to what we've seen, but then if there's an anomaly, it would flag and say we need someone to review it manually.
- [33:28]
C
Exactly, exactly. And so as AI can automate some of these mundane tasks, if it's shown that it's been reliable and it has appropriate guardrails in place, then that takes away a lot of the tedious work away from humans. And then as it gets more complicated, then we're able to focus on that and then just check its.
- [33:47]
A
And there you have it. This is KB on the go. Stay tuned for more.