IA on AI – The AI Oversight Gap

Summary

---

Podcast Summary: The Audit Podcast

Episode: IA on AI – The AI Oversight Gap

Host: Trent Russell
Date: September 3, 2025

---

Episode Overview

This episode addresses the “AI Oversight Gap” within organizations, focusing on how insufficient governance and controls around AI tools create hidden risks and costs—especially in the audit context. Drawing on recent industry reports, trends, and audit best practices, the host (Trent Russell) examines the prevalence of unapproved (“shadow”) AI use, the challenges of instituting effective policies, and the vital role of internal audit in mitigating risks related to AI adoption.

---

Key Discussion Points and Insights

1. The Hidden Costs of Ungoverned AI Use

• IBM’s 2025 Data Breach Report ([00:11]):

  • Nearly all AI-related security incidents originate from unauthorized or unmanaged (“shadow”) AI tools.
  • 97% of organizations with AI security incidents lacked proper AI access controls.
  • 63% of breached organizations had no governance policy for detecting or managing AI use.

• Definition and Terminology ([01:02]):

  • The team debates the terminology, settling on "shadow AI" as opposed to "Bring Your Own AI" (BYOAI) for ease.
  • Shadow AI is defined as employees using unapproved or unsanctioned AI tools in the workplace.

• Quote ([01:21]):

  “The most striking finding is that 97% of organizations experiencing AI-related security incidents lacked proper AI access controls. Access controls—that’s like audit 101 stuff.”
  — Host (Trent Russell)

• Host’s Recommendation ([01:47]):

  • Every organization should implement an AI governance policy and committee.
  • Lack of such policies increases—not decreases—risk; merely denying or “blocking” AI is insufficient.

2. Shadow AI in Organizational Spend and Behavior

• Real-World Example ([02:12]):

  • Guest Michelle Velez (referenced) noticed increases in ChatGPT subscription expenses before her company had an approved internal AI tool. After deployment of a sanctioned tool, unauthorized subscriptions decreased but did not vanish entirely.

• Quote ([02:46]):

  “Obviously, there’s still going to be shadow AI...even when you have those approved tools.”
  — Host

• Actionable Advice ([03:29]):

  • Auditors should document and raise findings if AI governance is absent.
  • Maintain an “inventory” of AI tools discovered during audits and share with governance bodies.

3. Public Accounting and Internal Audit’s Responsibility

• Eisner Amper Survey Data ([04:10]):

  • 80% of employees had a net positive experience using AI, but only 36% of companies had a formal policy.
  • Host speculates larger companies with internal audit departments likely have better maturity, but many are still underprepared.

• Quote ([04:57]):

  “There are organizations we talk to...and they still go ‘we have nothing.’ Or it’s ‘Yeah, we kind of have something, but it’s not super specific and it’s pretty broad.’”
  — Host

• Employee Use Patterns ([05:17]):

  • Only 41% of employees report AI use to their manager; 60% rely on free platforms.
  • 28% admit they’d use AI at work even if it were banned.
  • Host doubts even this is fully accurate — actual shadow use may be even higher, especially among AI-literate employees.

4. Mitigating Shadow AI and Setting Expectations

• Education is Critical ([06:21]):

  • Training employees on AI risks is emphasized as essential.
  • Setting realistic expectations: AI tools support but do not fully automate complex functions like audit report generation.

• Quote ([07:34]):

  “You’re going to need to be a little bit more involved on the engineering side...Definitely, I would highly recommend implementing—suggesting—an AI governance or AI risk training.”
  — Host

• Practical Learning Resource ([08:12]):

  • Host mentions a recent webinar showing how these tools work, advocating for education not just on risks but on practical functionality to reduce “AI anxiety.”

5. Hype, Vendor Claims, and AI Washing

• CFO Dive Article — Vendor Overselling ([09:02]):

  • FTC suit against Air AI for defrauding clients with exaggerated claims of AI’s capabilities (“AI washing”).
  • Caution to organizations: don’t be seduced by vendor promises that AI will “solve all your problems.”

• Quote ([09:43]):

  “This is what is referred to as AI washing...Propping up this idea that AI is going to solve all your problems—buy our thing, you’ll be good to go.”
  — Host

• Vendor Risk Management and Onboarding ([10:25]):

  • Host cites a prediction: By 2026, 80% of vendors will incorporate AI, making vendor oversight essential.
  • Organizations must add AI risk assessments and evidence of safeguards to the vendor onboarding checklist.

• Internal Integrity Check ([11:02]):

  • Reminder to assess one’s own organization’s marketing to ensure it doesn’t fall into the same overpromising or “AI washing.”

---

Notable Quotes and Memorable Moments

• On Access Controls:

  “Access controls—that’s like audit 101 stuff.” ([01:21], Host)

• On Shadow AI Use:

  “Even when you have those approved tools...you’re still going to have people using these shadow AI systems.” ([02:46], Host)

• On Reporting Gaps:

  “28% admit they would use AI at work even if it were banned.” ([05:44], Host reading survey data)

• On Expectations vs. Reality:

  “That’s not exactly how it works...Just being able to throw it into the tool…and it do it is not going to happen very often.” ([06:40], Host)

• On Vendor Claims and AI Washing:

  “This is what is referred to as AI washing... And so it is propping up this idea that AI is going to solve all your problems, buy our thing, you’ll be good to go. When that’s really not the case.” ([09:43], Host)

---

Key Timestamps

• 00:11 — Introduces IBM data breach report findings and the term “shadow AI.”
• 01:21 — On AI security incidents and the lack of access controls.
• 02:12 — Story of Michelle Velez tracking AI tool spend and shadow AI usage.
• 04:10 — Eisner Amper survey data: policy gaps and employee behavior.
• 05:44 — Discussion on actual shadow AI prevalence and employee bypassing of policy.
• 06:21 — The need for robust AI risk and governance training.
• 08:12 — Mention of webinar providing hands-on education about AI tool functions.
• 09:02 — FTC suit over AI washing; warning against vendor overpromising.
• 10:25 — Vendor onboarding and necessity of AI due diligence.
• 11:02 — Internal marketing review to avoid AI overpromising.

---

Takeaways for Audit and Risk Professionals

• Inventories of AI use, robust governance policies, and employee education are critical in managing AI-driven risks.
• “Shadow AI” is pervasive, and absence of formal controls compounds exposure.
• Vendor risk management must now include AI-specific diligence.
• Internal audit remains essential in setting expectations and ensuring reality matches AI’s promises, both internally and for clients.

---

Transcript

[A]

Welcome to the IA on AI podcast, part of the Audit Podcast network where we bring you weekly updates on AI from the internal auditor's perspective. Here we go.

[B]

From natlawreview.com the AI oversight gap. IBM's 2025 data breach report reveals hidden costs of ungoverned AI. We also talked about this report from IBM last week.

[A]

The article is linked in the show.

[B]

Notes highly recommend going there checking out the report. So overall, from this cost of data breach report from IBN the IBM, they found that nearly all of the AI security incidents from. Damn it. They found that nearly all of these AI security incidents stemmed from unauthorized or unmanaged shadow AI tools.

[A]

You might also have heard shadow AI referred to as bring your own AI or by bring your own O. BYO AI.

[B]

We kind of gone back and forth.

[A]

I feel like shadow AI is a little easier to say. We're gonna do a shadow AI audit. We're gonna do a BYO AI.

[B]

Okay, we're gonna call it shadow AI from now on here.

[A]

So anyway, it's from shadow AI. It's from folks that are using unauthorized unapproved AI tools.

[B]

The article from NatLaw Review goes on to say that the most striking finding is that 97% of organizations experienced AI related security incidents lacked proper AI access controls. Access controls, that's like audit 101 stuff. While 63% of breached organizations had no governance policies for managing AI or detecting unauthorized use access controls and governance policies.

[A]

That is like right up our alley at this point.

[B]

I don't know what excuse you have.

[A]

For not having an AI governance policy, an AI governance committee, and there's a bigger risk in not having those and saying no AI used in this organization.

[B]

Because as you'll.

[A]

You've seen here, I'm going to read another stat from another article here in a second.

[B]

And as we've talked about it seems.

[A]

Like every single week employees are absolutely using unapproved tools. And I was talking to friend of the show, Michelle Velez. We had her on the audit podcast I think about a year ago. And so we were just kind of catching up. And she would look at, I think it was P card spin. But either way she was looking at these basically chat GPT subscriptions and seeing which employees were putting those on their corporate credit cards. And this was before they had an approved AI or approved LLM tool within the organization. And so once they did move to an approved tool, she said you could start to see those subscriptions starting to dip off. And so now people were using the approved LLM or the approved AI tool as opposed to going outside. Obviously the there's still going to be shadow AI that's going to be used even when you have those approved tools. But if you're in that organization that still, for whatever reason, for whatever reason, has not implemented a tool or doesn't have some kind of AI governance committee AI governance policy. Please share this report with those that need to know. I would argue that if you don't have an AI governance policy, even if you are quote, blocking AI, you're not. But if you don't have an AI governance policy AI governance committee and you're the audit leader listening to this, I would immediately type up a memo, say it's a finding and hand it over. There's no hard and fast rule or control it's going to prevent shadow AI, but certainly help as we're in the field, maintain that inventory and share that. If you have an AI governance committee, share that with them. When you're coming across tools that maybe they aren't aware of. It's one of the toughest things for AI governance to account for is what tools are being used. This is where audit can immediately help.

[B]

All right, so to tie back in to the previous story, this is from inside publicaccounting.com Eisner Amper survey finds as employees embrace AI, employer oversight lags so reading directly from the article, it says that while 80% of employees report a net positive experience using AI at work, just 36% indicate their company has a formal AI policy in place.

[A]

I'll be honest, I didn't dig through this survey.

[B]

So I don't know what the population.

[A]

Of organizations or companies that they had as part of this. Your maybe small, medium sized business probably wouldn't expect that larger organizations that have.

[B]

An internal audit department, I would expect that number to be significantly higher than that, 36%.

[A]

But like I said, there are organizations that we talk to and we talk, we ask about AI governance. It's one of the things that we talk about with everybody and there's still some that go we have nothing. Or it's yeah, I mean we kind of have something, but it's not super specific and it's pretty broad.

[B]

So anyway, it goes on to say despite 84% of managers acknowledging some level of AI use among their teams, only 41% of employees inform their manager or seek permission before using AI. Notably, 60% of employees rely on free AI platforms rather than internally developed platforms or external AI tools paid for by the company. And 28% admit they would use AI at work even if it were banned.

[A]

I gotta think that 28% isn't right for those that are using it daily and are maybe you consider yourself AI literate. Imagine if they just stripped away all your AI tools and went, all right, get back to work like you used to. No way somebody wouldn't immediately download these apps on their phone and start using them.

[B]

And so I'm reading these stats as.

[A]

Support for if you don't have a robust AI governance policy. Specifically within this it says notably, again.

[B]

Notably, 60% of employees rely on free AI platforms rather than internally developed platforms or external AI tools paid for by the company.

[A]

And I know I mentioned earlier what Michelle said and that did decrease it, but also like I followed that up, it's still going to happen. You're still going to have people using these shadow AI systems. I think education around these risks is absolutely critical. Even setting expectations, which we're about to kind of get into on this next story, can be helpful. Also, I know we've talked to organizations that go, hey, we're just going to take all of our work papers and we're going to put them into this tool and then it's going to assess it and then it's going to spit out the answer. Or we're going to put all of our reports into a tool and then when we do an audit, we're going to put the support in there and then it's going to spit out an audit report just like all the other ones. And that's not exactly how it works. So there's this unrealistic expectation of what I can do. More so from the ease in which work like that can be done, those projects are absolutely possible. There's definitely audit teams that are doing that, vendors are doing that. But being able to quote, just throw it into the tool and it do it is not going to happen very often. Create an rcm pretty okay at being able to do something like that, Write my email, something like that.

[B]

Absolutely fine. But some of these more complex, you're.

[A]

Going to need to be a little bit more involved on the engineering side. So definitely I would highly recommend implementing suggesting an AI governance or AI risk training. You can develop that internally. I would make it a requirement of every single employee so that they really understand what's going on. I also think it's important for folks to understand this is setting the realistic expectations how these tools actually work.

[B]

I did a webinar with Audit board. So Today is the September 3, 2025 episode.

[A]

We did it. Either Last week or the week before. So two weeks ago and the first, I don't know, maybe 15 minutes we pull up and show how these tools actually work. And the most like the easiest way I explained it in such a way that I felt like some like AI literate people were sitting there going, this is not right at all. It is more analogous than anything. But you can check that out. It's on demand. If it's not on demand yet, it will be soon. And again, just those first 15 minutes, I think that's critical to be able to understand how these tools work, to understand and set expectations for yourself and the rest of your organization of how these work and to ease the AI anxiety or AI anxiety, as we've been saying recently on the threat of AI2 jobs.

[B]

So we talked about kind of the hype and the what's realistic with AI right now. So this is from CFO dive.com that's CFO dive.com AI startup reaped millions using bogus claims FTC suit says. So to kind of summarize this article, the FTC shut down or made a move to shut down Air AI who alleged that the company misled small business customers with false promises, that it's conversational AI. Quote, conversational AI service could replace human agents and generate huge profits, defrauding clients of roughly 19 million.

[A]

This is what is referred to as AI washing. And so it is propping up this idea that AI is going to solve all your problems, buy our thing, you'll be good to go. When that's really not the case.

[B]

From the AI.

[A]

I'm sorry, from the internal audit side. So for the vendors that are selling into internal audit with their AI tools, I haven't seen anything that made me go, you're absolutely full of it. They all seem pretty legitimate that I've seen. So I think for us it's pretty okay. It's the rest of the organization that I have a little bit more fear of something like this potentially getting them.

[B]

And this comes back to no surprise.

[A]

AI governance and vendor onboarding, I forget the stat. I think it's going to be something like 80% of vendors are going to have AI in their tools by 2026 or 20 in 2026. So this isn't something that is just going to go away, obviously. But as part of your vendor onboarding, there has got to be some kind.

[B]

Of AI assessment as part of that.

[A]

To understand security safeguards for the vendor that you're using, evidence of those safeguards.

[B]

And then also the other note on this is to consider reviewing your own.

[A]

Organizations, like AI related marketing and customer promises, and make sure this isn't something that's happening at the organization that you're currently working at. Thank you for listening, and be sure to follow the link to greenskiesanalytics.com in the show notes and subscribe Schedule time to see how green Skies can make the hype of AI a reality in your internal audit department. All right, that's it for this week. We don't really have a catchy slogan.

[B]

To end the show yet, so if if you have one and you want to send it to us, we'll be happy to include it.

[A]

And if we get a bunch, we'll just do a different one every single time. But until then. Well, I don't know until then, because we don't have anything to leave the folks with yet.

[B]

So have a good week.