
In the first episode of IA on AI, John Thomason joins Trent to talk about one of the biggest questions in the audit world right now: Is AI Taking Over Audit Jobs? They break down: What AI can and can’t do (yet) in audit How the role of the...
Loading summary
A
Welcome to the IA on AI Podcast, part of the Audit Podcast network where we bring you weekly updates on AI from the internal auditor's perspective. Here we go.
B
Hey Trent, so glad to be with you again. Always enjoy our conversations. My name is John Thompson. I am the senior vice president and principal at the Hackett Group. Building out a AI practice, AI products and, and a number of search services and products around AI. Been involved in this for 38 years. You know, started out way back when building data warehouses and business intelligence environments and now got into AI in 1991 I guess it was. So started working at IBM, working with the first neural network utility and have been on the forefront of it since then. So really been deep into Gen AI over the last three years and very excited about the conversation we're going to have today.
A
And real quick, for those that are maybe listening and aren't watching the YouTube video, you can't, maybe you can't see John's book Path to AGI in the background. Highly recommend it. Do a really good job of explaining technical concepts to people who aren't technical, especially kind of at an executive level. I think it would be, it's a really good read for, for all those folks. So anyway, with that said, so recently this has been probably the, the hottest thing in the news. And when I say news, I basically mean LinkedIn. It's about my only source these days, but big tech firms hired 25% newer graduates in 24 compared to the prior year. And there are things that are pointing to that that is because of AI automation. So that's from TechCrunch. And then I think it was this week or last week, Anthropic CEO, he said, and this is what I think I've seen Everywhere is that AI could wipe out up to 50% of like ENT white collar jobs over the next five years, which would just, I mean it's going to devastate the economy if that comes true. So with that said, how should John, from your perspective, boards, audit executives, how should they be planning for that kind of workforce disruption? I mean, that is, I don't even know if you can plan for that.
B
Yeah, disruption's coming, there's no doubt about it. I, I don't really agree with what Dario said. You know, he was kind of giving you half the equation there or half the picture. Let's take another example. I think it was two weeks ago, Microsoft laid off 7,000 people and there were a lot of folks commenting on that story. But what they didn't say in that story Was that if you went further or listened longer or read more, however you want to do it. Microsoft also announced they're going to hire 10,000 people. So they were laying off 7,000 to hire 10. Now why would they do that? They're doing that because they're laying off the people that are not AI literate and don't have the capabilities to become AI literate or have publicly stated that they would rather take early retirement or go somewhere else or don't want to be part of this new AI enabled future. So a lot of what's happening is that the jobs that we used to do or we used to have people do. I've talked to a lot of law firms and services firms, and many of those firms really have it wrong. You know, law firms, especially the older partners, sorry, guys, I'm old too, so I'm not throwing rocks at anybody but myself. You know, they're like, hey, people got to come in, they got to sit in the office, they got to be there for 80 hours a week. Got to sit next to me, they got to watch me, see how I do it. The world's changed, man. That's not the way it's going to happen. Right? So, you know, those, those new jobs or those early, those entry level jobs, they're going to be different. And how those people learn are going to be different. So again, I don't agree with Dario. I don't think we're going to wipe out, you know, the early career offerings. Just not going to happen. It's going to be different, but it's not going to be. It's not going to be apocalyptic.
A
Hey, everybody, we're going to take a quick break from our guests. And if you need to get analytics or AI actually working in your internal audit department, or if you already have some of it, you feel like you're not really getting exactly what you need out of it. You know, there's more. You're not getting that. Go to the show Notes, look for the Green Skies analytics link. Click it on the website. There'll be other links that you can click that'll take you directly to a calendar to schedule time. It's literally three clicks to get the time scheduled to get it figured out. All right, back to the show. That's what I was looking around to see. Okay, who is saying what in terms of where they are with AGI? And we can get into that a little bit. Where they are with AGI, how soon is it? Is it coming? And I'm going to hit on that.
B
A little bit with you.
A
And so I looked it up and it was like I just asked for like, what's everybody saying or something to that degree about AGI and when it's.
B
Going to be here.
A
And so there was like, you know, 20, 28, 2030, 2050. And then I went, okay. I said, who's saying this? It was like the CEO of every big tech company. I was like, yeah, no joke. They're saying this like they have stakeholders, like they, you know, like they, they're selling basically. But then I said, I like changed it and said, well, what about like the researchers who are like actually doing this? And it was a little different. And so I'm going to ask this in a different way. Probably obvious everybody where I'm going. But let's take like an audit example and the prompt is just something like, go audit socks, go do the financial statement audit. And we can even narrow it down just to like the IT general controls. Go audit IT general controls for my company. And then I'm just going to sit back and I'll see the final output, I'll review it, I'll check things off, et cetera. I'm almost hesitant to ask you on any kind of prediction because they're just brutal. But if I did, well, I'm going to ask you prediction on that happening when.
B
Yeah, let's, you know, let's answer the whole question. You know, when you go out and look at, and listen to Ray Kurzweil, Elon Musk, you know, Sam Altman, they're all saying AGI is here now or it's going to be here next year. And let's define it. You know, artificial general intelligence as defined by those guys and me too is, you know, when AI is as smart as an above average undergraduate. And that means that AI has memory, it has empathy, it has intelligence, it has everything that a person has. And you know, like you said earlier, those, those people are selling, they got a dog, they got a dog in the fight. You know, they're not objective observers. Now if you go listen to Yann Lecun and he's the head of AI for Meta and, and one of the preeminent researchers in, in AI and has been for decades. You know, he says something around 20, 50 or something like that. Now Rodney Brooks, the guy was the founding director of MIT CSAIL Laboratory Computer Computer Science and AI Laboratory. He's the only person out there saying it's longer than me. I say it's 123 years. He says it's 130 years. So we're virtually on the same page, Rodney and I, and people. I do keynote speeches, I talk to people all the time. I have all sorts of conversations and people come up to me or say to me while I'm up on stage, oh, you're an, you're an AI hater or you're an AI Luddite and, you know, you're against AI. And I'm like, no, I'm an AI optimist. You know, I'm a practitioner, I'm a pragmatist. I build these things. So, you know, when I look at, you know, we've been working on AI for 70 years and I love what Rodney Brooks says. You know, if you look at AI, you know, traditional foundational AI, it's been one of the most successful technologies we have ever built. I mean, it's impactful across all industries and all use cases and all, you know, companies. Now you look at AGI and it's even harder to do. So it took us 70 years to get here. Now I know exponential curves and, you know, quantum, you know, computing and all that kind. I get all that kind of stuff. I'm not a straight line projector. But, you know, I do think it's going to take us a long time to do it and I think that's exciting because it's hard. It's hard to do and I like to get out of bed and work on hard problems. So, you know, the smartest people among us are going to be working on this for a long time. I might be wrong. It might be 90 years, might not be 120 years, but it's not going to be five, it's not going to be six. So it's going to be a while. Now, let's jump back to what you said earlier about, you know, auditing SOX controls in your IT environment. I think that's going to happen almost immediately. You know, that's a very well bounded, you know, for the people listening or just listening and not watching. I'm drawing a box with my fingers. You know, I think inside that box you could do that almost immediately, you know, because we have all the AI, we have all the capability to do natural language processing and to do constraints and controls and target it in different ways and have information wrapped around the AI so it does exactly what we want it to do. So those kind of bounded functions, that's immediate, that's today, that's in the next six months. I mean, if you and I got interested in it and said we were going to do it, we'd have that application done and dusted and tested within months.
A
And, and that's on, that's still be, that's still going to be coming from more of, on a vendor side than the individual auditors with. I mean, I think on average, not on average, what is it something like less? 70% of internal audit departments have less than like 8 people or something like that. So my guess is they're not going to have the tech competency to be able to execute on that. And if they want that, they're going to have to go to a third party to get it.
B
You know, we used to say, okay, well rpa, you know, I'm going to say something that's going to be controversial. That's dumb. Automation, you know, if it always runs this way, RPA is your, your technology. But if it goes in and it branches based on different levels and different, you know, conditions, you know, you're going to have to do that through gen AI. And the great thing about it is you can do it. You know, when I sat down at EY and looked at the audit, you know, practice and they asked me about how much of it could be automated away, my estimate came back at higher than 90, 90%. And they were, they were surprised. They said, you gotta be kidding. And I'm like, no, you gave me the playbooks. I can write all that in a gen AI agent right now. So, so you know, the, you know, the automation in tax and in audit is going to be very high now. Are auditors, are auditors going away? No, I'm not here to scare the audience. That's not the way it's going to be. They're going to be augmented.
A
And when you're talking about like if we said go audit ITGCs for SOCs and thinking about all the different, I mean so many systems that are out there that are going to get touched, I mean just SAP, Oracle, you know, and then there's all, there's a bunch of homegrown, there's this little one, this little one, you know, you know, there's certain clients, when I was at EY, like high end on the Fortune 500, they had 60 in scope applications just for socks and you know, literally thousands of applications otherwise. And so is it, is it, is it your thought that right now, and I agree the technology exists. I've seen it from a few different startups now, hey, go to audit, the IT general controls relative to socks. Are you saying that, that it's going to be Kind of one shot and it can do it across all the platforms. Or is your. Is what you're saying like it's going to go, yes, you can do that one for SAP. And then there's going to be another set of agents that. Yep, that one can do workday or whatever the individual system is.
B
That's what it's going to be. Because each of these systems are unique and different in how they hold data and how you access data. So you're going to have to build up these, these agent farms, these agent workflows and you're going to be able to put the agents in different orders and say, okay, well this firm, the information is held in, in workday for this application, it's an SAP for this application, it's an oracle for this application. And you put them together in different ways and, and they'll be able to run across all those platforms. It's just going to have to be configurable. Like a Lego know, a LEGO kit. Yeah, you could have gathering different ways.
A
And that's some of the fear that I'm trying to alleviate while also grounding people in reality that it is like that's here, but it's not going to be this again, go audit everything and then just kind of sit back like it's, it's still almost individual IT GC. I know I keep hitting on IT GCs here, but it's a good example. And then you keep, and then you go to a different accounting process, a different accounting process for a given industry and things are going to be different there. So again, just trying to set the, the reality of where we are right now.
B
I, I think that's a really good point to hit on and a really good point to drill on a little bit to make sure that the audience is with us is that, you know, AI can do some really interesting things. Gen AI, you know, is. Everybody's all abuzz about it and has been for the last three years. You know, write me a sonnet for my, you know, Labrador puppy. And it goes and does that and it does a great job in that area. But you know, when you start looking at technical areas that have real requirements, you need to work on building it. It's just like any other technology. Everybody, you have to build an application that does that. Now, you know, we're moving into an era where we used to, you know, get the requirements, we build the application and the application ran two plus two was always four and it was a wonderful world. But, you know, it's not magic. You know, we have to look at these things. And we start now, we're all, we're always going to start from the model. You know, what is the model capable of? And then we're going to build up and we're going to get into the requirements for the application. And that doesn't mean that that model can do everything. It can't write a sonnet and do sox, you know, auditing it. That those are two different, radically different things. So when you say, you know, we're going to, we're going to, you know, focus on this area, these controls for socks, that doesn't mean that it's going to generalize over and do the other controls in a different area for, let's say, accounts payable or something like that. That doesn't work that way. You have to build these things for the purpose that you're interested in. So we have a long way to go. We have great tools, we have great technologies, we have great opportunities. We still have a lot of work to do.
A
Last thing, real quick, John. So kpmg, we've mentioned big four firms. I feel like audit podcasts, we kind of have to.
B
Right.
A
They did a survey of directors and found that while responsible use policies for AI are common, formal AI risk governments frameworks, not there yet for a lot of folks. And really one of their kind of root causes was that was basically boards, audit committees, they don't have the right expertise and structure to oversee AI risks. Is that how you're continuing to see. I'm sure it's evolving, but a question I often ask any cie I'm talking to is do you have some kind of chief AI officer, something to that effect? Because honestly, there's not a lot of them out there especially that are great. I mean, data analytics has been around for years and they still hear these stories about super, super technical. They suck at the business and really it's not a good fit. So anyway, how are you seeing that kind of role evolve?
B
Yeah, it's. It's a great question, Trent. And, and it's one of those things that, you know, you, you get into. Yeah, it's. It's just really intriguing. All the conversations I've had around risk management, you know, it's like that. It's just like you said, you end up with people on the call that are risk managers or super technical people or business process people or, you know, people like me, nerds, you know, and you try to get everybody to understand what's going on and what is risk in the AI area, and everybody's talking different languages and different words, and you walk away and think, that was the least connective conversation I've ever had in my life. You know, I don't think anybody understood anything anyone said. You know, you just walk out and it's like everybody's talking past each other, you know, but we will get there. You know, again, I'm an optimist. We will get there. And we did make good. We made great progress. When I was at ey, I spent a lot of time with the risk management department, and I think for the first six months, it was all just setting definitions so we would say the same. We would say the same words, and we finally understood what those words meant to each other. You know, we weren't missing each other all the time. So, you know, it's. It's. It's a classic case of. Of over communicate and over define so you can actually get on the same page. It seems hard in risk management, and I think part of the problem is that people like me, you know, coming from an AI technology background, we don't really care about risk. You know, it's not something we ever think about. You know, it's like, risk, oh, well, our life is all risk, you know, that's. That's what it is, you know, but then you end up in a room with an auditor and an accountant and a risk management professional. Their whole life is about risk. And you're like, do you care? And they're like, what do you mean you don't care? Right. So, you know, you almost have people from Mars and Venus and Pluto, you know, in a room, and over time, it will come together, but right now, it's hard for people to even talk to each other about this.
A
I think I might have asked you this before. The last time I know you've been. What's the number? I don't know. Give us. How many podcasts have you been on in the past two years, three years?
B
No. Maybe 250.
A
Yeah. And it was a lot. Yeah. And I don't even know how many you listen to, but I know there's times where I go, man, I wish somebody would ask me this so I could talk about it, you know, or go. It's not a really good fit for this audience. So I'm not going to try to interject that. Is there a question? When you go on these, you're just like, I wish somebody would ask me more about this.
B
Yeah, I. I always. I. No one ever asked me this, you know, and I wish they would ask me is that are you really concerned that AI is a negative force in the world? You know, that it is more disruptive and destroying than it is positive and accretive and aggregative? I mean, and that's not even a word. But, you know, people don't, you know, I always hear about, oh, is Terminator coming? Is more jobs going to be destroyed? Are all the college kids never going to get work? But no one ever goes to the other side and says, you know, what's the real upside here? You know, what is the what is the really, you know, utopian, you know, view of what's going to happen with AI? No one ever asked me that.
A
Thank you for listening. And be sure to follow the link to greenskiesanalytics.com in the show notes and schedule time to see how green skies can make the hype of AI a reality in your internal audit department.
Podcast Summary: IA on AI – Ep 1: Is AI Taking Over Audit Jobs?
Podcast Information:
In the inaugural episode of the "IA on AI" series, part of The Audit Podcast network, host Trent Russell engages in a compelling conversation with John Thompson, Senior Vice President and Principal at The Hackett Group. Thompson brings over 38 years of experience in AI, from building data warehouses and business intelligence environments to pioneering work in artificial intelligence since 1991. The episode explores the pressing question: Is AI taking over audit jobs?
The discussion kicks off with a pressing concern highlighted by Trent: the potential for AI to disrupt the auditing workforce significantly. Citing recent news from TechCrunch, Trent notes that big tech firms hired 25% more new graduates in 2024 compared to the prior year, attributing this surge to AI automation. Additionally, he references a statement from Anthropic’s CEO predicting that AI could eliminate up to 50% of white-collar jobs in the next five years, posing a dire threat to the economy.
Trent Russell [00:58]: “Big tech firms hired 25% newer graduates in '24 compared to the prior year. And there are things that are pointing to that that is because of AI automation.”
John Thompson responds by acknowledging the inevitability of disruption but challenges the apocalyptic predictions.
John Thompson [02:12]: “I don't think we're going to wipe out, you know, the early career offerings. Just not going to happen. It's going to be different, but it's not going to be apocalyptic.”
Thompson emphasizes that while AI will transform job roles, it won't necessarily eliminate positions wholesale. He provides the example of Microsoft laying off 7,000 employees while simultaneously planning to hire 10,000 more, attributing these changes to the need for AI literacy within the workforce.
The conversation naturally transitions to the topic of Artificial General Intelligence (AGI) and its anticipated arrival. Trent shares his findings from various sources, noting that while CEOs of major tech companies predict AGI's imminent arrival (e.g., by 2030 or 2050), leading researchers like Yann LeCun and Rodney Brooks offer a more measured timeline, estimating AGI’s realization to be over a century away.
John Thompson [06:00]: “If you go listen to Yann LeCun... he says something around 20, 50 or something like that. Rodney Brooks... he's the only person out there saying it's longer than me. I say it's 123 years. He says it's 130 years.”
Thompson categorically dismisses the rapid AGI predictions, highlighting the complexities involved in achieving true general intelligence. He underscores that current AI advancements, while impressive, are still far from the multifaceted capabilities that AGI would entail.
Delving deeper into AI's application in auditing, Trent poses a hypothetical scenario: delegating the task of auditing IT general controls (ITGCs) for Sarbanes-Oxley (SOX) compliance to an AI. He questions whether AI can handle the diverse and complex IT environments typical of large corporations, which may involve numerous applications like SAP, Oracle, and various homegrown systems.
Trent Russell [12:02]: “If you and I got interested in it and said we were going to do it, we'd have that application done and dusted and tested within months.”
John Thompson elaborates on the practical implementation of AI in auditing, suggesting that while AI can handle bounded functions effectively, it would require tailored agents for different systems.
John Thompson [12:35]: “Each of these systems are unique and different in how they hold data and how you access data. So you're going to have to build up these agent farms, these agent workflows... it's going to have to be configurable, like a LEGO kit.”
Thompson underscores that AI's current strength lies in handling specific, well-defined tasks rather than generalizing across diverse platforms without significant customization.
The conversation shifts to the governance of AI risks within audit departments. Trent references a survey by KPMG, highlighting that while responsible use policies for AI are prevalent, formal AI risk governance frameworks are still lacking in many organizations. A primary challenge identified is the lack of expertise and appropriate structure within boards and audit committees to oversee AI-related risks effectively.
Trent Russell [15:48]: “We have to build an application that does that. Now, you know, we're moving into an era where we used to, you know, get the requirements, we build the application and the application ran two plus two was always four... But, you know, it's not magic.”
John Thompson echoes these sentiments, discussing the disconnect between technical experts and risk management professionals. He emphasizes the need for clearer communication and shared definitions to bridge the gap between different departments.
John Thompson [17:46]: “When I look at, you know, we've been working on AI for 70 years... we have all sorts of conversations and people come up to me or say to me while I'm up on stage, oh, you're an AI hater... I'm an AI optimist... we will get there.”
Despite the concerns surrounding AI, both Trent and John maintain an optimistic outlook on the technology’s potential to augment rather than replace human auditors. Thompson stresses that AI will significantly automate routine tasks, freeing auditors to focus on more strategic and complex aspects of their roles.
John Thompson [10:58]: “Are auditors, are auditors going away? No, I'm not here to scare the audience. That's not the way it's going to be. They're going to be augmented.”
Trent further explores the necessity for audit departments to adapt by potentially partnering with third-party providers to leverage AI effectively, given that many internal audit teams may lack the technical expertise required to implement and manage advanced AI solutions internally.
Trent Russell [09:31]: “I think they're going to have to go to a third party to get it.”
In a thought-provoking segment, Trent asks John a question he often hears but rarely gets addressed: What is the real upside of AI in auditing? While the fear of job loss and economic disruption is prevalent, the potential for AI to enhance auditing practices, improve accuracy, and increase efficiency remains largely underexplored in public discourse.
John Thompson [18:18]: “No one ever goes to the other side and says, you know, what's the real upside here? You know, what is the really, you know, utopian, you know, view of what's going to happen with AI?”
Thompson advocates for a balanced perspective, recognizing both the challenges and the transformative benefits that AI can bring to the auditing profession. He envisions AI as a tool that can handle mundane tasks, allowing auditors to engage in more meaningful, analytical work that adds greater value to their organizations.
The first episode of "IA on AI" effectively navigates the complex terrain of AI's impact on the auditing profession. Through insightful dialogue, Trent Russell and John Thompson dissect the nuances of AI-induced workforce changes, the realistic timeline for AGI, the practical applications of AI in auditing, and the critical need for robust AI risk governance frameworks. While acknowledging the disruptions AI may bring, both hosts maintain a forward-looking optimism, emphasizing AI’s role in augmenting human capabilities and transforming auditing practices for the better.
Trent Russell [19:06]: “Be sure to follow the link to greenskiesanalytics.com in the show notes and schedule time to see how Green Skies can make the hype of AI a reality in your internal audit department.”
This episode serves as a foundational discussion for auditors and audit professionals grappling with the implications of AI in their field. It balances caution with optimism, urging the audience to embrace AI's potential while remaining vigilant about its governance and implementation.