← The Audit Podcast
Loading summary
Transcript31 lines
- [00:00]
A
You know what happens. There's always time constraints within audit, right? The audit has. The clock is ticking. You have to get the audit done, and then what happens is there's a squeeze at the end, and it's just really something that could have been fit and can be fixed with AI. So that's when we decided to jump in and see if we could develop a process within AI that would help take away some of that pain.
- [00:30]
B
Hello, everybody. Welcome to another episode of the Audit Podcast. I'm your host, Trent Russell. Today on the show, we have Ed Zimmer. Ed is the founder of controlsense AI. He's also the principal at Advanced Risk and Controls analytics llc, and prior to that, he was the Director of Audit Innovation at Citigroup for 30 years. So Ed has seen everything innovative in the world of internal audit, the world of tech and internal audit, from the beginning up until its current. We talked to Ed about what led him to launch controlsense, where the pain was coming from that he was seeing and talking to internal auditors, how Control Sense fixes those issues, and then his perspective on privacy security. When it comes to AI, when it comes to tools like Control Sense and other tools that have AI functionality that people are bringing into their organization, this one's a little bit different. So if you're listening through just the audio version, then there is a brief demo that Ed gives of controlsense. So if nothing else, highly recommend. Go to the Show Notes, check that out. You can just check out the demo piece. We'll link to that straight to the YouTube channel, as you can see what Ed and his team have built. And then if you already have something in place, if nothing else, take a look at it and go, oh, we could probably tweak ours to make it look like this. Or there's a good idea. We could work this way. For those that are watching just on YouTube, you will get the full version of this episode. So the demo is included in there. Well, that's it. Here we go. All right, Ed, what's a prompt that you've been using in your personal life? Usually we ask personal and professional, but I know we're going to get to some of the AI tech that you've been working with so people can glean from that there but a prompt that you've been using in your personal life?
- [02:13]
A
Well, I guess my perfect day is pretty simple. I love spending time with my family, preferably at the beach. And I guess my perfect scenario is seeing my grandchildren and my children playing together and they're in the ocean or they're in the sand and. And there's a bit of a sunset behind it. So I know it's not exciting, Trent, but, you know, you have to appreciate the little things because that's what makes life. That's what makes everybody happy, I think. And if you don't appreciate them, then that's an issue. So anyway, sorry. It's kind of a basic thing.
- [02:51]
B
No, I think beach life is the best life. We try to make it down when we can. Okay, what about a prompt that you've used recently in your personal life where you've like something usually travel related is where a lot of people go. But is there a prompt that you use within some kind of AI tool within your personal life that you found was valuable?
- [03:12]
A
Yeah. I've been trying to think about a cruise, believe it or not, to Greece, and I didn't want to go on one of those big ships. So what I did was I wanted to chat gbt and I asked her for some recommendations about some of the more exclusive, not, not exclusive, but smaller ships. And it turns out it costs more money. But at the end of the day, I think it really was worthwhile because my experience on the bigger ships was not great. It, you know, it's just like there's a lot of people there and I just wanted it a little bit more private. So that worked out pretty good. Does that make sense? And it's travel related?
- [03:51]
B
Oh, yeah, yeah. No, the. We did a cruise similar location a few months ago. The breakfast was. I mean, after the second day I was like, this is. The food's horrible. I think the smaller ones are better. But we did something similar. So we use chatgpt a ton of on that because we have a 7 year old and it was basically like, tell us everything that the seven year old can do. And they had like child hours at this place. So it mapped out like, hey, on this day your kid can do this and on this day he can do this. And so it put together a plan for us which was fantastic. And then we're not great with public transportation, so once we made it to land, we used it. We're on this train, this location we're trying to get to here, tell us exactly where to get off and get on. And for the first time in our lives, we didn't get lost on public transportation.
- [04:39]
A
So I love that you really can. You really can rely on it after a while, you know. Yeah, every decision you want to make, you can double check on.
- [04:47]
B
So anyway, hey, everybody, we're going to Take a quick break from our guests. And if you need to get analytics or AI actually working in your internal audit department, or if you already have some of it, you feel like you're not really getting exactly what you need out of it, you know, there's more you're not getting that. Go to the Show Notes, look for the Green Skies analytics link. Click it on the website. There'll be other links that you can click that'll take you directly to a calendar to schedule time. It's literally three clicks to get the time scheduled to get it figured out. All right, back to the show. All right, let's start with your background, because I know you have an extensive long career in audit, in analytics, in AI. And that's where I think the, the aggregation of all that led to launching controlsense. So maybe first start, if you could start us off high level, what is controlsense? And then what led you to launch controlsense?
- [05:45]
A
Yeah, well, you know, that's great because controlsense is really, you know, it brings together everything that I've done from my career, throughout my career, as you mentioned, I've been working on audit analytics and, and once, you know, I actually wrote a book on audit analytics that really, I think, demonstrates that I kind of still am passionate. Kind of geeky a little bit, but passionate about this stuff. But what came when you saw what AI can do and the capabilities that has. Well, that's where controlsense came up. Right. We decided to, instead of going into ChatGPT on all this stuff, to create a customized AI assistant that's geared specifically for internal audit. So it's not going back and forth trying to figure out all the audit requirements. It actually has a lot of capabilities to go in and be specific, to access quality standards, methodologies, and to go in and really develop an assistant that really automates a lot of the manual steps within our order process.
- [06:58]
B
And from that, what are some of the. I don't want to say features necessarily, but so maybe some of the pain points that Control Sense is designed that you designed to solve that. Maybe I don't want to call out other tools, but maybe some things that other tools don't do or some pain points, again, that controlsense is designed to solve.
- [07:21]
A
Yeah, well, you know, that's a great question because I have a lot of experience in audit, and some of the pain points that I experienced both as an auditor in my younger career and then as a manager is that there was a lot of inconsistencies in the development of audit deliverables. So there still are a lot of processes within audit that are manual and when people have different skill sets and different backgrounds, unfortunately, the end result is that the quality of audit deliverables, such as a risk and control matrix or issues or audit reports can become varied now over time. There's a lot of back and forth when it comes to the review process. And that can be painful at times because, you know, people go back and forth, there's, it's frustrating. You know what happens? There's always time constraints within audit, right? The audit has, the clock is ticking, you have to get the audit done. And then what happens is there's a squeeze at the end and it's just really something that could have been fit and can be fixed with AI. So that's when we decided to jump in and see if we could develop a process within AI that would help take away some of that pain and make the whole audit process not only more efficient, but higher quality. And I think quality is a key also.
- [08:53]
B
So you mentioned inconsistencies in the various audit processes like the RCM generation, report generation or report writing. Those are two specifically that you mentioned. But obviously there's a ton of inconsistencies that also can exist throughout the life cycle of an audit. What's controlsense doing to fix those issues?
- [09:11]
A
Well, what it really does is it takes, it goes into the deep pool of like audit best practices, emerging risk data and it extracts this information and it creates a very specific format that we can customize to meet the individual needs of each audit department and each audit methodology. So you know, right now we have our, I'll call it our base product is a very well formatted deliverables including announcing memos and risk and control matrices and some of the things we talked about. But what we can really do then also is the, we can customize and map the outputs to be specific to the individual requirements again of the methodology that every audit team has. So it really takes away a lot of the back and forth and a lot of. And it increases order quality. And when it comes right down to it, Trent, increased order quality is really the key because that is the differentiator when it comes to our order stakeholders relying upon the insights that we need to provide them. That's our primary job. And if we don't do that in a, in an efficient, high quality way, then we're going to be, we're going to be lacking from their expectations or not not handling their expectations.
- [10:39]
B
I know when I was especially an external audit, quality was basically it was about do we have to some degree, do we have typos in this thing? Do we test this thing correctly? I mean, even, you know, down to the typo level was considered quality when to my reviewed R work for quality. But to say there's an increase in audit quality, I could see some listeners definitely going the route of the quality of the testing, for example. But to your point, if, like I could see that being a mission statement or a vision for a CAE going, our vision is to always increase audit quality because if we increase across the board, if we increase it relative to the rcm, if we increase it relative to work papers, if we increase it relative to reporting, then all of that together actually adds value to the organization, really starting with the rcm, in my opinion. But anyway, just kind of want to get your thoughts around that.
- [11:39]
A
Yeah, no, exactly, because let's take the rcm. You know, almost every auditor should have a good understanding of what they're going to be auditing, either from prior work papers or from whatever research they do during their audit planning process. But what controlsense can do is it goes in if you identify the business that you're auditing, it will come back with suggested sub processes to the rcm, suggested risk controls over those risks, and even suggested test procedures that can be performed. Now, all this would be what I'll call a really good first draft. And then the auditor still has to apply their business judgment to go in and see whether or not the suggested risk controls and tests make sense from their perspective and then add to and change whatever they need to. And from a quality point of view, it just makes everything more complete because controlsense does go out and it has the opportunity to identify some of the key current and emerging risks that might not have been considered, let's say, in prior audit work. And let's face it, you might know this from your public accounting experience. Auditors tend to rely on prior work papers as a starting point. And you have to get current with this information or otherwise you might miss some very, very important risk and controls.
- [13:08]
B
Despite the mantra of don't say this is how we did it last year. I remember being in public accounting and I remember like they pushed that so hard. I had a manager. I had been, I think I was my first year as a senior. So I'd been there two full years going into year three. And she was like, why? Why'd you scope this in? Or why are we doing this test? And I went, that's how we did it last year. And I went, oh, no, come on, you know like even though it's pounded in my brain, still felt, still fell for it. Well, I will say I've been derelict in my podcast hosting duties. I know I've seen the Control Sense website before, but when you're talking about increasing audit quality the way you're talking about it. So apologies for if you saw me looking down, typing while you're talking there for a bit. I went and looked at the website because I was like, if I don't see that on the website immediately, then I'm not sure I believe what you're saying. And I'm not joking, that was my thought. And so I pulled it up while you're talking and it's the first sentence on there I went, okay, Ed really believes this.
- [14:14]
A
Yeah.
- [14:14]
B
So that was nice to see. And having known you as long as I have, wasn't super surprising, but okay. I feel like there's probably people, especially when it comes to tools and AI tools that are going, okay, well we have CoPilot, we have ChatGPT, so what's the difference? So like, what, what makes Control Sense different from, let's say a co pilot or a chat GPT?
- [14:36]
A
Yeah, well, you know what, controlsense, those are great tools, but controlsense really does go far beyond some of the generic AI tools. And the reason is it's specifically built for internal audit. It understands audit standards, quality requirements and workflows. But our key differentiator, as I mentioned before, is that we can customize the outputs to be specific and meet the, not only the quality, but the format and the methodology that each, every, every individual audit team has so that you don't have to go in and reformat everything. And I guess you could try prompts and get into specific prompts, but at the end of the day, controlsense brings this all together throughout the entire audit methodology, from planning, field work to reporting. But then even after that, we can go in and start monitoring whether or not any outstanding issues are due to be completed. And we can do follow ups with our audit stakeholders on any issue status so that we make sure that we're in sync with this. So it brings everything together into one cohesive platform rather than going back and forth.
- [15:59]
B
I'm a huge fan of having vision, mission, sticking to it, and that being the North Star. And it seems like yours is increasing audit quality, so I'm glad we hit on that. But I'm sure there's people that are going like, all right, how do we actually use Control Sense though? So how do they actually use it?
- [16:15]
A
Well, here you go. I mean, what I think we can do is before we, I like to show you a demo eventually, but before I do that, you know, we can just go into controlsense and let's start with order planning. You can go in and just type in your basic audit that you're going to be doing. Controlsense will come back with suggested risks that can be provided to the auditee as part of your audit planning memo to announce the audit. Then the next step in the planning process is creation of the risk and control matrix, which we talked about as maybe, probably the key step within the audit. Controlsense again, ask very simple questions and I'd like to show you the risk and control matrix later. It goes in and it suggests not only the initial process, but any sub processes, suggested risk controls, audit tests in a very, very well formatted document and tret. I've done risk and control matrices in the past. It can just be a bit of a pain going back and forth. And this is a really good starting point as a high. And by the way, these are all first drafts we're not taking away. We're not saying that this is complete. The auditor has to apply the judgment. But at the end of the day, the RCM will be, I think, completed much more quickly and also be more complete. Then you go into field work like other AI tools. You can grab some of the data files from, let's say transaction files and perform testing. But again it's in the same platform. So you can bring in the large data file of full transactions and then the experienced auditor or any auditor can go in. You don't need a technical skill. You just use some prompts and you can test key controls that are automated like a maker checker control or making sure that the same person didn't enter and approve a transaction. And you could do that across full data populations. You don't need analytic skills anymore to do this stuff right. And then you can also identify anomalies so that during field work you have a leg up on that. But then let's say in this process you do identify a finding. The first thing that you should really do is take that finding and share it with your stakeholder to make sure that there is accuracy and that the finding is, is something that, you know, make sure the factual accuracy is there. But what controlsense does is it'll just take a very basic finding and it'll create a very well formatted issue. Content root cause it has all the, all the areas of a well formatted issue and it does it very, very quickly. It does it almost instantly. Give it to your stakeholder. You go back and forth a little bit, but it really cuts down the amount of time that it takes to validate any issues. Finally, then, you're into audit reporting. Now controlsense will grab the final risk and control matrix. So, you know the one that we talked about earlier, you probably updated it based upon the auditor skills and experience, but it'll look at the test results to see if they're effective or ineffective. And it also will indicate whether or not there were any issues or any findings. So controlsense will just take that information, grab it and create a very well formatted audit report, literally in seconds. And I think to me that was one of the, the best things that I found because again, people with different skills and experience, you know, you have an audit team that they're trying their best and they could be really good, but nobody's going to do it in a really consistent way. And as a reviewer of these things, I would love to see a good first draft, so to speak, that can then be expedited. And I think you could cut down the reporting process very, very, very quickly.
- [20:33]
B
I think the point of the inconsistencies coming back again and now that you've spoken that a little bit more and really nailed that the way I was thinking about it. So in the field work, you can actually do full population testing on the data, which I had not seen that in a tool yet, an audit specific tool. So I thought that's really unique. And so if we talk about roles, you would have to have an analyst to do that. And then within reporting, usually those are inconsistent because we're not report writers for the most part. And the two areas that I see people get the most trained, like internal audit departments get the most training is on analytics and it's on report writing. And so to have the, not only the ability to do that at a more effective level, but then also the consistency across each one of those, I did not realize that that is, that's impressive. Okay, so for the. You can add, if you have any thoughts, you can add those.
- [21:34]
A
No, no, no, that's perfect. And you're right, it really, you know, it's really streamlined things. Not only makes them faster, but it just gives auditors more time to think, which is sometimes, you know, and I say that kiddingly, but that's what we get paid for. You know, we're paid to really provide insights and as opposed to worrying about formatting and whatever. Formatting is important. And that's why this is so Key, it can execute it very quickly. Okay.
- [22:04]
B
I laughed when you said that because there was a data analyst, an internal audit data analyst. And I asked him one time, I was like, what's the biggest pain? Or if you could have one thing, what would it be? And like, immediately he said, I just wish the auditors would think before they did something and go, hey, maybe I could send this to the DA person instead of me doing it by, I'm just going to do the same thing. Because that's how it, you know, we've always done it. So that's why I kind of laughed when you said that. Okay, so for the folks that are just listening to the audio, we are going to cut over to the demo. And so that's going to be on YouTube. It's also going to be on Spotify. So if you're Spotify listener, you should be able to see that on video. If you're itunes listener, now would probably be the time to switch over to the YouTube channel. We can put a link in there directly to this. All right, so if you're a listener and you didn't go check out the YouTube video, but you're still with us, highly recommend. Go check out the demo. But to that point, as we were going through the demo ad, one of the kind of burning questions I had and I feel like definitely auditors are going to have is, hey, how are we? How are we okay putting our data and our information into Control Cell?
- [23:16]
A
Yeah, as an auditor, I fully agree with that. And that's why security is really one of our major priorities in developing control sets. The way we did it, our large language models are firewalled inside the organization's security perimeter. So no data that you enter will actually leave your organization, leave your company's infrastructure. So that's all protected from there. And we are also, Trent, going to be moving to a SaaS model so that any companies that maybe don't have an existing LLM infrastructure can then use controlsense with confidence that the information is not going to be going out into any public domain. So all this information will be fully protected.
- [24:09]
B
Excellent. As considering your long career in audit, I would have been surprised if you're just like, yeah, we don't really know. We haven't thought about that yet. Well, we're going to work on the privacy thing some other day. So, yeah, zero surprise there.
- [24:21]
A
All right.
- [24:21]
B
I was really impressed with controlsense in general, but the ability to be able to test controls and do full population testing within it, maybe it's just the data side of me, I thought that was super cool. And then also the report writing because I hated doing that and it being able to do that amongst everything else. I also had hated doing rcms, so I think that's cool also. And I'm honestly jealous. That was full time auditor. We didn't have something like this. So very, very nicely done. With that said, I'm going to stop talking. I'm going to hand the microphone over to you. What do you want to leave the audience with?
- [24:57]
A
All right, well, thanks, Trent, and thanks very much for the opportunity. First, I just think we wanted to mention that we are going to schedule a series of webinars that's going to provide live walkthroughs that provides a deeper dive into controlsense. Just because of the sake of time. I didn't go into all the features, but I do think that if you could just take a look at the podcast notes, there'll be information in there regarding the webinars that we'll be performing with the schedule. And second, I just think it's important to go to controlsense AI. We do offer an extended free use of the platform so you can see how easy it is. You can see whether or not it actually will meet your particular department's needs. I really do think, and I hope it will, but it gives you the opportunity to kind of try before you buy. Okay. And I do encourage everyone to test the waters of AI assistance. I think it's certainly definitely here to stay and it's just going to always get better and just embrace the efficiency and effectiveness that it comes with the use of Generative AI, because I do truly believe that it's not going to replace auditors. But if you do resist the use of Generative AI, that might be the person that's replaced. So with that said, I do feel I very much appreciate everyone's time. And again, if you have any opportunity to take a look at controlsense AI, please do. Thank you very much.
- [26:39]
C
Hey everyone, thank you very much for listening to this episode of the Audit podcast. Whatever platform you're listening on right now, I'm sure there's a subscribe button somewhere, so please hit the subscribe button there. If you're listening through itunes or Spotify, feel free to go give us that five star rating. It only took me about 16 seconds to give myself a five star review and it really helps to get future guests to come on the show, so we'd really appreciate that. Lastly, be sure to check out the show notes and follow us on all our social media channels on Instagram, on LinkedIn and on TikTok. Also, if interested, please sign up for our weekly newsletter from the Audit Podcast.
- [27:12]
B
Thank you all. Have a great one.