← The Audit Podcast
Loading summary
Transcript1 lines
- [00:00]
A
Welcome to the IA on AI Podcast, part of the Audit Podcast network where we bring you Weekly updates on AI from the internal auditor's perspective from AIMagazine.com why Amazon has dropped its internal AI usage leaderboard. It says Amazon has removed an internal AI leaderboard after employees inflated token consumption, instead opting to track deployments. They show real code shipping. So basically, and there's other organizations that have come out and said that they're doing something similar, not nixing the program or the KPI rather, but we're tracking who is using AI the most and there's an expectation that everybody uses it to some degree and there's a baseline and they're going to measure against that. Historically that was pretty okay. And I'll try to distinguish between the two scenarios here. But for the most part, if you have a AI tool that you pay or your organization pays a monthly subscription to, for now you can still kind of use it to the extent that you want to. So like if you have a, you know, the 20 something dollar chatgpt one and you're just chatting, use as much as you want, should be fine. Similar to Copilot, as much as you want, should be fine. The issues are when you use the tools that are token based priced. So we'll use Copilot as an example. In Copilot you can basically use it as much as you want to. Not that big a deal. If you use their new cowork, that is a fee that you have to pay basically every time you use it. You can think about it like that. When you input something, when spit something out, you're paying for that. And the transparency on how that is being like the cost per, let's just say prompt is not great right now. So for example, I know someone who can legit write Python code without the use of AI. So they could do it pre AI. And they racked up a pretty sizable bill from not realizing how many tokens they were using. And I say that to say this person is very tech literate, very AI literate, very data literate and even they got caught up in this. So I didn't want to point that out. So make sure you understand the differences between what you can basically use all you want to and then also what is based on the amount of tokens that you're using. The other takeaway is relative to KPIs themselves or incentives in general. I know I've said to people like hey, you can go into for like certain AI tools, you can see who is using it. And how much or at least if you're an admin, again sticking to copilot. If you're a Windows admin, you can see number of props basically and how often they're using it per day, which is not fantastic outside of the person who maybe uses it like once a day or like very, very little. I don't see the need in posting or making public within your team like this person prompted the most and then this person and this person, this person. I do think it makes some sense to be able to look at that and go, hey Joe Smith. Hope there's no Joe Smith listening or on your team. But hey Joe Smith, you have not used it at all like ever or you never use it. What's up? Like, let's try to figure out why. And for the areas where it does make sense to use it, better ensure that they are using it. So that does come down to KPI. So examine whether like your KPIs are measuring actual business outcomes, quality efficiencies, gains, things like that, as opposed to just number of prompts that are going into the tools. The last thing I wanted to leave you with from this article, if you're looking at the YouTube channel, you'll see this. I'm about to highlight it. This cracked me up. All right, so Dave Treadwell is the Senior VP at Amazon. So in the article it says tokens are the units of data processed by AI models. And his quote was please do not use AI just for the sake of using AI, which is such a complete 180 from what we've been saying for so long of just like, just go use it. Use it for everything. You'll figure it out. Just use it for everything. And here we are going, hey, don't just use it for the sake of using it. So anyway, I did want to leave everybody with that. If you are not at a level of AI literacy that you feel like you should be at, and you do have some kind of monthly fee, pay a flat fee, doesn't really matter how much you use it. I would recommend use AI for the sake of using AI until you figure out where it does make sense and where it does not make sense. Thank you for listening and be sure to follow the link to greenskiesanalytics.com in the show notes. Schedule time to see how green skies can make the hype of AI a reality in your internal audit department.