← Embracing Digital Transformation
Loading summary
Transcript78 lines
- [00:00]
A
Welcome to Embracing Digital Transformation. Before we dive in, I wanted to personally thank you for listening. Many of the ideas we discuss on this show inspired my new book, AI Augmented Teams. If you're looking for practical ways to combine human expertise and AI to achieve better outcomes, I think you'll find it valuable. Learn more at Paydar AI Books. That is P A I D A R AI Books. Now, let's get started with the show.
- [00:30]
B
I'm realizing there's a lot of interesting topics here. It was like when I first started the research. I was thinking, oh, this would be great. I'll figure this out. I can give a solution, write a book about it, and people can cookie cutter it. But the more I talk to different organizations, I'm realizing it's a very custom solution for every organization.
- [00:52]
C
Welcome to Embracing Digital Transformation, where we explore how people process policy and technology drive effective change. This is Dr. Darren, Chief Enterprise architect, educator, author, and most importantly, your host
- [01:10]
A
on this episode, why 95% of AI projects fail and how to Succeed with Michael Chavira, the co founder and managing partner of Axiologics Solutions.
- [01:24]
C
Mike, welcome to the show.
- [01:26]
B
Thank you, Darren. Thanks for having me.
- [01:29]
C
Hey, I'm really interested about today because I have been quoting this stat. 95% of AI projects fail. And we're going to find out why, and we're going to solve all of those problems today on this episode.
- [01:42]
B
It's very, very ambitious.
- [01:44]
C
Very ambitious. Well, you know, but I'm an ambitious guy, right? So, yeah. And I know you've got all the answers. So, Mike, we'll see how it goes. But before we get started, everyone that listens to my show knows that I only have superheroes on the show. And every superhero has a secret identity and a background story. We won't reveal your secret identity, but we want to know your origin story. We want to know your background.
- [02:12]
B
Okay, great, Great. So, yeah, I'm Mike Shivera. I grew up in a small town in Wyoming. Powell, Wyoming. 5,000 people ended up going to the University of Wyoming. And from there I went into the military. I went into ocs, went to the naval background that way. And through the military, it brought me out to D.C. i was an intel officer for a while, and then when I got out, I was working at Booz Allen Hamilton for a while as a defense contractor. Like, doing that, I learned the market really well. Uh, and I thought that me and a friend of mine, Tom Stauber, decided, like, hey, I think we can go off on our own. And we. That was 17 years ago, actually. Almost. It was just our anniversary. 17 years, just a few days ago. Um, and so, yeah, we've been in business ever since then, and we've been. You know, we're. I met Tom at uva. We both got our system engineering degrees, and, yeah, we've been doing system engineering for. For the government, the military, and intelligence community ever since. And our service offerings have definitely morphed over time. But, yeah, I've been living in the D.C. area now for the last almost two decades, just doing this type of work.
- [03:24]
C
Well, first off, Mike, thanks for your service. I appreciate it more than, you know, and defending our country and doing great intelligence work for our country. I appreciate that. And then also, wow, what a great story, right? Going off out on your own, that's got to be a little bit scary. It is.
- [03:44]
B
I mean, it was. Looking back, it was not only scary, it was very naive how we did it. I'm not sure I would do it the same way again. I mean, we. We left our jobs, you know, and we started working immediately. And Tom, you know, to his credit, like, a week before, had twins, so.
- [04:03]
C
Oh, my goodness.
- [04:04]
B
So he was able to convince his wife to let him do this. And so, you know, she's big part of the story, too. I mean, I thank her a lot for. For supporting us throughout this whole process. And. And now his kids. I always know how old they are because they're the same age as the company.
- [04:21]
C
You say you always know. That's awesome.
- [04:23]
B
Yeah.
- [04:24]
C
So, Mike, let's. Let's dive into. You've been in the industry, especially around defense and government. You've seen major shifts in government. Yes, Over. Over the last. Frankly, just over the last two years, there's been major shifts in. In government. Big shift to AI. Why do you think we're having such a hard time tackling or getting the ROI out of AI that we were all promised?
- [04:52]
B
So I. So this. I said before, Darren, before we got started. I've been having this conversation a lot recently, and it started about a year and a half ago for me. You know, I'm getting my doctorate right now. It's something. It's been a lifelong goal of mine to get a doctorate degree. And when I was trying to think about where I should get my concentration in, you know, I started talking to some of the professors, and now my. My. My chair, Dr. Rajiv Nag. Him. And I started talking about it. And I had helped one of his other students with. With her dissertation, and he's like, hey, you have a great systems background. Like, you Know, he's like, you know, have you thought about looking at this from a systems perspective? And the more I dug into it, the more I thought that this was really interesting. And, you know, I looked at it from. From different angles, from a system's perspective. And then that MIT article came out sometime last year, actually, I think it was around this time last year. It came out where it said, like, 95% of pilots, of AI pilots are failing. And I read that article, I'm like, oh, well, here's the answer. It was like, you know, that changes my, my dissertation topic altogether. But the more I read it, the more I realized, like, hey, that's not necessarily the case. It doesn't really explain the how. And I started posting a few things online on my LinkedIn about why it's failing and my thoughts. And as I've moved through my research, I've now interviewed multiple companies, multiple people from different levels within the organization, not only the leadership. I'm interview some of the doers as well. Some of the people are just getting the job done. And I'm finding that everybody there, everybody's coming at it from good intentions, but they're looking at it from different angles and different lenses overall. They're not looking at, like, the social technical systems. And that's the push I'm taking is like, I'm looking at it from a system's perspective. Like, you know, did your leadership give you training? Have you changed your workflows? Have you looked at your processes and all these things? And every time I see, Every time I, like, I was just talking to somebody yesterday. Like, I'm, like I said earlier, I'm driving across the. The country at the moment, and I ended up. My phone rang. I ended up speaking to somebody yesterday who wanted to talk about AI integration. And he's like, I don't understand why it's not working. Like, I've told them what to do, I've given them the tools. It's just, you know, nobody's adopting it. And I, As I'm driving, I'm just thinking about. I was like, oh, man. I was like, did you look at training? He's like, no. Did you look at updating your workflows? No. You know, have, you know, people. Are you. Have you addressed the AI? Like, are they using AI? Oh, they're definitely using AI. I told them to use, you know, Copilot, but I guarantee they're using other models, they're using other things. And I was like, well, that's a. That's another issue. Altogether, you know, it's like we have to look at it from a government's perspective. We have to look at your culture. We have to look at these different things. And so I'm just, I'm interviewing from different levels and I'm realizing there's a lot of interesting topics here. It was like when I first started the research, I was thinking, oh, this would be great. I'll figure this out. I can, you know, give a solution, write a book about it, and people can, you know, cookie cutter it. But the more I talk to different organizations, I'm realizing it's. It's a very custom solution for every organization.
- [08:08]
A
Hey, sorry to interrupt the show, but I have to tell you about my new book, AI Augmented Teams. Over the past few years, I've worked with leaders across government, industry, and higher education who are all asking the same question. How do we use AI to move faster without losing trust, quality, or accountability? That's exactly why I wrote this book. AI Augmented Teams gives you practical frameworks and proven workflows to help your team deliver reliable, defensible results. With AI, it's not about replacing people, it's about helping people and AI work better together. If you're leading a team, driving transformation, or simply trying to stay ahead, I'd love for you to check it out. Visit Paydar AI Books to learn more and get your copy today. That's P A I D A R AI Books.
- [09:11]
C
Well, and it's, and it's very complex as, as you said. Even the tagline of our show talks about this. People, process, policy and technology. You need all four to make effective change. And I'm seeing the same things, Mike, exact same thing. People lead with technology and it's falling flat on its face because people haven't been trained. Their processes are the same. It's like putting not just lipstick on a pig, but mascara and blush. And it's just, it's just not working.
- [09:43]
B
Exactly. And like, you know, an example I had is one of these organizations I've been working with. I interviewed their leadership and they're like, oh, you know, last two years ago, or maybe it wasn't that long ago, but we instituted ticketing system to kind of help with, alleviate some of the emails and just all the troubleshooting that people have, like a help desk. We introduced a JIRA system and I've never used jira, so I've never, you know, I'm new to it, I'm very, I'm novice when it comes to it, but the leadership was like, we love it, this is great. It's an amazing tool. I can, I can log in, I can see the KPIs and all these different things. And I was like, oh, that's wonderful. I was like, maybe, you know, I'll look at JIRA for my own organization. And then I spoke to some people who worked for them about maybe two or three levels down, and they're like, oh my gosh, like, my workload has gone from leaving at 5pm at night, I'm now here at the office till 8 o' clock at night, 9 o' clock at night. I get a JIRA ticket, I get a phone call, I get emails from different angles, different people. And I'll handle the phone calls first because I just pick up my phone. But then I read the JIRA and like, did I just solve that problem? And then when I try and resolve it, sometimes it's the same ticket, sometimes it's not. And so it's creating a lot of confusion. So that's a process workflow that needs to be adjusted. That's training that also needs to happen. And this organization is very lucky. I mean, there are people that are very dedicated and they have a really great culture because I'm not sure how that that would go in other organizations. If staying until 8 o', clock, 9 o' clock at night just to try and fix a problem, it's pretty amazing. It speaks volumes about their culture.
- [11:24]
C
So that's interesting that you brought up jira, because JIRA introduced a new communication channel in the organization, but the organization didn't make any adjustments to the process they already had, so all they did was add more.
- [11:38]
B
Right, right. Yeah.
- [11:41]
C
And do you think AI is doing the same then? Do you think AI is just maybe possibly adding more?
- [11:48]
B
AI is going to amplify whatever processes or systems you have in place. If they're broken, they're going to amplify it even more. I mean, this company asked me to come in and help them identify where they can integrate AI. And as I'm going through it, it's like, hey, we need to fix some of these processes first. Because AI is just gonna amplify whatever's already broken and then people are gonna find more workarounds. Like, Excel is a, is a big thing. People use Excel all the time, you know, for these workarounds. Like instead of putting tickets into a CRM tool, I'm finding in multiple organizations now people are just using their own Excel files. They have their little notes set up in their emails. And so the data you're Getting and then it goes into the model. It's like the data that's going into the model is not full picture. So people don't trust the output of the model. So it's just a feedback loop that happens.
- [12:39]
C
Yeah. In fact I just posted something on LinkedIn today that said exactly that. If I saved four hours using Genai. But it takes five hours to correct the hallucinations. I'm at a net, I'm at a net negative one. I mean but we're seeing this because of the process thing and maybe data management is wrong and so there's a lot going on with this mic. So what do I do if I'm an executive? Right. Because I have to do something with AI because everyone else is if I don't I'm going to fall behind or am I going to fall behind? What I mean.
- [13:16]
B
Yeah, so what I've already fill out on this. No, what I've been doing is, is I like another organization I've worked been working with and multiple like I, I've been posting a lot of things on, on LinkedIn as well about this and ended up speaking some government organizations too. They asked me to come in and talk to them. Finding out the AI maturity initially is, is very important. So finding out like I go through and I, I look at basically get a pulse check of where the organization is. Like you know in marketing might be great. Like people are using AI there already in a lot of places but it may not be in operations. And so understanding is like where what kind of tools are you trying to, you trying to integrate and where is going to be important. And so I've been going through and coming up with I have a process now set up where I go in and assign an AI maturity for these different organizations internally and say hey this is you know and then I, I try and put it, I put together like a, almost like a two by two matrix of this is where it's going to be. You know, low effort, high return, you know, you know, high effort, low return. And let's, let's try and start like look at everywhere you can put AI in and let's take care of the smaller ones first. Let's take care of those, those low hanging fruit first because that's going to help a lot. And if we do that you can start building trust in the organization, making sure that it works. If it's, you know, you can actually go through and fix a lot of those smaller problems that people may have.
- [14:51]
C
Let's talk about the choosing of a Because you're choosing a specific workflow or use case to go after first. Right? You're not going after one whole org and saying everything's going to AI. You're. You're kind of selecting a use case first. Right. Or a process or a workflow.
- [15:10]
B
I call it a pilot. And so I go through, I was like, hey, this is where it's most likely to be successful. And then I put out a business, I put out a business case as to where, how it could be successful. And then I try and calculate an roi. Like, just like the example you were saying is like, with time saved, like, if you're saving four hours with spending five, that's a negative roi. And I found that it's much easier to not calculate the ROI that we traditionally have learned in grad schools. You know, I'm looking at like, how, you know, let's streamline your process. Let's streamline, you know, how, how much time is, is taking and let's look at time saved there and then we can start putting back as, like, and then I start. Go back and, and calculating those costs and things of that nature.
- [16:02]
C
Okay, so, so this. Because I, I got my Ph.D. mike as well in, in systems. That's the best way to put it, right?
- [16:12]
B
Yeah.
- [16:13]
C
In systems. And what I, I studied and did a lot of research in process re engineering.
- [16:22]
B
Right.
- [16:22]
C
And, and that's exactly what you're talking about is process re engineering. And it's almost like AI is a catalyst to help organizations optimize their processes because that's where the real problem is. Right? That's where the real efficiency gains come from.
- [16:39]
B
Exactly, exactly. You said it better than I could have said it. But yeah, this is a system engineering problem. And so I'm going through and looking at, in those pilots, I go through and try and come up with the training plan. I either, I've been holding like, almost like many sessions for people getting trained up on how to use the AI tool. You know, I've also looked at, you know, does it make sense to either buy the tool, buy a tool that's already been created, or should we just build a tool internally? I know for the government, everybody's been instructed to buy a tool. And so the question they had was like, you know, hey, like, almost the same thing. It's like, what? Like, like these organizations I've spoken to, they have companies in there building tools already, but nobody's using them, you know, and so, yeah, yeah, so it's like, all right, well, if nobody's using them, Nobody's adopting, adopting them. We need to actually go through and like, and, and kill them. Like let's get rid of them. Let's, let's set up a gate process and make sure that there's an adoption right there, that that is acceptable. And, and, and if it's not acceptable, find out why. And you need to listen to your people. You know, if it's not working, let's cut it, let's find another solution and we need to move quickly on that.
- [17:48]
C
So I, I've noticed one trend and may maybe you can tell me if you've seen the same companies are deploying these general purpose genius instead of finding a tool that uses a geni inside it that helps them get their jobs done better. So it's almost like I'm, I'm handing everyone hammers and they're, you, you know what I mean? Because everyone goes, oh, I just bought Copilot for everyone. Well why? Well, because I, I was sold Copilot, so I bought Copilot or we just bought a general license for OpenAI, right. For Chat GPT. Everyone now has access to Chat GBT. Why aren't you more efficient? It's a, it's a weird. But that's what I see happening. Are you seeing the same thing I am?
- [18:38]
B
And that's also, it's pull that thread a little bit more. You know, some people may buy Copilot for everybody or chatgpt, but if you're not accustomed to using it and you've trained Claude or you've trained, you know, Gemini or whatever, whatever you're accustomed to using, there's no incentive for you to stop using your own tools. And so now you have a shadow AI issue that pops up where, you know, this is where governance comes into play. And so I've been writing a lot of governance policies for different organizations as to, hey, we need to start putting out there. It's going to be hard for people to stop using the models that they are accustomed to. We need to put out a policy and almost training as to what information should not be going into these tools. And one thing that has been recently popping up that I've actually found very interesting is that NDAs are being now like when I'm working with different companies, NDAs are having to be signed about, hey, not putting in, you know, patented information into these things.
- [19:42]
C
Into these. Yeah.
- [19:43]
B
You know, and so it's just like, hey, like, you know, they're very protective of their ip and so now it's like they're, they're some of the, they're catching up, right. So there's documents that need to be created and need to be adjusted to make sure the companies are protected. Because one example that I have is there was a woman who used, she, she worked in hr. She had to write up a performance improvement plan for an employee. I don't know what, what model she used, but she just entered it, had created one on one of these LLMs, liked it, printed it out, had the person sign it, and then six months later she was using on a free plan, by the way, and she wasn't paying for it. It was one of those free plans. So she was just feeding into this, these models and then six months later this guy goes in and types in his name and everything that knows about him comes up. Including this. Yeah. And it was like, you know, the little things like that, like need to be adjusted and people need to understand, you know, what can be used and what cannot be used and things like that.
- [20:54]
C
I think people are under the misconception and frankly, I blame the big public gen AI because they say we're not training on your data. That is total garbage.
- [21:06]
A
Right.
- [21:06]
C
They're training on your prompts, they are training on, on your stuff. It's reinforced learning. They're, they're skirting some words, right, Some legal words, but their data, your data is being leaked into these sites. And I guess what people don't understand is they understand anytime your data leaves your laptop or your company boundaries, it's no longer your data.
- [21:31]
B
Yeah, you're right.
- [21:34]
C
You lost control of it. So people have to pay a lot more attention to that. So that governance, you're seeing this as another major hole that organizations have is they don't have strong governance in, in their organizations.
- [21:51]
B
The, or the governance needs to be adjusted to account for the AI.
- [21:55]
C
Got it.
- [21:56]
B
And so, you know, so we need to start going through and doing those things and a lot of it's going to flow up to the governance. I sit on several boards right now, and every single board that I'm on, this has been a topic of conversation like should governance be involved in the AI and to what level? And this has been something that's come up multiple times at this point. And so, you know, I've, I've seen it from different angles. So I see it from the board perspective, I've seen it from the executive perspective, you know, and now I've talked to enough of people who are working that are trying to, trying to get this stuff done that they're everybody's Trying to do the right thing. Nobody's trying to leak out data, nobody's trying to get.
- [22:33]
C
No, they're not trying to. It's just.
- [22:35]
B
They're just. Yeah, they want to get their job done as quickly as possible. And sometimes it's just easier to use a model that you've already trained than using the company's copilot or whatever it is.
- [22:55]
C
Right, right. Do you see specialized tools being developed and being utilized? I know developed that's happening, but. Or people looking for that silver bullet application that does everything.
- [23:12]
B
I would not do the one that does everything. What I've been trying to do is looking at those workflows, see where we can put the AI and developing a genetic tools with that way. So I've actually gone through. I'm not a coder, I'm a system engineer just like you. And so I'm not a coder. But now these tools are so advanced. I've been doing a lot of vibe coding, taught myself tools like Gumloop and Edatan and Lovable and things like that. And I've been very successful doing that and almost doing a proof of concept. Hey, this is the way I think it would work. I'm not sure about the security on the tool that I just built, but these are some other developers that you can actually bring on board to kind of help. Let's clean up the security, operationalize it. Right. So I use a lot of these tools as almost like a proof of concept. Like, hey, this is the way it can look, this is the way it can function. I would not use this just yet. Let's take it in and let's see if we can clean it up a little bit.
- [24:10]
C
So isn't that part of the danger though, with vibe coding and these quick proof of concepts, they see how fast they get to a poc. But to operationalize these things include security, scalability, reliability, all these things that these gen AIs, these frankly don't do very well yet.
- [24:31]
B
Right.
- [24:31]
C
So there's, there's almost like this. Wow, that's super cool. It looks polished. It looks really cool. It understands my use case really well. But it doesn't scale. And to scale it now is going to add some time. Do you think there's kind of a mismatch of expectations there?
- [24:50]
B
A little bit. I mean, I think, you know, like somebody like me like, like trying to vibe code. It's amazing how, you know, if I have an idea for a website or want to make some changes.
- [24:59]
C
Oh, it's so fast.
- [25:00]
B
I agree yeah, it's fast. It's very quick. It's easy. You can test out the tools and figure out how the way to make it look. And all I'm trying to do is speed up the process. Like, instead of paying a web developer to go get all this stuff done for me, I can actually take them almost a finished product and be like, hey, I want you to make it look like this. This is the way I want you to lay it out. You handle it from here. And you're saving some money doing it that way than having them go through the whole process.
- [25:25]
C
But it seems that last step is that last step. We're, we're hoping that it's as fast as the first, you know, five steps that we just condensed. And it's not, not yet. Maybe, maybe it will get there. So do you think that's part of the 95% fail rate that MIT is talking about is Geni is not getting me completely all the way there. And that last five, you know, that last 20% is taking longer than I expect and therefore my expectations aren't met. Do you think that's part of.
- [25:58]
B
I will say that I do think that's part of it. Now I will also caveat. My research isn't all the way done yet. And so I've been finding some pretty cool things throughout my research that I get really excited about. I think what's happening is people are not looking at the whole process overall, and maybe they're trying to. Maybe they're jumping the gun and say, hey, I've done this so fast. It works really great. And they try and put in production, and now they put in production. You know, there's no security. So, so, you know, so people are very savvy. I'm not going to use that. It's not secure. There's other people who are not trained, so they don't know how to use it.
- [26:33]
C
It's.
- [26:34]
B
It doesn't, it doesn't make sense. Like, something that might make sense to you may not make sense to me. And so that's where, like, you know, it's not like an Apple, like, you know, like an Apple product is very intuitive to use. I was dealing, you know, I got involved with a VC as well over the last couple years and I've looked at a lot of different tools that they've developed and that's something that keeps coming up. Like, some of these guys are very smart and they're building some amazing tools, but it's intuitive to only them or somebody who's Very technical. If you want to make it intuitive for the whole market, we need to simplify it quite a bit.
- [27:11]
C
Got it. So you hit right back to the people and the process part of it. A tool by itself doesn't really launch. Right. And so, yeah, I'm, I'll be very interested in your dissertation, in finding out what you found out on all this stuff, Mike.
- [27:27]
B
Yeah, well, me too.
- [27:32]
C
Have you already gone through, you've got it all outlined, you're just doing the research now or.
- [27:37]
B
Yeah, that's exactly where I'm at. I'm doing the research right now. I have a few chapters already written and I've, like I said, as I, as I, and I started doing the research and doing a lot of these interviews, I'm taking a qualitative approach just because it makes sense for, you know, for me to go speak to the different levels. It makes sense for me to do that, gather that information and pull in all these themes that I'm discovering. It's pretty important. And I, you know, to me it's, it's making sense to me, but it's, I'm a system engineer too, so I'm trying to make it so that makes sense for everybody and seeing if there's some commonalities there that, that I can share with, with, with other people.
- [28:17]
C
Well, I'm, I'm anxious to read the dissertation, but it sounds like you, you got to write a book. You got to write a book about this.
- [28:25]
B
Now that's a goal. That is. That has always been a goal of mine. But I, I don't know if I'm going to take a little break after, after all this for, for a bit.
- [28:34]
C
I remember, I remember, Mike, when I finished my dissertation, I took a break in about four weeks after I finished and it was all done. I felt like I needed to write because you're used to writing every, every week you're writing something or you're reviewing something. So, yeah, that's why I've, I've got a couple books coming out this year which should be very exciting on, on this AI journey that we're on. So it's, it's kind of exciting time. So, Mike, if people want to reach out to you, learn more about what you guys do, how do they reach out? How do they reach out to you, Mike?
- [29:12]
B
So I have a, you know, they can reach out to me through LinkedIn. I'm, I'm on. It's, you know, they can find me through my name because I've been just getting asked so many times, you know, from different people. I ended up setting up a company separate from my government company, contracting company called Logo's Edge, just to. Just to kind of keep things separate, keep them organized a little bit. And that's where I've been doing a lot of this AI integration work that's been keeping me busy on top of the dissertation. But that's what I've been doing so far for the last six months or so. And it's been very promising so far. So I've been doing that. Been asked to speak at a few venues as well. But yeah, they can reach out to me through Logos Edge or through LinkedIn. It's probably the easiest.
- [30:00]
C
Great, Mike. Thanks for coming on the show. Especially in the middle of Missouri on your way to Wyoming in an rv. This is awesome. For those that are listening, Mike is not driving right now.
- [30:13]
B
Pulled over.
- [30:14]
C
He pulled over. He told me he pulled over. I don't see things moving in the background. So. Thanks, Mike, for coming on the show.
- [30:22]
B
No problem. Yeah, you gave me an excuse to go buy the Starlink I've always wanted. So.
- [30:27]
C
So it's so jealous. I am so jealous. I have to tell you, I want one so bad. But yeah. Yeah, I gotta take a road trip. Where I have to do a podcast in the middle of the road trip.
- [30:38]
B
Well, let me know.
- [30:40]
C
Sounds good. Mike, thanks again for coming on the show.
- [30:43]
B
Thank you very much. I appreciate it.
- [30:48]
C
Thanks for listening to Embracing Digital Transformation. If you enjoyed today's conversation, give us five, five stars on your favorite podcasting app or on YouTube. It really helps others discover the show. If you want to go deeper, join our exclusive community@patreon.com embracingdigital where we share bonus content. And you can always connect with other change makers like yourself. You can always find more resources@embracingdigital.org until next time, keep embracing the Digital Transformation SA.