← The AI in Business Podcast
The AI in Business Podcast
Fixing the Pilot‑to‑Production Gap in Enterprise AI - with Ronny Fehling of HTEC
00:28:17
Loading summary
Transcript30 lines
- [00:13]
A
Welcome everyone to the Emerge AI in Business podcast. Today's guest is Ronnie Felling, Chief AI Transformation Officer at htech. Htech is a global engineering firm focused on AI centric software and hardware development working across financial services, med tech, automotive, telecom and enterprise software more than 20 engineering centers. In this conversation, Ronnie examines why the transition from successful AI pilot to full production deployment stalls in enterprises and why the root cause is almost always a sequencing problem rather than a technology or data failure. He outlines the conditions that distinguish organizations that reach production from those that don't, including the use of narrow bounded production slices built inside real workflows, decision gates with genuine kill switch authority and executive commit commitment oriented toward learning rather than destination. He also addresses why large scale top down AI mandates tend to reproduce the same failure patterns regardless of the capital behind them and what leaders should put in place before an initiative can realistically scale. Today's episode is sponsored by htech. If you offer AI products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title at the right type of organization and readiness. Emerge attract VP enterprise audiences who are already convinced that they need to move beyond traditional it. To learn the exact strategies we use to help leading AI brands and startups connect with the ideal enterprise AI buyer, visit go.emerge.com partner that's go.emerj.com P A R T N E R Now the conversation with Ronnie.
- [02:09]
B
Ronnie, welcome to our Emerge AI in Business podcast.
- [02:12]
C
Thank you very much.
- [02:13]
B
I'm excited about our conversation today. You've written about what you call the gap between conviction and execution reality. And I know that leadership plays a very big part in that for you. We've seen them say that oh, the pilot has worked, but somehow things still get stalled and there's nothing wrong that they can point out with the pilot or with the AI itself. But for some reason we're stalled from where you said, having lived this from different seats, what does that journey from a successful pilot to full production actually look like when you're inside of it?
- [02:48]
C
Most of the enterprises that I work with, they don't actually have that journey. So they have a pilot or they have had a pilot or they want to go on a pilot that goes well, you know, all things look good and then it suddenly slows down and oftentimes this slowdown gets misdiagnosed. It is a technology problem, it's a data problem, it's a change management problem. Recently heard also adoption problem, but I think it's usually neither. It's typically a Sequencing problem most of the time that I see is that the pilot was built next to reality, not inside reality. As soon as production hits, everything that wasn't in the pilot that you deliberately put out suddenly becomes the work. Integration, validation, certification, real users that have their real workflows, using it differently than you planned, the governance, the monitoring, the accountability or ownership of that. And I think it's often that the team that was set up to build the pilot was not structured really and was not thinking about that from day one and was not planning to carry any of them. And you know what I find now interesting with the recent announcements from last week alone, Tropic, OpenAI, Blackstone, Hellman, Friedman, they all announced their own deployment company with the big private equity houses, roughly more than $5 billion of fresh capital. And it's all aimed at really closing this gap. Even OpenAI, I think was saying earlier this year that they see that the limiting factor for seeing the value of AI in enterprises is not the model intelligence, but it is how AI or how agents are built and designed to run inside these organizations. And that's my thesis. Models, provider's own word. And I think the scarcity of these implementation engineers is one of the most significant bottlenecks to the adoption of AI in enterprises, as Blackstone I think said themselves. And that's a $0.3 trillion asset. Companies telling you that the constraint is delivery, not capability. So I think that gap is real and now you can invest in it. If I can give you a concrete example, I worked on a large telecom operator and there we found we identified a bounded AI assisted configuration workflow. So it's a good starting point, very narrow, deliberately. It's a first production slice. You hear me talk about these production slices a lot. It's real data, real systems, human in the loop, not completely automated, with explicit decision gate at the end. We want to test it, this should work and then we can move forward. So not a platform program of a horizontal data layer, not a full business case, a decision enabling step de risking how I called off. Everybody agreed for a while and then the decision logic shifted and I think that's very often problematic. MVP or this production slice started getting treated like a full investment case. You know, the business case being put through, we had to put the ROI in there for first a 6 and then 12 week program. That's a difficult ROI calculation. And these specifications that kept on coming, they just looked like an RFP for some mature production software. So I think the intent was good, the relationship was really good. And I think the organization wanted to be responsible, acting on the investment, but the timing is wrong. And the result wasn't failure. It was just slowdown, but you know, very harsh slowdown. More approvals, more specification, more governance, less room for iteration, less room for failure. And I think that is the actual failure mode in most enterprises. So what I've seen is the AI programs or these AI pilots, they rarely fail loudly, they slow down, they become more and more complicated, they lose the momentum and then by the time that anybody calls that out, 12 or 18 months are gone. So I think yes, the adoption gap is real, but it's further downstream of this. If the first production slice lives inside a real workflow and removes a real pain, I strongly believe adoption is not a problem. And there I can tell we had a manufacturing case in discrete manufacturing, quite tough, highly regulated. We built this very small system that helps these operators, blue collars, handle non quality events faster. And they came to us before saying these cause us a lot of problems. We get a lot of blame on that. And it worked inside their existing system. It didn't require any change management, it didn't require them to learn anything new. Was sitting slightly and very lightly integrated into their existing system. And the adoption was instantaneous. We didn't have to run a change management program. Their work just got easier. And I think that is the goal that we want to see in order for these pilots. And not even the pilots, right, that you immediately are in production. So it's not really moving but pilots from the pilot mode into production, but starting with production in mind and a very narrow slice. And then the idea what pilots are before you, let's test out and then scale it out. For me it's like, no, you see the actual result, it's been adopted. It is being used by real users that feel pain if you take it away from them. That's the pull that they then generate. And now you can expand the functionality across different operations of that or different business parts. Now you have a real scale factor that you can slowly put the value case and the roi.
- [09:19]
A
And it's interesting because as you're explaining
- [09:22]
B
this, it makes a lot of sense, but it also sounds so familiar because it's very similar to our traditional coding sdlc, the software development lifecycle. It's very similar to having certain slices, basically having an MVP and having rolling out a beta and alpha and those kind of things before we're flat out in okay, this is now being integrated. Is it the same idea? Am I getting it? Is the picture that I'm seeing, right?
- [09:52]
C
Yes. So the mvp and I think it's used in different terms sometimes the production slice idea that can be substituted with MVP is this idea that the slice by itself brings the value already. So it's not a proof point for
- [10:15]
B
the value, it's a proof point.
- [10:18]
C
This actually works. And from there I can scale.
- [10:21]
B
And then we built so foundation, basically, we start off with the foundation and we make sure that foundation is perfect before we start building on top of it.
- [10:29]
C
Exactly. It is really a decision enabling. So at that time, and you might have a gate even before, when you have the value case, roughly. But after this, you should have a gate. And this is how we think, we think in production gates. Typical. And this gate, I think the most important feature of this gate is the kill switch. So you get together and it's like if the numbers don't match to where we think it should be, we kill it or we have to further narrow it down. And that's where I think some of the problems are. If you have big numbers in mind, you know, at first, and you said, well, instead of narrowing, let's widen the scope. And that's, I think, where the failure mode then starts.
- [11:14]
B
And I think at the same time, an executive having to make that decision also has to keep a board happy and explain decisions. And there's a lot for them to think of, and you've been pretty direct about this, that the enterprises that do get AI to production aren't necessarily the ones with the biggest budgets or the most sophisticated technology, but they're the ones that figured out something internally, something organizational. So moving beyond proof of concept, like you just said, what is actually true inside of the organizations that actually make it and what is lacking in the ones that don't.
- [11:49]
C
So I think when it works, it's almost as I mentioned before, like, because the organization respected that sequence. Start with a value ambition, not a precise roi. Pick that bounded slice that actually matters. Build it inside a real workflow, not on the side, not, you know, in the would have, could have, should have situation, you know, of historical data on this. And I think the ones that are successful, they use these decision gates to reduce uncertainty, to reduce risk rather than kill momentum. You know, I think you have to let the operating model that might be there or might want to evolve, evolve through delivery. Not to design it on a whiteboard before, not first create all the governance structure, because even if it works or just as it works, it will evolve as well. It goes often from a centralized to a more distributed model. And I think on the use case discipline to trying to see where I'm going to start first, I think it's important not to pick the easiest thing, you know, the low hanging fruit. It means to pick a use case where the value is real, the workflow is real and you can actually finish it into production. Meaning that the execution friction, that might be data, that might be the technology, but most often is also the process. And the business reality that you're in allows that. I think these are the bounding steps and then it's not always that important to know which one is the biggest, the second biggest, etc. Because these need to go quick, very fast. You know, we're talking here six to 12 weeks to them. And you, you just said it before and I like it. Most of the companies, they pick these use cases that just looked very impressive. And I had a CEO say, you know, that have this wow factor and that's literally what he said in front of, you know, a steering committee. The ones that scale are use cases where someone is going to use this every day. And as I said before, if the business notice, if it goes away. And I think there's another part where I also think some of the push that we might see now with some of the announcements is this pushing for the foundation or the engineering foundation. And let's say we talk about big data layers. We had this 10 years ago around data lakes. And that's also where I would push against perhaps the conventional answer. And I think that the trap is that I see a lot of enterprises once again investing heavily in these foundation modernization, data platform governance frameworks. And they assume that the value will follow once the architecture is clean. It doesn't. We see that now very often. We worked with a company that serious money into data monetization. They expected the transparency and the data clarity on some of the products that they were distributing and out of that the ability to increase, let's say the margin, et cetera. This transparency supposed to come out of this big foundation modeling. So what did get cleaner over two years? The data got cleaner. The decision didn't, definitely didn't. So the standardization is not a value by itself or transparency. And I think this is also where I'm a little bit contrarian or I'm a little bit, let's say pushing back on some of the announcements that we have. With billions of dollars of new capital flowing into these PE portfolio AI services, both with philanthropic and OpenAI, I think we're going to see a wave of Top down mandated AI rollouts into these portfolio companies over the next 12, 18 months. Top down, vendor locked, governance heavy because it comes from outside. Some will work, but I think many will fail exactly the way that I just described because it's late state logic applied to early stage problems. Standardization, transparency before the value signal, platform commitment before the workflow. This is what we've seen. We often deploy small production slices and through that we built possibly the platform. It gets better all the time. If you demand ROI before the production is there, it will be hard. I think that the capital will be there, but the sequence will be wrong in many cases. So let's check that in 18 months.
- [17:05]
B
I'm excited, I'm excited to monitor this and I think we have listeners thinking now hearing what you're saying, following the picture that you're painting and seeing. Okay, yes, Ronnie, I agree with you. I can see why you're saying we're headed into this top down problem with vendor locks and all of that. How do we prevent that? What decisions should we be making now to prevent that?
- [17:28]
C
I think there are a couple of things that when we design it, when we're in the use case, what you actually need I think is usefulness from day one. The adoption is actually a symptom of a useful first slice, not a condition that you can engineer separately outside or that you put on top of that and choose that the work that you do removes the pain from the people doing. And you don't need the change program if it doesn't, no change program will save you. And I think it's these measurable milestones that matter. But this sequence still does. I think on the C suite side, there are probably things that need to be put into place that helps make it true before you can guarantee that something can scale. I think there are a couple of things on one side, commitment from the executive level to learn before we scale. No commitment to, hey, let's do AI. Let's have this abstract thing. We need to have AI in there. Although they get that pressure a lot from the investors and I think their commitment to running real production proofs and then change the plan if necessary based on what they see. Most of the executive commitment that I see is set up for the destination of AI, but not for the journey. Which means that if at the first decision gate that I mentioned before, we come in and say the data says we need to narrow further or we need to pivot or we need to kill, they really want to push through anyway, let's go bigger. And often that's where the programs go sideways. Another point, I think is this narrow entry point that's within a real workflow. I think this is the single most underrated decision. It needs to be bounded from the side enough so that we can finish something in six to 12 weeks. But it needs to be important enough that somebody cares if it works. And it needs to be, I don't know, embedded enough in the organization that it does touch a real system of record, you know, which at the end shows up in the P and L. If the first AI initiative is built next to that reality, it will likely survive the trip into production. But most pilots are purposefully and I think probably the most important. Well, I don't know if most important, but the one that I miss the most is that most leaders skip. They need to prove before you scale, but actually prove it not on a slide deck and know where I'm coming from. You need to prove that this thing works inside production. It needs to have real users, real data, real validation of this and real operational constraints, especially if you go in regulatory or compliance kind of environment. These complex engineering environments as we just love. I think this is that gap between the proof of the value or the pilot and the proof of production where most AI proof of value shows that something looks promises in principle. At worst, it's showmanship, at best, it's historical. But a proof in production shows that it can survive inside the system that has to carry it at the end, and especially in regulated or where we work a lot in these core operational environments, that bar is much higher than most people expect.
- [21:32]
B
It sounds like it's as simple as you need to slow down, to move fast. Like you just take it back a bit, slow down a bit. And then. Is it cliche to say slow and steady wins the race, especially the AI race. But is that the idea that our executives should have? Instead of making the Focus ROI and instead of making the Focus impress the board, it's make sure this works and the ROI will basically come by default.
- [22:00]
C
Yes, I think it's. Yeah, you can skim the cat that way. Depending on what you mean with slow down. I think when I say production in six to 12 weeks, that's incredibly.
- [22:14]
B
That's fast, right? That's super fast. But slowing it down and taking that narrow slice.
- [22:20]
C
Exactly.
- [22:20]
B
Instead of saying, are we going to fully convert? Let's.
- [22:23]
C
Yes, let's not start to transform the company. It is not transformed by the AI. It is really transformed by these in production slices where you see, aha, there's value in there. Organization absorbs this, this works, this doesn't. I think we don't have a top of the funnel problem. We have a problem of making sure that we do the things that we need to do. And most importantly, we kill the things that we don't need to do or we say no to it.
- [22:53]
A
Yes.
- [22:54]
B
And that should, the practical thing that you also mentioned was that we should miss it if it goes away. So if we think, oh, this is really working, let's take it away for a week and see what happens. Let's see if it, if it changes operations. Obviously not in high stakes areas, but just to make sure that, oh, this is not just another piece of technology or another tool that we've integrated. This is actually something essential and crucial and we can build from this.
- [23:17]
C
That's exactly right. There are different ways to encourage or to realize adoption. And it can be a B testing, there can be incentives. They can be. Most of the time the best thing is to take the pain away. And if other people see that you have taken pain away, or you say, let's try it again without it and then see, oh no, the pain increases immediately. Those I think are very good ways. Organizational structures are different on it and you might adopt it to it, but I think in general the governance should become operational, not conceptual. And I always say the structure and you might need a restructuring because it changes the way that we do things, but the structure should follow the signal, not the other way around.
- [24:08]
B
The biggest messages that our executive audience can take away from today. Is that the first thing you said, I think that was probably the first sentence, is that the pilot runs need to run with or next to the reality, not apart from it or in it or it has to be a parallel relationship. I like the respect the sequence part as well. Start with where there's value, start with a small slice that actually matters for the business. If you notice that it's gone, it was valuable. And I like that you said 6 to 12 weeks like we said. That's fast, but it shouldn't take longer. If it takes you longer than that, then we've obviously already have seen new technology and what we've been working on in the last three months might probably be outdated already. Anything else, Ronnie, that you want to leave our audience with today?
- [24:55]
C
Well, you know, I think I was kind of excited, you know, seeing last week's announcements of it because it does validate a lot what operators have been seeing for two years, you know, that the bottleneck was never the model. But I think at the same time. The mistake going forward now will be that we confuse more capital with better sequencing. And I think this quincing idea that says start in these slices, have those decision gates, then go further on that and to adhere to it and to stick with it and prove it, then go next.
- [25:37]
B
Ronnie, this has been a genuinely valuable conversation, the kind of conversation that I think our audience will be able to walk away with something that they can actually bring in their leadership meeting next week. Thank you for being so direct. Thank you for for keeping it real here. We've seen this now from your point of view and you've basically had a bird's eye view of the problem and the problems. It's been great having you on. Thank you so much for sharing your insight.
- [26:01]
C
Thank you so much.
- [26:16]
A
Wrapping up today's episode, let's look at the three key takeaways from our conversation with Ronnie. First, the gap between a successful pilot and full production is almost never a technology problem. It's a sequencing problem. Pilots built next to reality rather than inside it will encounter integration, governance and adoption friction the moment they move to production, and no amount of capital or change management will substitute for having started in the right place. Second, the single most underrated decision in any AI initiative is the choice of the first production slice. It needs to be narrow enough to finish in six to 12 weeks, important enough that the business notices if it disappears, and embedded deeply enough in a real system of record that the result shows up in operations that tool users who feel the pain when it's taken away is the only reliable foundation for scale. Finally, executive commitment to AI must be oriented toward the journey, not the destination. Leaders who set up governance and approval structures for a mature program before a single production proof exists will constantly push through decision gates that should have been kill switches. And that is where most AI programs go sideways. Position your brand alongside the Fortune 500 leaders defining the enterprise AI roadmap for the opportunity to showcase your solution to the executives currently funding and scaling global initiatives. Partner with Emerge to reach the decision makers holding the strategic mandate. Secure your partnership@go.emerge.com partner that's go emerj.com P A R T N E R for further executive level analysis and to join our network of leaders delivering workflow impact with a high. Visit emerge.com on behalf of the team at Emerge. We'll see you on the next episode.