E146: The 92% AI Failure: Unmasking Enterprise's Trillion-Dollar Mistake - How I Invest with David Weisburd

Loading summary

Transcript54 lines

[00:00]
Interviewer
An AI native solution. The framework is not just that AI is replacing what a human is doing, but how would you design the model with AI in mind?
[00:09]
Matt
I think most of the material benefit you're going to see is when you clean sheet any process to be like, how would I design this process knowing all the AI tools I have from scratch? And how do I use both technology and humans? And by the way, I think the example for that is going to involve both for a long, long time. In fact, I think humans are a core part of this solution. I think in Invisible we believe that it's the human machine interface where all the value sets. But it's not necessarily just giving all people on an existing process and a tool. It's redesigning the process to use all the tools you dispose of.
[00:38]
Interviewer
So let's talk about Invisible. Give me some specifics on how the company is doing today.
[00:42]
Matt
I joined in mid January. We ended 2024 at 134 million in revenue, profitable. We were the third fastest growing AI business in America over the last three years.
[00:54]
Interviewer
So how will deep seek affect Invisible?
[00:57]
Matt
The viral story was that it was $5 million to build the models they did. The latest estimates that have come out since in the FT and elsewhere would say it's close to 1.6 billion. I think the number that's been cited from a compute standpoint is like 50,000 GPUs. So if you had just told that narrative as the exact same story, but with 1.6 billion of compute, I don't even think it would have been a media story. The fact that it cost over a billion dollars to build that model means it is a continuation of the current paradigm. Look, there are some interesting innovations. They've had mixture of experts, they did some interesting stuff around data storage that does have some benefits. I'm using compute costs, but I think those are things we've seen other model builders experiment with already. If I think about types of data, they basically went around things that are base truth logic, like math, where there's a fair amount of synthetic data available. That's a fairly small percentage of the overall training tasks that I'd say most model builders are focused on.
[01:46]
Interviewer
Tell me more about that.
[01:47]
Matt
Think about training as kind of three main vectors. So you have base truth information where a lot of synthetic or kind of Internet broad based data exists. So math is a really good example of that. Then you have tasks like creative writing where there is no real kind of AI feedback, there's no synthetic data that's existing, there's no way to train those models without human feedback. But the most interesting one is you have a whole set of base truth information where you also don't have enough synthetic data. So an example of that I would give would be Computational biology in Hindi. The corpus of that is just not broad enough. Each branch of that tree and each topic you train off of will have a different approach.
[02:19]
Interviewer
And tell me about what invisible technologies does exactly.
[02:23]
Matt
We have two big components of our business, what I call reinforced learning and feedback which is the process on any topic where model is being trained. We can spin up a mix of expert agents on that particular topic. So that could be everything from. I mean I use the example of computational biology in Hindi. Our pool has a 1% acceptance rate and about 30% of the pool is PhDs and Masters. So these are very high end specific experts. The funniest one I talked about recently, somebody is like falconry in the 1800s things where there's just not a lot of good existing data. And look, I think models are going to be built on the full corpus of information that is mattered to humanity. So there's a lot of branches of that tree and we bring all of the different experts to help train those models. But that's only half the business where we're seeing increased focus and demand is on the enterprise side. The big challenge today and the kind of chasm that exists between let's call it Silicon Valley and the enterprise is there's a demand for broad based model development which is really important. But I think what a lot of the enterprise is looking for is how do I get those models to then work at 99% accurate accuracy in my specific context.
[03:21]
Interviewer
Tell me about some examples of enterprise models that have worked.
[03:25]
Matt
Therein lies a great question. The stat that I've seen most frequently cited is that about 8% of models today make it to production. The two largest high profile public enterprise cases I've seen are Moody's had a chain of thought reasoning example and then probably the most often cited one is Klarna had a contact center where they basically built up an entirely Genai center contact center to replace the old contact center they had. The realized impact in the enterprise has not materialized the way people expected it would. I am very bullish on where it will go but to date those are the only two examples I can cite. I can cite some pretty public struggles but there have not been many other realized examples that I've seen.
[03:59]
Interviewer
So there's hundreds of billions of dollars being put into this problem set. Only two successful examples where are the main frictions and how do you see that evolving over the next five, 10 years?
[04:08]
Matt
Most of that money to date has gone into building the models that are extensible, generalizable and moving towards greater levels of intelligent change of thought reasoning. We've seen unbelievable progress phase of the model building process. The challenge is let's take, let's say you're an insurer and you need to build a claims model. What you need to know is that your model works with perfect accuracy. You're not or 99% accuracy. The investments have been have led to material improvements. It's that the motion of then taking those models and fine tuning them in an enterprise context has not been standardized yet. The motion of how do I deploy a machine learning model with accuracy. You've seen a bunch of really good examples of that. Like straight through processing of mortgage loans is one example where those are being productionalized. They're working. There's a ton of examples of impact coming from machine learning deployments. AI has not really figured out it's what I call production paradigm yet.
[04:53]
Interviewer
The OpenAI's, the Anthropics, the XAI's of the world developing these incredible generalized models. And then you only have really two use cases for enterprises. You mentioned fine tuning. What are the other steps that a company needs to go through in order to make their AI work?
[05:07]
Matt
Let's take an asset manager that is going to build a system to do ongoing reviews of its assets based on its internal investments. Right. The first step you need is you need all your internal data organized and structured and in a place where you can use it and access it. That's probably the biggest challenge most people face is there's a joke. I like to say that when, when good AI meets bad data, the data usually wins. And I think the challenge is if your internal data environments, if you don't have a clear definition of your, your assets, your products, if you don't have kind of what I'd call organized core data domains, it's very hard to even use AI until you've got that organized. That's probably the biggest challenge I think the enterprise faces right now is most of the data is on systems that are, that are a decade or later old. It's not organized or mastered across those systems in a way that they can use it in gen. So that's one big problem. I think the bigger than issue is so let's say you get that all organized. David, I'll put you on the spot to give you an example. So let's say that you built a. Let's say you built a model, a gen AI model to produce summaries of investments in the financial services space and just kind of look at new investment ideas. How would you build a chatbot to do that? You spend money to do it. At the end of that you have a model that will start generating these kind of investment memos. How would you define a good memo or a bad memo at scale? So let's say it generates 10,000 memos. How do you know it works?
[06:16]
Sponsor
If you run a fund, then you know having instant access to fund insights. Modeling and forecasting are critical for producing returns. Which is why I recommend Tactic by Carta. With Tactic by Carta, fund managers can now run complex fund construction, round modeling and scenario analysis. Meaning for the first time ever you could use a single centralized platform to do everything from analyzing performance to modeling returns with real industry benchmarks. Most importantly, communicate it all with Precision to your LPs. Discover the new standard in private fund management with access to key insights and data driven analysis that could change the trajectory of your fund. Learn more@carta.com howiinvest that's C-A-R-T-A.com H O W I I N V E S-T.
[07:04]
Interviewer
It'S difficult to do at scale at 10,000, but I think on an individual basis a good model is a model that hits all the points and then has more clarity and more details on the sub points. So I would evaluate it based on did it get all the main key points of the investment thesis at a high level and then were the sub level points sufficient or covered the main topics?
[07:26]
Matt
What you're saying makes complete sense, which is you have a set of parameters or set of outcomes. You're looking for a memo. Even if you have that though, the question then becomes how do you evaluate that consistently across 10,000 memos? And I think this is the difference between backtesting of ML dataset vs gen is you need a way to actually go back and validate that what is produced works. And I think that has been the real challenge that the enterprise has struggled with is you may have a sense for what good looks like. You might say, for example, the definition of a good investment mo would be like at least a paragraph summary of competitive set some context on the market, including growth rates. Like you could set a set of parameters that you're looking for to answer, but then you have to wade through and kind of assess all that. And so what we spent the last eight years doing for the model builders and others is building what's called semi private custom evals where we effectively set parameters like describing that would say these are the definitions good, this is the outcomes we're looking for. And then we use human feedback to score those parameters. So we could go at big scale and say does this outcome cover what you're looking for? And we bring subject matter experts to Bayer to actually do that scoring. I think that's actually been the big gap is these are often things you can't score with a random person in the street. You can't just put it into market and hope it works. You need a subject matter expert say this looks generally good before any organization gets comfortable launching it. One way I've seen enterprises do that is I've seen a couple customers experiments already is they'll actually have their own employees evaluating this at huge scale. But if you think about the time suck of like having large numbers of people just reviewing gen outputs, that's very hard to do. So I think a lot of what we've now evolved to is on the enterprise side, a mix of these kind of evals and assessments of the models that are happening that we then help customers fine tune and improve their models.
[09:00]
Interviewer
Is there a gap between what a generalist searcher might want and somebody domain specific? In other words, if I'm making $100 million investment decision based on a MEM, that has to be much better than if I want to find out if, you know, dogs could eat a certain type of food. And what's the best practice for raising a healthy dog?
[09:17]
Sponsor
Post earnings reports are more than just a data dump. They're a goldmine of opportunities waiting to be unlocked. With Daloopa, you could turn those opportunities into actionable insights. Daloopa's dynamic scenario building tools integrate updated earnings data, letting you model multiple strategic outcomes such as what happens if a company revises guidance. With automated sensitivity analysis, you could quickly understand the impact of key variables like cost pressures, currency fluctuations or interest rate changes. This means you'll deliver more actionable insights for your clients, helping them navigate risks and seizing opportunities faster. Ready to enrich your post earnings narratives? Visit daloopa.com how that's-a L-O-O-P a.com how today to get started, One of the.
[10:04]
Matt
Questions you're asking here is what is the bar or the risk bar for production? Depending on the use case. And I do think it's different. As an example, if the goal of a chatbot is just to say something like Review restaurants and it's a consumer facing the risk bar on that is, does it say anything toxic? Is there any bias? You can put some risk parameters around it, but you don't really need kind of subject matter expert feedback on it. I'll give an interesting one. Legal, like law, the bar for accuracy on that is materially different than a consumer example. And many law firms are experimenting with this. But it's hard to assess something like a debt covenant agreement without subject matter experts weighing in. On its scale, the outputs are consistently good.
[10:42]
Interviewer
How do AI models evolve when parameters change? Is this something that'll always be needed to be refreshed?
[10:48]
Matt
There are two ways that models generally be consumed. Many models that will be consumed by consumers that are effectively just going to be what the model builders produced. Usually the way the enterprise is using these models is they are tailoring those models to their corpus of information. I'll give an example. Let's say that you have two wealth managers. Let's say, hey, I'm going to make up to Fidelity and T. Rowe Price. And they want to use, you know, experiment with things like robo advisory or question answering. They're not going to just use an off the shelf framework for that. They're going to tailor it off of all of the information that exists in their communications and their training documentation. Any model that's being trained at the enterprise is usually being trained off of the internal knowledge management corpus that that institution has. And so you're using the large language model from the model builder and you're tailoring it to your specific context. That process is called fine tuning.
[11:31]
Interviewer
Prior to becoming CEO of Invisible, you headed McKinsey's Quantum Black Labs, which is their AI labs. What did you do at McKinsey?
[11:39]
Matt
I focused on three main things. One, kind of all of our large scale data transformation data lakehouse data warehouse builds. So the first thing I mentioned, which is if your data is messy, it's very hard to use AI. I spent a lot of time focusing on that. I spent a lot of time doing custom application development. So building all sorts of different applications, whether that be for retention, pricing, contact centers kind of software, custom software that people could use to deploy models. And I do think that's an understated part of a lot of this is there is what a model does, but then there is the way that somebody can understand it and interpret it. And a lot of that is the user interface by which they consume it. And so I think that's something the enterprise is spending a lot of time on is what is the user interface by which people consume and think about and make decisions around these models? And then the third area I oversaw, the Gen Lab, which is McKinsey's kind of global genai tool build. We were doing it when I was there. We're doing anything from 220, 240gen AI builds at the top.
[12:27]
Interviewer
I want to double click on the enterprise side of what you did at McKinsey. You mentioned those two high profile use cases for successful enterprises built. Did you build any successful enterprise use cases while you were at McKinsey?
[12:38]
Matt
We definitely did. There's a public case you can reference that a couple folks in Quantum Black build for ING where it's effectively a chatbot. And one of the things they mention in it, very similar. What I'm saying now is a lot of what was required to put that in production was getting it to 99% accuracy. So they had a lot of parameters and fine tuning they did around testing, quality, controlling it, building audit LLMs to make sure that the outcome is good. We definitely did a lot of that. But the rough math you see across the industry is about 8% of gen AI models make it to production and that's broad based. The amount that kind of stall around the proof of concept pilot phase is pretty material. I think it will get better over time. But to date the challenge has been a lot of things I mentioned challenging data, an unclear definition of good and an unwillingness from the folks in the field to actually use and believe the outcomes of the models. And I think that that's going to take time.
[13:24]
Interviewer
There's this concept in the productivity space which is 80% done is 100% good. Is there like an 8020 rule here where you could use AI to solve many things and dramatically decrease your need for sales representatives, for customer support. And does it have to be 100% good?
[13:43]
Matt
That is a really complicated question. So there's another analogy which I'll use which we manufacturing lines in a factory. And so if I ask you the question, If I have 10 people on a manufacturing line and every one of them saves 5% of their time, what is the line savings?
[13:58]
Interviewer
I'm guessing half a person.
[14:00]
Matt
Zero. Because you can't take out a line and you can't take. No, no person can be taken off that line. You effectively just move to a world where everyone has a little bit more free time. And I think that's the challenge. The 8020 here is things like copilots have had a very interesting kind of last two years in that they they are helpful coding, co pilots, legal copilots, all these things. But it's unclear the degree to which they actually save any work. They kind of tweak a lot of things on the margin. And I think the difference I'd say with 8020 is I think to do that well, you actually have to re engineer processes. You have to say what does my end to end workflow look like for claims processing or whatever that might be? And how do I take out two full steps to actually get to a better level of efficiency? That's hard to use a software tool for. You need kind of people on your team to think about the workflow design. You need to redesign the actual process flow. That's been a bit of the challenge of the last two years is a lot of people have just focused on all different types of copilot pilots across all different industries and I think that's helpful. But I think the next phase of this is actually process redesign and moving to ways where you can actually totally restructure the way line works.
[15:02]
Interviewer
As an example, an AI native solution. The framework is not just that AI is replacing what a human is doing, but how would you design the model with AI in mind?
[15:11]
Matt
Most of the material benefit you're going to see is when you clean sheet any process to be like, how would I design this process knowing all the AI tools I have from scratch? And how do I use both technology and humans? And by the way, I think the example for that is going to involve both for a long, long time and humans are a core part of this solution. I think in Invisible we believe that it's the human machine interface where all the value sits. But it's not necessarily just giving all your people on an existing process and a tool. It's redesigning the process to use all the tools at your disposal.
[15:40]
Sponsor
Thank you for listening. To join our community and to make sure you do not miss any future episodes, please click the follow button above to subscribe.
[15:46]
Interviewer
So let's talk about Invisible. Give me some specifics on how the company is doing today.
[15:51]
Matt
We ended 2024 at $134 million of revenue. Profitable. We were the third fastest growing AI business in America over the last three years.
[16:00]
Interviewer
You just joined as CEO. What is your strategy for the next five to 10 years? And how do you even conceptualize a strategy given how fast the industry is changing?
[16:10]
Matt
We've had explosive growth in the current kind of core of the business, which is AI training, and we plan to continue to focus on that. Our goal is to work with all the model builders to get these models as accurate as possible and support them any way we can with lots of human feedback. So if you think about what Invisible there we have this kind of AI process platform where we trot out any individual task into a set of stages and then insert kind of feedback analytics at all of those different steps. We then have the AI and training and evals motion I described which is a set of modules. On the back of that we have a labor marketplace where we can source all of those 5,000 different expert agents on any given topic. The core of that will remain our focus. The shift I envision are kind of twofold. One, deepening our focus on using that for fine tuning the enterprise. This is something I think all the model builders are hopeful for as well is the more that we can help all of the enterprise clients figure out how to get the most of their model builds. They're focused on how to get those working that's better for everyone. Everyone is hoping to see many more examples and I, by the way, I'm very, very optimistic that over the next five, six years we're going to see many, many more examples of great gen AI use cases in production. It's just been, I think a period of learning. The last few years have been kind of a proof of concept phase for the enterprise. Really helping the enterprise get many of those into production is a core focus for us. The other big area that I'm going to evolve Invisible into is the analogy I would use is we're going to build a modern servicenow anchored in Genai. So Invisible's process platform will include much more data infrastructure. It'll include an application development environment and process builder tools and it'll include our kind of our really, really good services delivery team around that. So one belief I have is that it's very hard to do any of this with the push of a button. I think the age of software has kind of relied on the idea that you build something and people take that as is. And I think AI is much more around configuring and customizing different workflows exactly for what any given customer wants. You can envision what Invisible evolve into as kind of our AI process platform with lots of process builder tools where people can build very sector specific applications like claims and claims for insurance or onboarding for food and beverage or fund admin for private equity. So you'll have a bunch of different verticalized use cases we'll go after and a lot of really interesting core data infrastructure tools like data ontology Master data management, things like that to help people get their data working.
[18:15]
Interviewer
How do you avoid being the victim of your own success? So you come in enterprises, you streamline their AI models using the services model. How do you avoid making yourself obsolescent?
[18:26]
Matt
The funny piece of context outside there is 70% of the software in America is over 20 years old. The rate of modernization of that has been glacially slow. I know there's been a lot of kind of hype that says suddenly the whole world's going to be hyper modern. Everything's going to work in two years. I think this is a long journey over the next two decades where we get to a world where every enterprise runs off of modern infrastructure, modern tech stacks and functions much like the digital company, digitally native companies and built up over the last five years that will take time to get to. But I'm very excited about what our platform can do to enable.
[18:54]
Interviewer
That said another way, your total addressable market is every enterprise for minus the two that have built models. Even in those two companies. I'm sure they're looking to streamline other parts of the business.
[19:05]
Matt
I think that's right. Interesting thing if you look at what I would call the application modernization market. So all of the modernization of legacy systems that happens annually, no player right now is more than 3% of that. So it's actually a very fragmented market that is painfully slow in how it moves. And it's the main frustration points for most enterprises. Like if you ask the average CEO in any company that's over 10 years old, how happy are they with their core data, the kind of tools they use on a daily basis? Most are pretty frustrated. So I don't think this is something where the existing is really good and everyone's really happy. There's a lot of frustration that we are hoping to help fix. And I think Genai will be the root of doing a lot of that. I think there's a lot of tooling you can do to generate insights faster, to pull up reporting faster. And so we will be a gen AI native kind of application development platform.
[19:52]
Interviewer
You have a very unique vantage point in that you're the CEO of one of the fastest growing AI companies. You ran McKinsey's lab. Walk me through the AI ecosystem today in terms of how you look at the ecosystem.
[20:03]
Matt
I've talked to a bunch of VCs about this in the past couple days. The infrastructure layer, which is where most of the capital is going today and that's a mix of kind of things like data centers as well as the model builders. And you asked about the gap to kind of investment today to enterprise modernization. The challenge is that above the infrastructure layer you have what I call the application layer, which is individual tools for individual use cases. Right. And that could be, I mentioned claims it could be legal services. It's all the verticalized applications that exist anchored in Genai to solve problems. All of those applications today for the Most part are SaaS or traditional software based. So they are designed like all software of the last 20 years to be a kind of a push button deployment of a specific use case that functions like traditional SaaS software. I am skeptical that that is actually going to be the way that impact is realized with Genai for a couple different reasons. Software as a paradigm has existed that way because the idea was it took so long to get data schemas organized and structured and it took so long to build any custom tool that you had to invest all the money up front in building a perfect piece of software. Once you got data locked in on that software, it was very hard for anyone to ever migrate off of that. The term of this is your systems of record. Once you're locked in on any sort of a system of record, whether that be an ERP system, whether it be an HR database, you basically never leave as an enterprise because the data is really painful to move. And so that's been the conventional wisdom on how to build software for a long time. You've had some really public examples. Satya Nadella mentioned it. What Genai may enable is a movement where the value moves from the system of record layer to the agentic layer. So you actually move to a world where people don't stay on softw, that's sticky just because of the data. They actually want the best possible software for their specific institution. So you might have a world where people are building tooling that is much more custom to their enterprise. You might have a world where I have a react workflow that uses analytics that are customized to my enterprise in a cloud environment and I can stand that up in a couple months. And I think that paradigm is a very different way forward for technology now. I'm sure there are some that would dispute that. I'm sure there are some that will say software will exists as it always has. But I would say that the main feedback I heard from a lot of VCs is that most of the application layer today focuses on the standard software paradigm. And I think we're looking at something very different, which is we want to have kind of an application Development environment with a lot of configurability and customizability, the ability to build verticalized applications for specific sectors that will allow us to say not this is our tool, take it, but much more. What is the workflow you need bring that to life.
[22:31]
Interviewer
Let's say a company is looking a Fortune 500 companies looking to create a CRM. What would an AI native CRM look like for a Fortune 500 company versus just using a Salesforce?
[22:43]
Matt
What you would usually end up doing in that world is you'll look at Salesforce OR Dynamics or ServiceNow has one of these now and you will buy out of the box functionality like you'll buy, let's say their Contact center tooling but then you will end up customizing a fair amount of that to your enterprise. So you'll say my contact center is going to have this flow for services, this so for calls. And so even though you're buying the tool, you're going to spend a year customizing configuring for your workflow. CRM is a little bit different in that you do have several large players, you know, Salesforce, Dynamics and ServiceNow now that have built fairly good builder applications for that use case.
[23:19]
Interviewer
If you're successful as CEO of Invisible, what will the company look like in 2030?
[23:23]
Matt
I use the analogy now because I have a ton of respect for what they've done. My main North Star metric is that every gen AI model we work on will reach production. And so I'm really excited about working with all the model builders over the next couple years to continue to fine tune and train their models and get that working at huge scale in the Enterprise. And I think that's something that we will be a huge driver of.
[23:41]
Interviewer
What would you like our audience to know about you, about Invisible technologies or anything else you'd like to share?
[23:46]
Matt
What I don't want any of what I've said to come across is pessimistic. There's nobody that believes that AI will be more positive for the enterprise over the next five to 10 years. I think the last two years did not live up to the hype cycle, partly because there was a belief that you could just buy a product out of the box, push a button and suddenly all your gen AI will work. My kind of advice or view on the path forward is I don't think that will be the paradigm. I think every enterprise will have to build some capabilities around. What do I want to get out of these models? How do I train and validate these models? How do I make sure my data is adequately reflected in these models and that's a very doable thing. When we sit here five, ten years from now, there'll be some really exciting deployments in this, like the ability to stand up new software, new digital workflow. Companies based on Genai is going to expand significantly. But I do think it's been a bit of reality check for the last two years that, you know, this is not like I just stand up a piece of software, push a button and everything works.
[24:34]
Interviewer
How should people follow you? And Invisible.
[24:37]
Matt
You can add me on LinkedIn. I'll be posting about some of the updates we'll be having there and we're building kind of, I call it Data Insights function Invisible as well. We're going to start to bring bring as much of the truth that we're seeing and what's exciting and what we recommend to our enterprise clients so we can help them navigate what is a very complex and difficult world.
[24:54]
Sponsor
Thank you for listening to my conversation with Matt. If you enjoyed this episode, please share with a friend. This helps us grow and also provides the best feedback when we review the episode's analytics. Thank you for your.

Summary7 min read

Podcast Summary: "The 92% AI Failure: Unmasking Enterprise's Trillion-Dollar Mistake" – E146

How I Invest with David Weisburd

Introduction

In episode 146 of "How I Invest with David Weisburd," host David Weisburd interviews Matt, the CEO of Invisible Technologies, to dissect the alarming statistic that 92% of AI initiatives in enterprises fail to meet expectations. Released on March 14, 2025, the episode delves into the complexities of AI model development, the hurdles enterprises encounter in deploying AI effectively, and the strategic solutions Invisible Technologies offers to bridge this significant gap.

AI Model Building Challenges

The conversation kicks off with a discussion on AI-native solutions and the fundamental flaws in how AI is typically integrated into business processes.

“Most of the material benefit you're going to see is when you clean sheet any process to be like, how would I design this process knowing all the AI tools I have from scratch?” [00:08]
— Matt, CEO of Invisible Technologies

Matt emphasizes that true value in AI implementation arises not from merely replacing human tasks with AI but from redesigning processes to leverage AI's strengths from the ground up. He argues that human-machine interfaces are pivotal, allowing for a symbiotic relationship where both technology and humans enhance each other’s capabilities.

Invisible’s Business and Growth

Matt provides an overview of Invisible Technologies, highlighting its impressive growth and market position.

“We ended 2024 at $134 million in revenue, profitable. We were the third fastest growing AI business in America over the last three years.” [00:42]
— Matt

Invisible Technologies has rapidly become a key player in the AI industry, focusing on AI training and model fine-tuning. Matt discusses how Invisible's approach involves using both technology and human expertise to ensure AI models are not only built efficiently but also tailored to meet specific enterprise needs with high accuracy.

Enterprise AI Adoption and Challenges

The dialogue shifts to the broader challenges faced by enterprises in adopting AI. Matt cites a startling statistic:

“The stat that I've seen most frequently cited is that about 8% of models today make it to production.” [03:21]
— Matt

Despite massive investments in AI, only a small fraction of models succeed in being deployed effectively within enterprises. Matt attributes this failure rate to several factors, including poor data quality, lack of clear definitions for success, and inadequate fine-tuning of AI models for specific business contexts.

Invisible’s Solutions for AI Integration

Matt elaborates on how Invisible Technologies addresses these challenges through its two main business components: reinforced learning and feedback systems, and enterprise AI fine-tuning.

“We have two big components of our business, what I call reinforced learning and feedback which is the process on any topic where model is being trained.” [02:22]
— Matt

Invisible leverages a pool of highly specialized experts to train AI models on niche topics, ensuring that models can handle complex and specialized tasks with high accuracy. This approach is complemented by the company’s focus on enterprise AI fine-tuning, which involves adjusting AI models to function seamlessly within specific business environments.

Process Redesign and AI Integration

A significant portion of the discussion centers on the necessity of re-engineering business processes to fully harness AI’s potential. Matt compares the current state of AI in enterprises to factory workers who incrementally improve productivity without altering the production line’s core structure.

“It's unclear the degree to which they actually save any work. They kind of tweak a lot of things on the margin.” [13:42]
— Matt

He argues that true efficiency gains from AI require comprehensive process redesigns rather than superficial adjustments. Invisible Technologies assists enterprises in rethinking their workflows, enabling them to integrate AI in ways that transform operations fundamentally rather than just enhancing existing practices.

Data Quality and Organization

Matt underscores the critical role of data quality in AI success. He points out that many enterprises struggle with legacy data systems that are poorly organized and hinder effective AI utilization.

“When good AI meets bad data, the data usually wins.” [05:07]
— Matt

Invisible Technologies addresses this issue by helping companies structure and master their data domains, ensuring that AI models are trained on clean, well-organized data. This foundational step is crucial for achieving the high accuracy and reliability required for enterprise-level AI applications.

Evaluating AI Model Performance at Scale

The conversation delves into the complexities of evaluating AI model performance, especially when models generate large volumes of outputs, such as investment memos.

“We spent the last eight years... building what's called semi private custom evals where we effectively set parameters and use human feedback to score those parameters.” [07:25]
— Matt

Invisible Technologies has developed sophisticated evaluation frameworks that use subject matter experts to consistently assess AI outputs against predefined criteria. This ensures that only high-quality outputs are deployed, maintaining the integrity and reliability of AI applications within enterprises.

Future of AI in Enterprises

Looking ahead, Matt is optimistic about the evolving landscape of AI in enterprises. He envisions a future where AI models are not only successfully deployed but also continuously optimized to meet dynamic business needs.

“I'm very optimistic that over the next five, six years we're going to see many, many more examples of great gen AI use cases in production.” [16:09]
— Matt

Matt outlines Invisible Technologies’ strategic direction, which includes expanding their process platform to incorporate modern data infrastructure, application development environments, and sector-specific tools. This evolution aims to facilitate the creation of highly customized AI applications tailored to various industry requirements.

Invisible’s Approach to Customization and Scalability

Invisible Technologies focuses on providing scalable solutions that can be tailored to specific enterprise needs. Matt discusses how their platform allows for the customization of AI workflows, enabling businesses to design and deploy AI solutions that align perfectly with their operational processes.

“The motion of how do I deploy a machine learning model with accuracy. You've seen a bunch of really good examples of that.” [04:52]
— Matt

This approach ensures that AI implementations are not only accurate but also seamlessly integrated into the existing business frameworks, enhancing overall efficiency and effectiveness.

Addressing Common Enterprise AI Use Cases

Matt shares insights into successful enterprise AI use cases, such as Moody's chain-of-thought reasoning model and Klarna’s AI-driven contact center, highlighting the importance of achieving near-perfect accuracy in these applications.

“If you think about what Invisible there we have this kind of AI process platform where we trot out any individual task into a set of stages and then insert kind of feedback analytics at all of those different steps.” [16:09]
— Matt

These examples illustrate how tailored AI solutions can transform specific business functions, setting the stage for broader adoption and more innovative applications across different industries.

Conclusion and Future Vision

In concluding the episode, Matt reiterates his belief in AI’s transformative potential for enterprises. He emphasizes that while the path has been challenging, strategies focused on process redesign, data quality, and model fine-tuning are key to unlocking AI’s full value.

“I don't think that every enterprise will have to build some capabilities around... What do I want to get out of these models? How do I train and validate these models?” [23:45]
— Matt

Matt envisions Invisible Technologies as a pivotal force in this transformation, providing the necessary infrastructure, expertise, and support to ensure that AI models not only reach production but also deliver substantial business value.

Notable Quotes

“Most of the material benefit you're going to see is when you clean sheet any process to be like, how would I design this process knowing all the AI tools I have from scratch?” — Matt [00:08]
“We ended 2024 at $134 million in revenue, profitable. We were the third fastest growing AI business in America over the last three years.” — Matt [00:42]
“The stat that I've seen most frequently cited is that about 8% of models today make it to production.” — Matt [03:21]
“When good AI meets bad data, the data usually wins.” — Matt [05:07]
“I'm very optimistic that over the next five, six years we're going to see many, many more examples of great gen AI use cases in production.” — Matt [16:09]
“We spent the last eight years... building what's called semi private custom evals where we effectively set parameters and use human feedback to score those parameters.” — Matt [07:25]
“I don't think that every enterprise will have to build some capabilities around... What do I want to get out of these models? How do I train and validate these models?” — Matt [23:45]

Final Thoughts

Episode E146 of "How I Invest with David Weisburd" offers a comprehensive examination of why a vast majority of AI initiatives in enterprises fall short and how Invisible Technologies is positioned to change that narrative. Through Matt’s insights, listeners gain a nuanced understanding of the strategic adjustments necessary for successful AI deployment, emphasizing the importance of process redesign, data integrity, and customized model training. This episode serves as an invaluable resource for investors and enterprise leaders aiming to navigate the complex AI landscape and harness its full potential for transformative business growth.

Loading summary

Transcript54 lines

[00:00]
Interviewer
An AI native solution. The framework is not just that AI is replacing what a human is doing, but how would you design the model with AI in mind?
[00:09]
Matt
I think most of the material benefit you're going to see is when you clean sheet any process to be like, how would I design this process knowing all the AI tools I have from scratch? And how do I use both technology and humans? And by the way, I think the example for that is going to involve both for a long, long time. In fact, I think humans are a core part of this solution. I think in Invisible we believe that it's the human machine interface where all the value sets. But it's not necessarily just giving all people on an existing process and a tool. It's redesigning the process to use all the tools you dispose of.
[00:38]
Interviewer
So let's talk about Invisible. Give me some specifics on how the company is doing today.
[00:42]
Matt
I joined in mid January. We ended 2024 at 134 million in revenue, profitable. We were the third fastest growing AI business in America over the last three years.
[00:54]
Interviewer
So how will deep seek affect Invisible?
[00:57]
Matt
The viral story was that it was $5 million to build the models they did. The latest estimates that have come out since in the FT and elsewhere would say it's close to 1.6 billion. I think the number that's been cited from a compute standpoint is like 50,000 GPUs. So if you had just told that narrative as the exact same story, but with 1.6 billion of compute, I don't even think it would have been a media story. The fact that it cost over a billion dollars to build that model means it is a continuation of the current paradigm. Look, there are some interesting innovations. They've had mixture of experts, they did some interesting stuff around data storage that does have some benefits. I'm using compute costs, but I think those are things we've seen other model builders experiment with already. If I think about types of data, they basically went around things that are base truth logic, like math, where there's a fair amount of synthetic data available. That's a fairly small percentage of the overall training tasks that I'd say most model builders are focused on.
[01:46]
Interviewer
Tell me more about that.
[01:47]
Matt
Think about training as kind of three main vectors. So you have base truth information where a lot of synthetic or kind of Internet broad based data exists. So math is a really good example of that. Then you have tasks like creative writing where there is no real kind of AI feedback, there's no synthetic data that's existing, there's no way to train those models without human feedback. But the most interesting one is you have a whole set of base truth information where you also don't have enough synthetic data. So an example of that I would give would be Computational biology in Hindi. The corpus of that is just not broad enough. Each branch of that tree and each topic you train off of will have a different approach.
[02:19]
Interviewer
And tell me about what invisible technologies does exactly.
[02:23]
Matt
We have two big components of our business, what I call reinforced learning and feedback which is the process on any topic where model is being trained. We can spin up a mix of expert agents on that particular topic. So that could be everything from. I mean I use the example of computational biology in Hindi. Our pool has a 1% acceptance rate and about 30% of the pool is PhDs and Masters. So these are very high end specific experts. The funniest one I talked about recently, somebody is like falconry in the 1800s things where there's just not a lot of good existing data. And look, I think models are going to be built on the full corpus of information that is mattered to humanity. So there's a lot of branches of that tree and we bring all of the different experts to help train those models. But that's only half the business where we're seeing increased focus and demand is on the enterprise side. The big challenge today and the kind of chasm that exists between let's call it Silicon Valley and the enterprise is there's a demand for broad based model development which is really important. But I think what a lot of the enterprise is looking for is how do I get those models to then work at 99% accurate accuracy in my specific context.
[03:21]
Interviewer
Tell me about some examples of enterprise models that have worked.
[03:25]
Matt
Therein lies a great question. The stat that I've seen most frequently cited is that about 8% of models today make it to production. The two largest high profile public enterprise cases I've seen are Moody's had a chain of thought reasoning example and then probably the most often cited one is Klarna had a contact center where they basically built up an entirely Genai center contact center to replace the old contact center they had. The realized impact in the enterprise has not materialized the way people expected it would. I am very bullish on where it will go but to date those are the only two examples I can cite. I can cite some pretty public struggles but there have not been many other realized examples that I've seen.
[03:59]
Interviewer
So there's hundreds of billions of dollars being put into this problem set. Only two successful examples where are the main frictions and how do you see that evolving over the next five, 10 years?
[04:08]
Matt
Most of that money to date has gone into building the models that are extensible, generalizable and moving towards greater levels of intelligent change of thought reasoning. We've seen unbelievable progress phase of the model building process. The challenge is let's take, let's say you're an insurer and you need to build a claims model. What you need to know is that your model works with perfect accuracy. You're not or 99% accuracy. The investments have been have led to material improvements. It's that the motion of then taking those models and fine tuning them in an enterprise context has not been standardized yet. The motion of how do I deploy a machine learning model with accuracy. You've seen a bunch of really good examples of that. Like straight through processing of mortgage loans is one example where those are being productionalized. They're working. There's a ton of examples of impact coming from machine learning deployments. AI has not really figured out it's what I call production paradigm yet.
[04:53]
Interviewer
The OpenAI's, the Anthropics, the XAI's of the world developing these incredible generalized models. And then you only have really two use cases for enterprises. You mentioned fine tuning. What are the other steps that a company needs to go through in order to make their AI work?
[05:07]
Matt
Let's take an asset manager that is going to build a system to do ongoing reviews of its assets based on its internal investments. Right. The first step you need is you need all your internal data organized and structured and in a place where you can use it and access it. That's probably the biggest challenge most people face is there's a joke. I like to say that when, when good AI meets bad data, the data usually wins. And I think the challenge is if your internal data environments, if you don't have a clear definition of your, your assets, your products, if you don't have kind of what I'd call organized core data domains, it's very hard to even use AI until you've got that organized. That's probably the biggest challenge I think the enterprise faces right now is most of the data is on systems that are, that are a decade or later old. It's not organized or mastered across those systems in a way that they can use it in gen. So that's one big problem. I think the bigger than issue is so let's say you get that all organized. David, I'll put you on the spot to give you an example. So let's say that you built a. Let's say you built a model, a gen AI model to produce summaries of investments in the financial services space and just kind of look at new investment ideas. How would you build a chatbot to do that? You spend money to do it. At the end of that you have a model that will start generating these kind of investment memos. How would you define a good memo or a bad memo at scale? So let's say it generates 10,000 memos. How do you know it works?
[06:16]
Sponsor
If you run a fund, then you know having instant access to fund insights. Modeling and forecasting are critical for producing returns. Which is why I recommend Tactic by Carta. With Tactic by Carta, fund managers can now run complex fund construction, round modeling and scenario analysis. Meaning for the first time ever you could use a single centralized platform to do everything from analyzing performance to modeling returns with real industry benchmarks. Most importantly, communicate it all with Precision to your LPs. Discover the new standard in private fund management with access to key insights and data driven analysis that could change the trajectory of your fund. Learn more@carta.com howiinvest that's C-A-R-T-A.com H O W I I N V E S-T.
[07:04]
Interviewer
It'S difficult to do at scale at 10,000, but I think on an individual basis a good model is a model that hits all the points and then has more clarity and more details on the sub points. So I would evaluate it based on did it get all the main key points of the investment thesis at a high level and then were the sub level points sufficient or covered the main topics?
[07:26]
Matt
What you're saying makes complete sense, which is you have a set of parameters or set of outcomes. You're looking for a memo. Even if you have that though, the question then becomes how do you evaluate that consistently across 10,000 memos? And I think this is the difference between backtesting of ML dataset vs gen is you need a way to actually go back and validate that what is produced works. And I think that has been the real challenge that the enterprise has struggled with is you may have a sense for what good looks like. You might say, for example, the definition of a good investment mo would be like at least a paragraph summary of competitive set some context on the market, including growth rates. Like you could set a set of parameters that you're looking for to answer, but then you have to wade through and kind of assess all that. And so what we spent the last eight years doing for the model builders and others is building what's called semi private custom evals where we effectively set parameters like describing that would say these are the definitions good, this is the outcomes we're looking for. And then we use human feedback to score those parameters. So we could go at big scale and say does this outcome cover what you're looking for? And we bring subject matter experts to Bayer to actually do that scoring. I think that's actually been the big gap is these are often things you can't score with a random person in the street. You can't just put it into market and hope it works. You need a subject matter expert say this looks generally good before any organization gets comfortable launching it. One way I've seen enterprises do that is I've seen a couple customers experiments already is they'll actually have their own employees evaluating this at huge scale. But if you think about the time suck of like having large numbers of people just reviewing gen outputs, that's very hard to do. So I think a lot of what we've now evolved to is on the enterprise side, a mix of these kind of evals and assessments of the models that are happening that we then help customers fine tune and improve their models.
[09:00]
Interviewer
Is there a gap between what a generalist searcher might want and somebody domain specific? In other words, if I'm making $100 million investment decision based on a MEM, that has to be much better than if I want to find out if, you know, dogs could eat a certain type of food. And what's the best practice for raising a healthy dog?
[09:17]
Sponsor
Post earnings reports are more than just a data dump. They're a goldmine of opportunities waiting to be unlocked. With Daloopa, you could turn those opportunities into actionable insights. Daloopa's dynamic scenario building tools integrate updated earnings data, letting you model multiple strategic outcomes such as what happens if a company revises guidance. With automated sensitivity analysis, you could quickly understand the impact of key variables like cost pressures, currency fluctuations or interest rate changes. This means you'll deliver more actionable insights for your clients, helping them navigate risks and seizing opportunities faster. Ready to enrich your post earnings narratives? Visit daloopa.com how that's-a L-O-O-P a.com how today to get started, One of the.
[10:04]
Matt
Questions you're asking here is what is the bar or the risk bar for production? Depending on the use case. And I do think it's different. As an example, if the goal of a chatbot is just to say something like Review restaurants and it's a consumer facing the risk bar on that is, does it say anything toxic? Is there any bias? You can put some risk parameters around it, but you don't really need kind of subject matter expert feedback on it. I'll give an interesting one. Legal, like law, the bar for accuracy on that is materially different than a consumer example. And many law firms are experimenting with this. But it's hard to assess something like a debt covenant agreement without subject matter experts weighing in. On its scale, the outputs are consistently good.
[10:42]
Interviewer
How do AI models evolve when parameters change? Is this something that'll always be needed to be refreshed?
[10:48]
Matt
There are two ways that models generally be consumed. Many models that will be consumed by consumers that are effectively just going to be what the model builders produced. Usually the way the enterprise is using these models is they are tailoring those models to their corpus of information. I'll give an example. Let's say that you have two wealth managers. Let's say, hey, I'm going to make up to Fidelity and T. Rowe Price. And they want to use, you know, experiment with things like robo advisory or question answering. They're not going to just use an off the shelf framework for that. They're going to tailor it off of all of the information that exists in their communications and their training documentation. Any model that's being trained at the enterprise is usually being trained off of the internal knowledge management corpus that that institution has. And so you're using the large language model from the model builder and you're tailoring it to your specific context. That process is called fine tuning.
[11:31]
Interviewer
Prior to becoming CEO of Invisible, you headed McKinsey's Quantum Black Labs, which is their AI labs. What did you do at McKinsey?
[11:39]
Matt
I focused on three main things. One, kind of all of our large scale data transformation data lakehouse data warehouse builds. So the first thing I mentioned, which is if your data is messy, it's very hard to use AI. I spent a lot of time focusing on that. I spent a lot of time doing custom application development. So building all sorts of different applications, whether that be for retention, pricing, contact centers kind of software, custom software that people could use to deploy models. And I do think that's an understated part of a lot of this is there is what a model does, but then there is the way that somebody can understand it and interpret it. And a lot of that is the user interface by which they consume it. And so I think that's something the enterprise is spending a lot of time on is what is the user interface by which people consume and think about and make decisions around these models? And then the third area I oversaw, the Gen Lab, which is McKinsey's kind of global genai tool build. We were doing it when I was there. We're doing anything from 220, 240gen AI builds at the top.
[12:27]
Interviewer
I want to double click on the enterprise side of what you did at McKinsey. You mentioned those two high profile use cases for successful enterprises built. Did you build any successful enterprise use cases while you were at McKinsey?
[12:38]
Matt
We definitely did. There's a public case you can reference that a couple folks in Quantum Black build for ING where it's effectively a chatbot. And one of the things they mention in it, very similar. What I'm saying now is a lot of what was required to put that in production was getting it to 99% accuracy. So they had a lot of parameters and fine tuning they did around testing, quality, controlling it, building audit LLMs to make sure that the outcome is good. We definitely did a lot of that. But the rough math you see across the industry is about 8% of gen AI models make it to production and that's broad based. The amount that kind of stall around the proof of concept pilot phase is pretty material. I think it will get better over time. But to date the challenge has been a lot of things I mentioned challenging data, an unclear definition of good and an unwillingness from the folks in the field to actually use and believe the outcomes of the models. And I think that that's going to take time.
[13:24]
Interviewer
There's this concept in the productivity space which is 80% done is 100% good. Is there like an 8020 rule here where you could use AI to solve many things and dramatically decrease your need for sales representatives, for customer support. And does it have to be 100% good?
[13:43]
Matt
That is a really complicated question. So there's another analogy which I'll use which we manufacturing lines in a factory. And so if I ask you the question, If I have 10 people on a manufacturing line and every one of them saves 5% of their time, what is the line savings?
[13:58]
Interviewer
I'm guessing half a person.
[14:00]
Matt
Zero. Because you can't take out a line and you can't take. No, no person can be taken off that line. You effectively just move to a world where everyone has a little bit more free time. And I think that's the challenge. The 8020 here is things like copilots have had a very interesting kind of last two years in that they they are helpful coding, co pilots, legal copilots, all these things. But it's unclear the degree to which they actually save any work. They kind of tweak a lot of things on the margin. And I think the difference I'd say with 8020 is I think to do that well, you actually have to re engineer processes. You have to say what does my end to end workflow look like for claims processing or whatever that might be? And how do I take out two full steps to actually get to a better level of efficiency? That's hard to use a software tool for. You need kind of people on your team to think about the workflow design. You need to redesign the actual process flow. That's been a bit of the challenge of the last two years is a lot of people have just focused on all different types of copilot pilots across all different industries and I think that's helpful. But I think the next phase of this is actually process redesign and moving to ways where you can actually totally restructure the way line works.
[15:02]
Interviewer
As an example, an AI native solution. The framework is not just that AI is replacing what a human is doing, but how would you design the model with AI in mind?
[15:11]
Matt
Most of the material benefit you're going to see is when you clean sheet any process to be like, how would I design this process knowing all the AI tools I have from scratch? And how do I use both technology and humans? And by the way, I think the example for that is going to involve both for a long, long time and humans are a core part of this solution. I think in Invisible we believe that it's the human machine interface where all the value sits. But it's not necessarily just giving all your people on an existing process and a tool. It's redesigning the process to use all the tools at your disposal.
[15:40]
Sponsor
Thank you for listening. To join our community and to make sure you do not miss any future episodes, please click the follow button above to subscribe.
[15:46]
Interviewer
So let's talk about Invisible. Give me some specifics on how the company is doing today.
[15:51]
Matt
We ended 2024 at $134 million of revenue. Profitable. We were the third fastest growing AI business in America over the last three years.
[16:00]
Interviewer
You just joined as CEO. What is your strategy for the next five to 10 years? And how do you even conceptualize a strategy given how fast the industry is changing?
[16:10]
Matt
We've had explosive growth in the current kind of core of the business, which is AI training, and we plan to continue to focus on that. Our goal is to work with all the model builders to get these models as accurate as possible and support them any way we can with lots of human feedback. So if you think about what Invisible there we have this kind of AI process platform where we trot out any individual task into a set of stages and then insert kind of feedback analytics at all of those different steps. We then have the AI and training and evals motion I described which is a set of modules. On the back of that we have a labor marketplace where we can source all of those 5,000 different expert agents on any given topic. The core of that will remain our focus. The shift I envision are kind of twofold. One, deepening our focus on using that for fine tuning the enterprise. This is something I think all the model builders are hopeful for as well is the more that we can help all of the enterprise clients figure out how to get the most of their model builds. They're focused on how to get those working that's better for everyone. Everyone is hoping to see many more examples and I, by the way, I'm very, very optimistic that over the next five, six years we're going to see many, many more examples of great gen AI use cases in production. It's just been, I think a period of learning. The last few years have been kind of a proof of concept phase for the enterprise. Really helping the enterprise get many of those into production is a core focus for us. The other big area that I'm going to evolve Invisible into is the analogy I would use is we're going to build a modern servicenow anchored in Genai. So Invisible's process platform will include much more data infrastructure. It'll include an application development environment and process builder tools and it'll include our kind of our really, really good services delivery team around that. So one belief I have is that it's very hard to do any of this with the push of a button. I think the age of software has kind of relied on the idea that you build something and people take that as is. And I think AI is much more around configuring and customizing different workflows exactly for what any given customer wants. You can envision what Invisible evolve into as kind of our AI process platform with lots of process builder tools where people can build very sector specific applications like claims and claims for insurance or onboarding for food and beverage or fund admin for private equity. So you'll have a bunch of different verticalized use cases we'll go after and a lot of really interesting core data infrastructure tools like data ontology Master data management, things like that to help people get their data working.
[18:15]
Interviewer
How do you avoid being the victim of your own success? So you come in enterprises, you streamline their AI models using the services model. How do you avoid making yourself obsolescent?
[18:26]
Matt
The funny piece of context outside there is 70% of the software in America is over 20 years old. The rate of modernization of that has been glacially slow. I know there's been a lot of kind of hype that says suddenly the whole world's going to be hyper modern. Everything's going to work in two years. I think this is a long journey over the next two decades where we get to a world where every enterprise runs off of modern infrastructure, modern tech stacks and functions much like the digital company, digitally native companies and built up over the last five years that will take time to get to. But I'm very excited about what our platform can do to enable.
[18:54]
Interviewer
That said another way, your total addressable market is every enterprise for minus the two that have built models. Even in those two companies. I'm sure they're looking to streamline other parts of the business.
[19:05]
Matt
I think that's right. Interesting thing if you look at what I would call the application modernization market. So all of the modernization of legacy systems that happens annually, no player right now is more than 3% of that. So it's actually a very fragmented market that is painfully slow in how it moves. And it's the main frustration points for most enterprises. Like if you ask the average CEO in any company that's over 10 years old, how happy are they with their core data, the kind of tools they use on a daily basis? Most are pretty frustrated. So I don't think this is something where the existing is really good and everyone's really happy. There's a lot of frustration that we are hoping to help fix. And I think Genai will be the root of doing a lot of that. I think there's a lot of tooling you can do to generate insights faster, to pull up reporting faster. And so we will be a gen AI native kind of application development platform.
[19:52]
Interviewer
You have a very unique vantage point in that you're the CEO of one of the fastest growing AI companies. You ran McKinsey's lab. Walk me through the AI ecosystem today in terms of how you look at the ecosystem.
[20:03]
Matt
I've talked to a bunch of VCs about this in the past couple days. The infrastructure layer, which is where most of the capital is going today and that's a mix of kind of things like data centers as well as the model builders. And you asked about the gap to kind of investment today to enterprise modernization. The challenge is that above the infrastructure layer you have what I call the application layer, which is individual tools for individual use cases. Right. And that could be, I mentioned claims it could be legal services. It's all the verticalized applications that exist anchored in Genai to solve problems. All of those applications today for the Most part are SaaS or traditional software based. So they are designed like all software of the last 20 years to be a kind of a push button deployment of a specific use case that functions like traditional SaaS software. I am skeptical that that is actually going to be the way that impact is realized with Genai for a couple different reasons. Software as a paradigm has existed that way because the idea was it took so long to get data schemas organized and structured and it took so long to build any custom tool that you had to invest all the money up front in building a perfect piece of software. Once you got data locked in on that software, it was very hard for anyone to ever migrate off of that. The term of this is your systems of record. Once you're locked in on any sort of a system of record, whether that be an ERP system, whether it be an HR database, you basically never leave as an enterprise because the data is really painful to move. And so that's been the conventional wisdom on how to build software for a long time. You've had some really public examples. Satya Nadella mentioned it. What Genai may enable is a movement where the value moves from the system of record layer to the agentic layer. So you actually move to a world where people don't stay on softw, that's sticky just because of the data. They actually want the best possible software for their specific institution. So you might have a world where people are building tooling that is much more custom to their enterprise. You might have a world where I have a react workflow that uses analytics that are customized to my enterprise in a cloud environment and I can stand that up in a couple months. And I think that paradigm is a very different way forward for technology now. I'm sure there are some that would dispute that. I'm sure there are some that will say software will exists as it always has. But I would say that the main feedback I heard from a lot of VCs is that most of the application layer today focuses on the standard software paradigm. And I think we're looking at something very different, which is we want to have kind of an application Development environment with a lot of configurability and customizability, the ability to build verticalized applications for specific sectors that will allow us to say not this is our tool, take it, but much more. What is the workflow you need bring that to life.
[22:31]
Interviewer
Let's say a company is looking a Fortune 500 companies looking to create a CRM. What would an AI native CRM look like for a Fortune 500 company versus just using a Salesforce?
[22:43]
Matt
What you would usually end up doing in that world is you'll look at Salesforce OR Dynamics or ServiceNow has one of these now and you will buy out of the box functionality like you'll buy, let's say their Contact center tooling but then you will end up customizing a fair amount of that to your enterprise. So you'll say my contact center is going to have this flow for services, this so for calls. And so even though you're buying the tool, you're going to spend a year customizing configuring for your workflow. CRM is a little bit different in that you do have several large players, you know, Salesforce, Dynamics and ServiceNow now that have built fairly good builder applications for that use case.
[23:19]
Interviewer
If you're successful as CEO of Invisible, what will the company look like in 2030?
[23:23]
Matt
I use the analogy now because I have a ton of respect for what they've done. My main North Star metric is that every gen AI model we work on will reach production. And so I'm really excited about working with all the model builders over the next couple years to continue to fine tune and train their models and get that working at huge scale in the Enterprise. And I think that's something that we will be a huge driver of.
[23:41]
Interviewer
What would you like our audience to know about you, about Invisible technologies or anything else you'd like to share?
[23:46]
Matt
What I don't want any of what I've said to come across is pessimistic. There's nobody that believes that AI will be more positive for the enterprise over the next five to 10 years. I think the last two years did not live up to the hype cycle, partly because there was a belief that you could just buy a product out of the box, push a button and suddenly all your gen AI will work. My kind of advice or view on the path forward is I don't think that will be the paradigm. I think every enterprise will have to build some capabilities around. What do I want to get out of these models? How do I train and validate these models? How do I make sure my data is adequately reflected in these models and that's a very doable thing. When we sit here five, ten years from now, there'll be some really exciting deployments in this, like the ability to stand up new software, new digital workflow. Companies based on Genai is going to expand significantly. But I do think it's been a bit of reality check for the last two years that, you know, this is not like I just stand up a piece of software, push a button and everything works.
[24:34]
Interviewer
How should people follow you? And Invisible.
[24:37]
Matt
You can add me on LinkedIn. I'll be posting about some of the updates we'll be having there and we're building kind of, I call it Data Insights function Invisible as well. We're going to start to bring bring as much of the truth that we're seeing and what's exciting and what we recommend to our enterprise clients so we can help them navigate what is a very complex and difficult world.
[24:54]
Sponsor
Thank you for listening to my conversation with Matt. If you enjoyed this episode, please share with a friend. This helps us grow and also provides the best feedback when we review the episode's analytics. Thank you for your.