Loading summary
Transcript29 lines
- [00:00]
A
Foreign. Welcome to the first episode of the AI to ROI podcast, the Big Story. And I'm Ray Reich, founder and CEO of BenchmarkIT.
- [00:18]
B
And I'm Peter Buchanan. I'm the managing partner of New Plan.
- [00:23]
A
And my co author on the AI to ROI newsletter. And this week's Big Story episode is where we dive into the story of the week from the newsletter. But before we go into that, I just want everyone to know that we also have a second episode of the AI to ROI podcast every week where I will be interviewing a guest who has recently deployed an AI project in their company and they will be discussing how they justified the AI investment and then share the return on investment the project has delivered, or at least the leading indicators are using to be able to ultimately measure and report the ROI. I'll be hosting founders and CEOs not only from enterprise companies who have deployed AI solutions, but also the founders and CEOs from AI native software companies to discuss their vision for their company and real life customer stories centered on on the business impact and return on investment of deploying their solution in real life customers. But let's go back to this episode. Peter, can you introduce this week's Big Story that we'll be discussing?
- [01:39]
B
Sure. We're discussing a relatively new technology that we think is incredibly important to the next generation of AI applications. They're called context graphs, and basically what they do is they fill in a gap in the AI stack, in AI applications, infrastructure, applications, agents, all those things, and they provide basically connectivity so that when your applications are actually running, you not only see what happens, you see who decided to make those things happen, why they decided that under what constraints they made those decisions and what precedents they used to make those decisions. So context graphs take what's happening and it creates an environment around it so that AI applications and AI infrastructure just run a lot better.
- [02:40]
A
Okay, Peter, before we get into all those gory technical details. So this really came to light. I think it was sometime in the middle of December where two partners at Foundation Capital wrote an article on their blog that ticked over LinkedIn. I have a hard time having a discussion with AI founders without it coming up. And in fact, Foundation Capital said this is a trillion dollar opportunity. And even though I may question how big of an opportunity is, I cannot question how important context graphs are going to be to both validating and justifying the decisions that AI agents are going to make. And I just wanted to make sure I got it right.
- [03:25]
B
Here's what I think about it, Ray, is it's Not a trillion dollar opportunity in terms of, oh, they're going to spend a trillion dollars on this and it's going to dwarf spending on all other AI infrastructure and all other AI applications. And it's going to Dr. What this technology is a glue technology that takes the promise of things like agentic AI and makes it much more effective. So we have an issue of AI to ROI coming out this week on error rate in models and what that error rate actually means. Well, this is a technology that can take the error rate and AI applications down to the level where you can almost always trust these applications. So when you look at a trillion dollar opportunity, what it is is creating a trillion dollars of value for companies doing AI. So like when you read a McKinsey report, they never tell you the market size, they tell you the economic value that be driven in the world by something that's happening. And in essence, that's what foundation is saying, especially since there are no actual numbers in the report that get you $2 trillion. It's basically getting your attention, saying, pay attention. This solves a big problem and it creates a ton of opportunity for really enterprises of all sizes.
- [04:55]
A
So, Peter, we kind of had this Semantic web which was introduced in 2001, and then it kind of evolved into knowledge graphs, which Google, I think really were talking a lot about starting in the 2000 and tens. So are context graphs just a natural evolution of the Semantic Web in knowledge graphs?
- [05:18]
B
Yeah, it's like an injection of steroids, to be honest with you. So if you look at what a knowledge graph is, a knowledge graph will tell you customer X renewed a contract. The context graph will say, all right, well who approved that renewal? What policies went into making that decision of approving that renewal? Maybe you'll know that on the customer side, on your own side, in terms of doing that, what sort of rules and precedents did we follow and then what alternatives for renewing this? In terms of contract terms, governance responsibilities were evaluated when we did it, and then how confident are we that all of this went through and worked perfectly? And so basically what you get down to there is when you follow that chain from the initial sort of knowledge graph type area to the end, it provides the why things happen, which agentic systems generally don't know. They don't know why they're doing things, they're just told to do it. And so that's the context as you get all the way to the why. So it's a big difference from where we are today in most instances.
- [06:35]
A
Okay, so this is really about making AI trustworthy in production. It allows you almost to have an audit trail to go back and really understand, as you said, things like who made the decision, why the decision was made, what were the constraints, I.e. policy, et cetera, that the action was taken under, and was there a precedent that was used? But here's my question, and I think it's really, who should care about this? Is this a CEO, chief legal counsel issue? Is it the IT organization that's implemented who really cares about this context graph and an enterprise company that's deploying agentic AI?
- [07:19]
B
I think, I think there's a whole community that cares. You know, let's just talk about who goes into it. Well, you know, first of all, let's start at the top. The CEO has to justify these AI investments. They have to work well. They don't want their customers complaining about them. They don't want employees complaining about them. They don't want to have failures, they don't want to waste money. They have to talk every quarter to their boards about AI and they have to have, they have to have confidence that they're following rules and that they understand end to end. So it's auditing, it's governance. It's all the stuff you would have with the human process that you're actually executing. But you're doing it, you're doing it in AI. This, the CFO is sort of doing the same, doing the same thing, but they're, they're following the numbers and they're following the investment and they're following. Well, should we put more into something? If you're the chief risk officer or the chief AI officer, you want to know that you're not creating future litigation because your AI does a bad job. You want to have a big layer of confidence on top that chief AI officer is responsible for the success of projects, that the CTO wants the best possible, most supportable stack the business line. Managers, frankly, want what they do in their roles to be more successful. They want to have competitive advantage. They want to win more business. You know, they don't want to be lapped by competitors and they want the jobs of their workers to be more fulfilling. So this is a really, a glue technology. It's, it's something that's just missing most places.
- [09:02]
A
Now, Peter, I think it was an unfair question because of course, you know, the CEO is always going to care, but I also think it's use case specific. And now you and I, as we were talking about today's episode, you had like four different Use cases that you shared and say, okay, so here's examples of where this is really applicable. And I think maybe the first one is a little bit about data governance. But can you share kind of those four use cases that we discussed?
- [09:29]
B
Sure. So data governance, let's say you have a data engineer and he changes a database field. Happens all the time. There's new types of database that comes along. Well, the context graph is, is tracking everything that's sort of going on inside that AI stack from top to bottom. And so if something changes and it's going to break something that's downstream, say a dashboard, a report, a machine language model, a business process, any, any of those sorts of things, you know, you need to have an alarm that basically says hold on Sparky, you know, maybe we, maybe we need to give this better consideration. Maybe we need to have processes to actually do this. And so when you look at companies like Vanguard or Prudential that have these really complex end to end, highly regulated workflows, they want to prevent this sort of data breakage at all costs. That's a really good example.
- [10:37]
A
So I really, I really like the fact that those interdependencies, because agentic AI really provides the most benefit when it's an end to end process that could touch multiple systems. Right. Multiple processes. So this almost says, hey, if you're deploying a AI agent to do this, it knows every system it's going to impact and then it can predict potential issues. So it's very proactive in nature even before it goes into production.
- [11:06]
B
Right, that's, that's exactly right. And in production, you know, people change their releases all the time. Like software used to be. There's lots of applications have new releases every two or three weeks and so this is a great way to make sure they stay on the beam.
- [11:21]
A
Let me ask you about this. One of the things I see in a lot of companies is they're deploying AI and it's very departmental. They, they, the marketing team deploys it for some outbound marketing or optimized paid ads. And then the sales organization is leveraging it to automate the research and the outreach. And then customer success is using it to best manage the accounts. But I always think about that customer life cycle. There's so many different touch points, so many different contacts and today so many different systems of record possibly for each phase of that customer life cycle. Does context graphs, do they apply here also?
- [12:02]
B
Why? Yeah, so the average mid to large size company and their go to market stack from the beginning of Marketing to deep into success management has 17 different applications and so it's really hard to connect them together. So you've got your upfront demand generation applications, you've got your CRM, you close a customer, then you've got port tickets. But that's not, that's not all because some things feed go to market that aren't in the go to market system. Like how are the products actually being used, what's the billing history of the customer, what's the contract negotiation history of the customer? And so pulling that together is actually, it's difficult in the best of times for anyone. I mean few companies can do it from end to end and so. But you do have, with a context graph functionality, you do have the ability to create a truly connected feature and unify this data together and be able to dive into it. So you're not just understanding what the customer bought, but you're understanding their support experience, their product usage, their payment behavior, how they act in negotiations, all those sorts of things. And you know, it's a small number of companies that can do that and use that data for competitive advantage today. And context graph gives a lot more companies a better shot.
- [13:34]
A
You know, when you and I first met, I was leading the automotive industry at a pretty large Fortune 50 company. And one of the things they were trying to do was to get better visibility across their supply chain. Now that was initially to do some just in time kind of inventory management. But now as large scale OEMs are outsourcing component manufacturing to their first and second tier, it seems like knowledge grasp from a quality control and quality tracking could be a great use case in a manufacturing industry also. Is that right, Peter?
- [14:08]
B
It's totally a great use case and it's a great use case in several levels. So first of all you're going to want to trace the relationships between your suppliers. So the average manufacturing company has about 600 supplier relationships that directly affect their businesses. And then those companies have secondary supplier relationships that are in the thousands. And so tracking those, you know, understanding where everything's come from as things are getting into their plants and their expend in their expenses and they are producing their products, you know, they need to know, they need to know for security reasons, they need to know for product quality reasons, they need to know for a whole bunch of reasons. And then they need to be able to check things in specific production batches, they need to match those batches to customer complaints. And they also have to resolve quality issues much more quickly. I mean I have a client Right now that's done some work in basically in context graphs where they've been able to cut quality processes that used to take weeks down to hours with, with better results, which of course saves money and also makes their, there's a tier, it's a tier one auto supplier and it makes the, makes their OEM customers really happy when quality issues are resolved pretty much immediately. Because you're able to do it because you have a better access to data to get answers faster and do corrective actions.
- [15:43]
A
Peter, you know, we talked about the knowledge graph market, which, you know, kind of started back in the early 2010s and I think the latest data I've seen is only like a 1.1 to $1.5 billion industry. But people are saying the knowledge graph market is going to grow to almost $7 billion by 2030. And that's because knowledge graphs and context graphs are going to come together. So are there some well known vendors out there today who currently are leading that knowledge graph market who are going to quickly probably capitalize on this whole trend of context graphs?
- [16:22]
B
I think there are, there's different ways that companies are attacking it. So there's a company like Atlan, they market what's called a metadata lakehouse and they call it a context control plane for AI governance. And basically it's an open source product with support. They do impact analysis, policy data, et cetera. It's AI model governance. It's part of the story. And then if you go to a company like Neo4J, they have a product called Graph Rag. They also have a very deep POD partnership with Google Cloud. And so what they're doing is they're enabling developers to anchor their LLMs with knowledge graphs so that the models are more precise, they provide better context, they have better explainability and they work in highly regulated industries like finance, retail, healthcare. You know, you've got a company like Rider. So Ryder is marketing specific context graph so that companies can produce better content with better brand consistency and legal compliance.
- [17:37]
A
That's highly, highly valuable for financial service organizations who have all types of regulatory compliance about what they can say in their ads. I'd even thought about that, right?
- [17:46]
B
So you know, and so companies like Vanguard, again, Vanguard, you know, it's a great technology organization, Qualcomm, highly regulated, you know, they've got to get all their chips approved by the FCC to create a new radio for a, for a mobile device, for example. What I think is going to happen though, Ray, is I think all the entrants that are here, I Think they're lurkers are going to win this market. What I mean by a lurker is you have a company like Microsoft. Microsoft just announced a product called Agent365. And what Agent365 does is allows companies to observe their behaviors in action. So it's sort of the follow on to application performance management. In the, in the 2000s and 2010s my agents are doing this and oh my gosh, this one just went off the rails. And so you know, Salesforce has similar products. Sierra is an agent only company founded by a couple of ex Google and ex Salesforce people that's just knocking the COVID off the ball. And so these sort of big companies like those that are generating lots of revenue, well that's a logical product attention, right? You could imagine somebody that makes vector databases or other types of databases Moving or automated DevOps AIOps infrastructure moving up the stack to create a more integrated offering that's very packaged that would include this functionality. And frankly you could also see a situation where an anthropic or an OpenAI sees this as a problem. And they do like what anthropic, I guess it was anthropic did with their MCP server. They open source it to make content more accessible to models. But you could see a similar situation where one of the big model builders invents something that's super awesome. Open sources IT people can morph it, make their own products out of it, add commercial features on top of it. But one way or another this technology is just going to be available in the stack for people building applications. And it's also going to be quite useful for, you know, integrators building these sorts of applications. They're going to want to have this expertise. And so I think you've got, you know, there's that phrase pioneers get the errors and settlers get the land. I think there's these five companies that are out there that have products that are, you know, pretty strong today. They're early companies, buy them, they like the products. But I think when you look at it two to five years from now, the players are going to be completely different and the products are going to be very serious.
- [20:35]
A
Yeah, Peter, one of the things I want to do to wrap up is I've been thinking a lot about this conversation and I always like to end with a what to do about this for our executive listeners. So number one, right, if you have any level of regulatory pressure or compliance requirements, you're going to need to start thinking about this word Called, or term called explainable AI. And context graphs could help you actually explain why your gentech AI did what it did. By the way, if you're a native AI software company, you're going to get competitive pressure of people saying, hey, I can fully provide an explainable audible agentic AI history or audit for you. And quite frankly, I think that if you're not building this context infrastructure now, you're going to be forced to do it one to two years and it's going to be much more expensive and much harder to try to retrofit an agentic AI implementation that's not built on top of a context graph. You agree?
- [21:36]
B
I, I sort of agree with that. So I, I think, I think the objectives of what you want to accomplish with context AI, a context graph, they're the right questions to ask, oh, I want auditability, I want visibility, I want better control, I want, I want reliability, I want compliance. And so if you're a business executive that's looking at those applications, you and a CEO, you should be asking, well, how am I ensuring this in the projects that I'm taking from pilot to production all the way through? And what is this technology called? A context graph. But the question is, can you get what you want for your application today or is it pretty expensive? Because a lot of it is, say, customers. And so the other thing I think if you are a CIO or CTO or somebody deeply involved in what these projects are, is you should really be engaging with vendors and looking at different options out there. Like, if we, we go back far in our career, Ray, and we remember, we Both remember the Y2K crisis from the late 90s where the people who tried to solve it in 1995 spent way more money than the and spent way more time than the people who tried to solve it in 1998 and 1999, because a whole bunch of tools came along to make it a lot easier. And so, so I would be tracking this closely. I would be experimenting with technologies. I probably wouldn't make a, a selection that basically says this is going to be my context graph for life. I would be picking an application to prove that this actually works. Something that's pretty highly regulated. I would be really diving down into it and then I would be, you know, looking around, seeing what Snowflake, what Microsoft, what databricks, what Salesforce, what, what are companies coming out with that would make my ability to come up with a standard in my stack easier, because I don't think making that choice is 100% easy right now.
- [23:47]
A
Yeah. Well, to summarize, whether you're an AI evangelist, an AI advocate for your internal AI program, or an AI skeptic, I think it all comes down to context of who made the decision in an agentic AI system. Why was the decision made? What were the constraints the decision or action was taken under? Was there a specific policy that we leveraged and. And Israel precedent here? And so, for me, the context graph is such an important concept for any type of AI, pro or negative, that I invite you to come to the AI to Roi newsletter, the Big Story on Substack, and you can get into a lot of detail. Peter, thank you so much for your time today.
- [24:34]
B
You're welcome. It's a pleasure.
- [24:36]
A
Bye, everyone.
- [24:37]
B
Bye.
- [24:46]
A
Sat.