← The Digital Executive
Loading summary
Transcript15 lines
- [00:00]
Narrator
No one goes to Hank's for his spreadsheets. They go for a darn good pizza. Lately though, the shop's been quiet, so Hank decides to bring back the $1 slice. He asks Copilot in Microsoft Excel to look at his sales and costs and help him see if he can afford it. Copilot shows Hank where the money's going and which little extras make the dollar slice work. Now Hanks has a line out the door. Hank makes the pizza. Copilot handles the spreadsheets. Learn more@m365copilot.com work so good, so good, so good.
- [00:33]
Nordstrom Rack Announcer
New markdowns up to 70% off are at Nordstrom Rack stores now. Stock up and save big on shoes, tops, dresses, accessories and more must haves for summer. Join the NordicLub to unlock exclusive discounts. Shop new arrivals first and more. Plus buy online and pick up at your favorite Rack store for free. Great brands, great prices. That's why you rack foreign.
- [01:06]
Brian (Podcast Host)
Welcome to Coruscant Technologies, home of the Digital Executive Podcast. Do you work in emerging tech? Working on something innovative? Maybe an entrepreneur? Apply to be a guest at www.corazon.com brand welcome to the Digital Executive. Today's guest is Akhil Verghese. Akhil Verghese is the visionary founder leader of Krasimo, Steering the company's mission to bring reliable, enterprise grade generative AI to the market. With a background that includes engineering experience at one of the tech's strongest firms, he founded the company to deliver AI solutions built on engineering rigor, clarity of workflow and measurable business outcomes. Under his leadership, Krosimo focuses on guiding businesses through AI adoption, creating multi step workflow automation, deploying multi agent systems based on retrieval, augmented generation or rag, and executing rapid full stack AI assisted development. Well, good afternoon Akhil. Welcome to the show.
- [02:06]
Akhil Verghese (Guest)
Thank you so much for having me, Brian.
- [02:08]
Brian (Podcast Host)
You're very welcome. I appreciate it. And I know you're currently in London, so jumping some time zones, some calendars to head here is always a challenge. So thank you. I'm in Kansas City, so Akhil, if you don't mind, let's jump in to your story here with your journey. You just described taking a fairly meandering path into AI with stops that include IBM consulting and several years as an engineer at Google. How did that path eventually lead you to found Kresmo?
- [02:35]
Akhil Verghese (Guest)
Honestly, meanderingly, if that's even a word, pretty much sums it up. I started my first machine learning projects when I was working at IBM. Even though that wasn't my primary role. And then later on I was with a bunch of people in San Francisco and we tried to co found a company that was an intelligent assistant for your office. Unfortunately, this was before the Transformer paper and before large language models. So it was running on a technique called LSTMS and recursion neural networks and they weren't really able to keep context over large document sizes and things like that. So it didn't really work out that well. But through my time at Google, I started working towards the tail end of it on applications of AI, on more complicated customer service problems. So not your typical like hey where's my order kind of thing, but actually investigating audit logs, providing conclusions, spotting security leaks, things that it was really critical not to be wrong on. And obviously I was trying to adopt best practices from what I could find online and I just found that they weren't working that well and basically identified that we almost need to rewrite the book on how we approach software engineering when it comes to building systems for these very non deterministic things to function well in while still preserving the elements that make them special, which is honestly the non determinism and the fact that they're not predictable is part of. It's a feature, it's not a bug. But at the same time for enterprise systems, you need to build around that. And that's kind of what led to Krasno. It really wasn't well planned or anything. It was just something that kind of came out of that other companies started reaching out and asking if I could help out. I got Google's permission to do that and just moonlighted for a while along with my co founder. And then at some point it became clear that this was a viable business and we both left Google in June of last year.
- [04:23]
Brian (Podcast Host)
So that's amazing. Really appreciate that. And again, and I love this, the stories that start out here on the podcast. You found a gap right in the market and you said, gosh, there is a need here. Let me see if I can take this a bit further. I just love the backstories on these. It's, it's, it's just amazing. So I appreciate that. And Akhil Krosimo's mission is to bring reliable enterprise grade generative AI to market. What gap do you see in how businesses were adopting AI that convinced you there was a company worth building here? And we just kind of talked on that a little bit.
- [04:58]
Akhil Verghese (Guest)
Yeah. So I'm probably at risk of reinforcing a lot of Indian stereotypes here, but most of my friends in school wanted to be engineers or doctors. And I feel like it was interesting because I felt like one of the very fundamental differences that played a role in weighing which choice they went with was their love of being able to understand a system fully and being comfortable with ambiguity and uncertainty and the fact that answers are very rarely yes or no, but could be this, could be that. And so I found that engineers in general are very uncomfortable with that. And it's sort of built in. It's one of the reasons they became engineers. And what I found is that when applying these systems that work really well in demos to enterprise environments, you really have to figure out ways to. To separate out the magic of LLMs into chunks that are predictable, reliable, and most importantly, testable, so that you can bring in the level of oversight and everything else that you need to ensure that people actually feel comfortable deploying these systems. And so this is really the gap that I found, is that engineers aren't really wired to think this way. And it kind of requires us to take a step back, reevaluate the sort of risks we're comfortable with and where we're comfortable with taking them, and then identifying the areas we're not comfortable with, risk and finding workarounds, whether that's human oversight or human in the loop, to kind of ensure that those areas remain not just predictable, but also accountable. Because at the end of the day, you can never hold an LLM accountable for a mistake, and you can't punish an LLM. And so there are certain elements that even if the LLM did it perfectly, you need accountability there for legal liability reasons and things like that. And it's about breaking up that system in a way that all of this works well together.
- [07:04]
Brian (Podcast Host)
Thank you. Really appreciate that. I know there's a lot that goes into it. Talked to hundreds of founders and CEOs, folks like you on the podcast the last two years, and it's absolutely true. There's a lot that goes in to building an LLM, making sure the guardrails are there. And of course, at the end of the day, as they always talk about this human in the loop, truly, in your case, the human must be in a loop because the LLM is ultimately not responsible. So I appreciate that. And, Akhil, you've argued that AI models have become a commodity and that architecture is the real advantage. Can you unpack that for a business leader? Why does the system you build around the model matter more than the model itself?
- [07:47]
Akhil Verghese (Guest)
Okay, yeah, Great question. So I think the best way to understand this is with an example, right. And I think the gap here again comes into this engineering mindset. If something works, it works. It's usually binary. And that's just not the case when it comes to LLMs. So let's take a really simple example. Imagine you're an E commerce company and you want to have an agent that interacts with your customers and lets them do something really simple. So let's take, lets them change the date that their order is going to be delivered. So I've got an order that's going to be delivered tomorrow. I'm not available tomorrow. I want you to deliver it the day after now. Theoretically, right? And not even theoretically. I will put it in writing that modern LLMs can probably one shot this problem. I just tell an LLM, hey, someone's going to ask you to do this. You check that they are editing an order that they have control over. You check that the day they're asking you to change to is the day that we deliver on you. Check that maybe a few other things you put in the prompt and if all of that is a green light, you just go and change it. That I, I am 100% certain that that would work at least 99% of the time. Now there is not a single enterprise company that I know that would deploy that solution. And what is the gap? Right, and so what is the solution that an enterprise company would deploy? Well, what would happen in real life is you would have the person ask the question. You would have a predictable non AI based system that verifies who they are, that fetches their order numbers from a SQL query which is completely structured and deterministic and gets okay, this is the customer id, this is a session key, these are the orders. So they have three orders, which one are they talking about? And then you ask them which one they're talking about. They identify the order they're talking about either with a click or by saving it. And then you validate that and you say okay, this is the order you want to change. Then you let the LLM do its thing. You let it figure out what it is that they want, which day they want it, all of that stuff. And then you have the LLM check with an API call is this modification possible? So you don't just give it right access to the database. There's a control API call that says hey, you can change orders up to three days. The window is 9am to 3pm all of this is fixed in the system, right? And then the LLM can run this call. The LM then checks with the User. Hey, this is what I'm about to do. We're about to Change order number 1234 to 29 May at 2pm on Friday. 2pm to 5pm on Friday. Is this okay? And then when they press the thumbs up button, that's what triggers the update in the order database and then updates the entire system. Fundamentally what I've just described is the same thing. And if you can one shot this problem with an LLM for sure. But you've got to be so worried about prompt injection attacks. You've got to be worried about the cases where it gets it wrong. You've got to be worried about the cases where someone tries to manipulate it. So the gap here is not that is an LLM intelligent enough to do the task that we're talking about. It's the architecture around it that actually makes it something that an enterprise will deploy. And the thing to note here is that it is a binary system. A system like this is either deployable or it's not. It's either useful or it's completely useless. It can be wriked 99% of the time and be completely useless. If there's no accountability, if there's no systems to check that it does only what it's what you plan on it doing. And I think that kind of example sort of illustrates the difference between the architecture gap and the and just one shotting things with an LLM.
- [11:19]
Brian (Podcast Host)
I really appreciate that and you really broke that down for us. I do like that. Again, if you look back at this, you talked about the one shot and probably get 99% of it, right. But at the end of the day you talked about there's obviously there's a security piece of this, right. And you don't just want to give a full autonomy to the LLM, but having that human in the loop, whether that's the customer or a human that works at the organization in the service area, whatever that is, I think it's important. It's almost like having nowadays it's built into our security two factor authentication, right. So same thing with this. There's, there's got to be some sort of check and balances in place. I really like how you broke that apart, so I appreciate that. And then Akhil, last question of the day. As we look ahead to the future, you said you believe AI could free people from the modern 40 day work week and give us more time for self fulfillment and creativity. How do you see enterprise AI evolving over the next few years? And what has to go Right. For that more optimistic future to materialize.
- [12:21]
Akhil Verghese (Guest)
Yeah. So I think it was, this was part of an interview when I talked about the optimistic future versus the pessimistic future. And this is definitely the optimistic future. I, I do believe that that's possible for sure. Right. There was someone actually in London from the Capital Group that had kind of brought me in and was sort of getting my take on, you know, the real world impact on enterprise that AI is having today. And they, their thesis on investment and stuff like that was based on this very large picture view of the global economy. Right. So if the Global economy is $120 trillion, tech is sort of defined in a million different ways. But even if you take the broadest definition of tech, tech makes up about 16 trillion of that. Now about half of that economy, 120 trillion, is the labor economy. Right. Your salaries and things like that. So about 55 to 60 trillion is, is just salaries and things like that. Now when, when you look at the AI evaluations as they are today, and the projections that people are making, they are not restricting themselves to AI gobbling up a good part of that tech economy. They are essentially looking at AI gobbling up the 60 trillion part of the economy. Right. Or some percentage of that, because there's no way these evaluations make sense without that. So there is no doubt in my mind that that is the goal that some percent of that labor force economy is the goal of AI companies to gobble up. Now the question becomes, and this is not a question that I am qualified to answer, it definitely requires an economist, is like, how much of that can you gobble up before the harmony and the balance of the system as a whole kind of falls through? Right. And what is the ideal version of it? So the utopian version of it is anyone who doesn't want to work doesn't have to work because we've got UPI and their lives are comfortable, fulfilled. They find fulfillment in their hobbies and art and family and relationships and things like that. And then the people that do want to work, no one's working the kind of hours that we're working now. There's not this crazy burnout. Humans are still required for the more creative tasks, the more the tasks that we require accountability, like I mentioned. And the truth is that LLMs, while they're becoming incredibly smart, I think, I'm not sure who coined this term. I heard it from Andre Karpathy. But the idea of a jagged intelligence curve. So LLMs are kind of progressing along a very jagged curve. There are some areas where they're just brilliant, and then there are some areas where they're shockingly bad. And so humans will always be needed to sort of pick up the slack and fill in the gaps. And the question becomes, if that requirement becomes smaller and smaller, is that achieved by laying off everyone, or is that achieved by everyone working less? And obviously, I think the more optimistic solution, the more egalitarian solution, and probably the better solution for humanity as a whole is that everyone works a bit less. So, yeah, that's kind of the ideal future that I see. I think there's systems like that are never built by the companies deploying the technology. They have to be built by the people and by governments. I haven't seen a lot of proof that that's where we're headed. But at the same time, I also do feel like society does have checks and balances for this thing. There is a certain unemployment level that society tolerates, and this is just borne out by history. Like, I don't know what the unemployment level was before the French Revolution, but, you know, it was. It can't have been more than 20%, I think. So, Again, not a historian, but I think there is some balance where. Where society as a whole will push back. I think we already are seeing that pushback on AI, and so, yeah, I think we're headed for very interesting times, for sure. And whether that ends up being a good thing or a bad thing, I think is. Is very much in the balance. But, yeah, I'm very hopeful for the optimistic future. But let's see. Life.
- [16:09]
Brian (Podcast Host)
Yeah, like you, I'm very optimistic. We just need to understand that guardrails need to be in place all around. And there's some. There's people I know that literally work for maybe minimum dollars per hour under the federal rules and guidelines, labor laws, and they're fulfilled. They have purpose in their life. And so there's those situations, too, where, well, AI could replace that person, but if that person gets fulfilled doing that work and it makes sense for the company, for example, whatever that salary they make, then we. We need to ensure that stuff is in place as well. But I appreciate the insights, really do. And Ankiel, I really appreciate having you on today, and I look forward to speaking with you real soon.
- [16:50]
Akhil Verghese (Guest)
Thank you so much for having me, Brian. It was great, great answering your questions. They're really great questions.
- [16:54]
Brian (Podcast Host)
Bye for now.