← The Digital Executive
The Digital Executive
Ashwin Rao on Half the AI, Half the Answer: Why LLMs Can't Do It Alone | Ep 1284
00:22:10
Loading summary
Transcript15 lines
- [00:01]
A
This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome? That's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50 page restoration block. Or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it, ready to make anything online make sense. There's no place like Chrome. Check responses set up required Compatibility and Availability Various 18
- [00:27]
B
this episode is brought to you by Google Health. Stop chasing someone else's definition of health. What matters is what's healthy for you. Google Health offers a new kind of coach built with Gemini for effortless tracking, sleep insights and holistic coaching tailored to you. Visit googlestore.com to learn more and start a new relationship with your health. Requires Google Account, Google Health app, Internet and Google Health Premium subscription. Features subject to change availability and results vary. Not intended for medical purposes. Works independently of Gemini apps. Check responses for accuracy.
- [00:56]
A
Foreign. Technologies Home of the Digital Executive Podcast do you work in emerging tech? Working on something innovative? Maybe an entrepreneur? Apply to be a guest at www.corazon.com brand welcome to the Digital Executive. Today's guest is Dr. Ashwin Rao. Dr. Ashwin Rao has more than two decades of extensive experience across business, academia and Wall street with a consistent focus on creating outsized business value through technological innovations. Specifically, he has led science and engineering teams building products for trading, risk management, pricing, forecasting, supply chain, personalized customer experience. He brings unique skills in translating challenging business problems into technical specifications and solving them by blending traditional mathematical algorithmic methods with modern machine learning methods. Well, good afternoon Ashwin. Welcome to the show.
- [02:02]
C
Thank you Brian.
- [02:03]
A
Awesome. I appreciate you jumping on making the time. I know that you are in the Silicon Valley area. Palo Alto. Love doing a podcast out of that space. I'm in Kansas City, so I just appreciate you navigating time zones calendars to get here today. Thank you so much. And so Ashwin, if you don't mind, I'm jumping into your first question. You've had a remarkable arc. 15 years on wall street leading quantitative trading at Goldman Sachs and Morgan Stanley. Then Chief AI Officer roles at retail and distribution enterprises like Target and QXO, all while teaching at Stanford. And you recently joined 09 as executive vice president of NextGen AI. What's the connective tissue across trading floors, retail supply chains and academia. And what drew you to 09 at this moment?
- [02:50]
C
Firstly, thank you Brian for having me on your show. It's my pleasure. Yeah. So let me a two part Question. Let me address the first one. What's common across all the different things I've done? Brian, Even though on the surface it looks like I've done a lot of different things. Academia, 15 years of wall street supply chain at target and enterprise AI. Now, at the end of the day, a lot of this boils down to the core of what people like me do, which is we look in business problems, we try to develop a strong understanding of the important parts of the business that we need to capture, and then we translate that into mathematical models and then the next step is to implement software and algorithms that would solve those math problems and consequently the business problems that the math model captured. So if I reduce it to those grassroots, that is very common. And then I'll take it a step further that when I look at this pattern of like understanding business problems, math and software, and as I've gone across so many different industries and academia, I've just found so much commonality between so many disparate topics. For example, when I was doing Wall street trading versus when I was doing supply chain, at the heart of it, some of the math is extremely common. So I enjoyed going from one place to another because I've been able to take a lot of technical aspects and sometimes business aspects too, from one area to another. Because as I've grown older and more experienced, I've realized there's way more in common across industries and across academia than what I'd imagined when I started my journey, which is like almost 35 years ago.
- [04:47]
A
That's amazing. Thank you for sharing too. At the end of the day, and you talked about that there's way more in common across these various industries and verticals. At the core of it, there's always problems and there's always going to be people involved in these too. And so you can see a lot of where a lot of your experience has easily translated into the various industries. And I appreciate that. Wall street, teaching at Stanford, supply chains, et cetera. The core of what I took away here, Ashwin, is you look at business problems and truly try and understand them, map them out, translate them into mathematical algorithms in some cases and convert them into software to find those technical solutions. So I appreciate you breaking that apart for us. Ashwin, you've argued that large language models alone can't deliver reliable agentic AI for enterprise planning and execut that only by deliberately blending neural AI with symbolic AI can you build agent enterprises actually trust to run on their operations. For a leader who's been told LLMs can do everything. What's the case for neuro symbolic AI and where specifically do pure LLM approaches break down?
- [05:58]
C
Yeah, that's a great question. It's a big question. A lot of people are asking, can LLMs actually do everything? And before I answer this question, Ryan, I guess I neglected to answer the second part of your first question, which, which actually nicely dovetails into the second question. What actually drove me to 09 was I was talking to 09 for several months before I joined 09. And through those conversations I realized that 09 had over 15 years built a type of AI that is very complementary to the language model type of AI. So the AI online has invested in over 15 years is called symbolic AI, which I'll talk about in a minute. But the language model AI is what a lot of people today recognize as AI. Because if you ask any person today, when people use the word AI, they really mean these language models like ChatGPT or Claude or Gemini or something, that language model AI is only one type of AI and it is what I call Neural AI. Neural meaning because these models are trained on large neural networks or in simpler speak you could call it statistical AI. You throw a lot of data at the problem. You throw a lot of data, it learns, it trains and learns from the data and it infers patterns. The pattern recognition from large data is its power. That's what this neural AI does. And today neural AI is appealing to us in the form of language models. But they cannot do it all. They're very powerful, but they only half the story. And this is my message for leaders who might be thinking language models can do it all, they can do half of it. And the other half is what I alluded to earlier. It's the symbolic AI what's remarkable. And that's what drew me to 09. And over the last year I've been thinking hard about this and I'm very excited about neural symbolic AI because what I described to you with neural AI, symbolic AI is quite the opposite of it doesn't do this kind of big data training. Symbolic AI is the world of traditional mathematical logic and enterprise knowledge graphs. It's about structure, it's about detail, it's about precision. So if language AI, so the LLMs you can think about as the AI of language symbolic AI, you can think about it as the AI of mathematics and structure. If you think about neural AI as approximate AI, and it is approximate because it's never going to give you a precise answer. It's going to be people recognize now it's Probabilistic. So it's approximately. Neural AI is approximate. Symbolic AI is an exact science and approximate is not a bad word. Approximate is what makes it strong, actually. That's the power of statistical approximate pattern recognition. But it's also its weakness because it can hallucinate. I think in common human terms, if you think about, we have two types of intelligence and it maps quite nicely there too. We have this intuitive type of intelligence which is the neuralaya. And then we have this other type of intelligence which is a more rigorous or if you read Dan Kahneman's book, the fast thinking, slow thinking brains. The neural AI is sort of the fast thinking brain and the symbolic AI, the slow thinking brain. And humans have really. Humans have these amazing intelligence because they combine these two things very effectively. So that's the story of neurosymbolic air. Very complementary, very exciting. We are, I think, at a very early stage of a journey where we can now, at 09, this is what we're doing. We are combining these two things, tastefully complementary. And I think when you combine them, the result is greater than the sum of the parts.
- [09:51]
A
That's awesome. Thank you. And I appreciate you unpacking that. You talked about the, really, the big question I had was can LLMs actually do everything? And they can do a lot, but not everything, as you mentioned, and you kind of broke those apart. The, the neural or statistical AI, as you call it, learns from a ton of data and patterns, pattern recognition. And you called it the AI of language. And then the symbolic AI is AI of mathematics and structure, which is interesting. But you're really excited about neurosymbolic AI because really you're bringing that together to make something more accurate, obviously less hallucinations. And at the end of the day, what we want is something that is going to serve the problems that are presented today in our business world. So I appreciate that. Ashwin, you've talked about building agents that enterprises can actually trust to run their operations. Trust is the central barrier to autonomous decisioning at scale. What does it take to earn that trust technically in terms of explainability and reliability? And how should a CFO or COO evaluate whether an AI agent is ready to be given real operational authority?
- [10:57]
C
Yeah, this is really a brand question a lot of people are asking because people are starting to use agents in an enterprise context and they're not quite at the level of reliability that we are comfortable with traditionally. Leaders at the CXO level get comfortable with the level of trust reliability, and agents aren't quite, quite delivering it for those things. My take on this is that we're in a very early stage of this journey. We have to give this AI time to mature. Even neuro symbolic AI that I talked about in your previous question. It's also at the early stages. So we have to think about where do we trust AI, where will AI be reliable, what type of business problems. And then we think about where we would be in two years, five years. But let's only talk about today. Today, given where AI is, I would say that leaders should look at problems that are relatively low stakes. I would not allow AI agents. And when I say agents, I mean it truly has agency to act, to change prices, to make important strategic decisions. Like I want to open a branch in a new country. Like these type of important. Those are high stakes decisions. Anything high stakes, they are not ready to delegate to that to AI. So we have to climb that journey from low stakes decisions to high stakes decisions. And the way to enter this world of allowing AI to make decisions is deploy them in some of the low stakes decisions. So you learn where you feel comfortable, where you don't feel comfortable and the AI will improve and mature. And as this journey progresses on AI, our journey progresses in how much we trust, what kind of comfort we have at it. And as time progresses we will start allowing it to make decisions on higher stake decisions and tasks. There's always been this pattern in especially neural AI and statistically that this type of AI is good for problems where there's some tolerance for error. If I'm just chatting with Cloud or Gemini or ChatGPT and I'm just trying to learn something, there's a lot of tolerance for error there. It may not give me the perfect answer, but it gives me sufficient that I get my job done. Those type of things where I'm engaging with it. I'm traveling tonight and asked it some questions about PSA and transfer in London and stuff. And later I realized it was not perfect. These type of problems are perfect for AI. There is some tolerance for error there. But there are many things in enterprise context where there is not any tolerance for error. I would not allow AI to take agency over there. We have to wait. We have to cross that journey. And that is exactly why we are building neural symbolic AI. Because one thing symbolic AI gives you is gives you guardrails. It evaluates the results of the neural AI. It keeps asking it probing questions whether the neural AI got it right. It checks against exact boundaries. There's a lot of rules and constraints in our business that Symbolic AI ensures are validated. So the verification game is played by symbolic AI. If you think about neural AI playing the offense game, symbolic AI will play the defense game. And the defense game is what is going to increasingly create reliability and trust in an enterprise. And that really is our mission at 09. To not just provide offensive AI, but to also provide to bolster it with a defensive AI of symbolic AI.
- [14:40]
A
Thank you, I love that too. Just again, just to highlight some of the things you said. You know, neural AI is being really the offensive AI and symbolic being the defensive AI. Again, having both of those kind of like playing on a sports team. Right. You need both teams to win a game and, and to do it right. That's awesome. Business art I've done. I talk to a lot of CEOs and executives like yourself that are starting to deploy AgentIC AI in the enterprise. Some with success, some not so much. And you talked about that reliability and authority that, that issue that exists today and right now. Low stakes decisions, not a big deal. High stakes going to be a big deal for sure. And as you said, there's tolerance allowed for a lot of this stuff, especially conversational type stuff, but there's zero tolerance in these high stake in areas within the enterprise. And I'm glad you highlighted that. So thank you. Ashwin.09 talks about building the next generation of agile adaptive and autonomous planning and execution models for an increasingly volatile, uncertain world. As you look five years out, what does a truly autonomous enterprise actually look like? How much of planning and execution will be machine driven? Where will humans remain essential and what has to be solved technically to get there?
- [15:56]
C
Before I get into the technical aspects, I'll say that there's some cultural issues we have to solve in organizations. I think leaders have to be willing to invest and prepare in this transition. There are going to be a lot of changes in the next five years as AI is going to take over many of the functions AI is going to perform. All these agile adaptive autonomous activities, that's change management. So before we get into the technicals, I want to emphasize this is more important than getting the AI to actually do the job and make it successful, that we have to be willing to invest in this transition, we have to prepare for this transition, we have to prepare for this change management. But I think once that is done and we have this good attitude that will allow AI to come in into certain activities, I don't think AI is going to take over some of the higher order activities in an enterprise. So what do I mean by higher Order. If you think about typical enterprises, I could break up the decision making into maybe I'll say three levels of decision making. There are these strategic decisions that are made at the highest levels of the organization, perhaps with a C suite. Then there are these tactical decisions made. For example, you make a decision of what assortment should I have in my store this summer? How much promotion should I do for swimwear? I'm taking these examples from retail because that's my background. Those are tactical decisions, those are sort of mid level decisions. And at the lowest level every day we are making these operational execution decisions. How much inventory do I move from my warehouse to my store? How many people should I staff in my store on a busy summer day where people, a lot of people are pouring in to buy things? So these are very tactics. So not tactical. I meant operational. So the strategic, then this tactical and then the sort of the execution of operational level decisions. AI is going to penetrate from the bottom. AI is going to automate all of the operational execution decisions. The algorithms are going to decide how much inventory to move, what routes to take when the truck drives from the warehouse to the store. How many people should I staff in my store on any given day? So this is kind of where we are today. I think AI is increasingly taking on the decision making, planning on those lowest level decisions. Humans have been doing all of these operational decisions. Humans in spreadsheets have been figuring out how much inventory to move. How much labor should I have in my store that's going to get taken over by AI. So where do humans move? Human move? The humans move to the higher realm of the decision making. The humans who are doing more of the operational execution decision making, they would get involved in the tactical decision making. And I joke about this with my team and I say that I think everybody's going to get a double promotion. So if you were doing work at the level of a director, you're going to get double promotions, senior Director and then VP'd because you're going to make higher order decisions for the company. You're going to be a lot more. You'll go from operational to tactical. If you were tactical, go from tactical to strategic. So that's the movement that's going to happen. The other perspective I will give is we are going to do less of problem solving and do more of problem specification. I think this is the barrier I see in the future, as time progresses, that machines are going to do more of the solving and the acting and the getting things done. We are going to define what are the important problems to solve and we're going to get very detailed, perhaps very pedantic about specifying exactly what problem to solve and we are also going to decide which problems are worth solving. And this you can see that this already maps to the earlier point I made about higher order decision making is like what are the problems worth solving, where should I deploy my capital? And being very rigid and strict about this is the problem I wanted to solve and not this one. So that will take us so this divide is where will take us into this future world. I think Neuro Symbolic is going to deliver this agile, adaptive and autonomous capabilities in the way I described.
- [20:16]
A
Thank you. Really do appreciate that. Ashwin, just to highlight some things you talked about obviously right now and we see it there's cultural issues that we're dealing with in large organizations especially these need to be resolved up front. There needs to be a strategy on deployment, communication, acceptance and ultimately that's the change management piece you talked about. But decisions are being made at all different levels right now. Strategic, tactical and operational level decisions as you said and AI will certainly right now and they are at the forefront of these operational or execution level decisions I should say and I liked your term on double promotion that the fact that and you shared some example a director with more proliferation of agentic AI in organizations. We're going to see that where you you said that the double promotion is high order level of decisions. A director may see a promotion, a double promotion of like you said senior Director or executive vice President. There's going to be a lot of movements across the positions in these organizations due to the deployment of AI as it advances. So I appreciate your insights and Ashwin, it was such a pleasure having you on today and I look forward to speaking with you real soon.
- [21:25]
C
Thank you Brian for having me on your show.
- [21:28]
A
Bye for now.