Loading summary
Transcript52 lines
- [00:00]
A
How can I use AI to make my life easier, both personally and professionally? And how can I, you know, just improve my own productivities? From idea, from conceptual idea to product, AI was a partner.
- [00:15]
B
Welcome to the Think AI podcast. Each week we talk about the most exciting AI research tools, case studies and more. I'm your host, Dev Goyer, and I've been working behind the scene in data and AI for over 30 years. 30 years. Whether you are an AI expert, skeptic or something in between, this podcast is for you. Today I have a guest I'm really excited about. Priya Udeshi. She's the Chief of staff to the CIO, head of the IT PMO at MogoDB, one of the most important data companies in the world. And she sits at the intersection of AI strategy, IT transformation and enterprise execution. She's not just watching AI happen, she's in the room where IT gets deployed, governed scale across the global organization. Priya, welcome to the show.
- [01:10]
A
Thank you very much, Dave. I am very happy to be here and excited to talk about a topic that is very top of mind for pretty much everyone in the tech industry.
- [01:18]
B
Great. So let's start with your story. How did you get involved with AI and how has the evolution helped you what you're doing today at MogoDB?
- [01:29]
A
Yeah, no, for sure I'll give a little bit about just sort of my background. So I've been in the tech industry for about 17 years. I've kind of grew up through the channel of project, program and portfolio management. So I think being at that, like you said, the intersection of strategy, execution, operational discipline, it's really cornerstone to, you know, driving PMO leadership, driving modern IT portfolio leadership. So that's been my primary space. Um, I've been at Mongo now for three years. So I joined leading a technical program management function under our CIO's office. Again, driving technology enabled AI powered strategic delivery across the enterprise. About seven months ago, officially assumed the chief of staff role. So again, kind of getting that inner corner office of the cio CIO vantage point for all things AI. You know, I would say that my AI journey, you know, this word AI, has actually been in practice in our, you know, tech evolution for many, many years. Pretty LLM era. And so I would say, you know, prior to this sort of an official launch, you know, kind of pre GBT era, AI was really a wrapper for, I would say, all things predictive, analytics, machine learning, robotics, process automation. Right. So like rpa, you know, capabilities, any type of automation that you could Put some predictive algorithmic, you know, a boundary around, was considered AI at the time. And so, you know, a lot of the project initiatives that, you know, I had led or I had, you know, folks on my team leading were really in that space. Um, ChatGPT, you know, was launched just before I joined Mongo. You know, just as I was entering Mongo. We did, I would say, what most tech companies do. We started on our chatbot era. It was really focused on, you know, how do we, you know, we didn't want to just roll out ChatGPT at scale, you know, across the enterprise. You know, you have your security, all of those things are very important. So we didn't want to, you know, start pumping, you know, Mongo specific information or internal company data, of course, into those public LLMs. So we went on the journey of building our rag infrastructure, connecting a GPT style LLM to our data sources. We ended up building something called Mongo GPT which is actually still in existence today and it is used across the enterprise. So, so that was really, I would say the learning phase of kind of the AI evolution. Fast forward to the agentic phase, which is what I think we're in now. We kind of have two paths at least at Mongo. Enterprise agents and enterprise scale level agentic workflows definitely is CornerStone to our AI strategy. Looking at how we can improve the lives of sellers right across our go to market teams. Like how can we build GTM or kind of go to market agents to help sellers get everything they need to know about their accounts, their customers right at their fingertips. On the people team side, how can we build agents to address employee cases submitting simple task workflows. So there's definitely a good portion of our AI strategy that is, you know, focused on infusing AI into the, the daily workflows across all of our customers across the business, which again, working in it, our customers are every employee of the company. But then we also are looking at how can we actually democratize the use of AI for everyone in the company. I mean, you know, we talked about this when we first met. You know, Dave, like you and I should be able to create our own agents. You and I can create our own agents today, right? It doesn't, I'm not, I mean, I'm an engineer by trade, but I don't sit behind a computer and code, you know, in my role today, it's a very different, you know, kind of vantage point. But how can I use AI? We're all thinking that we're all Asking that question every single day. How can I use AI to make my life easier both personally and professionally? And how could I, you know, just improve my own productivity? So, you know, we, we, we do have, and, and that's a real thing that I think a lot of companies are facing is, you know, what we don't want happening is similar to every. And I would say being in tech, we've, you know, we talked about this the other day too. Like the notion of shadow it, right. There's, you know, when, when an enterprise solution is maybe not as quick or as fast or coming into your fingertips as quickly as you want, what do you end up doing? You, you go and buy it yourself or go and build a solution yourself or purchase a solution yourself. And we will and are seeing that happen with AI as well. Right. So if we don't, you know, keep up with the pace of innovation and speed that is necessary to get these agentic workflows and these AI infused into the day to day of every single employee. You'll see people creating their own personal accounts, you know, creating their own, you know, agent capabilities outside of maybe your company enterprise, you know, secured environments. And that's exactly what is counter to what we want. So those two streams, I would say enterprise, enterprise scale is definitely a stream. And then also low code, no code, you know, democratized agent creation is another avenue that we're looking at as well.
- [05:58]
B
No, that's really good to hear and that's a pretty powerful story that you have. I want to follow up on one thing, I think the other day we talked about it before that I want to ask you. I think I know the answer. I'm looking at three different buckets today. One is AI curious who are just seeing ChatGPT as everything. Then there are AI enthusiasts who are actually seeing the ChatGPT. They are seeing the other world like Claude and so many other open Source model Kimi 2.5 and so on. And then they are AI skeptic where they think if AI touches anything, it's gonna bring our world to the knees. Where do you put yourself and why?
- [06:40]
A
Yeah, I mean, of course the majority of my, I would say where I'm at is in the enthusiast bucket. But even being in the enthusiast bucket, there's a ladder, right? And there are so many rungs on this ladder. And I by no means will say I'm at the top rung, I'm probably at the lower to mid, you know, run. Of course I am, you know, exploring. I think just again having that vantage point within Mongo and seeing how, you know, we are bringing to life a lot of these, you know, AI infused capabilities. But, but then on the personal side, I had mentioned to you the other day, I, you know, got the itch to try to figure out how can I like, you know, start building my own stuff. And I built a lovable website and some AI powered digital products and I connected, you know, to a GitHub repo and I used Claude to help me, you know, do all of that. Which, you know, I really was starting pretty fresh from being able to do that. So I think there's a ton of power and a ton of opportunity ahead. I do feel that, you know, even with the skeptics and maybe the folks that are still using ChatGPT today, I believe that the evolution of where we're going to be with AI even a month from now is going to look vastly different. I have seen the timelines around the burst of this emerging technology is so much, far more compressed than any other emerging tech that at least I've been privy to in the 15 plus years. We saw the cloud boom and then the SaaS kind of boom and all of those still, they took time. People are still on their cloud journey today of migrating into cloud based environment. So but with AI, you know what the landscape looked like. Even with cloud, I would say, you know, that what the landscape looked like just a month or a few months ago was vastly different than it is now. So yeah, I think I would put myself in the enthusiast bucket, but I still have a lot more to learn in that space. So I'm just excited to keep learning.
- [08:22]
B
No, you're humble and I see more humble people will say I'm learning. And that's a good thing if learning stops. Because everything in your life you have to learn. You know, even if you're a musician, you are an artist, you are a player, you are always picking something up from someone else and that's a great thinking and the value that you have. So I appreciate that. I will also pick up on one more thing. I think the other day we talked about, and I could be wrong, Glean's agent framework that you mentioned. What is it and how do you see it fit in what you're going to do?
- [08:55]
A
Yeah, so Mongo has adopted Glean as our enterprise search platform. And that's actually the other, you know, we talked about enterprise scale, scale agentic solutions, we talked about democratized low code, no code agentic solutions. Enterprise search was the other. I'll just give a little bit of A story before I kind of talk about the Glean agent platform, but I having grown up in this PMO space in many of my parts of my career, part of my remit was to implement a project portfolio plan of record, right? Like, let's implement a PPM tool, a PPM system. We'll have one central quote unquote database for project and program tracking assets like I always think project are data objects. And just like any data object, they have attributes associated with them. Risks, issues, milestones, stakeholders, you know, resources, all the things that can inform the quote unquote health of a, of a project or program. So in many, you know, experiences of my tenure, I have had this remit of implement a system of record, implement a plan of record where we can pump all of our project data and run analytics and run, you know, data driven insights, obtain data driven insights off of them. And I've, you know, those have been successful. I've, you know, implemented multiple PPM tools. They've been successful. But even as a PMO leader, even with leading a team of whether they're project program TP, you know, program managers, TPMs, as much as you can really drive to say that, you know, this, this solution is to help you do your job better. The solution is to help you, you know, make life easier. There's always this administrative component right in the day to day of running a technical program in full transparency. The tpms and the project teams are not living in a PPM tool. They're living in docs and sheets and unstructured data sources, right? Like that is how they're making decisions. That is how they're, they're documenting, driving decisions forward. They're not going and updating a, you know, PPM tool. So like the, what I found in any of those implementations is it becomes a little bit of an afterthought even when you do all, all the things to try to not make that the case, to try that be, have that be the first forefront of driving program delivery, it still tends to be an afterthought. So what I see with, with Glean or with enterprise search capabilities is it solves that problem. Meet people where they are right people. And you and I both have different ways that we're using tools to help us do our job. And yes, we should have enterprise standard solutions and we don't want, we talked the other day about things like Sprawl and I'm also, I get the gift of leading our SaaS Sprawl initiative also. And so yes, that is a Real thing. You don't want to have this kind of hodgepodge of a bunch of different tools and different things that people are using. But what enterprise search does is it doesn't try to make this one size fits all of whether it's, you know, I gave the use case about project data, there's so many other use cases for that I'm able to just hey, hey, give me the latest on project 1, 2, 3. And it's searching the network for all of the different, you know, data sources that contribute to that. And it's, and it's translating it using AI powered summarization to translate that in a, into a meaningful output that I can Then again, whether I'm writing a QBR deck or giving a strategic update to our senior leadership team, that's the power of enterprise search. So that's where we started with Glean. We have since kind of moved away from mandating the pumping of project information, at least from my team standpoint, into a singular tool. The next stage of what we really want to do with Glean, and there is a dedicated team to focusing on this is how can we release an agent moderation framework across the enterprise so that we can build again build our own agents directly on the GLEAM platform. So that is, that is sort of our next, next phase of the evolution.
- [12:29]
B
Oh, that is beautiful. And it just reminded me of one of my own stories. I worked with a large healthcare organization and you know, they had several operating companies across the globe and we were managing different programs and every year we used to evaluate one of the PPM tools. So we had to do PMP and all that, but that's fine. But then we are looking at PPM tools and guess what? None of the tools were really solving the purpose. Like you mentioned, you don't live in the tool and all the PMs hated it. They didn't want it to go into the tool because they still have to see if you're entering the data there somehow that should made into the people who are looking at it but they don't look at it either. So then you're spending time in meetings and then saying the same thing that you entered over there. And then they had a legit reason that okay, I don't want to update it because I'm already providing it in so many other ways. And I don't blame them. We started doing it in our organization from that point onwards. So in our consulting business we are also using consultants and employees in India and other places and those, you know, they have different cultures, as you already know, different time zones and things. So we started using different methods, like you know, maybe WhatsApp teams chat and started collecting it and started putting back into the system in the BPM tool itself. But it's more on the.
- [13:50]
A
No, I just imagine if you had agents to do all that for you.
- [13:54]
B
Yeah. And those are happening as we speak. Right. So they created their own unified box. I have my own unified box. And those are guiding us what to do. Not that we don't want to connect human to human, but that connection should be about solving problems rather than providing updates. And that's one of the thing I personally hated most, that you are just sitting in a meeting where 10 people are providing update, which you could probably read off and start taking actions.
- [14:22]
A
You are preaching to the choir. I think about this a lot too. I mean, again, being in the PMO space, this is the evolution from outputs to outcome driven leadership to value driven leadership to what is that focus on business outcomes and how that really underpin everything that we do. I've always said the outputs, again, coming from the vantage point of program delivery, the outputs are important because they inform the outcomes you can't drive. And of course an outcome is like, you know, I don't want to say it's a, you know, you have a strategic priority and intended impact on the business that you want to have. There's a set of work that has to be done bringing that down to earth and decomposing that, that strategic, you know, pie in the sky, you know, intended value outcome. You have to do a body of things instead of, yes, there's tasks and activities and things that have to be done to realize that outcome, but the value is in driving towards that outcome. If you can automate, build agents to focus on the outputs to your point, then the shift in the conversation becomes very different. You're not talking about administrative tasks and activities and tracking progress against those because in a, in an ideal world you have agents doing that. That part for you. You're really the shift in the, whether it's a meeting, a report out, an update from one of these reporting tools, you're, you're able to, to look at a data driven report that gives you decision velocity. Right. That gives you what, what do I need to do next to actually drive progress towards this outcome? And that's where I think AI is going to really change how, how many roles, as we already know how many roles operate today. And I think with that intended shift into focusing on the outcomes and accelerating decision velocity across the business.
- [15:59]
B
Supriya, thanks for that great insight. One of the things we also like to hear from our guest is that how what you do in your job and your passion helps and solve the real world problem. Some of the things we already talked about, we personally work in manufacturing and healthcare space so we are focusing a lot more there. What's your experience? Looks like what customers at MongoDB ask for in terms of solving their own original problems and how you and AI is helping them.
- [16:32]
A
Yeah, no, thanks for the question. So yeah, as I shared earlier, like my passion and I think why you and I, we had such a great call even earlier this week too. You know, we're problem solvers. And so I think the crux of like the, the basis of how I grew my career where I kind of evolved my passion for this space of IT strategy, IT transformation program and portfolio leadership is rooted in that fundamental core principle of loving to solve problems, loving to help optimize. My husband kind of jokes and says I'm to type A for my own good.
- [17:08]
B
I can relate to that. I have someone at home also. Same thing here.
- [17:13]
A
It's you know, the relentless and ruthless focus on optimizing everything down from like my own personal day to day to you know, how we saw how we serve the business again, working in it, how we're driving solutions for our customers who are every, you know, employee of the business. Like, how can we ensure that we are delivering in the most efficient and effective way with again that relentless and ruthless focus on driving strategic value for, for end customers. And I jokingly say like, you know, it's it, it tracks that I grew up in this space of project and program management. I am again, I'm type A, I'm a Virgo, whatever you know, you want to say, like, you know, kind of contributes to that particular Persona. But it is, it's a Persona for I would say people that really thrive in driving efficiency. And you know, I said, he says I'm too type A for my own good. He also says I'm almost to a fault like if it's not done in the most efficient way. And it's like anything even like cleaning the house or you know, packing for, you know, trip and things like that. Like I'm, I try to just optimize and make, you know, as efficient as possible anything that I do. So again, and how that translates in from a professional standpoint, again that that's where, you know, when I say I'm an AI enthusiast, it's because when you talk about driving Efficiency and optimizing how you do things like that is what AI is for, you know. So I am going to continue to be a forever learner about how can I use the tools and the resources available to me. Of course, security is very important. So I think again, just being in. In it, you know, I. I don't think I'm the person that's going to just throw my likeness all over a bunch of different tools and see which sticks. Like I do want to be thoughtful about what information I'm putting out into what tools that I'm using and how I'm using those tools and how my information is being used within those tools. So again, I think taking a thoughtful, just again, both professionally and personally, how can I use AI to make me better at my own job, at everything that I'm doing, at everything that I'm focusing on? And how can that translate into actual value that I'm driving, both for the enterprise as well as our customers?
- [19:19]
B
Now that's a great talk. And one of the things you talked about being Taipei, I'll divert it to a better place. What's the difference between the leadership think AI does versus what you actually do it on the ground? Because sometimes, and this is a good thing with type A people, sometimes they are. They are driving it in the way it needs to be versus what is needed to be on the ground. So how do you use your personality type there to build better leadership within yourself and within your teams?
- [19:49]
A
Also, how I'm using AI specifically or.
- [19:52]
B
Yeah, you and AI both. Ultimately, your AI will be your clone, right? I mean, at the end of it, it will be sort of an emotionless clone of what you do, but it will do it in a better way. So your personality would always reflect in what you do in AI?
- [20:09]
A
Absolutely. And I think. I think we talked about this earlier this week too. Even with AI is exactly that. It is a partner. It can be. Even on this personal project that I tried really just with the intention of professional development, learning more about AI so that I could be better in my day to day job. From idea, from conception, conceptual idea to product, AI was a partner. The AI was in the passenger seat, I was in the driver's seat. And I think that always has to be the core. Like you. Yes, you. An AI is not going to be a replacement for your ideas. Your ideas are your own. There's a book that I read to my kids about similar like you want your ideas to be your own. That should never go away and AI will never replace that. What AI can do is help polish your ideas, sharpen your thinking, give you, you know, a vantage point that maybe you didn't think about. But you also have to be mindful that that vantage point is based on whatever information could be scraped from the Internet, you know, and surfaced into the LLM. So take it for what it is, but always feel grounded in your own intuition, if that makes sense in terms of how you, how what your approach to leadership is, what your approach to human empathy. Nothing replaces human experience and that will always be the case. And I think having been in the industry now for, you know, almost 20 years, there's, there's human experience that is, is, is valuable to me. It is part of my value. It is part of the absolute value of the value that I drive. And I will always use that to, you know, really influence, you know, my, what my approach to leadership is. AI is just going to help me sharpen that. It will never replace, you know, me and my thinking. But, but what I will say is at the same coin or like at the same time, I do believe that you know, a few years from now, in every professional role, AI will in some shape or form, even for the skeptics, be infused in their day to day into their workflows and what, you know, whatever that shape or form could be, will look different. But I do truly believe that with the accelerated, you know, timelines that we're seeing, with the evolution of this emerging capability, it's going to be in the hands and in the workflows of every single professional across the world. I think.
- [22:24]
B
No, you said it well and you just mentioned skeptics. I think everyone, whether they don't know it or do not know it today AI is already there. YouTube is listening and then it's showing you the videos that you like. You go to and start to do a room scrolling and then you see everything that you have done in here, wondering why it shows because you actually clicked on something and is building the traces of it. So you are using AI or you are the consumer of AI without knowing it. So it's going to happen. One of the good stat I found, I forgot where I found it. So call it conspiracy if we cannot define a fact here. But 3.3 months, AI is doubling its capabilities and capacities both right, which means, and I did the math, it will be about million times better than what it is today by 2030. And that's a very short amount of time, you know, just because how it is multiplying today. So I can only imagine what are the Goods and what are the bads it can bring in. On that note, what surprised you the most with AI? I mean we both have been learning about IT implementing in our lives and what you didn't expect when you started implementing AI, meaning you know, when you heard ChatGPT. I mean we both have worked. I started AI in 1996, 97, doing algorithms and things like that. A lot of things had to pre plan, you had to have powerful computers. You have to figure out these machine learning algorithms and get a lot of testing data and everything. But now that all has been gone, or at least people think it's gone. So how, what's, what was your expectation and the unexpected thing that you saw with AI today?
- [24:12]
A
Yeah, so I mean compute. You already kind of, you know, the compute aspect and how we can't keep up and keep up with the pace of the needs of, of AI from a compute angle, I think is something that every infrastructure and technology company is having to grapple with. But another one that I would say, and I would say I wasn't, it's not that I am surprised that it happened, but maybe I wasn't. But I'll tell, I'll share, I'll just share what, what it was, you know, when we talked about like the, the chat GBT coming out and the mon, you know, we created our own Mongo gbt and then you know, of course Gemini, we have Gemini in our Google suite. We have you know, enterprise search. We have this, this notion of sprawl, right? Agent sprawl is a real thing. And again this is where I say I can't say I didn't expect it because we saw it happen with SaaS. Sprawl, we saw it happen with tech sprawl, with process sprawl, with data sprawl. I mean these are words that every scaling technology company has to figure out. And again working in this space of program management and portfolio leadership, I have across all of these emerging techs have had to balance this. I've always had to play this balancing act and me and my team and the value that we deliver. I, you know I always say project management is a change management process. It was actually born from an IT gc. Like it's a, it's a controls process, right? It's how do we ensure the controlled delivery of technology. Right. In a controlled fashion, in a risk managed fashion. That was sort of at least my introduction into the space. You know, you know, 15, 17 years ago, project management was, was really you know, kind of anchored on that being a Very risk managed risk, you know, risk focused delivery approach. But as this evolution has happened, you know, we talk about outputs and outcomes. You have to balance process and risk and governance, dare I say the word governance with speed, agility, value, you know, value, acceleration, innovation. So how do you balance those two things? That's always been something that you know, it's a, it depends, you know, it depends on the company that you're in, the risk appetite there, there's pace of innovation, their pace for, you know, of growth. But that has always been a fundamental balancing act that PMO leaders have had to play. And so I think it's no different. That kind of, that balancing act is table stakes for the nature of my role. But how it shows up, right, what level of process, what level of governance versus you know, how quickly are we trying to deliver this thing? And you know, it, it, it, that's where I think it varies based on what you're trying to little to deliver. And I would say with AI because it's moving so quickly, like the old frameworks for governance don't really shape up anymore. Right? Like I'll give you an example. In the PMO space we build intake and prioritization frameworks where we're looking at, you know, value and complexity and how long is it going to take to deliver this thing? Just the, the time it takes to ask those five questions. You can spin up a solution with AI. So you know, we, I will say the AI agent sprawl. AI sprawl is a real thing. There is such a thing as AI governance. Now is it going to look the same way as traditional program governance that you know, we've typically overlaid on some of those, you know, historical technology solutions that we've delivered? No, but I think that was, you know, again, not an unexpected hurdle, just something that we are living the reality of that. Right. And if we don't kind of put some baseline guardrails in place for what, what is considered low code, no code, democratized agent creation that you know, you don't need to run through some intake and prioritization effort and you have the capability, those don't need to go to some central tracker, central, you know, roll up versus what is this other body of enterprise scale solutions, AI powered solutions that we're building that do need to follow because there is security implications, there is, you know, a mass change management effort, whatever the cases may be. You know, I think thinking of those two buckets in that way and thinking about AI governance as you know, how can we ensure again we're delivering in the most efficient and effective way but not losing sight of whatever the balance is needed for, you know, based on the company and risk, appetite and growth pace that we have to, you know, drive towards.
- [28:19]
B
Now this is great conversation and it pointed me to a few things. I'm looking at unexpected hurdles adapting AI and that happens to every new technology. When you go to let's say an EV based vehicles, then you have to have a charging issues. The blockchain came and a lot of fraudulent events started to happen. Right. So every technology innovation will bring that risk and you have to bypass or surpass that hurdle. Now going back to governance and I'm kind of putting all big companies, tech companies on spot back then. I'm 54. We came from a background where the software needs to be really stable and then only you start using it. And then we see Google's and Microsoft's and I want to pick one company here, they started releasing beta versions and if you relate our PMP head or project management head there that's not a good solution because you are having your customer test it out. But it goes in both ways because some customers are hungry so they want to get their hands on it. But most solutions today are released in beta versions. They are not governed. So I would argue on AI side the same way, if those softwares can be released in beta, AI can be released in beta. Now I understand that threat, threat is much bigger and that's why some of the organizations are going into responsible AI. What's your take on that? And that relates to tech sprawl that you were saying before?
- [29:48]
A
Yeah, I mean I think you're 100% correct. A beta is a, whether it's a POC, whether it's a pilot. Right. Like I mean even internally we roll out solutions to a beta group. Right. And that is the group that it's testing from the standpoint of the user experience as opposed to the stability of the platform. So I do think that those two things, Beta doesn't mean put something out into a production use case that hasn't been tested. Right. What beta means is it's beta, right. We're not GA yet. We haven't general purpose this solution. We want to get user. It's really grounded in the user feedback and the user customer journeys that are utilizing that specific, you know, specific solution. How is it actually helping them solve their problem? So I do think maybe thinking about, thinking about beta from that standpoint is important with AI. Like co generation is that's what everybody want you know, everyone's using, wants to get their hands on cloud code and augment and you know, cursor like all these big tools that are out there and they're very powerful. Again, as a non coder I've dabbled with cloud code and I'm like wow, like I can build again digital products, build AI powered things. So again that is becoming a democratized use case. I think it is something that companies are going to have to think about operational rigor. It has to be table stakes, right. We still want to look at things like percentage of defects, post production rollouts. Even if Those rollouts were 90% AI generated, right. There are still basic DevOps style checks and balances in place that we do want to make sure are put in place even with beta versions. So yeah, so that is, you know, security, reliability, durability. I mean these are things that have to continue to be table stakes even with AI powered development.
- [31:38]
B
No, well said. And one of the other point I also wanted to touch base on is that AI comes with. This is the biggest hurdle with AI which is hallucination. Yeah, how, what have you seen, how are you handling it and what do you see in future needs to be fixed or will be fixed that you have a hope for?
- [31:58]
A
Yeah, hallucinations. And this is where it gets very much more technical that the engineers of my team within Mongo are super skilled at dealing with. I will say we saw, at least from an end user standpoint, more hallucinations occurring with some of the earlier models. That's where the right reasoning, the right embedding, the way it's configured really does, does matter. And that's where even with beta you have to test that, right. You have to test these things even before you release something into beta and even within beta. That's where you know, it depends on what those workflows are. Are you building agentic capabilities in that, you know, solution that are really going to completely remove the human from the workflow? If so those, those various scenarios of testing to really test, you know, are those, what are those? How many, what are the hallucinations that we're getting and how are those going to impact, you know, the workflow in its journey? Those have to be really tested. Now if you're the agents are pulling information from various sources to give you an output that might not be actioned on, right. That's still the human that's actioning on that. That might be something you can test in a beta format. Right. Like that you could go before you go ga, get A few users, maybe we did that with some of the internal agent solutions that we've created. Ask it certain questions, I'll go to market one. Like pull up information about a certain account. Is that information the right information or is it hallucinating? Right, so that's the kind of user feedback that we do want to get because those users are the ones that are closest to the knowledge and the accuracy of that knowledge. But it is a very real thing. You know, I. With certain, you know, you see prompt libraries now all over LinkedIn people are still suggesting ways to kind of contain the amount of hallucinations that you can get. I do think that a lot of the more the latest models, like with Opus for example, I haven't really had that problem. It's become, you know, I've stored all my memory there and really, of course you have to review everything that an AI is generating for you and it's been pretty spot on. So I've, knock on wood, I've been, I've been. Haven't seen it as much there. But there are things that you can do in prompting that can contain the level of drift, if you will, with the data output.
- [34:07]
B
Yeah, no, that's very true. And one of the thing I remembered or what is happening, I spend about 30% of my time learning and where AI is heading and where the world is heading in technology, what you see a big shift today. So there are three things. One is prompt, prompt engineering. Second, which people are discounting a lot more is context engineering. And AI hallucinates a lot more because it's missing context because it only has a million token, the best model. The other ones only add like about 300, 400k in terms of token. And token are loosely coupled like words. Right. So only those many number of words it will remember. So then people started to evaluate and I built my own second brain using Obsidian. A story for another day. But it started to remember everything, but now that everything's still too much. So even that needs to be summarized. And when you summarize things, things gonna miss out, right? So context engineering is one thing I see as a big challenge and it is being solved. The third one I see, which is the biggest of all and Elon Musk of the world and others are trying to solve it as well. Infrastructure. I really see AI itself as an infrastructure. So recently I saw innovation called tiny AI, not a plug to them. I mean I'm still curious to see what they're doing, but it's like a small hard drive and it has all the AI models of the world sitting in it and you don't have to pay for it. It's really powerful and you can plug into your Mac or any device. And so you are bringing your own AI. So that's an AI as an infrastructure. I see that as a big shift, right? That's what is coming in AI. The third that is coming. So along with context I should have said that is memory. So memory management is a big deal. These are the three things which is coming that I see. And there are certain evolution or evolution or innovation that is happening. So you see openclaw has been talked about. Nvidia jumped onto it. So they created Nemo Claw. Then Claude code has started producing a lot more features such as remote dispatch and some of the other features. This is what I see coming. What do you see coming or what do you want to see it coming also is the question here. So from your own vantage point.
- [36:42]
A
Yeah, no, I think that's a great question. Just the memory aspect alone I think is huge. Like I will be the first to say, even in my current workflows, prior to Claude being we do have, you know, Claude now rolled out, you know, across the enterprise for engineering at Mongo. But on a personal standpoint too, like I had a running G B T from when I was using ChatGPT. I had a running chat because that's how it had quote unquote context or memory, right. But then I hit my chat limit. So then I would copy paste that chat and go to another chat, right. And then when I created my own personal cloud account, I copy pasted all of that and I stored it as memory. So I had to kind of do my own like stitching to create a starting point of my, my helping my brain, my brain partner, which is now what I, what I do use is primarily Claude. But yeah, so I do feel like exactly what you said, the context, creating context as infrastructure, that is your starting point. How do you create the right starting point? So you're hitting all the things that you said. You're minimizing hallucinations, you're starting from the right vantage point. It's indexing on the right priorities too. Because what I've also noticed, again, just from a personal, individual user, I'm giving it certain information and then when I need to need help writing an output for something, it's not wrong, but it's prioritizing data points that I would have never from the indexing was. It was indexing on things that I would not have considered pulling into Whether it was a report, a memo or an update, it's because it didn't have the full context. Right. So I 100% agree. That is a place where I think there's huge opportunity in all of the models. Right. Is a. We were starting it like with certain perplexity and things like that, where you can, you know, has access to all the models. That's great. But using that to actually build a context layer infrastructure base that starts. That is your starting point for, you know, it has the memory, it has the full context, so you can truly use it as a comprehensive brain partner for your individual workflows as opposed to trying to stitch together the way. The way I did.
- [38:46]
B
Yeah, no, that this is a great solution and that's a good tip for the listeners as well, that you need to build your own context. And I'm building in Obsidian. But you don't have to. You can simply put it in a Google Drive, Word document or something like that, but maintain it in AI. You can prompt it out so that whatever you talk to AI, it can start saving it as a transcript to it or you can copy paste either way and then keep feeding back. So that. There's a great example I use with my team also is that if what is context? Right. So when you go to someone's house and you say, they'll say what do you want to eat? And you'll say anything. But then you have lactose intolerance, you're a vegetarian, you have certain other things. You never gave any context there. And now you're expecting someone to guess your mind. That's what the prompt is. You're giving one line to the prompt and you expect that LLM model will guess it. It will guess it, but it has millions and billions and even trillions of data points for it and it will pick up something randomly that might apply to you or maybe which is majorly being used on an average. And that's the hallucination in a nutshell.
- [39:57]
A
Exactly. And I thought that's such a great example because you, you just talked about, oh, the lactose intolerant. I'm vegetarian. These are binary zeros and ones. But how about what have you eaten for the last 10 days?
- [40:08]
B
Are so true.
- [40:09]
A
Yeah, like these. That contextual historical trend analysis is also part of that. It should be part of the context in terms of what it's going to recommend to you to eat next. And yeah, I would say on that we're not using it as. At least I'm not using it as much anymore. But on the we were talking about projects and kind of the project tracking Notebook LM for a long time was, was a helpful.
- [40:30]
B
I love that. Yeah, I love that.
- [40:32]
A
G sheets and G drive all the way. So when you can. Instead of creating project folders with all the things like just creating notebooks was super helpful because then I could ask it exactly like it had targeted context because we would use it for specific programs and specific strategic initiatives. So throw everything associated with that initiative into a notebook LLM and make sure all of your artifacts are there so that it did have very targeted specific context. But yeah, I think you know, it still required the steps of going and creating the notebook, ensuring that you're connecting all your data sources. I think the mass like at least again on a personal level like here's my Google Drive, here's my email, here's my calendar, here's all the chat history and all of the things that we've talked about. And that's really like you said, the infrastructure layer of how to set context. If we can really optimize that then it's just going to, it's going to bolster the effect, the productivity that you know, gains that we are able to get from, from these tools.
- [41:27]
B
Great. No, I think you keep your eye on, on the things that. How it is saving the world or helping the world, whatever we want to call it. What would you say to the CIOs to prepare for now? I'm not even talking about future, but now because they might be lagging behind from where we are today. What should they be doing? And especially these are skeptics, right? So some people straight out reject AI and rejection on anything is not good. Like it's like earthquake is coming and say nothing will happen in last 10 years or 20 years. Nothing will happen but it could destroy. So what would you do to prepare it? Similar thing for CIOs. It's a storm and earthquake coming to you already here. What should they be doing to get started?
- [42:14]
A
Definitely. If you haven't already have an AI strategy, it's important. This is not something to sit back and see how it evolves. You need to have an AI strategy. You need to think about agentic infrastructure. Is that something you want to build in house? You want to go with one of the hyperscalers, stitch together a number of point solutions like these. Architectural considerations need to be factored into your AI strategy because that is going to inform buy versus build decisions. I mean everything AI is out there. Every company is, you know, kind of trying to, you know, hit the gold rush in terms of the, you know, AI products available to enterprises to drive efficiencies in their, you know, in their ecosystem. Of course, the number one question that every CIO and every tech leader is asking is how are you using AI to, you know, help the enterprise? How are you using AI to help the enterprise? Now having that strategy defined, having that, you know, those technical architecture decision points defined up front is going that can inform every buy, every build, every, you know, solution creation, every project or initiative, AI powered initiative that gets prioritized. The fundamental of it is having that AI strategy aligned to both with, of course, with the, at the CIO and you know, technical leadership level, but even with the rest of the, you know, the C suite. And so that's what we did at, you know, Mongo. The first couple of years, I would say were definitely innovation center. Let's, you know, get our hands wet, figure this thing out, build the Mongo gbt, build, you know, other things without really a clear strategy. And this year, I feel especially with our new cio, you know, we have a very clear strategy of how do we want to propel AI across the enterprise. And so I think that's really the most important thing. Think about AI governance. What does governance actually mean to your organization? Again, based on those things that we talked about, risk versus value, innovation and speed. How do we want to balance those two things? And having some semblance of doesn't have to be these bright, shiny, complex, robust frameworks. It's just a lightweight thing that you're going to use to assess power to mit, you know, AI initiatives and what your architectural approach is going to be to solving those, you know, those problems and building those AI solutions. Think about that now. Define that now because it's, it's very quickly going to become too late.
- [44:21]
B
Yeah, it's on your head already and you just have to deal with it. Whether deal with adaption or deal with resistance, it's up to you. It's going to hit you very hard. But AI is a friend is what people should take out of this discussion right now. One of the thing I also want to pick up on, you just mentioned that your organization is using cloud code. We are using it in a very siloed way because we are a small consulting shop. So I'm a little bit curious. Every strategy, strategy has few components. You have strategy, you have strategic initiatives, then you have phases, projects, workflows, and then you have measurement KPIs, how are you measuring or how the organization is measuring cloud Code performing and comparing because that's what most CIOs and CTOs would do. They will compare with the old world. This is how our developers were performing. This is the KPI we had. What's the difference? Are you noticing anything? Is there anything in place that is being giving you some differentiators.
- [45:24]
A
So we are in the maturity evolution. So I will say that we're building out what Some of those KPIs should be at the BA at the minimum. Baseline percentage of AI assisted PRs is something that we are going. We are starting to track now. Right. Historically we really didn't have that how much of our code is generated purely from AI. And then what are those delivery like what is the cycle time? How does the cycle time compare to what it was prior? So these are. We're building, we're in the process of especially now re establishing our engineering productivity, developer productivity metrics for success, our operational excellence metrics for success. And so these are some of the metrics that we're tracking. They I think the story is still unfolding. Right. Like time is now evolving to see how is this trend over time occurring? Are we seeing improvements in cycle time as a result of uptick in cogeneration? You know, code, code generated from AI? Are we seeing the complimentary, you know, tread down in actual cycle time? Those two things, you know, those insights are what we're working towards now.
- [46:31]
B
Okay, now that's again a great insight and I love this conversation. I know it will keep going into a lot more details if we keep unfolding. But two questions I don't want to miss which is coming to your own personal side. And you know, for me three things are very important. Leading with empathy, motivations and technology or solving business problems using technology. We talked a lot about it. Leadership. We talked about it. But then there are some personal side that you mentioned. The children's book story. I have a 15 year old and he tried a lot of different things. He also uses AI. I actually have a new company with him called Admit. Sure. So while he's preparing for getting into different colleges, I said why not you start building. So we got funded by Google and he started seeing those powers. He got trained a little bit and he saw goods and bads of AI and he can kind of coach to the people now. So I'm pretty proud of that. You have a similar story there. Let's talk about it. The children's book story you had.
- [47:40]
A
Yeah, so I. So my husband and I both work in tech, so we are very tech forward. I think in this house when we got our Google, Gemini was first introduced. Both I have a five year old and a seven year old, both girls, they were having such a blast with asking Gemini build me a picture of a princess riding a banana holding a hot dog and it would most random things so we would have some fun with that. My 7 year old is in second grade, they have Chromebooks books this year. So inherently the world is different, right? When we were growing up, we, my teachers wrote on chalkboards, we had notebooks and pencils and nothing. Right. We didn't even have cell phones in the house at that point. So it is inherently a vastly different world. We are very conscious about how we allow technology into their hands. I mean, simply put, we don't even let them use their iPads or watch TV during the week. So there is that. We still are very, I would say controlled and conservative when it comes to screen time and things like that. That being said, there is no reason why a 7 year old can't enjoy the power of AI. And so the example that we were talking about the other day is my seven year old, she's a wonderful writer. She loves to write, she loves to read, she's super creative. She writes her own little books. And so over Christmas break she had written a book, a children's book. She even drew all the pictures for it. The story is about little two moons, a mommy moon and a baby moon eating, they make a star cake. It was super, very sweet story. She drew all the pictures for it. We sat together, we uploaded her pictures and her story into Gemini and asked it to basically create, use her images as a starting point but create them into children's book ready images. Keep the same style of the pictures and kind of round out I would say the language and just from a grammatical standpoint and if there's missing pieces from the sentencing structure, you know, fill those in. We did that together. I got her comfortable with giving it a prompt. She's also learning how to type in school. So, you know, prompting, you know, Gemini specifically on, you know, I want to add this, I want to add that. And we built a children's book for her and I'm in the process of getting it stitched together and printed and you know, we'll read it every night before bed. And so yes, you know, one could argue, oh well, is that, you know, hindering creativity? And I say absolutely not. That is empowering her to think beyond the bounds of what, you know, she's able to put pen to paper on and giving her the tools to be able to prompt and to you know, kind of find the way to take her her own creation and, and create it. You know, a product, an actual physical product out of it is the journey that I'm helping to, you know, trying to instill in her at, at this age. But yeah, so I think, you know, it's here to stay. I think AI using AI responsibly is super important. And again, even for the children, you know, like this is the, I think the extent of what we want to be able to showcase to her at, you know, have her, you know, develop her own hopefully passion for technology the way her parents have.
- [50:40]
B
So yeah, no, no, that is great. And you know, you're encouraging that. That's really good. Good example. I also use about technology. So if you're a furniture maker, you use advanced tools that does not take away or scale your creativity. Same as when you use calculator when we're studying. You know, you could do calculation on paper, fine, but that's a repeatable task that you could let a calculator do it. Same thing happened with computers. So you have adopted technology all throughout. Why resistance on AI? It's the same thing. It'll bring a good and bad in everything. But if you adopt a good thing and you look at the good things, it's going to help you. And I definitely encourage kids. I in fact have a movement for disabled kids where I'm providing free education on data and I have a school community on that too. And I really want to encourage everyone to learn AI learn with the caution that it's going to bring a lot of risk to you. So like anything else, you use a digital card online, putting on Amazon, same risk you have, but in a different scale and proportion.
- [51:48]
A
So you need to be about those things and know how to respond if and when a risk gets triggered as opposed to ignoring it and acting like it's not going to happen.
- [51:58]
B
Yeah, and everyone needs to learn and educate themselves and educate others. And that's the key for any technology, including AI. One closing thought and question and this I ask everyone, what would you tell your past self? You know, your 16 year or 18 year or 21, whenever you were there. And I have a lot of story about myself too. I was, you know, my father used to pick me up, put me in school. So I got my wings when I started wearing braces. So I didn't want you to study so much. I was good in studies, but I wanted to travel. I went into music and I started working in different places. But one thing I missed at that time is I was not too reflective and I was ignoring what is happening around me and not enjoying. So I tell my past self that I should, should be enjoying that journey that has happening. A lot of great things has happened but I kind of keep moving on. So what would be your thing?
- [52:58]
A
That's such a great question, Dave. Thanks for asking it. I think always trust your gut and trust. Have trust and faith in your intuition is what I probably would have told myself because you know, when I. And I won't share my age, but I'm well,
- [53:18]
B
well in any case I'm older so that we'll settle down to that.
- [53:21]
A
Yeah, but it's, I remember it like it's yesterday almost. That's, that's the irony of the whole situation. It was not anytime recent but I do remember it as if it was yesterday. And just with this like we're talking about AI and technology, you know, my husband and I, we were recently saying too we are the last generation that remembers a life, life before all of this, like Internet. We had a TV in our house. That was it. And it was this big boxy thing and I had a big, you know, computer. No when I grew up as when I was my daughter's age. No iPads, no cell phones, no any of that. So. And we are the last generation I think to like have that, preserve that memory. So by the time I was 16, of course I think I had a very big looking cell phone. But, but yeah, I think it's your. We're such sponges for knowledge and that's a good thing. But I do feel like just reminding myself that, you know, your intuition even, and especially with AI coming in and all of these data is an abundance today and it's quite difficult to navigate through all of it and know who you are and know what it is you want. We were talking about the other day, I'm getting influenced, clicked on maybe one AI arbitrage ad on my Instagram and now I'm getting 15 of them. Are your AI consulting business star, you know, AI arbitrage business. So you know, yes, that, that, that, that is happening. We're all getting influenced by, you know, the, the, the data around us and the channels around us and those are so much more relevant today than they were, you know, when I, when we were, you know, 16 years old. So I think, you know, just the what I would have told myself back then, what I will tell my children, you know, because they are kind of the mirror of what I wish I could have told my my you know, 16 year old self is just trust your intuition lead, you know, really truly believe in your, you know it's, it's, it's almost like manifestation mindset. Like think about your gut is what's going to tell you what your true path is and always trust that intuition. Lead that path with intention. Don't just let the world influence you and bounce you around without really having asking yourself your question intentionally. What is the path that I want to go on in my life and
- [55:29]
B
just trusting that no this is so beautiful. And I also relate to that and I just thought these are forums that I look at so my doom scrolling is about about mindset, motivation, music. And there was one more amp so there are four amps that I constantly look at and see. So I'm welcoming all that AI sending those stuff to music right. So it's sending it to me and I love seeing those things and you know, I love connecting with people. So thank you so much for connecting with me. You have been incredible and where can people find you? I know you're on LinkedIn but is there where people should look at and find you? And I'll provide a link to people on my blog if that's okay.
- [56:14]
A
That would be wonderful. I love you know, similar to you, like I'm, I'm just excited to be in a space that is. Has so much opportunity for learning, you know. Thank you for reaching out and connecting this. I'm so happy that we, you know, we, we were able to meet and we have each other in our network now and so yes, I'm available on LinkedIn. Feel free to share the profile. I started writing recently as well, so I have a substack and I've just been sharing insights that I'm passionate about and I'm really, I think entering a phase of my growth and my professional journey of continued learning. And yes, I want to continue learning from both experts, novices, people that are in the journey with me, whether it's AI. I have a very strong passion for music as well. So we share that in common. But yes, thank you for this wonderful dialogue and just looking forward to staying connected.
- [57:05]
B
Thank you. Priya. That's Priya Udeshi, Chief of staff to the CIO and head of IT PMO@ MongoDB. If you're thinking about AI strategy in the enterprise, that is someone you want to follow. She just mentioned substack, so do go ahead and follow her. Have a great day.
- [57:22]
A
Thank you.
- [57:24]
B
You have been listening to Think AI podcast with Dave. Take one idea from this episode and turn it into action.