A (3:22)

What is different is with generative AI it has actually pushed AI in the front, in front of every user experience or every consumerization I would say or democratization I would say that's a significant change. Right before ChatGPT, many people don't know there is like a lot of MLDL in products. I mean Google has been using it for decades. From the time they were on there is AI in their product. Right? I mean but that's different. It is a few that was doing versus now it's democratized. People are experiencing it in a different way. That's the biggest difference. So if I take that to enterprise and you are aware that here we are talking about, you know, every employee in Palo Alto has their agent, which is the Panda AI. We don't have any more. You need something who to go to. You go to Panda. If Panda doesn't know what to do, it will drive you to the right person. So it kind of changed the experience of do I go to IT to open a ticket or do I go to HR to open a ticket or do I go to finance to open a ticket too? No, you go and ask Panda if you don't know anything, it will guide you. And in it, 72% of my requests are fully automated before I put Panda in place. We are about 12% automation. Over less than a year we have gone from 12 to 72%. It's not all AI, it's a combination of AI changing the user experience, people getting comfortable to go to Pandas the first place, then really the traditional way of thinking about your process re engineering and automation. Now with Agentic, those tools we built are available to the AI agent and it is actually our Panda version two is using them as tools for the agents as well.

B (5:10)

You know Mira, we used to say let me Google that for you. But internally we've started saying let me Panda that for you. I suppose we should call it Let me Panda AI that But nonetheless, can you walk us through how those internal AI agents like Panda AI are transforming the employee experience? You spoke about the UX and as a former designer, that's always one of the areas that interests me the most.

A (5:33)

So you know, it starts with understanding the problem. I mean first of all, when we started everybody thought through generative AI it's going to somehow magically solve the problem. But you have to start thinking about really identifying the real problem. And then for me, it's looking at what AI can do well and how you make AI to work for you versus trying to, you know, fit AI to what you do today. I'll give you an example what I mean by that. A lot of times people take AI and say, hey, this is what I do AI you just replicate the process of what I'm doing. Guess what? AI is good at certain things. AI is really good at information summarization. AI is really good at information retrieval. Now with agentic approach, it's actually starting to get okay and better day by day at planning and reasoning as well. But planning and reasoning is still a long way to go. However, if you go from, you know, 12, 18 months back to today, the retrieval and summarization has got really, really good. I mean, even translation has got multimodal has got better. So when it comes to information treatment, it has come a long way over the last two years. It has got really good. The agent. It's in the beginning of the journey. So when we started with Panda, we looked at all the ticket that comes to it and finance and hr. But I'll talk about it for a second. We had about 280,000 tickets a year and we are only 20,000 employees. Don't ask me, but people really like opening a lot of tickets, I guess. So we had that many number of tickets and we literally analyzed every single ticket and we put them into three buckets saying, hey, first bucket is information retrieval and information. People are asking how to kind of a question or how to kind of a ticket. The second is they're asking for some request. But these are deterministic outcomes. There's a request and there's a deterministic outcome for it. I call them as. These are service request kind of a thing. You are asking for a service and you're getting provisioned at that particular service. It could be an access, it could be a password reset. It could be somebody asking for extension of something, somebody asking for a certain specific data that they don't have. But these are very specific requests that has a specific deterministic action. Then you have something telling, oh, my laptop is broken in this case. Or somebody saying, hey, my financial data report doesn't look right in this case. Actually, it's a little bit of a break fix. You have to do a little bit of troubleshooting. That's a third part. Hottest category. I would say today we are really good at information gathering. That's about 20% of my tickets. I would probably say 19 and a half percent don't come to human anymore. Only it comes when there's no knowledge available or in fact we have also put now LLM as a judge in Panda version 2. So it actually immediately flags me saying, hey, I'm not getting data from this. However, I'm making this up, making meaning like I'm taking from all the other places and providing this input. You are the owner. Look at why this query is. I'm not having the knowledge. It actually has the ability to tell the owner that it's missing a knowledge. The second category, you know when a service request comes and the deterministic action, we did a very good job over there. We are 89% automated today of the request that comes to us that is actually more deterministic action. The real hard problem is non deterministic action are things that are actually break fixes. You have to do some troubleshooting. Where we are with that is we are looking at repetitive patterns. And I always tell bad code cannot be fixed by AI. If you have a quality issue, it's going to show up. You have to go and fix the defect.

B (9:14)

Samira, as you're talking about that UX and you describe this idea of going through a ticket system, is there both a process where you're expecting the agents to get better quickly, but you also need the humans to stop thinking about problem solving in the old way. Because now we've got this new paradigm where the, the systems are inherently smarter, more capable. And so your level of request maybe not quite a ticket is getting more complex or a little bit harder to say is my computer doesn't log in or I can't get into the system or I want to have access to that software. Is it like are the humans asking bigger, harder things of you?

A (9:55)

Yeah, actually here in this case, you know, when I talk about user experience, David, there's a couple of things, right? One is actually first of all, the user is not actually anymore calling a number and waiting on they're getting an instant response, right? It's actually an instant response of it's telling you yes, I can solve it or it is actually guiding you to the right place. But it's also not always you have to be scripted, right? Like let's say that if you say hey, my laptop is having problem, it'll give you a set of, you know, dialogue oriented things to diagnose better. And I'm also capturing typically in a break fix or if there's any problem in the past, the amount of time we spent in solving there's A significant amount going back and forth. I'm setting something to you playing telephone, right. With AI now I'm able to gather all the information even when it is not able to solve upfront and also not asking the same thing again and again. Number one, that changes the user experience because every interaction is moving it forward. Not like just asking for clarification because it captured all the things. It also has the memory of everything that's happening. That also helps if there's a human who needs to get involved to look at all the things that has happened. It gives me the history. Number two. Number three, it's also having an opportunity because the humans were never good at saying hey, can you tell me what's the feedback I could have done better? Or if I'm not solved, what is the thing that I missed? They never asked the question because then nobody wants to know that they didn't do a good job. Especially when we had third party vendors who are doing this. It's actually not in their own interest to ask those questions. Whereas Panda, if you're saying you're not, he's not right or she's not right, in this case it actually takes that input and it's becoming a learning experience, a continuous learning experience. To me those are some of the key, key differences from a user point of view, how the experience has changed and how they have become self sufficient and they can do it in their own time too. You don't need to be waiting for when it is going to be available to solve the problem, but it's on their own schedule.

B (12:05)

Yeah, it does seem like it's a very different experience than you would have had just a few years ago. Going in and putting in a ticket, hoping you put in all the right information and and then realizing when you did get that chance to either type back and forth or talk to somebody in that help position that you'd missed a bunch of the context or you've provided them information they didn't really care about to solve your problem.

A (12:29)

Exactly.

B (12:30)

That is my experience by the way. Rolling faster and being able to have that conversation back and forth. It's pretty wild sometimes you said he or she. It does feel like it has a level of personality and a person behind the interface. So I enjoy that side of it. Let's shift gears a little bit and talk about developer productivity. How is generative AI changing the way your engineering teams work?

A (13:01)

We started as engineering software developers productivity but over the last few months we have literally shifted this into AI driven SDLC or plc. You can call product lifecycle, right? I mean, if you gotta think about an engineer's job, the Engineer spends probably 20, 25, maybe 30% of their time coding. And that's actually the place where there was a lot of conversation in the market around, hey, you don't need software engineers because AI can write all the code. Guess what? It's only 30% of the time they're spending time in coding. The other 70%, they're doing design work, they're doing documentation work, they're helping with support and defect fixes, sustenance, engineering, you name it, meetings, you name it, right? And if you kind of think about a significant. When I look at like where the gaps in the product when it gets shipped, there is three areas where the product really has gaps or trouble. One, a defect that actually is coming because we haven't understood the user requirements, right, which is actually upfront process and the product gets shipped. Then you're kind of going like, well, the user says, that's not what I was thinking. That's not working for me. Two, there's a design gap that you got the requirement, right, but there was actually a technical design that actually led to a problem that doesn't get discovered until you are really in the design gap. It doesn't matter even whether AI generates a code or human rights are code. If you have a design gap, you're going to have issue regardless. The last but not least is where the coding issues are. So if you kind of think about out of the three, the top two issues are the most expensive issues. The gap in user requirement or product requirement is a bigger issue. Design gap is the next big issue and then comes real code issue. If you kind of focus all your effort on the AI writing code for you, you're only solving a small portion. Also remember, when you're building a product, there are other personalities who are involved here. There's a product manager, there is analyst, there's QA person, there is software developer, all these parties. And then you have your, you know, you can call it your users of the product or business. In this case, I have a lot of my users or internal users. So they're actually doing user acceptance testing, whether it's finance team or sales team or whatever it is, right? Because I'm building internal products, not just external products, right? So of course, yeah, when you think about it, I have to really, if I wanted to get a full benefit of it and AI has ability to certain things, I have to rethink about my SDLC process rather than just AI, you know, delivering code. Of course we have cursor. All my software engineers are using cursor as a coding assistant. They're using it and it does a phenomenal job. When it is a greenfield development, if there's a new product development, it does amazing job. But you have so much of product code already. It has to be brownfield. It has to be in a position to do it in a way that it will be able to do it in the brownfield. So brownfield it requires a lot more knowledge. What is this product is about, what does this technical design is about? So it can be, it doesn't give like a, you know, 20%, 30% efficiency. In a, in a green field I was able to get 70 to 80, 85% efficiency. But in brownfield we were not able to get good efficiency. So what we have done is we have reimagined the process. Today we have finished seven pilots, seven programs under this or seven projects under this. And it was a pretty good success. We learned out of this. Now we are going to go GA on every single work it will do moving forward. Is a AI driven sdlc. When it comes to requirements, we are sitting down with the business. Earlier we used to tell the business like hey, give you a requirement, some human has to record. That's where you get into the gaps. Today it's actually a zoom recording. It is actually all the documentation, all the problem statements they have thrown around emails, et cetera. We feed it into AI to generate our PRD in a certain way. Sam.

B (17:48)

You basically take the conversations that are going on in real time across that group of people or those different teams. Because I don't think it's just four people, you're pulling that all in and then that automatically generates something for the software developer or the product owner to react to and go like has this accurately captured what we're trying to do? And you can go back to that problem and really go like, are we solving the right problem before you do all that expensive work only to discover no, we've quickly written code to a problem that nobody cares about, nobody wants solved, or we solved it in a wrong way.

A (18:25)

You hit the nail on the head, David. Not only that, at times taking that problem, we will produce AI driven mockups right away and show it to them saying hey, this is the problem, this is what I'm capturing to think like it's a conceptual solution and take real time feedback. And a lot of times what we are finding is actually the requirement. So that's the direction we are Moving towards number two. Then we are using that to generate our user stories. Because now I can generate, because I have AI has done a much more complete documentation, I can generate a better quality user story that also is feeder into my test plan for QA as well, my quality assurance team as well. And since I have a better quality user story, if it's a greenfield development, I can literally feed the user story to AI to generate the code of my choice too, which it does a pretty good job. If not, the software engineer still has a very good knowledge to guide AI to get the right kind of coding that needs to be done. So it just is. It's like it requires my product managers, my analyst and my software engineer and my QA engineer, all of them to be AI enabled and AI savvy. This is redefining their job. In some cases. It's also the line between, in fact, our product managers and analysts are pretty much now the line is blurred because they both need to do a very similar job.

B (19:50)

So there's this, there's this famous guy, his name's Einstein and he talked about if he had to solve a problem in an hour, he'd spend 55 minutes thinking about it, minutes working on it. And what I'm hearing from you is that your teams have a third of their time writing code. But you can accelerate that, especially in that greenfield area. But by having the ability to capture the requirements, understand the problem, test out the ideas with generated ui, those types of things that allow somebody to think deeper, think faster, you can get more out of Your quote unquote 55 minutes of the hour doing the really expensive and hard things with AI and in a way that you've never been able to do that before, such that when you spend the third of the time or the 55 minutes in the analogy with Einstein, you get these accelerated outcomes that are on point. That's amazing.

A (20:43)

Absolutely, absolutely. And then the other thing we are doing is as the products are getting delivered, we are making sure because we have a good sense of documentation delivered throughout the process. Now that documentation could be used to automatically update my product documentation as well. So my product always stays fresh.

B (21:01)

Samira, you're talking about this fast paced innovation, the ability to roll out capabilities and get efficiencies that having spent your entire career in the IT space, it's got to feel unbelievable. But at the same time, we're here on threatvector security podcast. I'm curious, can you talk about the security challenges that maybe your most concerned about? And then, you know, for Our audience, what are some of the big blind spots that CIOs and CISOs that aren't at Palo Alto might be, you know, that they should really start to consider, you know, in this space where innovation and speed are coming at us so.

A (21:48)

Quickly, I think, I mean, you hit the nail on the head because first of all, I'm a first customer of Security products of Palo Alto, and some of the challenges we faced internally eventually became the very fast launch of products itself. To start with, anything, you need to have visibility, right? So if you're actually starting on your journey, if you think, oh, we don't use AI, guess what, your teams are using AI. Maybe you didn't know what they're using. So your first and foremost is get immediate visibility into the use of AI in the company. And that is, to us, the AI access security. You're probably aware of that. When we actually opened up everybody to be embracing AI in early 2023, right after ChatGPT was launched in November 2022, the very first product that myself, Lee, and we talked about was, hey, we need access visibility, or we need visibility of what is the use of AI in the company. That's how Access Visibility became the very first product. And we were in production for more than, I would say, close to two years now or from the time that product was a beta, right? So long story short, start with visibility because that's important. That's how you know who's doing what, what kind of rules from a data protection point of view, you want to launch, et cetera. The next thing is, I mean, when you're using AI, you pointed out very clearly there are security, security risk that you're talking about and vulnerabilities, new types of threat vectors you have opened up here. I know when you're using AI, the biggest threat vector is your model and your data. I mean, for AI, your data and your model are the brain combined together, what data it feeds and how the model is processing. It's like think about as neurons and the signals that are getting given to that neurons, and if I poison either one of them, AI can go haywire. So how are you going to protect your model? How are you going to ensure your model is actually the right model? Because today so many. I mean, of course when you talk about your frontier model, like a Gemini or OpenAI, you don't have to. I'm not saying don't worry about it, but that's actually a lesser risky thing for you because these are like frontier models. But if you take any AI application or any AI product. They'll have a number of models beyond just that one model. Even though the one model is powerful, it's not actually always conducive. I'll give you a good example. When we look at long documents, how we parse the long documents, Google Gemini can do a very good job, don't get me wrong. But it takes forever and it's expensive to do that. And I need a very specific job of just parse my long document which has tables, pictures, words into different form and give me a parsing. I want it in sub seconds. There is a small language model, an SLM that can do that only one job, but it will do that one job really, really well. Right. So I have that model along with Gemini in one of our products, IT products. And that SLM model is something that we have to host it internally and I'm pulling it out of hugging face or whatever it is, or it is actually a third party model. But still there's more than one model that you need to think about. So model scanning and making sure it's a good model, it has not been seeded or bugged and then the poisoning is not happening. That you need to know the security posture of the model itself matters. So we are not giving excessive permission to the model for doing things that it should be doing. How you give data access, that's another layer to think about. And then of course during runtime your prompting is like your code. Now LLM has become your operating system and your prompts have become the software code. So that has to be protected like a software supply chain. So that requires a runtime security as well. So we do use actually access for visibility and runtime security for anything from all the way from code to production.

B (25:54)

So you're talking about champagning the product for us before we ship it out into the market. Yeah, not too long ago I was at a conference talking about AI security and the workflow for that industry. And I think the thing that surprised people was that some of these open models or open source models that they could just pull down weren't secured. And it was like, well, you're borrowing code from somebody else, you have no idea what's going on.

A (26:16)

Yeah, it's open source. When we're open source is secure.

B (26:19)

Yeah, it was just that like light bulb moment and it gave me a little bit of indigestion where I was like, oh no, this group is going and playing with things that they don't really know where it came from. I don't Understand if it's, you know, had some sort of a malicious inject or, you know, maybe it's completely fine, but they were finding value in it and then integrating it into more and more of what they were working on. So I think some of those eager teams out there that aren't thinking and being thoughtful, some of the blind spots might be in how do you consume different models and protect yourself, protect your data. And it gives me great comfort to come back home here to HQ and pan and know that you're taking care of that for us and our teams here. Right. I'm not one of the folks that's having that face of shock during a conference. Mira, I know that your time is limited here, but I'm hoping you can recap for our listeners. What's the most important thing that you want them to take away from our conversation today?

A (27:24)

I think for me the biggest thing is AI is a tectonic shift and it's not a hype, it's here to stay. It's going to redefine the way we, it's not just tech industry, it's going to be redefined in my opinion, the way we live, we play and we experience life in the coming decades and centuries. Right. So it's like a moment of really a pivotal moment for how things are going to change. I mean there are theories about how fast, how change. It's exactly whatever is coming but it has started, the journey has started. And with any technology that you bring there's opportunities and threats. So know your threats. Security is super important. Security is much easy if you integrate from the get go as a design principle rather than trying to use it like a seasoning at the end of preparation of that food, it is not going to taste well. So think security integrated from the get go which means when you are dealing with your customers, partners, you know, make them realize what are the threats are with AI as much as it's a powerful tool, how can we help them to protect number three? For me, what is the other thing is Alto Products, I'm the first customer so if anybody wants to know about how we are using, happy to show what we are.

B (28:41)

Well, you heard it here folks. It's been 30 minutes that we are together. So AI has drastically changed in just this recording. And with that, Mira, thank you. I appreciate you coming on threatvector again and talking to us about this fast paced world that we're in where you're innovating as fast as AI can go and then making sure that we stay secure.

A (29:03)

Thank you, David. Really appreciate it. Foreign.

B (29:19)

That'S it for today. If you like what you've heard, please subscribe wherever you listen and leave us a review on Apple Podcast or Spotify. Those reviews and your feedback really do help me understand what you want to hear about. If you want to contact me directly about the show, email me at threat Vector Palo Alto networks.com I want to thank our executive producer Michael Heller, our content and production teams, which include include Kenny Miller, Joe Benecourt and Virginia Tran. Original mix and music by Elliot Peltzman. We'll be back next week. Until then, stay secure, stay vigilant. Goodbye for now.

A (30:06)

SA.