← The AI in Business Podcast
The AI in Business Podcast
Scaling Customer Experience with Operationalized Agentic AI - with Shezan Kazi of Dialpad
00:29:35
Loading summary
Transcript47 lines
- [00:00]
Shazam Kazi
Foreign.
- [00:12]
Podcast Host
Welcome everyone to the Emerge AI and Business Podcast. Today's guest is Shizam Kazi, Head of AI Transformation and AI products at Dialpad. Dialpad provides cloud based communication software that combines voice, video messaging and contact sensitive tools with built in AI for transcription and analytics. It is used used by organizations across industries to manage internal communications and customer interactions. Shazan joins us on today's episode to explain where autonomous systems actually fit inside customer facing operations. Why they should take the first pass on high volume requests and where they must hand off to humans. And how leaders can identify the parts of their process that are genuinely ready for automation. He outlines how organizations assess impact by tracking accuracy, customer satisfaction and the system's ability to follow instructions. And why starting with low risk, high value tasks before expanding to more complex cases is what ultimately drives meaningful gains at scale. Today's episode is sponsored by Dialpad for our Solutions partners. Position your brand alongside the Fortune 500 leaders defining the enterprise AI roadmap for the opportunity to showcase your solution to the executives currently funding and scaling global initiatives. Partner with Emerge. Secure your partnership@go.emerge.com partner that's go.emerj.com
- [01:38]
Marili
now
- [01:38]
Podcast Host
the conversation with Shazan.
- [01:41]
Marili
Shazan, thanks for joining us on the show today.
- [01:43]
Shazam Kazi
Hey Marili, I'm very happy to be here. Thanks for having me.
- [01:46]
Marili
In our last episode we had a great conversation with Greg on why this new way of AI is finally feeling different, why it's mature enough and leaders are seeing the signals where this is the AI automation is making a difference and replacing that old chatbot era. So, so today I'm interested in getting into the real practical side of this. So Shazon, what I would like to hear from you is where does this fit into for CX leaders into their actual workflow?
- [02:14]
Shazam Kazi
So I think the reason why we're seeing this truly being adopted today is because some of the biggest blockers in the past have been trust, right? So CX leaders, any business leader, they were happy to use chatbots like ChatGPT and Gemini for simple tasks, but they would see these models are sometimes hallucinating and asking them to rewrite an email required a whole different level of trust than autonomously exposing them to customers. That has over time improved. So these models have become more accurate. But also the companies of enterprise conversational AI software, the providers of these, of this software, they have realized that having a good looking chatbot that creates reasonably sounding answers is only going to get you halfway there. What you need is you need to work on that trust barrier. You need to show that you actually can provide data driven discovery. So tell the CX leader what they should be automating and you need to be able to prove that it actually works. Because screening an individual conversation is easy, screening 10 conversations is easy, 100 is doable. But when as you get to a thousand conversations and more, and we have customers today that are doing hundreds of thousands of conversations a month, it becomes impossible to do this manually. So you need the right metrics and analytics in place and you need the right data driven discovery tools in place to let these companies know what parts of the business they should automate. And this comes back to your question, where are we seeing this? And that is very customer specific. There is no single recipe. Although the use cases companies come to us with are always the same. It's always, you know, lead qualification after our support password resets, like IT support. And then we tell them, how about we look at your past conversations for the last six months and let's look at what problems got, didn't get solved or where did you have high customer frustration rates because someone had to stay in a queue for too long. And it turns out those use cases are totally different. And I'll give you a funny anecdote because we have a customer in the travel space. They thought the biggest demand would be their customers calling about flight changes. Turns out the cliente is mostly young people between 18 to 25. They were traveling abroad for couple of weeks at a time. They didn't care about flight changes because they're super flexible. The number one reason for calling for them was how do I do my laundry? But that's something nobody ever thought about, right? Because it's, it's not obvious and it probably doesn't apply to 99% of travel agencies. For them it was important and we were able to tell them because instead of guessing, we actually looked at hundreds of thousands of conversations and we had to smell. That was funny.
- [05:45]
Marili
And that makes sense to me that the use cases for each specific customer success workflow, it would look different. And there obviously is low hanging fruit as you mentioned, the passwords, identification, et cetera. If you look at the basic workflow of a customer experience, where in the workflow do we see the AI agents entering and making the difference? I'm supposing saving time and the scale as you mentioned, walk me through those workflows. Where would AgentIQ enter and where would it hand off to humans?
- [06:22]
Shazam Kazi
It should always enter at the start, right? Because you want to have agentic capture the majority of your volume at the very top of the funnel and it's going to do a screening and triage and decide is this something that I can handle? And the time it hands off is usually when knowledge gaps start to surface, workflow complexity exceeds the agent's capabilities, or circumstances or ambiguity arise that require for Newman to step in. Ambiguity could be something which is just plain ambiguous, but it could also be something like a lot of background noise or um, let's talk about a, a voice, a bot, I'll call it a bot, we usually call it an agent. But a voice bot talking to someone and asking for an order number a human would understand. If I'm calling whatever e commerce company and they ask for my order number, I'll have to look it up because I have no idea what my order number is. And that is going to take me some time to find that email. Certain cell phone providers don't allow you to have data usage and cell phone usage at the same time. So now you're struggling to find the order number because you cannot open the email. And that is very frustrating. So instead of having an AI agent pressure you to provide that order number, why don't you just hand it off to a human?
- [07:55]
Marili
And that's speaking to the friction, lowering that friction with your clients and ensuring that they have smooth experience, the easiest experience they could have. So they are obviously starting points where they, there's higher value and if we look across industries, they don't all have the same maturity. So those starting points, you saying it's right at that start with the AI agents make the first difference. And that does make sense because it's the lowest level, just identity, it's not ambiguous order numbers, etc. So walking through that workflow so you've got that first interaction where you sort of just identify, identifying the client and then moving them through the workflow where it gets more complex. How do, how do companies ensure they know where that breaking point is, where it becomes ambiguous? Is it also a continuous learning for the chatbot? And that might shift at some point where it can perform higher level tasks. Is it a continuous learning system?
- [08:54]
Shazam Kazi
To some extent they know because it's a continuous learning process and these models are self optimizing over time. But that doesn't work if you're just getting started on your agentic or AI transformation journey. You actually need to be able to have deterministic rules and algorithms that tell you when to hand something off. And the best way for doing that is actually to establish a confidence score for the AI agent for it to self assess if the answer it's providing is valid or ideally to have a third party oversight model we call ours Guardian, but could be any sort which monitors those conversations in real time and checks is it compliant, is it in line with policy, is the grounding in the knowledge base that's underlying, is that actually on point? And if it isn't, then just hand it off to human. Because one of the realities we're seeing in enterprise customer communications is agents, AI agents, they'll get this right, 60 to 80% depending on complexity of the use case. But still you have customers calling that just they do not want to talk to AI. And God, search from I think it was November showed it was 56% of customer support calls on ready to talk to AI. Now in a lot of markets and countries you have legal requirements to disclose that you're talking to AI, be transparent about it. And that just means one out of two calls are going to ask for a human to take over and not just force AI onto the customer.
- [10:48]
Marili
Ultimately I want to get to that governance part and the trust slightly later. Do you see that there is often that leaders misunderstand where AI fits in or is it pretty clear to organizations where the AI fits in? Or is it something that you still need to educate on where it fits, where it starts, where it ends within the workflow.
- [11:09]
Shazam Kazi
Everybody knows they want to do agentic, right? When we started off with Agentic, we were working on our messaging and marketing pitches and it turns out you don't have to sell it because everyone wants to do it. CEOs wanted, CFOs wanted, the boards wanted, the markets ask for it. But you know what that you want to do it, but you don't know what you need to do. And that's where that data driven discovery process that I talked about comes to play. So the answer is yes to do companies know they should be doing. But the answer is no. They don't exactly know what part of the workflow, what part of the process do I inject? And that is very much an advisory process, a consulting process where you need to understand the business processes. And the last thing you want to do is just point inject an agentic solution legacy process. You actually do want to redefine and redesign that process from scratch to ensure that over time Agentic can take on larger parts of it.
- [12:22]
Marili
And we see that across industries that it really is the responsibility of the organization and the owner of the workflows to remap the workflow. It's not about Sticking the AI on there, it's reimagining what could happen within that workflow and then bringing the agents in where they can make the biggest difference the fastest, with trust involved. So what I'm hearing from you is that for AI agents, it's still very much handling the deterministic parts of the process. And then humans doing the supervisory. And when it gets. Starts getting ambiguous, that's where the handover from AI agents to humans actually lives.
- [13:06]
Shazam Kazi
Yeah. So humans take on a lot of the edge cases, some of the edge cases AI is able to do, and they take on the escalations. Right. And that's the two things that happen. And an escalation can be voluntary. So because someone says, listen, I just don't want to talk to AI, or it can be involuntary, where AI just isn't able to resolve the issue.
- [13:28]
Marili
And do you see with these new systems that that shift you mentioned, it's a pretty big percentage of people that do not want to speak to AI at all. 56% is a pretty large number. So are we starting to see a shift that there is more trust, or is it still one of those points that haven't broken yet?
- [13:48]
Shazam Kazi
We are seeing a shift, but I don't know if it's statistically significant. It's a biased data set that we have. At DelPad, we have a relatively high number of customers that are already using our AI. Around 97% are using AI functionalities of some sort. So that means their customers have already been used to it. Right. If I look at my personal surroundings, I don't see a shift like my wife doesn't want to use. My mom doesn't want to use it. My assistant does not. My brother has.
- [14:23]
Marili
Are you. Are you saying it's a female thing? It seems to be the majority that are struggling with the trust.
- [14:31]
Shazam Kazi
I don't think it is. If you look at the core contributors at Dialpad, it's fairly balanced. Like our science team, our research team, our engineering team, and the product team, Even design. We have about 50. 50. That's just my personal surrounding. I have more females in my family than I have males, so it's.
- [14:52]
Marili
It's more personal. So on that note, you are saying that it's. Although we see that 56% of clients generally do not want to speak to AI, that's not a bottleneck within adoption, within companies, because they see the. The value there. So that's not stopping organizations of actually implementing it just means it's one of the things that you need to consider and Obviously understand that when you are adopting it, there is still that phase where you need to ensure that your clients as well as your company builds that trust. And it's just keeping in mind that they will be that slight educational curve, that trust curve. But what you're seeing is it's not stopping people adopting.
- [15:35]
Shazam Kazi
No, that's not stopping people adopting because it's not a. What's stopping adoption or what's stalling. Adoption is trust. Trust on two fronts. One, are the marketing claims of these agentic AI providers, are they actually true and will they, you know, will they hold true beyond an academic lab condition? And the second one is if I have an agentic AI company and they have their forward deployed engineers and they set this up for me and it works throughout a pilot, what happens once the pilot ends? Who is going to maintain the system? And am I as a company capable of managing and governing and continuously improving that system over time? So those are the two. It's trust and trust in the provider.
- [16:28]
Marili
So speaking about trust and governance, what are the specific analytics that organizations are looking at when they make agentic automation safe and scalable? What specifics can they look at within those analytics to ensure that it's safe and scalable?
- [16:42]
Shazam Kazi
Yeah, so there are ultimately two types of metrics people are looking at, right? One is, of course, everyone talks about roi, how when is this going to pay off? But ROI is very difficult to calculate just because there's so many variables and parameters that flow into it. And ROI becomes a side effect of ultimately csat. Right, Whatever you want to call it, resolution rate, customer satisfaction, the inverse of customer frustration rates. And the group of metrics that you're looking at is how many times out of a hundred is your AI agent doing what you told it to do? So how many, how often does it precisely and correctly follow your instructions? And those instructions, they go beyond just updating a record in a CRM. It's about how able are you, how often are you able to hit the right tone that resonates with the consumer, the one that's calling, how long does it take for you to handle that issue? Because these days most customer issues are urgent. Everything needs to be on demand. You don't want to wait for long. And while we might be able to minimize the number of rings that it takes for someone to pick up the phone, once that phone call gets picked up, you still need to be fast in handling that customer interaction. So average handle time, for example.
- [18:27]
Marili
I'm sorry, so you're saying csat, the customer satisfaction how is that measured? How do you accurately actually measure that?
- [18:37]
Shazam Kazi
So traditionally the way CSAT gets measured is before the call gets picked up, you get a message saying, when this call completes, can you please stay on and complete a short survey?
- [18:50]
Marili
We're used to those questionnaires before and after the call. Is that still the best way to measure it? Is it accurate enough? Is it reliable?
- [18:58]
Shazam Kazi
Well, it's very accurate. It's very reliable. The problem is nobody wants to do those surveys, right?
- [19:04]
Marili
Yes, exactly. So does it represent enough customers to make that a measurement you can trust?
- [19:11]
Shazam Kazi
Absolutely not. But fortunately a lot of companies these days, they have automatically calculated csat. So Dalpad calls it AIC Sat. What it does is it analyzes the call and uses different AI models, not just large language models, but intent detection, et cetera, sentiment analysis, to determine what that CSET is. And ultimately these automations are to make your customer base happier. It's not just about creating cost savings or speed up issue resolution. It's a lot about building customer loyalty, enforcing brand creating, customer delight. So CSAT is in my opinion, one of the most important, if not the most important metric when it comes to agentic automation. Not the manual, not the manual one, but the automatically computed one.
- [20:12]
Marili
And what I'm hearing from you is this is very much a balance as all companies, all organizations, all leaders would like to see ROI as soon as possible from any implementations from, from AI. But this is very much a balance between, yes, saving money, saving time, but also that customer set satisfaction which is a long term roi. So I guess when leaders look at this, they have to ensure to have that mindset that this, you cannot just look at the monthly or how many people you can, or the labor cost you are saving on getting less agents, human agents in there. This customer satisfaction metric is almost as important and probably as important as the, the cost saving. With that in mind, what do enterprise leaders need to look at today to ramp this up? They will be in the audience, people listening that are very early in the stages of maturity. Maybe they just started using chatbots for that very first contact with the client. How do they ramp it up? How do they ensure that they will get to a system that follows the entire workflow as far as it as they can take it. With agentic AI, what should they be looking at?
- [21:29]
Shazam Kazi
Introducing agentic AI to your business is transformation. It's automation, right? And as such, many of the same principles that apply to digital transformation, which we've had for 20 years, they still apply here. And what I'm trying to Say is you look at three different dimensions. You look at business impact, you look at risk, and you look at feasibility, or difficulty of implementation or integration, whatever you want to call it. But risk, business impact, and feasibility are those three metrics. And typically you start off small. You start with the ones that are lowest in risk, highest in impact, and somewhere medium on feasibility. That's what our skill mining, by the way, helps you do. It helps you discover, based on your own data, where you should be starting. And then you expand over time and you add additional capabilities. So you have your main agent, which might start off providing order numbers, order updates. Someone calls, where are my new shoes? And it'll tell you where those new shoes are. And you figure out, okay, now this is working. Now we're getting 80, 85% of those customer requests, right? Because we figured this out. And that's when you get to doing order updates. Let me change my shipping address. Have my shoes shipped yet? Because if they haven't, I actually would like to get them one size bigger and try out both and send one back. But now you realize customers have to send one back. So you have to expand and handle those returns. And that is how you roll it out over time. You start small and iteratively, you expand on the use cases, which ultimately leads to you covering both the whole spectrum of your workflow.
- [23:29]
Marili
So it's very much crawl, walk, run, following that. Absolutely.
- [23:34]
Shazam Kazi
That's how you do it. Yes.
- [23:35]
Marili
And this reminds me very much of autopilot on planes. You ensure that the autopilot handles the parts of the. That it's over and over the same processes. And then where it gets slightly, let's call it dangerous but serious, that's where the, the human would come into the loop and take over from the. What does that live handoff look like from the AI agent to the human? What I'm saying is, so it's one thing to be handed off from AI to an agent, and you have to restart the entire conversation. What does, what does that handoff look like?
- [24:13]
Shazam Kazi
It's an awful experience. And one of the reasons I got into conversational AI is because at the time, I had a dog that needed treatment and I had to call the insurance every other day. And I was talking to an IVR workflow. At some point, they had a voice chatbot. I would tell my story and I'd get passed on to a human. I tell the same story again, and then again you get transferred to someone else and you tell the story the way it's being done today or needs to be done today is your handoff from AI to human is seamless in a way that the system detects that a transfer is coming and that transfer helps prepare the human agent to take the call before you actually take it. So you start to read into what was happening. If it has to be spontaneous, instantaneous, then at least you need to have a summary of the conversation so far. And you need to have the key moments of that conversation highlighted and presented in a way that it's actually consumable within 10 seconds. Because you don't want the phone to ring for 60 seconds just so the human agent can read this. Right. So the agent experience, the user interface for the agent experience needs to enable that.
- [25:38]
Marili
And does that mean that the human agent can actually interact with that information as the they do with generative AI with LLMs, can it ask the system questions? Maybe the agent gives it a report, but can it actually ask it questions so it is slightly more informed, especially when it gets really ambiguous?
- [25:56]
Shazam Kazi
You could, but that would require you to multitask during a call. So it's typically not what we're seeing. It needs to be more proactive because if you talk to a customer, that customer got escalated to you for a reason and that reason was an escalation, meaning it needs a high degree of attention and you don't want to split your attention between a chatbot and the customer. Ultimately your attention needs to be focused on a customer. So anything the customer is asking for needs to be provided to you as the human call center agent proactively, so you don't have to type or look something up.
- [26:37]
Marili
Shazan, thank you for this conversation. I think you've grounded us in real practical terms of what this looks like. The AI agent gets involved in the deterministic tasks. As an enterprise leader, you have to ensure that you know where those fast to value points are. And that could be an advisory session or within your company itself and ensure that you do not just stick the AI on your current workflows, reimagine what they could look like. And then as you just said, it's the crawl, walk, run pattern that we see across AI get those early wins. And then what you're saying is customer trust, it's a measurable analytical metric that you can follow. And when you see you've gotten the trust of your customers as well as your internal pipe agents working on the systems, that's when you can start scaling it. But am I hearing you that we see this in many other industries as well, that when you start scaling. That's when you see the real value. Like the early parts you see small wins, but it's when you scale that you really see that true value.
- [27:46]
Shazam Kazi
Absolutely. Because ultimately these systems they self optimize and the best way to self optimize is by having large amount of data points that you can optimize on. So yeah, you won't see as essentially you won't see as much improvement in the early days as you will, which is, you know, when you start this you need to have trust in the vision that these systems are going to continue to get better over time.
- [28:16]
Marili
Thank you so much for joining us today. Looking forward to more conversations.
- [28:19]
Shazam Kazi
Me too. It was a pleasure
- [28:24]
Podcast Host
wrapping up today's episode. I think there are three key takeaways
- [28:27]
Marili
from our conversation with Shazam.
- [28:29]
Podcast Host
First, agentic systems create the most value when they take the first pass on high volume deterministic customer requests and hand off cleanly when complexity or ambiguity appears. Second, leaders need data driven discovery to understand which parts of their operations are actually ready for automation, rather than assuming legacy workflows can simply be augmented. And finally, meaningful gains come from starting with low risk, high impact tasks and expanding gradually as accuracy, customer satisfaction and
- [28:58]
Marili
trust improve over time.
- [29:00]
Podcast Host
If you have an AI solution, position your brand alongside the Fortune 500 leaders defining the enterprise AI roadmap for the opportunity to showcase your solution to the executives currently funding and scaling global initiatives. Partner with Emerge. Secure your partnership@go.emerge.com partner that's go emerj.com p n e r for further executive level analysis and to join our network of leaders delivering workflow impact with AI, visit emerge.com on behalf of the team at Emerge. We'll see you on the next episode.