Why Deepfake Fraud Beats Your Workflows, Not Your Technology - with Jon-Rav Shende of Thales Group

Summary

---

Podcast Summary: The AI in Business Podcast

Episode Title: Why Deepfake Fraud Beats Your Workflows, Not Your Technology
Guest: Jon-Rav Shende, Global CTO for Data and AI at Thales Group
Host: Daniel Faggella
Date: May 21, 2026

---

Overview

This episode dives into the rapidly growing threat of deepfake and synthetic voice fraud, focusing on why the real risk lies not in failing technology but in enterprise workflows and agent behavior. Host Daniel Faggella and guest Jon-Rav Shende unpack how attackers exploit customer service processes, especially under conditions of urgency and high-value business actions, and outline a practical response for business leaders—emphasizing shared responsibility, mapped risk journeys, and evidence-based controls.

---

Key Discussion Points & Insights

1. The True Exposure in Voice Channels

• Deepfake Fraud's Rapid Growth:
  • "Pindrop's 2025 voice intelligence and Security report had deep fake fraud attempts up by 1,300% in 2024, going from about one a month to seven a day, which is absolutely crazy." — Daniel Faggella [02:33]
• Where Are Enterprises Most Vulnerable?
  • Live voice channels are most exposed at the intersection of identity, urgency, and business action—especially in customer support, claims, payments, password resets, and executive-facing workflows.
  • Attackers exploit helpful agent behavior, system urgency, and weak escalation processes, not just technical flaws.
  • Memorable Quote: "A deep fake does not need to fool the entire company. Basically it needs to fool one individual, one weak point...take advantage of the good nature and expectations of good customer service from the agent..." — Jon-Rav Shende [03:47]

2. "Beating Workflows, Not Technology"

• Attackers leverage:
  • Social engineering, urgency, and partial personal data, rather than defeating advanced biometric or technical controls.
  • The most exposed channels are those where a caller can convince an agent to do something important without robust step-up verification (e.g., password resets, fund disbursements, account recovery).
  • Quote: "Assuming they're beating the workflow, they are using urgency, emotion, partial customer data, spoof numbers." — Jon-Rav Shende [07:52]

3. Shared Ownership and Accountability

• Who Owns the Problem?
  • Security owns the risk.
  • Operations owns the process.
  • Customer experience owns the friction.
  • True defense requires a joint control model with clear, shared accountability: "Accountability...has to be shared because an attacker is exploiting all three simultaneously." — Jon-Rav Shende [09:41]
  • The business owner must ultimately accept the residual risk with input from all teams.

4. What "Good" Looks Like in Practice

• It's not sufficient to have detection demos or more tools—real security means guiding better business decisions while a call is ongoing.
  • Control systems should enable step-up verifications and maintain an evidence trail for every action.
  • Focus on:
    • Human vs. synthetic/replayed/manipulated voice
    • Device and number reputation
    • Customer behavior patterns
  • Quote: "Good does not look like a demo that says we detected deepfake. Good looks like a control System that helps the business make a safer decision while the call is still happening." — Jon-Rav Shende [12:54]
• Avoid over-reliance on overlapping tools: "In one environment you might have 25 tools...and we're still being breached." — Jon-Rav Shende [15:50]

5. Surveillance, Privacy, and Evidence

• On Surveillance:
  • Shende paints a candid picture, referencing London's omnipresent cameras to underline that total privacy is already rare, so the focus must be on governing actions over governing just the tools.
  • Quote: "We don't have privacy anymore. Just to be honest, we really don't." — Jon-Rav Shende [17:38]
• Importance of classifying business actions by risk and ensuring every control leaves an evidence trail for compliance and audit.

6. Four-Step Action Plan for Leaders

[21:34]

1. Map Risky Voice Journeys
   • Identify where a call can change access, move money, release data, or override controls.
2. Define Decision Points
   • Pinpoint the exact moments agents decide to trust, escalate, deny, or apply step-up verification.
3. Specify Evidence Requirements
   • Anticipate needs of auditors, insurers, legal, and compliance before an incident occurs.
4. Layer AI as a Risk Signal
   • Use AI for risk signals (synthetic voice detection, behavioral anomalies, device reputation)—but automate only with high-confidence and always with human oversight on high-stakes decisions.
   • "You only automate when confidence is high...we have to be mindful that we do not let it silently approve high risk actions without appropriate controls." — Jon-Rav Shende [24:36]

---

Notable Quotes & Memorable Moments

• "Your voice channel is most exposed at the point where identity, urgency and business actions come together. And that's where deepfake risk becomes an actual business risk." — Jon-Rav Shende [06:12]
• "It's not about defeating your technology, it's about beating your workflows." — Summarized by host and guest throughout
• "The evidence chain is not a technical afterthought. It is a boardroom requirement." — Daniel Faggella [26:47]
• "We have 30 tools now. We're still seeing the same thing happening. So my perspective is never a focus on tools. My focus is about spotting that abnormal risk early enough and then taking the appropriate action thereafter." — Jon-Rav Shende [16:15]
• "The aim is not to try to solve everything...but basically understand that journey and then understand the gaps." — Jon-Rav Shende [24:57]

---

Important Timestamps

• 02:33 – Deepfake fraud’s massive increase in prevalence
• 03:47 – How attackers exploit agent workflows and customer service paradigms
• 07:52 – The problem in assisted-service channels and what attackers target
• 09:41 – Shared ownership model for defending against deepfake fraud
• 12:54 – What effective AI defense looks like in real-world operations
• 17:38 – Surveillance realities and shifting from tool governance to action governance
• 21:34 – Practical, four-step action plan for enterprises
• 26:47 – Key takeaways for executives: workflow focus, risk mapping, and robust evidence chains

---

Conclusion

This episode provides a pragmatic roadmap for business leaders tackling the urgent threat of deepfake voice fraud. Rather than buying more tools, leaders must map their high-risk workflows, create shared controls across teams, specify evidence for every action, and only then introduce AI as a supportive risk signal. The highest risk is not your weakest technology, but your weakest workflow—where urgency, identity, and business action converge. In regulated environments, having an evidence-backed, joint-control posture is not just smart—it's essential.

---

Transcript

[A]

Welcome everyone to the Emerge AI in Business podcast. Today's guest is John Rafshende, Global CTO for Data and AI at Thales Group. As AI generated voice tools make synthetic fraud increasingly accessible, enterprise contact centers face a risk that has less to do with technology and more to do with the workflows agents use to make trust decisions under pressure. John examines where the live voice channel is most exposed, why the convergence of identity, urgency and business action in a single call represents the highest risk point for regulated organizations, and how senior leaders can build a practical response framework that satisfies internal auditors, cyber insurers and regulatory requirements. Simultaneously, the conversation delivers a clear four step order of operations, from mapping high risk voice journeys to defining the evidence chain required before any AI risk signal layer is deployed. Today's episode is sponsored by Modulate. A quick note for our audience that the views and opinions expressed by Genre of Shende on today's program are his own and do not reflect those of Thale's group or its leadership. Emerge works with a selected group of AI vendors to reach Fortune 500 decision makers through research, media and direct access. If you want to be considered, download our media [e-mail] partner that's go emerj.com pa now the conversation with John. John, it's great to have you in our Emerge AI in Business studio today.

[B]

Thank you and it's a pleasure being here.

[A]

Yolande I'm looking forward to a conversation that our leaders will definitely resonate with and to give them some practical guidance. I think we would want them to, at the end of this, this conversation, be able to look at their own voice channel today and answer one question. If a deepfake or a manipulated caller hit their agents today or in the next 10 minutes, would anyone know before the call ended? And if, if that's the answer is no to that, then they should be listening to what they should be doing or thinking about or asking. And I think place to start is I think it was Pindrop's 2025 voice intelligence and Security report that had deep fake fraud attempts up with 1,300% in 2024, going from about one a month to seven a day, which is absolutely crazy. So I want to know from your CISO seed, where is the live voice channel actually most exposed right now?

[B]

That's, that's a really good question and it's based on several factors. So couple of things. When we think about this, the highest risk areas that we would look at would be from a CISO perspective And a C suite perspective, you're looking at things like your customer support call centers, your help desk, individuals that handles claims, payments, password resets, account recovery and executive facing workflows. The next logical flow would be, well, why? And that's because teams are basically trained to help quickly. They're measured on speed, they're measured on customer satisfaction and resolution. And most people within customer service, within these type of organizations, they are trained to be service oriented, to focus on resolution, focus on customer satisfaction. And those are all wonderful things to think about. Those are all wonderful things that we are measured on. And we expect csat, customer metrics, satisfactor and surveys to come on and say, yes, you did great. That's typically why at the end of a call, typically a service rep would say, hey, can you answer this feedback? Or you will get a link to a survey, can you please rate me? And so on. And with that in mind, because folks are trained to help so quickly, attackers know this. And because attackers know this, they don't just attack technology per se, they attack the process. So a deep fake does not need to fool the entire company. Basically it needs to fool one individual, one weak point. And fool is not necessarily the right word. I would say it's take advantage of the good nature and the expectations of good customer service from the agent to take advantage of that and inveigle change to an email, reset a password, approve a transaction or access a policy which may lead to bypassing a specific control. So when we think about this, the exposure is not only in the voice per se, it's basically in the journey. And we think about things like who is calling, what are they asking for? Is the device, is the number trusted? Is this a normal request that we are looking at? When we think about that, we have to think about risks to the environment. So we want to know if, hey, is this action high risk? And with all of that, then it falls back to the agent. Does the agent have a clear escalation path? And then afterwards, can the company prove why the call was allowed? And obviously, if you have a clear escalation path, you should have flags, you should have process controls, you should have risk measurements that will say, hey, this seems suspicious, this seems fraudulent, let's stop it. So basically for an executive perspective, the practical takeaway is simple. Your voice channel is most exposed at the point where identity urgency and business actions come together. And that's where deepfake risk becomes an actual business risk.

[A]

It's one of those problems that it costs you multiple amounts of money. A problem that costs you money to try and figure out when it happens. And then it costs you money to reimburse clients or it costs you money for regulatory fines, things like that. It's quite a costly problem to have. And I'm wondering if you had to walk into an organization tomorrow that you have very little context on as a ciso, which voice channel would you assume is already compromised, even not knowing the organization so well? So what is a known issue that you, that you're picking up on?

[B]

That's a wonderful question. So typically I would say the highest is any assisted service channels. Right. Channels that we are testing. So we're going to look at things from four areas. If you're thinking finances, you're thinking your claims and payment support, for example, that could span insurers, that could span banking, financial services. And the next most obvious one would be help desk. Right, Help desk. They would be the most targeted because you can compromise an agent or convince an agent into changing a password. And if you change a password, then it's game over. The other thing on top of that would be account recovery. Someone can say hey, oh my password's locked. They could have done some due diligence, figured out some account information and provide enough information to convince an agent that hey, this is a legitimate user, thus drive towards a password account recovery. Sorry. So that's where I would not start by assuming an attacker is bypassing or getting ahead of, for example a biometric model. Assuming they're beating the workflow, they are using urgency, emotion, partial customer data, spoof numbers. And today with biometrics and with AI we can generate synthetic voice. Now with that we can pressure the agent to execute one risky action that's approved and then it's game over. For me as a ciso, the first question I would most likely ask if I walk into that situation is what? Where can a caller convince an agent to do something important without strong step up verification? And that's tranel I will treat as already exposed.

[A]

I think with this problem, ownership is also adding to the problem because like you mentioned in the beginning, this is very much where we touch the customer problem. But then where is that line between it being a security problem versus it being a customer experience problem versus the it being an operations problem? Who is supposed to own this inside the enterprise?

[B]

Well, and again it rolls up to the CISO and it rolls up to the risk team. Ultimately, when we think about voice deep fakes, typically you would structure it around three owners. Security owns the risk, ultimately operations owns the process. And customer experience owns the friction in between all of that. So accountability in essence has to be shared because an attacker is exploiting all three simultaneously. So when I think about this, how I look at it is that if a call can lead to account access, money movement data release, identity change, or a control override, that is a security problem. However, if the agent workflow allows that action too easily to occur, then it's an operations problem. And if the fix creates too much customer friction, then it's a customer experience issue. So what does that mean for us as leaders? We have to be able to move away from debating who owns what and create a joint control model. So basically what I would look at is, hey, security will define the risk thresholds, security will define the risk thresholds. They will look at risk impact, cost of risk remediation, as well as cost of impact to the business, and then work with operations to design the workflow. Customer experience then will measure the the resulting customer friction and risk and compliance working together will verify the evidence and then we pass that over to a business owner who accepts the residual risk. And this is the operational model where we think about we have shared ownership, but ensure clear accountability.

[A]

It sounds very much like a teamwork makes the dream work scenario, which is a very practical way to think about it. I want to take us forward a step. We've now looked at the problem where we've kind of unpacked, where we see the different problems all adding to one big problem and taking us forward. I think it was Sam Altman that went on stage at the Federal Reserve conference in last July. And he said, well, he kind of warned the financial services industry. He said, and this is verbatim, I need to say it exactly the way a significant impending fraud crisis waiting on the financial services industry, because AI can now consistently fool voice print authentication. So when AI is actually applied well in life voice, not as a demo, not as the beautiful picture that we're trying to paint, but inside real workflow. What does good look like? What is the agent experience? What is the SOC experience? What is the customer experience?

[B]

You're right about AI right there. And it's a scary thing we're working with, right? Because we're seeing AI. I would even say month on month, I would say day by day, week by week, it's growing and improving by leaps and bounds, right? When we think about AI in the voice channel, when AI is applied well in the voice channel, good does not look like a demo that says we detected deepfake. Good looks like a control System that helps the business make a safer decision while the call is still happening. So what should we think about? We have to be practical here, right? We know within our corporate environments, enterprises are constrained by regulations, audit, compliance requirements and you know, we are within a box that we have to conform to when it comes to regulations and compliance and ethical behavior and so on. That's us. The them is we don't care about regulations, we don't care about compliance, we don't care about ethical behavior. We care about getting into your environment and getting out as quickly as possible with as much as we can that's profitable to us as an attacker. So looking at this from a practical perspective, we have to assess whether the voice is likely human, synthetic, replayed or manipulated. And then what we should think about is looking beyond the voice, looking at the device now that's associated, is it a known device? Looking at the number, is it a known number? Looking at customer patterns. A couple years ago I was part of a company, I designed the architecture, it was called Myveda and we were looking at identity based risk. And one of the things we looked at was patterns of usage across an environment. So John or Yoland wakes up in the morning, you have a routine, you log in, you look at your emails, there is a process you follow when you get to work. More systems understands that. So now if there was a potential for fraud or the potential for a malicious user stealing your credentials, they would not know your pattern, they would deviate away from your pattern and we would flag that immediately as potential risk or malicious activity. So knowing customer patterns is another thing we look at and ensuring that it's normal, normal behaviors, normal requests and so on. And then we look at how is the business acting upon a certain request based on a validated user. And you know, what we want to do is basically trick a step up verification before risky action is completed. And what that means is we have to be able to also curate and accumulate evidence. So why for example, did we trust a call, why did we block it, why did we escalate it? And where we are going at is we're not trying to get to perfect detection. We're basically saying can AI help us spot abnormal risk early enough, slow down that transaction and prove why we made that decision in the first place? So this ties into the whole Sam Altman discussion and on the risk and so on. So this is basically where we have to focus on. You know, as a ciso we see lots and lots of tools, lots and lots of end of presentations and pitches at the end of the day, we have to understand what's in our environment and what risks we are looking to assess, manage, mitigate and control. And when we have that identified, we basically then select an appropriate vendor to manage that risk. Today we have two too many overlapping tools in one environment. You might have 25 tools, right? And we're still being breached. We're still seeing risks occurring, we're still seeing, you know, identities being exploited. And we had another five tools, so we have 30 tools now. We're still seeing the same thing happening. So my perspective is never a focus on tools. My focus is about spotting that abnormal risk early enough and then taking the appropriate action thereafter.

[A]

And it sounds like this is something that can become quite intense, especially with all the tools available and new tools being integrated and launched almost daily at this moment. But how do we keep this from becoming surveillance? Or do we want it to mimic the idea of surveillance?

[B]

I think it's too late for that. You know, I mean, honestly, we have, we don't have privacy anymore. Just, just to be honest, we really don't. When I was at UNI in London, I remember taking the train up the tube up to Victoria Station and being a security student at the time, I looked as soon as I stepped off the tube onto the platform, it was Maybe, maybe an eight minute walk to exit. I counted about 50 cameras.

[A]

Oh, wow.

[B]

That I could have seen. That I could have seen, you know, and all I could think about was how many are unseen. So I know that anywhere in the world I go, there is a composite of me existing starting from when you walk into an airport, when you exit an airport, a train station, you know, a rental agency, there is always something recording. And as we move along with businesses with security cameras, you're always being recorded, right? So the whole idea of are we free from that Big Brother surveillance type thing is, you know, that's yesterday's news. We are in a sense, right? When you think about tools available and so on, as you were saying, if every new AI tool becomes a new control project, then that's a challenge, right? Because the program will collapse. So the model is not just to govern per se tools separately. The model is govern the actions, not just the tools govern the actions. So for voice AI, since this is the topic, that means we classify the business action. And basically when we classify the business actions, we look at low risk, medium risk, high risk, medium risk. For example, do we disclose account specific information? High risk is our credential, reset our payment details and so on. Releasing funds if you're talking financial services. And then we decide, well, how do we control that? Right? And we have to be practical in how we control it because we may have another tool arriving tomorrow that was not there today. And we have to ensure that the rules we have in place stay consistent. What actions are we looking at? What data is being accessed, what decisions are being influenced? What proof now of that action, that evidence chain occurs as we exit that and what happens if we have a false positive? Right? And I'm working on an AI security project and for me observability, visibility and then risk identification and remediation is really what I'm looking at from an AI perspective. I need to be able to reconstruct the part an AI agent takes from start to finish in order to ensure that I can provide or prove from a dividends perspective that that action is valid from the agent to the systems to the outputs.

[A]

And that I think is going to be crucial, especially in regulated environments where every decision, decision has to be backed up with why was it, why did the false positive happen? Why did we identify this as fraud or attempted fraud? So lots, lots to actually unpack there. We were heading into the part of the conversation where it's time for us to give our listeners a bit of an action plan. I think there's a big gap between oh, we should do something about this, it's a problem. And we've actually started looking into it and we've started deploying some sort of AI to do this. So that gap is quite wide at the moment. If a senior leader is listening to this and they want to move, what is the order of operations? What should they be mapping out first?

[B]

So obviously you need to understand what your systems or what your agents or what your AI ecosystem, what's the objective? And then you need to understand the risk there and then once you have that risk identified, you classify that and then you build, you know, your remediation plan for there. Now if we're thinking about voice per se, you have to think about what are risky voice journeys where can call change access, move money, release data, reset credentials or bypass or override a control. The second thing is define the decision points. So at what moment does an agent decide to trust or escalate or deny or ensure? We have step up verification. And the third thing is to find evidence required because now we have audit in the event of an incident occurring. We have multiple people that will require evidence. We have our internal auditors, we have our external auditors, we have our security consultants of the world that will come in to help us analyze for forensics evidence and so on. And that's a technology path, right? And then we have our internal incident response and our SOC teams, for example, our threat and response teams. And now that's the technical part, right? The technical folks, the non technical, we have to think about the cyber insurers, we have to think about legal folks, we have to think about audit and compliance. So when we say we define the evidence required, we have to consider what every one of those stakeholders would be looking for and then provide proof that we need to show justification of a specific decision that was taken to them. The fourth thing I would think about is add AI as a risk signal layer. So what does that mean? When we think about calls, we think about synthetic voice detection. We have to understand behavioral analysis, we have to understand the device reputation and call pattern anomalies, as we discussed earlier. And then that coupled with transactional risk, right? So that is your risk signal layer that needs to be clearly defined with stakeholders, with response plans, with action plans on how to respond if something goes wrong, who to call, what to do and what action as a result that we take in combination with all stakeholders. And then, you know, everyone talks about automation, right? Automation has been a big push, hadn't, has and will be a big push going forward still. So we have to be careful when we automate though, because we only automate when confidence is high. So with AI, a lot of us will be using AI to recommend or flag an issue and then run an escalation. But we have to be mindful that we do not let it silently approve high risk actions without appropriate controls. So the plan basically is understanding that call journey, understanding what can cause the most financial, regulatory or customer harm. Then we craft those journeys with evidence and escalation thereafter. The aim is not to try to solve everything, right? Not to try to solve every voice interaction, but basically understand that journey and then understand the gaps between we should do something and we should have an operational control in place.

[A]

I think the picture that you've been painting to me is kind of like realizing leaders should be realizing that AI fraud is not a moment, it's like you said, it is an entire journey. The fraudsters are beating your workflows, as you mentioned earlier. They're trusting that our agents will be so fixated on giving a good customer experience that they will look past things that are out of the ordinary. And instead of having 30 different tools to try and monitor that journey, it's better to sit and look at the different risks and maybe invest in one thing that will really bring it to a point where it minimizes the risk, where it matters most, and especially in regulated environments. Is that the, is that the picture that I should be painting for our listeners based on this conversation?

[B]

Absolutely.

[A]

Great. I love that. John, it's been, it's been great having this conversation with you. You've shared some very, very crucial opinions and insights and a great action plan for, for our executive listeners. Thank you so much for your time today. We really appreciate you being here.

[B]

Thank you so much.

[A]

Wrapping up today's episode, let's look at the three key takeaways from our conversation with John. First, Deepfake Voice 4 does not need to defeat your technology. It only needs to exploit one agent's instinct to be helpful. Which means the highest risk point in your voice channel is wherever identity urgency and high value business action come together in a single call. Second, before deploying any AI detection layer, organizations must first map their risky voice journeys. Because without that map, any tool you deploy is solving the wrong problem. And finally, the evidence chain is not a technical afterthought. It is a boardroom requirement. Internal auditors, cyber insurers, legal counsel and regulators will all need to see proof of why a decision was made or escalated. And organizations that built that chain before an incident occurs are the ones that recover faster when one happens. Position your brand alongside the the Fortune 500 leaders defining the enterprise AI roadmap for the opportunity to showcase your solution to the executives currently funding and scaling global initiative. Partner with Emerge to reach the decision makers holding the strategic mandate. Secure your [e-mail] partner that's go.emerj.com partner for further executive level analysis. And to join our network of leaders delivering workflow impact with AI, visit emerge.com on behalf of the team at Emerge. We'll see you on the next episode.