Podcast Summary: Experts of Experience - "AI Agents Explained: Build a Digital Workforce That Works 24/7"

Introduction

In the March 24, 2025 episode of Experts of Experience, hosted by Lauren Wood and Stephanie Post from Mission.org, the conversation delves deep into the transformative role of AI agents in enhancing customer experience (CX). Presented by Salesforce Customer Success, this episode features Bernard Sloey, Senior Vice President of Digital Customer Success at Salesforce, who shares firsthand insights on implementing Salesforce’s groundbreaking AI platform, AgentForce. The discussion explores the evolution of AI agents, their implementation challenges, measurable impacts, and the future landscape of customer interactions powered by artificial intelligence.

The Rise of AI Agents and AgentForce

The episode begins with Bernard Sloey highlighting the pivotal moment Salesforce is experiencing with AgentForce. He states, “[00:00] It's this transformational moment that I think every company is like, wow, how do I enable this and how can I use this for my customers?” Bernard emphasizes that Salesforce has been at the forefront of agentic AI, developing AgentForce early to stay ahead in the competitive landscape of customer experience.

Lauren Wood adds, “[00:10] Salesforce is unveiling its next generation AI platform called AgentForce 2.0,” underscoring the continual advancements Salesforce is making in AI technology. Bernard’s enthusiasm is palpable as he mentions, “[00:15] I’ve never been more excited about anything in my entire career,” reflecting his confidence in the platform’s potential.

Implementing AgentForce at Salesforce

As Customer Zero for AgentForce, Bernard shares Salesforce’s journey in deploying the AI agent within their own support systems. Initially treating AgentForce like a traditional chatbot with predefined responses proved ineffective. Bernard explains, “[00:32] Instead it was like, no, no, you're smart. We went back into AgentForce and we coached it just like you would coach a human employee.” This shift from rigid scripting to a more dynamic, human-like coaching approach allowed AgentForce to better emulate human support agents, addressing both hard and soft skills.

Lauren remarks, “[06:48] Agentic AI is the next wave of AI,” highlighting the evolution from generative models like ChatGPT to AI systems that can autonomously manage entire processes. Bernard further elaborates that AgentForce can handle tasks beyond simple Q&A, enabling actions such as resetting passwords or issuing refunds autonomously, which he describes as “[11:38] a game changer.”

Measuring Success: Metrics and KPIs

A significant portion of the discussion centers on how Salesforce measures the success of AgentForce. Bernard shares key metrics from their implementation:

• Conversations Handled: Since launching on October 11th, AgentForce has engaged in over 450,000 conversations. “[24:01] we've had 450,000 conversations with AgentForce.”

• Resolution Rate: Initially, Salesforce considered a resolution rate in the mid to high 80s percent as satisfactory. However, through customer feedback, they introduced a “Customer Confirmed Resolution” metric, showing a 76% success rate where customers confirmed their issues were resolved.

• Escalation Rate: The rate at which conversations are handed off to human agents has increased from 1-2% to 4%, which Bernard views positively as it indicates smoother transitions. “[37:37] I love the ticker numbers on the homepage...”

• Abandon Rate: This measures how often users open AgentForce but do not engage. Bernard notes, “[34:15] we're working on how do we kind of pull more people into the experience.”

Lauren praises Salesforce’s transparency in displaying these metrics, emphasizing its importance for trust and continuous improvement.

Lessons Learned and Implementation Challenges

The episode highlights several lessons Salesforce learned during AgentForce’s rollout:

1. Content Quality and Data Hygiene: Bernard emphasizes that “the agent is only as good as the knowledge it has access to.” Ensuring that AgentForce is trained on accurate and relevant content is crucial. He shares a learning moment where initial strict guardrails led to poor responses, prompting a shift to a more nuanced coaching approach. “[27:07] ...we taught it to think. We were treating it like an old school chatbot.”

2. Balancing Automation with Human Touch: Salesforce found that allowing seamless transitions from AI to human agents is essential to prevent customer frustration. Bernard notes, “[15:44] Agent Force is backed by humans. It’s if it can’t solve, you easily go into the conversation with a human.”

3. Soft Skills Integration: Beyond technical troubleshooting, AgentForce was enhanced to exhibit empathy and appropriate responses in sensitive situations. Bernard shares an example: “[32:10] if someone asks about our CEO, it will point them to our investor relations page.”

4. Managing Customer Expectations: Customers often expect flawless performance from AI agents, akin to human perfection. Bernard advises setting realistic expectations and ensuring that fallback options to human agents are always available. “[48:21] ...there's features they were asking about and they need to have access.”

Impact on Company Structure and Roles

The integration of AgentForce has prompted Salesforce to rethink its organizational structure:

• Creation of New Roles: Salesforce has introduced roles such as Prompt Designers and Directors of LLM Operations, which did not exist prior to their AI integration. Bernard mentions, “[61:38] ...we just hired a prompt designer slash engineer.”

• Team Focus: With over 100 team members dedicated solely to digital success, Salesforce underscores the importance of specialized roles focusing entirely on digital customer experiences. “[43:18] ...you need people that are invested in it.”

• Augmenting Human Workforce: AgentForce serves as a digital workforce that handles routine inquiries, allowing human agents to focus on more complex issues. Bernard explains, “[60:40] ...it's allowing us to go faster and to implement more videos at a more... time frame.”

Future Insights and Advice for CX Leaders

Bernard offers several forward-looking insights and advice:

1. Embrace Continuous Improvement: AI agents require ongoing refinement and should not be treated as one-time software releases. “[52:03] ...it's a digital employee. You gotta be constantly refining it.”

2. Foster Cross-Functional Collaboration: AI agents can bridge silos between marketing, sales, and support, enabling a unified customer experience. “[53:29] ...you suddenly have this agentic layer that goes across all of them.”

3. Leverage Structured and Unstructured Data: Utilizing platforms like Salesforce’s Data Cloud can significantly enhance AI agent performance by providing comprehensive, organized data repositories. “[29:44] ...Data Cloud allowed us to take all of these different data sources...”

4. Prioritize Customer Obsession: True customer obsession involves actively engaging with customer feedback and iterating based on real interactions. Bernard advises, “[72:38] I don’t think data is everything...sometimes the best decision goes against what data tells us.”

5. Prepare for New Roles and Skills: As AI integration deepens, organizations should anticipate and plan for new roles that manage and optimize AI agents. “[61:38] ...we have a director of our LLM operations.”

Notable Quotes

• Bernard Sloey on Transformational AI Moment:
  “[00:00] It's this transformational moment that I think every company is like, wow, how do I enable this and how can I use this for my customers?”

• Lauren Wood on Agentic AI Evolution:
  “[06:48] Agentic AI is the next wave of AI. We've been in the generative world using LLMs, ChatGPT, all this stuff.”

• Bernard Sloey on Continuous Refinement:
  “[52:45] It's not a one and done. It's a constant refinement, it's a digital employee and you gotta be constantly refining it.”

• Bernard Sloey on Customer Obsession:
  “[72:38] I don't think data is everything. Sometimes the best decision goes against what data tells us.”

Conclusion

This episode of Experts of Experience provides an in-depth exploration of how AI agents like Salesforce’s AgentForce are revolutionizing customer experience. Bernard Sloey’s insights reveal the complexities of implementing AI in large-scale operations, the importance of data quality, the necessity of integrating soft skills into AI, and the ongoing need for human oversight. As companies navigate this AI-driven landscape, the lessons shared highlight the balance between automation and human touch, the need for continuous improvement, and the evolving roles within organizations. For CX leaders and businesses aiming to enhance their customer interactions, this episode serves as a valuable guide on leveraging AI to build a digital workforce that operates seamlessly around the clock.