Experts of Experience: “Don’t DIY Your AI”: AI Agent Expectations vs. Reality
Host: Lauren Wood
Guest: Laura Miske, Customer Experience Manager at Secret Escapes
Presented by: Salesforce Customer Success
Release Date: April 16, 2025
Introduction
In the latest episode of Experts of Experience, host Lauren Wood dives deep into the transformative journey of implementing agentic AI in customer service with Laura Miske from Secret Escapes. The episode, titled “Don’t DIY Your AI”: AI Agent Expectations vs. Reality, explores the practical challenges and unexpected benefits of integrating Salesforce’s AgentForce into a dynamic customer experience strategy.
Embarking on the AgentForce Journey
Motivations for Implementing AgentForce
Laura Miske shares that Secret Escapes aimed to not only enhance efficiency but also position themselves at the forefront of AI innovation. “We wanted to be on the AI wagon before everyone else,” says Laura (00:56).
Pilot and Early Adoption
Secret Escapes became the first European customer to go live with AgentForce, marking a significant milestone. Laura emphasizes the excitement and learning curve involved:
"It’s a completely revolutionary technology. We're in at the very beginning of it and it's something that you need to take as a long term investment." (00:00)
Navigating Challenges
Complexity Beyond Expectations
While the seamless interactions of AgentForce impressed Laura, the implementation was more intricate than anticipated. She explains,
"If you read through some of the transcripts, it is impressive how seamless it is and how the customer didn't even realize they were talking to an AI agent..." (00:59)
However, setting up complex use cases that went beyond simple FAQs required more time and resources.
Measuring Return on Investment (ROI)
One of the significant hurdles was quantifying ROI, as traditional metrics didn’t fully capture the AI’s impact.
"The deflection rate we were expecting to achieve has not been as high as we expected." (08:21)
Laura suggests expanding the metrics to include customer satisfaction and interaction quality rather than solely focusing on cost savings.
Learning and Adaptation
Quality Assurance Processes
Ensuring high-quality interactions involved rigorous human-driven QA. Laura outlines their approach:
"We review a certain number of interactions per day and we have put together a form where we score the quality of the interaction..." (15:29)
This ongoing process required collaboration with the development team to address issues and refine the AI’s responses.
Training the AI Agent
Training AgentForce involved using historical customer interactions to teach the AI. Laura notes,
"We wrote up the use cases and we did a lot of testing based on different iterations of the same questions." (15:01)
Continuous testing and refinement were crucial for improving the AI’s effectiveness.
Impact on Team and Operations
Team Dynamics and Resource Allocation
Implementing AgentForce necessitated a shift in team responsibilities. Laura highlights the importance of having dedicated experts:
"Having a pool of experts that is focused on this full time... is key to get the best results in the fastest way possible." (16:45)
This focus ensures that the AI integration is managed efficiently without overburdening existing staff.
Scaling Without Cutting Jobs
Contrary to fears of job reduction, Laura views AI as a tool for scaling operations without increasing the workforce.
"Rather than seeing the AI as a way to cut jobs, is, I think, a way to scale up the company without growing the workforce so much." (17:48)
AI handles repetitive tasks, allowing human agents to focus on building long-term customer relationships.
Enhancing Customer Experience
Improved Customer Effort Score (CES)
One of the most noticeable benefits was the significant reduction in negative CES related to live chat interactions.
"Now that type of negative feedback is virtually dissolved." (30:44)
Customers experienced smoother interactions with the AI, enhancing their overall satisfaction.
Quality of Interactions and Escalations
AgentForce’s ability to detect and escalate complex or sensitive issues improved customer trust.
"Our escalation percentage... has increased, but as I said, overall I can see a better perception of the interaction with the virtual agent." (31:31)
This ensures customers receive the assistance they need without frustration.
Future Plans and Insights
Expanding Use Cases and Data Utilization
Looking ahead, Laura plans to enhance the AI’s capabilities by expanding the knowledge base and integrating more personalized data.
"I'm looking at different strategies we could take, but these are the main two avenues that I would like to explore." (34:39)
Leveraging comprehensive data within Salesforce will further empower AgentForce to handle a broader range of customer inquiries effectively.
Adopting New Functionalities
Future functionalities, such as workload management and case co-piloting, are anticipated to further streamline operations.
"I'm also looking at some of the new functionalities that will be coming up in Salesforce when it comes to workload management..." (34:39)
These tools will help free up agent time, allowing them to focus on more meaningful customer interactions.
Key Takeaways and Advice
Start Simple and Leverage Expertise
Laura’s primary advice to customer experience leaders is to begin with straightforward use cases and ensure robust data foundations.
"Start simple and make sure you have good data. Make sure you have a good foundation of knowledge base. Make sure you lean on the experts out there, don't DIY this..." (40:26)
Emphasizing a long-term relationship focus, she advocates for using AI to enhance, not replace, the human touch in customer service.
Foster Long-Term Customer Relationships
Investing in AI should be viewed as a way to deepen customer relationships by freeing human agents to engage in more meaningful interactions.
"The key to achieve the best customer experience in general... is to really listen and really try to help as that customer will stay with you for a very long time rather than a quick win." (41:14)
Conclusion
Laura Miske’s insights provide a comprehensive view of the early stages of agentic AI implementation, highlighting both the challenges and transformative benefits. As Secret Escapes continues to refine their use of AgentForce, their experience serves as a valuable guide for other businesses embarking on similar AI journeys. Emphasizing the importance of expertise, quality data, and a long-term approach, this episode underscores the pivotal role AI can play in elevating customer experience.
Notable Quotes:
- “It’s a completely revolutionary technology. We're in at the very beginning of it and it's something that you need to take as a long term investment.” – Laura Miske (00:00)
- “The deflection rate we were expecting to achieve has not been as high as we expected.” – Laura Miske (08:21)
- “Start simple and make sure you have good data. Make sure you have a good foundation of knowledge base. Make sure you lean on the experts out there, don't DIY this...” – Laura Miske (40:26)
This summary captures the essence of the episode, presenting a structured and comprehensive overview for those who haven’t listened while maintaining the depth and richness of the original conversation.