Podcast Summary: "Why You Should Not be Scoring 100% of Your QA Calls"

Podcast: Advice from a Call Center Geek!
Host: Thomas Laird
Date: January 8, 2026

Episode Overview

In this episode, Tom Laird tackles the persistent and often misunderstood issue of quality assurance (QA) in contact centers: whether scoring 100% of calls is actually beneficial or necessary. Drawing from his extensive experience as CEO of Expivia Interaction Marketing and Auto QA, Tom argues that evaluating every single call is typically wasteful, expensive, and—even more provocatively—less accurate than a statistically sound sampling approach powered by the right technology and models. He provides industry insights, mathematical rationale, enlightening examples, and actionable advice, especially for small to mid-sized contact centers.

Key Discussion Points & Insights

1. Two Primary QA Pricing Models

[02:10]

• Seat License Model: "The main one still is the old school...you're looking at the amount of agents you have, tagging them all with a license and you pay for that."
  • Typical costs: $100–$135+ per seat per month
  • Offers 100% call coverage but only up to ~85–90% accuracy
• Usage or Pay-Per-Call Model: "What we do is we utilize a usage model...you pay for how many calls you want scored."
  • Flexible, scalable costs
  • Ideal for centers with fewer than 500 seats
  • Relies on sampling rather than evaluating every call

2. The Myth of 100% Call Scoring

[06:00]

• "If you're paying to score every call in your contact center, you're not getting better data, you're just spending more money."
  • Coverage does not equate to accuracy; the process can become cost prohibitive and inefficient.
• Even leading AI platforms "really only have an accuracy...up to about 85%."
• Important Question: "What level [of accuracy] are you actually getting?"
• Costs vs. Returns: For 100% scoring, costs can be prohibitive, especially for smaller centers, while the accuracy gains are questionable.

3. Statistical Sampling & The Law of Large Numbers

[08:15]

• Tom references the statistical principle: "There's this statistical principle called the law of large numbers...you can apply these same principles to your QA to get you with 99% accuracy with a margin of error plus or minus 3%."
  • A properly sampled subset (say, 1,700 out of 60,000 calls) generates reliable, actionable data for a fraction of the price.
  • Memorable analogy: It’s the same math that "allows pollsters to predict elections...pharmaceuticals can improve drugs without testing every human being."
  • Typical error margin among human QA teams: 5–7%

4. Concrete Example: Sample Size vs. Seat License

[15:45]

• For a 60,000/month call center (~45 agents):
  • Seat License: $5,600–$6,000/month for 85% accuracy.
  • Sampling Model: Score ~1,789 calls/month for $1,500 with 99%/±3% accuracy.
  • "It makes zero sense for you to go pay...for a lower accuracy than you would actually get from scoring 1,700 calls a month using the law of large numbers..."

5. Accuracy Considerations

[10:45]

• "We are more accurate than human beings because we're 99% accurate, right? Within a margin of error, plus or minus 3%."
• Human variability—QA agents often disagree by 5–7% when scoring the same call.
• It's possible to sample less and achieve better, more defensible results by leveraging advanced large language models.

6. Addressing Compliance and Exceptions

[18:30]

• Compliance needs? "If you have compliance questions...we can just score those specific calls or those specific sections of the call."
  • 100% scoring may be mandated in regulated industries or clients with zero tolerance for error.
  • Usage models can still isolate and prioritize compliance elements, mitigating costs.

7. Common Mistakes in Current QA Practices

[27:15]

• Cherry-picking calls: Manual QA often biased—certain call types, lengths, or agents get disproportionately analyzed.
• Too few calls per agent: Undermines visibility, accuracy.
• Sampling not representative: Fails to reflect true agent or center performance; can be avoided with algorithmic, automated sampling.

8. How to Vet QA Providers

[33:00] Tom’s recommendations for those evaluating QA tools/partners:

• Ask about the type of language model used.
• Clarify accuracy rates and how they're measured.
• Investigate processes for updating forms and questions.
• Demand transparency regarding sample size calculations and stratification methods.

Notable Quotes & Memorable Moments

"It blows my mind why people are paying $135 per seat...when really they should be asking the question, why do I need to score all of these calls when I’m not getting any more data or insights from them?"
— Tom Laird [13:20]

"The law of large numbers is more accurate than 100% of your calls scored."
— Tom Laird [16:55]

"If your QA cherry picks calls...your sample size right there is already screwed up."
— Tom Laird [28:05]

"I want to make sure that people understand that scoring 100% of the calls isn’t always what it’s cracked up to be."
— Tom Laird [38:10]

Important Segment Timestamps

• 02:10 — Breakdown of seat license vs. usage models
• 06:00 — Myths about 100% scoring and associated costs
• 08:15 — Statistical underpinning: Law of large numbers
• 10:45 — Comparing AI, sampling, and human variability
• 15:45 — 60,000 call example: cost & accuracy analysis
• 18:30 — Compliance exceptions and hybrid approaches
• 27:15 — Classic QA mistakes (cherry-picking, unrepresentative samples)
• 33:00 — Questions to ask potential QA providers
• 38:10 — Final thoughts: why less is often more for QA

Final Takeaways

• 100% QA call scoring typically offers worse accuracy and ROI for most centers, unless mandated by strict compliance.
• Rely on sampling (rooted in statistical best practices) with high-quality models for better insights and cost-savings.
• When choosing a QA solution, push beyond marketing claims—demand clear, honest metrics on accuracy, methodology, and cost structure.
• Don’t be afraid to ask "the quiet questions"—about how many calls you need to score, what model is used, and what accuracy you’re truly getting.