Podcast Summary: Voices of Search – "Building a Company on Testing vs Consulting"

Date: February 5, 2026
Host: Jordan Cooney
Guest: Will Critchlow, CEO at SearchPilot

Episode Overview

This episode explores the strategic shift from traditional SEO consulting toward building a business rooted in rigorous, repeatable testing—especially in the fast-evolving landscape of AI-driven search and content discovery. Jordan Cooney interviews Will Critchlow about the origins of SearchPilot, the challenges that prompted its creation, and why a testing-first approach is now critical for SEO in a world of black-box algorithms.

Key Discussion Points & Insights

1. Why Shift From Consulting to Testing?

• Origins in Consulting:

  • Will Critchlow explains that after a decade in SEO consulting, his team observed recurring problems among enterprise clients:
    • Difficulty getting initiatives implemented
    • Trouble proving and measuring impact
    • Uncertainty about whether any value was delivered

• Aiming to Solve Systemic Problems:

  • These pain points led to the creation of a platform designed to facilitate controlled SEO experiments (A/B and geo-testing), allowing organizations to quantify the effect of site changes rather than relying on opinion or conventional wisdom.
  • Quote:
    • “We found the same problems over and over again. We found the challenges getting things done, challenges proving the value of the work that they'd done, challenges even understanding whether there had been value…”
      — Will Critchlow [01:19]

2. The Value of Testing in Modern SEO

• SEO Is More Opaque Than Ever:
  • With rapidly changing search discovery, increased AI integration, and less transparency about ranking factors, Will explains that relying on previous best practices is no longer effective.
  • Quote:
    • “The more it changes, the less you can rely on yesterday's best practices, the more it's a black box and the more that these algorithms are unknown and unknowable. The only way to operate, in my opinion, is to test.”
      — Will Critchlow [03:57]

3. Testing as a Boardroom Topic

• Elevating Testing to the C-Suite:

  • The ability to provide real-world data about what works and what doesn't is increasingly valued at the highest organizational levels.
  • SearchPilot has seen more of its work presented and discussed at the executive layer, reflecting greater recognition of data-driven SEO.

• Breakthroughs in the Last Few Years:

  • Will notes that the company has made significant progress—especially in recent years—driven by their focus on where they can add the most value.

• Responsiveness to New Discovery Channels:

  • The rise of AI and rapidly changing platforms creates urgency for live, adaptable testing over static recommendations.

Notable Quotes & Memorable Moments

• On the necessity of testing in SEO today:

  • “It's not a case of a set of ranking factors that we could speculate about like we could in 2009.”
    — Will Critchlow [04:21]

• On why SearchPilot’s approach matters now:

  • “That's where we're finding that we're adding the most value. And I'm very excited that the… growth of all of these new discovery channels… is bringing this topic… into the boardroom, into the C suite, into the executive layer.”
    — Will Critchlow [03:34]

Important Timestamps

• 00:44 — Introduction of Will Critchlow and the concept: Building a company on testing vs consulting
• 01:19 — Will explains the origin story and consulting pain points
• 03:17 — The shift to data-driven testing; SearchPilot’s recent progress
• 03:57 — Why testing is the only viable strategy in today’s opaque, fast-changing SEO landscape
• 04:21 — Comparison of past SEO (speculation-based) with today’s need for scientific testing

Conclusion

Will Critchlow’s conversation with Jordan Cooney illustrates the essential transition from traditional SEO consulting to systematic, platform-driven testing. In a world where algorithms are continually shifting and “best practices” often fail, investing in rigorous experimentation and data-backed decision-making isn’t just an option—it’s a business imperative. As search and discovery become even more driven by AI, SEO leaders must move beyond intuition and embrace solutions that prove real value.

Memorable Sign-off:
“Remember, the answers are always in the data.”
— Jordan Cooney [end]