Building a company on testing vs consulting

Summary

Podcast Summary: Voices of Search – "Building a Company on Testing vs Consulting"

Date: February 5, 2026
Host: Jordan Cooney
Guest: Will Critchlow, CEO at SearchPilot

---

Episode Overview

This episode explores the strategic shift from traditional SEO consulting toward building a business rooted in rigorous, repeatable testing—especially in the fast-evolving landscape of AI-driven search and content discovery. Jordan Cooney interviews Will Critchlow about the origins of SearchPilot, the challenges that prompted its creation, and why a testing-first approach is now critical for SEO in a world of black-box algorithms.

---

Key Discussion Points & Insights

1. Why Shift From Consulting to Testing?

• Origins in Consulting:

  • Will Critchlow explains that after a decade in SEO consulting, his team observed recurring problems among enterprise clients:
    • Difficulty getting initiatives implemented
    • Trouble proving and measuring impact
    • Uncertainty about whether any value was delivered

• Aiming to Solve Systemic Problems:

  • These pain points led to the creation of a platform designed to facilitate controlled SEO experiments (A/B and geo-testing), allowing organizations to quantify the effect of site changes rather than relying on opinion or conventional wisdom.
  • Quote:
    • “We found the same problems over and over again. We found the challenges getting things done, challenges proving the value of the work that they'd done, challenges even understanding whether there had been value…”
      — Will Critchlow [01:19]

2. The Value of Testing in Modern SEO

• SEO Is More Opaque Than Ever:
  • With rapidly changing search discovery, increased AI integration, and less transparency about ranking factors, Will explains that relying on previous best practices is no longer effective.
  • Quote:
    • “The more it changes, the less you can rely on yesterday's best practices, the more it's a black box and the more that these algorithms are unknown and unknowable. The only way to operate, in my opinion, is to test.”
      — Will Critchlow [03:57]

3. Testing as a Boardroom Topic

• Elevating Testing to the C-Suite:

  • The ability to provide real-world data about what works and what doesn't is increasingly valued at the highest organizational levels.
  • SearchPilot has seen more of its work presented and discussed at the executive layer, reflecting greater recognition of data-driven SEO.

• Breakthroughs in the Last Few Years:

  • Will notes that the company has made significant progress—especially in recent years—driven by their focus on where they can add the most value.

• Responsiveness to New Discovery Channels:

  • The rise of AI and rapidly changing platforms creates urgency for live, adaptable testing over static recommendations.

---

Notable Quotes & Memorable Moments

• On the necessity of testing in SEO today:

  • “It's not a case of a set of ranking factors that we could speculate about like we could in 2009.”
    — Will Critchlow [04:21]

• On why SearchPilot’s approach matters now:

  • “That's where we're finding that we're adding the most value. And I'm very excited that the… growth of all of these new discovery channels… is bringing this topic… into the boardroom, into the C suite, into the executive layer.”
    — Will Critchlow [03:34]

---

Important Timestamps

• 00:44 — Introduction of Will Critchlow and the concept: Building a company on testing vs consulting
• 01:19 — Will explains the origin story and consulting pain points
• 03:17 — The shift to data-driven testing; SearchPilot’s recent progress
• 03:57 — Why testing is the only viable strategy in today’s opaque, fast-changing SEO landscape
• 04:21 — Comparison of past SEO (speculation-based) with today’s need for scientific testing

---

Conclusion

Will Critchlow’s conversation with Jordan Cooney illustrates the essential transition from traditional SEO consulting to systematic, platform-driven testing. In a world where algorithms are continually shifting and “best practices” often fail, investing in rigorous experimentation and data-backed decision-making isn’t just an option—it’s a business imperative. As search and discovery become even more driven by AI, SEO leaders must move beyond intuition and embrace solutions that prove real value.

---

Memorable Sign-off:
“Remember, the answers are always in the data.”
— Jordan Cooney [end]

Transcript

[A]

The Voices of Search Podcast is a proud member of the I Hear Everything Podcast Network. Looking to launch or scale your podcast, I Hear Everything delivers podcast production, growth and monetization solutions that transform your words into profit. Ready to give your brand a voice then visit iheareverything.com welcome to the Voices of Search Podcast. A member of the I Hear Everything Podcast network, ready to expedite your company's organic growth efforts. Sit back, relax and get ready for your daily dose of search engine optimization wisdom. Here's today's host of the Voices of Search Podcast, Jordan Cooney.

[B]

I'm Jordan Cooney and joining me is Will Critchlow, CEO at SearchPilot. Search Pilot is the leading SEO AB testing platform for enterprise sites and they just launched the industry's first geo testing system purpose built for AI search. You've made a big bet on yourself launching Search Pilot. You've been in the consulting space and in other parts of the search, search, marketing, now, AI discovery industry. Why are you betting on testing and why is testing so important right now?

[C]

So we started building what became SearchPilot because of our consulting work. Actually the origin story is that we by that point had a decade of experience in the industry and had worked with a lot of these large organizations and we just seen that we found the same problems over and over again. We found the challenges getting things done, challenges proving the value of the work that they'd done, challenges even understanding whether there had been value in the work that they'd done.

[D]

Time for a one minute break to hear from our sponsor, Pre Visible. So you're looking for SEO help and you got a couple of options. You could start replying to spam from agencies that claim they can get you to rank number one on Google. You can pay an hourly rate for a consultant who will inevitably nickel and dime you with hourly charges. Or you can work with a cookie cutter agency to quickly launch a strategy less project with low success rate. None of those sound very good now do they? Well, that's where Pre Visible's integrated consulting model comes in. Pre Visible draws From a collective 40 years of SEO and digital marketing experience to unlock your organic growth opportunities. They build custom solutions that combine strategy, technical expertise, content and reporting to effectively operationalize SEO for your business. Pre Visible's four stage approach ensures that your SEO programs thrive by starting off with a strategy first approach. Then they support you in your efforts to create quality content, help help you identify technical issues, and most importantly, they'll work with your cross functional teams to integrate your SEO strategies to make sure that your SEO budget actually drives results, not just your agency's bottom line. So join brands like Yelp, eBay, Canva, Atlassian Square, all who rely on the SEO consultants at Pre Visible. For more information, go to PreVisible IO that's PreVisible P R E V I S I B L E.

[C]

We saw this in our consulting work. We saw it in their own work, their in house work as well. And that was the problem that we set out to solve. And since there we've kind of followed our nose a little bit to kind of say that's, that's what we're trying to do. Do we feel like we're achieving that? And particularly over this last two or three years, it really feels like we're making some breakthroughs. And this is why we've honed in on where we specialize, because that's where we're finding that we're adding the most value. And I'm very excited that the, the growth of all of these new discovery channels that we've been talking about today is bringing this topic to the conversation, into the boardroom, into the C suite, into the executive layer in a way that it hasn't been really for, for a long time. And that's proving good for us. And so, yeah, I think it's, I think it's important because it's, the more it changes, the less you can rely on yesterday's best practices, the more it's a black box and the more that these algorithms are unknown and unknowable. The only way to operate, in my opinion, is to test. And you've got to basically stop guessing because, yeah, it's not a case of a set of ranking factors that we could speculate about like we could in 2009.

[B]

And that wraps up this episode of the Voices of Search podcast. Thank you to Will Critchlow, CEO at SearchPilot, for joining us. If you'd like to contact Will, you can find a link to his LinkedIn profile in our show notes or on the voicesofsearch.com you can visit his company website, searchpilot.com if you haven't subscribed yet and want a daily stream of SEO and content marketing knowledge in your podcast feed, hit the subscribe button in your podcast app or on YouTube and we'll be back in your feed every week. Okay, that's all for today, but until next time, remember, the answers are always in the data.