Podcast Summary: The Exit Five CMO Podcast

Episode Title: Research-First CRO: How Data Leads to Better Marketing Decisions with Haley Carpenter
Host: Dave Gerhardt
Guest: Haley Carpenter, Founder of Chirpy (CRO Agency)
Date: September 15, 2025

Episode Overview

In this episode, Dave Gerhardt hosts Haley Carpenter to dive deep into conversion rate optimization (CRO) from a research-driven perspective. Haley unpacks why data—not just experimentation—should be at the core of all CRO efforts. She covers fundamental concepts, best practices for teams of all sizes, tool recommendations, embedding a testing culture, cross-pollination of D2C and B2B tactics, and avoiding common pitfalls around metrics, reporting, and company buy-in.

Key Discussion Points & Insights

1. What Is Conversion Rate Optimization? (CRO)

• Clarification: CRO stands for Conversion Rate Optimization, not Chief Revenue Officer (03:39).
• Core Definition: “You are optimizing conversion rates on a digital experience, whether that be transactions, forms, MQLs, whatever you have going on.” – Haley (03:39)
• Oversimplified / Overcomplicated: CRO is usually mistaken as just A/B testing, but actually breaks into two key buckets:
  • Testing/Experimentation
  • Research (user, UX, market, brand, etc.)
• Key Point: Successful optimizers start with research and use those insights to inform later testing.

2. The Case for Research-First CRO

• Start With Data, Not Assumptions: “The research bucket is often very overlooked, not talked about, not known about. So it’s really important to start there.” – Haley (05:24)
• Types of Data:
  • Quantitative: What is happening (numbers).
  • Qualitative: Why it’s happening (words, perceptions).
  • Behavioral / Perceptive Data: What users do vs. what users think and feel.
• Journey-Focused Approach: CRO should consider the entire user journey, not just optimize isolated sections.

3. Where and How to Start CRO on a B2B Website

• Custom Approach: "This is why I do custom scopes for all my clients..." (11:09).
• Broader Than Websites: Applies to apps, SaaS products, anywhere with a codebase you own.
• Priority Based on Data: Testing should be guided by analytics—volume and proportion of conversions dictate if/where you can test.
• Low Traffic Scenarios:
  • Haley’s testing threshold: ~200,000 monthly users, but depends on specific context and conversion rates.
  • Alternatives for Low Traffic:
    • User Testing: Preference tests, session recordings, qualitative interviews.
    • "Some data is better than no data." (16:27)

4. Building a Culture of Testing & Data-Driven Decision Making

• CRO as an Operating Philosophy: “It's really like an operating philosophy or a decision making philosophy, not just relative to a website or an app, but really as a business.” (18:40)
• Getting Buy-In: Start within marketing, evangelize results, then expand. Cross-functional teams benefit most—results often inform product, sales, customer service.
• Evangelizing & Reporting:
  • Slack/Teams channels to share test results and learnings (24:29).
  • Regular reporting: weekly, monthly, quarterly, both tactical and strategic.
  • Newsletters, business reviews, knowledge bases for research/test insights.
  • “Research...bumbs me out, a lot of times it's done and then it's just a slide deck that goes to die.” (27:23)

5. Scrappy Ways to Get Started with Research and Testing

• Tools & Tactics for Beginners:
  • Microsoft Clarity: Free, behavioral insights via heatmaps and session recordings. (31:05)
  • Hotjar: Paid alternative, highly recommended.
  • Immediate value: Watch real users on your site to spot friction/confusion.
    • “Just go watch 10 people hang out on your website and go poke around.” (32:44)
  • Winter for B2B messaging/value proposition testing (35:28).

6. Lessons B2B Can Learn from D2C/E-commerce

• Clarity in Messaging & Value Proposition: D2C excels here due to simpler funnels and more direct product-offer fit. B2B often gets lost in jargon and complex navigation.
• Foundation Over Features: You can't out-test weak messaging or positioning.
  • “If your messaging sucks...there's just nothing that's really going to fix that.” – Dave (38:17)
• Research is Non-Negotiable: Teams are often surprised how differently customers perceive value.

7. Social Proof as a CRO Lever

• Near-Universal Impact: “Every time social proof doesn't exist somewhere or it's very minimal and we do some research, it's called out, people notice, it immediately dings your credibility…” (43:01)
• Be Strategic: Too little feels inauthentic; authenticity and raw customer testimonials beat hyper-polished production.
  • “People want the real. They want the authentic. They want the genuine. They don’t want...the crap your marketing team created and put a veil over.” (48:03)

8. Common CRO Blockers & Mistakes

• Ego & Organizational Inertia: “No, we know best” mindset blocks progress. (48:50)
• Budget Constraints: True CRO at scale isn't cheap, but scrappy wins are possible at lower investment.
• Resource Limitations: Success needs buy-in, collaboration, and someone internally to champion the work.

9. Reporting and Metrics: What Matters

• Prioritize Bottom Line: Conversion events (revenue, form fills, MQLs) are primary. "The bottom line metrics need to be primary. That’s what we’re going after." (53:02)
• Avoid Vanity Metrics: Reporting bounce rate, pageviews, or engagement clicks as wins is not enough.
• No Excuses for Not Measuring Impact: "Not an excuse, completely unacceptable excuse. Yeah, yeah. That’s just it. That. It’s just not acceptable." (56:21)

10. Memorable Real-Life Testing Insights

• Surprise Results: Removing images from nav links on an ecomm site boosted transactions by 7%—counter to expectations. (51:15)
  • “I thought that would lose...it got a little over 7% lift in transactions. I was so surprised." – Haley

Notable Quotes & Memorable Moments

• “Start with data, start with research. The most common methodology is analytics, but research is so much more than that.” – Haley (05:24)
• "CRO is a system...you evangelize those results...CRO always works when it’s done correctly in some amount of time." – Haley (20:27)
• "If you’re not testing at all in user testing...what do you have? Usually nothing." – Haley (15:36)
• “If your messaging sucks, if you’re not well positioned, there’s just nothing that’s really going to fix that.” – Dave (38:17)
• “People want the real. They want the authentic. They want the genuine.” – Haley (48:03)
• “If you’re gonna go to your C-suite…and say we decreased bounce rate by 20%… who gives a flying shit? That’s not going to justify [the investment]…” – Haley (54:16)

Timestamps for Important Segments

• 01:13: Episode intro, meet Haley, CRO definition
• 03:39: Buckets of CRO: Testing vs. Research
• 06:44: Why research, not just testing, is the first step
• 11:09: How to choose what section of your website/funnel to focus on
• 14:03: CRO for low-traffic sites: alternative approaches
• 18:40: Data-driven culture as an operating philosophy
• 24:29: Reporting and internal evangelism tactics
• 31:05: Scrappy CRO wins: Heatmaps, session recording tools (Clarity, Hotjar)
• 35:28: What B2B can learn from D2C (messaging, clarity, value prop)
• 43:01: The undeniable value of social proof
• 48:50: Biggest blockers: ego, budget, resources
• 51:15: Shocking test result: removing images from nav boosted transactions
• 53:02: Danger of prioritizing vanity metrics over true conversion outcomes

Final Takeaways

• Start with research & data; testing comes after you understand what needs to be fixed.
• All teams, regardless of size or traffic, can and should embrace some form of research-driven optimization.
• CRO is a business-wide discipline and should be embedded in the culture—not just a one-off project.
• Effective reporting and storytelling are crucial for buy-in and scaling a CRO program.
• Focus on strategic, foundational improvements (especially messaging and social proof) before chasing advanced tactics like personalization or feature tweaks.
• Always tie your work back to business impact—the bottom line matters most!

For more insights from Haley or to join the discussion, connect with her and thousands of marketers in the Exit Five community.