AdExchanger Podcast Summary: "For Incrementality Testing, It’s One Step At A Time"
Release Date: December 17, 2024
Host: AdExchanger Talks with James Hersher
Guest: Olivia Corey, Head of Strategy at House
Introduction and Guest Background
The episode opens with James Hersher, AdExchanger's senior editor, introducing Olivia Corey, the Head of Strategy at House—a startup specializing in ad measurement and incrementality testing. Olivia brings a wealth of experience from various roles across the online ad supply chain, including positions at Starcom, Tubemogul, Netflix, Quibi, and Sonos. Her diverse background sets the stage for an in-depth discussion on incrementality testing and its evolving role in the ad tech landscape.
Notable Quote:
"I've been very lucky to be introduced to incrementality measurement pretty early in my career."
—Olivia Corey [02:44]
Understanding Incrementality Testing
Olivia Corey delves into the concept of incrementality testing, differentiating it from traditional attribution models. She emphasizes that unlike correlation-based attribution, which merely identifies associations, incrementality testing seeks to establish causation by determining what would have occurred in the absence of a specific marketing intervention.
Key Points:
- Causation over Correlation: Incrementality testing uses counterfactuals to assess the true impact of advertising efforts.
- Methodology: Comparable to randomized control trials in healthcare, it involves creating treatment and control groups to measure the effectiveness of campaigns.
- Types of Incrementality Tests:
- User-Level Experiments: Platforms like Meta and Google can assign users to treatment or control groups to measure conversion lift.
- Geo-Testing: Aggregates data across geographic regions to evaluate campaign effectiveness without relying on user-level data.
- Natural Experiments: Observes business changes following interventions without controlled groups, offering more directional insights.
Notable Quote:
"At its core, incrementality is the idea that we have the counterfactual to understand what would have happened anyway."
—Olivia Corey [10:57]
Challenges with Current Ad Tech Platforms
The conversation shifts to the complexities of relying on major platforms like Google, Meta, and Snap for incrementality testing. Olivia highlights several challenges:
- Trust Issues: Advertisers are often skeptical about platforms evaluating their own performance.
- Data Privacy Concerns: Post-iOS 14 changes have made it harder to track user behavior, complicating user-level experiments.
- Standardization Problems: Different platforms employ varying methodologies, making cross-platform comparisons difficult.
- Reliance on Aggregated Data: Geo-testing offers a privacy-friendly alternative but may lack the granularity of user-level data.
Notable Quote:
"With platforms like Meta and Google, they are getting better at targeting users who are in market to convert, making it even more important to validate incrementality independently."
—Olivia Corey [14:46]
Case Study: Netflix's Ad Tech Evolution
James brings up Netflix’s reputation for sophisticated data science and marketing operations, questioning whether their foray into advertising compromises their established standards.
Insights:
- Initial Approach: Netflix partnered with Microsoft to rapidly deploy its advertising platform, prioritizing speed over a homegrown solution.
- Future Direction: Olivia speculates that Netflix aims to internalize its ad tech operations to match its high standards, moving towards a more algorithmic and experimental culture akin to its product development.
Notable Quote:
"We thought, what if we could enable access to that level of experimentation... but do it as SaaS for less than the cost of a headcount."
—Olivia Corey [04:08]
Emerging Trends and Future Outlook
Olivia discusses the current state and future of incrementality testing, touching upon:
- Platform Limitations: As platforms enhance their targeting capabilities, distinguishing genuine incremental gains becomes more challenging.
- Diversification of Channels: Brands are increasingly seeking to diversify their ad spend beyond traditional giants like Meta and Google to mitigate risks and explore new opportunities.
- New Entrants: Applovin's recent expansion into the e-commerce space is examined as a potential new player offering viable alternatives for incrementality testing.
Notable Quote:
"Geotesting allows us to bypass user-level data issues by looking at aggregated sales in regions with and without treatment."
—Olivia Corey [09:01]
Deep Dive: Applovin's Foray into E-Commerce Advertising
A significant portion of the discussion centers on Applovin's transition from mobile gaming advertising to e-commerce. Olivia shares insights from early experiments conducted by House with Applovin:
- Initial Performance: Early tests showed promising attribution performance, aligning Applovin’s results with key channels.
- Ongoing Questions:
- Scalability: Can Applovin maintain performance as they scale?
- Customer Acquisition: Is Applovin effectively attracting new customers, or merely capturing repeat purchasers?
Notable Quote:
"In our experiments, Applovin is bringing in repeat purchasers who have engaged with the brands before, raising questions about true incrementality."
—Olivia Corey [27:20]
Impact of Regulatory Changes on Ad Platforms
The episode also explores recent regulatory shifts, particularly Meta’s new restrictions on health, wellness, and beauty advertising. Olivia explains how these changes affect advertisers:
- Conversion Optimization Limitations: Brands in sensitive categories can no longer optimize campaigns based on conversion data, impacting their ability to fine-tune ads effectively.
- Need for Diversification: These restrictions push brands to diversify their ad strategies and explore alternative channels to maintain performance.
Notable Quote:
"Brands are getting really creative with campaign optimization in the absence of conversion events, often facing efficiency hits but potentially reaching more diverse audiences."
—Olivia Corey [38:29]
Expert Insights: Stephanie Patterick on Ad Tech Evolution
Post-ad break, Stephanie Patterick, GM of Editorial at The Trade Desk, joins the conversation. She discusses the maturation of the ad tech industry, driven by consumer behavior, regulatory scrutiny, and innovation.
Key Points:
- Shift from Niche to Vital Player: Programmatic advertising has evolved from handling remnant inventory to supporting premium content.
- Regulatory Impact: Increased scrutiny from bodies like the Department of Justice is forcing greater transparency and value addition within the ad tech supply chain.
- Educational Initiatives: The Trade Desk Edge Academy emphasizes the importance of education in fostering a healthier and more innovative ad tech ecosystem.
Notable Quote:
"More people are spending time on the open Internet than in walled gardens, presenting a massive opportunity for ad tech to innovate with quality content and transparency."
—Stephanie Patterick [21:25]
Conclusion and Future Directions
The episode concludes with a forward-looking discussion on the state of ad technology and the importance of incrementality testing. Both Olivia and Stephanie emphasize the necessity for continuous experimentation, diversification of ad channels, and robust measurement techniques to navigate the complex and evolving landscape of digital advertising.
Final Takeaways:
- Incrementality is Crucial: Establishing causation in advertising efforts remains essential amidst increasing platform sophistication.
- Diversification is Key: Brands must explore and validate new advertising channels to ensure sustained growth and effectiveness.
- Education and Transparency: Ongoing learning and transparent practices are vital for adapting to industry changes and leveraging new opportunities.
Notable Quote:
"As brands diversify and regulatory landscapes shift, having a solid incrementality testing framework ensures that marketing efforts are truly driving incremental business value."
—Olivia Corey [39:02]
Additional Resources
For those interested in enhancing their knowledge of programmatic advertising and incrementality testing, The Trade Desk Edge Academy offers a range of expert-led courses and certifications. Enroll for free at edgeacademy.thetradedesk.com to stay ahead in the ever-evolving world of digital advertising.
This summary aims to encapsulate the key discussions and insights from the podcast episode, providing a comprehensive overview for listeners and non-listeners alike.