Marketecture: Get Smart. Fast.

Episode: Meta Advantage Plus and Incrementality: Findings From 640 Tests
Host: Ari Paparo
Guest: Olivia Corey (Houzz)
Date: December 8, 2025

Episode Overview

This episode presents an in-depth walkthrough of a comprehensive research report from Houzz, led by Olivia Corey and team, analyzing the true incremental value of Meta’s new AI campaign type, Advantage Plus, compared to manual media buying. Drawing on 640 incrementality tests run over 18 months, Olivia shares findings, methodologies, and practical takeaways for marketers navigating automation, attribution, and outcome-focused strategies in digital advertising.

Key Discussion Points and Insights

The Evolution of Meta Campaign Management

• Manual Days (2015): Buying ads was hands-on—constant audience testing and creative tweaks.
• Now: Major shift to automation where Meta and Google take control, including creative generation.
• Crucial Question: Should marketers still resist AI-driven automation, or is it better to fully embrace these tools?

Why Incrementality Matters

• Incrementality Definition: The difference between conversions caused by ads (incremental) and those that would have happened anyway.
• Anecdote: Olivia shares a personal speaker-buying scenario to illustrate non-incremental vs. incremental conversions.

  “Meta is just hammering me with ads. But I was already going to buy. … Those are the conversions on the top. That is an ad platform reported conversion that Meta will take credit for. But it’s not an incremental conversion.” (06:00)

Houzz’s Incrementality Testing Methodology

• GeoLift Holdout Tests:

  • Standardized across channels—evaluate Meta, Search, TV on the same footing.
  • Run using aggregated, privacy-compliant data ("sales by day, by zip").
  • Temporarily turn off ads in select regions and measure business impact.

• Measurement Nuances:

  • Includes not only DTC (dot-com) sales but also retail and Amazon to capture full omni-channel lift.

• Defining Metrics:

  • Incrementality Factor: Ratio of truly incremental conversions vs. reported by platform.
  • IROAS (Incremental Return on Ad Spend): Only revenue that wouldn’t have happened anyway.
  • PTW (Post Treatment Observation Window): Tracks delayed or lagged effects after test period.

Main Findings from 640 Tests

Meta’s Impact Is Undeniable—But Nuanced

Meta delivers, but there are caveats:

• Meta comprised 77 of the top 100 “lift” experiments—likely a function of high advertiser spend.
• Average 20% lift: Turning off Meta would, on average, decrease a brand’s KPI (revenue/new customers) by 20% over the test period.

  “What this means is that if they turned off Meta, their business level revenue would come down by 20% overnight or over two or three weeks.” (09:24)

• Impact is more immediate vs. YouTube or CTV. 96% of tests saw significant lift by midway point.
• 32% lift in omni-channel sales (Amazon + Retail) for brands with at least 25% non-DTC business.

Attribution wrinkles:

• Click-based attribution under-reports Meta's true impact by ~15%.
• Click + view model(s) over-report impact—so reporting depends on agency/team settings.

Advantage Plus AI vs. Manual Campaigns: The Unexpected Results

Advantage Plus (Meta’s AI automated campaign tool):

• Only wins 42% of head-to-head tests against manual campaigns.
• 12% lower IROAS and 18% lower spend than manual—a surprising finding.

  “I was certain that Advantage plus was going to outperform...And we didn’t see that.” (11:54)

Manual campaigns:

• Outperform on lagged effects—greater lift appears later (PTW), especially for high-consideration products.
• The algorithm may be “too good”—targets people already likely to purchase, so not always generating true incremental lift.

Not a Binary Verdict:

• "This is not an indictment of ASC [Advantage Plus Shopping]. … For 42% of the brands in the study, Advantage Plus actually did outperform Manual." (13:58)
• Brands shouldn't just copy findings; testing for your own business is critical.

Signal Engineering: A Trending Tactic

• Definition: Redefining the optimization event for Meta’s AI (e.g., optimizing ad delivery for "add to cart" or "site visit" rather than "purchase").
• Aims to drive prospecting (new customers) rather than just bottom-of-funnel (BOFU) conversions.
• Testers saw 121% increase in testing frequency of this approach.
• Early results:
  • 14% lower IROAS vs. purchase-optimized, but at 85% lower daily spend. Hard to compare apples-to-apples, but signals stronger omni-channel and long-term impact.

Notable Quotes & Memorable Moments

On the Discrepancy between Platform Reporting and Real Business Outcomes:

“We as marketers really should be striving to drive incremental outcomes. This is the age of outcomes.”
(06:30, Olivia Corey)

The “Boomer Buying” Joke:

“Manual campaigns is what someone lovingly referred to as boomer buying. But this is the old way... where you think you can outsmart the machine.”
(07:41, Olivia Corey)

On Why AI Sometimes Fails to Outperform:

“Is the algorithm actually too good? ... It’s actually targeting these people who are already going to buy? It’s like circling the bottom of the drain.”
(12:42, Olivia Corey)

On the Need for Testing Individually:

“Do not take this as gospel. You really need to test for your own business.”
(16:56, Olivia Corey)

Audience Q&A Highlights (17:37–26:31)

[17:37] Pattern of Success for Advantage Plus

• Smaller brands do better with Advantage Plus; less need to “outsmart” Meta.
• Split strategy (50/50 manual and Advantage Plus) worked worse—brands should “go all in” on one approach.

[18:47] Longer Purchase Cycles & Manual Campaigns

• Advantage Plus is better for quick, high-intent buys.
• Manual may win for products with longer consideration cycles due to more lagged effects.

[19:37] Meta’s Own Incrementality Attribution Product

• Just released; results “mixed.”
• Still early stage, not yet as robust as legacy optimization:

  “You have to hope and you have to assume that it’s going to get better over time and it will get to parity.”

[21:19] Impact of Andromeda AI Updates

• Recent AI system changes (like Andromeda) could mean more recent data may be more favorable for Advantage Plus; data not yet split to confirm.

[22:29] Creative and CTR Correlation with Incrementality

• Houzz focused on back-end outcomes (CPA), not CTR on creatives—still an open question if optimizing for CTR muddies incrementality measurement.

[23:20] Time Window’s Impact on Results

• Meta effects are quick and immediate, little lag relative to channels like YouTube.
• Advantage Plus leads to quicker lift than manual.

[24:21] Signal Engineering in Practice

• Brands allocating ~20% of budget to signal engineering/mid-funnel; if it works, they shift more funds from bottom-of-funnel tactics.

[25:32] Longer Tests and Patience

• Longer, 6-month tests would better reveal mid/upper-funnel value, but marketers rarely have patience for long windows.

Actionable Takeaways

• Meta is powerful, but it’s “not set it and forget it.”
• Advantage Plus is not always the winner—test head-to-head for your brand and goals.
• Signal engineering is rising—consider testing mid/upper-funnel optimization to break out of the “lowest hanging fruit” trap.
• Track omni-channel impact (including Amazon and retail), or you’ll miss a significant share of results.
• Attribution settings critically affect perceived channel value; understand and align your measurement to business outcomes.
• The landscape is changing fast (AI/algorithm updates); keep your testing and strategies up-to-date.

Timestamps for Key Segments

• [01:39] – Introduction from Olivia Corey
• [03:06] – The evolution from manual to automated ad buying
• [06:00] – Why incrementality matters (speaker anecdote)
• [07:41] – Old (“boomer”) vs. new (AI) campaign setups
• [08:54] – Core methodology definitions (IROAS, Incrementality Factor, etc.)
• [09:36] – Meta’s undeniable impact and attribution nuances
• [11:54] – Advantage Plus underwhelms; manual campaigns sometimes better
• [13:58] – Not a simple answer; importance of brand-level testing
• [15:00] – Signal engineering and new strategies
• [17:37] – Audience Q&A (smaller brands, split tactics, longer purchase cycles, Meta’s new attribution, Andromeda AI, CTR, lag effects, signal engineering spend allocation)
• [25:32] – On the value and challenges of longer incrementality tests

Conclusion

This episode offers vital, data-backed insight into the complex interplay between automation and true incremental business growth on Meta. Marketers are reminded to measure real-world outcomes, not just platform numbers, and to actively test across automation, optimization events, and attribution windows for the best results.