Loading summary
Transcript119 lines
- [00:00]
A
How do you operationalize your entire organization around incrementality and just this idea of being an incremental first marketing org and having that mindset and that approach with everything you do.
- [00:10]
B
We ran 58 tests last year, which is like more than one a week. The reason we're able to do that is we run just basically four separate lines of business right now. Between wallets, rings, travel, tech.
- [00:21]
C
How do you do that? What enables your team to be so effective at that and testing that fast? Like what is the culture? What have you said to them? How is it set up? I'm very curious and a lot of people would love to learn from that.
- [00:30]
B
We have moved to a place where I'm actually way more interested in just getting directional reads between strategies and channels. We're not trying to get in like a peer reviewed journal.
- [00:39]
A
Right.
- [00:39]
B
What matters to us is just improving the incremental roas.
- [00:42]
A
That's all incremental really means, right? It's additional. You would not be getting that outcome or that audience or that conversion if you were not doing that tactic.
- [00:50]
C
Teams that ran a certain number of experiments per year had like 17% lower CAC and it's just like duh, just an always on program where you're just constantly testing things.
- [00:57]
B
It's not all about incremental revenue. It's not all about iroas. If you identify any sort of event that you're optimizing for, you want to be measuring that as an incremental thing. And basically any channel can participate in that line of thinking.
- [01:07]
C
Show me it did something. You don't have to hold everything to the same. I think that's where it's a lot of brand spend is just wasted or not held accountable.
- [01:14]
B
I mean so this happened over the course of maybe the last like five or six days. McDonald's from like a marketing perspective is trying to create like founder led content. So they've got their CEO doing taste tests of the new burgers. He did the Big Arch sometime last week and then there was one earlier this week where he compares the McChicken Big Mac to the classic Big Mac. And he's just, people make fun of him because he's like, he doesn't feel it natural on camera. It doesn't look like he's a nerd.
- [01:41]
A
He's an, he's an absolute nerd. Is, is what the, what the take is?
- [01:45]
B
Well, I was joking that he doesn't, he doesn't, he doesn't quite strike the like guy I'd want to have beers with but he completely strikes the guy. I'd want to be running a quarter trillion dollar company. Like, absolutely. The guy's absolutely dialed in. And you could tell, not great for UGC content.
- [02:02]
C
But yeah, I was gonna say that was, that was definitely not a yapper.
- [02:05]
B
Not a yapper?
- [02:06]
A
Yeah.
- [02:06]
C
No, not a yapper.
- [02:07]
A
No.
- [02:09]
B
No. So, so he, he posted the video like a couple. And I also think it's funny. He's got like a, a specific profile. Chris K. Underscore McD is like what he posts from.
- [02:19]
C
So they have like, they have a
- [02:20]
B
dedicated profile for their CEO to be posting from. So that doesn't go over well initially. And then the Burger King CEO comes out and he's like, he just looks way cooler. And this is actually a great lesson in like authentic organic feeling, you know, video content. He's in a kitchen versus like, you know, some sort of like boardroom with the CEO. He's in the kitchen, he's wearing the apron. Guy looks like he hits the gym a little bit more for sure and just looks way more casual about the whole thing.
- [02:46]
A
So he looks like a guy you would have. He, he looks cool. Like, this is a guy. I would, I would have some beers with the Burger King CEO for sure.
- [02:53]
B
For sure. But I do think also from like a, I do think from a sentiment perspective, it's turned around for the McDonald's CEO. I saw a lot. I thought I was really going to have to come to this guy's defense on, on Twitter over the last couple of days, but it felt like a lot of the Internet was doing it for me where they said, hey, this guy actually is being extremely authentic. He's like, this is his, this is his natural sort of, you know, talking patterns. And this is the literal way that he discusses the food. And so I think, I think people are rallying behind maybe a little bit of his, his oddness. Now, the two things that I'll say are, one, I had the, the big arch yesterday. Very good burger.
- [03:31]
A
We got a fun episode today. Before we get into it, thank you to the sponsors Motion House Prescient Rich panel. And after.
- [03:37]
B
So Sam. Foreign. Just dropped their 2026 creative benchmarks report. And it's been getting shared everywhere. Slack channels, LinkedIn, Twitter, sharing it in our private group chats. And it's great because everybody's been asking the same four questions forever. What is normal? How many ads should we actually be shipping? What is a healthy hit rate? And which formats really win? The report analyzes over 575,000 creatives from 6,000 advertisers and over a billion dollars in ad spend to answer these exact questions. And the report has some really interesting findings, like the fact that only 4 to 8% of ads actually become winners and over half of ads actually lose. And for Motion customers, this report is especially helpful. You can upload it into your Motion dashboard with their runneth AI chat and compare it directly against your vertical benchmarks. Hit the link in the show notes. I promise you won't regret it. And as always, go to motionapp.com and tell the marketing operator, you
- [05:04]
A
all right, let's get into it. So we were talking a little bit the last few days about this idea of operationalizing around incrementality. We talk about House. Obviously House is an incrementality tool. And you know that when people think about incrementality, I think that's what they generally jump to is all right, let's do something to one set of markets or DMAs, let's not do something to another set and then let's measure the delta. Which is definitely like the best way to run an incrementality test. However, this incrementality approach is. It's not silo, just like running a GEO based holdout test. So we were talking about how do you operationalize your entire organization around incrementality and just this idea of being an incremental first marketing org and having that mindset and that approach with everything you do, not just when you're setting up a GEO based holdout test. So Cody, I'll kick it to you first. How do you bring an incrementality first approach to your entire marketing org?
- [06:04]
C
Yeah, I feel like there's a lot of different ways, you know, you can take it. I think number one is just like getting everyone knowing what your hierarchy of metrics are because there's so many, right? Like where does MER fall in? Where does MTA fall in? Where does mmm and just knowing like this is the only causal one. So it's only like the, like the real truth. And, and this is like what we're going to make decisions based on and just having a roadmap. But I just, I don't know. One of the things that like stuck with me, this was like a meta, meta analysis that I said with like Harvard or something. But it was like teams that ran a certain number of experiments per year had like 17% lower CAC and it was just like duh, like just an always on program where you're just constantly testing things and having the, I think the right mentality or philosophy, which is like very hypothesis driven and led versus like, like you're almost trying to be truth seeking and you're, you're trying to have more questions and answers and kind of using incrementality, incrementality to guide that like in everything you do. Like I think, you know, we don't want to be up and like, oh yes, this is what we're going to do. This is like what it's like always open minded, humble. And I think like that operationalizing like the, almost like the culture around it I've, I've learned is really important. So I think that's like the first place to start.
- [07:15]
A
The way that I've thought about this is letting that like letting, letting this incrementality approach like guide the KPIs that you are looking at. Because if you go in, if you, if you go into it without an understanding of incrementality and you're not looking for that, that's where I think you start looking at the wrong KPI. So I put a few examples here in the call doc. So one of the things that we are, we're looking at right now is we have a big meeting with, with TikTok today. We've invested a lot more in TikTok shop in the last six months. We're going to continue to invest there. So one of the questions we're asking ourselves is, hey, is this a net new audience that we're reaching and that we're getting to buy through TikTok shop or are we just cannibalizing our, our existing sales? So one of the things we're looking at is, well, what's the, what's the demographic data on TikTok Shop vs Shopify? Because if they're different, then chances are like it's an incremental audience. What we found is no surprise. The, the TikTok shop buyers, we have 11% of our audience that buys through Shopify is 18 to 34 years old. But on TikTok shop, that percent is 32%. So almost, almost 3x the percent of that demo. So that's like an incrementality first approach to understanding. Like, hey, are we driving incremental orders from, from different audiences on these channels? And the answer is yes. And that's giving us the confidence to, to go and invest more in this channel. And we think that it's additive retention. Yeah, go ahead.
- [08:39]
B
Can I. Yeah, a point of clarification here. 11 of Hexclad Shopify buyers are 18 to 34. And then you did an analysis of hexclad tick tock shop buyers and found 32. So you're just way over indexing in the younger audience on TikTok shop.
- [08:54]
A
Exactly. Yes, exactly. So we're saying, oh, wait, good, this is great. We're reaching someone new that we're not reaching on our Shopify funnel through the TikTok shop funnel that we have. So we're going to keep going and, and continue investing there.
- [09:07]
B
Can I actually, I want to make one more because this is, this is an interesting one because when we talk about incrementality a lot, we say geo lift studies. We run an experiment to measure incrementality and we talked about this at the Meta Performance Summit last year, where there are. Our ultimate goal is to be driving incremental outcomes. You won't always be able to run an experiment to truly measure and observe those incremental outcomes. So, like, what you just described is a way, it doesn't prove incrementality. You're not necessarily like, you didn't run an experiment and observe incrementality. But this is an extremely strong data point that gives you some conviction that they are incremental results.
- [09:45]
A
It's giving me conviction that the audience is incremental. And then I'm leaning into the growth in revenue we've had through TikTok shop to kind of fully connect the dots and say, you know what? I'm based on these data points. Yeah. Without running a holdout test, I'm confident that this is an incremental tactic reaching a new audience. And that's all, that's all incremental really means. Right. It's like it's additional. You would not be getting that, that outcome or that audience or that conversion if you were not doing that tactic. Right. I mean, that's, that's fundamentally what it means. We've been doing a lot of this in, in retention as well. Like our, our director of retention, Noah, has done such a good job of embracing this incrementality approach to the point where now like he comes to me and like he'll be, oh, I was testing this thing. It's been like, you know, we meet every two weeks and he'll be like, oh, I've been testing this thing in the last two weeks and we ran a holdout and here's the result. So we've been doing this on, we use this like AI personalization partner where we're doing like custom copy and subject lines and custom product recommendations. And we ran a user based holdout where we just split that audience and said hey, we're going to send some of these people through this AI powered experience, some of them to the static experience and then measured lift across those two cohorts. We're doing it with some like identity resolution partners where you know, we're basically saying hey, take that audience and put them in their own flow. So we're not mixing it up with any other flows. So yeah, we're, we're starting to take this and like apply it everywhere which is really exciting to see the team embrace it and like learn like realize that that is the way to pitch me, pitch everyone and like prove out the value. So those are, those are just two examples that, that I have top of mind that we've been leaning into recently. How are you thinking about this at your org, Connor? Like how are you. Let me rephrase the question. How are you bringing an incrementality first approach to your entire marketing org?
- [11:28]
B
Yeah, it's, I have an extremely similar answer to yours and that's actually why I made the distinction around geo lift testing versus finding the data points and thinking through ways to justify or you know, identify what may or may not be incremental. Because on one side the like extremely clear cut example of operationalizing incrementalities to make sure that we're running a lot of tests like what Cody said earlier where it's like hey, let's just we, we've got our multiple categories, we've got our house revenue streams for each of those. We're building out a testing, testing roadmap for each of those. We are constantly executing behind that. And then on the other side, retention is a great example where I think email SMS programs have never thought about incrementality. I mean they're extremely high ROI from like a cost perspective. So I think people are less conscious. You know this is like a total kind of BS number. But Klaviyo always says you get a 35x ROI from Klaviyo and it's like yeah, when you're getting like a 1.3x from Meta, ensuring you have incremental outcomes on Meta is significantly more important than if you're getting something that could potentially report as 35 to 1 on, on Klaviyo but that doesn't make a difference. So we, so retention is the same thing. Whether it's identity resolution, if you're testing something like Elevar Cookie enrichment or we're working with Ty right now or doing retention.com or whatever, all these Services that allow you to send more emails in many ways. It's easy for them to report on high revenue, but you should be setting it up internally so that you can observe what is it doing in addition to what would have otherwise already happened. So that's a big one. But retention's another tricky one because just because you sent an additional email or someone clicked an additional email still doesn't mean that it was necessarily an incremental outcome. What you really want to be doing is like, like a user level holdout or something. I, I don't think that as an industry we've really like, set up the infrastructure to measure, truly measure and observe incremental impact from retention. But right now I'm perfectly, perfectly comfortable with just giving ourselves the challenge of hey, let's set this up, let's get extremely comfortable with the data, let's go through the extra turns and the legwork to just give ourselves a little bit more information as to whether this might be incremental. And it doesn't necessarily need to be a perfectly executed scientific experiment. That's where we're at right now.
- [13:51]
A
Yep.
- [13:54]
B
Lately every market I already talked to says the same thing. Budgets are tight, goals are higher than ever, and I have to prove what's working, not just report it. And that is the new reality of marketing. If you can't afford to rely on guesses or platform reported results, you need clear causal proof of what's actually driving growth. And that's exactly where House comes in. Incrementality testing is the new scientific way to measure true impact, to see what's moving the needle and what's just noise. So you can reallocate spend based on fact and not faith. Cody, Connor and I all use House for incrementality testing, and it's become a core part of the modern measurement stack. They're now working with more than 40 of the top 100 DTC brands, which shows just how quickly this approach is becoming the new standard for serious growth teams. House helps you run real experiments across your channel so you can answer questions that actually matter, like which channels are truly driving incremental revenue? How much should I really be spending on Meta, Google or YouTube? And what's the halo effect of my ads on Amazon or retail sales? What sets House apart is the combination of rigorous science and marketer friendly design. The math under the hood is complex, built on causal inference and experimentation. But the platform itself is simple. You can choose your questions, launch a test in minutes and get clear, actionable results. You can actually use plus Every customer gets a dedicated measurement strategist, someone who's lived in the world of growth and knows how to translate data into strategy. They'll help you design smart tests, interpret the results and build a repeatable culture of experimentation across your team. With House, you aren't just buying a tool, you're buying the growth marketing team that can help you make the most of it. In a world where every marketing dollar is under a microscope, you need to know what matters with House by going to house IO operators, that's H A U S.IO/ operators and start allocating your budget with confidence.
- [15:33]
A
Well, I think part of it too is like marketers educating the technology companies and letting them know what we care about. And I think like Post Pilot has done such a great job of this. Like now they, it's native in their, in their UI where like any, like if you go and you're saying hey, I want to, I want to send a direct mail campaign to whatever, anyone that hasn't, that's ordered for me before but hasn't ordered again in the last a hundred days. Like they are by default holding out, running a user level holdout and holding out a small percent of that and then reporting on like an incremental revenue and IRO as number. And I think that's, that's like our job as, as, as marketers to like educate these like what we care about so that the tech tools can make their tooling as easy to use and easy to measure as possible. Like I, I love Post Pilot and we were testing their SMS sales with them for a while, which I think it worked well for us. But part of my issue was, is like oh, we're running a holdout. And I'm like, and it, and here's the iro I was like, come on, that can't be right. And then when I dug in a little bit more, well, they're running a holdout for like four days. Like guys, we can't, we can't like run a holdout for four days, not see the, the, the group that didn't go into your SMS flows, not convert and just say oh it's, it's incremental. It's like that's, that's not a long enough holdout to see if that, if that person that we didn't send into SMS sales converted. And like, you know, we are a unique brand in the sense like in terms of the, the bell curve on like consideration period and price point. But I think that's important and like that's Our, our responsibility like work with these companies to like ingrain what we care about. And I think, and I think post pilot like hurt us really well and they expanded the holdout only when we said hey we need to let these people be held out for a lot longer and give them more time to convert to get like a true IRO as read. So Cody, do you have any, do you have any like example like non geo based incrementality examples that you have been kind of like thinking through or working on with your team at Jones Road?
- [17:34]
C
I mean we, we are. I was, I definitely want to chat about GEOB stuff and more like the, the tactical media buying stuff. But yeah, I mean same thing. It's echo all Connor's points on retention. I think where it gets really important is when you have a variable cost behind it. So if you're doing direct mail, if you're doing like a SMS sales or shop or something like that where there's a variable cost like yes, if you want, if you. This is how I approach like incrementality is also very helpful not only internally on the org and making decisions and getting everyone understanding it but also externally. Any partner knows hey if you're going to be asking me for money like prove it with incrementality. Like this is what we care about and just very clear and firm about that. And so the best partners are great about that and we'll set up experiments like before we had data analyst in house like postscript did it for us and was able to proving commonality and build studies for us to do it. And like not all partners are going to want to do that because you know they're obviously it's not always going to look good so I always really respect that. Postpilot's been great about that as well. So like any direct mail we do is always going to be that. And the same thing with ad channels like Meta, Google, AppLovin, Pinterest. They all know like don't try to show me any other stories besides incrementality. You know like this is what we care about money where our mouth is. Like if, if you give us a good theory, a good mechanism, a good proposal, like we will test it and we'll align on a test together. But like we're not going to increase our budgets just because you know whatever in platform looks good or something like that. So I think it's a really, it's a really powerful thing for that to kind of like you know the way I vision I don't know if you guys saw that video of like, what's his name, the Palantir CEO with like the sword and stuff, that's how I feel. Sometimes you just always have people that are coming at you for money and you just have to like fight them away and like wear them off. And like incrementality is like really your best bet. Just because it's, it's so. It is. Everything has pros and cons. It's just so objective. It's so. It's like this was our number. That was not the number we need. How am I going to spend more if we're not hitting our number right? Build me a plan. So where, where, where I want to talk about. I think what is interesting is like, and Olivia talks about this on as well is like it's not just the running the test, but it's like how do you actually act on it, right? And like where I find the, where I don't want to say I've been struggling with, but where I'm trying, where we're trying to get better at operationalizing this on our growth team is like what I love about it is it's so objective, it's so causal and it's so simple. You get like two metrics, right? And in a sea of, you know, all of this data, you get two metrics. But then it's like, all right, you get a bad test, like why? Like then you kind of have to dive in, right? And like we're trying to take a first principles approach to really analyze why have a hypothesis. But I feel like what happens between the tests is actually probably just as important as the test. And that's the decisions you make, changing allocations, things like that or your follow up testing. It's not just like, oh, this was a static test. Meta was this. We're going to move on next. That's meta. No, it's like that was meta under those conditions. Let's analyze why. Let's go pull these levers based on a hypothesis. Let's go test again and let's go to. And I'm sure you guys do that really well, but I would love to hear like how you think about it, how you analyze, how you come up with ideas so it's not just spray and pray of like, oh, I'm just going to test this channel and test this channel next.
- [20:50]
A
Do you have any, do you have any examples, Cody? I would love to hear like what's an example of where you've like kind of strung together a series of tests In a intentional thoughtful way where like you had a hypothesis that led to test one based on the result, you got that that led right into test too. Like do you have. Because for us, like to answer your question, last year we were in, the year before was really like channel level holdouts and it was really channels that we were getting good signals on in our MTA impression and north beam that I also just had a hunch, had huge upside, you know, CTV, YouTube, like channels like this. And now if you go look at our 2020, like to answer your question, like, well, how does that play into your overall strategy? Well, if you go look at, at some of our year on year like total channel budget growth plans, the biggest growth channels in terms of net dollars are the ones that we got, you know, the best holdout results on. So that's, that's how that led into the next thing. But now we're starting to string together kind of some of the ways the things that you're talking about where as an example we tested view content last year for a long time, got a really good readout and now we're leaning into some of these other non purchase conversion objectives to see like how many can we stack on top of each other because we're reaching way more net new people with these tactics. So that's one example of how we're like taking that, that learning and then rolling into the next, the next batch of tests based on that. But how are you, what are you. Because I think you've been using House for, for longer than we have. So like how are you kind of stringing these tests together?
- [22:17]
C
Yeah, but I'm learning from you guys. Like I think we all have the same rep but like I know I've met, you know, met with him and like been like, oh, like this is how other people are doing it. This is how I like to like. So I'm always learning and trying to learn from, you know, Connor Dalton croons it runs a great program and stuff like that or something in like the bigger brands. I think what you described is probably like the best scenario you want to be in. It's like, hey, I think this is working. Let me go and try it again and spend more on it and see if it continues to work. Like that's like, that's like a scale test which is like amazing. Like that's what you want. It's like, hey, we think this is working. It's kind of not something that's going to look great in like an MTA or in platform like we think it is. Let's, we're there with like CTV right now. Like we have two good raids on CTV and we haven't really pushed like we got a good outcome in October part. And, and, and it's like, all right, we, let's, we want to go spend more everything. For me, that's like, what's the hypothesis? Let's not test for the sake of it. What's the hypothesis? What are we going to hope to learn and act on? Hey, we think that this is performing really well and we actually think we should be spending more. Like, perfect. That's a great actionable hypothesis. So I think some of the action actually starts with the hypothesis, but that's where you want to be with, with Meta right now. We've done this in the past and we're in this situation currently where Meta performance hasn't been what we wanted. And so that's a little bit harder and more challenging where you're like, okay, first I need to get a baseline right. It's been, you know, three months since I tested the channel, six months. A lot of things have changed. The good thing about Meta is at least for us, maybe not for hexcloud, but you. We can do like a one week test and get like a good. Yeah, all right. We ran it not where we wanted. Let's, let's call it 30% above where we need to be on our cost per new. Per new. But it's like in house you don't get that much data. You get right. You get incrementality, you get, you know, new versus returning. So at least like new versus returning the starting point. It's like, all right, is it exclude like, And I think there's just a lot of hypothesis, is it this? Is it that. And just go and dive in. This is where Claude has been great. Like my growth manager has been doing a really, really great job just diving in. And, and I, and I've done this in the past and kind of tried to be. Teach it, but it's like, all right, like let's diagnose this like first principles, like what in our account has shifted and changed over time. Like, let's build some dashboards and view stuff and it's like, all right. For us it always really comes down to it. It's a reach problem. It always seems like that. And so we needed a big change and so we just ran another one and we made a bunch of changes. I got this from Connor McDonald like a year ago where you guys, I think did a Lot of stuff. You added few content, you added video, you added exclusions. We just kind of had a bunch of different hypotheses and we did a lot at once because we needed a step change. So we. My hypothesis was that I think this over relying on attribution was hurting us and was really hurting reach. So we added a. We scaled up partnership ads a lot because we had a really good test on partnership ads. So we scaled up that. We added a new campaign that had partnership ads with Vue Optimization theoretically to drive more reels delivery. Just had some theories and hypotheses based on that. Our account was very heavy on VO volume and so we significantly switched it to CO conversion optimization. So we made like a bunch of changes, but they were all under the same thesis of like, we're not doing a great job reaching new people. And you know, obviously we talk about mid funnel, it's important, but I also think it's important to improve your CPMR and your reach even amongst the purchase objective as well. And so that was it. And we saw, you know, so like we're looking at those data, we're looking at North Beam mta. Like we saw our exclusions network as well. We saw our percent new going down. It's like there were a lot of signals that I think kind of told the same story. I guess that's what you're trying to look at and build the hypothesis. And then, you know, we tested it. Fortunately we were able to get, you know, numbers to where we need to be. Like, so we were pretty happy with it. It's not completely there. And so even now it's like, okay, we did analysis because we don't want to just get complacent and be like, all right, we're now where we need to be. Account's good. It's like, well, we moved, you know, our account to this much on co should have been higher. We have a one view optimization campaign. Should all of our campaigns be there? Like partnership ads are 35% of spend now. Should it be, you know that like. Right. Just like these different hypotheses that we've kind of gotten over time. And then I just think it's like, continue to roll up your sleeve. So that's kind of how we are approaching it. And sometimes it's like just attacking a channel and like, let's get four meta tests in a row. Let's test a bunch of different things based on hypotheses and then like, let's just roll first leaves and test them
- [26:35]
A
and Then all of a sudden you have a, a very optimized account in, in six months, hopefully.
- [26:40]
C
It's the, the hard part is we've done this a few times and then six months later it slips. It's like, sure, you know, that's, that's the challenge. But yeah, hopefully we're doing, we're doing the same demand gen right now. Like we, we've always had really good demand gen tests or YouTube in general. We scaled up in like October because performance looked really good, like seasonally post Purchase, North Bean, YouTube, whatever. So we scaled up probably way more than we should have, like probably tripling our budget compared to our test. And then we tested around there and it like wasn't great. And so we were like, all right, it's somewhere between the lower level of spend that we had and the higher level and we retested and, and just like not really where we want it to be. It's more repeat than we want, which is. You guys have heard me talk about that forever, but first time we've had that on YouTube. And so it's just like, it's the same thing. So it's like we just got on a call with House, we got on a call with our team, with our agency. You know, omg, it's just like, let's analyze it. And so the agency brought really good stuff to the table. It's like, hey, this is the change that we saw. Like, here's what was different from your past test. We had new launches in here, which we didn't do. We had, you know, we were optimizing for this. We should consolidate. So like, just, I think just trying to analyze, but like, I don't know, I think you just need like, intense, especially when things are not working well. Like, intensity to be like, I'm not going to get complacent. Like, I'm going to go really deep in the account, do some really thoughtful analysis, get a hypothesis, and then let's go test it.
- [28:06]
A
If you're still making budget decisions based on platform dashboards, you're not optimizing your gambling. This episode is brought to you by Pression AI, the measurement and forecasting platform built for scaling omnichannel brands. Because here's the problem. Attribution tells a story that's convenient, but not always the story that's true. Your best campaigns, they don't just drive direct conversions. They create ripple effects. They lift branded search, they drive organic traffic, direct traffic, and ultimately they improve performance across your entire funnel. Prescient is built to Capture the full impact of your marketing, including delayed conversions, cross channel effects, and the long tail performance that keeps working even after you've stopped spending. Where prescient really changes the game is forecasting. Because the real question isn't what worked last month. The real questions are what happens if we scale this campaign by 10%? Are we already saturated or is there still an efficiency pocket we're missing? If we cut spend, will performance hold or will it collapse? And are we balancing top of funnel and bottom of funnel the right way or starving one side of the system? With prescient, you can forecast at the campaign level, model different spend scenarios and understand the real relationship between spend and outcomes. So you stop wasting budget on outdated assumptions. Because a dollar in December is not the same as a dollar in August. So instead of defending your numbers in a budget meeting, you can walk in with a plan. And if you're ready to stop guessing and start forecasting, visit prussian AI.com/operators Pressure AI for performance marketers.
- [29:31]
B
So Cody, when you, when you're testing meta, like what's the split between doing a channel level holdout and, and like measuring the incrementality of meta versus you mentioned maybe at some point you'll be testing the percentage going to partnership ads. So like you wouldn't be getting a channel level readout necessarily, but you'd be testing which is which sort of mix results in the most in the best incremental roas. Um, like how often are you doing each of those tests?
- [29:57]
C
It's a good question. So tell, this is how I've done it. I, I was always, until like last year was under the impression that like, you know, to do proper testing you had to like very clearly control variables, test two things at the same time. Right. Like it had to be very, you know, strict. And that was like the benefit. And, and what Noah from House has like educated me on is like you don't necessarily have to do that. And it seems like they're like more comfortable with like subsequent tests. So like we will, like, I think you guys did this but like we will make a big switch in our account and just be like, let's test that. Like, and it's yes, you have some seasonal stuff because you're not doing two different things at the same time. But like we will, we will do that if that makes sense. So we might say, hey, you know, we think that, you know, conversion optimization instead of value optimization. Yeah, should be higher. Let's say we're at 5050 right now. We think we should be at 80% conversion. Like we'll just make that switch. We'll let it learn, maybe let it give a week of learn and then we'll launch a test. So it's like you kind of have multiple factors and maybe there's less precision, but I think that's helpful. I think if we really need more precision, like we ran a partnership test. I think it depends what you're trying to learn right now. We're trying to just like improve our total account and that was like most important. But it, but a month ago we wanted precision on partnership ads versus not because we had the hypothesis that partnership ads were essentially the same roas and North B maybe slightly worse, but driving better reach and that's probably more incremental and we should actually be pushing that harder. So like that was one that we wanted the precision and ran more of like a, A B against bau. The thing with that one is a little bit more nuanced. It was like a scaled up test. Like technically there was no region that was getting no meta at all. So like it was like directionally between those two cells was very, was, was fair, but it was like a very high cost per incremental because there was no like true holdout. But so we got the, we got the precision on that and then we took that into our overall larger test and that gave us confidence to really scale up partnership. Is that, does that make sense? Is that how you guys think about it?
- [31:54]
B
Yeah, well, I'll just, I'll just to parrot it back to you a little bit. You guys are doing a lot of consecutive channel tests. So like which to. For those listening, I, I think it's a perfectly like valid way to do it. That's what we did end of 24 into 25 was we just ran over the course of 12 weeks, we ran four tests or whatever and we were making changes over time. And what you're highlighting is the, the difficulty in that is like there's a seasonality component where it's like we ran our test between very early November and late February and there's just so much changing in that time. It's like you're going from big holiday to like Valentine's Day or whatever. Um, so that will like immediately skew like the, I'm going to say like quote unquote efficacy of the, of the experiment. What matters to us is just improving the incremental roas. Like we're not trying to get in like a peer reviewed journal.
- [32:46]
A
Right.
- [32:48]
C
Also to Noah's point, because when I Had some pushback. He was like, your KPI is so far from your target right now. Like, we're not trying to get a 3% improvement.
- [32:55]
B
Totally.
- [32:56]
C
You need like a 30%, like if you'll know. Right. If this work or not. Even without the perfect data.
- [33:04]
B
Yes, 100%. So I think that's a super reasonable way to do it. This is what Taylor Holiday and his podcast with Olivia and George Davis from Cozy Earth was really good. I would highly recommend it. George Davis, friend of the POD but he talks about progressive truth that when you get. Because I think a lot of people struggle with like, oh, I got an incremental row is for meta over the last four weeks. It's bad. What do I do? And it's like, that's not really the goal. The goal is just to set up in the case of Jones Road, consecutive tests so that you can make sure it's improving over time. Like, that's the ultimate goal. Ridge has gone through periods of time where we do that. We have moved to a place where I'm actually way more interested in just getting directional reads between strategies and channels and just saying, like, in the case of partnerships, I would, I would much prefer to run simultaneous cells that say, hey, this is business as usual, 100% no partnerships for the sake of this, this, you know, thought exercise. And then the other cell is 40 partnerships. And all we're going to look at, we're not even going to get an incremental roas because we have no true holdout. We're just going to say, hey, did which of these cells drove more lift? And that's, that's kind of where we're at now. And the nice thing about that, and that's just a, that's a different type of test. Right. It's like an A B test via a, a geolift, you know, measurement is that it's immediately actionable, one is better or worse. You can begin shifting budget to the thing that is more incremental. So we found that very valuable in getting clarity, removing seasonality and then making it immediately sort of easy to take action on.
- [34:40]
C
Yeah. And would you agree that like. Right, so it's like different strokes for different times in your business. Like you're probably in a better spot where you're trying to like fine tune your account a little bit. So like, I think that's the perfect thing for then versus like if you're needing to make some big changes, like, you kind of need to like go with a bigger approach, but like to get the precision and just like continually tweak the account and constantly get it slightly better. That's the way to go.
- [35:05]
B
Totally.
- [35:06]
C
Yeah.
- [35:06]
B
And I mean, the other big difference there is like when, when we run the AB test scenario with, you know, 40% of our account going to partnership ads, I have no need to compare that to a past result. All I'm doing is saying, like, in this time period and then moving forward, I want to find what's more incremental and we'll optimize towards that. What you were with the consecutive test and what we were doing in the same time last year was we wanted to make sure meta channel performance was getting better and like you kind of have to do that over consecutive tests and just make sure that you were building some sort of trend line in the right direction.
- [35:42]
C
Yeah, I think very, I think very directional. Especially if you're happy with the overall account or the overall business. Right. Like, you probably don't, you're not worried about that. And so it's like, hey, what's the best between it and. And I think that's where then also are you also like, you'll test that, you know, partnerships, whatever. But then you also might test the metaverse and the app love. And because you're really not trying to find like, hey, this, this channel I need to really improve. It's bad, but it's more of like, hey, where's my next dollar best spent? Let me get some directional.
- [36:13]
A
I, I think one thing I want to like summarize based on what we're all saying here is you often aren't testing like, I think most, most, most of the time you start testing against a true holdout. But then, but then moving forward, you're not, you're trying to find that like directional improvement because like our playbook has been, all right, let's do a full channel hold out. Like no spend in this channel versus our BAU spend just to like set that baseline. Great. If that looks good, then we move into a scaled up test. So now it's. And sometimes it's a two cell, sometimes it's a three cell. It's like, all right, let's, you know, cell A is BAU and then cell B, we're like simulating whatever 75, 50 scale up. So now we're saying, okay, like, is the channel incremental as a whole? That's check, check number one. Now like what spend level is it still incremental? And at that cost per incremental order, that's check two and then check three. Is often. Now let's go start to like tinker with some of these tactics. Like what happens if I run reach campaigns in YouTube or YouTube TV and now we're optimizing the channel that way. Like that's how we went about our view content test. Like it was not, it was not view content versus, versus like nothing. It was, it was our business as usual spend in both cells. And then we increased spend on top of that, cell A just had more purchase conversion spend and then cell B had the same amount of increased spend but in view content. So we're just trying to understand like which one got us the better, the better cost per incremental order here. And like where is our, our next dollar best spent? But we're not necessarily running it against like a true holdout. And like it's that directional read that you're, that you're looking for, right? Not necessarily like the total net read. You've already validated that hopefully earlier in a previous test.
- [37:51]
C
Do you guys have thoughts? So this was the, the cozy earth one and George was at the house cab for a little bit, so I chatted with him about it. But do you guys have thoughts about what you do when you're, you do get an actual holdout and like, let's say business is good, but you're like, you're not hitting your targets on, on this. Like, are you like, hey, it's just directional. I'm just like, you know, I know my business is good, so maybe there's some value here that's not being captured. Like maybe this is a hexcloud thing where it's like, hey, you know, a three week test. Yeah, I'm not profitable on a, you know, iroas perspective. But like I know that there's a lot more value created that just, it can't be measured in a, in a, in a three week.
- [38:28]
A
Yes.
- [38:28]
C
And are you then just comparing directionally? It's like, hey, I'm at least measuring the same across TV and YouTube and even though it's not first order profitable,
- [38:38]
A
it's still like, yes, what 100. So like in that, so that test that I, that I just explained, that was a three cell. Cell A was business as usual spend. Cell B was bau spend plus more spend into purchase conversion. Cell C was business as usual spend plus more spend into view content. So basically we don't have a true hold out. We, we are like quote unquote hold out are like comparison cell is just the BAU spend but because we're spending the Same amount and the other two cells just in different tactics we get a really clean like apples to apples IRO as read against cell A which again is just bau spend nothing else. So that's an example where if you look at just the IRO as it wasn't it's probably not profitable like net net but we get a really good directional read here and again we know that the our value captures does not is not contained within a a, and a half month test. So yes to your point, very directional. And I think Olivia would tell you and the house team would say the same thing that most of the time you're looking for directional reads not necessarily a net true end all be all read. Maybe unless you're selling like a 30 or 40 widget that's like super impulse buyee. But in this scenario we had, you know, you know it was like a $91 iro as on the view content versus a $78 on the more purchase conversion. I'm like well that's a, that's a great signal for us that we should go invest more into view content and probably not as much into purchase conversion. So and now we've, now we have that multiplier right between North Beam and what our IRO as read was. And now we've put a lot of money into view content. So and now we're moving into like what about video views, what about reach? What about you know, insert other non purchase? And that's how we're like stringing these together basically.
- [40:17]
C
So like, so that's a good, good example. Like will you hold video views to the same IRO escal that you'll hold purchase and, and not like I'm just making it up. Let's say you have to be you know, break even or profitable on purchase meta. Right?
- [40:31]
A
Yeah.
- [40:32]
C
Video views, does it get the same rigor? Right. Like obviously you'll, you'll run a longer test. Like will you compare that directionally? Or it's just like hey, we just need to know it's doing something but I just know that there's more value that it's creating. There's, there's you know, longer term stuff that like can't be measured in this test and like I'm okay with a little loss on this one.
- [40:53]
A
We're going to look at this test without a doubt, but with the caveat that this is running at a completely different time over a year later. We are going to set the test up the same way because basically the two events we want to Back into now are reach and video views. So we'll set the test up very similarly. Well, it'll probably be a three cell the same way that this one was a three cell and then we'll have again cell A bau Cell B and C will be BAU plus enhanced spend in these new tactics and then we'll be able to look at them side by side against that that sell a purchase conversion BAU spend. Yeah, we will come in here and look and like compare it to the view content results but it's not going to be like the number one thing we make our decision off of.
- [41:35]
B
All right, so I've got some interesting numbers to share from Ridge from our beloved sponsor Rich Panel. We switched our support stack at Ridge to rich panel about 15 months ago and our cost per ticket has dropped over 70%. That means same team, same volume and over $500,000 in annual savings. CSATs have not changed. We've been sitting at 96% week after week. So the automations did not come at the cost of customer experience. Our last platform talked a lot about AI but nothing was really changing under the hood. Rich Panel is genuinely AI first they came in, they rebuilt our workflows and we were live in under two weeks with basically no lift on our side. And they're about to roll out a returns portal which for us is huge because if they can do the same thing for returns as they did for customer service, that would be sick. If you want to cut down your support costs now and save on returns platform with an AI first platform, talk to Rich Panel. Go to richpanel.com demo tell them Connor from marketing operator sent you and they'll take good care of you. Thank you.
- [42:26]
C
Yeah, I'm very excited. That's how I've, I've done it with Reach. We don't currently have Reach going in our account or video views. I want to again maybe as we get some like new creative specifically for it but it's like hey I just need to know it's incremental but it's a little worse than here's how I feel currently with like ctd. So like Hana, I think we're running a three week test right now. We ran probably three week test before like very good efficiency. Let's say it's parity with Meta or something like that. Theoretically it's a more upper funnel channel. So like the same way we use roas lifted north beam, right? Like same roas but better percent new. We should put more spend there. Like I think the same thing probably applies for, for that.
- [43:04]
B
Right.
- [43:04]
C
Like there's probably some theoretical longer term benefits that TV drives. Right. And I think if you have some data on that or a hunch like like so my hypothesis, our team's hypothesis right now we think we're under spending on CTV relative to that because we have very similar IROs and I think we should accept actually a lower on ctv. So I'm curious like how you guys think about that. One thing I'm excited about is House is working on. I always hope I'm able to say this but like pulling in some like GA4 site metrics that you can actually see by geo to measure like a search lift. Because like that would be almost like my thought for like this is the new visits of a House test because like let's say you have the same CPA but but TV drives searchlift more, more effectively than a meta. Like I want to weight that up in my model and I want to spend a little bit more relatively. Like there's more total value captured there.
- [43:57]
B
You know it's funny Cody just I, I've said this before so this might come not come as a surprise and I know you're very anti brand search but if you are running brand search you could just break down impressions by DMA and then you can look at incremental brand queries driven by something like a CTV which might get you pretty close.
- [44:14]
C
Oh, have you done that?
- [44:15]
B
Yeah, a bunch.
- [44:17]
C
Yeah. I mean we just haven't found it to be incremental for us but that if House doesn't have this yet, maybe we do it just as part of a test. Just as part of a test to see that that's cool. Anything interesting when you've done that?
- [44:27]
B
Oh yeah, no, totally. Like I could pull for a future episode but like we've seen things like TikTok and YouTube Drive really significant increase in queries for rig wallet which is what you would expect as you move further and further into like where the, the value is generated from the view instead of the click. Like I think you just end up seeing different search behavior and different, I mean site traffic would be awesome to look at that as well. We've only been able to look at it via these Google brand search campaigns. But it's just another way to like help validate that something is happening when you're spending money on TikTok in these markets.
- [45:02]
C
Yeah. And then are you thinking like, like obviously that'll change your like click multiplier. But do you also feel like, hey, if like you're getting more searches in the same IROs. Like there'll be more future value.
- [45:13]
B
So we haven't gone like we've never looked at it as a key KPI. Right. Like even in all those instances we were still measuring like the tactic or the channel on an IROS basis. We just also will look at brand queries and. Cause it's just kind of an interesting extra data point. But we haven't actioned anything or made any like budget allocation decisions based on search lift.
- [45:34]
C
Okay. Yeah, I think like where, where I think it'll be really cool is having that plus the, the current house geolift of like Householder just to give like a little bit more nuance to it. Cause that's, that's the right. Everything has its pros and cons. I think the challenge is it's, it's, it's slow. You don't get anything back for two weeks and then there's not a lot of data. And so then you have to go and take that into your whole analytics stack and see what correlates. But having like a little bit more where it's still like causal I think would be very helpful.
- [46:00]
B
Yeah, let me ask, let me ask one like kind of in the weeds questions here, question here. Do you and Cody, I think you're hitting on this. Do you think there's a scenario where measuring the effectiveness of CTV requires you to also measure like purchase optimized campaigns on meta? And like another way of putting that is like for a long time we've always talked about like the quote unquote halo effect, which I think is like such kind of like, like a psyop at times by like these, these, these like top of funnel ad, you know, ad networks that like can't ever measure or quantify their impact so they just say hey, meta starts performing better. So if you believe in that, then theoretically you should never measure the channel that's creating the halo effect in and of itself that you actually need to look at the whole stack. I think that the, that, that the, the mix as a whole becomes more incremental with the two of them together. So do you guys think about that at all?
- [46:54]
C
I remember when I first like heard about incrementality stuff like people would explain to me and they'd be like, hey, you can like, yeah, you can, you can kind of run like stack tests and you can go and be like, hey, we're doing a podcast and at a home in this region or we're going to do just podcasts there. Like it just seems like they're like complicated tests. We haven't done it. I don't know. I would be very curious to get the house approach to that if like larger advertisers are doing that at all. We have not done it, but I think you make a good point.
- [47:20]
B
Yeah, it came to mind in the, in the Common thread podcast they talk about. Taylor was really talking about the importance of having an established. He calls it imr but let's just call it iroas for the sake of this, this conversation by channel and they're almost live like independent of one another. And you could just think of it as like a portfolio strategy of where you want to allocate the next dollar which is true like most of the time. Then I also wonder if there's not a more like compounding effect between strategies. Right. And it, what made me think of it Connor, was you talking about the view optimized campaigns along with purchase optimize that that sells more incremental. If you measure just the view optimize in and of themselves, I have no idea what that result would be that it might actually be the collection of them that becomes more impactful.
- [48:03]
A
Yeah, you don't have like a fair like where do I spend my next dollar? And that's ultimately what we're trying to answer in that test. Right. Should I put my next dollar into view content or should I put it into purchase conversion campaign? So if we would have set that up as a 2 sell where it was just view content like BAU plus view content against BAU only like yeah you get, you get a read on how incremental view content is but you don't know if more, if more spend in a purchase conversion would have been more incremental than than that view content readout. So that's why we've, that's why we've set it up that way. You know Cody, to your, to your question about like the channels that are higher funnel and like have a, a larger ROAS lift we've just been very thoughtful about when we run those scale up tests. So like we ran CTV in YouTube which you know YouTube does have our highest row as lift if you go look at north beam like we, we ran that test last year at the end of the year and then let the conversion or like the observation window run through our BFCM sale. So like we, we were just thoughtful about like we did the test during a demand gen time and then we let it run through the, the, the clearance period, the sale period and like that gave us a lot of confidence in that. But even that's still not the full, the full story of a channel that is that upper funnel for us. But I think that's how we've, that's how we've handled it so far.
- [49:19]
C
Yeah, no, that's, it's, I think it's really interesting. Yeah, I think you guys make good points on, on that.
- [49:25]
A
Well actually I want to, I want to ask Connor one more question on this. So, so Connor, last year you guys were in the top what, 0.1% of like CRO tests launched through. Or was it House? Or was it House or, or Intelligence? Both.
- [49:41]
B
Don't get, don't get it confused. Most House tests and what I, we were in the hundredth percentile of the intelligence test.
- [49:47]
A
So I think that's one amazing. Yeah, so I'm, I'm jealous of you guys because we have to run these holdout tests for pretty long periods of time. So like we can't run as much high volume tests. But I do like, I think we're running very high impact tests. So we are thoughtful about what we do decide to use those slots for. But I want to, I want to know, you know, you're a top, a 99.9 percentile house tester. What's on your roadmap right now? Or like what are you testing that you think is interesting and can have huge upside for your business? Like what are you testing? Why are you testing it? How are, like what, whatever details you can share on kind of what's in your current roadmap or what's live in your account right now.
- [50:22]
C
Can I ask a question almost more before you get into that? How do you do that? Like what, what enables your team to be so effective at that and testing that fast? Like what is, what is the culture? Like, what have you said to them? How is it set up? Because like I think a lot of people, I'm very curious and a lot of people would love to, to learn from that.
- [50:39]
B
Yeah, totally. No, and I was going to hit that because there are a lot of caveats to run it. We ran 58 tests last year, which is like more than one a week. The reason we're able to do that is one, we really run, as I've said many times, we run. Ridge is basically four separate lines of business right now. Between wallets, rings, travel, tech now. So we have all those revenue sources feeding into House so we can basically run concurrent tests. Those are functionally completely different brands. And then we also have revenue or Sorry. We have, we have other revenue streams set up for different markets. We can measure wallets in Canada and travel in Canada, things like that. So we just have a lot of surface area that we can run tests on. So that's number one. It's like literally if you're just measuring like your business as a whole, you, you literally cannot run 58 tests. Like it wouldn't be a good idea for anybody to do that. The other thing is we do skew on the shorter side of tests. We're running a lot of two, three week tests and that is specific to how we've decided to do it. We're largely at this point a B testing tactics, not looking for IRO as channel readouts, but more just what strategies are driving more of an impact and then shifting our budgets into those accordingly. And we feel we can observe the impact or the difference in impact of those tactics in a two or three week period. But there's nothing wrong, especially you know, for, for Hexclad and such a long customer journey, such a long consideration period. You guys should be running longer tests. I was at an event and they were talking about Purdy and Fig ran like a nine month test and it's like, yeah, we are, we are on, we are on the opposite side of that spectrum but depending on the brand and consideration period, like, like it's a whole spectrum.
- [52:23]
A
So I hope that was a high impact test they ran.
- [52:27]
B
Totally.
- [52:27]
C
Yeah, yeah.
- [52:28]
A
I hope they got it. It's like we just quadrupled our spend in YouTube or something.
- [52:32]
C
I think it was a brand. No jackalo. I think it was a brand test. I think it was like TV and podcast or something. Maybe not podcast. I mean it was TV and YouTube. Yeah. But I think it was like a long branding which is like very cool that he was willing to do that.
- [52:44]
B
For sure. For sure.
- [52:45]
C
Yeah.
- [52:45]
B
I mean there's something like equally impressive about having the conviction to do that. So anyway, that, that's, that's how we're running as mess as many tests as we are. We have media buyers across our different lines of business. So they are identifying like opportunities to test. A lot of those come from different thesises that we have internally. So like one of the next ones is actually percentage of budget going to YouTube. We continue to see success there. I think just based on our bias is basically we always skew meta dependent. Like I left to our own devices. I don't think we ever get meta to 40% of our budget. I think it always lands at 55 or 60 or whatever, just from comfort, basically. So like we're going to run a test where YouTube is 60% of the budget for one of our lines of business. We're going to see how that goes. So it just comes down to like, what, what? Depending on the line of business, depending on the buying behavior. I'll give you one more. I think I've talked about this earlier. We've been doing some great brand videos. We launched a product or a collection in January and we filmed this great brand video out in Hawaii. That's the sort of thing that never performs well. One of my goals coming into this year was like, hey, is there additional value to create by integrating brand first less Dr. Content into our mixes? So we just ran two cell test BAU with like no brand videos, all Dr. UGC, statics, et cetera. And then in the other cell, 20% was going to this brand video and we saw really meaningful lift dedicating 20% of our budget to those. So those are just the sort of things, depending on our goals and objectives, we can figure out whether we can run a test around it. Then we have enough kind of time and energy that we're able to get through a lot of those.
- [54:24]
C
Was this, this, this was a YouTube or meta. This, this brand video?
- [54:29]
B
So it was actually. So it was both. We had meta and YouTube in both cells.
- [54:35]
C
And was this standard campaigns purchase optimized or this was.
- [54:40]
B
They were. It's a great question. They were all purchase optimized. So we were just forcing spend on purchase optimized campaigns to the brand video in like new campaigns.
- [54:49]
C
New.
- [54:50]
B
I don't know if it was a new campaign or a new ad set, but if we were to just load up the brand and I'm saying 20% of the dollars went to the brand video. It wasn't like we just threw it in there and like hoped metal would would spend on it. We actually forced 20% of the budget there and that's what we saw the lift in.
- [55:05]
C
How are your metrics? Like what did your meta metrics look like? What did your reach look like? What did your north beam stuff look like? Whatever. I'm so curious about this.
- [55:12]
B
Yeah, no, I'll have to get back to you. We can make it a test of the week. We haven't done one in a while. Unlike the MTA data, we saw a 40% lift. So it was 40% more incremental in the cell that had the brand videos, which is. That's a massive win.
- [55:27]
A
And it was a true holdout. It was just Like Cell A has these videos. Cell B does not have these videos. And you lumped both YouTube and Meta into Cell A. Got it.
- [55:36]
C
Yeah, got it.
- [55:37]
B
So then there's, then there's questions like, oh well it was it like, we don't know if it was meta or YouTube. We had 20% of each channel's budget going to the brand video. We don't know which one actually like drove more or less of an impact. So if we wanted drill down further there but like directionally that it's super actionable because we launch, you know, we have eight campaigns a year where we're getting cool brand videos. And now we've got like another tool in the toolbox for how we want
- [56:01]
A
to bring those to market.
- [56:03]
C
And will you continue to think about like, like will you then say, hey, maybe we should put this in a View campaign or something like that? Exactly.
- [56:10]
B
So, so then the next step was hey, let's, let's figure out, we probably have the content available to us already. What would be a more evergreen strategy here? So like let's build a beautiful like just brand first video that's not tied to like a seasonal colle and just see if we can unlock some of that value on an ongoing basis.
- [56:27]
C
I love it so much. I was, I was listening to like Kat Cole on, on, on operators and she's talking about like, you know, shifting their stuff, full funnel, whatever. So we have, we have, I would say like our really first like hero video, like true brand product launch like in May. And so we'll have like some really good assets like that. And I've been thinking about that and like I think we'll do a big test even. Like again we're, we're going to want to do like an awareness test. Like I might work with YouTube or Meta to do like just like a, A you know, awareness lift test as well. But I think something like that is like, is like how do we make the most use of these assets? And like, especially with like sequence learning and Gem, it's like, hey, not every ad has to be even, even a purchasing. So I love that you did that. That's awesome. Have you seen this guy, Phil, I don't know his name on X, but he talks about adding like you can add like custom columns. That's a custom event. That's like a to meta. That's like a Google referral or something like that. Have you seen that?
- [57:17]
B
We have that set up.
- [57:18]
C
Yeah, yeah.
- [57:19]
B
So you can like we, this is another one we have on the roadmap. Is we want to try. I like and I don't know if good will come of this or not, but like I like spending more time on other sort of signal engineering. So one of them would be optimizing for brand queries. So we have a Google Tag Manager event fire for someone coming from Google. So if we optimize for that event in meta and we say hey, what we the event that we want to be creating is people coming to us from Google. You're essentially optimizing for a brand query. And I think we'd just be hitting potentially a different audience that way or at least a different part of the funnel.
- [57:52]
C
That's cool. We, we should do a whole signal engineering episode. I feel like we could have like so many just like ideas for like what you could potentially do totally
- [58:02]
A
Operators quick gut check here. Q1 is when everyone realizes the same thing at once. Traffic gets more expensive, growth slows down. So the question isn't how do I drive more demand, it's how do I make more money from the demand I already have? That is exactly why we use After Sell by Rocked. Most brands think upsells are about being aggressive, but they really aren't. They are about timing. And the best time to upsell someone is when they're already in buying mode, which is when it's right after someone buys. So after sell it lets you put the right offer at the right moment with one click. There's no reentering payment info, there's no extra checkout steps. Brands using after sales see around a 30% lift in AOV. And when you're running real volume, that adds up fast. But here's the part most people miss. It's not just upsells that After Sell ads. Once you're live, you unlock the entire Rocked monetization suite. Rock thanks monetizes your thank you page with premium non competing offers. Think Disney plus hello Fresh and brands are seeing $0.30 to $0.50 in pure profit per order Rock Pay plus as a clean wallet placement at checkout and kicks back another 10 cents to 15 cents in profit per order without hurting conversion. And in some cases it actually improves conversion. No inventory, no new ads, no operational lift, just margin. This isn't growth hacking, it's just found money. If Q1 is about tightening margins and getting paid more for the traffic you already earned, go to after sell.com/operators Activate Rock thanks or Rock Pay plus and you'll get the full after sales suite free for a year or an extended 60 day trial for post purchase Upsells So zooming out a little bit here then to like kind of summarize like the playbook for anyone that's like, hey, how do I even approach house? I think step one is just like channel level holdout, right? Like, hey, no spend versus spend. Is this channel incremental? Unless I think, you know, right. If you got to like 8 figures on meta, you probably can skip right to step two, which is then like scale up tests. It's like, great. Is the channel incremental versus no spend? Then it's like, how much can we spend and have it still be incremental? And then eventually you get into what kind of we're talking more about today, which is like optimizing the channel. Like let's show brand videos versus not show brand videos. Let's test view content versus more purchase conversion. And then, and then you kind of, then you stay in that, that third bucket for a long time until you're like, hey, I think we can spend a bunch more on this channel again. Then maybe you go back to, to step two. Do you guys think that's like a fairly accurate summary of kind of how the, the playbook plays out?
- [60:30]
B
Totally. You know, just. Cody hit one point that I think we could circle back to quickly. We said at the beginning, how do you operate? Not, not just operationalize incrementality, but how do you get the org bought in around driving incremental outcomes. And Cody was talking about awareness, working with Google for like measuring awareness and that's another great example that we're trying to focus on more this year is not necessarily driving incremental revenue, but just like incremental brand recall. Right. Like there's all edges. Any sort of event you want to be optimizing for. You want to make sure you're driving, you're driving it in a way that wouldn't have happened, wouldn't have occurred otherwise. So I think that's another way is like just saying, hey, it's not all about incremental revenue. It's not all about iroas that just if you identify any sort of event that you're optimizing for, you want to be measuring that as an incremental thing. And basically any channel can participate in that line of thinking.
- [61:20]
C
Yeah, show me it did something. You don't have to hold everything to the same, but show me at least it's not just, you know, I think that's where it's a lot of brand spend is just wasted or not held accountable. Same. Yeah, same thing for even like retail.
- [61:32]
B
Right?
- [61:32]
C
Like we now have 12 stores and so we'll pilot stuff at certain stores and we'll be like, hey, we're going to pick the. And like no, it's not a true holdout. But like, hey, that store is doing really well and we have like traffic counters at the stores. So like yeah, I completely agree with that.
- [61:47]
A
And like, and like make your team like prove it to you. You know, like I get the team comes to, to us all the time and says hey, I think we should do A, B and C. And here's why. Like part, part of building that culture of incrementality is in that like lens is just like the Socratic questioning of like prove this is incremental. Oh well look, here's the rp. It's like, but that doesn't prove it's incremental. Like, and here's why. So I think it's just like over time the team learns like, like what is and isn't an incremental, a data point that proves out incrementality. And I think that's, that's where like you just like start to get that buy in and create that like subconscious approach with your team. And now that's like just the standard. All right, so I want to, I want to summarize the overall playbook for running an incrementality program at your org when it comes to paid media. Especially step one channel level holdouts like like spend in this channel versus don't spend in this channel. Is it incremental? If you find out it is, then you move into the scale up test. All right, great. Can we spend 50 more? Can we spend 75% more? So BAU spend versus 50 increase or any increase in spend. You could do two different levels of increase and then you move into the optimizing of the channel. Hey, let's run brand videos versus not running brand videos. Let's run view content versus not running view content. I mean those are the, those are the three pillars of any incrementality program. If you're liking the show, make sure to like comment, subscribe and share with all your marketing friends.