E035: Black Friday Cyber Monday: How We'll Be Tracking, Analyzing & Maximizing Our Results

Summary

Marketing Operators Podcast: Episode E035 Summary

Title: Black Friday Cyber Monday: How We'll Be Tracking, Analyzing & Maximizing Our Results
Hosts: Connor Rolain, Connor MacDonald, Cody Plofker
Release Date: November 26, 2024

---

1. Episode Overview

In Episode E035 of the Marketing Operators podcast, hosts Connor Rolain, Connor MacDonald, and Cody Plofker delve into their strategies for managing the high-stakes Black Friday Cyber Monday (BFCM) period. They discuss tracking methodologies, budget allocations, team operations, marketing tactics, and the challenges of maximizing results during this critical sales window.

---

2. Setting the Stage for BFCM

The episode begins with Cody Plofker sharing personal updates, including his temporary relocation to Los Angeles to manage upcoming sales events and the logistical challenges of working remotely with his team and pets.

Notable Quote:

Cody (00:14): "We got a lot packed into November... Starting from Saturday, basically through the end of the year."

---

3. CPM Trends and Post-Election Impacts

The conversation shifts to analyzing Cost Per Mille (CPM) trends following the recent election. The hosts compare their observations with industry reports, noting a general decline in CPMs and discussing the implications for their marketing strategies.

Notable Quotes:

Connor (05:34): "We're down 20% from the previous period... Still up 67% year over year, which is crazy."

Cody (07:53): "Google told us retail has been weaker to start the holiday season... We're expecting a backloaded month."

---

4. Operational Workflow During BFCM

The hosts outline their operational workflows during the BFCM sale period. They emphasize the importance of daily standups, real-time data monitoring, and agile decision-making to respond to sales trends effectively.

Notable Quotes:

Connor (11:03): "We're starting a daily standup... Midday standups every day to talk about pacing and budget decisions."

Cody (15:53): "We're doing as much async as possible... Keeping an eye on efficiency and revenue pacing."

---

5. Budget Allocation and Channel Mix

Discussion centers around how the hosts adjust their budget allocations across different marketing channels during BFCM. They highlight the predominance of Meta (Facebook) in their media mix due to its scalability and incremental value compared to other channels.

Notable Quotes:

Cody (31:42): "We launched a couple House tests... Motion just raised $30 million in their series B."

Connor (36:48): "Our Facebook rep was hitting me up because our percentage of Facebook budget is as low as it's been all year."

---

6. Influencer Marketing and Upper Funnel Strategies

The podcast explores the role of influencer marketing and upper funnel channels like TV and CTV in driving net new customers. Hosts discuss the challenges of measuring incrementality and the tools they use to optimize these efforts.

Notable Quotes:

Connor (33:00): "We don't see much value out of influencer integrations this year... It's all about BFCM and holiday sale."

Unknown Guest (39:32): "Prescient AI has been a game changer... It helps us measure the impact across all sales channels."

---

7. Email and SMS Marketing Cadence

The hosts share their strategies for email and SMS communications during BFCM, balancing aggressive outreach with the risk of increasing unsubscribe rates. They discuss segmentation, frequency, and the use of plain text emails to enhance deliverability.

Notable Quotes:

Cody (44:00): "Two emails on launch day, one email every day, two on the final day."

Connor (45:34): "We'll share results on split testing send times... Ensuring our emails stand out in crowded inboxes."

---

8. Reactive Tactics and Inventory Management

Adaptability is crucial during BFCM, as unexpected challenges like inventory shortages may arise. The hosts discuss their approaches to adjusting offers, managing inventory, and maintaining operational flexibility to respond to real-time sales data.

Notable Quotes:

Cody (49:20): "Last year, we adjusted our exclusive offers based on early performance... This year, we're discounting kits to move excess inventory."

Connor (53:05): "Our biggest headache is managing stockouts in bundled products... We've prepared asset variations to handle such changes."

---

9. Tests of the Week: Innovations and Learnings

The episode features a segment where the hosts discuss various tests they've implemented to optimize their BFCM performance. These include CTV holdout tests, Rich Panel AI tools for comment moderation, and shipping strategy experiments.

Notable Quotes:

Unknown Guest (62:02): "CTV achieved a $68 cost per incremental... We're excited to see how it performs over the full test period."

Cody (63:30): "House has been instrumental in measuring channel incrementality... Their platform allows us to discover true ROI."

---

10. Lessons Learned and Final Thoughts

Concluding the episode, the hosts reflect on the importance of flexibility, data-driven decision-making, and team collaboration during the BFCM period. They emphasize learning from both successes and setbacks to continuously refine their marketing strategies.

Notable Quotes:

Cody (58:41): "Adjusting strategy mid-campaign is essential... It’s the right thing to do to hit our projections."

Connor (61:06): "Growth teams must remain nimble and recognize the competitive advantage of adaptability."

---

Conclusion

Episode E035 of the Marketing Operators podcast provides an in-depth look into the strategic planning and operational execution required for a successful Black Friday Cyber Monday campaign. By sharing their real-world experiences, challenges, and solutions, Connor Rolain, Connor MacDonald, and Cody Plofker offer valuable insights for marketers aiming to maximize their BFCM results through meticulous tracking, agile operations, and innovative marketing tactics.

---

Key Takeaways:

• Data-Driven Decision Making: Utilizing real-time data and advanced analytics tools (e.g., House, Prescient AI) to inform budget allocations and marketing strategies.
• Operational Flexibility: Implementing daily standups and maintaining agile workflows to respond swiftly to sales trends and inventory issues.
• Channel Optimization: Prioritizing scalable and incremental channels like Meta while experimenting with upper funnel channels such as CTV and YouTube.
• Effective Communication: Balancing aggressive marketing outreach with customer experience to optimize email and SMS cadences without driving high unsubscribe rates.
• Continuous Testing: Conducting various tests (CTV holdouts, shipping strategies) to identify and leverage effective marketing tactics during peak sales periods.

---

Sponsors:

• Motion: Enhancing creative analytics and advanced reporting features. Learn More
• House: Experimentation platform for measuring marketing incrementality. Discover House
• Rich Panel: AI-driven social media comment moderation. Explore Rich Panel
• Prescient AI: Cross-channel measurement and forecasting tools. Visit Prescient AI

---

If you found this summary helpful, please subscribe, like, and share the Marketing Operators Podcast to support the hosts and stay updated with the latest marketing strategies and insights.

Transcript

[Connor]

All right, boys, how we doing? Connor moved to la. My hypothesis that when the. When the big revenue numbers start rolling in, that he wants to be out in LA where. Where all the glitz and glam is. What's. What's the deal? What are you. What are you in LA for? Where are you staying? What's. Yeah, what's the story?

[Cody]

Yeah, well, I've got. There's a couple big things coming up. One, we launched our Black Friday sale today. We've got true Black Friday coming up. We got Cyber Monday. You know, I've got a birthday coming up. We've got Thanksgiving. All my family's out here, so it's like, we got a lot packed into November. So my wife and I were like, it's getting cold in Salt Lake. I'm like, okay, yeah, we could use a couple warmer months. So we're out in Atwater Village starting. I drove out on Saturday, brought all of our animals. So we got the three dogs and two cats out here, and we're out here from Saturday, basically through the end of the year. I think we're going to be here till, like, early the first week of January, something like that. So get super excited. And I got my whole. It's funny, like, a real relocation. Brought all the animals. I have my whole desk set up, so. Same laptop, same monitor, got the light, got the mic, the whole thing. So, yeah, we'll see how it goes.

[Connor]

Are you Airbnb?

[Cody]

Airbnb, yeah. And what's also fun and logistically, like, kind of interesting is on the property where we're staying, there's a goat. And the. And. And so we tried to introduce. We tried to introduce our dogs to the goat a couple times. Now it's not going well.

[Connor]

We got. You got time, you got a couple months. We got time.

[Cody]

Yeah, we got to come around. But. But, yeah, there's goats and chickens on the property, so it's like, it's a very interesting kind of environment to be in, in such a important, you know, sales period.

[Connor]

I know. I love. I love that you're fully embracing the digital nomad lifestyle and relocating for two months in the busiest time of year on. On the day that you launch. That's. That's. I love it.

[Unknown]

Yeah, we were. I was in LA probably three weeks ago, and I noticed the same thing as you did, though. The coffee shops don't open there till 8. It's crazy, dude.

[Cody]

Crazy. Yeah. The three closest cafes, me, didn't open till 8am And I've. I've, like, kind of railed against this in the past because I even think like 7:30 is too late. Even 7:00am I'm like kind of on the fence. I'm like, yeah, you're a really casual cafe. If you're not opening till 7:00am 8:00am Just blows my mind. Dude, that's, that's 11:00am Eastern. These people in Atwater Village, I don't know how they're keeping up. I don't know how they're optimizing their ad accounts if they're not drinking coffee until 8:00. Motion is a company that we've all been using forever. Hexclad Jones Road Bridge. We were all early adopters and now they're really gaining momentum. Motion just raised $30 million in their series B and a ton of new stuff is coming along with that. They've been pushing out new features. If you haven't checked them out in a while, you can book a demo and see all the new stuff that they've been shipping. But a few of the ones that really stood out to me 1. Creative research and Res was on our podcast, so he talks about this a bit. They want to improve the input into the creative strategist role. So one you have Creative analytics and two their new tool Creative Research, completely free where you can build swipe files from all your favorite brands. Connor, Cody and myself put together our own so you can start there and then add some creative from your favorite brands. 2. Motion released a new advanced analytics feature called Winning Combinations. Winning Combinations helps you instantly answer complex business questions like what ad concepts work best for specific product lines, what messaging themes are working across different influencers, what angles, hooks and concepts are working best across all ad formats. This will help you get real clarity from all your testing variables and it takes seconds to pull insights. Plus they have even bigger features cooking that they'll be releasing soon. So head to motionapp.com to book a demo and when you mention the Marketing operators, you'll get 50% off your first month. Or just start using the Creative Research tool as they are free. So an easy win for you and your team.

[Connor]

Hello marketers. Welcome to another episode of the Marketing Operators Podcast. And most importantly, Happy Black Friday. Hope everything is going well for you. We have an awesome jam packed episode today. It's all about how we spend our time, what we do, operations, meeting, cadences, tactics for when we actually launch our Black Friday sale. So what are the things that we're actually looking for and making decisions based off intra sale from the moment we launch through Cyber Tuesday Or Cyber Monday. Really jam packed. Really tactical stuff here. Thank you again to our sponsors, Motion, Rich Panel, Prescient and House. As always, if you're enjoying the show, please like subscribe and share. Guys, I thought it'd be interesting to chat about CPMs to kick off the episode. We were chatting about it last week. I think this is the first episode since the election's been. Oh yeah, it is. So I'm curious, do you guys. Have you guys been kind of following your CPMs and, and how they've been trending? What's that look like for your businesses right now?

[Unknown]

Put me on the spot. Let me, let me look real quick.

[Connor]

Okay.

[Unknown]

I was, I was looking pretty closely, like post election and then like, yeah, one or two days I did notice a drop and I'm like, I don't need to check CPMs every day now.

[Cody]

Yeah, yeah, I'm kind of in the same boat. I told you guys we didn't see much of an impact going into it. We haven't seen much of an impact coming out of it. At least on the CPM front. We had a super strong weekend. So that was. Election was the 5th, we're recording on the 12th. So yeah, the weekend of the. The 8th, 9th, 10th was like really strong for us and that's all we're really looking for. But for whatever reason, I always feel we're less affected by CPM. We just have a really consistent like 8 to $11 CPM that we get from meta.

[Connor]

Is that. Yeah. Interesting. Yeah, we were word. So we're down. We're down 20% from the previous period. So I was looking like Wednesday day after election through Sunday and then looking over the previous Wednesday through Sunday clip, we were down 20%. So that was promising. We're still up big year over year, but not as big. We're up 67% year over year in that same an L7 from today, which is down from where our year over year in October was, which was 85%. So still up, but crazy. Yeah. So I'm happy to see that it's a good time to see the CVMs coming down. I would've liked them to be down when we were, you know, building our funnel in the. In the previous four months. But I'll take what I can get. Cody looks like he's deep in Facebook ad Mode.

[Unknown]

We're about 12% down, so it's not crazy. But I mean, that's still pretty significant.

[Connor]

Yeah.

[Unknown]

So definitely saw a spike and seen it come down. We're not seeing the Same like, like I thought performance was going to get back up and kind of go crazy after the election. Election day was actually better. But you know, I think there's some other factors going on where our, our holiday offering is just not landing the way that we wanted. And that's obviously always super disappointing and, and frustrating. You know, we had a great October, you know, 40% above forecast. We're just below it on November now, but likely won't hit just with our, our holiday stuff not hitting. So we're going to scramble a little bit with our Black Friday plans to try to see if we can incentivize those, move some more of those kits. But yeah, it's, it's. I think people are definitely waiting now. I think there was definitely some CPMs up some fuzziness with the election. But you know, I think we've talked about it a lot but a lot of people, you know, Thanksgiving, Black Friday are so late this year that I feel like that just pushes the demand back. And so if people are normally starting to shop now, it just doesn't seem like it. It's like I don't know about you guys, but there's, there's still and I've been like sick this past week like over the weekend. So I was just on my phone way too much and like I would have thought that we went back to dc, dtc, Twitter, but it's really all politics still an election. And I would have just thought in my mind like hey, we ended this, people go back to their lives. But it seems like there's been just like still a pretty big focus and like a lot of mind share that's been taking up. So I think that coupled with Black Friday Thanksgiving being so late this year, it's just kind of pushing back the demand and it might be more of like a backloaded month. What do you guys think?

[Cody]

Yeah. So we got this note from our Google rep just today. So this is breaking news that will publish two weeks from now. But Google told us retail has been weaker to start start the holiday season. We estimate a combination of a weakening US consumer and the US election have led to a softer spending in Q4 quarter to date. So this includes a lot of October. Doesn't explain why JRB would beat by 40% and then back at forecast now does not necessarily mean holiday season at large will be weaker. Instead we believe that one the holiday season is starting three to four weeks later than the last several years post Covid and two, the holiday season this year will be peak year with fewer days between Black Friday and Christmas Eve. So that's what they're expecting, which is basically what we're seeing as well. Like we are trending very similar to last year's revenue. So it doesn't, it doesn't seem as if consumer spending is any different and it's hard to know for sure whether it'll be pulled back. But we can all hope for it. Google is as well.

[Unknown]

Yeah, that makes sense. I mean it's kind of nice to hear that it's obviously sucks, it's down, but that it is backloaded and they are expecting an uptick in a little bit.

[Connor]

Yeah, I think so. Ridge is live, Cody's live. I think hexcloud is the only, only three. One of the three mops not live. We launched Friday morning. So I'll, I'll definitely be shooting you guys some text updates. So far we're, we're pacing at. I'm, I'm. Hey, we got to have a place for them. We're pacing right at Forecast right now, which is very promising. Honestly. The same trend I was talking about in the last episode, we're like November hits and we didn't do anything differently. But revenues trending up, we're probably like 2.2to 2.25x daily revenue just evergreen basis compared to where we were at like the end of October. So seeing some good signals pre launch. But yeah, we're, we're gonna be refreshing Shopify often on Friday and kind of seeing where we land. Cool. All right, good transition. So we wanted to spend today. We've spent some time talking about different phases of BFCM production. We talked about like BFCM planning and production probably three months ahead of time. We talked about what are we doing when we're 30 days out. Now we're talking about intra sale. Like what's the kind of like operational workflow between us and our team? So there's like this totally different mindset right where pre sale you're being very generative, you're trying to build out the best funnels possible, the best offers possible. Then you hit like this intra sale moment and you know, for better or for worse, one of the beautiful things about E commerce is that you have this instantaneous access to data which allows you to be reactive in certain situations, which Cody already alluded to based on the sales data you're getting, which is, you know, simultaneously a good thing and a bad thing. So I think it's. That's what we're going to focus on today is like how does kind of Our mindset and operations and just what we're doing on a day to day shift when you're actually in, in the heat of it in Black Friday, Cyber Monday. So yeah guys, I guess where do we want to start off? There was a lot of notes here, right? We could start off with like the day of. We could talk about like any meetings or like budget scaling, campaign cadence, overall team collaboration. Is there any one area you guys would like to, to jump into, to, to talk about?

[Cody]

Well, let me, let me ask you guys this. At, at some point do you guys start doing, you start doing intraday reporting. So like hour by hour reporting and if so, when does that start?

[Connor]

We'll start that. We actually start our day. So what we, what we ended up doing is we ended up starting a daily standup which actually starts on. It starts tomorrow. I don't know why it should start Friday, but that starts tomorrow. And that's between growth team leaders. So myself and Cam, our leadership, Jason and Danny and then some of our finance team, Teresa, Niles, Jeremy. So we'll basically do those like midday standups every day and just talk about how do we pace against the previous day, how are we pacing today? Are there any budget decisions that we want to make based on that? But then like me, then the growth team itself, like me, Cam, London, Kyle, which is like kind of our ally, isn't our newest addition. That's kind of like the core internal paid media team. Kyle's a freelancer, but that's kind of the core like ad buying team. We are, we're constantly communicating like even more. Right. Like if one of us notices like a fat like hour hour bump in revenue and we're looking at like the intra hour spend and our CAC looks great and all the signals recommending like more spend is incremental. Like we're doing that all asynchronously but just making sure we have that, that real meeting time as well to like touch bases. But yeah, that, I mean that's happening non stop honestly from the moment, from.

[Cody]

The moment you launch. You launch Friday the 15th and then starting then you'll start making like almost hour by like you're really paying attention to hourly.

[Connor]

No, no, sorry, hourly sales stuff. That, that's really, that's really more Thursday through Monday, that's where we'll really be like touching base often. We're not really day trading ads until then. Maybe if we see like some crazy spikes above our projections, we might, you know, do a deep dive and decide to crank and then bring things Back down. Right. If it's like a random, you know, November 18th and we see like things take off at 6pm like sure, we might make some kind of like day trading type decisions in that moment, then bring it back down. But for the most part that's not going to happen until that the cyber weekend really got it.

[Cody]

Thursday, you said the day, day before Thanksgiving you'll start tracking like hour, hour by hour.

[Connor]

Yeah.

[Cody]

What about you guys? Cody?

[Unknown]

So what we normally do, just Evergreen Week, we look, we, we have a VA plus we'll, we'll look at some daily reporting. So we'll check pacing multiple times a day and try to refresh some dashboards just to make sure we're landing okay. Hopefully we don't make too many decisions and not making too many budget changes. But obviously we'll look to scale on a weekend if it looks good. So it's normally just done Async for these periods. I try to do as much async as possible. So we'll still have that where we have a va. We'll put in slack our pacing. That's essentially our mer amer and revenue growth compared to the previous day. So we'll monitor that just to see how we're pacing. In the past we've, we've always done or not always, but lately I've done a lot of like manual bids, big caps. We're going to go lowest cost probably all this year, which is big cats have not been working well for us. So you know, to do that we are going to be pretty aggressive with the budgets out of the gate where like that's one thing you don't have to kind of be as aggressive with the bid cap because you just only spend if you're getting conversions. But you know, for that being said, we'll want to have a pretty close eye on it. So yeah, we'll have a pretty close eye on it. So I mean we'll check it pretty often. I don't know how much we'll do the meetings. It's always been like we've done the sale where it's like four days and most of us are like with family and at home. So we kind of just do async and then we'll just huddle whenever we need to or call each other and be like, hey, what's going on? And we'll see. But we'll be in the office for a large part of us where we can kind of just come over to each other's desk and be like, hey, how's Pacing looking, can you pull it up? What are we thinking? And, and you know, so it's combination of that and looking at our, our efficiency, how is our revenue pacing? So we'll kind of do, you know, we'll predict and be like, hey, we are 40 above yesterday. Here's where we landed yesterday. 40 above that is this number. Here's where our spend is pacing. Like do we think we need to pull back? Do we need to push forward? And usually we'll try to be more aggressive if we think there's a chance of hitting it and then it's easier to pull back later. So I think that's a big, you know, component for us because we've got limited edition, you know, or, or exclusive shades. Like we've also got to be really locked in with our, you know, our supply and and demand planning teams to make sure we know how hard we can actually push based on inventory to make sure we have enough. It, it's probably fine. I think we buy pretty aggressively now. So that's another thing. In the past we've always had to huddle with them. How are we doing? How are we doing in this region? How are we doing in the shade? Hopefully that's not a concern this year. So it's, it's as much async like people are on even if it's you know, Thursday, Friday with families but you know, you're on but trying to do as much async as possible.

[Connor]

Yeah, I think the meet, like the budget change stuff is easier to do async when like the only variable you're talking about is like do we want to spend more or do we want to spend less? The ones that we have set up that are different than the daily standup I mentioned are, I don't know exactly. I think it's weekly. It's probably going to end up being more frequent when we launch. But we are meeting pretty regularly with our inventory planning team to understand like obviously like the growth team for the most part is like we're optimizing for top line revenue and inefficiency and a spend number. Of course we're talking with the inventory planning team to like help them dial in their like SKU by SKU forecasting based on like what we think is going to help us get to that top line number and that efficiency number. But we're doing a lot of, a lot of skew velocity meetings where we're talking about, hey, we're like, we're under, we're under our daily like recommended pace for this skew, like, what can we do to change that? Right? And that's when that's. That's easier to talk about on a call versus Async. Because anytime you're talking about, like, strategic tactics, I just think it's easier to collaborate in real. In real person or on a call or in, in person. So it's like, hey, we're, you know, we're at, you know, 300 per day here. We really needed to be at 500 per day. Like, what can we do to do that? Like, what can we do to move this, right? So it's like, all right, well, let's, let's maybe make the price point a little more competitive. Let's merchandise it more prominently on our product, on our collection pages. Let's do an additional email campaign around it. Maybe we do some, like, ads around it. So we're kind of having those a little less regularly, but also, you know, equally as important as kind of tracking that intra hour or intraday.

[Cody]

And that's kind of why I was asking, like, for us, there's like three kind of modes in Q4. One is like the more typical one where we make budget adjustments. Basically, Mondays will review the previous full week. So we'll say, like, did we land above projections? What do we look at at a channel level? How do we want to adjust budgets accordingly? We'll look at month to date reporting, etc. That basically happens, like, all year. Now we're in the point, and we have been for the last couple of days really. But now that we've officially started our Black Friday sale, where it's daily, but we're really not making that many, like, intraday decisions. And then week of, I would say it maybe starts the weekend before Black Friday is when we'll start looking at hourly reporting. Last year we started, we started hourly reporting that we'd set up. And I don't know how you guys would do it, but listeners might find it helpful. We didn't start doing it till maybe three years ago, but literally exporting hourly data from last year's Black Friday. And then we aren't able to get hourly exports from supermetrics. We just manually export every one to three hours. It'll get more frequent as we get closer to Black Friday. Someone's uploading that into a sheet. And then we're looking exactly how we're pacing. And that will basically start the weekend before Black Friday. And then we're doing it for a full, like 10, 11 days at that point.

[Unknown]

And those are you uploading Is that just revenue and spend or you uploading something else? Like revenue.

[Cody]

Okay, yeah, good question. So, so we're exporting. What I care about the most is revenue exported from Shopify. We're looking at that hour by hour, year over year. And then we're also getting spend from either a North Beam or Super Metrics can update for like up to the last hour how much we've spent. So we're looking at an intraday Me or Ame or something like that. So that's how we approach it. I think we're overdoing it, doing it for 10, 11 days. But that like Thursday, Friday, Saturday, Sunday, Monday is like essential. And I remember the first time we learned that when Black Friday morning of people understanding, hey, we're pacing 30% above what we were expecting or 20% below is incredibly helpful information to have. So I totally recommend everybody set up bare minimum that hour by hour, year over year, so you understand where you're like trending relative to question for you.

[Unknown]

On that because we'll, like I said we'll do and, and we'll refresh stuff pretty often to get that. But how do you know where you're going to land? Like, let's say you're looking at your pacing. You're looking hourly. Like, how do you, how do you see at 11am and like, what's your math to be like, hey, I think we're on track to hit our revenue goal. Do you, do you know where you should be certain hour you're looking at year over year and you're like, hey, this year we're, let's say we're looking to be 20% up year over year. Black Friday and by 11am we're up 20%. Is it that simple over last year?

[Cody]

And I think you said earlier, like all we would do, we don't do any fancier math and saying, hey, if by this hour last year we did 25% of our revenue, so we can expect another 75% of the day's revenue to come in between now and the end of the day. That's all we would do. We just extrapolate that out further.

[Unknown]

Okay. Yeah, that makes sense. Yeah. And so you guys have like the last year's data built out already because you've been doing this.

[Cody]

Yeah, we have a spreadsheet that got pretty crazy last year that we're going to refresh, but the team's working on that right now.

[Unknown]

No, but we've got a few spreadsheets like that. Even just our daily forecast versus actual. And it's so nice to have these documents that you have all this data in and you can look back a year from now and just be like what did we do on this day last year? Or even on this hour last year? And sometimes it's such a pain in the ass to build in the moment, but it's so nice to look back on these things and these spreadsheets that have the data and just be able to like easily compare to for when you're planning this year.

[Cody]

Did you know that platforms over report performance 65% of the time? In a recent study, house found that 82% of incrementality experiments showed that platform reporting was either underreporting or overreporting by more than 25%. And 60% of the experiments showed discrepancies of more than 50%. Is why marketers are moving away from platform attribution towards incrementality measurement in order to maximize their growth and efficiency. This is one of the many reasons that the three of us, Connor, Cody and I all work with House. So what is House? It's a self serve experimentation platform that allows you to configure regional tests and control experiments to measure incrementality and identify points of diminishing returns. What does that really mean? They tell you hey, These are the 40 zip codes you need to exclude from this campaign and we can look at what results did you drive in the targeted regions compared to that holdout group. Really awesome. We use it all the time. As an example, we just identified a lot of our non branded ring search and shopping campaigns were not nearly as incremental as Google as was reporting. So we've been growing a ring business over 100% year over year and spending less on Google. So we've been able to be more efficient with every dollar spent. House is built with cutting edge methods by PhD economists and data scientists who have built these solutions before at companies like Amazon and Google. The House platform allows you to test all your marketing channels both online and offline. Measure the impact across all your sales channels, dtc, retail, Amazon and calibrate your platform reporting for incrementality with House. Discover your marketing's true ROI and unlock new growth with House. Go to House IO operators spelled H A U S.IO/ operators to start your incrementality practice today.

[Connor]

Another way that we've looked at it is saying hey, we know that we had everything queued up to spend our full budget today and we know what our revenue target is, we know what our MER target is so it's like if we get to noon and it's like holy crap, our like we look like we're pacing to spend the entire budget and our mer targets like 25% lower than what we paced it for. Meaning your revenue is going to like definitely out, you know, beat whatever you had forecasted. The next question that we always try to answer and I, I tweeted this in the context of Prime Day 1 and it's the same application. What I asked was we did, we had a 10x MER at this revenue roughly on Amazon on Prime Day 2. How much of that revenue would we have realized if we didn't spend a single dollar on on ads that day? So the question is like okay, great, we're more efficient than we projected. Should we spend more? Like okay, we were shooting for a 4x MER and we're at a 6x but like is that next dollar of spend actually at a 1x like and then we'll go into like north beam and start looking at like net new visit rate in one day. Click in first time order revenue and post purchase survey to see what people are saying and like try to decide. I don't think we've actually gotten to a soup. Maybe there is no way to get to a super high confidence conclusive answer. But that's what we're always. Then it's like all right, great, we're beating target. We don't immediately spend more. Like we go look at CAC and am first to see how efficient acquisition is. But then again we still don't automatically spend more because chances are those people could have been in our funnel for a long time and now they're converting and it like it makes ame our look great, which is awesome. But do you spend more? Like what, what's your approach to, to that either of you, do you want to jump in?

[Cody]

I'll take a quick pass because I think we've been on both sides of this where we've ended certain holidays, like more efficient. I guess everybody does. You either end the holidays more efficient than you need to be or less efficient than you need to be.

[Unknown]

Yeah. No one's like hey, this was perfect guys.

[Cody]

Yeah. Yeah. Crazy insight, but yeah, actually last year I think we ended up on the, on the less efficient side that we were hitting our targets that we spent more. And I don't think we had. We didn't hold ourselves accountable enough to seeing the increases we needed to intraday to justify continue spending more. Because I think that's what you have to do. I think you ultimately have to say, hey, we are seeing, you know, this level of performance. It's above what we'd like. We'd like to spend more. If you spend more, you just have to be conscious you're not blending down your existing results, that you are actually, you should actually be increasing your growth rate. If you're going to spend more like.

[Connor]

Hour over hour testing where it's like, hey, okay, we decided to scale spend up by 25%. Did revenue stay flat? Did it also go up by 25%? Did it go up by 50%?

[Cody]

Yeah, that makes, I think that is ideal scenario. I think that's incredibly hard to do. That's what we're going to try to do this year is just hold ourselves really accountable to any sort of. It's really easy to wake up, be like, we're gonna do a couple million bucks today, awesome. We're gonna like just pump the budgets on Facebook. But ultimately we could be a lot more thoughtful than that.

[Unknown]

Yeah, no, I, I totally agree with that. I was gonna, I was thinking what you said a few episodes back of like just even evergreen, period, blending down spend. You know, it's not always that, that that extra spend is not always, you know, as incremental as you want it to be. And I, I've definitely done that during these periods. Um, I think in addition to that part of it depends on what you're optimizing towards. And I always try to be like super clear with the team on like what our goals are, what are we optimizing towards. And obviously it's, you know, cause some, sometimes it's revenue, sometimes it's profit, sometimes it's, you know, it's contribution margin usually is the North Star, but then you also have like inventory considerations. Like hey, we, for, for Memorial Day weekend, for example, like we had so much inventory and we weren't able to like really reuse it after it. And so for cash flow reasons, we, we were much more biased on the growth side. We were much less efficient. We still beat our contribution margin and EBITDA goals for that period because the volume was so much higher. And that additional volume allowed us to decide, hey, do we want to just. Because if you're above volume and you're above, you know, efficiency, like, like Connor mentioned in that example, you can decide, hey, do we want to just ride this and take more efficiency, just put more, you know, on the bottom line or do we want to ride this for growth, acquire more new customers, get more volume at less efficiency, move through more inventory. And so I think it depends. I think if you're, you know, in a. In a position where you don't have that much inventory, then you should maximize the margin of it and you should kind of pull back. But if you're in a position where you need to move inventory, you've ordered pretty aggressively or those cash flow considerations, or you. You're like, hey, we hit our. Our contribution goals. Like, we're good with that. Anything above that, we just want to spend into that. You know, I think it's just really clear for, you know, you to know and then all of our teams to know, like, exactly what we are optimizing towards. And the one other thing I would say, I'm more likely to want to spend more aggressively into that earlier in the period because theoretically, you can kind of recoup some of that value at the end, right? On Cyber Monday. I'm not going to be like, let's be as aggressive as possible because there's no lag to that spend.

[Cody]

Yeah, I guess you're right. I would even say that's hard to do just on Black Friday. Like. Like you're threading such a needle there by saying, like, hey, we are actually. We're fine with just moving through inventory. Like, you're not maybe unit economic profitable, but you're moving through inventory. Okay, those are all great. It's so hard to throw that needle there. And those dollars spent have so little value over the next 10 days. Like, if people aren't buying on Black Friday, at least what we see in our data, it's not as if we're prospecting someone who's going to purchase on December 10th. It's like that is your moment to do it. Such a high volume period. It's so noisy. I just think it's like, it's. It's really a tricky thing to nail.

[Unknown]

But you don't think so? Like, let's say we launch our offer on the 25th this year, right? We're gonna run it for eight days. You don't think that if you, you know, that spend from the first three days, some of those people might buy, like, Cyber Monday?

[Cody]

Totally. I mean, a little bit. Like, as you get earlier, as early as you can invest those dollars, the better once you hit, like, true Black Friday or true Cyber Monday, at least for me, I think that's. I think it's really hard to. To thread that needle, to measure it correctly, to know you're accomplishing the goal.

[Unknown]

You'Re setting so hard.

[Cody]

It's impossible and it's like infinitely easier to just give meta non incremental dollars and blend down all your results and just kind of light money on fire for a couple hours.

[Unknown]

Yeah, yeah. I think if I look back on all the years and we're inventory constrained some of them I think I've definitely thought we overspent on some days or during it. So it's very easy to overspend. I think we've always underspent leading up to it and sure, I wonder if that'll be the case this year. Have you guys ever felt like you've like over. Have you ever felt like you've overspent filling the funnel leading up to a Black Friday?

[Connor]

Yeah, yeah. I mean we did last year for sure. I mean just simply because we came in under our year end efficiency target that we were shooting for, which means like you know, de facto we overspent, which I think ultimately probably showed up mostly in like October, September because we were trying to see how low we could ride it because we had a very aggressive top line. But we also came in like, I don't know, less than 10 below our efficiency targets. We were pretty darn close and we, we beat our revenue targets for the whole year that we had set in, you know, Q1. So it was not a big deal. But yeah, I mean we definitely overspent last year. We, we look a lot at post purchase survey trends in terms of like consideration lag. Like if we're intra sale and we're really popping, we'll go and kind of see how, how is that less than one week, less than one day answer trending? Like if all of a sudden the number of responses we're getting there doubled, we might scale more like because chances are if we're above, if we're above efficiency and revenue projections, chances are we are going to increase spend. It's just a matter of how much. And then that's when we're going and saying like all right, what's the one day click row as what's the percent of net new visits? What's the percent of first time orders? We'll also then go look at post purchase survey and see how that less than one week, less than one day answer is trending. And then usually between those two data sources we can say hey, we're going to, you know, we're going to scale spend up just 10% or we're going to scale spend up 50% because our, you know, one day click row as like double day over day. And we think it's all net new visitors and our number of people that are saying they heard about us in one day or seven days also doubled. Like we'll kind of try to stack those data points to, to figure out if that, that that additional spend is incremental or not. It's, it's very much like a, an artsy driven science though it's not perfect by any means.

[Unknown]

Does, does your guys channel mix change from the period leading up to it to during promo period? Like are you, are you more aggressive on one channel or are you increasing budgets on all of the channels equally?

[Cody]

Well, I'll tell you guys I'm stoked on this, but we launched our Black Friday sale today. We strategically launched a couple house tests along with that. So we're going to get those midpoint readouts just before the week of Black Friday so that we have these additional data points around the incrementality of the channels. We did that for Meta and for YouTube. So I think with that information it'll dictate because otherwise it's, that's incredibly hard to say. Also we start on the 12th, on the 25th. Hey, we need to increase budgets 40, 50, you know, 100% this week. How are we going to do it? All you have is, you have, you have your MTA data, you have your platform data. We already said it's a high volume noisy period. So we strategically launched these house tests so that beginning that week we have some more clarity around the incrementality of these channels and I think that'll hopefully just guide those budget allocation decisions a little bit better.

[Unknown]

So you launched the test today or this week and are those just like standard like 5% holdouts on a, on a 2 cell on those just to compare like meta to, to YouTube essentially?

[Cody]

Yeah. Well, meta has a holdout and then YouTube is a scale test.

[Unknown]

Okay. Okay, interesting. That's super interesting. And what about like, like how much do you guys do influencer leading up to this period for Ridge?

[Cody]

Good question. Yeah, Influencer starts now. So we don't, we didn't have any integrations go live before Black Friday started November 12th and then all of those will go live before basically Cyber Monday. So that's like, that's what we tell all the creators. Oh, there's always some lag. Some like procrastinators who were like I need to post December 15th or whatever. So we expect some of those. But largely we're trying to like get all those impressions happening while we're on sale and ideally as much as we can During Black Friday Cyber Monday week.

[Unknown]

How about how are you Hexa?

[Connor]

We're definitely lower in our like our Facebook as a percentage of our media mix is certainly lower in the first two weeks leading into our sale launching. We're doing the, the like the next read in the next few days so I'm sure it'll go back up. And generally like Facebook is the channel we crank when we're, we're beating Target. But yeah, I mean we're probably 20 like, like significantly lower. Like 20 percentage points lower than like what are probably year to date Facebook mixes. So that's going into more, more upper funnel channels. Honestly it wanted more budget into Amazon which was an interesting one. But yeah, well and then we'll, we'll crank up once we actually get, get into the sale going live. But as far as influencer goes like we don't. We did this last year and like honestly we didn't see that much value out of it. Like we did this like enhanced campaign where we signed like like an additional like 75 micro creators and we just didn't see a lot of like traffic from it or revenue from it. And we just feel like are kind of always on like product seating like cpm. Impression arbitrage is, is ultimately the most valuable thing we can do to like maximize influencer for bfcm. So we don't do necessarily anything like incrementally different or not this year we do make sure that all of the people that do owe us organic posts. It's all about BFCM and holiday sale and it's all during that. So we are certainly using our existing roster to, to promote that. But there's nothing necessarily like net new happening aside from just like you know, we hope that we see a lot of that you know, impression kind of turn into revenue. Not that we'd really be able to measure it aside from our post purchase survey. But we're not doing anything totally like innovative or new to activate for a holiday.

[Unknown]

Yeah, I'm, I'm with you. It's, it's also hard like I, because you're like hey we've got the budget, we've got the spend, let's deploy it in all these places. But it's so hard to know if one of those new things is even working. So it's. Yeah, there's just so much noise. We're pretty similar. Like obviously we've done a lot of filling the funnel. So like I think TV spend has probably been like 17, 18 of spend. YouTube has been pretty high. We won't necessarily bring them down. We'll just bring other things up. And that'll be mostly Meta. So, like, if Meta's been 6, 50%, 55% of spend, you know, month to date, like during Black Friday, It'll probably be 80% of spend. Something like that. I. I just don't find the other channels have that, like, same liquidity, like the ability to just, like, scale that rapidly. Like, we can't launch a new campaign. At least I haven't done it. A new campaign on YouTube at like 100k a day. But like meta, that's like nothing. We'll scale. Applevin. I'm curious to see. I've heard and talked to some brands that are spending, you know, some. Some good money there, so we'll definitely scale that up. We actually launched a holdout that I got to see when it ends. I think it'll end Cyber Monday as well on Applovin. So I think we're like one of the first brands to run that. But yeah, we kind of went hard with the upper Fun. We launched ctv, got a little test of the week later. But yeah, Meta. Meta really is the one that just scales it up. You know, all the other ones, it's like, it's also so insignificant of like, we're spending, you know, $2,000 a day on Pinterest and we 3x the budget. It's still like nothing in the grand scheme of things.

[Connor]

Yeah, we had our. My Facebook rep was hitting me up in a tizzy last week because, like, your. Your percentage of Facebook budget is as low as it's like, it's low as it's been all year. Like, what's going on?

[Cody]

Like, dude, relax.

[Connor]

It's okay. Yeah, it's gonna come up. Don't worry. It's still net year over year.

[Cody]

I had a rep last year. Speaking of. Yeah, it's a great point that, like, obviously your. Your allocation percentages change largely because Facebook and. And Meta are the easiest to scale. So Cody made the point not necessarily scaling anything down. Meta's just getting larger. I remember it was this time last year. It'll be from the rep of an ad platform that I will not name. But they emailed me, like, it was like, you know, whatever week of Black Friday. And they were like, hey, I see budget increases on Facebook. Like, big ones. Like, why don't we spend that on Google? And we're just like, she would have. Oh, shit. I said the name.

[Connor]

We can blur that.

[Cody]

We can, we can, dude. Yeah. But classic. She said, yeah, anyway, whatever. We could leave it in. That's what the Google rep said last, last Q4 and it just, it just shows like a lot of people don't understand how to proper how, how channels can scale and what's more incremental to the business and meta and a lot of these top of funnel channels are just infinitely more valuable than us trying to pump more money into pmax. It's just like not. I have a lot more skepticism about it at that point. So um, yeah, I just think it's, it's funny the way different people approach it.

[Connor]

Yeah. Our, our Facebook. I love our Facebook rep by the way. They are unbelievable. They're super, super helpful but they're like, they come to the office, they're like hey, I think you should spend more and hey, I think you should launch your Sarah alert earlier. This is like, this is like last week, Wednesday. I'm like no, no, no, the we are not launching the sale earlier. If I send a message to my team right now, they are going to like take me out behind the woodshed and off me. It will not be good. And I'm like sharing all this data with him like why we're not launching it earlier. But you know, he's just doing his job, trying to get, trying to let them get, get them dollar dollars for, for Facebook. Not that they need too much help.

[Unknown]

We've been talking about upper funnel channels a lot especially leading up into, you know, peak moments. And honestly it's been one of the biggest unlocks for us as a business is, you know, helping us reach net new customers and getting them into our funnel. But the challenge is historically the stuff is very difficult to measure. You're not going to see it, you know, in platform roas clicks. So one of our secret weapons this year has been prescient AI. We onboarded with them about midway through the year and it's been a game changer and helping us to measure some of these very difficult to measure upper funnel channels like tv, like linear tv, like streaming. They've been able to move really quickly. The team's been great and actually get betas up with new channels that we've been testing.

[Cody]

So.

[Unknown]

So it's been great. We've been loving it. We're able to see and forecast if we were able to scale and spend on some of these channels what our incrementality would look like. Pression has become a really important part of our marketing workflow. It's one of the main tools that we use to set our budgets every month. And yeah, it's just become a really important part of our stack that we feel like we can't live without at this point. We love the unbiased cross channel measurement it has. Also love the Halo effects where we're able to actually see how it's impacting. We're just on D2C but I know brands that are on Amazon they're. They're able to get Halo effects to actually see how their upper funnel spend and their D2C spend is impacting Amazon. So it's just become a really big part of our workflow and I can't recommend it enough. We're using it. Hexclad is using it. Symbiotica Codery and dozens more impressions blowing up from everyone I talk to. Can't say enough good things about it. If you want to try it to measure some of these upper funnel channels that we're talking about, go to pressionai.com operators to book a demo.

[Connor]

Do you guys have a switching gears a little bit back to kind of like team operations if that's okay. There we go. Go with that.

[Unknown]

Yeah.

[Connor]

Do you do any like do you kill or pause or what Harley would call chaos monkey? Any of your evergreen. Just always on meetings when you get into like the middle of November just so the team can really go heads down and focus or do you try to keep that stuff on?

[Unknown]

Just looking at calendar now definitely like Thanksgiving week, stuff like that. Like try to just go like as bare bones as possible.

[Connor]

Yeah.

[Unknown]

You know so even like our, our weekly marketing meetings and stuff like that we'll, we'll push out and then anything that's like I don't know about you guys but like I just can't. When we're right up in that period and close to it like I just can't think of anything that's like bigger longer term. So like anything that's like next year planning I just don't have the headspace space for it. So that, that all for that two week period just gets put on hold.

[Connor]

It is so tough. Like every non like growth performance marketer in any org like mid November hits or earlier honestly I mean we've already started. I mean way earlier they start thinking 2025 and like it's always so frustrating. I always feel so bad like briefing the team on like 2025 planning asks on November 6th. I'm like it's just because you're not. You're going to get the most half baked version. Right. It's like we spend all year to, to maximize this 30 day clip. And yeah, it's impossible to think too far ahead when you're, when you're doing that. But you almost have to force yourself to do it at least a little bit so you're not like, you know, too far behind. And then, and then it's like your funnels are live and then you like immediately just switch right into 2025 planning. It's like, no, no break for the, for the growth marketers out there.

[Cody]

No, totally. We just went through the same thing where I was like, oh, it would be awesome if we reviewed like Q1 launches and we put together the marketing briefs. And then it was like, then I canceled that. I was like, there's just no way we do it. Yeah, it would be, it would be awful. Yeah, we don't. We keep our, I've talked about it before. We do a Thursday go to market meeting. We'll keep all those on the books. We do a Monday marketing all hands. We'll keep that on the books. I don't remember exactly what we did week of Black Friday and Thanksgiving. Obviously a lot of meetings get canceled. So a lot of those can kind of like fall by the wayside, but largely are key kind of structural meetings we keep in place. We also, with Thanksgiving or Black Friday where there are essentially no meetings, we'll have like the core growth team obviously working, which I think is beneficial. Yesterday was Veterans Day, so we had a call team. Worked like a half day. Basically. We did final sign off. So we use this time of like no meetings, clear schedule. Like let's just make sure we're as dialed in as possible. Didn't take too long, was super effective. And that's essentially what happens come Thanksgiving and Black Friday as well.

[Connor]

Yeah, I kind of want to do like, I kind of want to cancel all one on ones and just do like a growth team stand up like twice a week where it's like, hey, here's a Monday stand up. We'll maybe keep a Wednesday stand up on the books if we need it. And then Friday and everyone can just kind of rapid fire anything that's like top of mind, any fires and then get back to the regulars. Regularly scheduled programming.

[Unknown]

Yeah, I think totally fair to cancel one on ones and, and like that week. Yeah.

[Connor]

Yeah. All right. I want to talk about one of my favorite topics of bfcm. Cody. How hard are you going to spam your customers through email and SMS on Cyber Weekend? Let's hear it. What's that? What's that own media campaign calendar cadence look like? I got Ours pulled up right now, so I'm happy to share as well.

[Unknown]

Yeah, definitely more. I don't, I don't have it pulled up but we're, I think we're, we've reviewed it, you know, a few weeks ago. I'm always more my, my feedback whenever I see what we have laid out is more like this is the one time of the year where, where, you know, we can go extra. We're okay with unsubscribe rate going up because obviously, you know, it'll, it'll happen. But you know, it looks something like two emails. Launch day. Two SMS's launch day. Um, at least one email every day. At least one text every day. Definitely two on the final day. And sometimes some impromptu plain text emails, SMSs, if we have some extra, you know, inventory to move. Um, I don't think we go quite as hard as some like, I don't think we have any days where we're going three but like a normal cadence, our kind of, our, our full list will get probably, you know, three to four emails a week. And so we're going every day. We're going to have an eight day, you know, sale period this year. And so we'll probably have something. I think my director of attention told me she just briefed in 20 emails for, you know, to design last week. So yeah, 20 over eight days is what we're looking at.

[Connor]

So. So three is our max, except it's actually six because it's three emails split between like an acquisition segment and a retention segment. So like Friday we'll do acquisition retention, like morning acquisition retention, early afternoon acquisition retention Friday evening and then we'll do the same thing Monday. They have three, they have, we have three slotted out per segment on Tuesday too. I think that might be overkill but. So we might scale that back. But that's also just email. So like Black Friday we have three, like six total emails. Three to acquisition, three to retention segments and then we have two attacks or sorry, four texts. Same thing though. Two to acquisition segment, two to retention. So we'll see if, if our unsubscribe rate goes to 15. I'll let you guys know.

[Cody]

Yeah, we'll be doing four emails.

[Connor]

Black Friday. So nice to the same segment.

[Cody]

Yeah, yeah, yeah. I mean I actually, I don't know the amount of unique emails. We will break them up because we have a retention offer so customers will get a different one. I mean probably unique emails is probably like 10 on Black Friday or something with all the segments but a single person will receive four emails from us, which I think is so funny. Like, I mean, reflecting on the last couple years, like, Black Friday keeps obviously inching up. And then the other thing is slowly, over time, people realize like, hey, you could send a second email, Black Friday. And then the next year was like, I think we could send a third. Last year we sent four. This year we'll send four. Two. I'm pretty sure last year we had a fifth one planned. Like our director of retention was like, yeah, we just had like another one in the pocket basically, like ready to go. If we weren't seeing like unsubscribes, if we were happy with engagement, if we thought we could drive more revenue like etc. Etc. We just wanted that optionality so we can, we can really send it. I also think an important point to highlight here is plain text. Plain text on the day of Black Friday. Easy win, helps with deliverability, helps send out a little bit more. I think there's a cool message to tell around, you know, wanting to give the best savings or you can figure out some way to like tie it to your brand a little bit. So I'll highlight that because when, when I say we'll do four, at least one of those is a plain text and then the other one at least. People used to recommend this a lot in the past. But not sending at like a standard time. So don't send your email at like 12pm Send it at like 11:47 or something random just to try to like break it up a little bit. Because as everybody knows your inbox just gets flooded day of. So as much as you can do to like stand out, get deliverability, I think, I think the better. And then you basically just do that as much as you can.

[Connor]

If any of our listeners want to want to split test in 11:47 versus 12:00pm send time, we will share the results on our test of the week. We will do it. All you have to do is offer your brand as a testing ground. But I think, I think you hit on some good things there, Connor. Like how do you understand if the those additional emails are more damaging or value adding that? I think like it's all about trends. Like what's the trends in your open rate click rate? What's the trends in your up subscribe rate? What's the trends in your last click revenue? Like as long as those trends are staying flat, I mean you're probably good to keep sending until you really see like, oh shit now like unsubscribe rate just doubled. Like and last click, revenue is like down 30% from the average. It has been like, I think those are the types of KPIs that you can use to inform. Like what that, what that, like, you know, incremental drop off is right? Like, whatever the, whatever the. What's the reverse of incrementality if it, if it's not. Is it just non incremental? Yeah, yeah. Diminishing returns. Yeah. Here's one thing that I think is interesting. How often are you guys like, reactively changing tactics during bfc? And we started talking about this a little bit, I think about a month ago. But like, how often are you like saying, hey, this offer is not hitting, we need to switch it up, or, hey, this skew just went out of stock, we need to switch it up. Or, hey, we can't offer this as an upsell or a gift with purchase anymore. Like, we need to go update the product page and the copy and the ads. Like, is that some. I mean, obviously you try to minimize that, right? But is that some? I mean, you can never minimize or predict everything that is or isn't going to happen. So what does, what does that look like for you guys during bfcm? Connor, you want to kick it off?

[Cody]

Yeah, I. It's a great question. And obviously you want it to happen as little as possible. Almost every year we've had to make some adjustments. Like, last year was probably the biggest one.

[Unknown]

We.

[Cody]

It was. It's kind of odd the way it happened. We were doing a mystery wallet domestically, and we had a. We had a $99 Damascus wallet internationally. We had that. We saw the mystery wallet crush earlier in the year. And for technical reasons, it's just much easier to do just within the U.S. so we had this like, weird split test. And we realized very early, this was week of Black Friday, that the Damascus wallet internationally seemed to be performing better than the mystery wallet domestically. So actually like revamped that and that was like our week of Black Friday exclusive offer. That's the biggest change we've made in a long time. The year before that, we were off to like a relatively soft start. So I think we like adjusted some of the markdowns across products. We tested a gift with purchase. So you can be like, pretty iterative what we're doing this week. So again, It's Tuesday, the 12th, Friday morning. My team's putting together the first three days over like a comparable period. So probably, you know, Tuesday, Wednesday, Thursday last week to see exactly how buying behavior shifts across products. Because what we're looking at is or what I'll be looking for is how does buying behavior change between we'll have MagSafe still at 4 price versus the classic wallet at now a promo. So it's like, oh, does mag safe volume go to 0 or. And then are we worried about that from an inventory perspective? Would we want to roll that in? We have things like Collegiate that are currently not discounted. So we have like as I've said on this podcast, like a bunch of times we've had this like exponential increase in complexity and like massive expansion in the, in the catalog. So we're trying to be really thoughtful about how are certain promos shifting volume across that catalog. Then we can adjust accordingly. And if that has to result in or if our objective is to actually make it a better sale, then obviously we just have more levers. More things will get discounted. We can mark things down further. We have a lot of flexibility. We try not to make any adjustments. Week of Black Friday the last. Ideally the last change that I'd be wanting to make would be like the Friday before that weekend before Thanksgiving. This year it's the 23rd and 24th are so key that you'd like to think you're going into that period with kind of your, your best options.

[Connor]

We have the. I don't know how our inventory team are their wizards because we have the. Our. Our biggest headache is like, hey, you just sold out of one of the products in this $3,000 bundle that literally contains 30 different products. And now you have to go in and like basically make a different variant of that sku. Like it's still the everything but the kitchen sink bundle, but now instead of it having a black apron, it just doesn't. Or we swapped a Santoku like it was a chef's knife. We sold out. Now it's a swap in a Santoku. Nice. So that, that's the biggest like gymnastics we're always jumping through is kind of the iterative bundles and how those change. Now luckily those really only change on our like lower volume, higher order value bundles. Like that's never going to happen on the 12 piece or the six piece set, but that's always a fun one to to you know, we're usually switching out like three to six different bundles throughout the the core season just because we like stocked out of of you know, an individual sku and it's hard to project because like totally chances are that product is in a bunch of different bundles. It's also sold as an individual product. It could be an upsell so there's all these different places that it's moving and you have to like track velocity in all those places to see like all right, this is going to stock out in four days. Like all right, growth theme like go, go. Kind of get the backup ready. So that's the biggest headache for us. We've gotten better at getting ahead of it but at this point we've kind of created most of the assets we need for all the different bundle variants so we shouldn't have to do too much new production. But that's the biggest headache we face in a like an intra kind of tactic. Changing.

[Unknown]

Yeah, we, we decided today, finalize today we're going to, you know, change our offering. We're, we'll still have the Minis but we're going to, we're are, we are going to mark down our kits just because we have so many. So we're going to do like, we're gonna, you know, it'll be our first time really doing a discount that's besides just the Minis just because we have a lot of these kits. So we're gonna say like all, all kits that we have are discounted so that'll really be ours and so hopefully we'll just run with that through it. But in the past like we've definitely probably less of an offer perspective. Like certain things would run out or we, you know, we'd have inventory issues or shades. A lot of it was, was like marketing so we might have to like pull a certain skew from an email or reorgan just to kind of put less focus on it. So a lot of it is just, you know, like, like Connor does just getting really tight with the inventory team. See what's moving, see what needs a little extra love, you know, see what you can push and see what you, you know, you really shouldn't be pushing. We have, we've, we've stood up like a SKU and kind of have it be like a set because you know, it's hard to actually like kit things in the moment but have it be a set where it's like a bundle and gets to fill the single items on the back end. So like when we were smaller we did that find it's just harder as you grow and there's so many more moving, moving pieces. But yeah, definitely, if, if we have to. Hopefully you don't have to but there's always some last minute change that you know, we're trying to pull. But yeah, for today we're, we're actually briefing the team on it tomorrow, just finalizing details tonight because we want to give them, you know, a week, maybe two to, you know, get all their assets ready because they will have to edit a bunch of things based on this change.

[Cody]

So, so just to like double click on that a little bit, that is. You guys start November. You're at forecast now. You're thinking it's a little softer than expected. So next step was hey, what, what, what tools do we have to kind of increase volume here? And it was, hey, let's discount the kits. You decided that today and then when does that get implemented?

[Unknown]

Yeah, so that'll be on the, our, our, you know, Black Friday, I guess we'll call it a sale now. But the minis we're launching on the 25th, going through Cyber Monday. So that's eight days. So we're just going to keep the same timing, keep the same cadence. So it'll launch then.

[Cody]

Got it.

[Unknown]

We'll have it. Yeah, just be just on the kits. Yeah, we're, we're, we're, we're looking at way more than we would like. We have the possibility of doing a few things like kind of doing that, like bring back strategy that we talked about that you guys do as well. We can do that for Labor Day. We can do that, you know, during the summer. We can also de kit a lot of them. Almost all of the, the products in the kits are evergreen. There's obviously a de kitting fee plus the boxes can't get reused. But all the products can be, but we're, you know, we're, we're still looking at more inventory than we'd like to hold. It's not catastrophic or anything. We're not going to go bankrupt. But it's definitely more than we'd like to hold.

[Cody]

Right.

[Unknown]

So we, we definitely are, you know, weighing the pros and the cons and, and we think it's worthwhile from a, whatever brand perspective as well as, you know, margin perspective that it's better to move, you know, as much inventory as we can. So we'll go pretty aggressive with the discount. Just have it be, you know, similar to you guys do, but it just won't be off. Every SKU won't be site wide, but pretty much anything in that collection, you know, we'll have it be up to a certain amount off. But, but we'll, we'll probably just go set it and forget it. Like we're probably not going to change the prices throughout it. We'll probably pick a certain amount and just Run with it and see. See how we do.

[Cody]

Yeah, and I like that, that cadence, right? Like you guys identified, you must have identified it later last week. You have that, that discussion, you know, Monday, Tuesday this week, and you're like, okay, we're implementing it basically two weeks from today. I think it's such like, the team will keep up. Makes sense. If you're ever gonna promo, you do it during Black Friday. Like you. It's because you have a forecast, you have an expectation, you're worried about where you'll land against it. Like just like, just sounds well executed.

[Unknown]

Trying. It's, it's, you know, it's not the position we want to be in. I mean, there's worse things in the world. Obviously. October was great, so maybe November can be a little bit softer. But yeah, I was just telling like Chief Product Officer, I'm like, if we're going to make this, if we're going to do this, like, let's decide now, you know, Right. I want to give the team as much time as possible. I want to be definitive in our plans. So if we're going to do it, like, let's decide. And so just literally got everybody in a room, was meeting with like my project management team and brand team and Bobby's like literally walking by and I like, grab her and I'm like, are you okay if we say this, can we do this? And just like try trying to get approval right then on the spot. Because it's like, yeah, you've got to act fast. And like, same thing with our teams. Like, I think our team, what they're really good at is adapting and, and just, you know, from our project management team to like the rest of the team because obviously every. Whereas there's now 20 emails we're gonna have to edit and there's, you know, site stuff we're gonna have to change and new ads are gonna be. Have to created. And it's hard as, as you get bigger. I can't even imagine at hexclad size, you know, but it's hard when you get bigger.

[Cody]

There's.

[Unknown]

There, it's just, there's like harder to change momentum on things. So. So, so definitely it's always kind of cool to see the team like work really hard and pivot and, and do some of this stuff last minute. Hopefully we, hopefully we can limit it.

[Cody]

Yeah. And, and you said, you know, it's obviously not the position that you necessarily want to be in. I mean, you're making these decisions because you're not in the position you plan to be in. But making the adjustments, like, is the absolute right thing to do. I remember a couple years ago, this had to have been three years ago. We were probably sitting, like, just about right now, maybe a little bit later in the year, and we were like, hey, we're. We're not landing against forecast that we want. We had a VP of Ops who was like, now's not the time to change the strategy. I was like, what the hell are you talking about? Like, now's the time we need to change the strategy. The worst thing we could do is, like, act like, no, we can't change the strategy. We're gonna keep things as is. You get through Black Friday, you're that much further from your projections, you're sitting on that.

[Unknown]

It's like, what should we do next year?

[Cody]

Yeah, you end up, like. You end up screwing yourself because the timeline in which you can, like, you know, like, make up for that deficit is so much longer post Cyber Monday. But if you can figure it out right now and you can make the correct adjustments before this massive, you know, volume period, like, that's absolute best thing you could do.

[Unknown]

So, anyway, yeah, sorry, sorry. I was just gonna say it's such a. It's such a. An advantage, and this is literally what startups have. And I think this is like, one of the reasons we're D2C and we're not in wholesale is because you can't do this if you're in Sephora. You've got to give them, you know, your offering six months ago. And it's, It's. It's locked in, but it's, it's such an advantage and it's. I always tell my team it's a feature, not a bug. Like, because, you know, it's, It's. It's very natural to be stressed when this happens. But this is one of those areas where I try to, like, I try to use, like, positive encouragement for, you know, motivation and, like, shaping behaviors. But, like, I try to, like, call it out and, like, publicly call it out when people are like this and they're able to adapt quickly. And it. It doesn't. You know, it does, it does. They don't respond with, like, a ton of stress, but they just kind of, like, get it done. I always try to, like, call that out and publicly recognize that because that, like, that's the behavior I want on the team because it's going to happen. But it's, it's such a strength if your team is able to do that. And I'm sure you guys. You guys are as well. It's. It's such a strength and it's such, like a competitive advantage.

[Cody]

Totally. I was also going to say. That's why I said I really like the timeline that you guys were, like, making the decision and expecting the adjustments around, because we've also fallen under the trap. I totally agree. Feature not a bug. We've, like, maybe taken that too far in the past. We're like, oh, my God, we need to change this right now. We've got 20. It's Friday afternoon, and we need this live by, you know, 6am tomorrow or whatever. So you have to find that nice balance of, like, how can you remain nimble but, like, remain sane at the same time? And that's what your timeline sounded like. I think we're at that point now where it's like, we're just setting better expectations around how quickly we can turn these things around. And the biggest piece of that is just having those conversations a couple days earlier than you otherwise might.

[Unknown]

Yeah, I agree. You can't. You can't allow any time to go of, like, you know, not wanting to speak up or call a bad thing out. Like, anytime something happens, like, you've just got to shorten that feedback loop because the quicker you can make that decision, hopefully, like, the better chance you have of turning it around.

[Cody]

Yeah, exactly.

[Unknown]

We'll see. Hopefully I'm in on. On here in two weeks and we're like, hey, it worked. We'll see.

[Cody]

Yeah. Yeah.

[Unknown]

I also just want to, like, not just talk about the good stuff. Like, two weeks ago, we were recording. I'm like, yeah, things are great. This month is awesome. Like, we're at 40%. But, like, also, you know, we're. We're not that far down, but we are down. Like, not everything is great and not, you know, this is really the first time that this has happened for November for us. So it's super cliche, but, like, the. The two biggest misses that we've had this year. We've got one store that we opened that. That is not doing what we expected. And then holiday kits, you know, sucks financially. It's not great, but it's not going to bankrupt. Bankrupt us. But I think it's just such a great reminder. A, it humbles you, and B, it's like, this is how you learn. It's super cliche, but, like, we're definitely learning and having better conversations from these things and identifying who on a team is strong and who isn't. And so definitely grateful. For it. Even though it fucking sucks in the moment.

[Cody]

Totally.

[Connor]

Wow, that's a good one on bfcm. Shall we do some. Some tests of the week? Second. Second tot dub in a row?

[Cody]

Yeah, but yeah, let's do test of the week. Cody, I know you've got some good ones. I was going to make the joke. I was going to make the joke with that. We've been talking about bringing. Bringing a hunch of the week segment. Not data driven.

[Connor]

Vibe.

[Cody]

Yeah. So I'm ready for those. We'll hit one of those before the end of the year. Hunch of the week.

[Connor]

Call that Vibe. Vibe based media buying.

[Cody]

Yeah. Everyone knows ad comments are super important for social proof, but they're a mess to handle. Spam trolls, customer questions, good reviews. At Ridge, we had four full time people to moderate all of this, which was expensive but necessary. One of the reasons we moved to Rich Panel is we knew they'd be launching new AI tools and they just launched their AI Social media manager that handles all of our ad comments. It's built right into Rich Panel, our help desk software, which no other help desk has anything like. In the first two months, AI handled 11,000 comments, saved us 760 hours of work. It's like four months of one person's time. And the best part is it didn't cost us anything else. It came with our regular subscription. If you want to learn more about Rich Panel, you can join one of their weekly migration webinars. It's 40 minutes. They show you the platform, how to switch from your current system, and how much money you can save. It's great for teams to evaluate Rich Panel and see if it's a fit for your business. If you want to add social media superpowers while becoming more efficient, go to rich panel.com, book a spot for their weekly migration webinar and get your team a spot too.

[Unknown]

All right, I got a few, few I've been dying to share. We really. We wrapped up as much testing as we could prior to, you know, November. So tell me which one you guys going to hear? CTV household out quiz, Complete optimization on meta holdout or a. I talked to you guys about this a while back, but I haven't shared it here. A shipping test that got us about a dollar and a half in revenue per checkout.

[Connor]

I'm selfishly most interested in the CTV because we ran a CTV holdout recently too and I'm curious how your results do it.

[Unknown]

All right, let's do it. So first time CTV for us. We've Just been on linear and I don't, I definitely want to be careful about generalizing like hey, if this is our incrementality on ctv, this ports over linear. But you know, definitely something that we were really excited about testing for a while and I thought this was the time just leading up to it to, to fill the funnel. So we went with a four week test. I'm really trying to go with a longer test on you know, some of these upper funnel channels. So four week test with a four week look back, which means even though you know the spend is done, after that the holdout is done, you know, you're, you're still able to get the look back. And so that'll take us through Cyber Monday. So we kind of planned it that way. We did the same with that like meta quiz, complete test, which, which I'm pumped about as well. And so we got initial read. We obviously, you know, I'll update when we get the actual look back. But that being said, this was one of our better test results. So let me just pull it up super quick. And again, I also think that this is one where I expect to get better. So for us, a $68 cost per incremental, which is actually one of our more efficient cost per incrementals. Very good. Something we're very happy with especially I think on a two week period for tv. I expect that to get significantly better. So yeah, about a 3 1/2% lift on new customer orders, new customer revenue. I don't know what it did to total. For some reason we only looked at new on this one, but usually look at both. But I'm, I'm pumped about it. I mean anytime you can unlock a channel that you know there's massive scale on, I'm, I'm super excited about it, but definitely excited to get the, the look back after Black Friday.

[Connor]

What did you say? The, what did you say that CPI was 68 and, and like relative to other channels that was in like the. Is that like the best or is that like just top or.

[Unknown]

Because best but solid. Yeah, solid. Yeah, definitely very happy with it. It's very good. It's probably very similar to YouTube but it's at a smaller spend. But it's I would say very good for a, you know, what we would consider an upper funnel channel that you know, usually it's like, hey, I've got to you know, go in the negative or like break even at best because it's a longer term thing and hopefully over time. So for, for two week Period. I'm ecstatic about that two week period.

[Cody]

And then, and then, sorry, you said you're running a four week test on the targeted DMAs.

[Unknown]

Yeah, yeah. So four week test, that was actually just the initial read.

[Cody]

So the two week period.

[Unknown]

So we're getting, yeah, the four week soon expecting that to get better. And then we'll do that four week look back. That'll take us through Black Friday.

[Cody]

Perfect. Yeah, totally dude. Awesome test. So, so let's say, let's say results are like a little bit better than they are now, CPI of 55 or something. What would be the next step for you there?

[Unknown]

It's a really good question. Oh, the other thing about this is so you pay a premium to do DMA level testing with TV. So our CPMs don't quote me on the amount. They might be 50% higher than we would pay if we just went nationwide. And so you know, I'd want to be conservative about it. But you can say hey, if you take the same conversion rate, the same incrementality factor but you lower the CPMs by, by 50%, like technically it is better. So you know, that's, that's one thing you can look at. Well, I, I really want to, you know, I want to look at how, you know, impression our MMM is kind of saying and recommending we, we allocate. But I would probably, I would feel comfortable scaling it up because it was on relatively small spends, but definitely kind of scaling it up, maybe getting it to like 5% of media mix, kind of using MMM for a little bit and then depending on where we want to increase, maybe there's like a scale test in there I really like. And Nick, who we all work with, I think we shot it last week, has this idea for test where it's like not necessarily a holdout. Let's say it's just like half the country and you're looking for where should our next dollar be spent? Just take half of the country. West Coast I'm going to spend 100k in two weeks on TV and then East Coast I'm going to spend 100k on AppLovin and, and just see where the best income, you know, where your best. Because theoretically everything else should be even. And so those are the only two, you know, variables. It's more of a, an AB than it is a true holdout. You could run a holdout in there.

[Connor]

Right?

[Unknown]

So that's probably what I would think is like let's scale it. Maybe we don't have to do another test. Right away because we have other things we want to test but definitely confident enough to, to keep it running and scale that up.

[Cody]

Totally awesome.

[Connor]

Do you think, do you think they'll run a follow up test? Cody? Like a scaled, like Connor just said you're, they're doing like a scaled YouTube. We're also about to launch a scaled YouTube after finding out that YouTube was our like kind of what you're finding with CTV. Like YouTube was one of our relatively most incrementally efficient channels. Initially we just scaled it and then we were using like MTA clicks and views to kind of see how that efficiency was trending and we liked how that looked. But we are going to do a scale test. Are you going to do like a scale test at some point to see where CTV diminishes?

[Unknown]

It's a really good question. For every other channel I would say yes. Like that's what we have done for YouTube. We've actually ran a few, you know, three cell scale tests on YouTube. What we'll do is like we'll take mmm and like prescient was saying YouTube we should increase but it was above levels that we ever spent at. So we ran a three cell saying hey, what if we actually spent these daily amounts and, and it actually tracked very well. The only reason I maybe wouldn't on CTV is. Excuse me. The only reason maybe I wouldn't is that whole like CPM increase thing. So it's just like it's not as easy to, to, to geolift test some of CTV compared to like a digital channel. That would be maybe the only reason I wouldn't. Or if like I don't think we can get like, I don't know the scale that you can get on CTV compared to something like a YouTube or a meta. So if we just don't think it's like big enough percentage of our budget to make that big of an impact, like maybe I don't need that level of confidence. Um, but yeah, I, I, I think I have a number in my head where I'm like, hey, I'm comfortable scaling it to here based on this and if we really want to push it into like the next level, it's something we could revisit in the future. Have you guys, have you guys ran a CTV test? Cause I know you guys run a lot of ctv.

[Connor]

Yeah, we did. We ran a holdout on it. It was kind of right in the middle for us in terms of like relative cpia. So yeah, we scaled back a little bit and cranked up YouTube based because we ran those like around the same time. We kind of want to like, I don't know, we really like this idea of like benchmark tests happening in the background of like always like having an always on Facebook to sell, knowing that that's our core top of funnel channel and then like always hack. Because I just like, I don't like the biggest downfall of GEO holdout testing is it's only the incremental incrementality of that channel at that moment in time. And like it changes all year long. So our like. And this was Cam's idea initially was like running that Facebook channel test always on at the same time. So you always have like a really good kind of like context point or anchor point to understand how it is relative to that channel. So I kind of want to try it again with that because like the. We're just getting somewhat mixed signals from mmm and and like the holdout test. But I think there's just a lot of variables that I just mentioned that could be causing that. So there's probably some follow up tests for us to be done.

[Unknown]

Yeah, that makes sense. I think in addition to the hunch of the week, we should also do like a failed test of the week occasionally. I feel like I. I've also got a lot of tests that we've ran that that haven't done well and I'm sure you guys have as well, so. Don't want to only share the winners.

[Cody]

Dude. Totally fair. I mean test of the week does not. The criteria is not for it to be a winning test.

[Unknown]

Fair. All right, fair. I'll get some bad ones in there. Direct mail. We did this like 16 page holiday catalog. 80k of them. It's still early so we'll see. But I think that that one will not be a winner based on what I've seen so far.

[Cody]

Oh boy. I dick should be interested in talking about that quickly. 16 page holiday catalog landed recently.

[Unknown]

Yeah. Yeah, landed. Did you guys holiday launched which was like.

[Cody]

Did you guys use vanity URLs? Like can you track clicks at all?

[Unknown]

No, we didn't. No we didn't. So we're just going. We're going holdouts with that? We're just going holdouts.

[Cody]

Oh okay. Because I, I wouldn't write it off yet. I could share some crazy data. I was blown away. Yeah.

[Unknown]

What do you think?

[Cody]

It was probably 16. It was, it was a big catalog and this was, this was also a while back, maybe 2021 or something. But I was amazed we used vanity URL so we could see clicks and like it had this massive. It was like weeks that we saw traffic coming through and it landed on this day and it like slowly ticked up over time. Like peak traffic from the catalog was probably 10 to 12 days after it landed. So I think it's one of those things, like, especially if it's like a beautiful catalog. I got a Duluth trading one at my house that's just like sitting on the kitchen top now. If it's beautiful, if it's cool, people hold on to it. Like they end up flipping through it days after they receive it. So yeah, we'll have to circle back on that one. I definitely. I'll keep my fingers crossed. I wouldn't write that one off.

[Unknown]

Yeah, no, I agree with you on that. I actually haven't even looked like, I. I'll walk by and like our data analyst and director retention will be like looking at the dashboard that they set up to track these sold outs and I'm like, don't even show me. Like, I don't want to see it. Yeah, I think direct mail. You got to give it a. You got to give it a month at least.

[Cody]

Yeah, totally.

[Unknown]

I got. All right, I got one more. I'm a little skeptical of it. We ran that quiz complete test as well. Just like trying to fill the funnel. House is saying just two weeks into it is our best cost per incremental that we've had. Like I'm telling you, like stupid cheap cost per incremental. So I'm like very skeptical of it. Yeah, that's just, that's just taking our normal ads normal quiz funnel, but optimizing for a custom completion. So we ended the test. We'll get the results back this week from the four week period. So I'm curious to see if that changed. But even if like the cost per incremental doubled, it would still. Still be something we'd be very happy with. So. And I love, I trust their data completely, but just the number is so good that I'm like, I. I don't buy that.

[Cody]

Okay, let me ask a couple questions here. One, what, what was like a tr was there attributed rows and what did that look like?

[Unknown]

No, I think, I think in north beam it's probably like a 0.2.1 and meta. Maybe it's like the same thing. So like rough.

[Cody]

Yeah.

[Unknown]

Yeah. Cost per quiz completion is probably. I think it's two to three dollars probably. I don't know about unique. It's a custom event. So we're not able to see like A, like a unique. But yeah, it's saying two to three dollars. But I imagine the conversion from quiz completion, which is where we get somebody's email plus their quiz completion to purchase is significantly lower than, you know, quiz completion to purchase on a purchase optimized campaign. But maybe it's so cheap that it nets out.

[Cody]

Totally, totally. That is really interesting. And then my second question was, can you give us a sense of how much you spent and, and like how big was the holdout?

[Unknown]

I think it was probably like a 40% holdout. It was larger because we're not spending time, we're spending like a few thousand dollars a day on it. So you know, half, like 0.5% of our total spend at most. I think it was probably something like a 40 or 50% holdout just because, you know, you take a channel like Meta, as you guys know and like you don't want to hold out too much to impact the business. But if you're launching a net new strategy that you don't even know if it's going to work, it like work at all. Like you can go with pretty, pretty big holdout because you got nothing to lose.

[Cody]

Yeah, 100%. Well, yeah, dude, keep us posted. I think that's interesting. One. I wouldn't be. Well, I actually would be super surprised. But I like the idea that. I like the idea of optimizing for some sort of like more involved behavior on the site. Like we don't, we don't do any quizzes, so we've never optimized for a completed quiz. But what we'll do in the past, instead of optimizing for reach or for clicks, we'll optimize for like PDP views. But we're dropping them on a lander, so you're forcing them through a landing page or a collections page to get to a pdp. You're optimizing for a browsed user. That just makes so much more sense to me than like just straight clicks. It just makes way more sense to optimize for an involved event on site. Something like a browse where we're driving someone to site and we're optimizing for PDP views, but we're dropping them on a lander. So I like the quiz funnel instinctually. You'll have to keep us posted on, on any updates on that test.

[Unknown]

For sure. Yeah, definitely will do. I'm excited to get the next round back.

[Connor]

I think that would. I'm also especially curious. I think quizzes make a ton of sense for us. We've we tried like a tried one last year but it wasn't like a very great swing at it honestly. So maybe this is the, the data point I need to focus on this project in Q1.

[Unknown]

Well, have you guys like Hexagon? Have you guys done any like landing page testing or like, like landing page views or you know, view content or anything like that?

[Connor]

So we've, we've done it and we've always turned it off eventually because the one day click just never backed out. I also like over time have realized that like one day click ROAS or any probably click based return is not the way to measure that. Like we ultimately need to do a holdout with the way that you're doing it. So we'll slot that in. That's going to be basically our, our, our big picture strategy with House was do like all the channel level tests like the channel wide holdouts first get a sense of like incrementality per channel and then we'll kind of revisit each channel and actually do the holdouts on like the intra channel tactics like landing page optimized campaigns and top of funnel reach campaigns and all that. Like even some like creative like holdouts I think would be really interesting for us. So we're, we're getting there. But yeah, we just, there's like, I don't know, I don't trust any way to measure it other than.

[Unknown]

No, no, I think holdout's the only way. But it would be interesting for you guys with, with your AOV and also just your, your total budget and spend and consideration phase. I, I, I, I think it would be really interesting for you guys. Could totally see it working.

[Connor]

100%. Yeah, I agree. And we can ut tm them well and like cohort them in our, in our CDP pretty easily. I feel like. So sweet boys. Good app.

[Cody]

Hell yeah.

[Unknown]

Definitely. Happy Black Friday baby.

[Connor]

And that's a wrap on this episode of the Marketing Operators Podcast. We hope that Black Friday Cyber Monday is treating all of you very well and they're putting up some big numbers and some fun screenshots to share with your team. Thanks again to our sponsors, Motion, Prescient Rich Panel and House. And if you're like the show, make sure to subscribe like and share.