Sam (3:36)

Congrats on your best ever Memorial Day.

Connor Rowling (3:37)

Yes, thank you, thank you. And now we have to anniversary it. Now we're committed to Memorial days every single year. So a lot to look forward to. Cool. But we can get into it. We've got, we've got a bit of an agenda here before we begin. Thank you to our sponsors. Motion Rich panel after cell, prescient in house as always. Without them, we wouldn't be able to geek out on marketing every week.

Sam (4:07)

Sam.

Cody (4:34)

Okay, I just sent my entire team an email about this training that Motion's putting on. And you guys are going to block your calendars for this? Every operator I talk to is thinking about how to build lean teams and become more efficient. And AI is an obvious part of this. We talk about it on the marketing operators all the time because it can automate time consuming parts of the creative strategy process. And Motion, our sponsor, they're doing a live virtual event series with five sessions on how to adapt timeless advertising principles to the new era of AI first ad production. This is their legendary virtual event series. Make Ads that convert. Last year over 11,000 DTC marketers registered for it and it is going to be a banger. Listen to these topics and just an elite cast of the best of the best experts. You'll hear from names like Alex Cooper, Jimmy Slagel, Jacob Pal, Jenny, Kelly Rocklin, Mirella Crespi and very hot. A few of the topics that they're going to go over. Using AI for deep customer research Advanced prompting techniques for ads that don't look AI generated making sure your AI ads actually sell physical products using AI actors and voice models for scalable UGC building a lean AI augmented team of 2025. Here's one session that really caught my eye. It features Kelly Rocklin and she is going deep into using AI to make scalable UGC with AI actors and voiceovers without it sounding cringe. How to spin up high volume messaging tests using AI agents Super interested in what she's learned here. Expand on what already works without replacing the human spark that makes it work in the first place. How to test creative like a craftsperson, not a chaos agent. How to build internal creative testing systems for your AI creative, just to name a few. It's going to be really, really good. Highly recommend registering. If you want to check it out, go register for this. It's free and it's five total sessions. You can register at motion app.com forward/make-ads-that-convert that is motionapp.com make-ads-that-convert.

Connor Rowling (6:37)

We'Re getting into the weeds of meta advertising here and the important metrics. The new metric, that's all caps trademark. The new metric. Cody introduced it. What is your most controversial meta opinion, Cody?

Sam (6:53)

Well, well, I mean, I'll share what it is, but the funny thing is like normally on Twitter you can say something that's like very true. Like today's a Wednesday and you'll get people arguing with you. Be like, no, it's not. I was like, this is my most controversial opinion. Not a single argument. Even Barry Hot, who normally will argue with anything, came and agreed with it.

Connor Rowling (7:10)

Okay, what was it?

Sam (7:11)

What did I say? I'll pull it up. I always, by the way, whenever you send the episode plan over right before the show and I see a screenshot of my tweet ahead of like on top of it, I always get a little worried.

Connor Rowling (7:20)

Yeah, yeah. I've got to hold you accountable.

Sam (7:22)

Yeah.

Cody (7:23)

I got to go back to the headspace that you're in when you tweeted it.

Sam (7:26)

Yeah. My most controversial meta opinion these days is Cost per thousand accounts reaches the most important metric if you're spending seven figures monthly on a purchase objective. I even. Did you see how I like elaborated on that? So I gave it some nuance because I knew that people were going to be like, but that doesn't apply to us. And I was like, if you qualify for here.

Connor Rowling (7:44)

So we're learning.

Sam (7:45)

I'm learning, you know, maturing. Yeah, no, I mean, I think, you know, we talk obviously a lot about reach. And so again, specifically on a purchase objective, I've just seen anecdotally that like our best performing campaigns that have the best one day click roas the most incrementality with however we're measuring it and also the best percent of new visits also have the lowest cost per a thousand accounts reached. And there's been multiple tests that we've ran recently that we've fortunately band will see some improvements in there. Which is, which is very nice because it's something we struggle with and, and it's something that now I'm tracking very closely.

Cody (8:22)

Cody, when you say accounts reached, are you using because cpm cost per thousand impressions is obviously like the, the cost per eyeball metric most people are familiar with. Are you just using reach as the, the number that you're calculating it off of instead of impression? So it's cost per unique eyeballs instead of just total.

Sam (8:42)

Exactly. But it's not a custom metric. It's a, it's like a standard column or metric that meta has. You, you know, you have to add it to your columns. But yeah. So really instead of looking at what, what's our cost per every impression, what's our cost for every new impression that we're serving?

Cody (8:57)

What kind of delta are you seeing between CPM and cost per accounts reached?

Sam (9:03)

So depending on the bidding and optimization strategy, some pretty big ones. Um, but I mean on a seven day period, you know, I mean, so it's much higher. Right? Because like our frequency on a seven day period gets into like the twos and threes. On our best campaign that has these new bidding strategies, we're at a 1.6 and so we are, you know, we're less and we had a promo period, so it's higher. But let's say we have like a $35 cpm on average. $36 cpm on average. Just what we see in beauty. Our cost per a thousand accounts reach is anywhere between $56 and $130.

Cody (9:44)

Okay.

Sam (9:45)

Really just depends on the frequency to that. But that $56 is, you know, Less than half of our other. And that's on this like new bidding strategy. But we're rolling out a few tests and seeing the same thing. And so I just feel like it's a really good directional, maybe diagnostic to, to when that number climbs up on a campaign or an account to be like we have to do something, we have to change something. That's kind of how I'm thinking about it or using it.

Cody (10:13)

Interesting.

Connor Rowling (10:14)

And, and like the inverse of that is just frequency. Right. It's like you, you want the same, like a roughly the same CPM but lower frequency.

Sam (10:26)

Yeah, yeah, yeah.

Connor Rowling (10:27)

I think that's how, I think that's how the math.

Sam (10:28)

Maths, exactly. It's just like what percentage of your impressions are new?

Connor Rowling (10:32)

Right. And what do you think, what do you think is driving lower cost per thousand accounts? You said the bidding strategies at most of it. Is it like something about an offer or the creative, like some combination of all three? I'd be curious to know.

Sam (10:48)

That's a really good question. So the biggest difference we're seeing is this bidding strategy which I don't believe we can share yet. So. But that's the biggest difference. There are a few others. I think Creative is a very big one. But yeah, we're testing a few different, I guess, bidding strategies. I always mix up bidding strategies or objectives, but we're testing value optimization, we're testing purchase optimization. These new ones seems to be maybe reaching a new pocket of people. I think also Creative is probably one of the bigger levers. I just think that's where we struggle to break out and reach new people. And I think certain other brands do a much better job of having like very diverse Creative that continues to reach new people. So that's, that's, I mean our goal for us, but I think we're at least finding some wins with these different, you know, media buying strategies.

Cody (11:42)

I've, I've got ours pulled up out of that column. It's clearly the, the non purchase conversion stuff that's reaching more net new people has the lowest cost which you would expect, right? I mean if it's going to go and reach people that you're not hitting over and over again and then you got our purchase conversion that are way higher than like the view content or some of the other non purchase conversion stuff that we're running.

Sam (12:01)

Yeah, yeah, it'll be higher. But I think even thinking about it just amongst your purchase stuff like a, is like how does it track over time and is there correlated with performance or like between different you know, I don't know if you have different strategies running inside your account.

Cody (12:15)

Yeah, we're definitely seeing, I think the same, I think the same event that you're optimizing for, Cody, that we can't say yet is also very, very low on an order of highest cost per thousand accounts to lowest. So it'll be. Once we can talk about that, we can maybe bring that up a little bit more.

Sam (12:36)

I got to see if we should see if we can talk about that at the Meta summit. Maybe that would be a good. Hopefully I doubt it, but we'll see.

Connor Rowling (12:42)

It's become too, too big of a tease.

Cody (12:44)

This is, this is kind of one of those, I feel like one of those bidding strategies in that has had like the biggest. I mean we're seeing really good results out of it in our account. I know Cody, you've been super bullish on it. It seems like this is one of those things they've added in that maybe could actually have a, a fairly large impact on your ad account performance over a long period of time.

Connor Rowling (13:06)

Yeah, I could see it now. I'm curious, Cody. I think David Herman, friend of the pod, brought this up in the comments about running top of funnel. In order to drive down, you can drive down your like your blended cost per thousand accounts reached. There should be a acronym for this. We have too many acronyms but this should probably have one because Connor just said and we see the same thing. The most cost effective way to do it is just to not optimize for purchase anymore. So where do you land on that? Are you running some type of funnel and do you just feel like being able to optimize for purchases while kind of making it kind of like a sub objective of like also trying to reach new accounts is like that's the better mix.

Sam (13:52)

Yeah, we have, I think more to do. We've had a mixed success with different, you know, objectives which, which can share about but I think there's a lot of value and can't talk too much about this test but there's a lot of value of reaching new people in a purchase objective like because meta has the purchase signal on them. Not to say there's not a lot of value in going up funnel with your objective. Like we've all proven and I know that you guys have had a lot of success with it that can be very incremental. The exciting thing about this test is like on a one day click we have the same if not better one day click roas on these campaigns totally. But with you know, 20% higher net new visit rate or something like that. So it's like we've only ran a conversion lift study on this. Not a, not a House but like I would imagine it's like significant, significantly more efficient then you know. So I think that's the exciting part is like you're theoretically getting some best of both worlds but that being said ton tons of value in that. So we're running quiz complete as our only non purchase objective right now. We ran a really successful test in Q4 like a $20 cost per incremental and I didn't really believe it. So we tested again with the three cells similar to what you guys did at Ridge and we didn't find it to be great. It was incremental but not as efficient. I think we're going to, I don't think we had a big enough delta between like the spend levels where you guys were much more aggressive with it.

Connor Rowling (15:19)

Because you did, you did three sell for and you scaled up the campaigns.

Sam (15:23)

That were probably wasn't a big enough difference. Especially once you like regionalize the spend.

Connor Rowling (15:28)

Yeah.

Sam (15:28)

So I think we're going to, we're just going to go with the two sell and like scale into it.

Shane (15:34)

If you want to hit next level growth you need to move away from correlation based measurement and move towards causality. There is no better way to test your channels, your levels of diminishing return certain tactics within a channel than using a geo based incrementality testing tool. And that's exactly what House is. That is exactly why all three of us use House. House is a self serve experimentation platform that allows you to configure regional test and control experiments to measure incrementality and identify points of diminishing returns. House is really the, it's the most controlled, the most scientifically sound way to do any sort of marketing testing and experimentation. These things are very, very hard to set up on your own. It's rigorous. If you have one little variable messed up all of a sudden your data is not trustworthy. That's why House is such a valuable partner. All you need to do is go into your ad accounts and add exclusion or exclusion list, run the data or run the test. And not only do they set up the test for you, but they also help you interpret all the results. So they're handling experimentation design and experiment analysis and also even going as far as helping you make sense of what to do based on that data. And we at Hexclad have gotten some insane insights this year from all of our household out tests. So our core strategy this year has been doing channel level holdouts to really see which channels are driving the best and most efficient cost per incremental order. So we've tested YouTube, Meta, Google, PMax, TikTok. We're now testing AppLovin. We are getting a sense of which channels are driving the most incrementally efficient first time orders right now. And the amount of insights that come from that information is insane. It helps us inform where we develop creative. It helps us inform where we scale up budgets and certain channels and bring certain budgets down. Plus we are now able to use our incrementality results and actually plug it right into Prescient. So not only are we getting causal data, that is actual data that we can trust to make decisions off of, but now the media mix models and the probabilistic data from Prescient is even more accurate because they're using actual data to inform their models and the readouts that they're giving us. House is an essential addition to your measurement stack. Go to house.IO/forformators that is spelled H A us.IO/operators to start your incrementality practice today.

Sam (17:56)

Where are you guys at? You guys still on that view content pretty heavy?

Connor Rowling (17:59)

Yeah, it all depends. Like right now we're in a sale period so purchase optimizes hitting goal. So are like top of funnel our view content optimized Campaigns are like sub 10% of total budget and then at times like when we're in we call them evergreen periods. When we're off promo, the top of funnel seems to become more likely to hit goal and it can become a larger percentage of budget. So yeah, I mean just to, just to like connect a couple of the dots here because it's been a theme of the podcast all year, it feels as if from a marketing perspective we've more or less. I'm gonna make it sound bad. We've more or less had to bend over backwards in order to force meta to reach new accounts. And that's like yeah, we're gonna. That's what I was saying in January. Like we're brute forcing our way to reaching new people via this view. Content Meta has said for years that creatives a new targeting and that's how you do it. Which like I'm sure is true to a certain degree. And three, what we're talking about now is it seems like there's things in the pipeline that will allow us to optimize for purchases while still trying to reach new people and kind of Prioritize that objective. So, yeah, Connor, to your point, it does seem like it could be something really, really powerful in the pipeline. So hopefully our jobs become a little bit clearer, a little more straightforward over the next couple months. And luckily, we've got this new metric that we can pay attention to. Thank you.

Sam (19:24)

Should we talk about the next tweet? Yeah, this is me being a hater a little bit. This one, Let me pull up the document right below it kind of's got a screenshot of right below it. So this was what, two days after it, I said, and this was just joking, but I was like, I swear, you can make up a metric tweet about it. In a week, you'll see DTCX threads about why it's the most important metric in ECOM.

Connor Rowling (19:43)

Yeah, 100%. And we're, we're continuing the propaganda with this podcast.

Cody (19:47)

Cody, what percent of, what percent of people responded to your cost per 1,000 accounts with love for it versus pure, pure hate for it?

Sam (19:57)

Dude, like, I, I, I'm telling you, I said this was my most controversial opinion I had. Zero. People disagree. And normally, like, whatever you say on Twitter, even if it's like the most like, unarguable thing, like, people will come out of the woodwork and be like, no, you're an idiot, you're wrong. Nobody disagreed with me, so I gotta get a little spicier. But no, it's just funny. So definitely saw some, some people start tweeting about it and threat and like, this is not a metric I made up, it's a metric in meta.

Connor Rowling (20:23)

Right.

Sam (20:23)

But I do think it's an important one to look at. But I just find it, the human behavior, funny about how people latch onto it and start tweeting about it.

Connor Rowling (20:30)

Do you think it's, do you think it's related to rolling reach? Because let me explain it and you could tell me if I'm, I'm wrong here, because you're looking at cost per thousand account reach at the campaign level relative to other campaigns. So you're not necessarily looking at like, year to date unique accounts reached and like, how incremental was this new campaign. You're really just looking at, like, in a month to date or last week, like, what campaigns were driving the lowest cost per 1k accounts. Whereas rolling reach is like, looking at it over a much longer time and trying to ensure over the last year or two years that you are continuing to reach new people. So do you think those are related or do you think it's really just about the cost in the shorter term and the amount of unique people you. You served impressions to.

Sam (21:13)

Oh, I'm sure, I'm sure they're correlated. I think the one difference is there's no cost component to Rolling Reach.

Connor Rowling (21:19)

Totally.

Sam (21:20)

You know, so like theoretically you could be reaching the same percent of new people, but if your overall CPMs are going up because frequency is one component. CPM is one component of this. But definitely, I'm sure it's correlated. The way I'm thinking about using this is a just a diagnostic. All right. Our performance is down. ROAS is down. Let's look at this. Let's have this be one of the first things we need to do. And then do we need to shock the system a little bit? Do we need to put some more money into mid funnel? Quiz complete. Do we need to take some larger creative swings? Totally. That's how I'm thinking about. And then I think the other thing is like, like just having a dashboard that look at weekly, bi weekly, whatever it is. CPMs just trend line over time compared to cost per thousand accounts reach. Just to make sure like that Delta is not, is not shrinking, if that makes sense. Just so you're constantly reaching new people.

Connor Rowling (22:15)

No, that makes sense. And, and we've said that a lot like Rolling Reach does kind of feel like a tactic to convince advertisers to spend money upper funnel on like Reach Optimize, review optimize or whatever, which like some people are super skeptical of. And like our account is one where like we've basically reached every man in America only optimizing for purchase objectives. But what I've said for a long time and what you just mentioned is like, that's not necessarily the most cost effective way to reach all the men in America that like finding a balance I think is really, really valuable. I think that's how those two end up being related. And agreed that the cost component makes it a little bit more actionable than like just looking at how many unique people have we reached over the last 18 months? And let's make sure we're doing. We're increasing that 10% over the next six months. Is like you should be thinking about cost in that strategy.

Sam (23:05)

They're, they're probably all really comp. More more complicated than necessary ways of looking at frequency.

Connor Rowling (23:10)

Yeah, yeah, 1,000%. Well, because I was gonna say that too. I haven't like audited a meta ad account in a while. But like it feels like typically an easy win. You go in there and you're like, hey, this is a frequency of 11. Like let's just spend less here and let's add exclusions and like those things are what would drive down cost per 1,000 accounts reached.

Sam (23:30)

Yeah, exactly. We've struggled so hard with exclusions. Just whatever we have tried to date has not made a difference. If anything, a lot of things that you would think made it worse. We're trying one thing. Some did reach out on Twitter. I'm like, all right, cool, let's try it. I think it's called like waste not essentially like it's almost like a CDP where they like consolidate different like data lists. Because like apparently from Klaviyo there's like a lot of leakage and like meta can't pass it all back. And then obviously pixel, there's a lot. So I'll keep you guys posted if there's anything. But like we just struggled so hard with exclusions and just like my. This is probably. I'm going to tweet this later because this is actually my most controversial thing is like I just think exclusions are broken and like I see people have so much confidence in them and they talk about like best practices and I just, I just don't think it matters that much.

Connor Rowling (24:17)

I agree that exclusions are don't work the way they're supposed to. I wouldn't say completely broken. Go ahead, Connor.

Cody (24:25)

I was gonna say probably especially so for you Cody and your business where like I don't know what your reorder rate is in 12 months, but it's. I mean I bet people are what, ordering three, four, maybe five times in 12 months. I can imagine that'd be a lot harder. Whereas us, like we're still, I. We never see Facebook drop below like low 90% on first time orders. Now like net new visit rate will drop and frequency will go up, but it's still pretty much only first time orders. And like our reorder rates like it's not bad. It's not like anything like you guys though. So I think it probably depends on the business model a ton.

Sam (25:01)

I agree with that. I definitely agree. I think there's more signal to go after for us. Same with like when we talk about Apple oven and like why maybe it was more incremental on new customers for other brands. So I think there's more signal. But I think regardless there's still like a lot of leakage just with like.

Cody (25:17)

Are you guys gonna set any like I'm thinking about this cost per 1000 accounts reach now and I could see us almost setting some Targets based on when we're in a sale versus when we're in our evergreen period and like really trying to optimize towards that lower cost per a thousand accounts reached when we're in an evergreen period and then maybe a little bit higher when we're in a sale for a couple weeks and we want to like pump frequency. Are you, are you just kind of like keeping an eye on it for now, Cody? Are you actually like working with the team to be like, all right, now that I have a lay of the land on this metric, like, here's, here's what we want during X period, here's what we want during Y period. Or are you just kind of keeping a pulse check on it for the time being?

Sam (25:57)

Question. More of a pulse check. More of like a diagnostic which is like, hey, if performance is good, you don't look at it. It's like click the rate or something. But when performance takes a hit, it's like to me, probably one of the first things we should be looking at just to tell us like, hey, we, we need to do a better job of reaching people. And then, then it's all right. How do we do it? Do we test different optimization settings? Do we test, you know, different campaign types? Do we break out different creative styles? Um, there's definitely multiple ways to skin the cat as we've discussed. But I, I, I think that more than like looking at it daily and be like, I don't know, that's my thought. You know, I don't, I don't think it's something you optimize towards necessarily.

Cody (26:30)

Yeah, I asked. Cause we're, you know, we're running more view content than we ever have. Um, we really like the results we saw in last two weeks. We put like seven and a half percent of our budget into it. That was during a sale period. We're about to scale it up to like close to 15%. So I'm going to keep a pulse on, on that metric and see how us putting, you know, basically doubling the amount of spend we have in our view content, we'll see what that does to the entire account.

Sam (26:54)

Oh, keep us posted.

Cody (26:56)

Yeah, yeah, yeah.

Sam (26:59)

How other campaigns, if you're reaching more new people, if there's more signal.

Cody (27:02)

Yeah, yeah, I'll let you guys know.

Sam (27:04)

For you guys, it makes so much sense. I mean, obviously you're at such a large scale and also just with your aov, like, it just makes so much sense that a mid funnel would work very well for you.

Cody (27:14)

Yeah, yeah. Well, we're Excited to move further up Funnel. You know we've really only tested into View content and I think there's a lot more objectives that make a lot of sense for us. So we're going to do a mix of house tests and Meta's user level testing so we can keep finding wins there and keep getting more and more and more of our budget into these upper funnel campaigns. Sweet.

Connor Rowling (27:33)

All right, good little Meta deep dive there.

Sam (27:38)

You've heard us talk about Prussian AI before, we work with them at Jones Road, but you still might be wondering, what is it? What is this MMM thing? At its core, Prescient really helps brands get more out of every marketing dollar that they're spending by showing them what's working, what's not, and where to spend their money next. The Prescient team knows modern marketing is a sum of many things. Your media spend seasonality, other marketing campaigns, consumer sentiment and so much more. That's why they don't rely on pixels bias, platform measurement or old research models to help you make the most confident decisions. They truly built their MMM from the ground up based on models that adapt to your data and evolve over time, not the other way around. If you want to measure the halo effect that your campaigns are having on all of your channels, yep, talking about E commerce, Amazon and even retail depression can help you out. You can test new channels, predict future performance of those channels, know how much you should be spending on each of them, and continuously optimize how you spend your budget to make it as efficient as possible. Which listen, we all need right now. You can do this all without sacrificing your upper funnel budget or bottom line. KPI's prescient has proven to us how truly incremental and efficient some of our upper funnel channels like TV are. That was very hard to measure prior to it. So if you've been wanting to try Prescient, they just upgraded their model and their vision across measurement and retail is truly next level. They take marketing decision making where it's never been able to go to before. If you want to learn more and gain insights into how you can optimize your entire media mix in as little as 48 hours, then go to prescianai.com operators and book a demo today.

Connor Rowling (29:11)

Connor, you sent a really cool we'll pivot here. We're going to talk about AI and Shopify. Connor, you sent a really cool example. We mentioned this on last week's episode. Shopify Summer editions came out mid May, one of their big focuses, which we didn't really Talk about was improvements to Sidekick, which is their like AI enabled assistant in the Shopify store. You've been playing around with it. What's your take?

Cody (29:36)

Yeah, I think, I mean we talked about this like months ago too, about how like the next evolution of these data platforms are just being able to prompt it and you know, the, the technology going and pulling the information that you want and just bringing it right to you. I'm stoked on this because it's like there's really no excuse not to be super, super data driven anymore. You could maybe make the excuse before these LLMs on top of these data analytics platforms that. But you still didn't know how to use the platform. Right? Like you still like these platforms take time to get like Google Analytics. Like you got to know where to go for the information or North Beam or really any analytics platform you're using. You don't really need to, I mean not, I think a year from now or two years from now, you really won't need to know how to navigate the platforms because you can go into these LLMs, tell it what you want and it's going to bring all the data to you. So now you just need to know, obviously it's like, all right, now data is even more accessible. So the importance of knowing what's important is, is more important. But like, man, I was playing around with this and like anything I wanted that Shopify has, like, I just prompted it and in a minute I had it. And it shows you the workflow, which is really fun too. So you can kind of learn as well while it's pulling the information, like where these things live. So yeah, it was really fun to see. In Shopify's new mobile app upgrade, I'm seeing other analytics using it, like Edge Mesh. I'm sure more and more and more of these platforms are going to keep peppering it in because it just makes the tool so much more accessible.

Connor Rowling (31:06)

And what was. You sent us a cool example. Can you just talk through what that was so people get an idea of like how you're using it?

Cody (31:12)

Exactly, yeah. I mean, I was just seeing what it could pull. So I think the example I was looking at was like, was I looking at like add to cart rate? Yeah, I was looking at like, hey, tell me what the add to cart rate was on Monday of our Memorial Day sale. Like, how did that compare to last year? And then the. So I was like, all right, the add to cart rate was basically the same. And then the next question I asked it was, can you look at add to cart rate by source and tell me if which ones are marginally lower than the average. And then it was, then it sent us that. So it's like, all right, we know add to cart rate. We know the average add to cart rate. Are there any traffic sources that are maybe having a lower add to cart rate than average? Which means the intents probably a little lower or it's more upper funnel. I wasn't really like, it wasn't. There wasn't like an actual business case baked in there that I was, I was just trying to see what it could do. But you can start to get a sense of the types of insights. Like, all right, you know, paid TikTok has a, you know, 25% lower add to cart rate than average. Like there's some clear action items there to like either scale it down or maybe go see if that's more upper funnel than other channels. If it's not, then maybe you do turn it off or scale it down. Like, it's just the data is right at our fingertips and it's, it's pretty fun to be able to like go query these like really deep dives and see how quickly it can give you that information.

Connor Rowling (32:31)

Yeah, I was really, I was really surprised by the like the comparison piece that you mentioned. I was like, okay. Like I would have been so unimpressed if it was like, hey, what was our add to cart rate yesterday? I was like, okay, yeah, I don't know. That's like fine. But like, yeah, what you were asking for was like multi step. I think the example you sent us was like, it was sales on a certain. It was like sales on some sort.

Cody (32:52)

Of kitchen sales on our like kitchen utensils set and then how that compared to the previous weekend. Yeah, I will say I did try to, I prompted it to do some like forecasting for us as well. Like middle of the day on Monday. I said, hey, what do you think we'll do in revenue today? And it was like, let me go look at historicals, yada, yada yada. And it missed by like $600,000 in revenue. So it's. So it's not perfect. But I was, it's kind of cool to like, all right, make it go backwards in time and just pull information, but then also see like, try to make it predict for you and see how, see how well it does with that. Did not do well on the, on the revenue projection prediction. But yeah, I'm gonna, I'll be playing around with it and like I, I think it's just kind of a fun thing like as these questions you have, these data driven questions you have come up like going to Shopify and see if you can get the answer before you go and do all your like manual data pulling and jockeying of these platforms.

Sam (33:48)

Totally. Like I remember we were, we were getting, just trying to get some like frequently bought together stuff like a few months back and director of retention like requested it from our data person. Obviously he's got a backlog of stuff. So I used for what I saw, but saw a like GPT for like, you know how Shopify has their version of SQL? I think it's like shop Ql and I know like a little bit SQL but so I like prompted it and told it what I wanted and it wrote me like a query based on that to get it. But now you can just go straight into Shopify and be like hey, tell me the most frequently bought together things with this. And then I played around with this. You can also be like okay, what bundles would you create based on this? And then I think where it will go not that far off in the future is you can then just be like okay, create them because you can do that. I don't know if you guys have seen, but you can say hey, create a product and it'll create it. I don't know if it does anything with the assets, but it'll put in copy. I even had it be like hey copy, use this other product as the base and copy over all the same metafields. And it did that, which is pretty wild. So I don't feel like we're that far away from, you know, from that. It's like having the, the data analysis in the same place where then you can have some of the generation I.

Connor Rowling (35:05)

Think will be very cool, 100% what I'm super curious to see. And this is like everything that we've been describing is super exciting. And as far as the forecasting goes, like it's one of those things that probably just gets better over time. Will it get to the point that like your inventory planner can just be like prompts in the sidekick? I have no idea like that. That seems like really far off. I'd like to think for the sake of some amount of human work that like there will be enough nuance that some human will have to be in the loop. But we'll see on the generation point it'll be. I'll be super curious to see how Shopify connects to other places because I also think for enterprise solutions for, for A smaller business. I could see like all the data you'd ever need is living within Shopify already for our business. I feel like we, we. That process is connecting a couple different things between like, I'd love to connect to our master SKU table and where we place our images and things like that. Like, that's not crazy to think that it's like Google Drive integrations where you could basically prompt in, hey, we have all this information in all these different places now. You can like build out the product page in full, all the correct meta fields updated and not just duplicating them over from someplace else. And that feels like when we think about enterprise agent solutions, there will be half. There will have to be more points of connection. And that's what, that's what Motion talks about about all the time from a ad creative perspective is like they want to be the, the hub of that. So we'll see where Shopify goes with it. But extremely cool progress being made over the last couple weeks.

Sam (36:38)

I'm with you. Yeah, yeah, I think access to the data and I'm sure they'll do it. Or if not, as we talked about last, last one where you were saying like this is where the data warehouse and just like having the infrastructure is huge. Have you guys seen or played with like the. What, what's the theme called it. Is it Horizon, like their new theme that has some like gen AI capability or some coding stuff in it?

Connor Rowling (36:59)

No, I, I haven't played with it. We had it on the agenda last week. We didn't talk about it because it seems extremely cool.

Sam (37:04)

I, I want to play around with it. I gotta like make sure I don't mess anything up. Might have to do in like a dev store or something. But yeah, it seems, it seems very cool and I'm so into that of being able to just, you know, be like, all right, here's an idea for this block I have and, and just have it generate it for a test.

Connor Rowling (37:21)

Right? Yeah. For those listening, it's like. And there's a really cool demo. Harley tweeted it last week. But it's like built for AI generation. So you could just describe. You might have some sort of like collection section that you could just describe. Hey, when you hover over the image, I want to flip to some lifestyle image and that just becomes code. It's like natural language for theme editing. And the theme seems purpose built to support that. So extremely cool. If you're starting a Shopify store today. Yeah.

Sam (37:50)

You're gonna see those like one person businesses. You're about to see that like one person, $50 million business not too far off in the future. DTC 4.0. Yeah.

Cody (37:59)

No, no, no. Sidekick on desktop.

Sam (38:01)

No, there is. It's a hidden. Took me a minute. It's top right next to your, like, notifications.

Cody (38:07)

Oh, yeah, there it is. Yep.

Sam (38:08)

I know they're surprised that it was more. It was that hidden.

Cody (38:11)

Yep. Okay. Sweet.

Connor Rowling (38:12)

Awesome. So, yeah, Sidekick. Super cool. We've got this question here and we've already beaten around the bush a little bit, but is there anything else you guys would want to see from Shopify on the AI front?

Cody (38:20)

I think like, just what Cody said, like prompting it to actually publish and do things more. I think the data polling is awesome. But the more that this can become like an E Comm manager.

Connor Rowling (38:35)

Yeah.

Cody (38:35)

Is going to be so cool. And I think we all have things in Shopify we do that are pretty, like repeat things. Like we. We always run some of our. The same like fourteen hundred and a thousand dollars bundles for every single sale period. Like, I should just be able to prompt Shopify to like, give it the assets and it should be able to duplicate it, set the inventory, all the things without a. Without a person touching it, except for the person prompting it. And it goes back to like the same. The same conversation we've had about designers and video editors. Like, these, these AI tools should just make them more dangerous. Like, this should just make an E Comm manager even better at their job.

Connor Rowling (39:13)

Totally.

Sam (39:14)

So do you want. You. Do you think it would be better to, I think you said, like, be the econ manager, or do you think it would be better to like, allow the econ manager to be the dev.

Cody (39:24)

Like in the econ manager now do a lot more dev work.

Sam (39:27)

Yeah, like, that's what I want. I want. I want my. I want to be like, hey, we want to spin up this new section or this new feature or this new test. And. And I, I want my econ manager or director or whatever it is to be able to go in and be like, hey, make me this section and then like put. Push that into this theme and set up this test.

Cody (39:43)

Yeah. Yeah. Especially if it's like, I don't. Like, we're doing a ton of personalization landing page builds right now that are all based on a core listicle that we've tested into and I've put a lot of money behind. That is the perfect example where like an E Comm manager should be able to go use an AI tool to build the carnivore version or the, or the 50 plus version or whatever version. And I, and then I want the, the dev to be working on like, all right, what's the next like 0 to 3 page we're building? And they should still be using AI tools, but they're probably going to get in there the self and like do some coding. I think that's the delineation between our ecom managers, like iterating on pages. Engineering is, is building net new pages and then once we test into it and we like it, we iterate on it. That's the E Comm manager's role as part of that. That like second half of it.

Sam (40:33)

Oh, absolutely, I agree. I, I slacked drew from intelligems like 3 weeks ago. I was playing with Shop dev. I don't know if you guys have seen that. That's kind of like a little like, you know, code thing for Shopify. And I was telling Drew that like they should build something like that into intelligence. Like, because it. Cause I don't think the dev or the design has to be as perfect if it's just a test. But yeah, like you should be able to go into to IntelliGems and be like, hey, I want to set up a test where I want to, you know, add review stars or something like that. Like, if it's like a quick test, that should be like a super easy gen AI thing. And actually slacked them last night I was just like, hey, the more I think about it, the more I think you guys should really do this. So yeah, and then, and then you can take that, right? Because I don't know, maybe the code's not going to be great or the design is not going to be great and then if you get a winner, it's like minimum viable product, then you can have your dev team build it out like properly.

Connor Rowling (41:21)

And honestly, I think both examples are equally valid, right? Like all we're talking about is like people becoming higher leverage because E Comm ops can be delegated to an agent and development can be delegated to an agent. And if that goes to the same person and you just have this incredibly empowered E Comm marketer or whatever, that's awesome. It go to different parts of the org. The idea is like the, the human labor that is required to get some of these things done will get. It seems like there's a clear path to it being automated. And that's a. You know what it reminds me of is I was watching a bunch of videos over the weekend on Vibe marketing, which I think is a hilarious term and I don't even really know what it is, but this video is like this guy's basically seemed to be running his whole business from like Claude Chat and he was like, he was doing keyword research. He was like, I want to start a product. Find me all the long tail keywords with this volume that like aren't that competitive. Claude does that. And he'd connected it to like hrefs and all this other stuff via like N8 and it was really, it was really quite cool. But like the idea that this is what you were talking about. Cody, the one person, $40 million business just like hanging out on his keyboard, just vibing, doing keyword research, writing blogs. The guy didn't have a way to publish it to WordPress or Shopify, but like you imagine that's like in the pipeline at some point and it's just like one person who's just vibe marketing their way to like all the, all the ecom updates that you need.

Sam (42:43)

It's awesome.

Connor Rowling (42:43)

Sounds nice.

Sam (42:44)

I gotta, I gotta check out some of the stuff I've seen. I want to check some of it out and see if there's anything usable.

Connor Rowling (42:49)

Yeah. It's been a year since we migrated to Rich panel and I had no idea how impactful the right customer service tool could be. Here's what our customer service team has achieved since making the move. 1. Our order volume continues to rise but our support tickets, if not our self Service portal helps 47% of website visitors help themselves immediately without having to contact their agents. 2. Their AI powered social media moderator is awesome. We continue to spend a ton on meta as a part of our marketing mix and it flags and replies to thousands of comments automatically to help us make the most of our ad spend. Like many of you, I've been cautiously skeptical about certain automation and AI use cases. But our support quality hasn't dipped at all. It's actually gone up. We're at an all time high of 96% customer satisfaction score. All this and it's still 50% cheaper than other platforms. No limits on AI usage or tickets and fair predictable pricing each month. With market and tariff uncertainty, consumer confidence is in the gutter. You cannot afford bad customer experience or overpaying for older software tools. And migration is super simple. They handle your data import, agent training and automation setup. You only need two or three calls and you're live in 14 days. They guarantee a 30% ticket reduction in the first 60 days or your money back. So it's a no brainer if you want happier customers and a happier finance team, head over to richpanel.comdemo to book a demo today. All right, cool. Talked about Shopify AI. Sounds like we'll all be circling back here over the next couple weeks. We've all got a lot of cool stuff going on and things are changing quickly. Next in the docket we've got some operator hotline questions. So for those listening, we've got a link in the Show Notes. If you have any questions, submit them. I think it's included in the newsletter which everybody should be subscribed to. So if you have questions, send them through. Makes for great content on the POD and hopefully helpful to everybody consuming it. So one this is from a low eight figure supplement brand. They asked how would you run increment incrementality tests for a low 8 figure and plus 1 million ad spend? Brand currently 80 to 90% meta and the rest on Google search. Thanks guys. I love the pod. That's from Richard. Who wants to take this one?

Cody (45:02)

I mean you can. I mean I think that meta's user level incrementality testing is available for I believe every ad account. I don't think there's any like I mean definitely for this one. I think if they're spending that much. But I'm pretty sure any ad account out there can use that, that functionality. So I would just, I would start there like run a, run a in meta incrementality test which is a user level test and you should get a pretty good read as a. That's a, that's a great jumping off point.

Connor Rowling (45:33)

I think that's a good answer. I was gonna say, I was gonna take a different route. I was gonna say low eight figures. You probably don't need to be measuring incrementality like you'd like to think it's working or not. You're also a single channel basically 80 to 90% on meta and like Google search. I imagine that's like probably mostly brand and some non branded. It's like meta's either working or it's not increase spend. If your revenue goes up, you're driving incre results. That's how we ran the business for a really long time.

Cody (46:01)

Yeah, in platform, right?

Connor Rowling (46:03)

Yeah, yeah, yeah. In platform conversion Lift for Meta is free. I think we did determine that it is available to everyone. That's probably a little bit more a better of an answer.

Cody (46:11)

But you're just saying just period over period. Like if you're at this scale and you're like screw it, I'M gonna, I'm gonna scale up by. I'm spending 10 grand a day. I'm gonna go to 15 grand a day. Like you should be able to assume there's not like any insane seasonality happening during that period of time. Like you should be able to see that and like give that 30 days and like you should be able to see a period over period lift. If you scale up by 50 and your revenue only goes up 20%. All right, you're. It's probably not, it's incremental clearly, but it's not probably at the profitability clip that you want it to be at.

Connor Rowling (46:44)

Right. But that also then you could do the calculation. What was the, what was the delta in revenue? What was the delta and spend? You get some sort of like, you call that whatever. Yeah, yeah, maybe call it iroas. That's how, that's how I would start. For sure. Cody, would you have any other answers?

Sam (46:58)

For sure. I think both of your answers are good. I think low weight figures essentially, right. Demand generation on one platform, like your business is your increment incrementality test. It's like very rare that you're not, you know, if you're, if 90% of your spend is meta and you're not, and you're profitable and your business grows as you scale spend. Like that is your incrementality test. Like you don't have enough variables where it makes sense. You really want to. I would probably just scale spend and see if you're in multiple regions. Scale spend in a region. You know, like we didn't run, we actually didn't run a conversion optimized incrementality test on Meta until probably it was either Q, it might have been Q1 this year. And like we've ran, we've been on House for a while because I just knew it was so incremental. Like maybe I should have done it earlier, but like we launch a new region, Meta is the only thing we go with. You know, we double spend in that region. Business goes up. Like. Yeah, you know, maybe I didn't know the exact, the exact IRO s, but I knew meta was very incremental for us. So I think where, where incrementality is much more necessary is when you have multiple channels, when you're trying to find the, the right spend level, when you're trying to find how much to allocate between the channels and when you have channels, that attribution is not telling you something. But I, I think in this scenario I wouldn't if you, if you really need to go with the conversion left.

Connor Rowling (48:21)

Um, but yeah, yeah, you know, that's funny. I'll. I'll add for reference, we didn't run, we ran our first conversion lift on Meta 2023 when we were putting the disruptors meta program. And I remember talking to the reps and they're like, you've never done a conversion lift study. We did like $150 million a year or something. You've never done a conversion lift study. And that was my answer. I'm like, look, it's either working or it's not. Like, well. And I think that's probably the more important takeaway is you should be thinking from like an incrementality perspective. And there are many ways that you can try to measure it. Just looking period over period. Conversion lift in Meta is a good one. Just thinking about your entire business. We, we look at, we track this in like our main reporting dashboard, but even just like year over year. So month to date, year over year, what's revenue up, what spend up and what is our like, delta return on that? I wouldn't call that an incremental roas, but it is like, what are the changes in these two metrics and what is the, what is the relationship between them? You can get a really, really, you get really far, like just directionally trying to get that right.

Sam (49:22)

Totally agreed.

Cody (49:23)

And you could do it in reverse too, if you're a little more risk averse. If you're like, all right, I want to actually, instead of scaling up X percent, I'm actually gonna scale down X percent and then just do the same thing in reverse. If you're like, I, I have, I have zero risk right now for spending an additional 5k per day at, at a, at 5k of additional revenue or less or whatever it is. Like, I mean, if we do this, we do this all the time too. On a micro level. Like, we were not super bullish on the incrementality of the level of spend we were putting in Amazon. We scaled back by like 50% last year and I, over the next 30 days, our Amazon revenue dropped like 5% or something like that. So like we, we do this as well. Not everything has to be like, hold out, exposed, exposed. Like it's ideal, but it's. I don't know, it. That takes a long time to run. Like the period over period can be a lot quicker, especially if you don't have a super long consideration period. Like, you'll get that, that insight really quickly.

Connor Rowling (50:18)

Cool. So I like that point too. That's like a little bit more channel specific and scaling down. Amazon's a great one. We've done that before where it's like these high intent channels. You can like really easily spend money that's non incremental because they can just attribute like existing intent and conversions that would have already happened to like to the ads. So Amazon's one, I would say branded search too where it was like yeah, we just took that at one point in time. It was like 5% of our total budget and we took it to like you know, sub 1% and it like, yeah, did not seem to make an impact on total revenues and we felt good about it. And like I think if I'm a low eight figure brand I want to be thinking more like that than some sort of perfect scientific method.

Cody (50:55)

Yeah. And like you can still be data driven. Like that is still a very data driven approach even if it's not like a perfectly structured test following like exactly how you need to set up a test for it to be, you know, considered scientific or whatever. Like just have a data point that you're referencing as you make these decisions and you're probably going to make the right call 70, 80% of the time, I would hope.

Connor Rowling (51:21)

Yeah, that's a funny distinction. Like you can be data driven without being scientifically sound.

Cody (51:27)

Right, right.

Connor Rowling (51:29)

I like it. This next one's pretty long. This is from a leading west coast hospital system.

Cody (51:34)

Interesting.

Connor Rowling (51:35)

Okay, got some diverse listeners for sure. Okay, so I would love to hear your POV on the meta. What are the key platforms, techniques, workflows for marketing measurement reports with audiences that are non technical or external to the marketing team? I've got a bit more to read here, but like yeah, how are you as a marketer, Connor or Cody, your CEO? So I guess you get this from your team. But like when I've got to talk to my team, what's the best way to format that? They say if your marketing team delivers a quarterly measurement report to the C suite, who expects something resembling a slide deck with KPIs and data viz, that reflects the whole of your marketing activities mixed with manually written bullet points offering insights, context takeaways. He says slides seem too cumbersome at times when like a lot of that data just lives in a dashboard. But if you go pure dashboard, you don't get any of like the anecdotal or, or written context. So how are you guys putting that reporting and, and have you found a nice balance of how to deliver it to non Marketing. Let's just call it like C suite people.

Cody (52:35)

Yeah, I think it's hard first of all, like you're never going to strike that balance of like digestibility and too detail oriented. We started doing these like monthly and quarterly reports this year. That, that's the goal is like we start with an executive summary, like how's the business performing with our KPIs that are like representative of the entire business rev spen mer and then it goes down to traffic and then you get into the channel by channel report, which we've basically just spent some time talking about. A, what are the key metrics for each channel that we should care about? And then B, like what are the qualitative things that we should pull out of that? And that's what we do for now. It just goes into a single deck. And then every month, every quarter we create a new, a new, you know, title slide and then everyone kind of goes through the same format and we'll adjust it over time as we like go through it. It's certainly not optimized but it's, you know, something's better than nothing here. Totally. Just to get a pulse check and like go. It's. It's the exercise of doing it that's actually most of the value. So that's our approach. It's not perfect but it, it definitely gets the job done and it gives a pretty high level approach on or I shouldn't say too high. I mean it's high to medium insight on our, on our channel performance. I will say that I think the best brand that does this is Levels. They go super in the weeds with their investor updates on every single channel. So if you're looking for some inspo or something to work towards, I would go to go to Levels. And I actually think they're all public.

Connor Rowling (54:05)

They are public.

Cody (54:06)

Yeah.

Connor Rowling (54:07)

Do you remember the website?

Cody (54:08)

What is the website? Level. Yeah, levels.com and I think that in the footer is where they have their investor updates. Yeah. And they're awesome, like super deep dive on exactly what I just said. Like total business performance. And then they go really deep on every single channel. And then they also the third part is. And here's what we're doing in the next month or the next quarter. So it's like high level, channel specific. And here's what we're doing because of that.

Connor Rowling (54:38)

Yours, Connor, is it. Would you say it's. Is it mostly data or mostly context? Like what is that balance?

Cody (54:45)

I'd say it's mostly data. Like every, every channel Slide is it starts with tables of data. There's a lot of linking to creative, a lot of linking to, like, hey, here's our affiliate performance and here's the. Here's the pieces of content that drove the most traffic and the most revenue. So I'd say, yes, 75% data.

Connor Rowling (55:10)

Yeah, ours is probably the same bucket like our. Our workflow, and I honestly love it. And we'll have to figure out how this will change over time because, like, a lot of our centralized reporting right now lives in Google Sheets. So we have a Google Slide deck that's basically a template referencing all the tables of that sheet that we find valuable. So we just duplicate that over and then refresh it with the most recent data that we want to look at. And then we're just punching in the context or like additional points or next steps or whatever. So it is probably like 60, 70% data. And we're reporting on the exact same metrics. They're like, those tables are literally living in every slide deck. We actually do this weekly also. But I think it'd be the exact same workflow if we were to only do it monthly or quarterly. And that's a great flow. And it's like, so standardized. As we move to the data warehouse, which will improve, like, things will get slightly more accurate. They'll get more holistic across more channels. We'll have accurate cogs data, some of the stuff we'll benefit from with the warehouse, but you won't have that, like, really seamless refresh of like, Google Sheets tables. So we'll probably move to screenshots for Solomon. Like, we would just do that. Because I'd even say right now our, like, paid social director will do a couple slides, and those are just north beam screenshots and things like that. And it's like, I think that's totally fine. We're looking at it directionally and then that context, and we're typically talking through it as well. I also think it's really hard to put together a document that you don't expect to, like, be presented to some degree. Like, if you want it to contain every single bit of information you want, that's tricky. So whether it's a synchronous meeting or some sort of loom or something, I think, like, those are the additional layers, data, written context, some sort of brief presentation on it would be is our workflow.

Cody (56:48)

That's what we're still trying to figure out is like that final piece, like, what's the best format to review this? Because that's a. That's a lot of information. There's a lot of color required to paint that picture. So that's like. Is this like a 60 minute or 90 minute meeting that we have like in the first week of every new month or quarter Reviewing the last month or quarter like that's, that's the, that's like kind of the final piece. We need to like really button up and then probably come back and audit the actual information we're pulling.

Connor Rowling (57:19)

Totally. Cody, anything you want to add?

Sam (57:20)

Yeah, everyone's going to be a little different. We've. We've done a bunch of different things I think because I don't have like a CMO marketing leader like I, I do the just top line just look at it myself and then like share it in. Our marketing meeting weekly is just blended metrics, pacing, you know revenue contribution margin, spend, ame r stuff like that. We used to have like the channel level go over that weekly. We really switched that to like one on ones and Async. So I have everybody. So like depending on who it is either in a meeting or a lot of it is now even like I've just moved a lot of things to Async and we only meet if we need to meet. But like director retention will share her KPIs for the week. Right. Like just repeat revenue email and SMS revenue list growth. Right. You know and, and my expectations for for everybody, you know so paid social will will share theirs Acquisition and paid social metrics influencer will share theirs. My expectations for anybody is these are the numbers we were this much above or over above or below forecast and why so here's the numbers, here's my analysis, here's what I'm doing about it and I think anybody director even even manager level above like that's the expectations same thing meet with you know if I meet VP retail she's sharing the same thing on retail she's sharing. Here's how we're pacing compared to our forecast. Here's why we're. We're below. Here's you know here's what we're doing about it and then same thing. Demand plan gets updated weekly. So that actually is in a deck it used to be I think especially because I am CEO we have a bunch of different teams. Everybody kind of formats things differently. The marketing team will do it. We have a data warehouse but we do it all in a sheet but it gets auto populated from our data warehouse but I just prefer it in a sheet but we have it in power bi as well, we have the same data in Power Bi. Yeah, I just like to keep it easy. So a lot of it is just like a document, like a, you know, like a one on one document. It just gets screenshotted in there. Like our VP retail will like print out like the Power BI thing and like bring it to our meeting. So it's all different. Our demand planner used to do it in email and it just became too cumbersome. So she puts it in a deck and then she'll like take a few slides out of that deck and she'll send it to every director in our company just so everybody can see like how we're doing compared to our demand plan in terms of like unit sales. Um, so yeah, there's a little bit that's all over the place. I think you've got to find what's, what's, what's best for you and your team. And then the one thing we do so paid social, we'll do like a monthly paid social review. We used to do like a separate performance review and a creative review and we kind of do it all together. We'll, we'll, we'll first start whoever's leading that channel. We'll be like, here's meta spend. You know, we'll look at roas, maybe we'll throw in that cost per thousand accounts reach now and then we'll look at every creative concept we launched. So that's probably like the only like big monthly one.

Cody (60:06)

I really like that, that very simple, high level format, Cody, because you could take that, that framework and then apply that to like any level of detail. But I think what you said was here are my KPIs, like the Actuals. And then everyone's gonna have different control metrics or forecasts to compare them against. Here's my analysis on why the KPRs, what they are. KPI's are what they are. And here's what I'm doing about it. Like I love that. Super simple. And then you can take that in any, any number of ways.

Sam (60:35)

Dude, I'm anal about that. And that's a culture shift over the last year that we've really implemented and change and our VP of retail is doing in retail right now is just accountability. Like I don't love week over week metrics for that. I just think they strip accountability. It's like, oh like for example, if somebody's gonna report you on week over week right now they're gonna be like, oh well, we're down 60% but the sale's over, you know, and it's like, all right, how, like we forecasted that in. How are you against your forecast?

Cody (61:02)

Yeah.

Connor Rowling (61:02)

Right.

Sam (61:03)

So I think that is very important. And then what are you doing about it if you're below, like, it's fine to miss. This is what I've told people before because I've had people who didn't do a great job of this and they're no longer here. But it's not, it doesn't look poorly upon you if you miss your numbers. It looks poorly upon you if you're not willing to own them and, and take accountability for them and then share what your plan is to improve them. But if you try to sweep it under the rug, like that's a big no, no 100%.

Connor Rowling (61:29)

All right, Solomon, hope that's helpful. If you're listening to this podcast, I know that you're working hard to drive revenue with upsells and across your E comm site generally. But if you're not monetizing your post purchase experience, you're leaving easy profit on the table. Most brands and everyone listening are optimizing the transaction journey. But stopping at the thank you page and that is a big mistake. After Sell is changing that. We've been using their Rock thanks feature at Ridge and it's a no brainer for us. Here's how it works. You show premium non competing offers from brands like Disney plus and HelloFresh on your order confirmation page. AI driven personalization picks the best offer from hundreds of reputable advertisers. And brands like True Classic, Blendjet and Jones Road Beauty are already using it, seeing 30 to 70 cents lift per order. No disruption, no fees, just extra cash in the bank. And the numbers speak for themselves. After sale customers generate 20 to $30,000 in pure profit per 100,000 orders. There's a 16% average engagement rate which is way better than traditional ads. And it's a new revenue stream disrupting the checkout. It's an easy win and it's completely free to try. Plus, After Sale is hooking up marketing operators, listeners because you can sign up today and you get their entire upselling suite for a year which we also use very heavily. So visit aftercell.com operators to try rock thanks and see the money rolling in. Stop leaving money on the table. Turn your thank you page into more revenue with After Sell. All right, cool. Let, let's talk. This is, this will end up segueing into the test of the week here. Shane roasted, another friend of the pod. One of the few Multi appearance appearance guests. He tweeted, this was a couple days ago. But he goes, say you ran a test where revenue per visitor is flat conversion rate up 5%. AOV down 5%. Are you taking more conversions in lower AOV or less conversions in high AOV?

Sam (63:22)

You say it again one more time.

Connor Rowling (63:24)

All right, you run a test. Revenue per visitor is flat, conversion rate is up 5%. AOV is down 5%. He uses 5%, which like is maybe a little tricky. Like I think you could say 20%. And this question's maybe a little bit more, you might have more of a response to.

Sam (63:37)

I understand. Okay, so same profit, same revenue. You're just changing how you get there.

Connor Rowling (63:42)

Are you getting fewer customers at a higher AOV or more customers at a lower aov? Which would you choose?

Cody (63:50)

I would choose the less customers higher aov. But that's purely based on our business. It's pretty clear when you look at our lifetime value data that when we capture more value on that first order, it also turns into more value over any time horizon you look at. So I'm taking the higher order value person. But I, I think you asked 10 different businesses, you should get 10 different responses because some people, it might make, it might not make sense to do it that way. Like maybe a consumable like Cody is going to say, I want to like optimize for first time order velocity and first time order customers.

Sam (64:27)

Such a good question.

Connor Rowling (64:29)

I, my only concern with that one would be what is driving the higher ltv? Is it that they, is it be, well, what's the right way to phrase this? Is it because they spent more on their first order or is it because they're just willing to spend more over time so that if, if you got, if you like, are you not acquir that person at the lower AOV and then are they deciding not to spend more over time? I think you'd still have a subset of customers even at the lower AOV that just spend more over time because they've got more money or want more of your stuff or bought into the brand.

Sam (65:00)

We've found that the biggest predictor of LTV is first order aov. My, my theory is if somebody has the money to spend, they have the money spent on the first purchase as well.

Connor Rowling (65:12)

Yeah, but I think this one's tricky though because I mean, we're talking about like everybody converting at the same value, right? So it's like that doesn't really happen, at least for any of our brands. Anybody has the ability to purchase more products but if you could only if like, if you say like let's say we sold creatine gummies or whatever. And yeah, if it's 50amonth, you might find that people are subscribed for a longer period of time because they're like less price sensitive. They spent more on the first order. Is that person not converting at $30 on the first month and not staying longer? I, I think that the behavior is still the same despite what that initial order value is.

Sam (65:48)

I think it depends on your business model a little bit. For us. I'm going to most likely without knowing more context, I'm going to pick the higher AOV because you're making more profit. You do have some, you know, while you do have fulfillment variable costs, you, there are even some fixed costs amongst your variable costs. Like you're probably paying the same shipping regardless. And so let's say you're paying $8 shipping. $8 is a lower percentage of $90 than it is $95. So your landed gross margin is probably slightly better even on the same revenue per user.

Connor Rowling (66:22)

And you think that'd be the same if it was like 20% in either direction. You take 20% fewer customers at the higher AOV.

Sam (66:29)

That's steep. I think it would depend what it is. And is that something that you think is sustainable and that you'll see those same results outside of that two week period? No, because that, that would be wild. I'm trying to think what, what test you could run that you would get that result. I guess maybe like a price, like a 20% higher price.

Connor Rowling (66:53)

Yeah, yeah. No, and I was, I love this test of the week. We've been thinking about this a lot at Ridge and I've mentioned it on, on previous podcasts before. Like we reduced our, our prices in international markets because, well, we saw a disproportionate increase in conversion rate. So we were not like net net the same. If we were net net the same, we'd probably keep the higher aov. Let me make a couple points here. We'd probably keep the higher AOV because. And Metab has a great thread on this that maybe we can include in the show notes. But like at the higher aov, not only will you have maybe higher gross margins considering some of the other variable costs like Cody mentioned, but you also, it reduces inventory risk. Right. If you're selling it at a higher value, like you can actually pay back your suppliers quicker. So I think that's helpful. Like your, your, the, the inventory that you have to keep on hand Just costs less relative to the retail value. So that might simplify things a little bit. What we've been, I wouldn't say struggling with, but just experimenting with a lot is like okay, if you reduce prices 20% and conversion rate goes up 40%. So now net net you're, you're increasing more profit, you're acquiring more customers, but you're taking on more inventory risk essentially because you have lower margins on all your products. Like when do you make that trade off? So like again, the AOV plays such a role in your ability to acquire customers at scale that it's like I think, I think that trade off becomes a little bit more interesting and leads me to my test of the week. We launched a product in April, maybe early May. It's $150 wallet. And it was kind of a soft response and like I really just wanted the data point of like what would this look like at a lower value? So we reduced it 30%. We went from 150 to 95. Average order value reduced 32%. Conversion rate went up 90%. So net net profit per visitor is up 25% here. So like good like that. It seems like pretty clearly a win. But even if it were a little bit closer, like let's say if, if it were like a 10% profit per visitor, it's like I do think that becomes a question as to like, is that, is that a benefit to the business given all the things we mentioned, the lower gross margin, the higher inventory risk, the like, is there some sort of like, do we just like having higher priced products on the site and like, does that make gunmetal look more affordable? I think it's like a really nuanced thing. But the other one that I would love your guys's take on is if conversion rate doubles and we want to acquire customers on this product, we've doubled the signal going back to Meta. Does that not also dramatically increase our ability to like scale ads? Is it easier to scale with higher conversion rate with more signal? As long as your AOV, AOV here is still 120 bucks. It's not like we're selling thirty dollar wallets and can't tolerate like a two dollar cost per click or something. So I just think there's a lot of balances there. What do you guys think about that price in relation to conversion rate from meta? Ability to optimize, ability to scale?

Cody (69:57)

I mean it should, it should. I mean that's how machine learning works, right? Is like the more input you give it, the better it is at, at doing the thing it was designed to do. So I think the caveat is as long as you're not passing it like a totally different consumer at a much higher rate. And now it's like now you're getting all this signal that's not really your core consumer. Clearly that's not what's happening in your situation. Like it's, it seems like it's still a similar consumer. So yeah, I mean, it should, it should improve the ad account's ability to target people and to scale efficiently, I would think.

Connor Rowling (70:30)

Yeah, 100%. That's where I think. And what I have to do from here is like, we also have to do a basket analysis. Like I mentioned, the Wallet's now at $95, but the AOV is $120. So like, even at lower margin on that product, we're still selling quite a few accessories that we're going to have really high margins on. So it's like, anyway, there's just so much value to, to create outside of like the, you know, unit profit. Especially when you think about ltv, it's like, yeah, if those cohort values are roughly the same, we get another 10 bucks from each of those customers and I'd want 90 more customers. Like that seems pretty straightforward.

Cody (71:05)

We, we ran a similar test, Connor, during our, our Mother's Day sale. At the beginning of the sale, I always get, I always want to, I like to pepper in, you know, lower value offers to every sale we have. The pushback I always get internally is, well, we're going to cann sales and my take is I think we will a little bit, but not much. And I think the conversion rate bump will get way outweighs any sort of AOV drop. So we tested in the first five, seven days of Mother's Day, we tested only our sets on sale and then our aprons versus every entire stack which was like sets and then all the individual products. We saw AOV come down like 5%, but conversion rate went up like 20%. Same thing. I'm like, great, we're in a sale, so we are getting way more revenue per user during the sale. And we're still pushing most folks into the sets which we know have the best lifetime value. Like Win Win. We're maximizing profit right now and still getting people into the sets that maximize value over, over time. And now I have data to be like, yes, we should do individual product skus. We should do like 200 to $250 offers. As long as the merchandising doesn't the merchandising just has to make sense, right? If our individual products are 30 off, but our sets are 22. Okay, well yeah, of course we're gonna have people going into the individual products. But if the individual products are like 10 to 15, but you get, you know, 40 by buying a 12 piece set, like, yeah, we're gonna push people into the stuff we want and we're still gonna have these like safety net offers. Because I'd rather have someone buy a two hundred twenty dollar cart than nothing at all. One hundred is my approach.

Connor Rowling (72:44)

One hundred. Yeah, dude. And that's what I've been geeking out on this for like the last 18 months. Just because I think if you were. If you're doing tens of millions of dollars a year in revenue and you know, even more if you're doing whatever you guys are doing, like a hundred billion dollars a year, like small adjustments like that just create so much value. Like it just goes so, so, so far. So anyway, I've been thinking a lot about this. Shane's question was good. Led nicely into the test of the week.

Cody (73:11)

So I call it the funnel clearers. Like we just did this with Memorial Day sale or the last weekend we did a flash sale on knives and a board. It's like we've been running knife sets, offers the entire sale. If you haven't converted yet on a set, you're probably not going to convert at this point. All right, well let's get something in front of you that's different, that's fresh, that's lower barrier to entry. And sure enough, like our, obviously our individual knives and that board like skyrocketed throughout the weekend. And we added like, I haven't looked, but we probably added an incremental six figures in revenue on knives that we would not have gotten if we didn't roll that out now. I was strategic about it. I'm like, let's do it at the end of the sale because I want everyone to go a seven piece knife set if they want that. Okay, great. Now let's roll it out. For the last few days we saw a big, big, big bump in individual knives.

Connor Rowling (73:57)

Hell yeah. All right guys, anything else you want to talk about?

Cody (74:01)

That was, that was another good quick hits.

Connor Rowling (74:04)

Great app. Thanks for hanging. All right, thank you again to our sponsors Motion Rich panel. After Sell prescient and house, make sure to like subscribe comment on YouTube, submit operators hotlines questions. If you want us to talk about your e comm issues in the future and share with your friends and family and colleagues. Thanks again for listening.