Loading summary
Transcript101 lines
- [00:00]
A
All right, we're here for episode 119 of the Marketing operators pod. We got all the guys back. Cody's starting today off by trolling me online. So it's a. It's a tough first episode. Back after some time in the woods. How are you guys doing?
- [00:13]
B
Doing fantastic, dude.
- [00:15]
C
I'm good.
- [00:15]
B
How are you doing? You went viral yesterday. You went pretty viral yesterday on Twitter for your. For your chair setup. There were. There were some. There were some haters out there.
- [00:23]
A
There's some haters out there. Don't let the looks deceive you. Our. One of. Our creative strategist, Paige has one of these in the office in la. And I tried it last time. I was. And I had. I was at least 50% more dialed in on this ergonomic chair than I was sitting, laid back, slouched in my. In my desk chair. So I'm locked in now. First. First episode for those listening.
- [00:46]
C
I mean, maybe we can describe it. I'm going to struggle to describe it, but you've got. There's like a. There's like a knee section, almost like, you know, if you're sitting at a pew, that's. It reminds me of like a Catholic sort of hue situation. And then no backing, and then it sort of wraps around, and then you. There's something to support your butt, so it's like. It's forcing you to sit upright, and you're kind of like suspended on top of. But you're.
- [01:09]
A
I think the big thing is like, yes, you're sitting upright, but, like, it leans you forward a little bit. So I just, like, I'm just like. But not in a weird way. I'm not like, right now.
- [01:17]
C
You're on it right now.
- [01:18]
A
I'm on it right now. And it's got a little rocker feature, so if I want to rock a little bit, I can get a little motion going.
- [01:23]
C
And.
- [01:23]
A
And it's unreal. Just go look at. Go look at Cody's tweet and you'll see what the setup is. It's pretty great. I highly recommend.
- [01:31]
B
Yeah, we'll have to put your tweet plus my quote reply in the show notes, but if you guys have ever seen Silicon Valley, there was that guy Gabe, who was that really annoying guy who had the pants chair. And I don't know. That's what it reminds me of. It's just like. It's probably a great thing. It's probably really comfortable, but looks a little silly.
- [01:53]
A
I'm still waiting for you guys to come visit me. In Denver. So when you get out here. Actually, you know what, Connor? Next time in la, which is next week, you got to come to an office workday and we'll, we'll pop you in paige's chair for 30 minutes and, and you tell me how you're feeling after that.
- [02:05]
C
Yeah. You know, shout out Paige. It's amazing office fodder. Like it really gets the people talking, you know, it's making it on the podcast. She's selling chairs to you. There's probably this is going to be big for this weird chair business.
- [02:19]
A
Eric Banholz replied and said someone in his network runs one of these brands that sells these ergonomic chairs. So I'm gonna hit her up and, and try to get my affiliate link going. Cause Cody's right, that, that tweet yesterday was my, my best distributed tweet in years. So I, I need to get the affiliate link going.
- [02:35]
B
And by the way, just listeners. So you guys know the only reason we're talking about this is cuz Connor wanted to talk about David and other Connor and I were like, we've talked about David protein, the last. And like protein in general. Probably the last like 10 episodes. Like I think we gotta cool it on the protein.
- [02:48]
C
It's true. But thanks for squeezing it in there. We've got the streak. The streak is technically still alive.
- [02:54]
A
Yeah, it's like, it's like our snap streak. It's the David protein mention streak. So appreciate that, Cody, for keeping that going. All right, we're talking, we got a bunch of stuff to talk about today. We want to start off with some, some CRO testing. I think all of us have been doing some interesting things on the CRO side of the world. Connor, I want to start with you. You. I was not here last week. I think you guys talked about it a little bit. You just launched a new theme test. So I'm curious, like what was the hypothesis with that? What did you end up doing and what results are you seeing so far? And when you say theme test, is this like entirely new web build? When you say theme.
- [03:31]
C
Yeah, yeah, yeah, yeah. So let's get into it. Cody and I just touched on it briefly and I said I'd come back with some more notes and I've got those.
- [03:36]
B
You said if it did well. So if we're talking about it, it means it's doing well.
- [03:40]
C
It's been live for less than 24 hours, so I'm not calling it one way or another. That's.
- [03:44]
B
That seems early enough to call it.
- [03:45]
C
I did yeah, yeah, it's funny. Um, no, actually for the first like hour we had this weird tracking issue. So the results were actually really bad. But what was funny in that was I was talking with our VP of E. Comm and she said it started out and it was like negative and she goes, she goes, I feel good about it because she goes all, all bad tests turn good and all good tests turn bad. So she was excited to like we've really got like a tinfoil hat on over at Ridge. Yeah, I thought that was really funny too. So we've just got to go in live now. We'll talk about it. I cannot say one way or another whether it will be a win or not. But I've kind of got like how we are thinking about winning in this scenario. So I can get to that in just a second. But I'd love your guys thoughts here. This is, these are the things that I was prioritizing. One, it's a complete theme rebuild. So totally fresh. We worked with Van on it. So Xavier who did the hexclad site, he did the simple modern site. They're great to work with. Total pros. So full, you know, bottoms up rebuild of the theme. And we had a number of things that we wanted to do. I'll start with expanding categories. So we've talked about this quite a bit. We've dramatically expanded categories products within those categories. One of our main focuses was just like how are we providing the navigation tools and sort of like the site infrastructure to support what's a way more sort of nuanced business. That's what I've got here is like number one and that just comes down to like really thoughtful hamburger menus, collections, filters, different collections page tooling so we can navigate between the like hundreds of wallet SKUs that we have at this point. So that was a really big focus for us. Two, we've got in category education bundling. So once you're not navigating between what you want, how are we just how do we have the site, you know, functions to like educate someone about what they need and bundle the products that they need to. So what I put in that bucket is like being really thoughtful about product cards. Once you get to the PDF, how are you navigating to and from kits, sort of things like that. And then three and then I've got a list of like small sort of like tertiary goals that we have. Three is just improved subjective design choices. I spoke to who is the guy that you use? Cody in Canada or have used Andrew. Yeah. I spoke with him and he was like, look. He told me years ago, he was like, look, your site's good. He's like, you guys don't really have a speed issue. Like the site's good. He goes, really? If your founder just wants to redesign the site, he's like, that's totally valid. And that's why we redesigned like half of sites. And I'm like, yeah, that's, that's totally a valid reason for us to include here. So that's the third one that I have is just like we've had more or less the same theme that we've been building on top of for 10 years. So let's just sort of blank slate it, make the design choices that we want and sort of just feel better from a subjective perspective about, you know, the web choices that we're making. So those are the three big ones that I had for goal goals of the, the site redesign. Do any of those stick out to you guys or if you had to redesign your site today? And I know both of you guys are thinking about this all the time, like what are some of the things that you list as sort of top priorities?
- [06:49]
A
I mean, your last one was like the core. I wouldn't say it was like purely like when we did our redesign because we did the Same Thing in 2022 with Van Group where we like did a full on site redesign, totally different experience, and just split test the entire theme against the original one. The core reason for that was we just felt like our website was not indicative of our, of our branding and like the brand ethos, which is just like sexy, badass cookware. That was like the reason we designed it. And then like that was. That was number one. And then once we tested that and found out that that was converting better, then we went into the other reasons that, that you talked about first, which was where we probably should have started, which is like product merchandising. We were adding new categories like quarter over quarter over quarter and they were not discoverable enough. So then we went into like the navigation optimizations, the just like category discoverability stuff. So yeah, I resonate with all those.
- [07:47]
C
I've got a few others here. I'll jam through them quickly you guys. Let me know if anything stands out.
- [07:51]
A
1.
- [07:51]
C
Oh, cleaning up old code, easy. 1. And I actually think this is something that I think of as like a really long term benefit. We've just made not always the best technical decisions over the last 10 to 13 years. So this was our chance to sort of again like Clean slate, the whole thing. Performance, site speed, a big component of that. It is quite a bit quicker. Not significantly. We spent a lot of time on site speed earlier but sort of a nice benefit. And then these are some of the more like tertiary ones. I've got site schema for SEO and LLMs. As we're really digging into the site code, we're like, we're fleshing it out as much as we're possibly going to do it. Let's make sure that we are setting up from a schematic perspective our website in the way that we need. I've got accessibility and compliance. Let's just clean all that up and like reduce all the ADA risk that we can as we're going through this process. On site tracking is another one where it's like for rebuilding all the functionality, let's tag all the buttons and carousels and you know, navigation menu items so that we can just track on site behavior better. We've got Shopify markets and international localization. Our previous theme wasn't super conducive with supporting that. And then similar to site scheme is cleaning up product information hierarchy. We've spent a lot of time and I've geeked out about it on the podcast before thinking about the taxonomy of our products and like what is all the information that we want to append to any given sku and how do we make merchandising decisions or on site decisions or ad naming convention decisions that support that. And like let's just make sure all those dots are connected as much as possible. So that's what I've got. Those are like my, this is the big high level objectives as well as like some, some tertiary ones that we've been thinking about as we've gone through the process.
- [09:33]
B
I knew this was going to come back to naming convention somehow with you
- [09:36]
C
it always does, dude. It's all, it's all interconnected.
- [09:40]
B
No, it's cool. I love that. I think that's really smart. We actually had a. Not a new design, we actually had a new theme go live today. We like refactored our theme so like very similar. I think we will probably do like a redesign next year. We just kind of went one step at a time. We have been on single page architecture as spa. I don't know why. We had this like really fancy Polish dev agency built our original site and like it's faster but like really I, from what I've talked to, you know, developers like a terrible way to build sites just for like maintenance and stuff like that it's almost like trying to go headless. And so we just finally decided to like rip the band aid off. Like, it messes up like we can't run intelligence themes tests with it. It messes up tracking on a lot of things. So we actually like just went live today with it and did a lot of components. So I didn't touch anything front end, anything visual that'll just be separate. And we're just kind of like for now a B test our way through, you know, design changes that we want to do. But definitely cleans up theme and even like it was interesting as we were doing it. This was actually Andrew's idea and I did it. I just ran, I just downloaded the theme files, like when we finally got it to preview, tossed it into Claude and Codex and was just like, hey, we want to run an overall audit, but run a speed audit. And it found a lot of things that like they probably should have found. But like we found like a bunch of different apps, still had scripts, like super old apps. We had like, you know, like super old stuff. So like, I think it's one of those like pain in the ass things. But it's just like very necessary maintenance to do whether it's, you know, what we did or what you guys did. And one of the other things I would, we kind of did here a little bit. But like I would do in the future. Like I talked to Stuart from Revo maybe like a year ago or six months ago about like a lot of the work that they did to really enable them to be super AI, you know, forward and just like building their code code base very like component driven. Like I would do that. Obviously we're using cloud a ton for pages and stuff like that. Like, I'd also be thinking about that as like how we're, how we're building it, you know, for, for. For. For agent decoding as well.
- [11:34]
C
Right. So in the case of refactoring the theme, are you guys testing that or are you just. Did you roll it out and you'll
- [11:39]
B
be like, no, we just rolled it out. Yeah, we're like heavily QA because it's actually our third time trying to go live and found some small things each time and ro it back. But no, but it is pretty fast. We at least did like the normal like page speed stuff. It was cool, like a good hack. I tweeted about it. But like I would recommend everyone just take their theme files, give it to Claude or Codex, whatever you use and just be like, hey, how do I Improve speed here and it'll just like, it'll just find stuff. So I recommend that. But now didn't, didn't test it.
- [12:05]
C
Motion just dropped their 2026 creative benchmarks report and it's been getting shared everywhere. Slack channels, LinkedIn, Twitter, sharing it in our private group chats. And it's great because everybody's been asking the same four questions forever. What is normal? How many ads should we actually be shipping? What is a healthy hit rate and which formats really win? The report analyzes over 575,000 creatives from 6,000 advertisers and over a billion dollars in ad spend to answer these exact questions. And the report has some really interesting findings, like the fact that only 4 to 8% of ads actually become winners and over half of ads actually lose. And for Motion customers, this report is especially helpful. You can upload it into your Motion dashboard with their runneth AI chat and compare it directly against your vertical benchmarks. Hit the link in the show notes. I promise you won't regret it. And as always, go to motion app.com and tell the marketing operator sent you.
- [12:57]
A
One thing I noticed in like kind of the second wave of you explaining this test is the first wave. You're like, we're testing like here are the objectives, here are all the things we changed. And like immediate knee jerk reaction is like, this is a huge swing. You're changing a lot of stuff. But then when you explained it further, you actually made it. You've actually been thoughtful about making sure that all the different variables you changed weren't like, couldn't be manipulated in the same variable. So like the upsell for example, like you guys, you did a lot of optimization on call it product category discoverability with like your hamburger menu. In theory that should improve conversion rate because more people can find the products they want. And it should also improve AOV because your products are more discoverable if you also would have tested a bunch of your upsell components which in theory could also move the needle on aov. Well now you don't really know which variable is is moving that aov. Was it the hamburger menu changes or was it the upsell component? So you were thoughtful in like making sure that you didn't have two of the same change affecting the same variable. Right? Like that's. And then you'll, you'll go into those op. Those like iterative tests after that. Is that, is that true? And how you kind of set that up roughly.
- [14:08]
C
I mean we're still, let's say, you know, Whatever result we get from this test, we won't. We don't have so few variables that we'll be able to point to one thing and say, like, this is what drove this. We still have many things that are different between these two themes, but it is a. It is. It's just a more, like, palatable amount. Like, I'm comfortable with, like, the amount of differences we have now. Whereas if we went in with a new homepage redesign and all new upsells and, like, a new way to navigate to kits, we. And we've got some cool components built out that I'm excited to test. We've got. We've got, like, 50, 60 wallets. I would love to test being able to navigate to more of those wallets from the pdp. So we built out this, like, it's almost like a tabbed swatch set where you could go from, like, bestsellers to NFL or to limited edition, but without having, like, 60 swatches selected there. So it's just like, way more navigation. That's one of those tests where, like, if that's worse, it could be way worse. And I'm not just going to, like, throw that into the bucket. So the things that we have tested, that we have testing, quote, unquote, now I feel like we've got a good grasp of, like, how that might impact performance. And then we'll get to this, we'll be able to transition over and then we'll begin rolling out the new features sort of one by one.
- [15:22]
A
Yeah, I love that. I. I think there also is a way. We just did this with our product page. There also is a way to take these big swing tests and still port over, like, all the findings or the optimized setups that you've tested into in the previous version. Like, we just did this with our product page, where we totally. Like, we basically just gave the entire product page a facelift. With that being said, if you go and look at our. Like, if I go and look at our PDP now, a lot of what's here we tested into, like, our accordion sections and the information we include there, like, these. You know, we tested into a what's included section on a landing page. And we took that learning and implemented in this new PDP build this, like, social proof video carousels. Like, these are all things we tested into in the previous version of the product page or. Or even off the product page, and then pulled it into this updated PDP refresh, which also drove a lift. So it was like kind of the. It was like kind of the finding that sweet spot and like this is a net new big swing. But we didn't just like abandon everything that we've tested into in, in terms like what's the optimized, what are some of the optimized components on this page? So I think that's, that's an important note here. And I'm sure you guys did the same thing, right? It's not like you just like totally wiped out all of your, your learning from the last decade and just like abandoned it. It's like you took those and put it into this newer, overall better experience. So you're still starting at like level five instead of level zero.
- [16:40]
C
100%. Oh, yeah, yeah. Like, like most of the features we built we've tested into over the last couple of years. This wasn't like a. We're throwing the baby out with the bath water and we're restarting from, from some like brand new perspective. It very much. A lot of stuff that we did was informed by, by previous tests we've run. I've got, I've got one other question for you guys on this.
- [17:01]
B
So, Connor, so you tested incrementally and then you shipped pdp. But when you shipped that pdp, did you a B test that or just ship that?
- [17:08]
A
No, no, we, so we, yes, we had, we had a bunch of incremental PDP tests over the last, I mean, God, two years. And then we tested this net new swing and we just. And we did test that as well. So it was like stacking those incremental wins and making sure those showed up in this new updated PDP design. And then we split test that against the globally too. Like every single PDP across every original pdp. So it was like a ton of data that went through it and it drove like a 5% lift in RPU. So we felt, we felt pretty good about that.
- [17:38]
B
Okay, cool. But there was probably like a bunch of variations on that. Bunch of, you know, different variables on that new pdp.
- [17:45]
A
Definitely, like a lot of it. Yes. I mean, I'd say if you go look at our pdp, like if you go look at section by section, I'd say at least 50% had something different. So now this is, and this is what, I'm curious to maybe bring this back to you, Connor, because now what we're doing is basically our, our like, Gantt chart of our PDP has been like iterative testing and that's gotten us to a more optimized product page experience. Then we did this big facelift that we also split tested. That included a lot of the things we. We learned in our iterative testing. And now we're on. And that, like, kind of brought us up here. And now we're going back into the iterative testing to see, like, which of these sections are more important. Like, what happens if we remove the. The Gordon Ramsay approved section in the TV commercial. Like, what. What kind of lift does that. Does that bring to the page or drop to the page? Like, so now we're kind of going back into the. All right. We have a new experience that's converting better and we feel like it's just an overall more aesthetic on brand experience. And now we're going to back into some of these other sections again. So it's kind of like building up to a peak. And now we're like almost coming down the other side of that in a way. So are you guys doing that too? Connor, Is that what the approach will be?
- [18:52]
C
And it's funny, what you describe is very like Sisyphusian, where he's pushing the boulder up the hill and then you let roll back down. Yeah, we'll. We'll take a similar approach. I mean, we're probably not. You guys. It sounds like you guys have a more rigorous system potentially. Like, we've been rolling out new features. We've rebuilt many of those features. We're retesting the pdp and then we've got a plan now for additional features that we'll test. And then. And then for really, when it comes to more PDP testing, it comes down to things like, I'll give you guys a good example. And then I've got another example that I like here. As we launch new products or expand colorways in given products, that typically influences what sort of PDP models we need to test. And sometimes we can pull from different categories. But power banks, for instance, are really the only product that we have that people are interested in buying multiple of the same silhouette. Our other bundling tactics are like cross silhouette. Nobody's buying two or three wallets. They're buying a wallet and a pen and a key case or something like that. So. So those two kidding experiences, a multi silhouette kidding experience versus a single silhouette kidding experience, I think are different. And for power banks, what we tested into just recently, which I'm really excited about, is a really flexible two and three pack bundler where you can select different colorways at each of those. So we built that out. We tested into that. Fantastic. We feel good about it. We're going to like refresh it into the new VAN theme, because we just found this win over the last, like, six weeks. And then when we launch drinkware in a couple of months, we'll have this new feature to deploy for drinkware. We'll be able to test it again. So for us, again, it's a little bit less, like, I don't know what you want to call it, like, scaling up and down the PDP of a single product and more developing modals that we can then test across category for different purposes.
- [20:37]
B
Connor, the reason I asked was because we're redoing our PDP as well. And, like, even, like, how to debate the other day about it internally. Like, so we did exactly what you kind of said on your figures on the homepage, where, like, we tested probably four things recently on our homepage. So we tested technically, not homepage, but we tested Nav, right. So we got a winner on Nav, and then we tested something on our hero, got a winner on that. So that's like, all right, cool. That's our new, you know, version. We tested, like, a few different sections, I think probably, like the hero plus, like, two to three other sections. And so I don't really want to take the time going through every section on the homepage incrementally. Right. It's like, focus on above the fold. And so now what we're going to do is we're going to redesign our homepage based on that. We'll probably test it, but it'll just be like a redesign. We'll have a bunch of variables. So that's what we did. I am. I think that's probably the best way to do pdp. I just don't want to take that long, like six months or whatever it is, you know, three months, like, to. To test, especially because there's. Right. If you think about, like, a, like, homepage, like, there's not as many features. There's so many features on a PDP that, like, you could be there forever. So I'm like, I just don't really want to take the time. So I'm like. And. And I'm normally like, I used to be in the mindset of, like, let's test everything. And now, especially with AI, like, we can develop a new pdp. Like, you can develop a PDP in a day if you want to, you know, like. Like a very different one. I'm like, I kind of just want to approach it a little bit more like an art project. It's like, let's just make the best PDP we can possibly make and like, let's test that. And I'm, I'm pushing for that. And I think some of my team's like, oh, we want to test it more incremental, like, how will we know? And I'm like, there are also ways to know. Like, it might not be a strict AB test, but like, let's say we have a winning variant. Then we can go into heat map data, we can go into, you know, other data and like kind of build a hypothesis of like, hey, you know, even though we change multiple variables, like this likely is Y or, or not. So just, I don't know, like, I just don't want the speed. So do you guys agree? Do you want to push back? Do you disagree on that at all?
- [22:35]
A
I like it a lot. We're doing the same thing with our like one of our hero landing pages right now where we've put like literally tens of millions of dollars to spend through this page in the last three, four years. And now the new project on it is let's go ship this to the brand team and have them make the most beautiful, aesthetically pleasing on brand version of this and just start putting spend through it and see how it compares to the version before it. Our, our CRO lead also has, has, has this interesting idea or approach rather on this, on this like big swing testing concept. Like, because we did the same thing with our homepage like I don't know, a year ago or six months ago. And his, his thought was, all right, let's take this big swing where we're changing a bunch of different things and like, let's just test it against the original version. Make sure we, we have in fact built a, an experience that's generating higher conversion AOV rpu. But then his, he has this interesting approach where his next step is like, assuming the new version performs better. Let's do a multivariate test where all you're doing is removing one section for each variant. And the goal in that is seeing which section is the most important. So you can see like which, which removal, which variant, which section remove drives the biggest drop in rpu. And that tells you which section is, is driving the most performance on the page. And you can almost like stack rank all the different sections based on like whichever one has the biggest drop in RPU has the most importance in driving performance on that page. And that can kind of like steer you a little bit in terms of where you go and optimize next. It's like, oh, if this like Gordon Ramsay Social proof section, when we removed it, that drove the biggest drop. Okay, well maybe we go and like focus there because assuming that the inverse is also true, if we go and improve that section, it'll also drive the biggest lift versus going and improving like a different section on that page. So I thought that was like a cool, thoughtful idea on like how to, how to go about this, like start with the big swing and then like back into the components that are, that are driving the biggest lift.
- [24:38]
B
We talk about incrementality a lot, but how do you actually operationalize it to make your business better? That is one thing that I've been really leaning in with my team recently and HAUS has played a tremendous role. We use it for all of our experiments, all of our geolift tests, testing, but we now use it for our MMM as well. I've been a design partner, I've been one of the early design partners on how to cmm. C stands for causal. So it's one of the only MMMs, if not the only MMM that I've seen that's actually using your causal experiments to build the model. And so that allows me to trust the data so much more. So it's not a black box, but actually informs our roadmap and has been so crucial for allowing us to operationalize around incrementality. The house team is world class. I can't speak highly enough about them. They've also built a really amazing community with some of the best DTC growth operators out there. If you want to check it out, learn a little bit more about cmm, go to House IO operators to start making better data driven decisions today.
- [25:30]
C
One I'm all for. I think it's an interesting thought to say if cost of development has been driven so close to zero. Like do you need. Because I feel like part of incrementally testing the PDP is like also saving on development resources. But if, like, if those, if those become far more affordable and all of a sudden you could rebuild an entirely new PDP really quickly and test that, then it, then it does make sense to just take larger, more explorative tests and then kind of decompose it with Connor's process where you're like. Because at some point you're going to find a PDP that you love and you'll want to know, well, how do I improve it from here? And then it's just a matter of like, yeah, identifying at a more granular basis what is making the impact. And the heat map example that you gave Cody as Well as Connors I think are both good. I like it. Let me ask you guys this so we'll, you know, I'll give, I'll give. I've already refreshed intelligence probably a hundred times, but I'll give it, I'll give it quite a bit more time before we start doing any forms of analyses. One thing we do that I think is really helpful again in terms of like decomposing the results of, of a given test. We'll look at within the theme test like we'll be able to identify if it's driving better or worse performance by category. And I think this is a really interesting way to like try to better understand what's working and why or the inverse of that. One of the things we saw in the, in the past, I've mentioned it before, we found this. We were testing a new hamburger menu many months ago. It had pretty negative performance and it was almost exclusively on our ring program. Or actually though the like, the like interesting data point here is that the tech category was unimpacted by this test and then rings had a significant decrease in revenue per session. And what that ended up being was like our font size was too small, which, which totally tracks for me where it's like if you're buying tech accessories, if you're buying a, you know, a new cool trendy power bank, like you're going to be a little bit less impacted by slightly smaller text. Whereas I think we have like an old, I mean we know for a fact we have like an older male audience for our ring program and they were impacted by the like you know, 14 point fawn instead of 18. So us decomposing the test I think will be like the next step for us. But if you guys were in my shoes, is there anything else you'd be looking at to try to identify directionally, you know, what's working or not in this theme test?
- [28:00]
B
Yeah, I mean I'd look at heat maps. I, I used to look at recordings more. I don't as much but you know, potentially looking at that, um, I mean I'll talk about later but I'm big like user interview fan so like you know, even like subjectively like getting ideas for next test, you know, like what's working. Well I always find that looking at like creative things when you have data is very different. Like sometimes you look at an ad, you're like all right, cool ship this, this looks pretty good. And you get data back and it's like oh, of course like the thought like the hook the hook rate is low or something like that. So I actually think it's interesting to like get some data and then go look at the site yourself and be like, oh, now I see that's maybe not the best UX or we could do it there. So that would be like the process I would do.
- [28:41]
A
Yeah, we've been, I mean we've been testing some new offers lately and we've been looking at like a lot of heat map data to see how people are engaging with these sections and then making iterations based on that. It's like, oh, we, we built this section to kind of dummy proof the offer and make it very clear what it is. But then you find that it's like in the bottom 25% of like engaged components on your website. So it's like, okay, clearly this isn't doing what we need it to do, so we need to adjust this. I've also, we've been launching a lot of like products that are similar to other products. So as an example, we just launched a grill topper, like I don't know, a month ago, a month and a half ago. We already have a barbecue grill pan. It's a super cool product. It basically just allows you to like cook all the food you normally would in a pan on the stovetop on your grill without like all the veggies and stuff falling through into the grate so you get that like that grill taste. And then we launched a grill topper which is just a bigger version. You can cook a bigger meal on it. So we've been having a lot of back and forth on like, what, like where should we put these like graphic treated assets on the product page to differentiate between the two, the two products. And there's just it, you know, you have like all these people in the room with all these different opinions, right? And it's like, no, we don't, we don't need to like let our, our subjective opinion drive where this graphic asset lives on the product page. We can go into clarity and in five minutes look at the parts of the page that are getting engaged with the most. And that's where we should put the, the graphic treatment. Like put it in the, and it's always the carousel. Like that is always where what gets the most engagement. It's like, let's make the second image that like comparison graphic where there's no reason to put it anywhere else. Like that is clearly the data driven decision here. And like you just end especially with website. Like everyone has an opinion on website because everyone's channel touches the website and I think it's important to like have some sort of data driven framework there. So we've been, yeah, I mean I guess a long winded way of agreeing with Cody on like the heat map or Microsoft Clarity like click and engagement data.
- [30:44]
C
I like it. Well, if you guys can keep your fingers crossed, make any sort of sacrifices to the CRO gods. It's a really big test we got going on. I'd love any help that I could get.
- [30:55]
A
I'll sacrifice my old desk chair for you. I'll burn it right in the office here. Thank you.
- [30:59]
B
Thank you.
- [31:00]
A
Okay, so continuing on the vein of CRO testing and we haven't done a test of the week in a while. We'll call this the test of 1h. I guess I, I want to bring this up because what I think is interesting is everyone talks about especially in creative like the creative flywheel and volume and you know we're at a point now where we're in like I don't know, seven, eight markets and we, we certainly don't have the same resourcing in all of our markets. And this, the size of the prize isn't, isn't as big as it is in some markets as others. So like Japan is one of our smaller markets growing but, but still smaller. And like we're just not, we're not launching you know, 50 ads a week in the Japanese market the same way that we are in the US So I think we've had to kind of like just be more thoughtful about where we point all of our resources over there. And one of the big changes we made year on year is just we were really chasing revenue targets and just doing a ton of bottom of funnel marketing last year and it was candidly just mismanaged. And if you would go to look at our like net new visit rate in, in Meta, last year was like 20% in the Japanese market. And this is not a big market, this is like a low seven figure. But if you go look at what we were doing, it was just like offer ads, offer offer ads, offer after offer ads. So the big thing this year was launching more evergreen creative and just like training the algorithm to go and reach more new people which we've successfully done. UGC in Japan is like where UGC in the US was in like 2019. Like you could launch a new UGC ad and it will have like a 6% outbound click through rate and CPA will be cut in half. So that was like phase one here this year Is like we need more, more upper funnel, more creator LED ads. And it became clear in the first quarter that like the egg pan was our big unlock over there. Like we launched a bunch of UGC and the egg pan UGC had like a third the CPA of any other product. So like great, we've just unlocked our HERO product. Fantastic. That's a huge learning struggle with some inventory. We stocked out a couple times. Now we're in a better place and we're still producing, you know, egg pan creative, but again we just don't have all the resources over there that we have elsewhere. So the big issue was well, AOV is low. Like we improved conversion rate in a major way but AOV also came down CVR up more than aov. So I was like, all right, this doesn't, that's fine. But we were over indexed on the egg pan and AOV did come down. So the big win we've had lately is not necessarily peppering in another, you know, 15 egg pan UGC creators. It's been how do we drive up, how do we drive up AOV on the egg pan and and also simultaneously get more sales on these other products. So basically the, the existing funnel was, you know, egg pan ad, mainly UGC going to this egg pan landing page, going to the egg pan product page. And that was, that was a winning funnel. So the new test was the hypothesis was well, if we instead of directing people from the lander right to the product page, take them to a collection page and we create like call it three new bundles that are just combining the egg pan with our other top selling SKUs which is like a fry pan, a high sided pan in the rice pot and then maybe one SKU that's like all four of them together and then we can like position a discount. We will maintain conversion rate because the egg pan is still there as a solo SKU but then also see a nice bump in AOV. So that's been, the test is just 50, 50 split between direct to PDP and collection page. You know, the, the other downside hypothesis would be oh well, you're adding an additional click into the experience and that could hur right and people could drop off. And that's been like a massive win for us in the last two weeks. Like we're seeing CVRS flat. It's actually up slightly on the collection page variant. But AOV is up 23%. Revenue per user is up 27%. We're getting more sales on other hero products because of the way that we're merchandising it. So we're by just like we're, we didn't have to introduce a bunch of new ads around these products, but we're less overinvested on the egg pan now, which is great for inventory levels and a number of other reasons. And now Shopify's up like again, this is coming from like very small numbers, but it's up like 160%. So I just think it's a great example where people get so hyper fixated on like more ads, more ads, more ads. And often it's like once you find an unlock with your ads, you can often like look down funnel and, and continue to optimize that way. And we've been pretty jazzed on this and now we have like a very optimized destination. We can kind of go back to the ads and start developing more winning ads and like keep stacking these wins. So that's been, I've just been very happy with that. It's a very like classic product merchandising play. Whereas like, you know, we were like sending traffic to the homepage last year, like no product merchandising, no product funnels. So it's very much like, you know, 2 out of 10 to a 4 out of 10. But a big win for our Japanese market this year.
- [35:40]
C
That's so cool. So one. Actually, I didn't realize. It sounds like the way that you guys are running Hexclad in Japan is pretty significantly different than in the U.S. was that always the intention when you launch there that like, hey, we don't, we don't expect the like 12 piece to be the hero product. We think we're going to have to go into this market with, I think you said a rice cooker or a rice pan, rice pan, egg pan, like all these different things. We're going to like approach this. Like we're going to refigure out what our hero product is here. Was that always a goal or is that something that was identified and then you guys have started testing into more recently?
- [36:13]
A
No, that was not the goal. We, we developed this product and the obvious, the obvious place to launch it is in Japan. And then even when we did that, it wasn't like, it wasn't until we actually started building funnels around this product that it really started taking off, which is like again, it's the classic, if you build it, they will, if you like launch it, they will come is not the truth. Like if you build funnels around it, they will come and they will Convert. So no, it took us like, I mean, first off, we didn't even have this product when we launched the Japanese market. So it was like, you know, at least a year, maybe even longer, until we even launch this product. Then another six months before we like actually started building funnels around it. And then we found these unlocks in the ad account and then the AOV bump. So no, it's been like figuring it out as we've gone was very much not our hypothesis going in that like the egg pan funnels would be what? Because it's not a super pricey product. I mean, it's, it's a single. It's, you know, a single pan, basically. So it's been, it's been interesting.
- [37:13]
C
Rich Panel CEO just predicted his own SaaS product will lose 90% of its revenue and he's the one building what will kill it. They are betting that AI agents will replace traditional support tools entirely. No dashboards, no seats, no workflows. So what did they ship? An autonomous AI agent that handles customer support from end to end, reading order data, issuing refunds, updating shipments without a human ever touching it. It means you can run a faster, leaner support team at a fraction of the cost while resolving tickets instantly. 24. Seven rich panel is cannibalizing their own business to build the future of CX and support for brands. Because if they don't, someone else will. Go check out what they're building@richpanel.com and learn how you can automate your entire support operation today. I love the idea here, right? Like you're having success with the egg pan. You were driving people to the pdp. You're selling it. Fantastic. By adding this, like interstitial where someone can choose between a couple different bundles, you've increased AOV without sacrificing conversion rate at all. I think that's a great learning lesson. Can you navigate to the egg pan bundles from the single egg pan pdp?
- [38:24]
A
No, but that's a great idea. Like, to have like a variant selector right on that product page is a. Is a really good idea. We did make, we did set up the core website in a, like, in a way that, like, if you go to our hexcloud jp, our hero section will navigate to the collection page that we're running the split test with. So we made the core website very like. Our hero section is now all about the egg pan. That then takes you to a collection page that has the egg pan plus the collections. And then what. What is not what is the same as our hero business is our evergreen offer merchandising. Like, we are always, we always are able to run offers because we're such a big bundling brand. So it's like, all right, if you bundle these things, you get a discount and that's an evergreen offer. So that's what's really working here is we're able to say, hey, if you buy one of these. One, one of these product or SKUs rather, that has two products in it, you're getting 10 off. If you buy the. The SKU that has all four products, you're getting like 20 off. And the. And the. Now the second best performing SKU is the four. The four product bundle, which is great. I was expecting it to be one of the. The two product bundles, but yeah, so we'll probably sunset some of the bundles eventually that aren't doing as well and just make it even more streamlined. But I like the idea of adding the, the variants to the, the Egg Pan product page. It's a good idea because, yeah, that's
- [39:45]
C
where I go all the time. It's like, it's the same idea of what you brought up earlier, where the best way to test that idea is with the interstitial. Interstitial collections page giving people the option to click between a couple and then it's like, oh, yeah. And I have no idea, could be worse. But can you compress that further and just like give those people those same options? If you make it visible from the single Egg Pan pdp. The other thing is. So this is a paid funnel you're describing. You guys must also merchandise the, the new Egg Pan bundles from someone landing on the homepage as well.
- [40:17]
A
Yeah, so that's. Yeah, that's what I was saying. We're like, if you hit the, if you hit the homepage, it's now all about the Egg pan. And then the CTA takes you to that collection page. We also like this collection page. It's kind of going back to what you talked about earlier. We actually added additional components other than just a pure collection page with additional SKUs. Like, we have a HERO section with a nice image that kind of explains the product. We have like a HERO card as the first SKU in the collection, which also has like a hype reel overlaid on it. So it very much was not a single variant test. It was like, how do we just make a much, much, much better experience which includes multiple variables. And then of course, now we can go and kind of back into some of these more iterative optimizations. Which is, it's interesting. This is like a pattern across all of our, our CRO programs.
- [41:06]
C
Right. And that's what I was going to say is it's super fun. This is a great example of like you build all these tools and strategies selling seven piece sets in the US and then you get to apply a lot of them to the egg pan in Japan. We're in a somewhat similar boat right now. This is a little bit different, but with TikTok shop, like our hero product is just our tracker card, individual tracker cards for like $30. It's far and away our most popular upsell. It's extremely popular to be bought along with the Ridge Wallet. But we're seeing success with it as its own hero product in both TikTok shop. There's some Amazon success and we're trying it on D2C now. And it's like, it's a completely different funnel AOVs of like 40 bucks or whatever. Because we're rolling out bundles, we're looking at cohort values because all of a sudden we're getting someone in who hasn't even bought the wallet yet. So we're seeing higher sort of incremental LTV post purchase, which is super cool. And it's just fun. You get all these like you've developed all these learnings and strategies over time and they kind of, you get to reconfigure them in new ways depending on product, market, channel, et cetera.
- [42:09]
A
It's fun to. Yeah, it's fun because there's a lot of difference, but there's also a lot of similarities, right? Like how we're setting up the test. Like we've run similar AOV boosting tests in the US just around different products. And again, it's like it feels like buying media. I feel like I was at like homestead in 2020 buying like launching new UGC ads in an account that's never launched ugc. And you're like, like literally almost overnight the CPA on those new ads is half of what it has been for the last 30 days. And you're like, it's just like so obvious, you know. Whereas at like our brand scale now, these, these wins aren't as like big, right? You know, you rarely launch a new ad that just like cuts the ad account CPA in half because you've been launching literally tens of thousands of ads for the last four years. It's just harder to find those wins. So it's fun to, fun to kind of launch new ads in these less mature ad accounts and Less mature markets as far as just like Facebook ads goes. And see, see those like kind of like 2019, 2020 wins that you used
- [43:05]
C
to see is are you guys going to move. Is the egg pan live in the US and are you going to be applying any of the strategies from Japan and testing them in the US it is.
- [43:14]
A
We actually just launched it in the US like two months ago and it's, it's not as big of a product over here. Obviously. It's a very niche. Like basically the, there's this style of omelette that Japanese culture makes. It's really, it's really cool. It's like this very like delicate process I guess. And it, they're really good. So it, and it's just less of a known thing over here, so it hasn't performed as well. And like we might roll out some of the strategies but honestly I don't even know if the size of the prize is, is worth going down that rabbit hole. We still have it on like if our, our U.S. site, like we have a homepage section for right now because we basically our go to market strategy for new products is every time we launch a new product, for the most part we'll create a homepage module for it and that will sit there until we roll out the next product that is worthy of a homepage module. So we've had that on our site for like, I don't know, two, three months at this point. And like again, it's, it's not, it's performing pretty well, but it's not like it, it's changing our business or anything like that.
- [44:15]
C
Yeah, I need the egg pan. I'm looking at it now. Dude. It could be a great TikTok shop product.
- [44:19]
A
Yeah, we should, we do not have, I don't think we have it live there yet, so we actually should. Honestly, the rice cooker is probably more of the. Has more upside in, in American culture because I think more people are probably likely to, to want to be making white or brown rice than this like Japanese style omelette.
- [44:35]
C
Dude, true love it.
- [44:36]
B
Have you had one of those omelets? They look so good. I always love to watch people like making them on Instagram.
- [44:42]
A
Yeah, I've never made one, but I have had them last I was in Japan a couple years ago and yeah, dude, they're, they know how to make some eggs over there, I'll tell you that.
- [44:49]
B
They look so good. Yeah.
- [44:52]
A
All right, so we're, we want to, we want to kick it over to Cody because Cody's been doing a lot of. Cody, you've been going deep on. On CRO process and like using AI to totally revamp your CRO program in an efficient way. And just we were texting last night. I think you're doing a lot of things that most people aren't doing. So drop some wisdom on the, on the listeners right now on the latest and greatest and CRO in 2026.
- [45:16]
B
I'll talk like no AI first. Because like the way I try to approach these, like building these AI workflows is like, what's the best in class version of like fully human, right? And like, we don't necessarily have the bandwidth or budget or team to like do all of that. So then it's like, all right, how do we actually enable this? But that's like my thought process. And one thing I like, I've always been really. I actually think I used to be like, pretty bad at CRO. And like we used to be. And like, we just like, you know, come up with tests like, all right, that's, that's a cool test. And like not do it from first principles. So like, I mean, obviously that's the most important thing. So number one is like having a framework for testing and have just like piece things together from what, you know, we've learned. Seeing other things, seeing like agencies do things well, but like really try and be really specific. So like we have like a, you know, a few different points, like always above the fold, highest traffic pages, like probably like no brainers to a lot of people, but I think just like codifying it and having it, you know, there. So highest traffic pages right above the fold. Obviously, if you can identify high traffic pages that are lesser converting or lesser revenue per session, like huge opportunities there, just like trying to be, you know, high leverage with all of these things. So like, there's a few different things. And then one of the ways that we test, which I haven't seen many people do, and I think it's controversial because everyone has different feelings about like, like stat sig and that you need is we were working with this agency and they never ran like a B test where it's just like two variations. It's almost always they would run like, it's technically not multivariate, but it's like four variations. If you have enough traffic and the way they would their thought process, if you're taking a, you know, you only have a certain number of blocks and. And this actually works really well in the CRO area era where, you know, you have more Ability to, to Connor's point of like, cost of development going to zero, you have more ability to develop tests than you can actually run the test. And you know your testing slots. And so rather than just taking a two week test and doing, you know, a, B, two variations, they're usually doing four variations. So most tests that we run look like this. So you have obviously a control that's one of your slots, then we have four other slots and the variations stack. And so, you know, if I'm testing a homepage, I'll. I'll make it up. Right? And you're testing like a hero module. You might change the button copy on variant one, and then on variant two you're going to keep that change, but you'll add on one other change on top of it. You might change the button copy, plus you might change, you might add a new button and then on variant three you'll have the button copy, add a new variant and you're going to change the image. And so at least that way the one that wins, you can isolate and be like, all right, what's a step below it that it didn't have? Um, but you're able to just test a lot faster. So I think that's one thing that has helped, helped us to be able to improve hit rate. You're just getting more shots on goal.
- [48:01]
A
Wait, so can you, how do you, can you talk me through how you would like, interpret the results from a test? So it's basically like plus, it's like plus one. It's like original new variant is V1, V2 is like two new variables. V3 is like three new variables.
- [48:19]
B
Yeah, so you can always isolate what the variable is. I'll give a specific example. We just redid our collections pages. And so on the first version of Control, we just changed like, you know, in the top where you have like the different categories. Like, we like change the UX of those. On V2 we kept the new version. Plus on each product card we indicated how many shades there were. On V3 we kept those, but then we actually added the shade swatches. And then on V4 we kept all those. But rather than driving just to PDPs, we had a Quick Shot module. And so each test was slightly different, each variation was different, but it only had one component. So like the Quick Shot module didn't do well, even though the one before it was a winner. So we were clearly able to say like, hey, that's the variable that is losing here. Like, people don't want Quick Shot Model.
- [49:12]
A
It's almost like the. Like, what I said, but in, like, the inverse. Right. It was almost like big swing. And then you, like, remove each component, but you're saying, like, you add each component. Which I think is better. I think that's a better.
- [49:27]
B
Yeah, yeah. But, like, so how I'm gonna approach this PDP test is actually designing the perfect pdp. Like, that's where I'm starting. And then from there, we're gonna kind of remove components to kind of get back down to, like, it won't be perfectly clean to get back to our control, but it'll be similar. So I guess it's. Yeah, it's a different way of doing it. So that's how we run almost every test. And again, I think you can only do it when you have enough traffic. And I still think some people don't think it's stat sig enough, but I'm willing to give up a little bit of that, you know, perfection in stat sig to test fast.
- [50:01]
C
And in a test like this with the pdp, will the main KPI be conversions, or will you look at things like, you know, add to cart, begin checkout. Will you look more upper funnel so that you don't need as much?
- [50:16]
B
That's a good point. I think I said this. I think it was maybe when you and I were recording, like, I'm really bad at looking at, like, the funnel steps, and usually I'm just looking at, like, revenue process per visit. You know, I think we should do better at that because I think that would be interesting. Just give it more data.
- [50:33]
C
I had this thought earlier as. As Connor was teeing you up. This is mostly a joke, but, you know, you're going so deep on AI. You're going so deep on CRO. Do you think you're approaching any form of, like, CRO psychosis?
- [50:48]
B
I mean, definitely, like, AI psychosis in general.
- [50:50]
C
Yeah, yeah, yeah. It's like. But it's the CRO version of that. It's a very particular form of AI psychosis related to making optimizations on your website.
- [50:59]
B
It's oddly specific.
- [51:02]
C
It's a look. It's a low tam. Low tam tweet potential here. Yeah, I thought I'd get you.
- [51:08]
B
No, very, very much like, so we have this. The HERO system, the skills, things like that. We can rate things. And so I. Forget if I said it, but so I have this, like, you know, agent, like a, you know, a Hermes agent in Slack now, and my director of growth and income, like, put a test in and screenshot it. And I just tagged the agent and I was like, hey, rate this test. And it obviously did it based on our skill, which is our framework. And then I asked it to mock it up and it was like super cool. It just like it actually like gave a. It created an HTML and it just put the screenshot in Slack without me asking. Yeah. So I was like, that was like a little bit of a psychosis moment. I'm like, holy. Do that
- [51:46]
C
before Q4 hits. The smartest e commerce operators aren't just asking how do we spend more, they're asking how do we make every visitor more profitable? That is where After Sell comes in. After Sell helps E commerce brands increase revenue per visitor, contribution margin and incremental profit by optimizing the highest intent moments, cart, checkout, post purchase and even the thank you page. Brands like hexclad, Ridge and Pela are increasing their AOV across every cohort using these zero risk upsells we've tested and they have no negative impact on conversion. It's just more free money in your pocket. And with ROKT thanks, you can offset your after sales subscription. No redesigns, no risky funnels, just better economics. Want to see how much money you can make with after sale? Go to after sales.com Rocket Revenue Forecast, put in your numbers and see how much more profit your funnel will make before BFCM starts. Okay, so user, you guys are working on building the quote unquote perfect PDP right now. Are there any other like high leverage touchpoints that you guys are testing frequently?
- [52:44]
B
I mean we're, we're trying to do it all right now. Like that's because we can just, you know, we just had, I think she's on second week. Our new Ecom zero person like now between her and I like handing things off. I'm obviously going to do less now. But like you could, you know, you can, you can parallel path 5 different tests at the same time, you know, and just have them in different windows open. But yeah. So the other thing I want to talk about is like I have always been really bad at like, like heat maps. I struggle with like I have a hard time going into Heat Map and looking a. I think I was the first person has a Heat Map API. That helps because I can then just have Claude be like, hey, what's, you know, what am I looking at? But I don't know, I've just like struggled at like just looking at heat maps or even like user recordings like I used to watch. I'm like, what is this like so this person left like what is it telling me? Or I just don't have the patience. The thing that has always made the most tangible sense to me is, is asking customers. It's just like the qualitative and I just think that people are not doing enough of that. I think software does it well. You know, like I just think in D2C it's like much more like data driven and much less so we, we do a few different things. But I've found it to be incredibly helpful and obviously AI, but I think in general I just get so much more out of like a bunch of either customers or prospects looking at a site and being like, this is what I'm confused about. So like typical, you know, user research. I know you guys know, but for listeners is like, you know, there's, there's sites like user research.com or there's a bunch of other things where you can, you can give people a prompt and have them just pretend they're a prospect. Go to your website often they're like, you know, stay at home moms or whatever. But you can even like do it by demo, depending on what tool you're using. And they're just like, hey, this, I don't, I'm confused. I'm looking for this product and I don't know where to find it or whatever. And then obviously if you. The old way to do it as manual. Now I can talk about how we do it with AI, but you can consolidate a lot of those together and you know, try to automate that. That's been, I think one of the, one of, if not the biggest thing for me is so there's obviously you have all your, your quantitative, your GA4, your heat map, your intelligence. That's all important. You have your reviews. Like you could obviously use AI to analyze your reviews, but, but just that user research and whether you're physically getting on a call with people, talking through it, whether you're using one of these softwares, whether, like whatever it is, like I think people should do it more. And it's been incredibly helpful for, for us and any big project. Like we kick it off like we'll do a study before we're doing any
- [55:06]
C
big project and you'll use some of the tools you mentioned.
- [55:10]
B
Yeah, yeah. So we have like, we use two. So we use Typeform, which is just like a form builder. We have a customer advisory group. And so we'll, we'll give it to them and we'll just be like, hey, thinking about redoing our Product page, go to it, open it, take the survey, answer, you know, these questions. Right. And some are open ended, some are before, after that. Obviously has an API that feeds Claude. We'll obviously use, you know, unit reviews, stuff like that. And then we use, we use Listen Labs is what it's called. I know Simple Modern uses it too. It's, it's very cool. It's like, it's, it's like Typeform had a baby with like actually talking to customers. And so you create your questions but people will, for each question people will actually record themselves on video. And so it's, and so you know, obviously then AI will, their, their AI models will, will scan that and consolidate the insights. Um, but so you can literally get people talking and being like, hey, I'm going through the website, this is what I'm confused about, you know, but you can send out a hundred of them, hundreds of them and then either put it in cloud or have their AI thing like analyze it to, to like find similarities. So it's almost like you get the, you get like the depth and the insights of actually talking to customers, but you get like the speed of like surveys. So yeah, those have just been so incredibly, you know, helpful for us.
- [56:27]
A
Then are you, are you just taking all this data? Actually 1, 1, 1 question for that. Are you doing more of the, of the surveying or more of like, like actual, it sounds like you're not doing actual, like you're not necessarily doing real interviews or someone on your team's not actually getting on a zoom call or Google Meet with someone and asking them questions. It's either the surveys or Listen Labs and it's like a recording of the person.
- [56:51]
B
Yeah. So it's, it's combination of both of
- [56:53]
A
those and then you upload all the, that data into Claude and, and Claude can then like look at all of it and say, hey, here's the number one issue that is popping up the most kind of the same way that people will like upload reviews to Claude and say like what's the number one thing people like about this? Or the number one thing people don't like and then create like ads and emails and stuff around that.
- [57:12]
B
Yeah, so we have like, yeah, so we have like APIs from them that, that feed in. But yeah, so essentially we get data from Junip, which is our reviews we get from Typeform, which is like our customer surveys. We do Listen Labs which is like the user research prospect stuff. There's a few other sources like post purchase surveys we have. And so that's like the qualitative side. And so that'll be part of, like, when we kick off, like a CRO process or something like that. It'll have that. And then it'll also use the quantitative side. That's like your heat map, your GA4, your intelligence test. And so those are like, six, seven, like, really rich data sources. And it'll pull together and it'll build a brief. That's like, the first step is to, like, build a brief and, you know, put it all together. Here's. Here's what people are saying. Here's how the data validates that or doesn't validate it, you know, and it'll, like, put it into brief and actually, like, recommend how we should test. So, like, that's our first, you know, step of it. But I think, I think a lot of people listening probably do a really good job with the data side and whatever. I just like, however you do it, I just think, like, talking to customers, that qualitative, like, user research is for me, incredibly helpful. Like, we'll. We'll run one of these, listen, lab studies, and I'll have, like, 20 ideas to test. I look at Heat Map and I'm like, I don't know what I'm looking at. I don't know. Like, I just, I personally struggle with it. So I just have a. An easier time with the qualitative stuff.
- [58:37]
C
So are there any. I'd love, like, are there any examples that stick out in your mind of things that you began testing or learnings that you developed from this more qualitative approach?
- [58:48]
B
Yeah, yeah, Let me, Let me think. I mean, you know, we. Our last six tests have been winners, and I've never had that before. And I truly think it's. It's a big part because of this, just because of the system, you know, on. On testing, some of the homepage stuff, we tested this, like, shop by category, where we used to just have. We used to just have, you know, because what we would do before is like, oh, we think people want bestsellers. Let's test bestsellers. And it's like, well, let's ask people what they want, you know, because I got a lot of our. And you mentioned earlier, a lot of our, like, best practice tests have not done well. I think that was like, my biggest wake. I was like, all right, we have to do something different because, like, we're testing bestsellers, we're testing social proof, and it's not working. It's like, what do people actually really want? Let's just ask Them. And so, yeah, we used to just have, like, featured products here. And it was like, all right, what do people want? What we found out was existing customers. And it's also why I think you need a combination of existing and, you know, new, new to brand people prospects. Existing customers want to see what's new. They want to see your launches and what you have coming in. But right, somebody new to site, they want to see, you know, what are the best sellers, what are highly rated. So we have this one, rather than having a bunch of different sections, we have the shop by category that has different tabs that you can just filter on. That's best sellers. New kits. Face, eyes, lips. And so that was one that we built that performed really well.
- [60:13]
A
Once Plofker. Once Plofker CRO, launching the agency. It's gotta be coming also. You need to. You need to. In good DTC fashion, you need to come up with a name for your, like, variable plus one approach to testing
- [60:29]
B
and then do, like, one of those tweet comment threads about it.
- [60:32]
A
Yeah, exactly. Yeah.
- [60:34]
C
We need to be coining more terms. I had breakfast with Nate Lagos over the weekend and he's all about that now. As marketers, it's our job.
- [60:41]
A
Yep. More acronyms. There's not enough. We have to name things and then. And then shorten it with an acronym.
- [60:47]
C
No, I'd like to move away from acronyms. And I'd like more. Maybe I've mentioned this before, but, you know, you've got. You've got, like, Bulgarian split squats and Romanian deadlifts. And, like, those names are so much cooler than if they were just, you know, deadlifts and split squats. So we need that, like Cody's.
- [61:05]
A
Cody's testing. It's like blank multivariate CRO testing. Yeah. All right. We gotta fill in the blank Cody for next episode. So that's what we're doing the intro on.
- [61:15]
B
We'll just brainstorm them for an hour.
- [61:17]
A
Yeah, yeah.