← DTC Podcast
DTC Podcast
Ep 520: How Pilothouse Tests 30+ Creatives Weekly on Meta: A Practical Guide | AKNF
00:24:46
Loading summary
Transcript44 lines
- [00:00]
Eric Dick
Cpms have been down a bit the past month. Cost of traffic is more affordable right now. Overall returns seem to be climbing a bit often July and August can be a bit of a lull. Good time to take advantage, stock the pond as much as possible.
- [00:14]
Jacob
How do we think about testing at scale on these platforms?
- [00:18]
Taylor
This is a big question. I think it really comes down to people may have noticed in their ad accounts they've got value rules that are coming into advertising settings and the ability to set those. From what I've heard, Meta is a looking at trying to give advertisers more input at that level.
- [00:37]
Jacob
Prime day is in July. What changes in Meta on Prime days? Do you get more conversions? Are you ever driving to Amazon?
- [00:44]
Eric Dick
Yeah. So basically it's just.
- [00:53]
Jacob
You'Re spending real money to drive traffic to your site. But the truth is 98% of that traffic leaves without ever giving you an email. Meta knows who they are, but you don't unless you use Instant. Instant helps brands identify a anonymous visitors. You can send more abandonment emails, build stronger retargeting audiences and actually recover that lost revenue. Brands like Fate the label saw a 4.5x boost in their abandonment flows. Massco increased Shopify revenue by over 15%. Third Love, Toys R Us and the Oodie. These are all massive stores that use Instant to see an average ROI lift of 11.2x. They'll even guarantee you a 4x ROI or your money back. It only takes 15 minutes to set up and you'll start seeing results fast. If you're serious about getting more out of the traffic you're already paying for, now's the time to fix it. Get 50% off your first month at Instant One DTC. That's Instant O N E DTC. It's all killer, no filler. I'm Eric and I'm here with the Pilot House Meta Brain trust. We've got a smattering of topics to cover about Meta today. One of them is Cody Plofker's exciting post that he made about how brands are testing at really high volume. So I'm excited to dive into that, but I'm just curious, like on the overall temperature check, how is Meta performing for our clients at a high level right now?
- [02:14]
Taylor
Taylor I'd say things are looking pretty good overall. It's definitely been an interesting, like recent couple of weeks this year. To date things have been solid. The platform's kind of doing its thing as it always does. But more recently I think we're in that kind of transitional period of the year. So, you know, coming out of Mother's Day and Father's Day, moving into, I guess at the time of this recording, we're just a few days from the official start of summer. So it's definitely feeling like there's a bit of a switch up in some of the ad accounts as far as, like, what's working and where some of the trends are. So that's been top of mind for us right now. But I think, I think given the challenges a lot of brands have faced this year, whether it be with tariffs or other components like that, a lot of brands are starting to get ready for going up in volume and being prepared to jump into the Q3, Q4.
- [03:07]
Jacob
Season, stocking the pond, as we like to say every year on this podcast. Jacob, any other observations from clients in the market?
- [03:15]
Eric Dick
No, yeah, pretty similar. Like as a whole across, you know, all our clients, CPMs have been down a bit the past month. Like cost of traffic is more affordable right now. Overall returns seem to be climbing a bit. So. Yeah, well, well, you know, it's basically three days from summer here. Often July and August can be a bit of a lull for a lot of a product unless you, you know, have a summer product. So good time to take advantage while the, the traffic is there. And really, yes, like you said, stock the pond as much as possible because the moment you get into September, October, November, it's just like every month is gonna keep up ticking and you're gonna wanna have momentum by then.
- [03:56]
Jacob
So this, so this post by Cody that you is actually a reply to an Olivia Corey comment about how I can't keep track of how we do creative testing on here. Are you guys all launching new creative into business as usual campaigns right away? Do you have a testing sandbox? If the latter, what's the criteria for graduating an ad out of the testing sandbox and into business as usual? And Cody's answer was that they currently use abo, which is. Is that Ad set Level Bid optimization.
- [04:24]
Taylor
Ad set Budget optimization? Yep.
- [04:25]
Jacob
Yeah, Ad set budget optimization. They're launching 30 concepts a week. Each ad set is at a low budget. And then they're maybe graduating things out of there. I'm curious how maybe. Taylor, kick us off here. How do we think about testing at scale on these platforms?
- [04:41]
Taylor
Yeah, this is, this is a big question. I know, I don't know the exact number of replies in that thread, but I know there are a lot from a lot of people who are well respected and well known in the industry. And I think like the fact that it is such a big discussion you're like, that probably hints at the sentiment that I'm going to share with. I feel like I'm the it depends guy, but I think like I've been having a lot of these conversations recently as well with various people in the industry. We go back time and time with our brands as we, we take a look at how we're testing for them. I think it really comes down to a lot of factors that extend beyond even the ad account specifically like, because when it comes down to it, you know, there are some brands I work with where I might even launch net new concepts directly in a scale campaign. There are some brands I work with where I run the ABO structure given the creative volume and the performance of abo. There are some brands where I'm testing in a CBO with ad set minimums and graduating from there. A lot of it kind of comes down to the risk appetite and the hit rate or the success rate of the content and how you're, how you're approaching it that way. So I think it's, I think it's really interesting. Like some brands, you know, they have formulas developed, they know what works and we've been learning more and more about what Meta's algorithm prioritizes, which is an interesting topic related to this, like how they understand what is net new creative versus what is not net new creative or notably different creative. So it's, it's tough to give like an umbrella statement for it, but brand to brand, I'm doing it a little bit differently and a lot of it is kind of treating it like I want to diagnose what the best opportunity is for that brand given where they're at, how all of their channels work together, where Meta fits into that and ultimately where we want to go from there.
- [06:31]
Eric Dick
Yeah, I would say it's like, it also comes down to exactly what you're trying to get out of the test. Like is it just trying to find the winning ad as quick as possible? Is it that you're actually need an apples to apples, like hey, we need to test the different colorways of this ad because we need to translate that learning to our website. In that case, ad set budget is kind of needed because you actually need an equal spend for that comparison to make sense. If you're just like we need to test all these ads to find the winner. Yeah, like Advantage plus probably makes more, more sense now. Like Meta's gonna, you know, find that winner quicker with their, their AI like delivery system and everything. So Yeah, I think it comes down to exactly what you're trying to get out of it and then really the pathways become more clear as well.
- [07:21]
Jacob
It's almost like you could test really big concepts at the ABO level, but then all of the like subtle variations within those concepts within asc. I don't know if that really makes sense, but I guess the problem is if you're not forcing budget into all of your concepts, you may not give them a chance to find their audience, I guess. Right. And with this idea that your creative is your targeting and that as you maybe target different avatars with your creative, you unlock different aspects of the meta audience that it then has to find. So I guess part of it is that you're forcing meta to assess each one of your creatives when you're doing it in that aboard standpoint. Like how often do we, do we make creative that just doesn't even get a good look? Like that just doesn't. You know, obviously you've got a bunch of winning creatives that are, that are building, you know, building the account. But like when you try to run new creative, it just won't ever get a foothold. Is that something we run into?
- [08:15]
Eric Dick
Yeah, like definitely run into it. And like by philosophy, if we are running a pointed test, we try to get basically like whatever your average order value of your product is, you know, maybe get at least that amount of spend to every creative to give it that chance at least. But you're also looking at all the metrics, the clicks, the add to carts and yeah, if you launch them all in an advantage plus I'd say like if you launch 10 ads in there, three or four of them might get like 2 cents a spend a day. Like there's just those bottom dwellers that matters just knows somehow immediately like those aren't going to work. But then you actually test them in an ABO campaign with dedicated spend and like a few purchases do come through. So yeah, again just goes back to like if you need an apples to apples comparison and you're actually trying to extract insights for other reasons beyond just immediate performance, then that data is valuable. Even though it's not converting, it's actually not wasted spend. You're learning from that spend and translating it across your ecosystem.
- [09:17]
Taylor
Yeah, to that point too, Meta operates why I think a lot of brands will see all the spend goes to like one or two ads because meta predicts that the action rate is going to be higher from that. So like CTR for example might be a metric that it's it's weighing. And this gets back to why I think like AI and media buying and like the art of media buying is, is still really prevalent because there are, you know, as a functional team working with a brand, we're going to have moments where all of us sitting at that table, taking a look at the performance together are going to say this concept we're coming up with is probably going to have a really high ctr, but we don't know if that's going to actually convert out the same way as some of our other ones that maybe don't have as high of a ctr. So now we need to think about that and where we place it in the ad account strategically, like that's where a lot of that comes into play in my eyes based on what you're saying as well. Jacob.
- [10:11]
Jacob
So Cody's comment here is when you're testing a lot of different concepts in an abo, you're forcing spanies. Things scale too slowly. He says the wasted spend is as big as issue. Jacob disagrees. But then he says it also violates everything Meta is telling us about how Andromeda Lattice and GEM work. I'm actually not familiar with Andromeda Lattice or gem. What is, what does he mean by that statement? What is, what is Meta telling us about how those platforms work and what are they?
- [10:36]
Eric Dick
These are like Meta's like AI delivery systems that they're implementing. So like Andromeda is, I mean I don't know like all the specifics but basically the behind the doors of how Meta's like algorithm works and it's a self learning system, that's what's fueling like Advantage plus. So he's just alluding to that essentially like Meta's trying to work towards this, these systems that they're investing in and they want us to just put in inputs and like hey, here's your results and that like they're quite far off, that there's a lot of management, there's a lot of strategy and like to the point of this tweet, it's kind of like a minor outlook of the bigger picture, how much manual involvement is needed. And that's what he's just saying like the ABO is kind of do go against what, what they're pushing towards but at the same time it's so far off that there's, there's a balance needed for sure.
- [11:34]
Jacob
What is Meta's, dare I say propaganda? What are Meta's talking points about how they would want marketers to solve this problem? I Assume it's just throw everything in ASC and let Andromeda sort it out. Is that accurate? Taylor, like what, what is, what is Meta's company line about how people should be testing this amount of creative?
- [11:51]
Taylor
Meta's kind of stance, to my understanding, based on conversations we've had, is really lean on the machine learning and like just keep fueling it with content. So worry less about the structure, keep going. And like it was last year they were doing a lot of interesting like tests looking at how many ads you should put in an ASC campaign. And I know at some point it was getting up to like 30 to 50, like it started getting kind of crazy and it's, it's the idea that Meta is continuing to try to flush those signals out on their end and those AI systems as, as Jacob mentioned. So you just keep feeding it more content based on what, what we learn about the system itself.
- [12:26]
Jacob
And then Jacob, back to Taylor's original point. Are you the it depends guy as well where you really are testing some your Hot Swap and Creative in, in bau, in business as usual campaigns, in some cases your, your asc, you're aboing all over. Like what's your general sort of strategy? Do you have a general strategy when it comes to testing creative? Are you. It depends.
- [12:46]
Eric Dick
Yeah, like I mean initially, if you're still kind of trying to unlock what's working like angle wise, you know, unboxings, reviews, like all these different kind of broad angles. I would say like we do a lot of ABO testing still just again to like get those true, true insights across the board. And the way we structure our test though is very pointed and actually doesn't require a lot of the budget. You know, once you're at a certain scale and you're spending, you know, X amount a day, the test budget really is like 5 to maybe 10% of your total budget. So it, it really doesn't hurt that much to be running these tests, you know, getting the data back and learning from what isn't working. But I would say more matured campaigns where you just kind of, you just know what's going to work, what's not. You know, you're, you're more in like iteration phase versus discovery phase. Those ones I would say you start rolling more with. Just fire it into, you know, Advantage plus, fire it into the catalog, whatever it might be and, and let Meta take it because there is data that led you to that point. That's my outlook on it, I guess.
- [13:53]
Jacob
And we haven't talked about it. As much lately like or we haven't named it as such. Are we still talking about the pilot test? Are we still talking about going with, you know, is that what you were just referring to? Is that. Do we set up the pilot test in these ABO structures where we take all of these different themes and they get a certain amount of traffic and we go by click rate or by these other lower funnel metrics?
- [14:12]
Eric Dick
Yeah, yeah, pretty much looking at the full funnel click rates, but also like view content cart rates and pretty clearly there's losers and top ones. It might not be proven out on the purchase side, but taking those signals and then moving those winners into your foundational campaigns, your evergreens, your Advantage plus. And that's so yeah, pilot test in a way is almost like a battleground and definitely something we're always still running. And that is what is at that 5 to 10% usually of the total budget. Not every campaign runs them now, but definitely those newer campaigns trying to elevate to that next threshold and in that discovery phase definitely run a lot of those.
- [14:56]
Jacob
And then what was your comment, Taylor, about what Meta considers net new creative? Where is that relevant and what did you learn about that?
- [15:03]
Taylor
Yeah, so it's really interesting. So I've been diving in and having, having various conversations and one of those conversations has been about what what signal Meta is using to try to understand how to deliver and cohort users to build those delivery patterns. So when it comes to some of that, I know they're kind of working on, on some different types of bid multiplier type signals. So weighing psychographic and demographic components against what it knows about the brand, one example might be it might weigh like targeting women higher based on the product offering is a pretty simple delivery pattern. And then as far as the creative itself, it's pretty interesting on the static side of things, Meta considers the relationship between the message and the visual itself. So there are cases where if the text is the prominent message in the creative, that may end up getting priority over the actual visual component in and of itself. So it does score that and look at that. And then on the video side, it's the first three seconds that Meadow really considers strongly in the current weighing from, from the conversations that we've had. So you know, testing different like end components of longer videos doesn't matter as much in theory as trying different hooks, intros and those first three seconds to basically say, hey, this is a different video concept than the other ones. So there are ways that you can kind of come back to aligning your Test strategy based on all of that information and continue to customize it accordingly.
- [16:39]
Jacob
A lot of these insights you said came from a call with, with our North Beam Rep about what he's sort of seeing across. Maybe, maybe set that up and share any other sort of insights that he shared.
- [16:48]
Taylor
Yeah, so big, big shout out to the, the team over there. They've been, been doing awesome, awesome work as always. So that was one of the big ones. The other one was the, the bid components. I think the other thing too, like speaking to this, this testing structure component, if you're choosing between like moving quickly with momentum and you have a sense of what's working versus you want to have a more methodical approach, you're maybe trying more contrast, taking some bigger swings with your creative. There's kind of two main schools of thought that, that are still relevant based on the conversation that I have, which is you launch a test campaign with your group of new concepts, you see momentum start to pick up. This might be in an ASC campaign or a CBO campaign where it's set up to just go, if it works, and then you basically just flip that into a scale campaign. You don't really change anything there. You just let that go, scale it up and then keep going and build your pipeline out accordingly. That's one school of thought for faster momentum. And then the other one is more so the ABO structure where you have graduated process and you're, you're controlling it. Again, it kind of depends on what's most relevant for your account and what you're looking to do based on stability, how much growth you want, et cetera, et cetera.
- [17:59]
Jacob
What was the other point about cohorting users by behavioral and psychographic details?
- [18:05]
Taylor
Yeah, so that's kind of the idea that a lot of people may have noticed in their ad accounts. They've got value rules and whatnot that are coming into advertising settings and the ability to set those basically bid multiplier type things. From what I've heard, Meta is looking at trying to give advertisers more input at that level. So prioritizing how we might want delivery and almost adding a manual component to that where you can combine your strategy with customer Personas or products, et cetera, et cetera, and then scoring that accordingly so you can tell it what it values and it can go and try to find those users, which then enables you to spend less time at the campaign level and focus more on just feeding it creative in alignment with that. So again, nothing, nothing, nothing else kind of beyond that at this point, I think it's still a ways out, but it is a pretty cool, cool potential insight that might be something to watch for.
- [19:05]
Jacob
And then maybe one of the last points we want to hit on is sort of an update check in this health and wellness category in with Meta advertisers. We did a podcast on it a little while ago about how they were deprecating any sort of personal identifiable data points including conversions and all sorts clicks and all sorts of stuff that makes it tough. And I think Jacob, you and I had a chat yesterday about a Jonathan Snow post about the health and wellness death spiral that campaigns can find themselves in when they aren't able to gain purchase or gain purchase data and they end up kind of flailing and performing poorly. Is that something we're seeing?
- [19:44]
Eric Dick
Yeah, like if you're in that health and wellness space and you're marked under this like core setup it's called now there's certain data that's limited being passed back which means Meta isn't able to define your audiences correctly, which means you're not necessarily self fulfilling new customer goals which means your budget is like yeah, you don't have that control. So if you are in health and wellness and marked undercore setup like other reporting tools are basically required. I'd say like getting North Beam on on or whatever it might be triple A any you know of those reporting tools and as well as just mapping your you know, before data and after. Because yeah, it's like different. There's like eight different buckets people are being put under in health and wellness and it can be like different data that's limited based on those buckets. Not every brand is getting hit with it. It kind of depends how you're messaging the brand and product. But yeah, we're definitely seeing that as an issue for a lot of brands.
- [20:47]
Jacob
And then I guess maybe your insight from the your North Beam talk Taylor, about how a lot of supplement and health brands are turning to Applovin. Maybe in this lack of data that Meta is giving there's a lot more increase in app Loven spend for these brands.
- [21:01]
Taylor
Yeah, that was an interesting discussion point hearing that there may be a pattern in kind of more budget share going to Applovin for some of those brands. So potentially something to look into for anybody who's navigating that issue on Meta and working to try to work around that.
- [21:18]
Jacob
We haven't talked about Applovin in a while but they're still crushing for E Comm. Is that true?
- [21:23]
Taylor
Yeah, it's interesting from What I've seen the whole app side of things has been it's definitely had some extremely powerful moments and then kind of also had its own challenges like you'd encounter on any channel. So creative is really important, but it's still a pretty prominent growing side of the space, so definitely something to watch. And I think especially as we, as we get closer to Q4 as well, though, we'll continue to see, see where it goes, but a little bit more from. From having more brands exploring it. I think brand fit with that placement seems to be a big, big thing. And then the potential nature of it being like an assisting platform in a lot of cases as well, based on what you're doing on other channels and then kind of capturing that has been an interesting pattern that we've been, we've been diving into a little bit to try to learn more about and see. See how we can best use it.
- [22:20]
Jacob
Nice. Well, we. Our Mother's Day is down. Father's Day is down. We've got July 4th, big, big holiday weekend in the States and events. Oh, Prime Days. That's right, yeah.
- [22:32]
Eric Dick
11Th, which is always big across all channels. And then, yeah, big Q4, hopefully here. Seems like, seems like it's going to be a good one.
- [22:41]
Jacob
When you say that, I'm just curious when. So it's, you know, Prime Days in July. What changes in Meta on Prime Days?
- [22:47]
Eric Dick
A.
- [22:47]
Jacob
Do you just. You get more conversions? Are you ever driving to Amazon or mentioning Prime Days in your ads? How does prime play into Meta?
- [22:55]
Eric Dick
Yeah, so basically it's just buying intent is high those days because Amazon is doing a lot of the legwork making people aware it's Prime Days. So a lot of people online buying intent is high and a lot of brands that are on Amazon on their website will still run like a, a Prime Day sale. They'll either match, you know, match the price of Prime Day, or basically use it to capture that audience that isn't going to buy you on Amazon anyways. Right. Use that messaging, Prime Day sale, and you bring it to your campaign front. You, you know, you, you run campaigns. Um, so yeah, it's more or less just taking advantage of the intent. And some brands definitely will be sending people to their Amazon like from meta ads and whatnot. But just the tracking side of that is obviously very hard and whatnot. And we found that they are kind of different ecosystems. Like, people that are gonna buy on Amazon and see a meta ad, they're still gonna search for you on Amazon anyways. So on Meta, usually you want to stay focused on just purchase intent. Get people you know buying through Meta or buying through the website. Those Amazon buyers will still buy through.
- [24:03]
Jacob
Amazon, so you don't have to give up that 20% margin and no one can afford to. So yeah, keep it on Meta. Cool. Well, thanks for coming on today guys. This is interesting. Look forward to checking back in with you as we approach Q4. Thanks for listening to today's episode. If you're not getting the D2C newsletter, you can subscribe for free at directtoconsumer. Co. And if you want to learn more about Pilothouse's all killer no filler services, take off to Pilothouse Co. I'm Eric Dick and this has been the DTC podcast. We'll see you next time.