← DTC Podcast
DTC Podcast
Ep 601: 25% of Your List Drives 75% of Revenue. The Dangers of AI Email Segmentation
00:21:21
Loading summary
Transcript44 lines
- [00:00]
A
You might never notice that this thing is actually bleeding your program. If anybody out there is thinking about using a service that does AI customer segmentation, I'd like to discuss some of the stuff that you'll see. So many people are showing up and it's like their site's just got problems. A lot of people come in and they're just limping. Okay, let's just get you not limping.
- [00:32]
B
It's all killer. No filler. I'm Eric and I am here with the return of Jordan Gordon. Jordan Gordon Levitt. I called you today. I don't know if you've ever got that nickname before, but thought that was pretty good. And usually I have a clear idea of what we're talking about. And it's been a while actually since you've been on the podcast and today I'm just going in blind because you say you have a good topic. Let's grip it and rip it, first of all.
- [00:53]
A
So whenever I've Googled my name, Joseph Gordon Levitt is what always comes up. And I've got to scroll through that to find any of my cell phone social medias. So what I wanted to discuss today was I'm not going to name any names, but I want to just discuss the next kind of service that I'm just totally going to trash on and say, nice.
- [01:13]
B
Right, here we go. Hot takes trashing on a service.
- [01:16]
A
Nameless. Yeah, yeah. So we, we use on the world's best email retention podcast, we, you know, often bring on the people who run different services, our contacts. We. I just recorded with someone from Rebuy, don't know if it's live yet, you know, and so because we're trying tons of tools, we just. People come and they've got all these different tools. We wrap our heads, we figure them out, you know, and then either one, I have them on the pod, or two, I trash them. That's kind of what's going to happen eventually, right? And so if anybody out there is thinking about using a service that does AI customer segmentation, I'd like to discuss some of the stuff that you'll see, you know, with, with services in that space and also just talk about from a theoretical standpoint, what, what those kinds of services do. And from that theoretical standpoint, we can also, we can also discuss the, the, it's similar the, the RFM tool that Klaviyo has, although we're not trashing that today, but, but, but from the theoretical standpoint, what happens when you start doing this segmentation? What happens to your inbox placement. So those are the topics because you've
- [02:28]
B
articulated over the years a very clear sort of methodology as it comes to how you think about, you know, customer life cycle retention, all those things. So what you're saying, when you start turning it over to AI, you gotta be aware of the bargain you're making.
- [02:41]
A
Yeah, that's right, the Faustian bargain. So your results, broadly speaking, are an intersection between inbox placement and message quality. So, you know, it doesn't matter how good the message is if it's not anyone's inbox, and it doesn't matter how many inboxes you're in if it's, if it's a completely disgusting message. It's just you're not going to get very good results. I mean, technically, inbox placement is higher on the hierarchy because it must be in the inbox to get any kind of reaction. Even bad messages can get some kind of reaction. But everybody out there don't take it to say that I think, you know, quant is better than qualitative. That's not what I'm saying here at all. So ultimately speaking, I've said this before, the more inboxing placement you get, the better. And you broadly, you want to get as much inbox in placement as you can before it starts causing something you would define as a problem, which might be my, my audience is getting less engaged. You know, I want to have a really engaged audience. I'm going to stop it there. Or of course I'm, I'm in spam, right? Those kinds of, of things that would cause you to, you know, to not want to beef up your inbox placement. And so from this pure theoretical standpoint, before we go into, into any specific tools, when you start segmenting, what you do is ultimately decrease inbox placement, of course. Right. And so if we go back and look on, on some episodes that I had on the world's best email retention podcast, we did American Eagle versus Everlane, we did Urban Outfitters, and those guys are emailing me like 1.5 times a day. Right. If you're on any large fashion brands, you're getting a lot of emails. So I would say they are not doing a lot of segmentation. So if you get a tool that does AI for customer segmentation, you know, the, the, what they say they're going to do is go through and find the most engaged addresses and message the most engaged addresses. So what that will mean in practice is you just message way fewer people. And I ran this analysis today just before coming in here. And I, I looked at a client and I looked at people who'd been on the site in the last 60 days as a share of, of mailable list revenue for that client. And 25% of the addresses that that client was emailing were on the site in the last 60 days. And though they were driving 75% of the revenue, broadly speaking. Right. So there's like, obviously like an 80, 20, you've got this small group of people that is driving 75% of the revenue. Awesome. But also that small group, that 25% was driving 50% of the clicks. And so it's. Sometimes you people shop in journeys, you need to, you're going fishing, you need to tease them in. They got a click. Hey, the clicks triggers, the browse abandon and all this stuff happens. Those clicks are important. So you can't just walk away from 75% of your audience. And so these tools that purely, I mean, we do, our segmentation focuses on addresses that are likely to drive traffic. But if you go too hard down that path, what you do is you don't get enough inbox placement, which means you don't get enough opens. And if you don't get enough opens, people fall off your list and your list just starts to shrink. So these tools in general that really focus in on engaged addresses, they will shrink your list, they will kind of decrease a bunch of click traffic that will also decrease what's going into all of your other funnels. And it kind of just, it just really narrows the whole thing.
- [06:19]
B
Now, are these tools black boxes where you can't, you can't sort of say, hey, like, shift it back towards being more aggressive? Because I imagine brands can see that they're like, number of mail, emails sent goes way down.
- [06:31]
A
Yeah. And if you're watching what's going on, you'll, you'll notice that. But remember when we, we went through this whole thing for these, these services that email people who've never subscribed to your service, we had this big talk for those kinds of services, not naming any names. And so, yeah, of course, if you're looking and you know what to look for. So after we'd dealt with enough brands blowing up and we knew what to look for, we could tell right away if those things are causing problems. Spoiler alert. Almost always. And so, sure, we can look at that. But if you're, if you are Joe Ecom and you're just trying to get by, right. You're just trying to increase units for, for this great thing and email is not the number one thing on your mind. You might never notice that this thing is actually bleeding your program choking you a little bit.
- [07:21]
B
Like a black belt jiu jitsu match that I know you love so much.
- [07:25]
A
That's right. Yeah, yeah, yeah. Me and Hoist Gracie. So, okay, look, that was the theoretical framework. And I'll say I, I will loop in that theoretical framework. Also, there are tools out there that are not total garbage, but you also must think about that. Like with Klaviyo rfm, right? That does zero in on specific addresses, you know, and you would want to use that in conjunction with your campaigns. You know what I mean? And so you can use that to like, you can feed different flows and you can have much more complex rule sets. You can pass things around and then also use campaigns to make sure you get that blanket inbox placement to keep getting the opens and the clicks. You need to kind of just to just keep your retention system firing. Now here's the. In practice, if anybody out there is thinking about using one of these tools that does AI for customer segmentation.
- [08:17]
B
I saw a few at Shop Talk. I'll tell you.
- [08:19]
A
Yeah, yeah, yeah, yeah. We had this, like, situation. It's always this way. Oh my God, I hope you don't mind the hot takes. It's always this way. We're like, we go through and we're like, this is, this is garbage. Why, why did all these suppressed addresses get unsuppressed? So the system goes in and unsuppresses a bunch of suppressed addresses that the AI says are like high potential addresses. And we're like going through and eyeballing these things and it's like, you know, clearly spam email addresses and just like. And go through and like this address has never, ever, ever engaged. There's been no measured engagement. I mean, essentially, like, the machine is garbaging garbage out and it's just like spewing garbage back into the list. And we're having these like, total PTSD from the days when we were dealing with tools that message people who'd never subscribed to your brand. Right? Oh, my God. All this garbage is being put back into the list. And yeah, we get on a call with one of the account teams and my number one, Bri, who's brilliant, was talking to the guys and they're trying to pin us up on it being like, oh, no, no, we're, we're going for the high quality addresses and going through. No, no, no, look at this. This is garbage. This is garbage. This is Garbage. This is garbage. And saying, oh, we don't always get it right. Well, look, you're never getting it right. This whole thing looks really bad. And then of course, I've been in these situations with these really, really, really garbage tools where they're trying to pin it on you and they're trying to frame it so that it's your fault, you know, and it's like, oh no, you've got to use our system. What you're doing is wrong. And it's like she just started laughing because it was so obvious, you know, when someone's like, you know, kind of laughing at their account team in the
- [10:10]
B
following suppression tools or suppression lists, seems to be like a cardinal rule that these tools would have to filled in. Right.
- [10:17]
A
It's actually unsuppressing suppressed addresses.
- [10:20]
B
Yeah.
- [10:20]
A
Because the AI says it's a high potential address. Who knows why? Oh, oh, yeah.
- [10:24]
B
You made the mistake in suppressing them in the first place. AI knows better.
- [10:27]
A
That's right. We made the mistake. Somebody who thoughtfully suppressed this list was wrong. And we're going to let this AI do it.
- [10:36]
B
So we spent some time thrashing on this anonymous tool. What are some other tools that you actually find a lot of use in, in the Klaviyo environment?
- [10:45]
A
Yeah, oh, some, some totally unrelated tools or some.
- [10:48]
B
Yeah, even, or. Well, first of all, is there anything else you want to say about, about turning the Faustian bargain brands make when they turn over their segmentation to smart AI? Because it can really, you know, send you some bad signals.
- [11:01]
A
My assumption is at some point someone's going to crack this, you know, and, and it'll just. Segmentation won't be as big of a part of what we do, you know, and segmentation is like, it's not the funnest part, you know what I mean? The funnest part is making cool messages, for sure. I'll just say I don't think anyone's cracked this yet. Like, clearly this is a popular product that I've seen people use. Clearly people haven't cracked this yet or
- [11:28]
B
you haven't seen it, to be fair. Maybe there are brands, CDPs out there essentially that have cracked this. Maybe the ones at Shopify were telling the truth. But you gotta be buyer beware when you're letting AI take the reins.
- [11:40]
A
Yeah, for sure. And like, I mean, so look, tools in the space that we like. We're, we're super into sonar from Triple Whale. It's, it's, it's not the same. It's not AI segmentation What it's doing is it's increasing inbox placement by capturing more addresses. So if we, if we leave the fact that this tool is garbage, if we leave that hot take away and go back to the theory. Okay, well the only reason you'd want to decrease inbox placement is if you are having trouble with your reputation, otherwise you want to increase your inbox placement. So what is something like sonar or Black Crow does this as well. These tools that extend cookie length and grab, however they do it, they go in the session and they hey, this is the fingerprinting devices, whatever it is they're doing and they get more site abandoned, email sent, more browser, more intent.
- [12:37]
B
They're able to ring more intent from the reg.
- [12:40]
A
Those are awesome because they increase in box placement from addresses that have shown high intent. So your aggregate engagement goes up while your total email sent goes up. You improve your reputation and you improve your revenue. That's what you'd want to do. You would not want to go and find a tool that's just going to crater how many addresses you send to. Even if it worked. Even if it worked. I mean I've never seen this stuff work. The, the thing that I always see work is find your safe send size, your safe audience, right? And give them one clean funnel. Let's say you're like, oh, we're going to start micro segmenting all these groups. Well if the more you segment groups the more messages you need, right? Because like just for instance, splitting up by male and female is an obvious example. If you split, if you segment by segment by male and female, you need a message for males and a message for female or what's the purpose of the segmentation? So the best thing you can do is push everybody through one clean funnel and then one clean funnel with your automations and then yeah, then you can maybe get cute if someone does a category abandon or something like that. After that's all set up, do we
- [13:48]
B
use any tools for AB testing on the CRO side?
- [13:51]
A
Yeah, we use Shoplift on the CRO side for our testing. That's our official tool.
- [13:55]
B
I went to the, the sweet party of my friend drew at IntelliGems at Shop Talk and it was cool. I feel like it's a pretty interesting tool, has some pretty good use cases. You should check that one out.
- [14:07]
A
No, we do, we have a client that's on intelligems right now. They've just taken the trial. They've taken the trial and no complaints so far. I'll say that we might have Someone from intelligems on the world's best email retention podcast in two months, who knows?
- [14:20]
B
Oh, nice. You should. And then I'm just curious from a personal level, how's it going? I know you've been, you know, you've recently, I guess it's been, maybe it's been over a year now where you've been handling both CRO and retention in the pilothouse ecosystem. What. How do you feel about having such a broad purview?
- [14:36]
A
I'll just tell you the stuff that we're, that we're thinking about. So one like super key finding. Since I've been involved, so many people are showing up and it's like their sites just got problems, right? And so at first we were like, hey, we're all this super classy stuff and lots of split tests and we do do that, but a lot of people come in and they're just limping. And so a lot of the work we've been doing is like, okay, let's just get you not limping and then we'll, then we'll start getting cute. So that's just like a key finding. And so, you know, I came here from really big brands where the site was already working when I got there. One of the service levels that we've been, that we're, I'm just like right now working on this with, with our executives and it's, it's going to be, of course, I'm sure there'll be an episode for this on the world's best email retention podcast where I'll sum it up for the audience to try at home. But there's like 15, you know, really solid lifecycle touch points. I can't spout them off the top of my head. But of course like your top landing pages, your pop up home above the fold Home below the Fold site 1 Browse 1 Cart 1 Check out 1 welcome Post Purchase Category above the Fold PDP Cart Checkout Recommendation set or your discovery set.
- [15:53]
B
Turns out you can pretty much.
- [15:55]
A
Pretty much I can, yeah. So look, we got 15 of these, right? And it's like there's certain views that you take on these 15. And you know, views would be for instance, structural, like a funnel view. We talked about that before in the fall. But also like the Persona view. Are you, are you speaking to the right, are you speaking to the right people in the right way in there? Some, some technical discovery, et cetera. Half a dozen views. So what you should set up at home if you're in the audience trying to Figure this out is you need to catalog these kind of. These top touch points and actually grab every single touch point. This is, like, very arduous and. And it's just one. But it's one of the things you got to do is catalog each of those touch points, and then you've got to make a call on each view. On every single view. Are we good or not? Are we good or not? On the structural view, the path that people go through, Are we good on Personas? Are we good on brand? Are we good on et cetera? It's kind of like now that we are bringing together, now that that team is helping bring together all of these channels, it's just been clear as we've been working with more and more brands that this is one of the biggest things that's just laying on the floor waiting for someone to pick it up, right? Is. Is going through those. These key touch points and making sure that they match on all. On all of your views.
- [17:15]
B
Big thing to pick up.
- [17:16]
A
It's a big thing to pick up. I'll give you an example. I will give you an example. This is live. This is live from kind of what we're doing. Imagine a pet food brand, premium pet food, and the customer Persona is a mom. The mom is buying it, and the mom is someone who's looking to spend an extra 250 bucks a month on pet food. And you go to the site and home above the fold says, free package of kibbles with each thing. It's like, whoa, whoa, whoa. That's not what we're talking. Right? That's not what we're talking about. That's. That's not. That's not what's going to close the deal. When. When mom is thinking, I want to get three extra years of life out of my pet. That's what mom's thinking. When she's spending 250 bucks extra on pet food, mom is not thinking about a free bag of kibble. So there's this clear life cycle touch point, core touch point, disconnect. And you just have to go through and catalog them all on a scheduled basis because especially when you've got a big team. It's funny. It's like these touch points, they just end up changing. Someone's like, oh, hey, you know what? We don't do that pack anymore. And so then someone goes in and remakes the page for that pack, but what they remake doesn't match some. Some other kind of, you know, feature or benefit or whatever rule. So these. These touch points, even the automated touch points, they walk away from you. Right? This service level would be kind of like a little bit of a glue between landing page website and klaviyo touchpoint, landing page, website pop up and email and how all those touch points interact.
- [18:57]
B
I think we've been talking a lot on the podcast about Andromeda and essentially having your key customer avatars, ideas of who your customers are and knowing where in the lifecycle they are. So being able to manage all that seems herculean. But it's, it's what brands have to do right to, to meet customers, where they're at.
- [19:16]
A
And, and that's again that's the key finding is like everyone, they, they show up, either one, they're limping, it's like okay, let's hope you limp or two, this is just not what's getting done. So like I, I, I'll say to everyone out there in audience land, it's like if everybody owns it, nobody owns it. And that's the thing about this, that's the key thing is like we're talking about, it's all post click. Although you know, that's because we're not. That's because we do post click. But of course like you just said, the advertisements part of that too. The, the ad is obviously part of the lifecycle and so if you're saying oh, the ads person is going to make sure that this matches and then, then whoever's doing the landing page is going to make sure it matches and then whoever in the email is going to make sure it matches and there's not someone whose role it is to just go through and make sure that whole thing matches like spoiler alert, it's not all going to match.
- [20:01]
B
Yeah, well let's leave it at there. Remind everyone to go subscribe and download the world's best email and retention podcast to a burp. Thanks for the in person appearance and buyer beware on AI segmentation tools. You definitely put, put them on blast. Maybe, maybe we'll get some. Maybe we'll get some. Who will hear this? Who will be like that's not at all what our tool does. And they'll come explain why their tool,
- [20:29]
A
if anybody wants it to reach out to us and show us why their tool is awesome. I'd love to hear it. Please contact me. You know, come to the come to Pilothouse co, fill out the lead form and I'll speak to you directly.
- [20:40]
B
But risk getting anonymously flamed on the DTC podcast.
- [20:43]
A
Well, make sure you don't contact me and tell me your tool works and then have a tool that doesn't work.
- [20:47]
B
There you go. Simple as that. Thanks, Jordan. Thanks for listening to today's episode. If you're not getting the DTC newsletter, you can subscribe for free at Direct to Consumer. And if you want to learn more about Pilothouse's all killer no filler services, take off to Pilothouse co I'm Eric Dick and this has been the D to C podcast. We'll see you next time.