SPECIAL SERIES ==> Your Attribution Data Is Lying to You 🤥 <== | BATHROOM Break #95 COLLAB: The Marketing Millennials + Do This, Not That

Summary

Podcast Summary

Podcast: Do This, NOT That: Marketing Tips with Jay Schwedelson
Episode: Your Attribution Data Is Lying to You 🤥 | Bathroom Break #95 (Collab: The Marketing Millennials + Do This, Not That)
Date: February 16, 2026

---

Episode Overview

In this special “Bathroom Break” collab episode, Jay Schwedelson and Daniel Murray deliver rapid-fire insights on a pressing issue in marketing: attribution data, and why it’s often misleading. With humor and a candid tone, they expose common attribution mistakes, share practical alternatives for measuring campaign performance, and reflect on how marketers can better understand what’s actually moving the needle. The conversation moves from data fallacies and tactical fixes to winter sports banter, offering both actionable tips and entertainment.

---

Key Discussion Points & Insights

The Problem with Marketing Attribution

• Attribution Obsession:
  Marketers are fixated on attribution, but most attribution systems give a false sense of accuracy.
  Jay (A) [02:39]:
  “Attribution measurement is completely, complete garbage.”

• Misuse as a "GPS":
  Attribution should be thought of as a compass (directional), not a GPS (turn-by-turn instructions).
  Daniel (B) [02:59]:
  “Don’t use it as like a GPS...Directionally, it could tell you these channels might be doing better than this channel. But...as the roadmap, it’s the wrong thing to do.”

• The Fault of Last-Touch Attribution:

  • Over-reliance on last-touch attribution heavily favors channels like Google Search because they're at the end of the journey, while discounting earlier influences.
  • This results in misallocation of budget and credit.
    Daniel (B) [03:51]:
    “Most companies...automatically put more money in Google search because that’s usually like the last touch attribution...there had to be an action before they came to your website.”

  Jay (A) [04:31]:
  “There’s going to be more last touch attribution assigned to these AI tools, even though it’s the same stupidity as assigning it to Google Search.”

---

Practical Fixes & Alternative Ways to Measure Impact

• Holdout Groups:

  • Randomly select a portion of your audience (5-10%) and exclude them from receiving campaign communications or ads.
  • After the campaign, compare results between the marketed group and the holdout.
  • The “delta” reveals the true lift driven by marketing activity. Jay (A) [04:31]:
    “You take 10% of your database and you hold it out...Then what you do is after the campaign concludes...the delta between your holdout group and the people you marketed to is your real performance.”
  • Caveat: Not perfect, but a major improvement over standard attribution.

• Geo-Based Attribution Testing:

  • Focus spend (OOH, radio, CTV) in a test region, then compare results to other markets.
  • Measure for incremental lift. Daniel (B) [06:23]:
    “Pick a geo...just concentrate–say out of home there or radio there...see if there was a lift in that area with doing that one tactic...You could tell there should be an incremental...lift.”
  • Downside: Pricier than holdouts, but highly insightful.

• Red Flags with Channel Credit:
  If one channel looks like an outlier (driving all credit), that’s a sign your measurement model is flawed. Jay (A) [07:19]:
  “If internally you’re assigning a lot of value to one channel and you’re doing multi-channel marketing...that should be your red flag.”

• Multi-Touch Attribution & Self-Reported Attribution:

  • Use multiple models (multi-touch, media mix, incrementality) to get a triangulated understanding.
  • Easy win: ask on lead/contact forms, “Where did you hear about us?”
    Daniel (B) [07:50]:
    “The easiest thing to do is just add ‘where did you hear about us’ on your form...And then we could now invest more into those channels because they’re actually telling us.”

---

Notable Quotes & Memorable Moments

• On Misplaced Attribution Confidence:
  Jay (A) [02:39]:
  “Attribution measurement is completely, complete garbage.”

• How to Think About Attribution:
  Daniel (B) [02:59]:
  “Attribution should be seen as like a compass...don’t use it as a GPS.”

• Budget Allocation Warning:
  Daniel (B) [03:51]:
  “They just say, oh, here, executive team, look, we can put more money in Google because Google is doing so good for us.”

• On Holdout Best Practices:
  Jay (A) [04:31]:
  “The delta between your holdout group and the people you marketed to is your real performance.”

• Multi-Touch Modeling Real Talk:
  Daniel (B) [07:50]:
  “Facebook’s a walled garden, Google’s a walled garden. They’re going to make it look better because that’s how you spend more on those platforms.”

---

Lighthearted Moments

• Altitude Sickness Banter ([00:39]–[02:34]):
  Jay and Daniel open with jokes about Jay’s Colorado trip, altitude sickness, and drinking water with Advil. Their comedic rapport sets a friendly, informal tone.

• Winter Olympics and Curling Digression ([08:59]–[10:38]):
  The conversation detours into the Winter Olympics, with musings about curling’s status and a playful exchange:
  Jay (A) [10:16]:
  “I don’t believe those are the best curlers in the world...they’re just the people that tried curling.”

---

Timestamps for Key Segments

| Timestamp | Segment Description | |-----------|-----------------------------------------------------------| | 02:39 | Why most attribution data is misleading | | 02:59 | Compass vs. GPS analogy for attribution | | 03:51 | The pitfalls of last-touch attribution | | 04:31 | How to set up and learn from holdout groups | | 06:23 | Geo-based attribution testing | | 07:19 | Red flags in assigning channel value | | 07:50 | Multi-touch models and self-reported attribution | | 08:59 | Olympics/curling banter and light close |

---

Actionable Takeaways

• Don’t trust attribution as gospel—use it for directional guidance, not literal action.
• Implement holdout groups or geo-based testing to understand campaign impact.
• Be wary of any one channel receiving all the credit, especially in multi-channel campaigns.
• Use multiple attribution models AND simple, self-reported data to triangulate the truth.

---

Tone:
Conversational, humorous, and candid, blending practical marketing wisdom with lighthearted personal anecdotes.

Best for:
Marketers, growth leads, and anyone struggling with measuring the true impact of their digital campaigns.

---

Transcript

[A]

Foreign.

[B]

Welcome to a new special series called the Bathroom break. That extra 10 minutes you either have to listen to marketing tips or use the bathroom or both. But I don't recommend both. But that's your choice.

[A]

This collab is going to be super fun. We have Daniel Murray from the Marketing Millennials and me, Jay Schwedelson from the do this, not that podcast and subjectline.com each episode in the series we are going to go over quick tips about different marketing top topics. And if you want to be in the bathroom, fine, just don't tell us about it. Thanks for checking it out.

[B]

We are back with another bathroom break. And for those who can't see the camera right now, Jay is literally in like this puffer jacket and he has like a hot toddy in his hand with a beanie on and he's in some cold place but he won't tell me where he is because he's on the road doing some weird stuff. But like, he's really looks like he's gonna get bundled up. I don't know what he's doing.

[A]

You might be the stupides alive. First of all, I'm on the road in Colorado and I'm not bundled up. I don't have a hot toddy. Although I would have one if I knew what the hell it was. But I will tell you this, it's two hour time change from where? You know, south Florida. It is crushing me at like 7pm I'm so tired and they scare the hell out of me. I, I drink so much water though. Like you're gonna, I've gotten altitude sickness before and Colorado's like high and I don't want to throw up. So I'm like drinking water and taking Advil. Like, do you get any of that when you go to high altitudes?

[B]

Yeah, especially I've been to, I went to Salt Lake City once and I went like up to Park City and just like quick ascent. I'm like, I needed to get that tea. There's like special tea that you're supposed to drink to help you with that. Yeah, I've got some bad headaches in altitude. I feel like we're weak being on sea level.

[A]

Oh, I, I'm very weak. It's funny, I was at a meeting in Utah and I didn't realize the altitude was like 5 or 6,000ft. In the middle of the meeting I had this crashing headache. I went to the restroom, I threw up in the middle of the meeting like I had. And then we had to go back to the meeting, I didn't know what was going on. It was terrible.

[B]

It was so funny. I was with my friend one time and we were walking in this place and it was like 10 years ago and he was, he's walking and he's getting so tired and he goes, what's the altitude here? And I'm like, turn around. And he's like, I'm like, it's sea level, dude. He was just like sweating because he was just so tired. There's an ocean behind him and he's.

[A]

Like, he thought he was idiot.

[B]

I was dying.

[A]

That's amazing. All right, so let's get into what we're talking about today. So attribution. There's not a marketer on the planet that doesn't focus on attribution, but there are some tricks to actually getting attribution sort of. Right. But in general, attribution measurement is completely complete garbage. Daniel, are you on the. The garbage train?

[B]

Yeah, I, I think I've always said this and I've been a marketing ops person before I decided to like go full content and social. But I always said attribution should be seen as like a compass where you directionally can find some answers, but don't use it as like a gps. And a lot of people use it as I need to turn right, turn left, turn this way, turn that way to get to my destination. Because that's what attribution is telling me. And I think that is the wrong way to use directionally. It could tell you these channels might be doing better than this channel, but I don't think people who use it just as the roadmap, it's the wrong thing to do.

[A]

Well, especially last touch attribution. Right. I mean, are you on the train that last touch attribution is the worst? Before we get into ideas on what you should be doing.

[B]

Yeah, I mean, I mean the modern day buying cycle is most people have done the research before they get to your site. So. And the problem is that I've seen it, most companies have been at is when they do that, they automatically put more money in Google search because that's usually like the last touch attribution because someone's branded searching you or they searching you online or they come in straight to your website. But there had to be an action before they came to your website. And it people don't want to dig deeper into that. They just say, oh, here, executive team, look, we can put more money in Google because Google is doing so good for us.

[A]

Yeah. And I do believe with all the, the LLMs and all the AI tools and they all will eventually start taking ads. I don't care what Claude says, that there's going to be more last touch attribution assigned to these AI tools, even though it's the same stupidity as assigning it to Google Search. And so there's one, there's one tactic that we do a lot with our clients I don't think enough marketers do, which is holdout groups. So what a holdout group is, let's just use a round number and say you have 100,000 people in your database, right? And you have a whole campaign you're pushing, whatever it is you're pushing, you know, to get new demos of your new SaaS product, or it could be a consumer product or you're trying to push some sort of new offer or whatever. And you take, you could do this with 5% if your big file is big enough, or 10% and you take 10% of your database and you hold it out. You don't allow it to get any of the emails, you don't allow it to be used for retargeting and remarketing. You do the best that you can to have those 10% to not be exposed to any of the media that you are aggressively pushing out there. And then what you do is after the campaign concludes and you take all of your net new customers that you've got, you could do it over a window of time. So say, okay, I'm going to take all the customers we got 30 days post the end of the campaign and all the customers we got during the campaign and then you match it back to your complete database, right? And you say, okay, the people that we marketed to versus the holdout group, what was the actual lift that we got by doing the marketing? Or was there no lift? Did our marketing have absolutely no impact? Because the holdout group performed exactly the same. And the delta between your holdout group and your and the people you marketed to is your real performance. Now, people, it's not an exact science. There's a lot of moving parts there, whatever. But it does give you a much better view on how effective your marketing is. I don't think enough people do that.

[B]

Yeah, I think two things that I used to do is one is geobase. So what I would do is pick a geo that and just concentrate to say out of home there or radio there or connect to TV there and just see like get a baseline of what it is now and for the last year and see if there was A lift in that area with like doing that one tactic and there isn't a left you will know but usually you could tell there should be an incremental like branded search lift or some left. That's one, one thing I, I try. I, I've done a lot when I've done as a geo based attribution testing because that will really tell you if something is lifting up in that area. It's more expensive though. So that's like the negative part of geo testing. But it's helpful to see like does my tactics work in that area or not?

[A]

Yeah. And I think just in general, if internally you're assigning a lot of value to one channel and you're doing multi channel marketing or multi platform marketing, that should be your red flag. Like wait a minute, we think that, you know, we're driving all this on Instagram, but really simultaneously we're running campaigns on Facebook, we're doing email, we're doing this all to the same people. It's silly to just assign complete value to wherever it is that ultimately the person's converting in that last moment.

[B]

And a lot of people now are going to multi, multi like attribution models, like they have like multi touch attribution, they have like media mix models, they have incrementality testing, they have like a bunch of things as like zero in on what. And that's actually a better way to do it to have multiple, multiple things going at one time because Facebook's a walled garden, Google's a walled garden. They're going to make it look better because that's how you spend more on those platforms. So actually having multiple things going at a time and compare and tries it. But the easiest thing that I liked it like the easiest thing to do is just add like where did you hear about us on your form? That's like a simple thing to just get someone to like at least start getting directionally. And that's how a lot of people we used to do that and we used to get, oh, we heard in your YouTube channel, we heard it in this. And then we, we could now invest more into those channels because they're actually telling us where they're hearing this from. So it's not exact science but it works sometimes I'm curious about something since.

[A]

I'm in cold weather right now. Are you an Olympic sky? Are you like all in on the Winter Olympics?

[B]

I am because it's like something I didn't grow up in cold. So it's so interesting to me how good people are on these things because I know how hard it is to ice skate, and I know how hard it is to ski. When I see these people going 90 miles an hour, I just think how I would die if I went one second off that. I just thought about. These drops are insane. I read the funniest tweet the other day. It said the winter. Lynn. So, so crazy, because there's people going around with knives on their feet and jumping off of hills, and then there's curling.

[A]

I feel bad for the curling people because they always get so much hate or whatever. Here's my problem with curling and all of it. I watch the curling people and I say, I don't believe those are the best curlers in the world, because it's not like, you know, Michael Jordan ever tried curling. These are just the people that tried curling. So I know they're very good. I'm sure they worked very hard, but they're not probably the best in the world at it. It's just they're the only ones who did it smart.

[B]

I mean, that's. That's the way to do it. Capitalize on a market that is not being able to do. But there you go. I feel like you would be a good curler.

[A]

Thank you. Thank you very much. I really. I appreciate it. It's the nicest thing you ever said to me. Well, with that, I hope everybody goes out there in curls, if that's what you say. I don't even know. What is that? What is that?

[B]

What is the.

[A]

Actually, I don't know the word curl. I don't know. I think it's curls. Let us know. Leave it in the. Leave us a comment. We'll see you at the next one. Daniel, come on, man. I gotta get back to work. Get out of there. All right, while he's still in there, this is Jay. Check out my podcast, do this, not that, for marketers. Each week we share really quick tips on stuff that can improve your marketing and hope you give it a try. Oh, here's Daniel. He's finally out.

[B]

Back from my bathroom break. This is Daniel. Go follow the Market Millennials podcast, but also tune into this series. It's once a week, the bathroom Break. We talk about marketing tips that we just spew out, and it could be anything from email, subject line to any marketing tips in the world. We'll talk about it. Just give us a shout on LinkedIn and tell us what you want to hear. Peace out.

[A]

Later.