Bonus: The Hard Truth About Marketing Measurement with Constantine Yurevich of SegmentStream

Summary

---

DTC Podcast: The Hard Truth About Marketing Measurement

Guest: Konstantin Yurevich (SegmentStream)
Host: Eric Dick
Date: September 17, 2025

---

Episode Overview

In this bonus episode, Konstantin Yurevich, founder of SegmentStream, returns to the DTC Podcast to deliver a candid, unvarnished look at the current state of marketing measurement in the DTC (Direct to Consumer) world. The conversation challenges widespread industry myths, dissects the main measurement methodologies, uncovers why so many approaches are flawed or misused, and reveals what actually works for DTC brands seeking real growth.

Yurevich argues that, despite years of hype around fancy models and "holy grail" solutions, much of what the industry relies on is either fiction or deeply misapplied. Instead, he prescribes a pragmatic mix of evidence-based attribution, first-click, self-reported analytics, and a strong focus on finding new customers, not just maximizing revenue attribution.

---

Key Discussion Points & Insights

1. Marketing Measurement: Levels of Abstraction and Mistrust

• Yurevich’s Analogy: People using marketing measurement platforms are like kids using Wi-Fi—they trust it works but have no clue how it really does ([01:21]).
• Danger of Abstraction: Unlike Wi-Fi (which just needs to 'work'), blindly trusting marketing metrics can actively mislead and cause harm, not just confusion ([03:55]).
• Technical Complexity: Modern measurement is an uneasy fusion of data science, mathematics, and domain expertise—models without business understanding are "detached from reality" ([02:14]).

---

2. The Three Main Approaches to Marketing Measurement

A. Empirical Observation

• Definition: Attribution based on observed touchpoints—e.g., a click leads to a purchase; self-reported answers like "How did you hear about us?" ([05:02]).
• Advantage: Simple, hard to fake, directly tied to user actions.
• Drawback: "Correlation is not causation"—just because a user touched a channel doesn’t prove true incrementality ([05:41]).

B. Pure Mathematical Modeling

• Examples: Marketing Mix Modeling (MMM), "Bayesian MMM," regression-based approaches.
• Danger: "You can build and prove any story by putting correct assumptions." The analogy: You can mathematically model either a round or a flat Earth; math alone doesn't guarantee reality ([05:02], [13:55]).
• Modern Pitfall: Priors (user-inputted beliefs) can bias models to whatever outcome stakeholders desire. "Confirmation bias in its play" ([13:55]).
• Limits: Extremely costly, usually irrelevant for 99% of advertisers, and often overtaken by rapid platform changes ([11:08], [13:55]).

C. Causal Effect & Statistical Interpretation

• Methods: GEO Holdout Tests, Lift/Incrementality Studies (espoused by platforms).
• Limitations:
  • True AB testing is mostly impossible on full-funnel marketing ([16:55]).
  • GEO holdout suffers from small sample sizes (e.g., splitting just 200 regions), leading to huge, unhelpful confidence intervals ([17:47]).
  • Reports claiming Facebook or YouTube are "undervalued" by 2-6x are based on ungrounded statistical frameworks and marketing biases ([20:58]).
• Reality Check: These methods can at best give yes/no incrementality answers—“Is my channel incremental or not?”—but never a precise, deterministic ROAS ([18:27]).

---

3. Industry Myths and Narrative Control

• Skepticism About Platforms: Facebook and Google promote methodologies (like rapid MMM and lift studies) that ultimately encourage more spend, not true insight ([13:55], [41:11]).
• Conflict of Interest: Many attribution vendors and agencies receive incentives and subsidies to "prove" the platform's effectiveness ([45:00]).
• "Attribution Is Dead" Hype: Yurevich: “For the last 20 years, we haven't done any progress. This is a hard truth.” The industry churns new narratives to justify more ad spend, masking the lack of actual measurement advances ([44:40]).

"The narrative essentially is: spend more, spend more, spend more. You don't need tracking, you don't need measurements. Spend more. Brand awareness, brand awareness, spend more. No need to measure."
— Konstantin Yurevich [45:00]

---

4. What Actually Works: Practical Recommendations

First Click vs. Last Click vs. Multi-Touch

• First-Click Advocacy: Despite prevailing advice, Yurevich argues first click attribution can be highly effective, especially for smaller and mid-size brands. It's empirical, transparent, and interpretable—key for finding new customers ([32:14]).
• Why Not Last Click?: The industry unfairly demonizes last click. But there’s silence on the quirks of first click—likely due to business incentives for selling complex modeling instead ([32:14]-[34:24]).
• Identity Graphs: Crucial to accurately stitch user journeys, especially across devices and channels ([35:24]).
• Ignore Impression-Based Attribution: Labeled “complete scum and...heavily abused by ad platforms just to claim contribution” ([33:36]).

Self-Reported Attribution (SRA)

• Best Use: SRA (e.g., “How did you hear about us?” fields) should supplement empirical data, not stand alone ([39:18]).
• Optimize for Big Levers: Only ask about major channels where clicks aren’t easily tracked—e.g., podcasts, YouTube, influencers, etc. ([41:50]).
• Combining Data: Match SRA answers with tracked user journeys to identify donor/contributor channels ([40:47]).

Marginal Analytics

• Focus: Once attribution foundation is credible and robust, analyze budget elasticity by observing performance at higher/lower spend levels—grounded in actual deterministic measurements ([42:34]).
• Diminishing Returns: Regardless of attribution model, returns diminish at scale; this pattern is observable and actionable ([54:40]).

---

5. SegmentStream’s Approach

• Not Wedded to One Method: Emphasizes a polyglot approach—using empirical observations, self-reporting, optional causality testing, and robust identity graphs ([30:41]).
• Consultative Methodology: Prioritizes deep client engagement to build reliable infrastructure, customized to real business needs ([52:13]).
• Processing Self-Reported Attribution: Uses AI/LLMs to categorize free-text responses into meaningful attribution logs, giving flexibility and surprising insights ([52:13]).
• Biggest Value: Their expertise—helping brands avoid fairy tales and "cut through bullshit" to uncover hard truths ([57:08]).

"Our idea right now is to work with people who want to hear hard truth...In the long run, they're going to win."
— Konstantin Yurevich [57:08]

---

Notable Quotes & Memorable Moments

• On Confirmation Bias in Modeling
  “You actually can build and prove any story by putting correct assumptions using priors. And this is what we see all over the place, that this approach is perceived as a holy grail and is promoted everywhere.”
  — Konstantin Yurevich [13:55]

• On the Illusion of Incrementality Measurement
  "[Geo holdout] gives you a confidence interval where it's going to say actually incrementality of your Facebook ads is from 1% to 11%. So Facebook actually contributes either to 1% of your revenue or to 11% of your revenue. So it's huge, it's a huge confidence interval."
  — Konstantin Yurevich [17:47]

• On Platforms Abandoning Transparent Attribution
  “Why Google and Facebook launched this open source Bayesian MMM frameworks...instead of launching them as a software, as a service within their platforms? ... If you have responsibility, you show that actually this channel is incremental and people start spending more money and two years later they find out that it was complete scum. You’re going to go to the court.”
  — Konstantin Yurevich [49:27]

• Pragmatic Advice for Founders
  "You should perceive marketing measurement as going to a doctor ... you need to have different opinions from smart people ... Otherwise you're gonna fall into confirmation bias trap ... the truth is very important even if it's not always very comfortable."
  — Konstantin Yurevich [58:47]

---

Timestamps for Key Segments

• State of Marketing Measurement & Analogy — [01:21]-[03:55]
• The Three Main Approaches to Measurement — [05:02]-[20:00]
• Industry Narratives and Confirmation Bias — [13:55]-[20:00]
• Geo Holdout & Incrementality Test Limitations — [16:55]-[23:00]
• Why First Click Is Underrated (and Practical Steps) — [32:14]-[38:18]
• How to Use Self-Reported Attribution Effectively — [39:18]-[41:50]
• SegmentStream’s Distinct Approach — [30:41], [52:13], [57:08]
• Founders’ Mindset and Avoiding Bias — [58:47]

---

Episode Tone and Final Thoughts

The episode is blunt, skeptical, and refreshingly candid. Both host and guest puncture fads and reject complexity for its own sake, instead advocating for practical, transparent, and evidence-based solutions. The hard truth: Most “advanced” measurement methods serve advertising platforms’ interests more than advertisers, and simplicity (well-designed attribution, SRA, and sound tracking) often wins.

---

If you’re seeking marketing measurement that serves your business—not just platform narratives—Konstantin’s hard truths are essential listening.

---

For more details on marginal analytics, check out the previous DTC Podcast episode with Konstantin Yurevich.

Transcript

[A]

There are many different approaches how you can measure marketing. I would separate them in three main categories. You actually can build and prove any story by putting correct assumptions. When someone builds a model, they have a sales call with a digital marketing director and they ask you what is the problem? Which channels do you believe that are undervalued? And they say, we believe that TikTok is undervalued. And they just put all this priors into the model. It's indeed a perfect mathematical models. But this marketing research is not grounded in reality, is not grounded in empirical observation that you've increased budget in TikTok and then your revenue grows. It's just based on the way how you tweak the data and how much value you attribute to particular channel.

[B]

It's the DTC podcast. Welcome back, Constantin. Today we are taking an overview of the marketing measurement landscape out there. How you doing? Welcome back.

[A]

I'm good. Thank you Eric, for having me.

[B]

Okay, let's talk about this. I've just been going over some of your LinkedIn posts.

[A]

I see.

[B]

You know, we could frame this either as the marketing measurement myths that are pervasive in the industry, but let's maybe just start, zoom out a little bit. Give me your overview of the state of market measurement in the, in the DTC space.

[A]

Yeah. So recently I've posted on LinkedIn that sometimes when I observe my kids, it's very interesting to observe them. They use their iPads, they use their iPhones, they connect to WI fi and essentially WI fi for them is something like really essential, like electricity or even like air for us. And they have no clue how it works. They have no idea about all these data centers, they have no idea about all these underwater fiber cables. They have no idea about tcp, IP protocols, they have no idea about HTTP protocol. It simply works. They just click the button. They know they need the password and now it works. So I would say I don't either.

[B]

Really, realistically, I don't either. Children and I are about the same level. Like the light switch, it just goes on when I press it, it just goes on.

[A]

And this level of abstraction, at some point it's nice to have it and not to worry. And some other people took care of it. And in marketing measurement right now I can compare. Like it's very, very good analogy. Like there are so many are just working. Like you launch your YouTube ads, Facebook ads, you see some numbers in marketing measurement that were sold to you using specific story and you have no clue how it works. You don't understand at all you just trust believe these numbers. The only difference is that WI fi is very utilitarian. You can just connect and you can just use it. While with marketing measurement and with advertising in general, it's little bit more complex. You are not able to see the effect immediately, but then you can evaluate it only within one or two years. And in my opinion, like right now, it's too technical. It's very technical. There are so many algorithms, there is so much statistics, there is so much empirical knowledge. But the same time, if you just take pure mathematician without domain knowledge, again the same problem, you can make beautiful mathematical models, but these mathematical models will be completely detached from reality, from business and from empirical knowledge. So I would say marketing measurement landscape right now is like a very tight combination of understanding data, science and mathematics, as well as empirical knowledge and understanding like business and marketing in general as a whole.

[B]

There's just with your kids and my kid, when she wants to play Roblox and there's no WI Fi, she just can't play it. Whereas in our case there are all these, there is all this data out there and it's a little more complex because if you follow the wrong data story or your data is too abstracted and you're not getting what you actually need, it can actively lead you astray. Not just, not just not play Roblox.

[A]

Right. It was like when, when I first purchased a computer. I've had like Windows 95 and I had a very, very small hard drive and I didn't know how operation system works, but it was too much space that it was consuming on my hard drives. And I started deleting some files one by one and seeing whether it's still loading or not. And at some point it stopped loading and I had no clue like how to restore it. So it's very, very similar.

[B]

Very similar. So what, what is a brand to do then? What, first of all, I guess what are the myths? Like what are the, what are the active calculations and measures that are out there that marketers might be looking at that are maybe not giving them the actual data they think they are?

[A]

Yeah. So there are many different approaches how you can measure marketing. And I would separate them in three main categories. So first one like is empirical observation. So empirical observation is like when you go outside, you see the sun, the sun is rising, the sun is falling, and you just observe this. You have no clue how it works. You have no clue about the shape of the earth, whether the earth is circling around the sun. You just see that it goes up it goes down and you see lots of causations and correlations related to this. So first wall is like this empirical direct observation. So we can put all attribution models there. So if attribution model is fair, you see a click on the ad, you see someone came to the website, after that you see a purchase. So you see a direct customer journey observation, there is no way to interpret it, there is no way to model it. You just have a direct empirical observation to the same category. We can attribute self reported attribution. When someone came to your podcast, you ask them how did you hear about us? And they tell you, oh, actually we follow you on LinkedIn or we've heard about you from our friend. So I directly tell you how I heard about you. And again, this is like direct empirical observation. Why it's great because it's very simple. There is no way to fool you unless you can clearly see how this attribution model is built. Like there is a clear evidence that something happened before the conversion, or someone told you how they've heard about you. The challenge of this method is that we all know correlation is not causation. And sometimes even if someone clicked on your ad or someone saw your ad, this is not direct proof that this ad was incremental. And actually that without this click there would have been no purchase. So this is downside. So the second category, this is the most interesting one is like pure mathematical modeling category. Like in this category we can put MMM, we can put all this new Bayesian MMM, all this new gen MMM, all this modeling based on impressions, costs, etc. As well as different econometrics models. And the idea of this regression modeling is that essentially you can build whatever you want to feed the data. So just to give you example, so we can build a mathematical model of our solar system and Earth, but the same way you can build an absolutely accurate mathematical model of flat Earth. So you know this like flat Earth conspiracy theory. Like the idea that you can build, you can build absolutely accurate mathematical model of a flat Earth with like a dome, the firmament. Yeah. And it will be 100% correct from mathematical perspective. So the idea of mathematics is that you give the system some assumptions and then you feed the function. So essentially you feed the answer to the question and to assumptions. So any mathematical model is based on assumption that you cannot prove or disprove. You just say, I believe this is correct. Now build the model that will show how the system works based on these assumptions. Like in physics we have gravity, we have no idea what it is, it's just G. It just equals to this. We just need to have this number. Otherwise our functions do not work and do not converge. So this is the area like this area was really popular in 1950 when MMM actually appeared. And back then it was something that was used by huge corporations like Coca Cola, like Procter and Gamble. Because essentially you have no other way to have empirical knowledge and to like see, to make all these observations how your products are consumed all over the globe. The best you could do is to have some market data and also understand like okay, in France we invest like 100 million per year. In US we invest like 500 million a year on marketing and you can build some correlations. And because of this huge budget, because of no other way to do any other measurement, because it is very, very time consuming, you were using this mathematical model to give you at least some clue about what was going on. And of course the hardest thing about MMM and all such models is to understand the baseline. So baseline means what gonna happen if you would be doing nothing like just how market gonna behave. And in like maybe 50 years ago markets were not that dynamic. Right. But nowadays sometimes what we can see, we see some, I don't know, we see statistics for the previous month on Google paid search and then we see the next month and we see that performance dropped 20%. No change in budget, nothing, it just dropped. What is the reason of this drop? Is it seasonality? Is it a competitor that for some reason need to hit the quota and they increased budget two times and now you pay much higher fees for your clicks? Is it the problem with your stock? Is it because a competitor now has a sale period, or maybe your sale so there's oh, it's microeconomic factor or maybe, I don't know, bank rates changed or like we have no clue why it happened. So there is such a, economics is such a complex system that there is such a level of complexity that it's just impossible to feed all the data points into the model to, to make it accurate. So that's why if accuracy is not something you care about, probably. But now I'm talking about traditional MMM that requires all this effort to collect the data about all the promotions, about all the competitors, about like auctions on each ad platform, like all the things that usually required at least two years of preparation. Because if you just decided today I would like to make an MMM and but you didn't think about this for the last two years. You don't have the data. Because if you want to Implement MMM in two years, you need to start tracking all your promotions, all the competitor auction information, like everything that happens, like the amount of stock units that you have, how many pages and products out of stock. So you need to start tracking all of this for two years before you can even think about making mmm. So it's difficult, expensive, extremely expensive, extremely time consuming. And when I say that something is useless, let's make also an assumption that when I'm talking that something is useless, it's useless for 99% of the advertisers as well as it just doesn't make any economical sense. So it's of course any technology has its own application, but when they say that something is completely useless, let's say it's not 100% useless, it's useless for 99, sometimes 0.9% of the advertisers. So, and after that, like what happened with all these mathematical models, we have all this GDPR, we have all these restrictions in tracking, etc. And the market started searching for like new holy grail of marketing measurement. And here were like new gen or like fast paced or Bayesian mmm. Like a new way of MMM appeared and everyone started promoting it everywhere. And first of all, it was promoted by Facebook and by Google and they say, okay, we can build MMM really quickly. There is no need to collect all this data. You can just import cost data from different ad platforms and essentially cost data and impression data is all you need. There is no need to collect anything else. Let's just build based on cost data and all the changes that we have and they've added such a thing which is called priors. So priors is like, we will build you a model but you tell us like, how do you feel YouTube is performing? How do you feel Facebook is performing? And you add this prior.

[B]

That's subjective.

[A]

Yeah, yeah. And this is where like lots of startups got their real traction. Because for the first time marketer or digital marketing director who buys a software can give the vendor all their assumptions. We think that our YouTube is undervalued. We think that TikTok, we invested $1 million on TikTok, but we don't see any traction. Our CFO is unhappy. We need to prove because we believe that TikTok actually is valuable. So you give all these priors actual numbers. Like we believe that TikTok ROI is between 2x to 3x and you put this as assumptions to all these models and eventually modeling is really good and model builds you a regression line that feeds all these assumptions. In the best possible way and shows you results that you wanted to see. So this is a confirmation bias in its play. So you actually can build and prove any story by putting correct assumptions using priors. And this is what we see all over the place, that this approach is perceived as a holy grail and is promoted everywhere. And you don't need to wait for two years, you don't need to collect the data, you don't need to invest hundreds of thousands of dollars. You can just pay some fee and it will import the data from all ad platforms and will build you the model. And of course, when someone built the model, they have an interview or sales call with a digital marketing director and they ask you what is the problem? What problem with marketing measurement do you have? Which channels do you believe that are undervalued? And they say, we believe that TikTok is undervalued. We believe that our YouTube generates a lot of brand, but actually people and they just put all these priors into the model. And now you have a perfect model. It's indeed a perfect mathematical models taking these priors into consideration. And this is how these models work. And yeah, and I see a lot of case studies where like there is published like oh, we understood that TikTok is actually undervalued five times or I see lots of marketing. But this marketing research is based not is not grounded in reality, is not grounded in empirical observation that you've increased like budget in TikTok and then your revenue grows. It's just based on the way how you tweak the data and how much value you attribute to particular channel. Yeah. So this category I call pure mathematical fiction. So you can model whatever you want. It will be 100% data driven. 100% data driven. You can show this to your CFO, but this has nothing to do with reality because you can model whatever you want.

[B]

So what's the third category of marketing measurement?

[A]

The third category is quite complex because again it can be twisted in so many different ways. So it's a combination of causal effect and statistical interpretation. So for example, when you run a GEO holdout test and you stop running ads in one region and continue running in another region is all types of AB tests which are unfortunately impossible in marketing analytics because you don't have two different realities to a B test. So you have one budget, you have some regions and there is no way for you to split audience deterministically and understand what is the effect. So right now there are lots of also fairy tales about incremental Attribution or so called lift studies which are promoted by ad platforms and again like non technical marketers. So the idea overall is nice, like you can split all the audience because Facebook can see all the audience, all their users on a, on a user level and they can randomly choose who going to see the ad, who not going to see the ad. So overall idea looks like genuine a B test. But the question is how you measure results. Because like when you show ads to some people and do not show ads to some people, at the end of the day you need to measure whether they've made a purchase or they did not buy. But how are you going to do this? The only way to do this is using attribution. Because there is no way to do this on a user level without attribution. And the results of attribution will 100% depend on how well you can stitch these customer journeys. So when you showed ads to someone and they clicked on these ads, it's very easy for you to stitch using click ID et cetera. If you didn't show ads to someone. So you didn't show ads, but you know this cookie, cookie, 1, 2, 3 didn't see the ads. Now they go to the website and they make a purchase, but Facebook doesn't see this purchase. Yeah they promote some PII stitching like matching. You can send first name, last name, email, etc. But still this non viewable stitching gonna be much much worse compared to viewable stitching. And this way incremental attribution is always biased towards stitching much better. Those who saw the ads, who clicked the ads and the group that didn't see the ads, even with good matching rate might have 40% stitching. So 60% of conversions that happened anyways without ads are not considered. So by default you have this, even if you trust that their algorithms are genuine, just by design you will see two times, three times higher incrementality that it really is. So there is no way to run a real increment like real lift study, real a B test and the only thing that we have is a geo holdout testing. So geoholdout essentially it's not a real AB test. It's not like you have millions of users and you split them randomly and you have this normal distribution and everything is following all the statistics rules. Instead you have like 50 states or at best you have 200 plus DMAs like specific like market areas defined by Nielsen. And what you can do you can split these areas like 20 areas. So it's like Imagine like new, like pharmaceutical companies testing new medicine and they need to make a study. And imagine they have only 200 patients. They don't have more, they have only 200. So there is such a huge chance of noise and choosing the wrong person because more people you have like more like balanced and soothed. The statistics is. But now you have only 200 DMAs. And now there is a method which of course we cannot just make an A B test between DMAs because those are different. And yesterday they behaved similarly, next week they behave a different way. The only way that actually was proposed by Facebook is to build a synthetic control group. So you choose some test regions, let's say California, Florida and Nevada. And then you choose a combination of different states. Then you apply specific coefficients. For example New York, there are 1,000 purchases, but you want to fit it together with some other state. So you apply 0.8 coefficient. So you just apply some function so that you build this not real but synthetic control that behaves very similarly to test regions. And then when you make a test, you do not do, you do not compare your conversions directly to New York or to other states that are in control. Instead you compare it to synthetic control. And the biggest issue here that because the distribution is not normal and because it doesn't follow like normal distribution rules, how you evaluate effect, etc. Or confidence interval. These confidence intervals are extremely high. So essentially it's quite normal when you're going to measure incrementality of your Facebook ads, for example, in a region. And you will have confidence interval where it's going to say actually incrementality of your Facebook ads is from 1% to 11%. So Facebook actually contributes either to 1% of your revenue or to 11% of your revenue. So it's huge, it's a huge confidence interval. And whenever you're gonna run, and also so this is the first problem, you can never ever understand real incrementality. So when I'm talking about geo holdout testing, I always say this is a methodology to give you yes or no answer whether my ads are incremental or not. Also it can give you like worst case scenario, for example, if I measure that the incrementality of my ads are from 1% to 11%, so at least 1% incremental. So you can test another channel and it will be at least 3% incremental. So it's not real incrementality. You cannot calculate real incremental roas of this channel, but at least you can evaluate lowest boundary of Incrementality so that you can compare it to other channels. While again around the social media and some companies even publish industry reports where they say that they've concluded hundreds of incrementality tests and they see that actually Facebook ads are undervalued like two times. YouTube ads are undervalued six times. There is no way to measure this deterministically. Even if you're going to measure like YouTube ads iros, it can be anywhere from 0.5 where you're losing money to 5.5. So it's because of the huge confidence interval. So we have assumption with this because it's not a real a B test. We have very small number of regions. This regions behave. First of all they are not selected randomly, they are selected specifically. They have different demographics, they sometimes have different seasonality. Some regions that are close to each other can behave. So in our platform we also have this methodology. So we have GeoHoldout testing methodology and many of our customers use it. But we explain that you cannot just measure incremental ROS using GeoHoldout. Actually there is no such thing as incrementality measurement in terms of deterministic incrementality measurement. There is no such technology. It's promoted everywhere that if you, you should go away from last clique, you should start measuring incrementality. But it's just blah blah blah blah blah. There is no methodology that can help you measure incrementality. I mean deterministic incrementality. We have exact number how many conversions this particular channel brought us. What is the real roth of this channel? There is no such methodology. There are only methodologies that can tell you whether your ads are incremental or not. With a huge confidence interval. Like it can be, yes, it's incremental. It gives you from 5 conversions to 500 conversions. Now you can do anything you want with this information. And most of the companies, they don't use these confidence intervals. They don't even show these confidence intervals to clients. Because if they gonna do this client's gonna be confused. Like what? Like we've shut down all our ads for one month. We didn't get revenue. Like we wanted to measure actual IROs of our YouTube ads. And now you say that actually our IROs is from 0.5 to 5.5. Like why have we like wasted so much time, so much money just to have this like number? We'd rather use attribution. We'd rather go inside YouTube ads and at least see post view like post click like first Click attribution and we will know approximately and what we also observed that whenever we run GEO holdout and with this huge confidence intervals, the attribution results always fall inside this confidence interval. So for example, attribution shows that Facebook brought 100 conversion. Okay, we want to calibrate our attribution and run incrementality test. Incrementality test says that attribution says it's 100 conversion conversions. GeoHoldout says it's somewhere between 50 to 250 or from 20 to 200. So it always falls inside this interval. So essentially it doesn't even add much value on top of the attribution. So where we found it useful only when we do not observe any conversions like tv, connected TV or some other channels that are extremely like podcast for example. But again with podcast it's very hard to split regions, right? It's very hard to. Probably when you're going to publish it on YouTube you can exclude. But you wouldn't be doing this because like when you, when you do organic stuff, you want it to be distributed as much as possible because you already invested in content production. So essentially what we found that you can use it maybe for connected tv, but also there are so many ways how connected TV can be tracked much better. And also you can test it for some channels where you don't even believe that these channels are incremental. For example, you invest a lot in retargeting, you invest a lot into brand search and you're like, oh, we already invest like 100k a month in brand search. Is it really incremental? Does it give us any additional revenue? Or we'd rather reallocate this budget to some other channel, upper funnel channel. And you can run GeoColdout and GeoColdout can give you yes or no answer, but everything else is just a fairy tale. All these industry reports that say, oh, actually we evaluated like 100 customers and YouTube or Facebook is seven times or six times undervalued. They're just taking this observational result like, okay, they've measured incrementality. So recently we've finished an experiment and the incrementality observation was 9% but the confidence interval is 1 to 10%. Usually we expect confidence interval to be symmetrical from both sides. So it's like 9% plus minus 5. But this works for normal distribution when you have millions of users, et cetera. For geoholdouts, it's asymmetrical. It's always asymmetrical because the way how this confidence interval is Calculated is different. It uses the same approach as in medicine. We use placebo effect. We just reshuffle all these regions again and we just run a GEO holdout test again and we see that there are some regions that were in test now in control and we see that we've been continuing showing ads but now we're again measuring criminality which is 7% and they say okay, even if in AA test we can measure 7% incrementality, can we really trust 9% number? Yeah, you cannot. That's why we have and so we cannot in geo holdout tests we cannot use this observation number. We should always look at the confidence interval and adjust this number and understand worst case scenario, best case scenario, neutral scenario, et cetera. And in many cases what we see that like everywhere, everywhere on social media I see that Ross is underreported. What I usually see that Ross in ad platforms is usually over reported. So it's absolutely different story. And again it depends who wants to push different narratives.

[B]

I'm sure the platforms want to obviously, you know, take as much credit as they can. Where, where does segment stream fit in these three categories of marketing measurement?

[A]

Oh, it's a good question. I would say. So we use everything but the idea is that we don't put all our eggs in one basket and this way we do not push like heavily any specific approach because our whole business doesn't depend on specific approach. Our whole business depends on expertise and very deep engagement with each client to find the best possible way. So that's why for example we have GEO holdouts and actually I was a big believer in this methodology like for me like oh it's so genius, you can just split regions and you can and this is first like causal part when you understand that you make an action and you have a result. But then when I started deep in research with real data and with statistics and I understood that oh we invested like one year developing this module to the platform and now it has such a limited application where we can just answer some yes or no for very specific channels some clients do not even care about. But there are some companies whose whole model depends on this methodology. So so far like I can tell you what I've observed so far in terms of like what really works and what I would be doing for my if tomorrow I going to be running D2C store importing some products from China with high tariffs.

[B]

Belts, I hear belts are really big. Belts are the new thing. So let's make it a belt business.

[A]

Yeah, if I'm Going to make a belt business. And I understand that margins are very important for me. Tariffs hit hard. I need to be profitable. I need to grow as much as possible. So first of all, I will always. So a lot of people say last click attribution doesn't work. Last click attribution is biased. You should never trust last click. And like lots of posts about last click attribution, my question is why do we even talk about this? Why do we even talk about last click attribution? Why don't we talk about first click attribution? So why there are no posts like first click attribution is biased. You should not trust first click attribution. You should use mmm. So everywhere I hear last click attribution is like old school, people are losing money. You should switch to MMM and your holdouts. Why no one is writing about the same about first click attribution?

[B]

Good question. It's funny, I feel like first first touch. I remember all the platforms were always wanting when I, when I was in my marketing days more than with the media here. They were always wanting me to trust impression tracking or you know, when I couldn't see good results, they were always trying to say, but look at the impression tracking. You can see that this is working. So I think the platforms want first touch attribution to be a thing. But I think it's always thought that the last click is.

[A]

Yeah, okay, not first touch, first click. Let's. Let's forget about impression. Hopefully everyone knows that impression based attribution is complete scum and it's heavily abused by ad platforms just to show impressions to whoever entered your website from different channels. And then to credit the contribution, like claim the contribution. Let's put aside impressions. But why no one is posting first click attribution. Everyone is writing about last click attribution. It's super primitive. It existed for like 20 years. You just need to track one click and then like stitch it the same way like you stitch last click to a conversion. Why then there is nothing about first click attribution on the Internet.

[B]

Is it too hard? Because it requires all that stitching to the final click.

[A]

It works exactly the same way when you. Even easier. Once you have a click, you can ignore everything else that happens and just have a conversion. With last click you need to reattribute to the last click. So it's much more complex technology in a sense than first click.

[B]

Well, you tell me.

[A]

I'm not sure because it works and it is very simple. Who gonna pay you like 50k or 100k for MMM or geo holdouts or all these data science, etc. If you can just use first click. If just an average D2C brand who is not very big right now, they invest 100k a month, they can just implement first click and they will see actually a picture which is quite grounded in reality. It's a direct empirical observation and it's very interpretable. You can explain what you are seeing. You can see that actually there was of course first click and true first click are different things because even now if you're going to go to your analytics, you can see a lot of first click attributed to email just because of cross device, et cetera. But still it's like in a sense it's one of the best attributions that you can find right now. And what I would be doing immediately I would do first click. What we do in segment stream, we understand that first click and true first click are very different things. So what we need to do is to build a very extensive identity graph under the hood so that whenever you send a D2C newsletter to one device and another device, we have all the proper parameters Everywhere like user ID, email # and stitch it so that we understand that this is the same user across different devices. So there are many methodologies how this could be done and it requires some work, not such a hard work as you need for traditional MMM to implement proper infrastructure so that we can start collecting this data. And at some point we see that without identity graph and with identity graph we see different numbers, sometimes they differ two times for some channels. But again it's a clear empirical evidence grounded in observation that now it was the same user with the same user ID with the same email, with the same phone, with the same IP address, with the same specific parameters that we also add that you can send to a friend and we know that actually you've sent this link to a friend and we can link you back to your initial click to the ad. But now it's like fully grounded in reality model that can be fully explained and the explanation is quite good. Like this was the first channel that ever brought this user to the website. I like this explanation. Yes, with first click maybe it's not so easy to explain like retargeting impact and lower funnel impact, but I don't care, like what I care about is what really brings new and this is what everyone is talking about which channels really bring new customers to the website.

[B]

New customers, exactly.

[A]

And you can also like, you can also stop tracking returning customers at all. This is what I would do. I would track purchases only for new customers. I don't care about returning customers. I don't want ad platforms to show ads. Again there is definitely an incrementality in showing ads to existing customers, but this will happen anyways. Platforms will abuse this audience anyways. So I would say this will be a nice bonus. But I don't want to reward any platform for bringing back my existing customers that I can re engage with in.

[B]

Different other ways because they're all incentivized to get you to overspend on that bottom part of the funnel which is I think of what we talked about in our last interview where they're already going to convert any.

[A]

So it's essentially like first click or also multi touch attribution. So we have both in our platform. For someone who is really focused on just driving new audience, they need to grow, they need to scale. First touch is perfect for someone who really wants to make everything like super balanced. Multitouch might be a good idea. But again some might ask okay, but first click is nice. But there are some channels that we cannot touch measure based on clicks. For example, they view our TikToks, they view our YouTube videos, they viewed our podcast. There is no way like right now after this podcast many people gonna view it but there is no way for them to click. Maybe there is some link to segment stream website they can click but very small portion gonna do this while they're gonna be a huge brand awareness. So so what we found the best way to measure it is a self reported attribution. It's another evidence based attribution. But the problem is that many brands use it in isolation from their main attribution. When I say isolation they just whenever imagine like we're going to publish this podcast and in our HubSpot whenever you submit a lead form we ask you how did you hear about us? And they will say DTC podcast, DTC podcast, DTC podcast. So I'm going to say like LinkedIn, Facebook, etc. If we're going to use this attribution just standalone, it is also very biased because first of all not very small. But not everyone gives an answer. But also answers can be very not equally distributed across different channels. Maybe D2C audience is very loyal and they recognize D2C podcast and your brand and they immediately can like remember oh we saw it on D2C podcast because it brought a lot of emotions and it's high quality content etc. But if they saw something on Instagram they might not connect the dots immediately or Google search or something like very obvious that they don't even pay attention to. That's why like there is no, no need just to use this attribution. Instead we combine this together. We use a methodology which is called reattribution. So whenever someone comes from direct or from organic or from brand search or from any other channel which is considered like direct, we connect their user journey with an answer they've given in a self report distribution. And imagine someone came from direct and they answered how did you Hear about us? D2C podcast or YouTube. So we have channels which are called donors and contributors. So we never assign contributors to donors. But donors like brand, organic, direct always are re attributed if the answer is different. And we see that for many brands like 90% response rate, how did you hear about us? But also we bother only about the channels that we cannot track. So there is even no need to add all these options like criteria, retargeting, X platform. You don't need all of this, you just need. Okay, we cannot just confuse them.

[B]

Yeah, they don't know what they even came from.

[A]

Yeah, you know like what you call like brand awareness channel. If for you it's YouTube, podcasts, influencers and TikTok. Just keep these options here, maybe add like online search and other. You don't care about all this like micro stuff. You care only about these major channels where you need to get as many answers as possible. If really someone remembers that they saw Your ad on YouTube and some people might say oh, people do not usually remember, sometimes they forget and they might give you a wrong answer. But again if you're doing brand awareness, it's already like keyword is brand awareness. If someone cannot remember how aware did they hear about your brand? Probably this was no brand awareness. So when you have really good. For example I remember do you know the Monday.com platform? And few years ago it was crazy. Like they've had all these YouTube ads all the time appearing, appearing. And even if now you're gonna tell me how did I hear about Monday.com I will tell you YouTube ads like three years later I can tell YouTube ads. Because yes, I was annoyed by these ads. We didn't have YouTube Premium by that time. But I still remember. So this is brand awareness. I still remember. So this combination of attribution, the best attribution that fits your business model and your goals identity graph to be able to stitch everything like emails, phones, IP addresses, click ID propagation, etc. Etc. And reattribution using self report attribution already going to give you amazing results, like from the start, great results. If implemented correctly, it's amazing. After that I would say don't bother about further enhancement of your attribution. Apply marginal analytics. And we talked about this on our previous podcast and maybe you can show the link to the previous podcast where they can learn in depth what is marginal. Because now once you have some model that you can trust, because you can explain it, you clearly understand how it works, it is grounded in reality, it's like pure observation of the reality. You can now start applying math and some statistics because marginal analytics is already a third method where you have some causal inference and you have math. So you start shifting your budget, increasing budget, understanding elasticity of each campaign. But this elasticity is not measured based on some math, it's based on the attribution that is quite deterministic and you've chosen. So that's why there are so many talks about attribution is dead, et cetera. But in reality, like what I've posted recently, for the last 220 years, we haven't done any progress. This is a hard truth. This is a hard truth, especially for me working and especially because usually I'm a very big believer in something. Like, I have so much enthusiasm and whenever I see some new technology, we immediately have a meeting with our product team, data science team. We invest time, research, implement, and then again I see, oh again it doesn't work. It's a fairy tale, like huge confidence intervals. We cannot trust this data. It's just a marketing strategy, like, and all the time. And for me it was very, very hard to accept that the best thing we have right now is attribution. It was very hard to accept because like five years ago I was like saying, attribution is dead. We need something new. For me, it's not easy to accept this after investing 10 years in finding something better. But I can say like for the last 20 years, with everything I know, do you know this Dunning Kruger effect? When you don't know much, you're very enthusiastic, you can sell, you're very confident, you show off, but more you know, more you understand that you know nothing. And this is something that is happening to me after I evaluated all these methodologies with real data, with real clients and understood that yes, there is 5% of improvement, the research that is being done, but 95% is just huge budgets that are bumped into social media and informational field to push specific narratives. And the narrative essentially is spend more Spend more, spend more, spend more. You don't need tracking, you don't need measurements. Spend more. Because not everything can be measured. You can use some other methodologies, but you should spend more. Brand awareness. Brand awareness, spend more. No need to measure. So this is an overall narrative, but then you just need to dress it up in very beautiful data driven solutions marketing research, different partners, partner vendors that are sponsored by ad platforms. And I was really surprised that there are so many vendors that take money from ad platforms that subsidize their pilots to prove the efficiency of specific channels. So for me this such a conflict of interests.

[B]

Speaking of conspiracy theories though, I saw your post about platforms are actively removing the first touch attribution. Why do you like speak like why if it is it's an effective way to see downstream results. Why do you think they're removing it?

[A]

First of all because like let's start with like Google. Google is still a like I would say even if you're going to implement first touch it won't give you such amazing results for YouTube display demand gen like for upper funnel channels it still won't give you such amazing results compared to MMM, compared to post view, etc. But for paid search which is a primary driver of the revenue for Google like it might show that actually like 70% of your paid search is just cannibalizing your organic it's just not incremental. People just click it because you have a Chrome browser because they just don't want to type full address of your website because so it's reality like when you apply first touch like rows of Google shrinks dramatically. So why would you want to have this attribution? Also it's not very good for optimization because click might have happened like three weeks ago, four weeks ago and now you have a conversion. It's a huge delay in optimization. So that's why first touch is good to have more strategic view when you can analyze longer period of time. But if you need to optimize right away, usually we tell our clients to separate different channels like upper funnel, mid funnel and lower funnel based on the rows they see based on first touch and reattribution and identity graph. And then inside these three portfolios already optimized based on multi touch attribution because it's much faster. Yes, we now understand the funnel level but still you cannot like wait three or four, four months to make a decision when your first click actually is activated. So that's why our goal is as experts, as consultants to find the best approach to work on tactical and strategical level. So I would say this is the main reason why first touch distribution was removed completely from most of the platform. Because it just shows you much worse your roles is not as amazing as the roas that you're going to see based on last click. And also in each platform there is not deduplication whenever someone clicks on both Google Ads and Facebook ads every platform going to claim the credit from this touch. But there are more like the question is why Google and Facebook launched this open source Bayesian MMM frameworks. If those are so easy, you just need to pull costs etc. Instead of launching them as a software, as a service within their platforms. Why? Why they just haven't integrated it inside ad platform just to be able to measure the effect because they don't fully trust it? Maybe because if something is not like open source, you can use it at your own risk, etc. Now you have responsibility. And if you have responsibility, you show that actually this channel is incremental and people start spending more money and two years later they find out that it was complete scum. You're going to go to the court. That's why it's much, much better strategy to launch this open source than to have this army of individuals brainwashed by different stories and sometimes kickbacks and these pilots that are subsidized by ad platforms. But now you have lots of individuals, lots of influencers, lots of small agencies, lots of vendors that even if someone going to be responsible for something, there is no responsibility for a big guy. The same thing. Why geo holdout testing geo lift library was abandoned by Facebook Why it was complete like no new commits on GitHub for the last three or four years because they understood it just doesn't make sense to run this geo. You can make so many mistakes, it's such time consuming methodology, you can lose money. You cannot run your ads. It's so risky. But at the end of the day, after the concluding the experiment, you have some incrementality with huge confidence interval. And almost every time your attribution results fall within this confidence interval and they just abandoned this library.

[B]

What's the most valuable thing that most advertisers are using segment stream to uncover right now?

[A]

I would say when we start working with clients, we do not just plug into their website and just show some numbers. We analyze their whole system. How they track their users, how they track conversions, how they track their email subscriptions. Do they add proper tracking parameters to all the emails? Because emails are a great way to stitch customers between devices. So we help them to build the whole infrastructure and the whole architecture of proper tracking and identity graph building and finally building the attribution they can trust. Then of course we always encourage them to add self report attribution. And we also have a very robust methodology of self report attribution. We never ask them, just add a dropdown on the checkout form, how did you hear about us? With three options, because these three options might be limiting the customer and sometimes you don't even know what findings you will see when they're going to answer and how you will compare this to what you are tracking. So for us it's very important to see to compare what we track with what customers actually say. And that's why we always encourage to put like a normal text field with a free input, like you can input anything you want. And then we process this using LLM and categorize into different categories. It gives us flexibility. First of all, it gives us a lot of finding, okay, what customers that come from brand actually say what customers who come from organic actually say where they came from. And then we can process it into proper categories and then we can apply re attribution. But also we can learn sometimes we find some answers that we didn't know about, like different programs, some affiliate websites we didn't hear about. Word of mouth actually is amazing way to understand what is the strength of the brand. So and also it gives us flexibility to retrospectively change these categories if for example, we miscategorize something. Because if you just put very strict like select box, you just have three or five options which are very strict. There is no other way. And also sometimes people are biased just to choose the first one. They just choose the first one and that's it. So first of all, building this really good attribution that you can trust with all these technologies that we have, like identity graph, self report distribution, retribution, assigning donors and contributors, etc. The next step of course is marginal analytics. Because every DTC brand cares about actual margin and people talk a lot about IROs like incremental ROAS which does not exist. But the closest thing we can get to it is to apply marginal analytics. And we've tested. Even if you're going to take different attributions like last click, first click, at some point at specific budget level, returns start diminishing and it doesn't matter which attribution you use, you're going to have campaigns that scale linearly at a high budget and you're going to have campaigns that Scaled really well and had amazing roas with any attribution. But then returns started diminishing. Now you have 0 incremental value, but you keep investing money. And this is an elasticity problem and this is the calculation of diminishing returns for every single campaign. And of course we work a lot with lead generation businesses, but probably this is not for this podcast that have absolutely different set of problems. Like with e commerce, everything is straightforward. Someone purchased, you have money. One thing that really helped us is to make a clear distinction between returning customers and existing customers. And we understand that if you're going to focus on existing customers, you might be leaving money on the table. While if you're going to focus on new customers, existing customers going to see your ads anyways. So this is like a collateral dam. I know. It's like even if you're going to exclude everyone everything possible in all ad platforms, you will see that existing customers still see your ads and there are still conversions from existing customers attributed to ad platform. You cannot just, you just can't prevent it the same way. Some people say if you use first click attribution, your retargeting will be not evaluated properly and you will not. Because if we have real first click retargeting would always get zero value because retargeting cannot be the first channel. But just because our reality is so complex and imperfect anyways, you're going to see a lot of customers who come first click from retargeting just because of the complexity of the world. And it's not so by design. We use this flaws of the system just to be able to find the model that can utilize these flaws.

[B]

And it's ultimately about finding those new customers.

[A]

Yeah, but coming back, I would say still like even though we have all these sophisticated technologies, geo holdouts, marginal analytics, automatic integration, applying costs, like we even have AI agents that completely manage your budget. So you just integrate attribution, reattribution, marginal analytics and you can just click apply button and you don't even need to go to add platforms make bidding strategies change, budget like everything going to be done by AI using API. But the biggest thing still that our customers find valuable is our expertise. So we, we try to cut through bullshit as much as possible. Yeah, there are like some people say that the easiest way to make money to lie to people that love to be lied to. So they really want to hear the lie. And some people just tell them the lie and everyone is happy. And it works like that. But our idea right now is to work with People who want to hear hard truth and even though it's hard in the very beginning, in the long run they're going to win. It's also before we also try to sell truth to people who want to be lied to. It's a horrible business now we don't do this anymore. Yeah, yeah, yeah.

[B]

So well, if you're in our audience and you are ready to hear the hard truths, you've got to go check out segment stream and just I'll put your, your LinkedIn. You're always dropping your hot takes on LinkedIn so you got to follow Konstantin on LinkedIn. There anything else you'd like our listeners to do regarding segment stream?

[A]

Yeah, so I would say you should perceive marketing measurement as going to a doctor and imagine but you need to be serious. Imagine like you have a serious problem like cancer or something. Of course you wouldn't go to just one doctor and this doctor will tell you like you need to cut it out or you need to have different like opinion from different worldviews from different. Maybe, maybe you can even go to some shaman to check like you need to have. Because problem is so serious that when you have this level of problem which is usually like when you invest budget, it's like it's core of the business, especially in D2C you need to have different opinions from smart people. You can talk to them and make sure that you don't just follow the story that the best aligns with your beliefs. Otherwise you're gonna fall into confirmation bias trap. And this is something very important for founders to know about because still when you have head of digital who is working for salary, is afraid to be fired, is afraid to make mistakes, is afraid of hard truths, I can understand this. And that's what is heavily exploited probably by all these people who spread these fairy tales and myths. But when you're a founder, when you really see the money in a bank account, when you really invested your time, your life and maybe your finances into particular business, I think the truth is very important even if it's not always very comfortable.

[B]

Love it. Well, thank you for coming on the DTC podcast today. Get in touch with Konstantin if you want to know the hard truths. Awesome as always. Thanks man.

[A]

Thank you Eric. Was pleasure.

[B]

Thanks so much for listening to today's episode. If you're not a subscriber to our newsletter, you can do that right now at Direct to Consumer all one word. I'm Eric Dick and this has been the D to C podcast. We'll see you next time.