← AdExchanger
AdExchanger
There's No Such Thing As An Attribution Easy Button
00:50:00
There are many misconceptions about ad measurement. But the biggest thing most marketers are wrongheaded about is in thinking there’s a single easy button for attribution. It simply doesn’t exist, says attribution expert Madan Bharadwaj, founder...
Loading summary
Transcript104 lines
- [00:13]
Alison Schiff
Welcome to Ad Exchanger Talks, the podcast devoted to examining the issues and trends in advertising and marketing technology that matter most to you.
- [00:26]
Sarah Sluice
Today's episode is sponsored by Philo Ads. Philo Ads is built for brands that want premium connected TV placements without the premium price tag. With highly engaged audiences on fan favorite Networks like Discovery, TV One, MTV, A&E, Own, Lifetime and more, Philo Ads delivers a CTV experience that gets results. Kick off your next campaign with Philo Ads today at ads Philo TV.
- [01:02]
Alison Schiff
I'm Alison Schiff and this is AdExchanger Talks. Thanks for listening and or watching. My guest this week is Madan Bharawaj, the founder of M Squared, which is a measurement startup focused on advanced marketing attribution solutions for marketers. Their approach is built on the notion that no single attribution methodology fits every business, so there's no such thing as a one size fits all solution to attribution. We'll talk about what that means from a practical perspective and we'll touch on some of the specific, spicier topics, like whether brands can actually trust the measurement they get from Google and Meta. But first, it's time to get your tickets to Programmatic IO New York, taking place in New York City, of course. And cool news. Madan will be speaking, so if you want to get smart about attribution, you should snag your spot. Pro Guyo is September 29th and 30th and podcast listeners get 25% off their ticket when they use the code PODCRUSH. Use P O D C R U S H all one word and all in caps. See you there. Hey Madan, welcome to the podcast.
- [02:07]
Madan Bharawaj
Alison, thank you so much for having me on. I'm a big fan of the show for many, many years. Looking forward to this. Thank you for having me.
- [02:15]
Alison Schiff
So I know you've listened to Ad Exchanger talks for a while and we're delighted to have you as a guest now. And that means you know how we like to start out these episodes, which is with a classic question. What is one thing about you that not a lot of other people already know?
- [02:32]
Madan Bharawaj
Yeah. So great question by the way. I love this. If you go out, look up my LinkedIn, it looks like I'm a classic engineer turned sort of ad tech martech kind of a guy. But even before that, my very, very first job was in publishing. When I was in second or third year college, me and a couple of my buddies, we started a publishing startup back in India and ran it for two years. In fact, I stayed back for a year, trying to make it like a real thing. I failed gloriously, shut down shop and came over here to the US and the rest is probably public history, but that's probably a piece of history that nobody knows about.
- [03:06]
Alison Schiff
I do love the phrase failed gloriously. That could maybe be the name of your memoir, right? Like failed gloriously. Colin Madan Bharawaj Story but you did not fail gloriously because you segued into what is a very successful career in analytics and measuring media. But how did you segue from computer engineering and electrical engineering, which is where you started, into analytics and this very weird industry that we both found ourselves in?
- [03:37]
Madan Bharawaj
Yeah, it's a fascinating industry, by the way. And I grew up as a data scientist in the industry way back, talking about early 2000s. And my very first couple of jobs was in very technical engineering jobs. So I would work with five data points, optimize 50 cents worth of efficiency in some sort of a control system or something like that. And my very first job I fell into, I was looking for a job in the big publishing companies like either Google or Yahoo or something like that. I found a job on the buy side with an agency called Aegis, which is now Dentsu. And the very, very first time I walked into the office, I asked him, where is this thing that I'm a data scientist. I'm going to write algorithms for optimizing search SEM media. Where is it running? So I was expecting to be taken to a back room and show a big server and whatnot. They said, hey, look at the little machine right below your desk. That's the thing that's spending $10 million a day. And I'm like, what are you talking about? And this thing was like lying on its side a little bit. I'm like, why is it lying on the side? Oh, it's got a loose cable. And I'm like, wow, this thing is the One that's spending $10 million. I don't know, it's a week or a day or something like that. And it was a. I got hooked on it. I got hooked on the whole thing. And it was fascinating. I was. It's kind of like heady times back in those days in search, in the early search years, that's how I got into ad tech, just through the sort of as a data scientist. But eventually I kind of grew up to have my own leadership roles and eventually starting my own company.
- [05:07]
Alison Schiff
This is a total tangent, but that story about a little device on its side to make sure that the wire is in the right place. Reminds me of stories my dad would tell about being a radio engineer in the 70s. And they would be playing these tapes and maybe three tapes to a series, and they had the first tape. When the first tape ended, they put on the second. And then all of a sudden they're like, looking for the third tape. Where's the third tape? And they realize that someone had put the third tape as a doorstop to keep the door open because the engine booth was so hot. So they're running around, they finally find it, they pull out the tape from the door, throw it on. What happens behind the scenes?
- [05:46]
Madan Bharawaj
That's right. That's right. Whatever you have to do to make the world spin, right?
- [05:51]
Alison Schiff
So I want to keep meandering down your career path for a bit because you have been focused on ad measurement and analytics for nearly two decades now, which means you've got the long view. And I also commend you for keeping your sanity, because ad measurement is complex. It's constantly evolving. That's what makes it interesting, but also kind of infuriating because people just keep clinging to these outmoded methodologies and it really is enough to drive you nuts. But before M Squared, which is the consultancy and also like a network of experts that you created that focus on solving attribution challenges for brands using methodology agnostic approaches, which. We'll talk more about what that means in a bit, you were at Visual IQ before it was acquired by Nielsen, and I was doing a little research on you, even though we know each other, just to get some more facts. And while you were there, you helped launch the first algorithmic multi touch attribution product, which is very interesting. And then you also co founded Measured, and you were the cto. And I think most Ad Exchanger readers would know what Measured is, but it's an independent measurement company and it really helped push the concept of incrementality into, I don't want to say the mainstream, but, like, what counts for the mainstream in this industry. And obviously, like, so much has changed over the years, but I don't know. Unfortunately, a lot has also stayed the same, which is why ad measurement is still kind of a mess. And it's one of the most debated topics in advertising and one of the most important topics, because there's a dotted line right from that to your wallet. But I want to talk about principles, like, what hasn't changed? What are some of the core underlying foundational truths of ad measurement that remain true, consistent, like, regardless of how technology changes or how the industry Evolves.
- [07:43]
Madan Bharawaj
Amazing. What a great question, right? I mean, you have to go back to the original Wanamaker quote, right? Half my advertising works, half my advertising doesn't. I remember looking it up, it was 1910 or something like that, you know, I mean, I would say it's not probably half right now, but the adage is still predominant, still very much true, which is you still can't tell what's working, what's not working. Primarily because measurement at its very core is measuring human behavior, right? We're trying to put a creative or a piece of media in front of people and asking them for their money. You're trying to change people's behavior and trying to get them to activate and do something you want them to do. That at its essence is trying to change hearts and minds, change human behavior. That's never going to be a solved problem because it's always moving on you and you learn something about humans and then you kind of, you exploit that, leverage that and use that for driving your business and eventually that changes on you, right? So it's always moving, right? So that is probably why it's never been a solved problem. I don't think it's ever going to be a fully solved problem. But it's a pursuit, like a pursuit of happiness will pursue measurement forever. And that's one truth that's probably has been the same and probably will be the same for forever. Second is as we meander through the various kind of zigs and zags through the industry, one thing has always been true all the way back to the CPG years when apparently the very first media mix model, the story is that it was run by the Koch CMO. He was funded by the Koch CMO to understand TV's impact on retail sales. And apparently Nielsen came out and there's a whole sort of story behind it. But the point is that even from that point to now, it's always been about understanding business outcomes. The effect of marketing media advertising on business outcomes. It's always been true. Incredibly challenging to do because in the center of that is actually human behavior. But that's been true all along. Whether you look at multi touch attribution marketing, mixed modeling, incrementality testing, post purchase surveys, qualitative survey studies, all types of causal studies are fundamentally trying to understand the stimuli which is marketing investment to outcomes, business outcomes. That's been true the whole way along.
- [10:01]
Alison Schiff
I hate to ask a facetious question, but isn't it also just about placating people because they have their ideas, fix a And they don't want to be told otherwise. They just want you to prove what they already think they know is true.
- [10:13]
Madan Bharawaj
That's the thing we have to navigate as professionals as to People have pet theories. How do you hold your integrity when you consult around measurement while trying to write a report card on what people's works? Basically they worked many, many years on building a campaign, a brand and an execution. You issue a report card, how do you interpret that? How would you present that in a way that is fair and empathic is definitely a strong point of the consulting lifestyle that comes along with measurement.
- [10:48]
Alison Schiff
So I It's time to get nerdy. Like to get nerdy as hell. So I I spent a lot of time in preparation for our conversation today reading articles and explainers and posts on the M Squared website. And one post I read is about the influence of the marketing funnel on marketing measurement. And there's this section about the different types of marketing and measurement tactics. It's like very broadly speaking. And you point this out in the piece, you can categorize measurement tact into two types, base attribution and advanced attribution. So that sounds pretty obvious, right? Like base, like basic and advanced is advanced. But what is really the difference between these two types of attribution? And why is the industry moving like away from base and more toward advanced attribution, like this embrace of causal and incrementality based approaches, which is definitely something we've been noting at Ad Exchanger over about a decade.
- [11:47]
Madan Bharawaj
Yeah. Just a way to simplify these really complex concepts. There's so much jargon in the industry. Right. So we want to be able to make it simple so people can absorb these concepts. Base attribution is this very simple idea of all the correlational and tracking techniques we've come to call attribution over the last 20 years. Last click attribution, Google Analytics, UTM attribution, platform attribution, like app attribution. A lot of these are correlational and tracking techniques that we've come to call attribution. And then there is what we call causal attribution, which is to understand from a statistical perspective what caused the actual outcomes to happen. So there you have techniques like randomized control tests, incrementality testing, marketing, mixed modeling, which are trying to understand causality and assign causal drivers to the outcomes that actually happened. Right. So those two big buckets we call base attribution, advanced attribution. Because a lot of these things are not just purely incrementality, incrementality testing, regression modeling, Sometimes post purchase surveys can be thought of as poor man's incrementality. Qualitative studies can be interpreted as causal. So a lot of those things can. A cluster of things can be thought of as causal. So we call that advanced attribution. And base attribution is all the tracking things that we do that we've always done forever and ever. The big reason why you even need these two things is that almost all causal techniques are episodic in nature. If you run an incrementality test last month on Meta, that's your read on last month's meta efficiency. If you ran a search incrementality geo test like last quarter, that was the efficiency of last quarter, right? So how do you use that? Or a marketing mix model, which is mostly only giving you meaningful reads on a quarterly basis for daily planning, weekly reporting, next week's sort of optimization. How do you do that? You have to connect those two things together. All the last click, or the base attribution typically is daily or even real time, whereas causal attribution is episodic. And you connect them together through something called multipliers. Say, hey, When Facebook reports 100 conversions, the multiplier is 70%. So only 70% of whatever Facebook says is actually the last click. Attribution is truly incremental. Therefore 70 of those hundred conversions are estimated to be incremental or causal. And 30 is what we call baseline. So to be able to establish this concept of multiplier, it's simple to establish a base attribution and advanced attribution framing. So now you can create an operational framework for measurement that can really drive marketing operations, reporting, forecasting, planning, all those different things that marketers do on a daily basis.
- [14:27]
Alison Schiff
Okay, got it. So you need both things and they complement each other. It's not like base then gives way to advanced attribution.
- [14:34]
Madan Bharawaj
You got that? That's exactly right. You need both. Causal attribution by itself is more like studies. You do it once, maybe a couple of times a year, maybe 10 times a year, depending on how much you need studies done on a particular channel or a particular type of outcome now. But you need daily reporting, weekly planning and sort of CFO reporting, CMO reporting and whatnot. And the way you connect that is through this concept called multipliers. And you need both. You're absolutely right.
- [15:00]
Alison Schiff
So that leads me to a question about misconceptions that you encounter among marketers when it comes to ad measurement. Because I do think there is a conception that we're evolving toward better Measurement, which is true, but that we're eschewing older methods, which is not true because not only is everything old, new again, but you need a combination of methods to really get to truth. So what are some of those misconceptions that you encounter about mta? Mmm, incrementality testing? Like what are people commonly wrong headed about?
- [15:36]
Madan Bharawaj
Almost the most important thing is that there is no one technique that rules all. There is no one type of mmm, one type of test, or one type of approach that is right for everybody. You have to bring them together. The magic or the mystery is in how it comes together for your brand or your particular business unit you're working on. That is it's almost never in either incrementality testing or mmm, it's not either. Last click attribution or causal attribution. Right. It's about bringing them together. That's one big thing, which is that it's not either or, it's kind of, it's almost always an. And number one, number two is that there is no one real one truth, so to speak. Because underlying we are human behavior. We can only understand human behavior to certain level of. What do you want to call it? Precision, Right. So when you measure it through an incrementality test and you estimate it through a regression model like mmm, they'll both give you similar answers, but not exactly the same answer. So the idea is that you need something called triangulation so that you can triangulate multiple sources of ground truth to make one investment decision. That is a sort of an important kind of concept that's slowly beginning to like take root in the industry, that there is no one source of truth. Right. You have to take multiple sources of truth and be able to use all of this. And most marketers triangulate naturally. It's not something that you even have to be told that there is a new technique called triangulation. Many tenured marketers, when I work with them, if I go and present a read, they will kind of bounce that insight I'm giving them with their more intuition for the business and what they kind of viscerally understand about what's working, what's not working, and they're triangulating to other things they know about the business. Right? So what we have done is that we kind of formalize that approach into a sort of a proper formal analytics technique. And now you can do it in a sort of a more formulaic scientific way as opposed to having to do it or inventing a new version of it every time.
- [17:41]
Alison Schiff
Regardless, though, do you think most marketers really know how their ad dollars are performing? Because I get the sense, putting aside this sort of natural triangulation that happens just from everything, you're saying that marketers think they know how much of their is being driven by media, at least directionally, but in reality they do not have a clear picture. So, yeah, I mean, what you just.
- [18:04]
Madan Bharawaj
Said is actually the most accurate way to describe it, which is they have a directional sense. Most marketers are investing their dollars, they feel that in the business, but they can't exactly pinpoint what's working, what's not working. But when something is absolutely working, they understand something is kind of bombing on them. They kind of like have a good gut sense about it. But can you quantify it? Can you say that the Facebook program contributes 25% of the total business? Well, they're not able to say that. That's where techniques actually come into play to quantify it, calibrate that so that you can make more sort of tailored, designed decisions around it. That's where a lot of the techniques come into play.
- [18:44]
Alison Schiff
How precise can you get? And are marketers looking for a level of precision that's just not possible and also not even needed? Right.
- [18:55]
Madan Bharawaj
Yeah. This is an incredibly salient point you're making. This is very coarse, to be super honest about this, is that a lot of times growing up in the industry, we would present the CPA numbers as 45.55, ROAS is 2.333 and whatnot. Honestly, there's a big bound of uncertainty around those numbers. It always existed. Whether you measured on a last click basis or a causal basis, whatever always existed. We're so used to presenting it in this false position narrative. Honestly, that's any advanced measurement you do has a ton of uncertainty around it. And you're right. It doesn't have to be super precise for you to make a ton of decisions. Some decisions, maybe 2 out of 10 decisions, need a lot of precision, but 8 out of 10 decisions are kind of like more clear. Whether it's like left or right, right or wrong is kind of pretty obvious once you take the right approach to it. Right. And that's where most of the alpha is. And then there is still a lot of alpha left in making those, those big investment decisions. Right. But you will get 80% of the way if you just take a good process to it and stick with it, as opposed to like doing it once and not kind of like following through.
- [20:08]
Alison Schiff
One reason that marketers don't have a great Sense of ad performance is, is because of the walled gardens, right? I mean, there's been this narrative about the walled gardens grading their own homework for years and years. It was one of the first concepts that I covered at Ad Exchanger over a decade ago. You know, they provide metrics that reflect their own ecosystem. They overstate their own impact. I mean, obviously they do, but yeah, I mean, can marketers trust anything about platform provided measurement?
- [20:43]
Madan Bharawaj
In most of the times, they can trust it to be consistent. That's what they can trust. It's usually biased in one way or the other. It's biased either too strongly in the platform's favor or sometimes their attribution is so weak that you feel like it's working, but there's no signal. They're not able to actually attribute the conversion event to the media event. So one of those two things is right and it's consistently wrong. So that's where all measurement, multiplier, the concept comes in. If you can get causal measurement in place, that gets true measurement, true value. As true as you can possibly get in the world we live in. Now you can attach a multiplier to correct for the biases. Now you get a much closer estimate, but on a daily basis, right. You can have Facebook and AdWords and TikTok and you know, I spot all report wrong numbers, but consistently report the wrong numbers. Now you can have a multiplier adjusted number that translates it to your currency, your brand's currency. Now you have the cheat sheet to be able to make investment decisions and sort of get the alpha required to maximize your outcomes as opposed to the platform's outcomes.
- [21:53]
Alison Schiff
So consistently biased. Kind of like that one uncle at Thanksgiving dinner. So you know what he's going to say.
- [22:00]
Madan Bharawaj
That's right.
- [22:01]
Alison Schiff
And you can, how you can manage it, right.
- [22:02]
Madan Bharawaj
Once the. Once you know the uncle's gonna be drunk and he's gonna say weird things, you can put him in the corner of the room, put somebody right next to babysit him and no, the rest of the party can go on.
- [22:12]
Alison Schiff
So we're gonna take a quick break now, but when we come back, we're gonna talk about being practical, like how to put some of these concepts into practice. Because if I'm a marketer and I want to do better measurement, I can't just do better measurement. I have to change my mentality. I have to make certain changes within my organization. It's a process. So stick with us and we'll get into that.
- [22:50]
Sarah Sluice
I'm Sarah Sluice, executive editor of Ad Exchanger and ad monsters. And I have with me here Reid Barker, who is the head of advertising for Philo. Welcome, Reid.
- [23:00]
Reid Barker
Hey, thanks for having me.
- [23:02]
Sarah Sluice
So let's start by telling us a little bit about who watches Philo. Why do users come to your service and who is the audience?
- [23:11]
Reid Barker
Yeah, so Philo is the streaming service that has the linear channels from the cable, entertainment, and lifestyle networks you've known and love, from Warner Brothers, Discovery, Paramount Global, Hallmark A&E, plus the entire AMC library. So with that entertainment and lifestyle focus, that means our audience is primarily women. We see them watching it on the biggest screen in the house, and active users are watching linear television more than three hours a day.
- [23:39]
Sarah Sluice
So we know that you're different on the audience side. But let's talk a little bit about the tech, the programmatic pieces that our viewers love. Agency buyers are flooded with CTV options, fast networks. So tell us what differentiates Philo on the ad side.
- [23:56]
Reid Barker
Yeah, you know, the big conversation is all around dollars moving from direct linear to programmatic. And Philo's been programmatic from the start. That means we've been in programmatic advertising more than seven years. That means we understand that programmatic is not just a machine talking to machine that you plug in a pipe. It's about really understanding what goes on there. So we feel ourselves are really have a lot of expertise in this area. That means that we need to have the best signals both for identity as well as context, the most transparency, and really listen to the buyer so that we can say yes to what they need to fulfill their KPIs.
- [24:34]
Sarah Sluice
Great answer. Reid, you and I were both at can. What are some of the trends and innovations you saw there that are exciting you in the CTV space?
- [24:45]
Reid Barker
Yeah, I've been around innovative ad products forever and seeing some of the things like pause ads, interactive ads, suddenly becoming standardized in a way that makes them available via programmatic channels is just. It's great to see they're actually finally going to get adopted. And the other thing is just everyone's talking about mergers, sell offs, spinoffs, and I feel like I'm a Jack Nicholson at a Lakers game with the front row seat of what's happening. And in addition to actually just watching, I'm also on the field. So it's amazing and amazingly terrifying all at the same time.
- [25:17]
Sarah Sluice
Well said, Reid. I couldn't say it better. Thank you for joining us on the podcast.
- [25:22]
Madan Bharawaj
Thank you.
- [25:39]
Alison Schiff
All right, we're back and we're going to get practical as promised. I'm an advertiser. I'VE jotted down that WannaMaker quote on a post it and taped it to the side of my computer screen. I stared it every day and now I'm just like, I'm ready. I'm ready to find out which half is wasted. How do I get started? What's the first thing I have to do if I want to benchmark every channel in my mix and then start using multiple measurement approaches to get a real sense of what's up? That seems like a multi step, step, multi phase and probably never finished process.
- [26:11]
Madan Bharawaj
That's a great way to put it. But there is almost certainly now tangible, finite steps you can take to kind of just get better and the steps you would take it depends on the place you are in your journey, so to speak. Right. Let's start at the very beginning. Imagine you are a brand new E commerce brand. You just got launched probably in the last six months, eight months, a year or whatever. You're struggling to get any demand going right now. You're trying to put ads on meta ads on TikTok, find some search and stuff like that and you're trying to get yourself going. You don't need a lot of fancy measurement. Honestly, last click measurement is plenty for you. It turns out there are two types of last click measurement fundamentally. One is called platform attribution, one is called UTM attribution. They're both base attribution, but they're both kind of incremental for a brand that doesn't have a baseline. If you don't have a ton of demand in market, you can use either one or you can use them both together through something called triangulation. That's plenty of signal for you to optimize. Now as you grow a little bit, as you get to a point where you have maybe a million dollars in sales, maybe about 2 million in sales, what will happen is that your Google Analytics, which is your UTM attribution, and your Facebook attribution, Google Attribution, the platform attribution, will start to diverge from each other, right? And as they diverge from each other, it'll be harder and harder to rely on either one of them fully. That's when you start adding something called post purchase surveys. Now post purchase surveys, it doesn't work for all the brands equally, but it's a really good proxy for incrementality. It's what we call poor man's incrementality in my class. It's a good when you ask a customer out right after they bought, hey, where'd you Hear about us, right? Whatever they remember is usually not a bad proxy for incrementalities. And that can take you a little bit further. So almost up to 2, 3, 4 million in sales. And as you get there, that's not precise enough for you. Now you get to a point where you feel like, well, this is not good enough. I need a little bit more inquiry, a little bit more calibration. You start doing platform lift studies. You do Facebook live study, a meta lift study, a YouTube study or whatever it is, and it gives you a little bit more calibration. You start adding multipliers. At this point, say, okay, what is my last click? What's my causal? What's my multiplier? So you start getting a little bit more calibration, right? And then you start doing geo tests. And then eventually you get to a point where it's like, well, I have too many channels now, a lot of different complexity. I have different business outcomes that I want to measure. Now you get to something called a marketing mix model, right? You can slowly stack up your complexity. And along the way, the most important thing you'll add is what's called maf, Marketing Accounting Framework, which is you will now come up with an approach for measuring marketing in a way that both finance and marketing can shake hands and agree on the metrics that are important for the business that both marketers and finance can get aligned to. And that particular approach is called marketing accounting framework. Somewhere when you're about 15, 20, $25 million in GMV or revenue, it's when you still start thinking about new customers, returning customers, you know, high value products, low value products, high LTV customers, low LTV customers, Business unit A, business unit B. Right. Trying to think about that more thoughtfully, intentionally so you can understand what media drives, what kind of outcomes, and how to account for the total holistic impact of the media. And translate that to more granular metrics like contribution margins, gross margins, and profit and stuff like that. So everybody can get aligned behind those metrics.
- [29:43]
Alison Schiff
I mean, it really is a cocktail.
- [29:45]
Madan Bharawaj
It is a cocktail. It is a cocktail. That's absolutely right.
- [29:49]
Alison Schiff
What would you name an attribution cocktail? Just, I always wonder how cocktails get their names. Some of them are very creative. We'll put a pin in that. Maybe anyone who's gotten to this point in the podcast can send me an email, allisonexchanger.com and let me know what you would name your attribution cocktail.
- [30:08]
Madan Bharawaj
And maybe we'll host a lot of events. And I'm happy to name it after the ad exchanger name that pops up. Absolutely.
- [30:14]
Alison Schiff
That would be the specialty cocktail. Well, maybe we could call it Last Click Won't Die. Like, I'll take a Last Click Won't Die, please. Like, by the way, why won't it die? Everyone agrees it's an oversimplification. It ignores the fact that a customer journey exists. But it's like a cockroach, right? Like the world will end and all we'll have left is cockroaches. And last click attribution.
- [30:38]
Madan Bharawaj
That's right. I remember doing a video in 28 or 9 that this is the year last click attributions get replaced. 2008 or 2009. I remember that. 2025. I'm still talking about Last Click Attribution wouldn't die. It's just one of those amazingly durable things. Do you know the story of how it was born?
- [30:59]
Alison Schiff
No. Please do tell.
- [31:00]
Madan Bharawaj
No, apparently this is again, it's all sort of, you know, what do you call it? Legends. Right. Modern legends. Apparently it was invented in omniture. Remember Omniture?
- [31:12]
Alison Schiff
Of course, yeah.
- [31:14]
Madan Bharawaj
Back in those days. So it was sort of a very simple way to. This is back in late 90s when there was no E commerce. Early early days of the Internet were not. And they would have to like, find something to attribute to events that came through some referral source, say, look at the referring URL and call that Last Click Attribution. Right. And it was so simple to implement. Every platform implemented it along the way because there was a standard and 25, 30 years after, it's everywhere and cannot be taken out.
- [31:48]
Alison Schiff
It's also like a lesson in, I guess, not making predictions. Even though journalists like myself will always force people to try and make predictions and then hold them to it. Because when you say this year is the year of something or you put a stake in the ground, you're almost always proven wrong. Right?
- [32:07]
Madan Bharawaj
That's right. That's right.
- [32:09]
Alison Schiff
So I want to switch gears a little bit and talk about the role of AI and machine learning and how it's changing ad measurement and attribution frameworks because it's going to transform it completely. So how does AI unlock new forms of measurement or new approaches? I think of, I don't know, the ability to address really complex multichannel customer journeys more easily. And this is pretty simple, but I feel like it would have a big impact. Insights like actionable insights super quickly and then also being able to optimize your budget allocation really quickly and in an automated way like that. That Kind of thing.
- [32:51]
Madan Bharawaj
Amazing question. A lot of us have been thinking about this, as you can imagine, day and night. There's an amazing discussion in MIT by a bunch of professors. I'm trying to remember the Susan ati. Oh yeah, she's Stanford. A couple MIT professors whose names are escaping me right now. Amazing discussion about AI. The fundamental sort of crux of the conversation is that most of AI is trained on current data, right? Whatever data we have out there. And most of the AI reasoning is correlational, not causal, because there's not a lot of causal data out there. It's a lot of like, you know, not a lot of testing and measurement, incrementality testing metrics are out there for it to do sort of causal thinking. Most of it is correlational thinking, right. So there is a data collection and testing and sort of a causal infrastructure and causal insights that need to percolate the world. That's still to be done. So that's a sort of a still chasm that we have to jump over. Right. But once you jump over it, the insights, mining it for insights is going to become super, super efficient. There's still a lot of AI based, I would say automation that's just taking over the space, right? You can run these media mix models very, very quickly. But what is the right data to feed a marketing mix model? So that it's truly incremental, so that you're controlling the right for the base variables and what are the right control variables? Right. Those are all things that we are developing the framework and the literature around and the research around that is happening right now. So the truly causal data collection and the data we want to call literature or the body of knowledge is being built right now as we speak. So most of these organizations I work with are in the process of trying to put down the infrastructure to understand what's causal, what's correlational. Now once you do that, once you have data that gets pumped out, where you can look at weekly data, daily data, that's all causal now to generate insights so you can actually have super cognition on what's working, what's not working, where are the sort of pockets of demand, where can you unlock demand very quickly? And all that stuff that's going to be super, super exciting. We're not yet there. I think once we get there, I think all possible efficiencies that you can have will be accessible and we'll be living on the efficient frontier, offer investments all the time. So we'll be living anything we'll know we would have Optimized towards. The only thing will be the mystery of the unknown. What can human beings be poked to do? Can I poke people in a certain way so they'll move left or right? What we don't know is the only thing that will be mystery. Anything that can be known will be used, exploited, maximized to its fullest extent. I think we're not there. Honestly, it's at least a few years away from being there. But that's the journey that all of us are walking towards.
- [35:57]
Alison Schiff
I think it'll happen slowly and then all at once, right? We'll just be talking about it and talking about it and suddenly it'll just be there and it'll be the way that people do business and they'll really have to change their mindsets. But by the same token, you have the rise of privacy regulations and privacy focused platform changes. Putting aside the fact that third party cookies aren't going away. La da da da da. God, if I had like a nickel for every time I've said and written the word deprecation, I would be I could retire straight up. But yeah, like how do all of us could for sure, like how do these changes and like more privacy focused scrutiny from regulators, like how does that change what's measurable? And like. Any advice for how to deal with with signal loss from a measurement perspective?
- [36:46]
Madan Bharawaj
Great question, great question. Now all of causal measurement today is not all of it, but most of it is basically statistical, meaning that we don't look at user level data, we look at cohorts of users or we look at signals like time series signals to estimate causality. Right? So by definition most of the causal measurement does not use user level data. Right? So number one, it's kind of future proof from a privacy perspective because we don't use user level PII data to get the measurement number one. Number two, other side, the way you calculate multipliers is by looking at base attribution and causal attribution, bridging the two with the multiplier. Now, as you have signal loss, the multipliers have to get readjusted on a regular basis, right? And we don't see that much instability today. But it does happen and every so often we see some jumps that we have to go and account for. So I think the idea of having a practice in place, an advanced measurement practice in house, so you are cognizant that of the right framework and having at least the cognizant, if not the infrastructure, at least the awareness, that's what you're going to do is probably the biggest thing you can do if you're a marketer or an agency supporting a marketer. I think having to think about, as you said, signal loss and what kind of measurement framework allows you to handle signal loss is the big sort of like, you know, leap you can make today.
- [38:12]
Alison Schiff
Well, what do you think is the. What's the most overrated ad measurement tech right now? Like, something people are talking about. And you're just like, guys. Like, you're just. You're missing. You're missing it. Like you're just focused on something that's.
- [38:26]
Madan Bharawaj
These are all my friends, okay? If I say something, somebody's being offended. But I've never understood data. It's been like, you know, programmatic. When people sell data, right? And especially for measurement, they'll say, hey, it is like so and so, like a data set for you to measure. And there's these panels of like, you know, 100,000 users, 50,000 users and whatnot. And the data sets are super noisy. They are very, very sort of unlikely to be accurate and stuff like that. I find that that's the most. I struggle with it. I've audited a ton of these data sets. It's so challenging to get any of these right. So that's what my inspiration also comes from. The fact that, okay, you can be wrong, just be consistently wrong so that I can use you. I can use you in a way that is useful. That's what I've come to rely on. That. If you take for example, the last click attribution that Meta does, the last click attribution that Google does, the last click attribution that HighSpot does and Pinterest does, they're all last click attribution. So in principle, it's the same algorithm. But actually, if you look at the underlying data assets, the underlying device graphs, and then how they imply policies on, like how Apple imposes a policy on Facebook as opposed to Google, and so on and so forth, and how it transforms the actual last click algorithm that eventually gets manifested in those in those platforms. It becomes its own currency. So Facebook's last click attribution is its own currency, and so is Google and so is Meta. So is. I spot. And so anyway, the long story short, when somebody else says, I have another data set, that's the truth. And you can get perfect measurement through that. I'm like, whatever, just pick your poison. Say whatever you want to say. Just give me consistent reads, right? So that I can calibrate with that. Is there is a place that I would advocate for.
- [40:21]
Alison Schiff
That's such an interesting perspective, right? Like you just need your baselines and then you can figure it out. But if everything's all over the place, then you can't do the math.
- [40:28]
Madan Bharawaj
That's right. That's right. At least having a stable baseline across all of these things. And all the big platforms do, all the big platforms because for their own reasons, they have to have stability, just run their businesses, right. So they, it's just natural business stability forces data stability and whatnot, so allows us to rely on that. But like big events like iOS 14.5 was a big disruption to the space. I mean, a lot of us got thrown off, all the calibration got thrown off and we had to start from scratch, right? That happens. But that happens once. It's like a big sort of seismic event. It happens like you know, every five, seven years, right. So also keeps us busy, keeps us, you know, we have jobs to solve something. So keeps us all employed. So I'm not going to complain.
- [41:12]
Alison Schiff
Well, by the opposite question of what I just asked, what's something in the measurement space that's not like getting the attention it deserves? Something super interesting that people aren't talking about enough?
- [41:23]
Madan Bharawaj
Ah, it's a great question. It's a great question. Okay, I'll tell you what. Again, not something that very many of my friends in management would like. There is no paint by numbers, right? There is no one size fits all. There is no one mousetrap. So unfortunately today what happens is that if you're two DTC brands, take an example, if you're a DTC brand, a DTC brand, B, looks like you're just the same thing. You're selling T shirts, you're selling T shirts. But actually, if you get close enough to their businesses and figure out what they're trying to solve from a go to market perspective, one brand is trying to figure out performance marketing and they are saying, well, all I care about is making meta work and Google work. If they don't work, I don't have a business. That's all I care about. Another brand is trying to figure out retention marketing. They're like, you know what? It's not about new customer acquisition. I have to keep these customers happy, otherwise I don't have a business, right? And somebody else is saying, you know what, I really cannot take those go to market. I really have to take a very influencer LED brand. LED go to market, right? Very different go to market questions they're asking. Measurement has to wrap those business questions and Give them alpha or answers that answer those questions. So it's very bespoke. Therefore what happens? The actual measurement that has to be put in place is very bespoke and custom for every brand. And so there is no one mousetrap that does a good job for everybody. And also, even if it works well for you right now, it changes on you every six, eight months because the questions you're asking are changing on you all the, all the time. Right. And therefore what happens is that most of these tools just work for some use cases and stop working. And they're just prestigious in a company for a long time. Therefore, my point is that it's a point of like, there's definitely a role for tools and technology, but without expertise supporting all the tools and tech, it's just going to be another piece of tech nobody uses, a bunch of reporting nobody touches. And it'll, you know, last click live on for another 20 years and we'll still be talking about this, you know, 2045 and we'll be talking about last click stop.
- [43:30]
Alison Schiff
So we're nearing the end of our time and I'd like to do a lightning round. Yes, ma', am. If you'll play along. So five seconds or slightly longer answers only. And I do mean that. So are you ready?
- [43:42]
Madan Bharawaj
Yes, ma'. Am.
- [43:43]
Alison Schiff
Okay. What is the most common mistake you see even really sophisticated marketers make when they're trying to, to measure the impact of their media.
- [43:55]
Madan Bharawaj
Believe in last click. No matter what they say, they still last click is the truth that they're always there. That's the gravity that they're always drawn towards the black hole.
- [44:03]
Alison Schiff
You'll take it out of my cold, dead hands if you. I feel like the last click is going to be the answer to this question too. If you could wave a magic wand and fix one thing about the current state of ad measurement, what would it be?
- [44:17]
Madan Bharawaj
People's minds to be more open to different answers than what they are used to seeing. That's the magic wand. It's very hard to change people's minds because they've seen this answer a hundred times over. Right. And so to have a different answer and be open to receiving, that is the most challenging thing.
- [44:33]
Alison Schiff
It's also good advice for life. So when you're teaching, because I know that AT Squared, you guys also host these master classes with marketers and teach them about different tactics and you bring in experts. And I know there's no such thing as a stupid question, but do people ever ask any stupid questions? And can you share One.
- [44:56]
Madan Bharawaj
Honestly, there's no stupid questions, per se. No, honestly. No, that's not true. Any questions are welcome. Honestly. And that's the whole point of having a class. That's a safe space to ask questions. No, that's not true.
- [45:09]
Alison Schiff
What's the best question you've been asked that made you go, huh?
- [45:12]
Madan Bharawaj
Oh, plenty. Plenty of questions. Best, Best question. Well, most of the hardest questions are almost always about communication. It's not really about the measurement, the human aspect. The human aspect. How do you communicate this complex soup of what we call advanced measurement to stakeholders, to your board, for example, who don't care for the nuance. Right. They're like, am I going to make money or not? I'm going to sell this company in three years. Are you going to sell it to me? Sell? Or are you going to get the company ready or not? Right. It's a very simple. They kind of have to translate all of this gobbledygook into something very simple. Executive comms is the most complicated thing because there's no, like, playbook, Right? It depends on the personalities who are in the boardroom and the executive suite and your people who are in the operating team and whatnot. That's the most complicated, sophisticated thing you can do, which is communications.
- [46:12]
Alison Schiff
So I know it's about a portfolio of approaches and that's the best way to do this. But if you were absolutely forced to choose, are you team MTA or team mmm?
- [46:24]
Madan Bharawaj
So again, MTA is base attribution. Okay. There are only two causal techniques. Incrementality testing, marketing, mixed modeling. So if you ask me, hey, would you trust an MMM or an incrementality test? Incrementality test is the gold standard with measurement, but you can't run it. There's only so many tests you can run at any one point of time, so you need both. But if, gun to your head, if you had to pick one number that represents the investments that you're going to make, then you should choose an incrementality test.
- [46:54]
Alison Schiff
And last one, what is the most overused measurement related buzzword right now that you wish you never had to hear again?
- [47:02]
Madan Bharawaj
I'll tell you. Mta.
- [47:03]
Alison Schiff
Mta.
- [47:05]
Madan Bharawaj
Many people will be like, no, I'm really trying to solve multi touch attribution. I'm like, I know they're trying to say causal attribution, but they'll use the word MTA for causal. And I have to translate in my mind and have to find the right opportunity to have that conversation. Yeah. Most overused is mta. Absolutely.
- [47:24]
Alison Schiff
So, guys, let's make Madan happy and let's understand what MTA is, and let's stop using last click because it's time. Yes. By the way, I came up. While you were talking, I came up with a few cocktail names. And you can take these if you want to, please. Complex soup. I'll take a complex soup, please.
- [47:45]
Madan Bharawaj
Doesn't sound like a cocktail. It sounds like a soup. How about something fizzy?
- [47:51]
Alison Schiff
Incrementality on the beach.
- [47:52]
Madan Bharawaj
Oh, that's very cool. That's very cool. I take that.
- [47:56]
Alison Schiff
How about Snappy Cappy?
- [47:58]
Madan Bharawaj
Snappy Cappy.
- [47:59]
Alison Schiff
Snappy Cappy.
- [48:00]
Madan Bharawaj
You gotta say something with causal. Something with, like, it has to, you know, use the word causal somehow.
- [48:06]
Alison Schiff
Yes. I don't drink enough. We could just call it the. The cat. The casual. I keep saying casual when I mean causal. The causal cocktail. I do that so often in stories. Our copy editor is constantly like, do you mean causal? Because I don't think you mean casual attribution, which is definitely something else. How about Manhattan Mix Modeling?
- [48:33]
Madan Bharawaj
Ooh, that's very cool too. Manhattan Mix Modeling. And the causal cocktail is what you said.
- [48:38]
Alison Schiff
The causal cocktail.
- [48:39]
Madan Bharawaj
Causal cocktail. I like those.
- [48:41]
Alison Schiff
This one is my favorite, though. The desired action.
- [48:44]
Madan Bharawaj
The desired action.
- [48:46]
Alison Schiff
I'll take a desired action.
- [48:48]
Madan Bharawaj
Yeah. It sounds kind of wrong, though.
- [48:50]
Alison Schiff
It does a little bit. But is it so wrong that it's right? I don't know.
- [48:54]
Madan Bharawaj
That's right. That's right.
- [48:56]
Alison Schiff
All right.
- [48:58]
Madan Bharawaj
Yeah, it was fun. Really fun. I don't get to, like, talk 45 minutes with somebody as fun and educated as you, so thank you so much for having me. I had a ton of fun.
- [49:08]
Alison Schiff
All right, well, next time you're in town, let me know and we'll grab a. A causal cocktail it is.
- [49:15]
Madan Bharawaj
All right, thank you so much. Appreciate it. Thanks for having me.
- [49:27]
Sarah Sluice
Today's episode was brought to you by Philo Ads. Philo Ads delivers premium content, a CTV first audience, and zero stress ad buying that works for everyone from scrappy startups to savvy marketers. So shake up your ad buying strategy and launch your next campaign with Philo Ads today. Visit Ads Philo TV to learn more.