← Limited Supply
Limited Supply
S12 E6: Redefining Media and Marketing Measurement (with Michael True, CEO of Prescient AI)
00:41:42
Loading summary
Transcript68 lines
- [00:01]
Nick Sharma
Welcome back to season 12 of limited.
- [00:03]
Mike True
Supply, the only commerce podcast with unfiltered and refreshingly hot takes.
- [00:08]
Nick Sharma
I'm your host Nick Sharma and when.
- [00:10]
Mike True
I'm not recording, I'm behind the scenes scaling your favorite celebrity and consumer brands. Let's start talking all things direct to consumer I'm going to tell you about a software that my agency, Sharma Brands uses. With our fastest scaling brands historically last click advertising channels. Think Facebook, Google or TikTok. They take too much credit for what they do. As a result, it's reduced confidence for brands to effectively scale their upper funnel structure strategies, which puts them in a pickle of higher CPAs. A few years ago without Prescient, I would have said there's no real way to track what channels like YouTube TV or even newcomers like Applovin are doing. But with Prescient you can now look at daily multi channel forecasting and optimize your paid media without any bias. Prescient allows you to see the halo effect that is usually unrecognized with top funnel channels and allows you to measure those down to the campaign level. See the impact your upper funnel media really has on E Commerce, Amazon and even retail stores. Plus once you go live, since it's not a pixel based platform, you'll have data and results populating within a week. Try it out and go to prescientai.com Limited to learn more.
- [01:17]
Nick Sharma
Welcome back to another episode of Limited Supply. On today's episode we're actually going to dive into mmm, which is media mix modeling. We've talked before about mta, we've talked about incrementality. Today we're going to talk about the third one which is mmm and we specifically brought on Mike True, who's the creator of Prescient or Prescient however you want to say it. They're MMM tool that a lot of our largest clients use. So and Prescient's also the sponsor of this season. But Mike is a good friend, incredibly talented, incredibly smart, knows this space better than anybody else and we broke a lot of things down to a very basic level. So it's very easy to understand and sort of follow along. And specifically if you run media on channels that are view through channels, channels like TV, TikTok, AppLovin, YouTube or you're selling in places like retail, Amazon, you know, things that are not your dot com, this episode is definitely a must listen for you. And yeah, I hope you enjoy it. If you have any questions you can always hit me on Twitter, DMs or by email you can hit up Mike. His email is at the end. And I hope you enjoy the episode. All right, Mike. Well, it's an honor to have you here. You know, I think a lot of people think that real time tracking is now dead and they might open up their meta dashboard and think, you know, these numbers can't be right, but it might be working. And so I'm excited to go through that whole thought process with you today. But to start off, could you give us just a quick background on what is mmm? And then I want to get into just who you are and why you built Prescient.
- [03:00]
Mike True
Yeah, of course. Good to see you, brother. Thanks for having me on. MMM stands for media Mix model Marketing. Mix model. They've been around since the 60s. It was the primary form of measurement where we spent on a billboard or catalog, tv, anything for the newspaper. And they still wanted to know how was my spend driving retail sales? And so they built these statistical models that tried to look at spend and revenue and find correlations between the two of them. Easiest way to think about an MMM nowadays is it's going to help you measure your top of funnel spend by essentially taking credit from your last click and redistributing it back up to campaigns where you have the highest level of confidence that the credit deserves to be to that campaign. And then it'll provide some like, budget optimization and recommendation models on how you should shift your spend. We can unpack some of the nitty gritty that goes underneath the mmm, but in a general thesis, it's top of funnel measurement. And then how do you, you know, reallocate and re optimize your budgets?
- [03:54]
Nick Sharma
Basically, if you run paid media, this is the episode to listen to. So, Mike, I want to get into your story real quick, so tell us just quickly. You know, you're like, you're a fun dude, but you're also a big software guy now, and you weren't always a big software guy. So how did you get to, you know, where you are today and really just talk about like the story of prescient? How'd you get into mmm? How'd you know that this was something to focus on? You know, I know Prescient separately and outside of this conversation, obviously. But I know that you did. You started doing this before anybody else was doing this in a different industry. So walk us through that story just to give the listeners some context.
- [04:31]
Mike True
Yeah, I'd say I never thought I was going to start a software company. I was always doing large scale software transactions within the analytics Space. So I don't know if anybody remembers, like the IBM Watson back in 2014 that beat the jeopardy. So I was on that team and predominantly focused on healthcare and financial services. I mean, it was, it was interesting. I know we were just catching up about music, but I was at a Byron Bay Blues festival in Australia at the end, said, I'm going to go build a platform that I'm going to sell to a record label. And initially went to go build a model that would predict where artists should tour and recommend local venues, things of that nature. Covid hit touring stopped. I was back against the wall. I'm like, go back and sell software again. You know, it's a good job. But one of my advisors said, hey, you know, with COVID everybody's going to be inside. Labels are adjusting their strategies to spend, do awareness tactics, spend and, you know, and earned to drive awareness for people to stream songs while they're inside their house. The problem is there's no Google Analytics behind Spotify and Apple, so you don't get that last click. Music is the most difficult industry to measure of any industry, right. Because you're gonna think of the noise around Cardi B when she is promoting a song. She's on the James Corbin Show. They're doing Twitter live streams, they're doing all sorts of these viral moments of a dance that goes viral on TikTok, but they're also spending across seven to eight, you know, digital advertising channels, just like you would as a, as a DTC brand. So idea came from a music festival and evolved into where we are today.
- [06:04]
Nick Sharma
And could you just talk like I remember. So for just, just so you know, the things that you think in your brain are so simple, make no sense to most people. So what I'm thinking about is you were showing me this demo of Prescient a while ago and you were literally showing me how you were predicting. You were able to dictate where you could put, where you could allocate, spend and calculate it down to the number of streams that it would output or number of music sales it would output. And that was kind of the catalyst for building this. Could you just talk a little bit about that and how that worked and also like how you even go about calculating that stuff.
- [06:38]
Mike True
And I think it's. I'm going to layer on, like, when you think about an mmm, the questions you want to ask. Whether it's us or other MMM vendors is one of the big things when you go to a label exec and they say, well, how do you get them confident in your form of measurement. How do you say hey, they knew that there was a song by Cardi B is a very derogatory song but it blew up on TikTok where little kids were going to record their parents reaction when they heard this song and it went absolutely viral. The label execs knew that there was a halo effect from that organic onto these streams and we were able to start quantifying that. But the first question was how do you get somebody confident in your measurement? And what you do with an MMM is you do a series of back testing. So we got Cardi B, Roddy Rich, all these songs. We got all of their old streams by day by platform and then we had all of their spend by day by platform. So you can think about this, your Shopify to Spotify and then you have the same similar ad channels and then we had some of like events that took place. Think of those as like a tent pole moment, like a PR thing which a lot of brands might be familiar with. And we would train our models on that and then we would see could we predict how her songs were going to perform in the past they've already happened, the songs happened. So we knew the actuals and we train up on everything until that song and then we just use the spend the seasonality to see if we could predict how that song would go. We're coming 93 to 98% accurate consistency with consistently with these artists. So mmm, historical measurement. Right. That's how you can initially think of how we started in music and then how that translates over today. The next question we said hey Mr. And Mrs. Label execs. If we had that same budget for these songs and our model had spent it, here's how many songs we think incrementally you could have gotten compared to how many songs you achieved in the past historically. So yeah, I remember was February of 2021. We were watching Cardi B's up song come out and I'll never forget that we called the GM of Atlantic Records. There was no tech platform available. Was just send us data, we'd run it through our models, we'd give them a report, we type it out and we said hey, if you shift From Spotify to YouTube and Facebook to Snapchat, here's how many incremental streams you're going to get over the next 30 days. And they took our spend recommendation and we predicted it at 96.3% accurate. And that was really that aha moment of wow, this worked in an industry with no last click data. A lot of noise with this level of accuracy, hey, it makes sense for us to go explore turning this into a software platform where marketers can log in and get those insights in a consumable way.
- [09:06]
Nick Sharma
And then as you shifted this from the music industry to the world of consumer brands, how did you then, you know, what did you change? And also, how do you, like, I think the thing that I think about, and a lot of people probably are wondering is like, how, how can you predict, like, what data sets can you use to predict the future with such uncertainty? And how do you do that with your mmm? And then we can get into the specifics.
- [09:31]
Mike True
Yeah, yeah, of course. So your spend does not drive 100% of your revenue, the seasonality, this promotional period. So just word of mouth. There's just truly organic things that take place where, you know, I cut my knee, I need to go buy a band aid because I cut my knee. That had nothing really. Maybe I had exposure to, you know, a brand that was associated to that. So mmms are using historical data to try to understand and quantify the impact of how much seasonality drives for your business. Right. How much word of mouth, like, what is truly organic? So if Your brand made $10 million last year, we're going to look at your historical data and say, all right, 65% of your revenue was generated from your spend. Right. And that spend is going to fight for credit. Right. So the inputs that you have from there, you have to see some sort of spend by day. And then we look at impressions. What's unique to us is we look at actually GA session level conversions. So like paid, organic and direct. And then we look at the revenue and we're looking at this from your Shopify, from your Amazon, from your retail, from your GA or GA4, and then all of your different ad channels that could range from connected TV to podcast to meta and Google. And things can happen that you can't predict and the models will be wrong. Right. You can't predict if Covid is going to happen, you can't predict if tariffs are going to take place. But what you can do with MMMs and I can talk more about what's specific to ours is the models like to see change. And so when something changes, it's going to want to try to know something else happened. But if you tell the model what happened now, you can start to like, quantify the impact of COVID quantify the impact of tariffs alongside the impact of all of your paid media spread totally.
- [11:10]
Nick Sharma
Before we dive into the prescient differences. Can you speak to the difference between, you know, as if you're talking to my mom here, the difference between mta. Mmm, incrementality and even just looking at.
- [11:22]
Mike True
In platform data, MTA would be, I want to know deterministically, right? Because an MTA is going to say this person based off an IP address did this, this and this. And I'm very sure of it, right? It's indisputable. This is exactly what happened based off of clicks that took place across our different ad campaigns. Now what that's great for is for channels that are very click based channels, bottom of funnel channels, but it's not very good for something like linear TV or connected TV or podcast or YouTube or TikTok where they're more view based channels. Now you could think about an incrementality test is hey, if I'm running five channels and I shut everything off and see what the impact of, of that one channel is or vice versa, right? You're just kind of like shutting off off spend. You're looking at like, hey, what was that incremental impact? Because the other things are no longer in the equation anymore. And what incrementality is great for. It says, hey, at this point in time this channel is incremental, right? But it doesn't tell you what to go do next and you have to go run it again and recalibrate and run it again. And MMM historically would run once a year, maybe now twice a year, quarterly or maybe monthly, sometimes weekly. MMMs are statistical models that are using historical data, right. To try to find, hey, what is that sweet spot of how much you should be spending and then forecast out what does that predicted impact if you make those changes now, they're all unified to work together. And it's funny, my favorite quote is, it's George Box. It says all models are wrong, but some are useful, right? I am a firm believer and I'll stand by this. Since coming into E commerce, people will say, well this is my source of truth. And I'm like, no, there's no source of truth. The source of truth is you. There's a variety of forms of measurement that are designed to tell you different things for brands that have different marketing mixes. You know, I love these, love all the pure play MTA players out there. We'll send them three to five deals a week that if the brand is just comes to us in the Shopify and Meta and Google, they do not need an MMM at this time and they should go dial in that with an mta and then as they're going to start going into upper, more upper funnel channels, they're going to go into Amazon or retail. Then this is when you're going to want to start considering an MMM and making sure you're asking the right questions about the validation of the MMM incrementalities. Hey, put the holdout data inside of mmms is a big thing you're hearing a lot of people talk about now. So can you explain that?
- [13:46]
Nick Sharma
So taking incrementality data and putting it into mmm, can you explain what that means and also like an example of that.
- [13:52]
Mike True
Yeah, so I was just looking at a shout out to. It's a great incrementality provider platform called House. They are a do a lot of incrementality holdout results and we share a ton of mutual clients and you know, when they were looking at YouTube for example, they're showing what is that cost per incremental acquisition? And they show that from those holdout results. And the brands say, hey listen, like I'm really confident that this holdout is accurate and I want to put in, you know, this measurement into your model. So now our model believes. Right. We call them priors. Think of priors as a prior belief. Our model is using the measurement from that holdout test. We have other clients that say, hey, I want to put in my post purchase survey results into your mmm. We have some clients that have this is a real big mind blown from here. But we have other clients that are running Google, Google's Meridian, which is an open source mmm. I suggest the folks out there take a take a look at that. There's Robin, which is an open source MMM and they're taking results from other MMMs and putting it into our models as well. And so now these marketers have the flexibility to use, we have the most reliable forecasting.
- [15:12]
Nick Sharma
So you're using first and second or third data party sets in addition to like everything you gather from a brand and even stuff they try to upload like post purchase survey results or retail sales. And you're basically creating a very accurate model based on all the data that you're sourcing together. Is that right?
- [15:32]
Mike True
Yeah, we're truly unbiased in a sense, like because MMMs are designed to say, go do this next. Right, Right. If you do this, if you shift from YouTube over to AppLovin and Meta over to CTV, we predict that you're going to get 2,000 incremental new customers and your cap will go down by 20% on Amazon. Right. It's a great use case that people use our platform for, but there's some sort of measurement that is telling them how much they should be increasing or decreasing the spend. You can use our halo effect measurement. You're going to get your own MMM measurement from us. But if you have results from post purchase surveys, this is, it's a three, two row, as we saw. It's a, we're saying it's a 2, 8 and incrementality saying it's a 3, 4. And you want to use the 3, 4, by all means use the 3, 4. If you want to use the post purchase survey, use the post purchase survey. And our models are really focused on putting the power and the decisions back into the hand of the marketer and then using us for that Forecasting and optimization. Agnostic to any form of measurement.
- [16:41]
Nick Sharma
I've always gotten the question of what data set do you operate off of? And having bought media like so much media online, I feel like internally I get almost like a gut feeling when you read the data and you sort of intuitively know what to do. But how do you explain what, how do you explain that answer to somebody, you know, a founder who's listening and they're like, well, I'm going to trust my marketing person with this. You know, do they have the gut feeling for that?
- [17:08]
Mike True
Are you saying that you should be trusting intuition over data?
- [17:11]
Nick Sharma
No, definitely not intuition over data. But to your point of like, you know, if you have three different reads and you're trying to figure out which one to optimize toward. Right. Whether it's the 2.8, the 3.2 or the 3.4, usually if there's no one source of truth, you're sort of using the data to calibrate where you should go toward. But for me, that's always been somewhat of a gut feeling or intuition. Kind of like your own data layered on top of the data you're seeing.
- [17:37]
Mike True
Yeah, it's a great question. We have a feature coming out within our model that's actually going to recommend and tell them transparently which form of measurement would be best suited for the model. Right. And so they can use their kind of intuition. But if you have three different forms of measurement, well, help me figure out what is that best one that should be used. And so more, more to come on that. But it's a very strategic priority of ours to be transparent instead of just assuming that, you know, one form of measurement is best for. Mmm. Right. Could be other forms. And well, we're going to help them transparently guide them towards which form of measurement they should use. It's a big black box today in that sense and the industry is going that direction.
- [18:24]
Nick Sharma
You know, like one thing I was thinking about yesterday was with the concept of AI agents getting more and more mainstream, like I think right now we're about to, in the next three, four months we're going to see a bunch of SaaS, well known SaaS, companies release agent use cases or allow you to use agents that they've created. And then I think the six months after that there's going to be a whole world of open source agents. And then the six months after that it's going to be a marketplace of agents. And you know, one of the things that about these agents is, is just like your AI, it keeps getting stronger with more sets of data and more access to data and more context. Do you think that as, as AI becomes more normalized to use in decision making and pattern recognition, that a lot of these platforms that have walled gardens around their data are going to start allowing you to export data more freely? So that prescient cannot only bring in all the truly unfiltered data from all the platforms to give better recommendations, but that might include just a bunch of unfiltered data that today you just can't get.
- [19:34]
Mike True
I think a great example of that is retail. Traditionally we talk to brands, I'm sure, especially ones that are not owned and operated where they own their own brick and mortar store. Like we have brands that sell in Costco and Whole Foods and then like gnc. I mean the list goes on and on. Target, Walmart and Beauty is a tough one with like Sephora and Ulta Beauty and like. So how often do you get the data from them? Right. Are you getting a generalized national report? Are you starting to get DMA level? At what frequency does it come in? And I'm starting to see even the retail media networks now as well. People are starting to open up their data because they understand the importance of having it integrated into different AI platforms and models. And it's almost a necessity at this point. Right. Because you, it's, it creates a level of stickiness for retail providers. You know, the more data they're going to give you, the more better insights you can get. Ideally the more products are going to be sold throughout these locations. And so I think you're going to see a democratization of first party, third party data based off of what you just described on the AI kind of forefront. Yeah, that's spot on by the way I couldn't agree with you more.
- [20:45]
A few years ago without Prescient I would have said there's no real way to track what channels like YouTube TV or even newcomers like Applovin are doing. But with Prescient you can now look at daily omnichannel forecasting and optimize your paid media without any bias. Today, brands like Port and Leather, Saatva, Jones Road, Hexclad and so many more of our clients use Prescient to understand what these top of funnel channels actually do to impact their sales in retail. Amazon and their DTC site Last Click Channels have taken too much credit for far too long and reduce confidence in upper funnel channels. Prescient allows you to see the halo effect that is usually unrecognized with top of funnel channels and allows you to measure those down to the campaign level. See the impact of Upper funnel on your revenue roas and new customers today by going to prescientai.com limited to learn more. That's prescientai.com limited.
- [21:40]
Nick Sharma
All right, next thing I want to jump into is how are, how are brands using Prescient today like or just mmm in general in modern day applications? You know, if I'm a, you know, if I'm a brand it sounds like if I'm a brand that's just really spending on meta, Google, maybe a little bit of TikTok and driving sales on Shopify, I probably don't need it yet. But if I'm now adding in, you know, CTV, AppLovin, maybe some podcast advertising, I'm starting to do some pop ups, PR events, you know, tell me a little bit about like how does, how does MMM work if I'm a brand doing 30 million versus you know, a brand doing 400 million.
- [22:21]
Mike True
Yeah, I think some I'm going to give 2Q, I'm going to give the halo effect use case and I'm going to give the budget optimization use case. Brands that are on Shopify and say hey we need to go diversify our revenue mix. And so they move into a marketplace like Amazon and just like with, with Last Click and Search, you know, Amazon tends to have inflated credit because it's not kind of seeing what's pushing people over there. And so they use Prescient for which unique, again this is unique to us. Our models go down to an individual campaign. Right. Which for an MMM is very statistically it's challenging. So and our models will run every single day and so they'll use our platform to look at hey, how is this YouTube campaign that we just turned on, you know, a couple weeks ago impacting sales on Amaz, Amazon or new customers on Amazon for brands that have a subscription. They don't care about Roas but they still want to know that new customer count. And so given them that holistic view to understand, hey, the work that we're doing on our paid social, there is a halo effect over there and people know that there is now they can stand by, they can justify it, they can go have meaningful conversations with their finance teams, agencies, with their clients and working closely with us on those strategies. So halo effects is going to be a big one. The same thing for retail, right? I'm going to give a shout out to a quick aside for anybody on Shopify. Pos please reach out to me. We're doing some design partner stuff. We're actually able to quantify these halo effects of, you know, YouTube or linear or TV or meta on your retail sales. And this model's been worked for about eight months now. It's, it's with some of our early design partners. It's been in a very effective insight for them to have more holistic, a more holistic perspective and opinion on how their made paid media spend is driving revenue across all channels. So that's use case number one, what happened yesterday, what happened in the past using these halo effects. There's been a lot of buzz around tariffs right now actually if any of our clients are listening to this, you'll see an email coming out from me tonight talking about these tariffs. There was some news yesterday on coming from that some of the talks between the US and China on tariffs. It's encouraging, right? Is it solidified? You know, it's still to be determined in a lot of our clients they're doing a lot of scenario planning and modeling of like give me the worst case scenario of these tariffs and what are some of the first things to go is people cut down on some of their media budget and we're seeing it a lot, right? We are engaging with, with our clients where we have an optimization tool where you can create dynamic list of certain campaigns based off of performance. And so hey, I have a $250 CAC target, right? We're going to have a list of all those campaigns that are left less than a $250 cac and we need to cut down our media budgets by 25% and so they're using us to reduce their media spend on the selected channels. And then we're going to come back and tell them, hey, here's how you should reallocate those budgets. So here's the, here's the best we think you can do based off of a reduction in your spend has been a, a very relevant topic of how to utilize our platform. In some cases we have brands that are saying I have an additional half a million dollars. And you sit down and you talk to the finance team, you talk to your channel managers and, and your agency and everybody put a plan together and say, well, how should we spend this additional $500,000? Well, what do you want to do? Do you trying to optimize for CAC on Amazon? Are you trying to drive new customers through the dot com? You're trying to go pure top line revenue growth, right? And, and then just simply in 20 to 40 seconds selecting those campaigns, increasing the budget by 25, 5,500 thousand, clicking a button and you know, 40 seconds later you're going to have a predicted media plan where you can look at saturation. We are currently spending here. Here's what you should spend here to give some of that validation and justification of why we think you should change that spend alongside we know the seasonality of your business, we know what that impact should be. And one quick aside, I know you didn't ask this question, but I have to share something that I have been seeing a lot from, from this data is there's a lot of wasted spend that takes place around holiday seasons. And you see people trying to chase that revenue or row as like right during that tent pole moment. It's ineffective. It's largely in a sense like driving down the highway, opening up your sunroof and just letting money fly out the top of the window. There's a relationship between a ratio between your top of funnel and your bottom of funnel. And then there's a time component to that, right? So you think of ratio, top of funnel, bottom of funnel, time component that's aware of upcoming seasonal moments, right? And then you have the third kind of piece is like these saturation plots, diminishing returns. You spend $2 and make 10. You can't expect to spend $2 million and make $10 million. Eventually it's going to saturate. The optimizer knows that you need to start spending 45 days in advance of this holiday moment to make sure that ratio between your top of funnel and bottom of funnel is at that right element. Right? Because the bottom funnel is going to do its work just based off of this seasonal moment. And so that's another I would say big use case of Media planning.
- [27:38]
Nick Sharma
Yeah, that's. That is actually a really interesting point and I think a really applicable use case here. So in that specific example, what is that onboarding process to integrate Prescience so that you can get to a point where you can see all of that data and predictability inside the platform?
- [27:59]
Mike True
It's usually anywhere from like 12 to 12 to 16 minutes. I had some early people time.
- [28:03]
Nick Sharma
That's it.
- [28:04]
Mike True
Yeah.
- [28:05]
Nick Sharma
And so what all are you connecting?
- [28:07]
Mike True
You connect your, your Shopify, we have, we can connect into other e commerce platforms like Salesforce, Commerce Cloud, your GA and then all of your ad channels. So anything from Tatari to Mountain to Keens to Neon Pixel to you know, tenuity on the TV provider side, the trade desk, pod scribe. So into Podscribe, your Meta, Google, TikTok, Reddit, everything under the sun is really point and click. Your Amazon Seller Central 3P we can ingest and then Amazon Vendor Central 1P we can also stand up as well all of your Amazon ads.
- [28:41]
Nick Sharma
How do you, how do you account for channels like out of home or direct mail?
- [28:45]
Mike True
Good question. We have a little data schema. We call it like a Google Sheets. Right. And it's very simple. They just kind of campaign. We just ingest via Google Sheets.
- [28:55]
Nick Sharma
I see, so you can basically ingest everything to your point earlier. You can either present a couple different options of models or Prescient will create its own model for that brand specifically. Right. And then within what, a week all your data is populated and ready to go.
- [29:12]
Mike True
Yeah. So there's no great question. We talk about this all the time. There are no humans that are going to tinker with this model on your behalf. It's all learned from your historical data. We're not sharing other brands data with you. We have a philosophy that, you know, the research that we've conducted is we should have our core belief of how your business generates revenue based off of the data that we have available associated to that.
- [29:38]
Nick Sharma
Yeah.
- [29:39]
Mike True
And yeah, I mean it's, it really is like 20 minutes to get stood up and running early on in the days. I'm like, take a stopwatch, time it and if it's below, above 20 minutes, like I'll buy you a steak or something like that. And yeah, use that on like our seed round investor deck. It was like, dude, no way. This is 11 minutes to get everything connected. And then the models train really fast for retail and there's some, you know, maybe some external data points you want us to have. You know, there is a little bit of legwork where you're gonna have to send us an external export or upload it in some fashion. But 99% of the onboardings are point and click.
- [30:14]
Nick Sharma
What is the product feature that you think blows people's mind the most?
- [30:19]
Mike True
Oh, it's a good one. I think it's our optimizer for sure. Well, you know what it's like when we get. I was on the phone this morning with a private equity fund that invests into a lot of consumer CPG companies. And I was walking through and it's like they sit together when these board meetings and they're like, I know that there is a halo effect happening here. And I'm used to an mmm that would run once a quarter and it's like too much time has already passed. So it's the speed in which our model can recalibrate, which is every single day. The granularity that it goes down to an individual campaign almost feels like an MTA in a sense. Right? Because it's running daily, it's getting very granular. But it's having these very powerful machine learning models that are giving more of a holistic view view. I'd say that's one side then the optimizer for people that are like, hey, I'm sitting here in a team or trying to figure out like a lot of intuition and gut instincts, but we have a big channel mix and I really just want to take some big swings. Right. I remember at Shop Talk, I was chatting with a, you know, one of our clients, publicly traded companies, like, dude, I want to go back. We have a board meeting, I want to take some big swings. I get 5 to 20% that, like, depending on what the data says, I want to swing it. And so for people that can go in and click buttons and set up different scenario modelings and press it in 20 to 40 seconds, you're going to get a. A recommended media plan. They say, I don't like that. I'm going to switch it a little bit, click another button. And you could do it unlimited times at no additional cost to the client. Where, you know, we. Computational cost, we figured out to do it a cheap way. So speed, granularity, flexibility and accuracy, I think are the things that our people are most excited about our platform or find value in our platform.
- [31:57]
Nick Sharma
Yeah, okay. Speaking of, I'm curious, just as a observer of somebody who's in, you know, you have the admin log into prescient, what are some of the most interesting underrated channels, tactics, strategies you're seeing brands do that maybe Even you thought were going to be just a crazy, you know, this is not going to work. But then you know it runs and you're just like wow, this was an amazing move.
- [32:23]
Mike True
I was not too sure about Applovin. There's a lot of buzz around it and we've come out with our second AppLovin benchmark report. We've seen on average brand spend hover between that 5 to 7% of incremental budget. It wasn't taken away from somewhere else. It was, you know, additional budget on top of that. Another one is YouTube. I always believe that YouTube was a strong performer.
- [32:43]
Nick Sharma
Yeah but it's always, it's always been a view through channel so it was really hard to, to measure.
- [32:49]
Mike True
Yeah, you know it was. Who was it from? Cody. Cody from Jones Road. He did a holdout test on YouTube, found out it was incremental, used our optimizer to scale the YouTube and then ran another test after that. And like these, it all matched up within like a certain percentage range being like wow, this channel really checks out and I love that textbook play. It was like validated, it was incremental. Use an mmm for what it was designed to do to scale it and then validate it again. But YouTube has been a, a great view based channel. I think we just analyzed about 2,000 YouTube campaigns over the last year now. So we have a pretty decent sample size of, of the spend. Granted you have to have good content, you have to have like for sure strategy, all that stuff. We're not a silver bullet. It's like oh spend on YouTube. With our model we're going to make a gazillion dollars like but for the people that have YouTube dialed in in the right strategy, the data is showing it's a very reliable channel depending on the brand. Linear TV has been, has been an interesting one for some beauty brands in particular.
- [33:46]
Nick Sharma
And, and when you see these things like this beauty brand for example, are you seeing that halo effect or the, the results online on Amazon when it's an omnichannel brand?
- [33:57]
Mike True
Yeah, I looked, we just, we just did a retail analysis for a beauty brand. 47% of their revenue over the last year came from their media spend on retail. We back tested on this one. This was insane. We back tested, we call it a map. But think about it inversely our map was like 6.4% meaning we were off predicting daily revenue for this retail brand at about 93 and a half percent accurate out 90 days. Right. So again like how do you feel? Confident and hey, 47 of your, your media budget drove of retail. Well I feel confident because these guys are so dialed in with a high level of accuracy on these back testing methodologies and explanations of well we're going to tell them trend. It's like just because the brand exists and they didn't spend any money, this is how much revenue you're going to make. Then you layer on things like seasonality and word of mouth and truly organic. Right. So we decompose all that down. But that would be, that was a very interesting insight is a good chunk of their revenue is coming from on retail is coming from their media spend. And I got a message from their, their one of their chief digital officers and was like we can now sit down in our next board meeting and just have a very thoughtful conversation and we're going to change our strategies. I love to hear insights like that.
- [35:19]
Nick Sharma
Yeah, yeah this, this data set from Prescient really allows you to make big swings pretty confidently. And you know especially in those like rooms with people who are wearing suits they don't understand anything about the you know, they don't understand the day to day of Facebook ads or managing spend. But when you show them this type of reporting it helps so much getting the point across the table 100% man.
- [35:46]
Mike True
There's another pretty. Should have landed this one on but I'm going to backtrack for a quick second. You know when I was talking to the private equity guy he was like hey, we wait three months or six months and too much time has already passed. Something that we've just recently released is a as we call it a scenario tracking ability. So for example I'm gonna if it's cool, I'm gonna walk you through the work the walk the workflow. So hey, we want to keep our budget the same and we want to scale campaigns that have less than a $250 cac. So you're going to select those campaigns and you're going to create optimization scenario. Okay. You lock in on that optimization scenario in the platform. You can say accept or reject. Reject. I accept this change. Tell us the date you're going to implement it. I reject this one for whatever reason. Tell us so our models can learn and we're going to see when you make those changes to your media budgets. Now this scenario is based on a forecast over the next 28 days when you start spending and we start to see conversion, we're seeing performance of each one of these campaigns. We're going to tell you at some point in the middle of that flight if you're which campaigns are on track to hit that forecast. It's a very unique way of thinking about an MMM where you don't have to wait even till at the end of the flight to finish. Tell you in the middle of the flight, it's not going to work 100% every time. @ least you now have a heads up of like, which ones are working, which ones you should consider reaching, reshuffling around a little bit. So I thought that was an important one to highlight, just in the way people feel about MMMs, because they're so programmed to say, Nielsen runs once a year, a million dollars PDF report. Now it's like, wait, like we can run these so fast and you can tell me, like, if I'm halfway through there, it's like I want to, I want to lose weight, right? It's like, hey, if you keep doing what you're doing or if you pull up or down these levers, like you're going to, you're going to hit this, this weight target faster is essential until you get there and step on the scale 30 days later and see if you actually hit that target weight or not.
- [37:39]
Nick Sharma
Yeah, totally. What? Okay, so future of attribution to close us out here, where do you think the future of attribution is? You know, there's all of the. There's a lot more growing concerns around privacy, so there's a lot less trackability. You know, tell us a little bit about where you think it's going.
- [37:57]
Mike True
I think you're going to continue to see the consolidation of different forms of measurement together, like you're seeing with MMM plus incrementality. And how do you start layering in creative level analytics, but generative AI, like, you know, people are at motion and they're doing all these really interesting things on the creative side. So I just think it's going to be more of a universal media measurement powered by Autumn Augmented. It's going to sound word salad or whatever, but augmented automation, like, not. There's not. Everything's not going to get automated. Right. Like big portions of media buying will get automated, but, like, I think there's going to be some augmentation of human, you know, interactions, if you will.
- [38:40]
Nick Sharma
Yeah, almost like checkpoints in the process.
- [38:42]
Mike True
Correct? Yeah. Curated insight, super simple, clean and easy. Like I've always envisioned this, like newspaper. Like, you wake up and it's like, here's what's going on. Like, you know, just kind of like very conversational in that way.
- [38:54]
Nick Sharma
Right.
- [38:55]
Mike True
You know, it's like, hey, I want to go buy. Talking with your wife. I want to go buy a house. And, you know, soon. And then you wake up, you see the newspaper, it's like, oh, the Fed's gonna hike the interest rate in 30 days. Like, okay, we should probably go look at a house sooner than that. Right. So we can take advantage of. That's sort of like thinking. I'm think where AI will go. Here's what to go. Here's what's happened, here's what you should do, and here's what we think is going to happen if you do it.
- [39:16]
Nick Sharma
Yeah. Fully agreed. All right, Mike, anything we missed that you want to cover, I would highlight.
- [39:21]
Mike True
You know, we are into, you know, retail. We are into different verticals. So like gambling sites for any of the audiences that might not be in The E commerce. D2C in the case, airlines, hotels, you know, financial services platforms. The models are now diversified to support different industries. And so. Yeah.
- [39:38]
Nick Sharma
And how much media do you track every month? Because this number is crazy.
- [39:43]
Mike True
North of $200 million a month.
- [39:46]
Nick Sharma
Wow, that's amazing. And if people want to learn more about prescient, if they want to talk to you, they want to ask you a question about. Mmm. How do they find you? How do they find pressiant?
- [39:56]
Mike True
Prescient AI.com I'm on LinkedIn, Michael. True. And then my email is mike prescient AI.com I like meeting new people and talking. As you can tell from this conversation, I have no. I don't have any shortage for words. So give us a shout. Even if you're just curious about mmm or you want to learn about the type of questions you should ask when doing MMM valuation. No two models are the same and the math matters.
- [40:22]
Nick Sharma
Amazing. Well, thanks for coming on, Mike.
- [40:24]
Mike True
Appreciate having me on. It's good to see you, bud.
- [40:25]
Nick Sharma
You too.
- [40:27]
Mike True
Thanks for listening.
- [40:28]
Nick Sharma
We'll be back.
- [40:29]
Mike True
Next time to cut through the noise on CPG retail and E commerce. If you enjoyed this episode, why not share it with a friend? And be sure to subscribe wherever you listen so you don't miss the next one.
- [40:43]
Sam.