← Marketecture: Get Smart. Fast.
Marketecture: Get Smart. Fast.
Episode 183: Ask Olivia Kory Whether AppLovin Ads Work & She Explains Attribution vs Incrementality
00:52:47
Loading summary
Transcript128 lines
- [00:00]
A
Here at marketecture we know what you need more of in your life. AdTech Podcasts so I'm thrilled to tell you about the MediaOcean Podcast Network. Three different shows currently live and a bunch more launching soon. There's Femme Forward, a show about women in leadership, Add it up, which is basically AdTech101 and FRAP rap with G Dog, which defies all descriptors. Check them out@mediaocean.com podcastnetwork that's mediaocean.com podcastnetwork network available on Apple, Spotify and YouTube. And thank you MediaOcean for supporting Market. This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50 page restoration block. Or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it, ready to make anything online make sense. There's no place like Chrome. Check responses, setup required, compatibility and availability. Veris 18. Welcome to the marketitecture podcast. This is Ari Paparo. I'm joined by Eric Franchi and our guest today, Olivia Corey, a second time guest. She is the chief marketing strategy officer at Houzz Haus is a company that does incrementality testing. She has just a ton of insights and knowledge. Effectively the most important question we have which is do the ads work? Eric, what sort of questions do you want to hear from Olivia about?
- [01:41]
B
I mean I want to know if ads work.
- [01:44]
A
Do they work? They must work. They must work, right?
- [01:47]
B
Has our life worked? But Elijah, Just kidding Olivia. I mean she is so prolific and you know she has so many interesting nuggets she puts out there. So a just having a being able to talk to her and not just see a kind of view of the tweets. But then also she's done a lot of work on Applovin and I'd love to dig into that one. Like Applovin, does it work? If so, you know, theories on why it works and like why is this thing just exploded over the course of the past year or so and then yeah, let's get into AI and all that other good stuff.
- [02:17]
A
You probably didn't see it. Vive was in the news again this morning. The deal hasn't closed yet. I think it's moving along well. From all I know they were in the Wall Street Journal's daily morning email. I forget what it's called like CMO Daily or something because they have a new kind of controversial, somewhat hysterical commercial for themselves. And they have this commercial where they have the CMO and the CFO falling in love. Like kind of a Sexy commercial where CMOs and CFOs start getting the. Getting the connection with each other. And it was featured as the main article in the Wall Street Journal today. So kind of another, another quiver in the belt of like good marketing matters for ad tech, B2B companies.
- [03:01]
B
So cool.
- [03:02]
A
So one follow up is last week we launched our MCP server from maddb and people reacted. We have a whole bunch of signups. Mad DB is kind of our little stealth project, but a whole bunch of people signed up, people are using it. If you have any feedback, hit me by email. I'd love to hear how you're using it, what you're getting out of it, because it's definitely in the realm of experimental products we're building and the more people use it, the better it'll get. So please let me know. And if you're new to this, you can access our MCB server, just go to MAD DB AI and then register and then it should be pretty straightforward from there. With that, let's. Let's do the. Do the thing. Let's have Olivia Corey, the chief marketing and strategy officer of House, to talk about whether the ads actually work. All right, welcome, Olivia. The Queen of attribution. The caffeine and caffeine.
- [03:59]
C
No, it doesn't. Incrementality.
- [04:02]
A
My bad. The former queen. It's like one of those British titles like the Duchess of Attribution and the Queen of Incrementality or something along those lines. Show the audience your caffeine situation here.
- [04:14]
C
Yeah, well, real quick, Meta's new ad product for optimization is called Incremental Attribution and somebody wrote that's like an oxymoron.
- [04:25]
A
So.
- [04:25]
C
Yes. I was just telling you before we hit record that there was some research that published this week saying up to five or six cups of coffee is actually good for you. And so I used to start my morning with a venti, four shots of espresso, and now I feel like that article gave me permission to add another coffee on. So I was saying I'm. I'm testing the limits of my caffeine intake, so we'll see how this goes. It might. It might be messy.
- [04:49]
A
How do you feel about microwaving coffee if it gets a little cool?
- [04:52]
C
I'm fine with it.
- [04:54]
A
Yeah, I'm kind of fine with it too.
- [04:56]
C
I'm not in terms of taste buds. Like, I don't really you know, I don't drink fine wine, so I don't care.
- [05:04]
A
So. So, I mean, your. Your. Your career is about testing stuff or so. Or do you live your personal life by, like, what studies say are you, like, optimizing against, you know, all the academic research?
- [05:15]
C
I feel like a lot. My husband is. He's, like, life maxing, you know, reading all the Huberman books and the.
- [05:22]
B
The.
- [05:23]
C
You know, the. The Peter Atia. Although I'm not sure we're allowed to talk about him anymore, but. Yeah, so maybe we should cut that.
- [05:29]
A
Wait, what's. What's wrong with Peter Atia? What?
- [05:32]
B
He was. His. His name came up in the. The Epstein files. He was. Had a reputation and an image of being, like, this squeaky clean, you know, kind of like what you would call him, like, biohacker, medical doctor to the. To the stars.
- [05:49]
C
So. So he's. You know, I feel like he's always up on the latest research. Like, we have the Whoop. And I'm. I know I have a running joke with him that it's ruining my life, but it's been fun to look at the data around. Around. Around health. And I think it's been. It's been interesting to see, like, how they've. How these. These companies, like, whoop have, like, gamified. It's the first app I open every single morning.
- [06:10]
B
So how many ounces or cups are in a Venti?
- [06:14]
C
I think there's 300 milligrams of caffeine.
- [06:17]
B
No, it's gotta be more.
- [06:18]
A
Well, you know. You know the whole ridiculous thing about the ounces in a Venti, Right?
- [06:22]
B
Yeah. First of all, it's like 20 ounces.
- [06:24]
A
Yeah.
- [06:24]
B
Which is good and a half.
- [06:26]
A
Right. But 20 is Italian for 20 is venti. But they don't use ounces in Europe, so it's a ridiculous name. It's really be like, Italian for 350 milligrams, but it's not, and that's hard to say. All right? That's not why people are here. They're here to hear Olivia give us her wisdom, her, I think, second time on the show about the latest going on in the big topic of what ads work and what don't. So you teased up an interesting question, and maybe not all of our listeners really knew what you meant. What is the simple difference between attribution and incrementality?
- [07:03]
C
I'm thinking of giving a coffee shop analogy right now. Stay on it. One of the stories our founder Zach likes to tell is, you go to a coffee shop, you Order a cappuccino as you're paying for the cappuccino is a sign for a cappuccino right at the register, you buy it. Did that sign at the register make you buy the cappuccino or are you going to order it anyway? And that is the difference between attribution and incrementality. Attribution is looking at correlations. I clicked on an ad, I saw an ad, and I bought this thing. Incrementality teases cause and effect. I saw an ad and it influenced me to buy this thing. And so that's the simple way to think about it is like almost analogous to randomized control trials in healthcare. Like, we have a placebo, we have a counterfactual to understand what would have happened in the absence of that intervention. And that's how we establish causation or incrementality. And that's why it's so funny that some of these new products from the ad platforms are called incremental attribution, because those two things really, they don't coexist nicely. It's like correlation causation. And so what does that actually mean when you say incremental attribution? So it's been, it's been interesting to work through.
- [08:21]
A
And the inherent challenge is that the coffee shop owner has that sign up whether, whether you wanted cappuccino or not. So unless they have a whole bunch of coffee stores and could start, you know, holding out and not putting the sign up, and some of them, it's really hard to tell. And in a coffee shop, you even have your name written on the cup. So this increment attribution should be easy, but, you know, not so much. Let's talk about, though, what you've been doing because you're a regular your company. When I say you, I mean you and your company are regular publishers of kind of interesting data and studies about the current state of the world. So we have a lot to talk about. Let's start with Applovin. So you've been kind of one of the more vocal, independent voices about whether Applovin really works or not. So does it work?
- [09:07]
C
It does, it does.
- [09:09]
A
We're done. Buy the stock app. Ticker symbol buy.
- [09:13]
C
No stock advice here.
- [09:15]
B
This is not financial advice. Do not listen to us.
- [09:18]
A
It works, though. Okay, who's it work for? How's it work?
- [09:21]
C
You know, I grew up in the DSP world, open web, so I shared a lot of early skepticism of, like when Applovin entered the e commerce market. Is this really different from, you know, all of these other kind of programmatic plays. And so I was like, you know, why, why, why is this getting so much airtime and so much attention? But what I my. And I'll share the data. My hypothesis is, I think what they've done really well is like the effects of Applovin ads are short term. They work immediately. It's like the meta of it all where a lot of these businesses, like, you know, they have these cash conversion cycles, like they want to see immediate ad effects. And so I think that's what Applovin has done. Well, whatever they've done with their machine learning model is like these ads work fast and I think that's what, what's been, what's been able to kind of help them really break into this market because we, I mean we look at so much data and that is not the case with most ad platforms. We see this with YouTube and with CTV with a lot of the social platforms, we see really encouraging results. But it takes time, you know, like I don't even recommend a CTV test under 8 weeks at this point. You know, so that is, I'd say that's been very helpful for them as they've broken into this market is, you know, you know, the kind of growth marketer, performance hacker Persona when they can see their bank account going up immediately, like that's the kind of dopamine hit they need and they want to keep going and keep investing.
- [11:01]
A
So it. Does their platform easily allow for incrementality testing?
- [11:05]
C
Yes, that is one thing that they, they did, they did well out of the gate when we saw they started the E Comm pilot, gosh, what was it now? End of 24. And a lot of our customers were seeing encouraging results in their attribution model and they were like, all right, we need to validate this with incrementality. And Applovin opened it up to third parties right away. There was no. And maybe we'll talk about this later when we talk about affiliates and some of these other channels that are just like really, really hard to measure. They made it very easy to measure, very easy for these, these third parties to come in. And, and so yeah, setting up an incrementality test with Applovin is as easy as it is in any, any like Meta or Google.
- [11:50]
A
Following up on this question of the speed of reaction, does that mean or does that result in certain products being more suited to certain channels, like a high involvement purchase, one that requires a lot of thinking, maybe works better on ctv, whereas One that is impulsive might work better in applovn. Have you seen anything like that?
- [12:10]
C
Yeah, I think that that logic makes sense. I think it plays out in the data. I'd have to look more closely at category level breakdowns in some of the other channels, but we have seen that lower AOV more impulse purchases like apparel, footwear are doing really well in app and on the flip side like subscription products a little bit harder because that's like, you know, a longer term commitment. So.
- [12:34]
A
And would you see that in like TikTok and Instagram Reels and stuff like that?
- [12:37]
C
Also we, we have some category level data on TikTok. Let me pull that up.
- [12:42]
B
I think, I mean it makes intuitive sense. While you're pulling it up, if you think about their core business, right. It's the app downloads. There is no lower consideration product I would argue than an app download. Right? Like you know, flashy, flashy, check it out, you know, download the thing. So this makes intuitive sense. It would be interesting to see if this is, you know, the case for some of these other platforms though.
- [13:03]
A
Yeah, it seems like from my experience TikTok, the, the products being offered are all probably like sub$100 kind of purchases. I don't, I don't think I've seen any like, except for those Chinese companies that, that are trying to sell industrial plastics and stuff like that which are, which sometimes show up buy things on TikTok. No, are you not? I'm not, no.
- [13:25]
C
We actually have this data. So advertisers with an AOV of less than $50 spend three times more on TikTok versus advertisers with AOVs greater than 50.
- [13:36]
A
Yeah, totally makes sense. Let's talk about some of those hard to track channels. So last week we gave you a shout out because we were talking about the little scandal with FIA and how they were taking, allegedly taking affiliate commissions where they shouldn't have been. And it begged the question about whether affiliate is incremental at all. So what do you think? Is affiliate incremental at all?
- [14:03]
C
I wish I had a more satisfying answer here. We've only run a few test tests on this because affiliate is so hard to test, they make it so difficult. Like you can geosegment these affiliate offers and for the audience, the reason I care about geosegmentation, it's the way we get that holdout group, it's the way we get the control where we say turn off ads in part of the country and that's our control group. So geosegmentation is critical to enabling these studies. And so it's not easy in addition to not being able to geosegment, it's just not easy to go in and shut off. If you have an offer with a capital one or you know, whomever, they don't make it easy to just say like we want to pull down this offer for some period of time and so we've done it before. It is something the advertiser, the brand has to really push for. Like you really have need to have the stomach to go to your partner and say we are doing this, please make it happen. And so one interesting way we've studied this is we've, we've had them pull certain product skus. Like we're so we can almost use like the product skew as the control of like pull this product sku from the affiliate program and then we'll look at sales of that product versus sales that were continuing to get promoted. And, and so in the, you know, very few studies that we have we, we didn't see much in terms of incrementality. But that's a you know, sample size of like two. So I wouldn't, I wouldn't, you know, take that to the bank. I think this is, this is the case with RMNs, with Amazon search with affiliate is like, I wonder if they are, they're limiting like their advertising kind of TAM is just limited by not offering incrementality. When you think about those three RMN's, affiliate and Amazon, those are things I think probably belong in the like brand search category of like everyone's wondering about the incrementality. Like it's kind of like a poster child of an incrementality problem. And if they could, I think all of them have are very promising channels. Like if they opened up their channels to more rigorous measurement I think we'd see more growth. Like you see it with, you see it with CTV and all that, you know, so I'm, I'm, you know there's the flip side of the coin is like maybe you don't see incrementality but I think advertisers are holding back dollars just because they are skeptical and they, we haven't had a way to actually get evidence on it.
- [16:35]
A
Well doesn't Walmart introduced an incrementality product right around the new year? Right. So is that not available to third parties? You have to use, you know, their, their tooling for that.
- [16:43]
C
I haven't heard much about Walmart, but that's cool and that doesn't surprise Me knowing what I know about Walmart's like ad tech sophistication. So that's great. I think that's, that's what I would want to see more of.
- [16:55]
A
Yeah, you would want to see more of that. You know, retail media is often, is so skew based that often there's not a lot of tech in the ad serving part of it. So let's talk about some of the mechanics that are going on here. What's how the market's sort of changing. So one of the big changes that's happened really over the last five years has been the growth of C APIs, conversion APIs. And it started out obviously with closed systems like Meta. I don't know who, I don't know who introduced the first API, but like, you know, we could ask Eric Sofer to give us a history of APIs at some point. Slow newsweek. But like, but it feels as though it's gone from like this interesting, interesting new method to almost replacing pixels as the dominant method. Is that, do you agree with that assessment? And what are kind of, what are you seeing? What are the kind of pros and cons of this movement towards the APIs?
- [17:50]
C
I think I posted a tweet once saying find something you love. As much as Eric Soufert cares about capi, I love it. I love the passion so much. Cappy. So I think capi and Pixel sort of do the same thing. It's different technology. Capi, you've seen a rise in capi because it's server side, it's more privacy safe. It feels like the more modern approach. I, I mean, I'm curious if you guys agree. I'm not as close to the, you know, the, the data engineering pipelines as I once was, but I think it's still easier to implement pixels. So like, that might be why like you haven't seen them fully die, is that it's just easier to implement. But, but the, the capi is definitely like the, I think it's, it's better coverage. It's just probably like a more modern approach to the pixel. And then the, I mean all of these ad platforms will tell you that the better signal you feed the system, the more performant your ads. So I think it's, it's less about measurement and more about what is the signal that I'm feeding to the ad platform to help it optimize and like train their machine learning model to go find the right people who are most likely to purchase. And so we've seen a lot of brands Experimenting with like, what to send through the capi of like, whether it's like a predicted LTV metric or some are even experimenting with like sending what they believe to be the most incremental conversions back to say like, hey, only, you know, really use these in your, in your, in your model. But to not send a capi event is to leave a lot of opportunity on the table. You, you still, you still are, are subject to the same issues of. All these platforms are going to claim credit for the same conversions, but we don't really see it as a measurement vehicle as much as it's like an optimization vehicle, if that makes sense.
- [19:50]
A
Yeah, that's an interesting point because you have a choice as a, as an advertiser about what to send in the capi. And in the old world of pixels, like before capi you had sort of a tag manager and then you had a bunch of pixels and you could actually have a lot of business logic in your tag manager that would say, you know, only send this pixel if you know the UTM parameter is whatever and therefore you're limiting who gets the data. But with capi, I think the default is send everything to everybody. Right. Which makes your attribution problem dramatically harder.
- [20:24]
C
Is that accurate to the extent that you trusted platform based attribution to begin with? Like, we don't really see anybody kind of using, using that as their source. Like you know, if you're working with House, you are, um, you know, you're, you're likely, you've kind of likely taken that journey from platform reporting to MTA to mmm to incrementality. And so using platform reporting is any semblance of source of truth is just mostly a thing of the past. Just given the double counting like the, it's the, it's the age old problem of if I add up all these conversions, it's like, you know, three times more conversions than I have in my bank account. So that's the issue.
- [21:07]
A
So they're going to tees up the subject of last week's podcast, which was the billion dollar investment in Apps Flyer from effectively media companies that are reliant on their measurement. You know, Google and Unity and Meta and, and Maloco all invested in them. But it sounds like from the way you're positioning it, Apps Flyer is a form of kind of attribution. Mma. Mmm, mmm. Mta. Sorry, that's what I meant. Multi touch more than an incrementality platform is. So do you have thoughts on, on the, you know, long term sustainability of that type of offering and you know why they invested so much.
- [21:48]
C
Yeah, they, so they do that work of like de duping the all of the, you know, conversions that the platforms might claim credit for. An Apps Flyer does the work of saying, okay, now the, this one actually belongs to Meta, this one belongs to Google. I love this investment, by the way. I was just geeking out over it. I saw you guys had BQ on the podcast. We're big fans of him at House and I haven't listened yet, so I've been looking forward to listening.
- [22:15]
B
Yeah, I like it. I didn't know it was bq. We would have called him that.
- [22:18]
A
Yeah, we didn't know that either.
- [22:19]
C
Oh yeah, yeah, he's bq. I, I, it's mostly secondhand but.
- [22:23]
A
Well, I mean, obviously if House could get a billion dollar investment from Google that you'd probably take it.
- [22:28]
C
I so to answer the question, attribution is still ubiquitous in this and especially in mobile. Everybody has Apps Flyer, you know, everybody we work with, they're using us in addition to Apps Flyer. And, and the reason is you still need a daily signal. You still need a daily kind of granular down to the ad level view of performance. Even if we know that it's biased. What we've seen is that like that bias is systematic to some extent and we can adjust for the bias, but you really need a real time view of what's happening in the business. And it's been something we've been thinking about at House is this data has value, like attribution data certainly has value. This is something we've learned and it's because it's, we talk about growth marketers and they're in these platforms all day long, they're making tweaks like you kind of need to be able to see things down to some level of granularity. And so that's what we're seeing is like you actually have that raw data coming through from an apps flyer and then we can adjust it, we can debias it, if you will, for incrementality so that you can, I mean, imagine like a CFO needing a daily or a weekly report. You can't just be like, no, let's hold off for the incrementality study one more month. You just don't have that kind of time.
- [23:56]
A
So is it like, is it like, I, I'm a marketer and I'm getting my attribution reports and I'm saying, well, my, my ROAS is, you know, whatever, 200% on this channel. Great, let's keep giving the money and then in a medium term over say a 30, 60, 90 day period, I'm getting incrementality results that say actually it's really not 200, it's 150. Right. And that's fine, that's just new information that makes your models better. And then over like a annual period you get an MMM report media mix modeling that says like actually billboards are where it's at and you know, you probably shouldn't, should be spending less money online.
- [24:36]
C
Certainly happening. Our mission at house right now and what we've been heads down on is connecting all three of those so that you can make decisions faster with more data. I think the, the journey we've been on is we started in incrementality, we started in experiments, we now have several thousand experiments in terms of this cross customer database of causal kind of signal to work with. And so what we've done is we've trained a causal attribution model. What we're doing is kind of de biasing the attribution data. So we have a pixel now funny enough, it's just for that raw daily stream really of what's happening down to the ad level. So we're debiasing those estimates with our incrementality tests. If a brand has incrementality tests of their own, great, that's even better. But if not, we kind of cold start them with our database of experiments. And then so that's like for the micro day to day granular decisions that need to be made. And then we've taken that same model to the MMM to say, all right, macro channel level adjustments. We're using an MMM kind of anchored in your experiment data. And the, and this is where I'm going to get very heady and I hope we can get to agentic media buying at some point. That's just a lot, that's like a lot of crap. It's experiments, mmm, mta. It's like even I work here and I'm like, man, that's like a lot to look at in any given day. And so we're kind of of the belief that like humans probably shouldn't be trying to triangulate all this data. It's not a great task for a human. And so that's what we've built our, it's called Architect, but it's an agentic kind of budget allocation tool. We were very honest that like it's not like automating all of the tasks of a media buyer, but the actual budget Allocation question of where do I put my next dollar today to make more money? We think that that can be automated and we think that there's a better way to synthesize all of these sources and like, kind of triangulate than to just, like, go log into each of these at different points in time. So that's the vision. And we have some customers who are starting to take these recommendations from Architect. And there's a, there's a. I would love to get your thoughts on this. There's a trust and safety component of automated buying that you have to overcome. That has been the biggest challenge is how do I, how do I, you know, make. How do I avoid the extra zero or, you know, how do I account for the fact that my CEO says we just need to be on brand search no matter what? I need to see our company, our brand in the first position. So, like a lot of the, kind of, A lot of the things that make it feel risky, we're trying, we're really trying to be thoughtful about that.
- [27:27]
A
There's a pretty big difference between a agentic model that says, hey, spend your incremental dollar on meta or, or, you know, add some parameters to your meta campaigns versus agentic media buying, where you're placing new iOS out of a whole cloth. Where is the comfort level right now
- [27:46]
C
where, where we, I think, have a, I think a unique pov and where our, our customers trust us most is like, they're already working with us in some capacity to make these budget decisions. Like, this is what we do with our incrementality test is like, all right, based on what we're seeing, here's like the recommended next best action. And so what we're doing today is a, it's a kind of a, you know, a view where it says, here are the budget moves you should make. Move this much budget from Google search into YouTube, move this from here to here. That attribution product tells you exactly which campaigns to move into. So that's sort of why we had to build the more granular view. And then it says, do you want us to do this for you based on our API connections into the ad platforms, or do you want to do this yourself? So that's where we're at right now. It's, it's, it's not a lot like, it's, It's a very kind of simple budget allocation question. But I don't, I don't see, you know, it's, I think where some of these other companies may be faltering is like an understanding of, of how these businesses actual actually operate and building that context into the system in a way where it's like, all right, brand search is not moving. Like this is, this is an immovable right object like certain, like line items in the P L that you just, you aren't touching. And so it makes the recommendations make more sense when we bring in that business context. But I, you know, I'm, I'm, I'm very curious what you all are seeing. Like where are, where's the industry at and where does this kind of, you know, line up to what you've been seeing?
- [29:18]
A
Well, I'm unemployed podcaster so I haven't seen anything but, but we talk about, you're the one sitting on a giant mountain of data. But I think it's a good point to put a bow on it. This was a great conversation. We're really going pretty deep here on all these questions about what ads actually work. We have a lot of news this week. Google, OpenAI, etc. So we'll take a quick break and come back with our news of the week. Right now, get up to 15% off
- [29:53]
B
select storage solutions put heavy duty HDX totes to good use.
- [29:57]
A
Protect what's important to you.
- [29:59]
B
The solid impact resistant design prevents cracking and the clear base and sides make items easy to find even when the totes are stacked. Find select shelving and tote storage up to 15% off at the Home Depot. To organize every room in your home from your garage to your attic, visit homedepot.com how doers get more done. All right everybody, we are back with the news of the week. Let's start by talking about Google. So Google kicked off earnings season yesterday with the first announcement out of big tech. Let's talk about it. So revenue, total revenue was 119.8 billion for the quarter of the stuff we care about. Ads 81.6 billion. YouTube 11.1 billion up 13%. Search and other 63.3 billion up 17%. So things are nicely growing. Cloud however, up from 13.6 billion the previous year's Q2 to 24.8 billion. This year's Q2 up 82%. 82%.
- [31:07]
A
Unbelievable.
- [31:08]
B
So we know what the story is. We know where the growth is. The third thing that we, you know, now kind of become a metric. We track Capex estimated to be 195 billion to 205 billion for the year. That's what they're spending on building out AI infrastructure. And the stock took a hit largely because it was the first quarter of negative free cash flow in a very long time. Google's ads are growing. Google's cloud is exploding and they're spending a lot of money. This is an AI sort.
- [31:40]
A
Yeah, I think two things. One is I want to repeat my rule of thumb that I talked about the other day, which is 81 billion in ad revenue and another 11 for YouTube added up, that's around 90. So if you ever want to say, how big is Google? It's about a billion dollars a day in ad revenue. That's the easy, easy, easy cocktail party chatter makes you sound smart. And that's the only thing I care about. Secondly, I saw these numbers, I put them into ChatGPT and I asked it like round numbers, roughly when is Google's cloud business going to be bigger than its advertising business? When will it be a cloud company primarily with ads as a secondary line of business? And it said probably in the early 2000s. Obviously you can't grow at 80% every year but. But the lines are going to cross at some point. Right. So that's kind of an interesting head scratcher. Our children or our grandchildren might think of Google the way we think of IBM as probably not IBM, but primarily a cloud tech company.
- [32:42]
B
Yeah, yeah, absolutely. And I mean I didn't go super deep into the numbers. I guess the assumption is that the network business continues to be flat or declining.
- [32:52]
A
At this point we care so little about the network business that we don't even put it in the show notes and that's our entire worldview. You know, it's like probably another bad quarter for the network business is desperately trying to, you know, at this point what's going to happen is that the employees who maintain the network business are unfortunately at some point are going to pass away from old age and then there'll just be no one left because they're just sitting there waiting for the judge to say something. And like, you know, we all get older and it happens. And I hope they're keeping in good shape because like they're going to be the last ones to know how any of it works.
- [33:28]
B
Olivia, what you got, if anything on this one or Google generally?
- [33:32]
C
We are getting a ton of questions. It's probably one of our next research pieces on the effects of AI on the effectiveness of, of search ads. Like what, what is going on? Do these, you know, how, what is the role of Google search in this new era? And you know, what, what they're seeing, you know, that just anecdotally it's like it doesn't work as well as it used to. I don't have any data on that and I don't know if it's true. It's often, you know, most likely a function of like macro factors and business headwinds. So. But there is a, there is an open question of like, how important is the SERP these days?
- [34:13]
B
Yeah, that's a good point and a great transition. By the way, I don't know if either of you read that New York Times piece on Google AI mode this week. It was, it was good and external validates what you just said. Olivia. I'll pull out a couple of things I saw in the article. So number one for AI mode, people are writing far longer queries. They're on average three times as long as keyword search. So they're like much richer and they're probably having a better result on what happens. People are spending more time on AI mode than search. So AI mode, they're on it between one and nine minutes. Search are kind of in and out. And then There was one third party study that said in 75% of sessions, users never left AI mode for the web. So that's why search might be declining in performance because nobody's going anywhere. Right. And taking an action. So Google said this is flawed. Small sample size, non standard behavior. It all points to AI mode being like fundamentally so different than search mode, you know, and that's what's growing and that's what, what people are doing right now, not just on Google, but everywhere.
- [35:20]
A
Yeah, and I think this follows trends that we've been talking about on this POD for years. Like I think it's going on a year or a year and a half ago. We're starting to say, well, what should happen? Well, we expect fewer, more queries, deeper queries, less clicks, higher CPCs, probably better conversion rates. All these things are happening effectively as expected. And the one question we had a year, year and a half ago was will the market share of search change with new entry and OpenAI? And Google's doing a hell of a job protecting its on the share side of things for sure.
- [35:55]
B
In response, Google said it's sending billions of clicks to the open web. It's continuing to evolve the product, but it's very clear what's happening here. All right, let's keep talking about AI. So Business Insider, shout out to Lara O'Reilly on this one. I thought this one was a neat get by them. They had a post, I think it was just Yesterday, speculating that OpenAI was building an ad network. Apparently they caught wind of a job posting about a role that, you know, was working with publishers and seemingly around building an ad network, asked OpenAI about it and apparently OpenAI quickly deleted a couple of lines in the job description and changed a bunch of others. So maybe there's something here. And it was a very good piece. Basically, the theory is that OpenAI with ChatGPT needs more inventory already. So apparently there's like an inventory bottleneck because the queries are so long, so context rich that advertisers are not able to spend enough money. So it's number one, what do you do? You build an audience extension product. I think that's plausible. Or number two, they theorize that this idea is perhaps like getting ahead of all of these publishers that are very, very mad at them for scraping their content, scraping their data, and actually doing something that gives them money. You didn't think either of these were too, too plausible, right, Art?
- [37:22]
A
Well, my opinion is whatever it is, but you actually tweeted you had a secret opinion you were going to reveal on the podcast. So what? I think we should skip me and go. Let's hear it.
- [37:34]
B
Okay. All right, so here's, here's reason to believe there might be something here. Currently, ChatGPT is a product for free users. I don't know what the percentage of users that have like a paid subscription of any kind versus the free users, but it's only a segment of the users and presumably it might not be the most high value ChatGPT users. Right, right. Makes sense, right?
- [38:00]
A
Yeah, makes sense. High value ones pay.
- [38:01]
B
Yeah, exactly. I looked at the terms of service and I didn't see anything about them being able to use the data from paid users to build and inform an ad product or an ad targeting product. So theory is the most valuable users are the paid users. They're collecting all the data on that. Whoa. What their version of Fan could be, you know, taking the audience data, the user data, and then building a product around that. That could be pretty hot. So basically my idea is they're, they're building fan, but for ChatGPT, if indeed they're building anything.
- [38:43]
A
Yeah. So my, I will give you my opinion, which is the if. And I did a lot of this research when I wrote my book, because it's pretty interesting that, you know, Google was the one that developed a network business and really Facebook Meta has a very small one. It's not very important to it. And the difference is the reason Google did that was because they had very little owned and operated O and O properties. Back in the, in the day before YouTube was a real thing. They had acquired YouTube but it had very little monetization. Monetization. So the, the way to, to take their demand and leverage it was with a network. And what we've seen over the last couple of years is as YouTube got bigger and bigger and bigger, they cared less and less about the network because they have oh and O and they keep 100% or not 100%. They keep a much higher percent of the ads that show there. And so you could say, well, OpenAI is in the same predicament where they don't have any display or video inventory really. So if they want to offer advertisers a complete product that works across mediums and across the funnel, they need to get inventory from elsewhere. But they, it has kind of the same drawbacks of all these networks, which is you're only keeping 60% of it, or I'm sorry, 30% of it, more or less. So it always becomes less interesting than the core business. So maybe, maybe not is kind of where I'm at.
- [40:00]
B
Yeah, I think the valuable user data might be the story here. I don't know. Olivia, what you got any thoughts on this one?
- [40:06]
C
I'm, I'm bummed there's not more skill in the free, in the free tier. We've heard this as well. Some of our brands and partners, customers are testing OpenAI and they've just, they're not able to scale to the levels that we would need to even test it. So I, I, I don't know. I want to, I would love for them to figure out a way to be able to scale this business because I think it's a huge opportunity. It's been an interesting journey. When they first rolled out ads, it felt like they were taking the, the Netflix like very high CPMs, you know, very curated list of early brand advertisers and it felt it was going to be more of like a brand play to some extent. And then it felt like total, total reversal there where they, you know, Eric wrote about them launching a cappy and they were like going bigger on. All right, ad tech, let's open this up, let's make a more self serve platform. And so I'm admittedly just a bit confused on the ad strategy. The ad network itself doesn't feel like the answer. But yeah, I'm bummed because I want advertisers to have more places to spend ad dollars.
- [41:26]
A
Do we think the scale issue is because of the number of users which is very large. Like a billion people use it or is it because of the number of ads, the ad load per query, which I think is pretty low.
- [41:39]
C
Eric, what did you say? How many users are on the free plan?
- [41:42]
B
So it's not a billion users, it's the user that are on the free plan. Right.
- [41:45]
A
So that's most of the billion. I mean you have to expect it to be most of the billion.
- [41:50]
B
Yeah, globally I would imagine in US maybe other high, you know, kind of high GDP countries. I don't know. So it's. What is that number? And then yeah, the queries. That's the thing. It's. These are so, so like kind of data rich and unique and fine tune I think. How do you even organize that to understand like what's available?
- [42:13]
C
Who cares? I would just, just throw ads on there. I don't like, I don't, I don't know how. That's where it's travel. I don't know how performant it has to be. It's just, it's, you know, everything is an network. It's like surface area, you know, for attention. That's why I'm wondering if they've over revved too hard on the performance aspect of this whole thing and they should kind of go back to where they started of like $50 CPMs. This is more of a contextual, you know.
- [42:41]
A
Yeah. I mean the obvious play was just AdWords. Just like reproduce AdWords. Right. But the thing about the Google search product is that when you search for something it's like, you know, there's 12 blue links but 10 of them are ads. And, and that's definitely not the user experience in a chat, in a ch experience. So you know, we'll see. I mean maybe this ties to our conversation last week about how eMarketer lowballed the estimate for how they're doing saying whole market is a billion this year, well below where their forecasts are. I mean maybe we're, we're just, we're reading a lot into this one Business Insider article. But if, but, but Olivia's data point that people are having a hard time delivering is kind of an interesting point that I haven't heard previously.
- [43:27]
B
I know reading into things is what we do here on this podcast. Also on Business Insider. Like I want to give them props. They announced that they're launching a fast channel powered by Vagnite. So I was blown away. Business Insider apparently has 38 million YouTube subs across all of their various channels. Question is, how can. What is the percentage they can convert into fast channel watchers. There's a lot of fast channels. So I asked Claude and Claude said there are 1960 fast channels globally. We probably watch, you know, in our personal lives a couple. And most of these fast channels have at best a 50 fill rate. So it's like, you know, hard to be discovered and then hard to monetize. So maybe Bagnet can help here. But I think this is cool, you know, being able to, you know, to a degree. Right. Control more of your destiny is. Is the way with Fast.
- [44:24]
A
Yeah. I think there's an opportunity to be CNBC for streaming. That's not cnbc. I think that cheddar John Steinberg tried this years ago and he probably would have succeeded if he hadn't taken all the money he got when he sold the thing for hundreds of millions of dollars. But if he hadn't been that successful, he would have been successful the other way. I don't know though, how you get consumers to install your fast channel. You know, I, I talked to an executive at World, which was owned by Applovin, just like to educate myself, and I was like, what's. How, how do people get new fast channels on TVs? And he was like, it's impossible. Like, basically, you know, we need another fast channel. Like, we need a hole in our head. People. Don't you gotta do deals, right?
- [45:07]
B
Yeah, you need to invest. That's how, that's how you do it. It.
- [45:10]
A
Yeah, basically.
- [45:11]
B
Yeah, you got to pay the, pay
- [45:12]
A
the, pay the piper, pay the OEMs, pay the, pay the Roku's of the world. Whatever you have to do.
- [45:18]
B
Yeah, no, exactly. So I don't know. Kind of cool. All right, what are the amp records? Why. Why should we care?
- [45:26]
A
You know, I have podcasts on my mind a lot because I run a podcast. I don't really run it. I'm sort of involved in architecture and we run podcasts. And it's a, it's sort of. Of fascinating in that like, while we're all trying to make measurement better and the ad tech ecosystem better, in my opinion, the podcast world is getting worse. Significantly worse. Because podcast used to be kind of hard to measure, but relatively uniform, that everyone listened to audio and everyone got an RSS feed into their player. And over the last couple of years, the movement to video has broken all of that. So effectively, if you think about the top three platforms, which are Spotify, Apple and YouTube, they all have radically different technologies for supporting video. And this is a big one. None of them really support video. Ads. So while all the podcasts are moving to video, the ads are way behind. They're very hard to insert. They're impossible to measure consistently, and YouTube is kind of its own island. So the world for a podcast publisher or advertising advertiser has actually gotten worse over the last several years. Less measurable, less accountable. While consumers love podcasts and the ads clearly work well, you can ask Olivia to tell me if I'm right or not on that one. So the Amp Accords is an effort by a friend of the pods at. At sounds profitable, which is a kind of a B2B media company. They teamed up with, like, a pretty good list, including Spotify and UTA and some advertisers and DraftKings and some other folks to try to, like, like, throw the gauntlet down and say, we have to fix this. And there's even a video by, like, Galloway and Kara Swisher talking about how that needs to be fixed. So, like, it's a lot of noise. It's a lot of smoke. We'll see if. I can't imagine YouTube moves too far off where they're positioned, but maybe Apple. You never know. So that's what the Amp Accords are, and I wish them good luck.
- [47:21]
B
Olivia, talk to us about the performance and incrementality of audio ads and maybe podcast ads in particular.
- [47:29]
C
Podcast measurement is such a mess. I love this one. I love this, this line, I think. I mean, we see it even at house, that podcasts are a huge source of pipeline. The only way we know that is because we have, like, a How did you hear about us? And we see podcasts, but we don't. No one ever writes, like, the specific podcast. So.
- [47:51]
A
Yeah, I have to interrupt you here. Are you advertising on architecture? Do I have to have Jeremy hunt you down?
- [47:57]
C
Not yet, but I'm sure you're listening.
- [48:01]
A
You live in the same neighborhood, right? As Jeremy. I'll just have him come to your house with a coffee and, like, sell you some podcast ads.
- [48:09]
C
Yes, we, we. I, I. So I. Eric, you said something about how everyone has a podcast now. Do you think that I shouldn't start my own podcast? What do you. What's your. What's your take? Are there. Are there too many?
- [48:21]
A
Well, you're asking the wrong people. Eric's got, like, five, and I'm. I'm a guest. I'm a guest on everyone else's podcast. I would not recommend doing a podcast if you want to make money on advertising, because the. Getting the critical mass that, like, this podcast has kind of the smallest possible critical mass where someone would be interested in advertising on it. Right. And it's taken years, so. But if you just want to, you know, have something interesting to post on LinkedIn, then, then go for it.
- [48:50]
B
Yeah, I think you should.
- [48:52]
C
Our. The insight is that we think we have this open house format where it's, it's sort of confused about its identity. It's like a part webinar. Like, we'll do like live webinars. We'll bring customers on and we'll talk and then we publish it to YouTube after. And so it's like people call it a podcast, but it's not like the format itself doesn't lend itself well to like walking and listening on your way to work or whatever. And so that's part of it. And then number two is we just think there's a ceiling to how much, how many people will listen to a corporate podcast, like, sponsored by house. And so we're like, should I spin off and start an Olivia podcast? But it seems like so much work and anyway, well, I'll get back to the, to the, the point and the real question that you asked.
- [49:36]
A
Yeah, what is the real question? I asked like, oh, do podcasts work?
- [49:41]
C
It is like, so it's, it is.
- [49:45]
A
It's so bad.
- [49:46]
C
You have. It's so bad. There's something needs to be done about this. Like, it is, you can, you can't geotest. So we, our best solution for podcasts is like either dial up podcasts or dial down and we'll do a time test and observe like a before, after. Kind of like, how is, you know, we will forecast what we expect from the business and then we'll look at the actuals versus a forecast. And then some, some brands are doing like dynamic podcast buying, but that's usually not what they're really excited about. They're excited about like the host red. So it's so frustrating because it's like such a big. There's so much belief in this format and we're pretty sure it works, but there's no measurement.
- [50:26]
A
Yeah. And also when media buyers or even a lot of the ad tech companies talk about supporting audio, there's a totally different market which is streaming radio. And streaming radio for the most part does work because it supports vast and sports, it's programmatic, whereas podcasting is very spotty and you can't do host red and etc. Etc. So it's important that you figure out what you're actually buying there. Good luck with the Amp Accords look it up, support it. Get someone from Apple on the phone, see if they'll support it.
- [50:58]
B
I personally think you should launch a pod and put it under your name. Just, I mean, all the stuff that you tweet out there and all the people that you end up engaging with, like, I think there's a podcast in that whole thing, so I think you should totally do it. But my two cents.
- [51:11]
C
Thank you, Eric. I'll need advice on how you. How you manage the time. Like, I'm just nervous it's gonna become a distraction from my. My full time job.
- [51:20]
A
But, yeah, maybe.
- [51:22]
C
Maybe you just need to shoot from the hip and I need to, you know, not worry about preparing.
- [51:26]
A
No, unlike Eric and I, you have a real job, so you may have a. Have a conflict.
- [51:34]
C
Ari's like a legend at house. Like our product people went to his book signing in San Francisco, so I appreciate. You don't need to have a. These days. You've already. You've already. You've already made your name.
- [51:45]
B
That's amazing. And we should end it there on a high note.
- [51:49]
A
All right, there's a great conversation. Olivia, thanks so much for joining us.
- [51:52]
C
Thanks, guys.
- [51:53]
B
See you next week, everybody. Thank you for subscribing to marketecture.
- [51:57]
A
New interviews are added every week at marketecture tv and your favorite podcasting app,