← Always Be Testing
Always Be Testing
#86 Beyond Best Practices: First-Principle Thinking in Performance Marketing | Svet Kazanjiev, Principal, Blue Services
00:38:22
Loading summary
Transcript39 lines
- [00:00]
A
Foreign. Welcome to another episode of the Always Be Testing podcast. I'm your host, Ty degrange, and I'm really excited to talk to Svet today. Svet, how are you, sir?
- [00:17]
B
I'm doing great. Thank you for having me. Excited to chat. There's so much interesting stuff that we can discuss.
- [00:25]
A
Absolutely, absolutely. Svet is amazing. He has achieved a lot in his career, to say the least. East Stanford, full merit scholarship to being at McKinsey. We've been in performance marketing for a similar long length of time, 20 years. What of all the fields and career options, what drew you to the sphere of digital marketing?
- [00:50]
B
Well, there are two things that I absolutely love about online marketing. One is the measurability and objectivity of this field, which is one of the fields where the truth in performance really drive the outcomes. And the other one is that I just really enjoy the cat and mouse game between advertisers, SEO experts, social media experts, and the platforms on which they run their campaigns, obviously staying within the rules and carefully managing risk. And as you said, I've been in online marketing for over 20 years. After Stanford, I was at McKinsey and then I moved to a company called Nextag, which is one of the really kind of granddaddies of the online marketing industry. It was one of the first companies that built a sophisticated, fully managed keyword platform of million keywords across dozens of industries. And it was a comparison shopping company. So it was doing keyword arbitrage in the early 2000s. And as an academic, as an environment, as a work environment, it was very different from Stanford and McKinsey, which are very genteel, very academic in many ways. It was more like a Wall street style pressure cooker where if your boss is not happy with you, they will throw the phone at you. And that, yeah, one of the first things that, you know, really a little bit shocked me was on my first month in the company, after enduring like six hours of interviewing with the CEO to get the job, he goes with me in the elevator and he goes on this absolute rant about how much he hates strategy consultants. I come from McKinsey, right. It was definitely interesting, but we built really mathematically sophisticated, algorithmically driven strategies and we managed to make a lot of money across a variety of industries. So it was a wonderful first experience to learn about online marketing. And one of the things that actually it helped me also, it was my first experience managing teams. And it was interesting because my. One of the big challenges in that company was balancing maintaining this really fast moving, learning, driven culture, while at the same time a little bit providing a shield for younger junior team members from the pressures and kind of from this very aggressive environment that existed. So I learned to a little bit service this buffer and to absorb this pressure for my team. That was.
- [03:50]
A
That's amazing. That's very commendable and I think it's such a great leadership trait. It's not always common, so commend you for that. And it's impressive that you, you learn so much through that. It's. That was an era where there was definitely some wild, exciting growth opportunities in search in particular in that realm. So it's fascinating to hear that you're able to live through that and bring so much of that learning to your experience now.
- [04:20]
B
Thank you. Yeah, it's really. I think that some of the learnings that we got from that time is just relying on first principle thinking, technical depth, rigorous testing and analysis. I love the name of the podcast, by the way. And so Today I help B2B businesses, E commerce companies and services businesses, scale the things that work, debug the problems that areas or the things that don't work particularly well, and build high performance marketing engines and teams.
- [04:57]
A
I love it. It's kind of fascinating to think, you know, you're talking about scaling what works and thinking in terms of algorithm. Thinking like algorithm is very fascinating concept because it's. So much of it has taken over. Maybe, maybe it's a helpful way to describe a little bit about your role today and more about that.
- [05:16]
B
Yeah, absolutely. So today I lead sophisticated marketing organizations where efficient, scalable customer acquisition is a company priority. Sometimes I work with them as an external consultant, sometimes as an executive on the team, and sometimes by running an outsourced marketing team. And what we like and what we enjoy doing is tackling hard, challenging problems. And what differentiates us, as I mentioned earlier, is a highly analytical, highly technical approach that challenges the conventional wisdom and embraces innovation and first principle learning.
- [06:06]
A
That's awesome. Yeah. What are, is there kind of obviously first principle learning, I think is something that, you know, is critical, important. It's, it's not as maybe common as you would, you would hope to be. I think it's actually kind of uncommon. Do you have something that comes to mind in your career where it's like, okay, this is an example of first principles thinking in action that you feel like more people could adopt or maybe learn from?
- [06:34]
B
Well, I think that it is something that should be applied to everything we do as marketers. One of the big trends right now is this new wave of automation and AI that is happening. And one of the big drivers of that is Google and other platforms moving further and further towards fully managed type of campaigns where you just set the goals, you provide the assets, you provide the targeting and you hope that you'll get performance. But in most instances, just purely following these best practices is not going to get you very far unless you're willing to pay through the nose for your performance. So you need to really understand how these algorithms work, how your, what your, your inputs are affecting the campaigns, and also be willing to kind of create a cross deployment of the new campaign types and the older campaign times types. It is a little bit, to use an analogy from another field, it's a little bit like the innovations that Deep Seq recently made in the AI space where to gain an advantage, competitive advantage in the world where they were constrained in computing power and the money that they could spend for training, they had to go to a subcuda level. So they had to actually go to a level kind of like as an assembly language level optimization to get better performance and to create top performing models. So you need to do that. You cannot rely on, on the cudas of the online marketing world. You need to go low level and you need to understand how things are, how things work in order to actually get the performance that you want.
- [08:34]
A
I love that. It reminds me of something that came up on this podcast yesterday around like if you don't understand something well enough, if you're speaking in terms that are too high level, you're speaking in terms that are too much jargon, you know, you're, you're probably not understanding it at the level that you should and those that are very plain spoken about it and, and, and just get it intuitively, I think that's a sign, you know, like that they're, they're at a deep enough level to keep it simple, to remove multiple steps, to get the best outcomes. And so that comes to mind when you speak in those terms for me.
- [09:08]
B
Absolutely, absolutely. You need to understand the algorithms inside and out. And I think that one of the things that we need to really do as marketers is constantly hone our technical aptitude and our hacker mentality. And when you, when you look at the algorithms that we live around, which is the environment that we live in, they are built by engineers. So we need to think like engineers. We need to think algorithmically in order to thrive in this algorithmic environment that we live in.
- [09:47]
A
Absolutely. And when you think about, you know, you've had so many Experiences across consumer cross B2B. Obviously we have as well and are betting there as well. How do you kind of view the differences between those two? What have you observed there?
- [10:05]
B
Well, this is a huge question and probably we could do a podcast just only answering this question or maybe even just starting at E commerce and maybe an episode on E commerce, an episode on services, an episode on B2B. But let's give some high level insights here. I think for E commerce there are probably three key elements that differentiate good performance. One is focus on profitable growth. The second one is winning the comparison shopping experience and the third one is winning the data game. So profitable growth, why is that important? Different products, usually those at lower price points have lower margins because you need to pay for shipping or you need to in some way provide incentives. You tend to make less money on those. The ones that are more expensive, obviously they tend to be overall more profitable. You need to have the right pricing strategy and you need to have the right upsell cross sell experience to be maximizing profitability on your shopping cart. Comparison shopping experience. Right now, customers relentlessly comparison shop. So you need to provide the best price, otherwise they'll go to Amazon or to your competitors. So this is a must. And in some ways you need to own your niche while maintaining your profit margins. This is easier said than done, but you need to do it. It's not an optional thing, it is a must. And that that will allow you to essentially to avoid being directly price comparison shop, being kind of crowded out by Amazon or Walmart and then winning the data game. Because right now just data is essential for dynamic retargeting, for shopping campaigns, for all of that optimizations. And with those types of campaigns it's junk in, junk out. So you need to have the right data, the right pixelation, the right events and the right content there. Going to services. That's a super broad category. It has lots of really different players. But let's for a second just focus for the purpose of our discussion on local marketplaces there there are several things that are very important. One is geotargeting. Geotargets even in Google are imperfect. There is a lot of optimization, There's a lot that you need to do. You cannot just rely on Google to do the right geotargeting for you. The second aspect is just figuring out the service mix and the profitability of the different services and how those map to keywords, to landing pages, to assets. So this type of flow is very important. And the third, a third element that I will highlight Here is understanding the underlying secular trends. Particularly if you're a marketplace business for demand and for supply and where you cannot really, in theory, Google has its own seasonality and kind of weekend adjustments that they apply, but you cannot really rely on them because those adjustments are too generic. They don't necessarily apply to your business. So in many instances you need to take a much more hands on approach to, to maximize the roas of our of your budget. For B2B, obviously setup is key. Really. There's two components that I want to highlight here. You can very easily overcomplicate things, but lead scoring is essential. So again, you need to have the right data set up. Otherwise it's junk in, junk out. If something is not measurable and actionable in your system, you're not going to be able to the right lead scoring. And the second thing is you need to have a feedback loop that kind of really optimizes the right blending of those lead scoring variables so that you can get a precise measurement of what drives value for your business and align your marketing spend with that. So probably that's kind of really the most important, single most important thing for B2B businesses. Having the right instrumentation and variables inputting into your model and then having the right feedback loops to optimize those. And it's, it sounds very simple but in practice there's lots of complexity under them.
- [15:03]
A
B2B is continuing to, it's growing rapid rate on the partner marketing side of things. As you know, it's, it's growing, growing in terms of our exposure and wins and learnings in terms and even conversations we're having on the podcast and that theme comes up a lot of like the power of instrumentation and audiences and structure is like almost, you know, 2x on B2B it seems it's not that you're, you don't, you don't value that fidelity or the audience data on consumer, because you certainly do. But, but it just, it's seeming to be very powerful, particularly when the complexity is higher. When you're dealing with CRM, when you're dealing with sales qualifications, when you're dealing with dynamic human elements that are, that are qualifiers, there is more involved in some ways and I also observe there's less people that seem to really know it at the level you're speaking of it. Which is exciting.
- [16:02]
B
Yes, no, totally. It is an art and a science. Right. You have this aspect that you need to have for your feedback loop for your scoring algorithm to work. You need to have the right variables in your model. And you need to identify them and sometimes you need to invent them and then you need to instrument them in the right way because otherwise it's never going to work. And with B2B because you're trying to get to the sales qualified lead, you need to do it. There's so many factors involved. It's not as simple as just progression down the conversion funnel as it is in E commerce.
- [16:44]
A
Yeah, I love that. Couldn't be more accurate. I love that commentary. You've obviously seen a lot in the world of search years detail, the spend levels have been insane. When you look at the world of paid search, which is quickly, you know, changing. Help us, you know, debunk some urban legends here. What, what are some of the biggest M misconceptions? What are some of the things that you think people fall into traps on in paid search?
- [17:14]
B
Yeah, there's just so many of them. Paradoxically, our field being as numbers driven and measurable is just filled with urban urban legends that masquerade as best practices. For example, let's take an example. One of the very common best practices is once you make a change to AI driven optimization algorithm like let's say some sort of smart campaign, you change the assets or you change your conversion setup, you need to wait for X days and not make any modifications to that. Well, in practice, if you see that the idea there is that this will give the machine learning algorithm enough time to audition your goal and assets properly and make sure that you know, it understands basically it's, it understands your setup. It's related to this concept that exists in advertising technology design of exploration versus exploitation. So you need to do the right exploration, you need to make sure that your setup gets properly auditioned on different types of inventory, on the different targeting levers and so on. But if after three or four days it looks like you're losing your shirt on this setup and there is no volatility that indicates that sometimes you're meeting your goals. There's basically no sign of life. Probably you should be cutting your losses and moving to new tests or iterating on your test rather than just blindly following the guideline on how long you should be testing. And there's so many of those. Like we can talk about row is bidding, we can talk about smart bidding, we can talk about importing of external goals into the platforms. All of those things are filled with misconceptions where you need to understand your business and you need to understand how the algorithm works in order to be able to grow your business.
- [19:29]
A
Yeah, the Algo has just taken such a significant precedence and it's changing significantly. So I think you're exactly right that it needs to be known and addressed and be well aware of. Do keywords still matter? That's the question that we're all asking, right? What do you think?
- [19:51]
B
Yes, well, searches continue to be the most important source of signal on Google's platform. And we know that for driving raw performance, if your business field allows it, search beats any of the other targeting technologies. So they matter. Now the question is what is going on and why are the new campaign types that Google is rolling out so different and kind of really putting keywords under the hood? Why is reporting for keywords going down so much? When you look at performance max demand generation campaigns, app campaigns, with all of those, the keywords are kind of a little bit bundled into the picture. But Google is trying to sell this more holistic experience of driving performance for the advertiser. Why are they doing that? Really? There's two reasons in my view. One is that they they're strategically helpful because these types of campaigns help Google more directly compete with meta on market share for demand generation display earlier part of the funnel performance. So by blending the keyword performance with the other targeting technologies that are more demand gen focused, it allows advertisers to get good performance on par or in many instances beating what they see with meta. So it helps Google get more market share from its top competitor. The second reason that they're investing in these types of campaigns is that they are a great strategic fit for the AI generated future of search. So Google is moving toward away from keywords and being more like an answering machine. And as this happens, they need to incorporate more of contextual signal into add a deeper understanding of longer queries than they currently have. So by blending the decision space, if you will, for optimization into those smart campaigns, they are positioning themselves to be effective in this new world and still make a ton of money because they are a business and they thrive on ads. That's what they do as a business. So that's I think the reason that Google has been investing so heavily in these media type. So for a marketer, what does that mean for us as a marketer? You need to be practical and balance investing in these future campaign formats that are going to be dominant in the coming months and years. But at the same time you need to really do also the things that are right now currently driving performance. So you need to strike the right balance based on your industry between keyword based campaigns with different smart bidding strategies. That are appropriate for you and these more fully automated campaigns and find the right marketing mix there.
- [23:13]
A
That's great. Yeah, it seems like there's this just critical balance to make sure you're tapping into keywords and the algo. And it's not necessarily an easy balance and I think a lot of people are not doing one or the other well enough. But it's exciting to hear your perspective on it in terms of, you know, making sense of the data. Right. What are some of the biggest mistakes and you can think of like data measurement, attribution. What are some of the biggest mistakes there?
- [23:42]
B
Well, I think that one of the most common mistakes that I see over and over is under tagging. If something is not visible to Google or to Meta, then the platforms will not be able to optimize based on it. So frequently a checkout will go through different steps and often those checkouts right now are on JavaScript so the URL will not change, which is fine. But then you have a problem. You need to make sure that in some way you're conveying to the user that their progression through the funnel. So you need to send events, you need to instrument events so that the corresponding platform that you're trying to optimize can see that this user went 80% through the conversion and they did not complete the final step or something like that. Otherwise you better have a really high frequency, low value purchase so that Google or Facebook or TikTok or whatever your platform is can optimize based on that. There's enough data to optimize on the end conversion. But a lot of the advertisers, a lot of the players in B2B, they have a very high value and conversion. So being able to fully instrument the different steps becomes very important. And another problem that I see very often related to data is poor alignment between performance and data. So what you send to Google or To MeT should be closely aligned with what makes you the advertiser money. If your purchase, if your monetization event takes multiple points of touch and you use something like Google Ads Pixel and Meta Pixel, then the problem is that every platform is going to claim credit for every conversion that they touched. That means that they are going to be optimizing and overweighting you on things like retargeting which you want to have, but you don't want to be over reliant on it. So you need to have the right setup there. For Google Ads, the solution is very easy, which is, or relatively easy, which is you can incorporate, for example, GA4 Google Analytics conversions and those conversions, unlike the Google Ads conversions, are they account for cross channel attribution already. So you are not going to have a situation where your Google Ads campaign claims credit for a conversion that actually was more facilitated by your meta campaign. So this cross channel attribution is very important. And it's notable that Facebook is now trying to incentivize advertisers to import their GA goals into Facebook. I think that they're realizing the same problem, that ultimately you can do so much smoke and mirrors and you need to drive real performance for the advertisers. So that is like a really common problem that I see across advertisers. They just like they will over rely on the wrong types of pixels on the wrong types of data to optimize their spend.
- [27:26]
A
Yeah, it's such a great point. There's so many things and details to ensure that the instrument and measurement is set up correctly. And then it's. I find it shocking how it makes common sense like each platform is incentivized to tell you that the platform is performing well, but at the level at which they do that and the, and the percentage of time that reasonably sophisticated marketers and brands and consultants are not acknowledging that well enough is really shocking to me. And you know, in every single performance marketing channel, this is a miss where if you have anywhere remotely close to the spectrum of a set it and forget it approach, you're literally throwing money out the window.
- [28:12]
B
Yeah, well, I would say I would take even one step further. I would take this one step further. I would say that if you make some of those mistakes, they can easily take your business down.
- [28:22]
A
Yeah, that could be the highlight of our conversation almost because it puts the stakes pretty high. And I think it really, it doesn't overstate how important it is to get it right. This is good stuff. There's a lot, there's a lot here. There's a lot to, you know, be, be mindful of in this space. And I think it underscores the importance of really getting someone like yourself in the corner of a brand and at least having eyes on things to make sure that, you know, do they have an insurance policy in check? We're investing a lot in this area, talking to as many consultants, experts in attribution and incrementality. Mmm. Providers, MTA providers, you name it. But there's a lot that can be done and it's just surprising how many people aren't tackling it. You've obviously thought about, read about, worked a lot in AI and automation. It's taking over, taking the world by storm in both fear in conjecture, but then also some real reality. With LLMs rolling out a few years ago, the power of each one is rolling out features almost weekly. You have more visual options. What's actually driving advertising performance in that world from your perspective beyond what's just buzz and bs?
- [29:45]
B
Yes, it's a great question and it's a fascinating question. LLMs obviously have taken over creative generation on both the textual and the visual asset side, but there's a lot other stuff going on. When I was running Marketplace optimization and media at Turn, which was an early DSP player, we used threenet and controlnet control theory to optimize advertiser and partner agency performance. Now there are models that incorporate reinforcement learning for things like budget pacing, auctions and conversion optimization. There is multitask neural networks for event and value predictions. Transformer models are entering the space very strongly for sequential behavior and models like AutoML are used for model tuning and for targeting. So there is so much stuff going on there. A lot of it is not LLMs. But what we will see in the next chapter is as Google moves towards an answering engine, the two worlds are going to merge because the LLM generated content in the answer is going to be the one that will be driving the ads which are getting optimized by these other types of machine learning that I just mentioned. So it's getting more and more complex, more and more interesting. And as an advertiser you need to understand these technologies and to be able to position yourself to best benefit from these algorithms.
- [31:43]
A
Yeah, well said. It's insane the pace of change that's coming. And I love hearing where your head's at with all this stuff in the tactical side of search, like where should marketers be thinking about our friends over at Bing as a search engine option? How do you think about that conundrum for brands that are maybe hesitant to to invest or curious be going beyond Google?
- [32:09]
B
Yes, I love Bing. So my answer is absolutely they should for several reasons. One, Bing is 15% of Google search volume. Depending on the industry it could vary a little bit and it is less competitive. So you can get better CPCs. The optimization is a little bit behind, but you can get very competitive CPCS there. The second reason that Bing is great is that Bing SEO, for example, is a great lab for Google SEO. Their webmaster resources are in many ways unparalleled. They're the best. And we haven't talked as much about SEO so far, but it is a Huge part of the revenue stream of the marketing mix of most businesses. And while it's slowly decreasing, it is not going to completely go away. It is going to be transformed by these models. So you need to think like to perform today, to make money today as a business, you need to think about SEO and Bing is a great way to fine tune your SEO strategy. And then last but not least for B2B businesses, Bing integrates LinkedIn targeting with search functionality at very affordable CPCs and that can be very, very powerful and unique. So being able to integrate these tool targeting technologies can be very, very powerful for a B2B business.
- [33:52]
A
Love that. Yeah. I love the emphasis on the B2B opportunity and I'm a big fan. I think in the right situation it's hard to really miss it. And love the call out coming down the home stretch of the pod just wrapping up. I love all the great learnings we've shared. What are you most excited about in the marketing world right now, Svet?
- [34:14]
B
Well, I just love this coming wave of AI search. It is at some level scary for people who are entirely in SEO because things are going to be brutal. Coming months are going to be brutal in many ways. But paid search, paid advertising is not going away. This is what Google does, this is their ad business model. So it will just go through a major transformation and when that transformation happens, it will be very exciting for a good marketer. Change is exciting because it means disruption, it means new opportunities. So I can't wait. And some of the changes, like the double serving, for example, that Google announced recently across placements is indicative that they are getting ready for making more dramatic changes in search very soon. Because it's clearly a revenue boosting change. It is clearly intended to help kind of protect them a little bit, to give them a little bit of a buffer as they make these changes to an answering engine and figuring out what works best as they go. So it's, I think that the change is very imminent. They will be happening more and more will be happening in the coming months and I can't be more excited.
- [35:45]
A
That's awesome. Yeah, no, it's, I think, I think we thrive on that early stage and the emerging channel phase of things. That's where a lot of the exciting performance and value can be found. So I think I'm with you there. What's a piece of advice you'd give to today's marketing leaders and what would it be?
- [36:05]
B
I think that we should use AI tools every day, read about AI models, study AI experiment, read research papers. And no, just using ChatGPT in answering mode is not enough. Like we need to really understand these technologies because this is the environment that shapes our industry and it will determine if this coming wave will. If with this coming wave you will be among the winners or the losers. There always are those categories.
- [36:43]
A
Yep, absolutely. Absolutely. Where can folks follow you and learn more and kind of keep tabs on all things Svet?
- [36:51]
B
Thank you. Yeah, I think the best way probably is LinkedIn. So reach out to me and I will respond. We are currently actively looking for more clients, more opportunities to create value. So it's an exciting time to partner.
- [37:08]
A
Absolutely. Any, any interesting reading your perusing right now that you want to recommend as books to the audience?
- [37:14]
B
Yeah, I think that I have been reading Peter Norvik AI textbook. It's absolutely awesome. It's super detailed and he is an excellent teacher. So there's so much stuff that you just open a random chapter and you read through stuff and you can learn so much, you can get so much additional insight. It a little bit reminds me of the Feynman lectures in physics which similarly inspired me when I was high school student back in the day.
- [37:47]
A
I love it. That's amazing. That's such great stuff. It's been such a pleasure to get to know you over the years and continue to learn and talk and share insights with each other. And I know our audience appreciates where you're coming from. It couldn't be a more appropriate fit for the always be testing pod. So thanks for coming on and thanks for sharing with us.
- [38:07]
B
Thank you. It's been a pleasure to chat with you always, Beth.
- [38:10]
A
Thank you.