← The Digital Marketing Podcast
The Digital Marketing Podcast
Digital Measurement And Why It's A Waste Of Time!
00:18:20
OK, we know, its a controversial title but in this episode Ciaran and Daniel discuss digital measurement through analytics and look into attribution modelling. How can you best identify the channels and marketing activity which are driving the success...
Loading summary
Transcript45 lines
- [00:00]
A
Welcome to the Digital Marketing Podcast brought to you by targetinternet.com hello, and welcome back to the Digital Marketing Podcast. My name is Kieran Rogers.
- [00:19]
B
And I'm Daniel Rolls.
- [00:20]
A
And today, Daniel, we're talking about digital measurement and why it's a waste of time.
- [00:24]
B
Yeah, this is a slightly controversially named one. This comes from a number of things that we've both been talking about.
- [00:31]
A
So.
- [00:32]
B
So I just did a second edition of digital branding and I never mention my books. Do you write a book? Yeah, exactly. So we're doing the second edition and I asked Kieran to write a few words for the book. When we're talking about measurement, and that was looking at kind of display advertising, pay per click, and some of the.
- [00:50]
A
Challenges around that, I'll be honest, I really struggled. I dragged my heels on that for days and days and days. And then I think I read an article on E consultancy and it was just red rag to a bull for me. I'm like, no, that's wrong. I disagree. And it all came out.
- [01:10]
B
Yeah, it did. And it was a great thing.
- [01:12]
A
A little bit of a rant, if I'm honest.
- [01:13]
B
Yeah, that's good.
- [01:14]
A
So I hope you enjoy it if you ever read it.
- [01:15]
B
So we'll post some excerpts onto the website as well.
- [01:18]
A
Kieran's rant.
- [01:19]
B
So if you Again, show notes Targetinternet.com podcast and you will find all the show notes there. But just tell us where this all started from.
- [01:25]
A
Well, we'll see. I might get very animated listeners, so.
- [01:27]
B
That'Ll make a change. Ready? Peaceful and quiet.
- [01:31]
A
More so than usual. But where do I start with this? Well, the article I was reading was all about digital measurement and cutting the fat and boiling everything down to just what drives results. And, you know, there's a lot to be said for that. But I kind of disagree with the fundamentals on it in that where do I go with this? I don't think measurements good enough to be able to take all of your decisions based on what the measurements showing, at least what a lot of people are choosing to measure is the key thing. Right. So a lot of businesses out there will rely on analytics to tell them what's going on, how stuff's working and how they can do better. And there's nothing wrong with that. Okay, let me get that straight. However, there are some apps, absolutely fundamental flaws in the default kind of attribution models that a lot of these systems use. So let's take Google Analytics as a starting point and any of you use that, you'll be aware that it sort of works on a last click attribution model. So I suppose we should just explain what an attribution model is before we get really stuck into this debate.
- [02:48]
B
So attribution modeling basically says, if my path through to conversion, so me doing the thing you want me to do on the website, for example, buy a product or fill in a form, there might be a number of steps that have got me there. So I might come in from search, I might come in from display advertising, from email, then I come back from search again, and so on and so forth. And attribution modeling is trying to say, what's the value in each of those steps and how important are each of those steps as well? And there's a number of ways of kind of measuring, but you need to start by setting your goals, which is by saying, what do I want you to do? So that attribution modeling tries to play around with how we attribute value to each of those steps.
- [03:24]
A
What marketing activity did you do that drove the end result? And whatever your end result is, it could be you're looking for leads, or it could be you're looking for sales in your online shop, or it could be something completely different. But that's essentially what it is. And so as digital marketers, it's really useful because you get a sense of, right, we did this and it's driving this. Right. So the problem is that the default attribution model that most people rely on is just kind of wrong. I caveat it by saying that most attribution systems are wrong, but there's less wrong and you need to sort of get more. Let me explain what I mean by that. So the way Google Analytics works, it generally works on what's termed a last click attribution model, but generally the last click activities. Let's say I visit your website through an organic link and then I visit you through Facebook on one of your social links. The last and I buy or I sign up or I become a lead. The last click in that journey would have been Facebook and therefore that would get all the credit. Great. Do more Facebook Facebook stuff. And this is the problem where the attribution model is heavily going towards last click is, well, what about the organic link that I clicked on first? Where does that come into it? If I hadn't had that, maybe the Facebook interaction would never have taken place. It's complicated, right?
- [04:45]
B
One thing I'd also say as well is that we are talking about attribution models and Kieran's point about things being Less wrong is really important. This is a model and actually if you wanted to really know how people make buying decisions, you'd have to follow them and watch them and ask them questions and see how they interact with the world on a day by day basis. Because we're tracking all their digital behavior. That digital behavior is going to be impacted by their thought processes, what people say to them, what they see, all those kind of different things and you might get really clever. And actually the only reason someone's done a search is because their friend in the pub the night before told them something.
- [05:19]
A
So there's an answer to this. You need to place some JavaScript on their friend in the pub to track what they actually.
- [05:27]
B
Well, this is it. Until you can actually map someone's neural networks, you won't know how they make decisions. And there's some privacy implications of that, to say the least. So I think this is it. These are models and we're trying to get the least wrong model. It doesn't mean it has to be perfect because we're looking at grouping and therefore saying, generally speaking, if we do more of this, we're going to get.
- [05:45]
A
Yeah, yeah. So last click has its flaws. There's another model you can look at which is first click and that's an interesting one because that gives you, you know, what starts the journey and maybe that's kind of right. Well, dunno, not really. It's kind of almost as wrong as last click because you're looking at one extreme to another. Still interesting and definitely worth a look at because you get to understand a little bit more about the customer user journey. What's generating those first clicks, what's generating those last clicks. You've got other models where you equally share all the attribution across all channels and all touch points, which is known as a linear model. I'm not such a fan of that because I think that just skews things even more, really. The one I would say is definitely worth looking at is one called time decay, and that's a really interesting one. I think that gets a little bit closer to the truth. So, you know, it looks at the time difference between those first clicks and those last clicks and it attributes more waiting if it happens quicker between first click and last click. But as time goes on, so maybe two or three weeks pass, so the weighting of that first click would diminish and I think that's probably getting a little bit closer to the truth, but it's still not perfect.
- [07:05]
B
My problem with attribution modeling this way it's also arbitrary. It's all like me going, yeah, I reckon that makes sense. Yeah, let's do that. And then you apply a model and go, oh, look, turns out that was correct. And actually the only reason it turns out is that was correct because we've applied the model to it. So if we say, well, I think the last paid search is really important, you go, look, it turns out it is. So there is a danger with this.
- [07:23]
A
And there is. And this is why I get so fired up when people come out ranting and raving about, oh, you've got to just take decisions on data and be ruthless and cut the fat and what have you, because we can't measure everything that accurately yet.
- [07:40]
B
Let me give you an example. So we did an experiment. So this is through to the target Internet website, and we essentially did a load of paid search in Google, which was great. We did a load of Google Display network advertising, which was great. And we then went to load Facebook advertising, which was great. Okay. So we went through and did all three things separately. Then we ran them all at once, and the whole was greater than the sum of the parts. So basically, when you've got all three running, you end up getting better results than you do individually, just spending the same amounts of money. Now, what that shows us is the user journey is probably more complicated than we realize in the fact that people are jumping between channels, the brand is involved in this, the brand reinforcement is involved, and so on as well. But also what happens when we do those three things? Our direct traffic goes through the route. So what's happening is some people are coming through the Facebook, the Google Display network through the paid search. But a lot of people were seeing that stuff, not actually clicking on it and then just typing in the website address, or they were coming through it once, then coming back through another channel, then coming back directly. Now, the reality is I can see these assumptions going on. I know my end result was better, but I can't actually prove that 100% because I can say I did these things and this is what happened afterwards. But I can't show you the absolute journey because a lot of people saw things, didn't click on them, then later kind of came back. So in reality, you have to experiment and the data won't be perfect, but you're reading the data, interpreting it, drawing a hypothesis and working through that cycle. So it is important to measure, but it's not always 100% effective.
- [09:16]
A
And it's a little bit more of an art form than a science. I would argue what I think you find is you begin to play around with different attribution models. As long as you remember none of them are accurate to within 100%, you know they've all got their problems. You do begin to understand how all these different mediums that you're working with, be it paid, social or organic or direct to website or email or pay per click, how they all begin to connect together. And this has been my experience and my journey going through all of this as I've tried to seek out more of the truth or a better truth is that as you play around, so you begin to understand how all these different channels and activity are connected. So I do tend to think of these as cogs. And you're beginning to see the inside of the machine and see, well, if I turn that cog, this happens and if I turn all three of these cogs, this happens. And actually you can get like cogs and machines, you can get leverage a lot greater from the sum of the individual parts when you put them all together. However, there are still, particularly with web analytics, there's a whole raft of stuff that just isn't measured. And any of you brand marketers out there, you're going to love what I've got to say on this because for me it was a bit of an epiphany. I've always been in the camp of, yeah, brand shmans is just kind of, you know, it's all a bit woolly and stuff.
- [10:37]
B
Well, there is another reason for that, because branding is used as an excuse in digital a lot. So someone will say, why did you do that Facebook campaign? And they'll say, oh, it's a branding campaign. Which means they don't know why they did it and they don't know how to measure it. But actually branding is a hugely important thing.
- [10:50]
A
So I've always sat in the kind of against branding campaign, but actually recently I've come around to it because actually one of the things I've learned, I've been running a lot of display campaigns and display activity. One of the things that I've learned playing around with it and turning things on and turning things off is that actually that activity does drive a lot of extra traffic. When you turn it on in volume, suddenly you start getting way higher levels of traffic coming in through lots of different mediums where it doesn't come in, it doesn't come in through clicks on your, on your, on your display campaigns. You know, people don't click on those things, but they do see them, they do Clock them and it does sort of warm them, warm them up to who you are and what you're about, you become a little bit more familiar so that when you do reach them through maybe some email activity or a social media campaign, they're that much more receptive to you because you're someone that they know. Right. Rather than a complete stranger just trying to sell them stuff. So as you play around with this, so you begin to learn this. Now, the problem is here that if I haven't clicked on the activity and gone through to your website, but I've been influenced to it, there is an absolute disconnect between your analytics and those different activities. And the only real thing you're sort of left with is sort of cause and effect. But if you've kind of taken our advice in previous podcasts on work out all of the different things that you've got to measure, and those aren't necessarily just digital things. You can begin to build up a bit of a picture. But you do also have to follow our advice and be quite brave and turn things on and turn things off to actually prove that you're right.
- [12:29]
B
Because of these problems of attribution modeling and how you kind of go for a modeling, I took a slightly different approach and we've used the multi channel funnels report a lot and I've always talked about this a lot. We'll put the link into the show Notes, the measurement framework that I kind of recommend. And what that says is you use this multi channel funnels report to work out in what percentage of cases has this channel contributed to someone ending up doing the thing you want them to do. Now that's great, but the only problem with that is it doesn't take into account offline things. It doesn't account for ads that people have seen but haven't clicked on. So what I think you need is a dashboard that shows you all your digital measures, but also other activity that didn't lead people through to the website. So you might have done some display ads. Well, let's show those measures. But even if they didn't click through to the website, let's show how much we were doing. So if suddenly my SEO traffic goes up, I could say, well, why did that happen? Well, actually we were doing loads of display, then maybe it's built awareness and people have searched for it. I'll give you another example. We started spending a lot more money on advertising of all different types and lots of traffic from them channels. But our direct traffic was going through the roof and it was by far the biggest traffic source. I thought, I've done something wrong. I haven't set this up properly. So I audited my analytics code. I reset up all the tracking code on all of my channels. Still the same results. So I was lucky enough to speak at a conference with someone who's a real analytics expert. And I said, can you do me a quick favor? Look at my analytics. And he went, yes, branding. So basically we've done all these other channels. It's built awareness and people are therefore going through and searching and awareness of us and so on as well. And it's not the answer I wanted at all.
- [14:05]
A
And those noises he made, were they a crucial part of the process? Because I love that.
- [14:08]
B
I think he was charging me by the Merlot.
- [14:09]
A
Was he?
- [14:11]
B
So the reality is that actually this direct traffic thing is a challenge and analytics is not perfect. Even if you have got an E commerce business, there will be things influencing people that you're not aware of from that point of view. So it is an art as well as a science, and I think you need to keep that in mind. So what are the solutions really? I mean, it's looking at time decay, starting at time decay, looking at customer attribution and working out what you feel works and then adjusting it.
- [14:35]
A
Also experimenting with those different models. Look at, you know, learn what first click activity drives and how that interacts. Look at all the different models, begin to get a sense of how your individual medium cogs all link up for your business. And it is fascinating because depending on what you've been up to over the previous six months, you're going to get lots of different results. But that's the fun of it, right? You're like an artist painting with colors and mixing things and creating beauty.
- [15:04]
B
I think that you also are now very. It's very easily possible to go into analytics, find a fact. So you identify a fact and you make a hypothesis. And the hypothesis, I think this is telling me this, you then implement something to test that hypothesis where improving something, changing something, whatever it may be, turning something off even, and then you go back and say, did it have the impact that you thought? And you work that as a working cycle and there are no failed experiments because actually you're learning something each time as well. So you have to be in this loop. And if you don't resource this, if you don't actually spend the time looking at the analytics and implementing these experiments, you'll find yourself in a situation where you've got kind of data paralysis because it's not really telling you anything useful and you can make the wrong assumptions as well. So take a look at your data. But as Kieran says, be an artist, paint some pictures with the data.
- [15:50]
A
Yeah, and there's a couple of things I wanted to recommend. So if you're new to attribution modeling, I want to dig into it. There's an article by somebody who I'm utterly in awe of, but I. I don't think I can pronounce his surname correctly. Is Avanash Koushik.
- [16:07]
B
Avanash Koushik.
- [16:08]
A
Yeah. Have I pronounced that right?
- [16:09]
B
I believe so.
- [16:10]
A
Do you know what? I've been scared of saying that for a while because I follow his blog and read, but no one ever tells you how to say his name.
- [16:16]
B
I've seen him speak at conferences and I'm pretty sure that's how he.
- [16:19]
A
He has a brilliant blog called Ockham's Razor and there's an article on there which you'll be able to find. It's called Multi Channel Attribution Modeling. The Good, the Bad and the Ugly. And I definitely recommend having a read of that. He goes through all the different modeling systems that he's looked at and explains why each one is least wrong as he works up to actually kind of a custom attribution model and how to apply it within Google Analytics. A really, really good read, really thought provoking and he encourages you to experiment and apply that kind of knowledge to your own model. And the final thing. It's been a while, Daniel, since I've done a Winston Churchill impress. I'm not sure I should redo it because we got a few complaints last time, but I'm beginning to think the famous saying of Winston Churchill there are lies, damn lies and statistics. I feel he should actually be updated and it should be saying, well, there are lies, damn lies and last click Attribution Modeling.
- [17:19]
B
And again, we're gonna finish the podcast on one of Kieran's moments. So thank you very much for listening indeed and find all the show notes Targetinternet.com podcast and want to hear your comments as normal. And we'll see you again on the Digital Marketing Podcast.
- [17:40]
A
Thanks for listening to another episode of the Digital Marketing Podcast brought to you by Target Internet. If you'd like to get more information on the show, get hold of back issues of this podcast or get details on any of the links we mentioned, please visit our website at www.targetinternet.com. if you've enjoyed the show, we would love to read your feedback. Please rate us in itunes. Or even better, write us a review. Or if you have any questions, please get in touch. We'd love to.