
Ten years ago, on a cold dark night, a podcast was started, 'neath the pale moonlight. There were few there to see (or listen), but they all agreed that the show that was started looked a lot like we. And here we are a decade later with a diverse...
Loading summary
Michael Helbling
Welcome to the Analytics Power Hour. Analytics topics covered conversationally and sometimes with explicit language.
Tim Wilson
Hi, everybody. Welcome. It's the Analytics Power Hour. And this is episode 261. You know, a long time ago, actually it was January 3rd, 2015, the first episode of the Analytics Power Hour aired. And we didn't think much about the future back then. I think we were just trying to share ideas and keep a conversation going with people that we liked from different conferences we'd attended. And we were trying out a new format and wow, 10 years can go by really fast. And what. Over the years, we've had some amazing opportunities take the Analytics Power Hour to places we never would have imagined. 261 episodes, multiple live events across multiple continents. And a chance to interact with so many amazing analytics people who made the choice to listen and interact with us along this journey. And to everyone who's been a part, both as a co host, as a guest, and as a listener, I mean, we can only just say thank you. Thanks for an amazing 10 years.
Julie Hoyer
It's been a good run and we're done.
Tim Wilson
That's it. And this is the final. No, it's not. But it is the year in review episode. All right, so one of my favorite episodes every year. And you know, in our profession, a lot of times in analytics, we get more attention for what's wrong with everything. In fact, we analytics people, we tend to be good complainers about all the different things, things that are broken and don't work right and people don't listen. And maybe this time around we're just going to do something where we focus on the positive, Tim.
Mo Kiss
And do your best. And it's just an hour.
Val Kroll
In this.
Tim Wilson
In this episode, we're just going to talk about some of the best things we saw this past year. So let's jump into it. Let me introduce my co host, Tim Wilson. Welcome.
Julie Hoyer
Two continents, is it?
Tim Wilson
Third.
Julie Hoyer
We're doing Antarctica in 2020.
Tim Wilson
I sure hope so. Live with the penguins.
Julie Hoyer
And we can do that in Australia, apparently, now. Oh, I mean, speaking of that.
Tim Wilson
Mo. Kiss. Welcome.
Josh Crowhurst
Hi. How you doing? I feel like today's gonna be a real role reversal because I'm feeling particularly pessimistic and I might have to steal that from. From Tim today.
Tim Wilson
All right, well.
Mo Kiss
Freaky Friday.
Tim Wilson
That's right. And of course, Josh Crowhurst. Welcome.
Unknown
Hey, yeah. Good to be here.
Tim Wilson
Awesome. Thanks for being here. Val Kroll.
Julie Hoyer
Welcome.
Mo Kiss
Hello.
Josh Crowhurst
Hello.
Tim Wilson
Holding things down from Chicago, 4th Floor Productions. That's right, 4th Floor Productions. Julie Hoyer. Welcome.
Val Kroll
Hey there.
Tim Wilson
Awesome. And I'm Michael Helbling. All right, so it's worth mentioning as we get started, like each of us does kind of very different things in our day to day. And so what I'm excited about, as we talk through some of these topics, I actually think there's a lot of different ways we look at the world and a lot of diversity in the what we handle. And so I think actually what we're going to talk about, it kind of makes me really excited because when we go through this stuff, we'll all have different perspectives and different experiences and sometimes maybe even differing opinions. But that's usually never happens.
Josh Crowhurst
Right.
Tim Wilson
If you've listened to this show for.
Mo Kiss
10 years, for 260 episodes, we've always been on the same page.
Tim Wilson
That's right. In lockstep on every idea. All right, let's dive into it. So we're going to talk first about the best thing we saw this year in data collection and pipelines in our work. So I don't know who wants to kick us off, but I'll throw it out there. It's sort of like one of the things we do in analytics. We collect a lot of data. What was something awesome you saw in that area this year?
Josh Crowhurst
Okay, well, full disclosure, I feel like this year has been the year that I have really started to work more closely with Snowflake and full disclosure, that's who Canva uses as one of our partners. And I've just really been blown away by some of the technology they've been building. The Snowflake feature store is particularly cool because I feel like lots of companies try to build like a feature store, CDP or whatever you want to call it in house, especially, you know, if you have an in house data warehouse. And Snowflake have made it like they really simplified the ability to build a feature store that then particularly can be used for ML models. So that's been really cool. Snowpark is the other thing that's been pretty awesome to see because it lets our analytics engineering team program in multiple languages. I know my team in particular have managed to build a bunch of pipelines this year using Snowpark that have, you know, saved like days of full time, like FTE every week. And so for me, it's really been kind of leveraging what other companies are doing well and then kind of using that to drive our innovation. But like I said, I am very much on the Snowflake bandwagon in 2024.
Julie Hoyer
And they did not pay for that endorsement.
Josh Crowhurst
They did not.
Val Kroll
Well, speaking of partnership I'm glad you brought that up because one of the greatest things I've seen in this year in like data collection in general is actually we've been working like cross groups, skill sets, capabilities, we call them practices on some clients. And it's been really great to see we've started to partner better around implementations in general. We actually have had a couple clients where we've had implementation work and helping run their paid media. And in the past that hasn't always happened as often when we're like in an engagement with a client. And so we were able to kind of take a step back and talk about like, how do we take all of this into context when designing our implementations and better serve the client? And kind of the big picture. Talk about that a lot obviously here on the podcast of like, how do you take that bigger context into consideration? And so I was really happy that we were able to start to actually put that within our process for some reimplementations that were we're currently doing for clients. And it was interesting too because we had to really break down the understanding for the client and even internally like the best practices they do in paid media compared to the best practices we do in certain reporting tools like Adobe, for example. And so making that a single connection point was actually something we spent a lot of time on. And I think it was really valuable for like us and then obviously the client and the output they're getting.
Tim Wilson
You know, it's interesting in my world, a lot of my clients this year were struggling with various parts of consent management. And that's been around for a while. But there were some changes to some things this year that a lot of companies had to grapple with. And it was cool because one of the folks here at Stacked Analytics, Charlie Tice, who's pretty amazing, so not something I did in data collection, but I was got to be part of a team that did some amazing things. And just watching him break that down into very simple terms for our clients and create elegant solutions that actually solve the problems that we're facing to get that right. Because it's kind of really imperative to get that stuff right now. Like you can't sort of, you can't really fake it. And it's crazy to me how badly as an industry we still are at that aspect of data collection. So that was my highlight for the year, is just sort of how we were able to track down and really get some good solutions in place for companies this year. And mostly Charlie did it all, but I was very thankful to Be part of it. It's time to step away from the show for a quick word about Piwick Pro. Tim, tell us about it.
Julie Hoyer
Well, Piwick Pro has really exploded in popularity and keeps adding new functionality.
Tim Wilson
They sure have. They've got an easy to use interface, a full set of features with capabilities like custom reports, enhanced e commerce tracking and a customer data platform.
Julie Hoyer
We love running Piwick Pro's free plan on the podcast website, but they also have a paid plan that adds scale and some additional features.
Tim Wilson
Yeah, head over to Piwick Pro and check them out for yourself. You can get started with their free plan. That's Piwick Pro. And now let's get back to the show.
Julie Hoyer
My contrarian. Well, first I will say I've not worked with it so I've just seen kind of the videos around it. But on a data collection, the Nightjar from Moonbird AI very much Adobe analytics focused thing. Just what I liked about it is they seem very, very focused and not like this is gonna solve everything. They're like we have identified a problem that we have lived and felt the pain with. But I haven't actually used it. I've only sort of I know the people who are doing it and have liked what they've been saying it is doing. But for me there seems to picking up more signals around doing less data collection has kind of been what I've gotten most kind of excited about this year. And I probably pointed to Matt Gershoff kind of laying out the just enough just in time mindset as opposed to just in case and you kind of stack, I mean Annie Duke just last month came out with something where she was kind of theorizing that we over collect data partly so that we can talk about all the data that we looked at if something doesn't turn out as we, as we'd like it to. So I've been and even Jen Koontz has kind of been also talking about that A lot of it's been under the starting point of privacy and how we need to be very disciplined about what data we're collecting on users for privacy reasons kind of writ large. But I've spent a good chunk of this year thinking around I think a lot of organizations, they have enough data, it's clean enough and maybe they shouldn't be worrying about doing more or better or more sophisticated. So that may be that's tying up what I've seen with kind of the, the focus of like my personal journey this year. But I, I really liked Gershoff's framing of Just in case is what we default to. But getting just enough just in time that, that framing is a, as a way to go about thinking about data collection.
Josh Crowhurst
Me too. Like, that actually is like I started to do Chef Kiss and say it instead of just having the emoji. But I mean, you know, it's a podcast. The emoji wouldn't work. The only, I guess the only caveat or the catch is we can see that in the data, I guess community in 2024. But I don't feel like our stakeholders are there yet. And I think that's the really difficult thing is like we might say to them, you know, just enough just in time. Was that, Did I get it right?
Julie Hoyer
Yeah, just enough just in time. Yeah.
Josh Crowhurst
But I, I still, I still feel like there's a tension right where the business is like, track everything. And you're like, no, that's really not a great idea anymore. So maybe that's, maybe that's where we get to for 2025.
Julie Hoyer
I mean, I mean, I think that's, it's a great point. I think that it's an easy thing to default to. I mean, I go back over A. Probably 15 years ago being told, hey, this tracking is going to be really easy. We just want you to track everything. Like it's easy to articulate. So there's definitely a business partner management because it's. They feel like obviously we need this like in the absence, like clearly we need the data. And I think AI has driven a lot more around the, oh, you got to have the more data, the better to feed into the monster machine. And it's got to be really, really clean. Ben Stancil had a, had a post last month about kind of quest calling that into question, which I thought was really good as well. So I think, I think we, I think there is a need to bring along our, our business partners with. Wait a minute, what are you trying to do? Don't let's not be prescriptive that you need this data. There's no value in the data. There's only value in what you're doing with it. So let's talk about what you're going to do. But that's a. I agree. I think it's got it. Maybe it's got to start with the data community to have that mindset to not just kind of take when somebody says we need this data to just say, well, not having it or having it. Clearly, yeah, we don't have it. We can default to. We need it. Instead of defaulting to maybe we don't need it and we should question it without, you know, burning our relationships.
Mo Kiss
So, and even, even if you were to look like retrospectively and Julie, I've heard you talk about this like several times and I love the way that you put it. So if I bastardize it, please correct me. But the number of times when like the just in case actually saves you is actually pretty slim. Like, trying to torture data for the purpose it wasn't intended to be collected for gives you that like, you know, where's the wind blowing direction kind of take on things, but it's actually not always customized to answer the specific question at hand. And so how much are you really missing out on? Right. Like I, I've heard you. I don't know, Julie, if you have thoughts, this is definitely coming from, from your brain, but I, I totally agree with you.
Val Kroll
Yeah, no, I would say most of the time people say, oh, well, we're collecting, you know, data in the vein of this question. We should be good. And then by the time we really get into it, it's like, oh, actually, sorry, you're missing like a key, you know, setup, configuration, the way the data is actually coming in or is implemented, like it's not going to do what you think it's going to do. So I run into that way more than, oh, cool, you already have the data. It's perfectly set to answer this question that like, never happens. Like, I've, I've learned my lesson.
Mo Kiss
Serendipity.
Tim Wilson
Well. And of course with any topic we're going to find some things to become, you know, concerned about. But let's move on to our next category. So this year, what was the best thing we saw in the area of experimentation and optimization? Like what, what things did we see in that category this year that we liked?
Mo Kiss
Okay, so I'll go first in this one. I'm actually excited to talk about this. So just before I left further earlier this year, I was working with the optimization team on a cool project for consent banner optimization. So as we know, consent banners are a little bit wild wild west. As you mentioned, Michael, some changes happening this year. And so I say wild wild west because we don't really know kind of like what works in terms of right practices or how do we create the right transparency and allow people to really hold the remote on their preferences. So the team started with a lot of research on like, what capabilities are out there. OneTrust offers some a B split testing so we, we could do a little research on what levers there were to pull. Then we did a lot of research to develop a POV on what further wanted to recommend to clients to be testing and what did we want to ethically support to make sure that we weren't injecting any dark patterns and that we were really focused on aiming on education and obfuscation of information to really increase brand trust. And then we put some of that together to run some tests on the own for their website. And the first one which was really just kind of playing around with some of those basic, I'm embarrassed to say it almost button color principles of contrast just to really make sure people didn't think it was a modal, that it wasn't going to stop their behavior but that it allowed them to interact and set their preferences. We saw a huge increase. And so we were able to kind of bundle that together to put together a nice solution to be able to share with clients and having those that case study already in hand after, you know, 60 days of research and work because it was definitely a labor of love to pull that together quickly. But it was really kind of fun because it kind of felt like something I hadn't seen before. You know, I was able to engage with lots of different clients across a lot of different industries. But there's definitely thematic some of the similar things that come up time and time again. So it was fun to kind of engage with this totally new and kind of like total clean blank slate.
Unknown
That's cool.
Val Kroll
So.
Unknown
So I missed it maybe. But what are you optimizing for in the cookie banner? Yeah, like what's the actual metric that you're so the.
Mo Kiss
Some of the times when you're testing it's just about interaction with it altogether. So that as. As maybe new regulation comes in that affects it that people that they know how to get people to interact with the banner itself to either set your preferences or accept all cookies. But there was some other tests that we hadn't seen the results of by the time I left. So definitely reach out to Lucy over at Further if you want more information. But we were trying to figure out how could we assess its impact on brand trust. That was one that we were really interested in because with some companies with some sites like the cookies are the cookies like you have to. To be able to navigate. But because the banner has to be present if there's really no other leverage than like the limited levers that you have to pull, I should say that brand trust is the one that feels like the most meaningful way to invest in that. So that was One that was kind of like in the works.
Unknown
Yeah, very cool. Well, maybe I'll chime in with one here. It's a little bit out of left field, but I've been becoming really interested in experimentation in the realm of sports science this year. So, yeah, I've gone down a lot of rabbit holes on studies that have been done on how athletes should train to do better in endurance events. So like marathons, ultra marathons, cycling events. And there's been a lot of RCTs done a lot of research and experiments into it. And one area that I'm looking at is like the research into just training most of the time in a low intensity, really light. Like if you're, if you're training for a marathon just doing like 80% plus long, slow, light running and not just feeling like you need to work as hard as you possibly can, like every training session. And like, this is, this is personally relevant for me because I do this long distance paddling, endurance sport. Right. So I'm kind of the nerd in the team, like going out there and being like, okay, what is, what does the science say? And that's totally not the culture at all. Like, it's really like, people are very resistant.
Julie Hoyer
Trust the data. We should back up, you know, easy.
Unknown
Yeah, yeah.
Julie Hoyer
This guy.
Unknown
Yeah. So I'm just, I'm just the one nerd in the corner doing that. And it's, it's, you know, it's definitely not the culture to train in that way. But the thing is, our rival team this year kind of did that, like they put that into, into practice and just did all this really light training. And even within their team, I know there was a lot of skepticism and, and doubt and people were questioning it. And then we had our races and like, they smoked us. They completely smoked us. Whereas last year, you know, they didn't. So I know that's like, you know, n equals 1. But I still thought it was, I still thought it was pretty interesting to see, you know, applying some of these, these learnings from other sports and the exercise science that's being done and getting a good outcome. So anyways, maybe I can, I could push for next year to. Yeah, to listen to the nerds.
Tim Wilson
I was just glad to hear this wasn't associated with like a gambling thing or something. So that's good.
Mo Kiss
It's.
Josh Crowhurst
Josh, I find this really. When you discovered this, were you really surprised? I feel like this is counterintuitive to what I thought the research said about like doing short burst in intervals and like, yeah, it seems really. Is it just because it's endurance sport that it's different?
Unknown
Yeah, so it's, you sort of need to have both. And I'm, I'm definitely looking mainly in an endurance context, but I think this also applies to like, shorter events. But basically the idea is that you do the long slow as a way to build up your aerobic base, which is gonna help you basically absorb oxygen more efficiently into your muscles and clear the byproducts more efficiently. And then you still need to do that high intensity as a component of your training. Sorry, getting into the weeds a little bit.
Josh Crowhurst
No, I'm here for it.
Unknown
But, but you only need to do a little bit of it because your system, your system reacts really quickly to high intensity training. So you get those adaptations quickly, but it reacts really slowly to the long endurance training. You need to get that base, but it just takes way longer to do it so you can get away with, you know, just maybe leading up to the races, you incorporate more of that high intensity. So that's, that's sort of my understanding of it.
Julie Hoyer
I can say I've seen, I've been reading, I've actually been seeing a lot of articles about, about that. My challenge is that like, I'm like my, my, the range from my max effort to trying to go slow is like really, really small. So if I'm, if I' I'm like, wait a minute, oh, I'm supposed to slow down. I'm like, I'm already going pretty slow, you know, so, yeah. What do you do?
Mo Kiss
You already optimized, Tim, is what you're.
Julie Hoyer
Saying can't go much. Like, you can't really call it a, a run if you're walking.
Tim Wilson
My 80% looks like walking.
Julie Hoyer
Yeah.
Tim Wilson
Oh, that's a good one.
Val Kroll
Yeah, I love that one that I wanted to bring up is actually against the popular opinion as well. I had gone to one of the latest TLC Friday sessions and it was Georgy Georgiev and he was talking about the difference between observed power and observed MDE compared to like what you set when you're designing your test and how a lot of people will conclude because they, you know, the test ends and then they check their power and they say, oh, it was underpowered, we like can't use it and its conclusions. And he was pretty much saying that's not true and you shouldn't actually look at your observed power because it's different than the, the power that you use to like design your test. And I don't fully understand it. Enough to give you a nice synopsis here. But he has blog posts out and some LinkedIn posts about it, but it was really interesting. And he just said how it really changed his understanding and way he actually uses power and even, like, mde, and that a lot of people in practice, you know, are still relying on observed power to make conclusions, and that that is actually not a best practice. But I know that's not a popular thing yet. And I think a lot of people. It's going to take a while to see that change in the actual practitioners. And I was pretty shocked. And it was a really great session. So highly recommend. And I'll definitely be re watching it to try to wrap my head around.
Julie Hoyer
Can I share that you and I were slacking each other during that session? I just pulled that back up to look at it where I was like. You were like, yeah, this is hard. And I'm like, yeah, if you're not hanging on. I'm really not hanging on. That's okay, I'll ask. I've had no shame. I'll ask a question. And I asked the question, and the response was like, yeah, I'm gonna have to take that offline. Which I interpreted as being like, that was the dumbest question anybody.
Val Kroll
No, he said it was a really good question. He just needed time to, like, he wanted to think about his answer.
Julie Hoyer
I think that was kind of the. Oh, that's a really good question. You're adorable. Idiot. I'm not gonna. Yeah, I lost you, clearly, at slide two.
Unknown
Did you guys see those recently? Sorry, sort of tangentially related, but did you guys see those recent, I guess, memes about, like, statistical significance? And in some of the really terrible analysis out there, they're sort of classifying, like, p equals 0.10 as, like, borderline semi significant or, like, you know, extremely statistically significant for, like, a low value, like some. Some of these, like, really awful descriptions and kind of just the arbitrary nature of that.
Mo Kiss
Oh, my God.
Unknown
P equals 0.05. Just in general, maybe, like, I feel like I could.
Julie Hoyer
I'm not going to name them, but there are people who've already gotten triggered by that. Yeah, there are some very divergent schools of thought as to how useful it is to go down the path of parsing those versus not cattle worms. Yeah.
Val Kroll
Deep breaths, Tim.
Julie Hoyer
Save us, Michael.
Tim Wilson
All right, let's move on to another category. One thing most all analytics people are involved with is getting data out into the hands of people who need to use it, whether it be marketers or operators. Or things like that. What did we see this year in reporting? Like, what was our. What were the best things we saw there? It's not always the most glamorous part of the job, but it's necessary and crucial.
Val Kroll
Ooh, Michael, I'll start because you're gonna be really excited about this one. Okay, so if you all remember the great episode we had with Cedric Chen and it was about XMR charts, well, I have seen them in the wild for clients and they went over really, really well. Lucy, we already name dropped her earlier. She is just a rock star. She did it for one of our large clients that heavily relies on us for reporting. And it did a great job of showing, you know, variation like we talked about, and showing the client that, like, you're always going to have variation. Your KPI will move up and down. But, like, let's put a little thought around when we should really pay attention to it. So I know that they did a lot of like, education and change management with the client. They worked really hard on the different visualizations to kind of show those bands and how they were setting them and when their KPIs they cared about actually needed to be paid attention to. So I think they said it went really well with the client and it's been a great tool for them this year.
Tim Wilson
Awesome. And yes, I am a huge fan of XMR charts now.
Mo Kiss
Thank you.
Unknown
I have another one that's a shout out to another former guest from this year. So there was an article by Eric Sandosham recently, one of his weekly articles he's putting out, and he did one called the Joy of Business Reporting. So, yeah, counteracting that narrative of reporting is just something you have to slog through and get through. It's a necessary evil. Not the most exciting part of the job, but actually he made a few points about how it's a great way to gain deep subject matter expertise in how your business works and gives you an opportunity to link up with, like, people across the company that you otherwise wouldn't have a chance to talk to build your network. And I just thought it was a nice perspective to see, you know, what are actually the benefits that it'll bring to you as an analyst to. To do some of this instead of just sort of seeing it as okay, monthly reporting. Again, I got to go chase the stakeholders. The numbers are late. You know, that whole thing, you can kind of look at it in a different way and say, yeah, actually this is kind of a good opportunity, especially if you're, you know, Relatively early in your career. It's a great way to build some of those relationships and understanding. So I thought that was a nice article.
Julie Hoyer
I like that. I mean, it is, it's like the double edged sword. You can, you can go work with your business partner and have them just. I was, I was having coffee with somebody a little while back and she was like, oh my God. Like my, the head of marketing is just like, where's my dashboard? Where's my dashboard? And she's like, no matter what the context, whether it's a. She just perceives that everything that comes out of analytics and data science winds up on a dashboard. And this was driving this fairly senior person who's like managing the analytics and data science teams a little nuts because she was having to sort of push back. So that the flip side is saying, oh, if the reporting exercise is an opportunity for me to ask you questions about the business and what you really need, then that is an opportunity being mindful of. Don't turn into an order taker. Where they say, and I want to look at this and I want to look at this and I want to look at this and I need to slice it this way and that way. Like that's the challenge. But I do like that framing. I mean, anything out of Eric.
Mo Kiss
I was gonna say, I don't know if we're gonna do a category for best of matchmaking in 2024, but I will definitely take credit for the love affair that is Eric and Tim.
Julie Hoyer
But also like, I found him first and now I'm like, yeah, obsessed. Yeah.
Tim Wilson
As obsessed as we all are, interestingly enough, in this area for me this year, I think this is where I used AI was in using reporting and developing reports. I feel like this, yeah, like lots of other places I use AI. But like, in terms of like practical application to my work, it had the most impact in terms of a lot of times. And I'm not. I've been around a long time, so I've forgotten a ton of stuff that I used to know really well. And so going back into the weeds and like creating things and data, I'm a little rusty sometimes. And so AI was amazing to be like, okay, this is. I know what I'm trying to do and it'll just be like, oh, you just do this, this and this. And I'm like, oh, that just saved me like two hours of googling for an answer and I'm off to the races. And it helped me do things that I'd never done before that actually were really helpful because sometimes when you're trying to, like, think through a way to display data in a certain way, like, you're limited in different ways by, like, the tools you've got and what you're trying to show is not well supported by both the underlying data as well as the platform you're on. And so getting to a couple of really cool solutions and then being able to show that to clients and then being like, wow, how'd you do that? And be like, that's why you pay me.
Julie Hoyer
No big deal.
Tim Wilson
You know, even though it's like I was. Stayed awake half the night being like, I wonder if that's possible. And then AI could be like, I was. What I want to do in the AI is like, oh, I think if you did this, this and this, and boom, you're, you're, you're off and going. So that was like the cool thing for me this year. It was sort of like leveraging AI to compress some of the work and help me to remember stuff I used to know how to do better.
Julie Hoyer
So using it in the context of kind of the execution, the development of the report.
Tim Wilson
Yeah, building stuff.
Julie Hoyer
Yeah, interesting.
Josh Crowhurst
I funnily enough had the same thing, but I actually put it under analysis. So one example, I had someone in my team the other day who I'd asked them to like, run a query. It was a bunch of data sources I'm not super familiar with because, you know, also I haven't written a line of code in probably two years and I was overseas, so I didn't want to like wake this person up because it was Australia time. And I basically was like, I just want to make sure that I'm like, it's the logic that I intended when I asked him to pull this data. And so I was like, hey, you know, chatgpt, here's the query. Can you tell me what this query is doing? Is this the calculation it's doing? And I was like, I'd never thought about using it that way, but fuck, it was a beautiful thing. Like, it basically came back and was like, this is how it's calculating this field and this is what it's doing. And I was like, great. I didn't have to bug one of my data scientists to be like, am I definitely interpreting this correctly? Because it was really complicated. And see, I would put that under analysis, though. Tim, what do you think that's under reporting?
Julie Hoyer
No, that could. I mean, that's kind of in the. I mean, those are both aspects of kind of supporting the, the development tasks, you know. Right. So like that's, that feels like to me, that feels like something that cuts across, like legit cuts across both of them.
Josh Crowhurst
Yeah, I suppose you could, because you could do the same thing, right. With your queries for reporting purposes.
Tim Wilson
Well, since we're talking about analysis, what is, what are some of the best things we saw in analysis this year?
Julie Hoyer
I'll start with like there's a whole movement and maybe this cuts across reporting and analysis as well. That, that was like some of the worst thing I saw, which is all of the hype around, you know, you point AI at your data, throw your data at AI and insights will emerge, which, I mean being the inbox manager for our inbound pitches, there are certainly plenty of people who would love to fill your ears, dear listener, about how their AI solution is going to generate insights. So that was very, very off putting because it feels so wrong. But the flip side, and I'll credit seeing Jim Stern at MeasureCamp Austin, seeing Jim again at Marketing Analytics Summit, John Lovett at Adobe Summit. I saw him then again at measurecamp Chicago. And both the really pushing for using generative AI as kind of an ideation companion to think about hypotheses, to like be a smart companion to. You know, it's kind of like the like rubber ducking on like super, super steroids to say I can have an exchange where I am forced to be in conversation and I am encouraged like using prompt engineering as a, as a way to get to ideas for analysis, to get to hypotheses to think through, to kind of, you know, think a little bit more broadly. That to me seems really, really useful. And even with like John sort of building GPTs, you know, specifically around training them how to be good ideation companions was, was pretty exciting.
Josh Crowhurst
Well, I think, I think fundamentally like I was saying to someone the other day, I feel like ChatGPT is changing my job and it's making me better at my job in some places and worse at my job in some places in some cases. Like yes, it is definitely the, you know, ability to come up with a list of hypotheses or what often happens with me is like I will do all the thinking, get 90 of the way there. It's the last 10 that I really struggle to do. And so I've been leveraging that to do the last 10% very, very effectively. But yeah, I don't know, I feel like I use it across the full stack. So like companion bounce ideas off, ask stupid questions. That's probably my favorite one I'll be like, explain this thing to me. And previously, like, I probably would have struggled to find a simple way to do that.
Julie Hoyer
And you keep saying ChatGPT, is that like your, is that your one go to or do you.
Josh Crowhurst
Yes, that's. Well, I use that because we have enterprise and so we can put confidential information in there. So it makes a really big difference when you can like upload a data set and be like, hey, the other one that I loved so much. We had this Slack thread the other day and we were like doing an investigation into a metric that had declined. And everyone's like, what's going on here? And of course we have all the like, Brisbane best brains in the business and you have all these senior data scientists, like updating the channel of what they've looked at because it's cross, you know, cross company, there's all these senior leaders in it. And I, I caught up on the thread, like, I don't know, seven hours later, and I was like, what the fuck is going on here? Like, I can't tell what we've looked into, what we haven't looked into, what's next, like, where are we at? And I basically just copy pasted the entire CH into ChatGPT and was like, give me a summarized version of what we've looked at, what we've ruled out and what's still outstanding and who should follow it up. And it was beautiful. Like, I mean, so good, so succinct. And then I pasted it back in the channel and was like, hey guys, here's where we're at. And everyone was like, again, chef kiss. You know, as this way to just like keep everyone on track almost without anyone having to digest all that information and do the summary. But I, I think the, the point that I made that I probably don't, that I take for granted is like having an instance where you can put, you know, confidential company information is, is a real game changer, right?
Val Kroll
Absolutely.
Julie Hoyer
Yeah.
Tim Wilson
Yeah.
Josh Crowhurst
I'm just not gonna talk anymore because every time I do it's like stone cold silence because I'm making really shitty points and I'm being super annoying. You are not.
Unknown
Mine's just totally unrelated, so I couldn't think of a segue, but totally agree with what you said. I'm dying to have a. Our own instance of ChatGPT. Like, my company still blocks access to all gen AI applications despite having a gen team and coe in the company. So that's where we're at.
Josh Crowhurst
Do you know, do you know what my Founder did the other day. My founder the other day was like I think everyone just needs to like brainstorm this. And people were like writing on post its with pens. She's like no, no, no. Someone grabbed some like perma markers write in big letters and we, because we had like 30 people in this brainstorm, everyone threw it up on a board and she just took photos on her phone, uploaded it straight away and it summarized brainstorm. And I was like boom. It's pretty nice.
Mo Kiss
Some of those like virtual whiteboard tools like the murals and the miros have some of that built into where you can like like Canva and like canva didn't realize that one we use. Yeah, but you could like the summarize is super, super helpful. But there's other ones where you're asking it like you can draw a lasso around a certain ones and say like what's the theme of these? Or like you can have it do assistance. A couple things that it's pretty manual that definitely is a time saver especially if you're in a facilitation mode or you know there for a day long workshop with clients kind of thing.
Val Kroll
Mo you gave me some ideas of how to use it without having to put like client data in there. Because we obviously we can't do that.
Josh Crowhurst
Yeah.
Val Kroll
For privacy stuff. So. But yeah, more you have given me good use cases of how to use it for the ideation and the creative side but a little more applicable to the day to day because I'm like I don't get to be that creative in my job, you know. So thank you.
Josh Crowhurst
I'll keep throwing you because I just keep finding more and more ways. And I honestly was like, I think I've mentioned this example before but like we had another like metric deep dive where something had gone down and one of my stakeholders had put like what are the possible hypotheses for these in ChatGPT. And what came back was very good. Like and the thing that I really liked about it, it actually reminded me so much of analysis of competing hypotheses where it kind of like came up with 10 hypotheses. It was really structured. And then we actually I was like let's lean into this. And we used it for our analysis and we're like okay, we've ruled this one out. This one we've ruled it. Like we didn't take the same process exactly as analysis of competing hypotheses, but it definitely had that mentality of like okay, here are all the causes. Like let's disprove them. And the thing that I keep coming.
Julie Hoyer
Back to is, but why did you not. Why did you not tell it. Why did you not tell it to take an analysis, like an ach or ach?
Josh Crowhurst
I don't. That is interesting. And I will try it. The only caveat is I don't know if the conclusions it would draw about the different pieces of evidence would be sufficient. Plus, you would probably need so many varied pieces of evidence in there. But I'm gonna try it. I'll come back to you.
Val Kroll
Well, it kind of goes back to. That would be a little more like asking it to do some fact finding for you, which then you have to check. Like we always say, like, check it, gut check it. Which I think would be hard to your point, like asking it to do that. And then you'd have to go through the exercise to check it. Kind of compared to what you're doing, you are. You're doing more of the brainstorming, creative summarization, get you moving in the right direction stuff.
Tim Wilson
Nice. All right, last category. How about we talk about what we saw this year that we loved that was in this. In analytics strategy or strategy related?
Val Kroll
I. I have to start out with something that Val actually sent me in a very. At a very timely time that I needed to read this because, you know, you always have existential moments where you're like, what am I doing? Should I be more hands on? Should I stay more generalist? You know, we all have those.
Julie Hoyer
And then it's like, julie, get back on the podcast. We're trying to have a monthly call here.
Val Kroll
Yeah, exactly. But VAL sent me a quote by Adam Grant, and I feel like it perfectly summarizes what I hope we see more of in 2025. Like, people embrace this. The quote is, the hallmark of expertise is no longer how much you know, it's how well you synthesize information. Scarcity rewarded, knowledge acquisition, information abundance requires pattern recognition. It's not enough to collect facts. The future belongs to those who connect dots. And I was like, oh, my heart just, like settled my, you know, self talk where I was kind of spiraling. So I really loved that.
Tim Wilson
So like an analytics translator.
Julie Hoyer
Yeah, I mean, I'm.
Tim Wilson
I'm sorry.
Julie Hoyer
Like, I wasn't already. Like, I was like, the switch was 80. Triggered by this Anyway. Yeah.
Val Kroll
Really?
Mo Kiss
You're triggered by this by Adam Grant? That's all you had to say?
Julie Hoyer
No, I mean, it's stating that, like, that is not new. Like, that. That. That is not new. Like, there is.
Josh Crowhurst
I.
Val Kroll
Okay.
Josh Crowhurst
I was gonna say that. Tim. I was gonna say his perspective is not new, but I think he sometimes has a way of packaging an idea that makes people, I don't know if I want to say, believe in it, but makes people want to follow it forward, like, in a really concise way. So, like, that's the. That's the value add.
Julie Hoyer
Okay.
Tim Wilson
Anyway, I love that. I'm glad it was hopeful because a lot. A lot of people, like, I don't think that what you were grappling with, Julie, is uncommon at all. Like, as people progress, they're trying to figure this stuff out. And with everything going on in our industry, with what we're just talking about with generative AI, like, there's all these things that people are trying to say, like, what is the shape of my career? And so it really is helpful to get guidance or guideposts from people that like that.
Julie Hoyer
But you know what's going to happen is a bunch of people are going to say, sweet. So I can just take all this stuff and throw it at generative AI and tell it to connect the dots for me. And I found the shortcut. I mean, okay, well, then you can.
Mo Kiss
Swim on past them.
Tim Wilson
So, Tim, none of us here are gonna do that. We're all gonna come to you. Okay? So don't worry about it.
Val Kroll
But I think it's nice to show that, like, also, there's been a lot of talk in the. Not a lot of talk in the industry, but there's definitely been a change, right. Of, like, especially in being, like, a consultant. Being able to deliver a point solution for a very specific pain point is valuable. Sure. But there has definitely been a bigger demand, right, from clients of, like, now that. That's like, table stakes. As I know, Tim, you're gonna say it should have been all along, but, like, there is a bigger push of them wanting bigger pictures painted for them, more guidance, help them connect the dots. Like, they are now feeling the pain of, like, I have so much stuff going on, I don't know how to make it work together. They still want to grab onto. We'll switch the tool to what I'm comfortable with. But then they do that and they're not getting right the the outcome they were expecting or the benefit fit from the swapping the tool that they thought. And so, like, this quote to me was also, hopefully, to Moe's point, like, if more people are seeing this and there's a critical mass of people understanding that, like, it's making everything you have work together and seeing the Bigger picture. Like, one, the industry's feeling the pain. I think they're pushing towards wanting more strategy. And two, I thought this did succinctly say, like, that I could pass around to my colleagues at work and say, like, hey, us thinking this way, like, yes, it's painful and it can be hard sometimes, but, like, we're on a great learning curve and it is, like, way more valuable than classically, like, the great work we've done. Like, yes, we're still going to do that and need that expertise, but, like, we do have to upskill in this area. And so I think that's why it was so encouraging.
Josh Crowhurst
The thing, though, that I would add, which I was going to say earlier, is that, like Tim said, I don't feel like that's new. Like, I feel like I. That kind of, like, working to connect the dots is kind of a core data scientist. Like, that's what makes a good data practitioner. Right. And I feel like it's still a work in progress. And what can sometimes be attention particularly is that, like, I feel the industry right now. I don't know about anyone else, but I feel like our foot is down and we're moving so super fast that it almost feels like a bit of attention has been taken away from that core skill and we need to, like, refocus. I feel like I still work on this with my team all the time. Like, it's not yet. I don't feel that way there.
Julie Hoyer
I mean, because my, my concern is that this goes to say, this goes to our business partners and now they've just been given another cudgel to say, well, this is great. You gave me what I asked for. But you, But I need you to really, like, connect the dots and I really need you to do the pattern recognition or they throw it. I mean, I mean, it's. I'm not, I'm not opposed to, to the idea, but like the, the data practitioner who is constantly coming up short from the expectations of their business partners, like the critique, I mean, it is, it is a partnership that there's. To me, there's more about communication and partnership and deeply empathizing with and understanding what our business partners. Partners really need is a lot, way more important than where analysts a lot of times want to scurry off and say, let me just keep digging into the data and let me just keep finding something. So I think the danger is how somebody parses it because if they, if they read too much into, you know, synthesizing the information, they're like, well, I just got to get more and more data and I've got to synthesize it it. And then I'm going to come back with, look, I found this relationship between these two things that are completely divorced from the actual, you know, business context. So I'm not just trying to shit all over it, I promise.
Val Kroll
No, no, but it's just interesting. The way you're reading it is definitely in a different light than I was. And I think some of it is like, the space I was in. I was primed to read it and take it completely differently than you're taking it. So it's not to say like, yours is wrong and I disagree. I agree with what you say. It's just interest. I didn't have that point of view when I first read it.
Tim Wilson
I guess I like it.
Josh Crowhurst
Okay, controversial question. How important is data strategy right now? Like, I made that comment about things moving as fast. Like, I feel like it's moving as fast as possible. I feel like there are a lot of documents sitting on shelves somewhere. I don't know. Someone. Someone said something to me in 2024.
Julie Hoyer
And I had this, like, you mean this year?
Josh Crowhurst
Yes, this year. But I am already in 2025. My mind is in 2025. It has been for the last three months. But someone said something to me which was, oh, well, the data strategy should just connect up to the overall company goals. And I was like, well, fuck, that's obvious. I think that's harder to do in reality sometimes. But I guess I'm just having this, like, existential crisis of, like, what is the purpose? And, like, and I'm thinking when I say data Strategy, they're like 12 to 15 page document that probably does sit on a shelf. Like, is it as useful as it used to be with the pace that we're at? Like, I don't know, like, is it more about the behaviors and the ways of working that are important or the concepts and less about the how?
Tim Wilson
At times like these, Mo, I like to think about the Canadian band the Arrogant Worms and the song they sang called Star Trekken across the Universe. Always going forward because we cannot go in reverse. And I think when we're going so fast, in my opinion, that is when data strategy is actually even more crucial because we're going so quickly and we have to respond so quickly. We do have to have a good strategy for where we're trying to go, or else we'll get pulled 45 different directions and end up nowhere.
Julie Hoyer
Counterpoint. Okay, to put me in Moe's Camp. Yes, I think, I think most if you say data and I struggle with, I mean I've asked what is a data strategy? Because I, I don't even know so and I, I've seen it defined different ways. One of the more recent ones in discussion with a company, the data strategy is like where, what data are we going to have? How's it going to be hooked together? What are we going to gather, what are we going to collect? How are we going to manage it? And nine out of 10 of those come down to, okay, here's our strategy. We've got to spend the next 12 months getting, getting all the data hooked into these. Just we need the whole year to kind of execute it, which I think that that's where they default to and they'll wave the flag of AI and all the data has to be super clean. And so there is a tendency, I think, I don't know Mo if I'm articulating the same thing you're saying there's a tendency to over index towards collecting, getting all the stuff and getting all the process in a good place with this, this belief that you've got to build this strong foundation. And that means the next year then we'll be off to the races and the generative AI will be so much more powerful and that's why it winds up as opposed to being more nimble.
Tim Wilson
What's interesting, Tim, is what you just commented on is actually the execution on the strategy being too slow, not the strategy itself.
Josh Crowhurst
I think, I think both are true. But we throw truth.
Tim Wilson
Yeah, but I think that's the, the issue you have there is sort of like, well, how do we execute on the strategy effectively? I, I mean, and it does. Your strategy, I think Mo, to your point, does have to fly back to what is the business trying to do. One of the things we did this year with one of our clients was, you know, as you're going along helping companies do stuff, you get to an inflection point and you start, you get a chance to actually say like, hey, does our business better, better serve by like altering our strategy? And then what are the steps we can take now with what we've got? And then how do we need to adjust out into the future to build that strategy forward? And like that's work we are doing. But I, I tend to take personally a much more iterative approach which is sort of like, okay, if this is where we see the puck going, don't be like, I need 12 months to get everything ready to be perfect. The Boil the ocean. Work is never right. It's always sort of like, here's where we're at and here's the five steps we can take in the next 90 days that are going to push us a little closer.
Josh Crowhurst
And I think, sorry, but when you write a strategy, you don't run a 90 day strategy. It's typically one to five years minimum.
Tim Wilson
I didn't say 90 day strategy. I said here's where we're at today and to get to our strategy, here's what we're going to do now and then across the next so that we're not taking forever to get there. Because you gotta be incremental about it or iterative.
Julie Hoyer
You.
Tim Wilson
I just don't think it works to try to do the Big Bang every time. Sometimes you have to do it that way, but it's. I don't think it's effective every time.
Josh Crowhurst
I don't think we do the Big Bang. I think the problem is exactly what Tim said where we said, we're gonna spend the next 12 months and we're gonna get everything up to scratch and everything's gonna be perfect. And then 12 months later, a bunch of shit happens in the business. You got a bunch of new tools, a bunch of other shitty data, and you're like, we're gonna spend the next 12 months making everything perfect because if it's perfect, we can help the company. Company achieve their user and revenue goal. And you're like, no one gives a shit.
Tim Wilson
Yeah, but that's, that's not strategy, that's execution.
Val Kroll
I was gonna say. I definitely saying strategy. I have to, I just have to give you the context. I definitely was not meaning like data collection strategy. I was definitely talking concepts and the other things you mentioned, Mo, just, just to put that out there. I can't go on the record people thinking I was talking about the collection side because I was.
Josh Crowhurst
I definitely wasn't talking about collection, but I think that's where we landed.
Julie Hoyer
But, but I think I've. I've learned this. I mean, I'm finally starting to understand that the generally accepted thing is like data versus analytics, that the data. Data tends to be the collection, the piping, the governance, the management. But at the same time, I worked with somebody for years who, when he would do a data strategy, he meant more. Julie, what you meant. So there is just. Yeah, it's exhausting. This starts to feel brutal.
Tim Wilson
It sounds like a podcast episode, but. All right.
Josh Crowhurst
This is. Sorry, just to round it out though. This is why I think it comes back to maybe what the direction I need to go in is less about the what are we trying to achieve and like what are the behaviors and ways of working we want to use to get there and to support the business. And I don't know if those. Those are the same things or different things, but that's kind of what's been rolling around in my head is like, what do we want to stand for as a team? And anyway, yeah, this is. We do need a whole episode.
Tim Wilson
Totally agreed.
Mo Kiss
So I'm going to take us on a little bit of a left turn for my good observation for 2024. Hopefully a little less contentious. We shall see at this point.
Tim Wilson
Don't count on it. That's right.
Mo Kiss
I feel like the jumping on happens later and later in the episode. So I am a little nervous. No, I had the opportunity to attend.
Tim Wilson
Okay.
Mo Kiss
Well, you were there, so you know that I was there and there were a lot of great presentations. I really was impressed overall. But there was one in particular by Noam Levinsky, who was the CPO of Grammarly and his presentation was a little provocative. It was have LLMs killed Grammarly? And so it was just super, super interesting. And I have the recording in the slides that we'll definitely link in the show notes if you're interested. But one thing that really has stuck with me ever since he presented it is thinking about the question, has the problem that you're working on truly been solved for your customers and your users? So the example that he gave to kind of like illustrate this point was back in the day, I GUESS in the 70s and 80s, there were these things called Thomas Guides, which were essentially like these paper, like little manuals that had like maps and places you could go so that if you were out of town, it was one of the best ways to like navigate the city. And Thomas Guides were kind of outrun by MapQuest in the 90s, because now you could just type in where you wanted to go and print out your directions. And that was considered the solve. But he thinks that where we are today with Gen AI is the MapQuest stage, because we all know what comes after MapQuest, which is the smartphones and the constant access to Google Maps. And that that's interesting to think about Gen AI just being at MapQuest. But the part that really sticks with me is he was saying actually having Google Maps on your smartphone or in your car, like always accessible, still isn't the solve that if you're really obsessed and focused on, if you're actually solving the problems that that would be self driving cars because that's what gets rid of the need for navigation altogether. And so like that's actually where you get there. So he was kind of giving some examples in like the product sense, but thinking about how to be really deeply connected with the problems that you're solving. I've just found lots of, lots of opportunities to share that story and applications of, of the way of analyzing just really deeply understanding the problem that your users and customers are facing. So that was definitely a takeaway.
Julie Hoyer
So in a data or analytics strategy context, it's really getting to that perfect dashboard.
Val Kroll
Right?
Julie Hoyer
That's really where we land.
Mo Kiss
There's the takeaways.
Tim Wilson
Awesome. All right, well, this has been sort of the intent, but we did pretty good. And it's also nice to see that after 10 years we're 100% on the same page and aligned on everything analytics related. So I guess we can't quit doing the podcast yet. We still got work to do. So 2025 is looking to be a good year, I think. Thank all of you, Mo and Julie and Josh and Val and Tim. Thank you for being on the podcast with me and doing this show together. It's always not only enlightening, but fun.
Julie Hoyer
And thank you.
Mo Kiss
Thank you too, Michael.
Tim Wilson
Oh, that's what I was looking for. Thank you.
Julie Hoyer
I'm gonna leave that hook out there for a while to see if anybody was gonna bite anybody.
Tim Wilson
But I think as you're listening, you know, maybe you've got things you're thinking about in 2024, learning things that you want to take into 2025 or things in year 11 of the podcast. You're like, you haven't talked about this enough. Sounds like we learned today we're going to talk about data strategy just a little more, but there's probably lots of other things and we would be on.
Julie Hoyer
The list for a while.
Tim Wilson
It has.
Josh Crowhurst
I have had that on the list for like three years. Have it be noted.
Julie Hoyer
You have? Yep, yep.
Tim Wilson
But it's not important, Mo, because we're too fast paced taste so.
Julie Hoyer
And I maintain the list that backs up your. Your contention.
Tim Wilson
So anyways, but we would love to hear from you. There's a bunch of great ways to do that. The measure Slack chat group is one of them. Obviously you can email us at contact analyticshour IO and so please feel free to reach out. We actually really enjoy hearing from listeners and as we hear from you, we do incorporate what we hear into our show topics and to our guests and things like that. So we do appreciate It. You know, no show would be complete without a huge thank you to Josh Crowhurst. I know, Josh, you're here.
Julie Hoyer
So he's got his work cut out for him on this one.
Mo Kiss
Josh can say, you're welcome.
Unknown
You're welcome.
Tim Wilson
Don't mean to make it awkward, but we really do appreciate you. Thank you very much. And I think, you know, this has been 2024 has been a very interesting year. A lot of learning, a lot of growth, a lot of change. And I think that's always the case in our industry. And I think I speak for all my co hosts when I say, no matter what 2025 brings in the next 10 years of this podcast, remember, keep analyzing.
Michael Helbling
Thanks for listening. Let's keep the conversation going with your comments, suggestions, and questions. On Twitter @nalyticshour, on the web at analyticshour.IO, our LinkedIn group, and the Measured Chat Slack group. Music for the podcast by Josh Crowhurst.
Julie Hoyer
So smart guys wanted to fit in, so they made up a term called analytics.
Tim Wilson
Analytics don't work.
Michael Helbling
Do the analytics say, go for it no matter who's going for it. So if you and I were on the field, the analytics say, go for it. It's the stupidest, laziest, lamest thing I've ever heard. For reasoning in competition.
Julie Hoyer
You should try.
Tim Wilson
To troll them on Blue Sky, Tim.
Julie Hoyer
Well, they're not on Blue Sky. I went to where they were.
Josh Crowhurst
This is Blue Sky.
Tim Wilson
I'm not the new social network for liberals who don't play Twitter anymore.
Josh Crowhurst
I don't want another network. I don't want any networks.
Tim Wilson
I know.
Julie Hoyer
It's a. It is like Twitter was in, like, 2010. It's delightful. I love it.
Tim Wilson
Want to kind of wing that? Or, like, do we want to just sort of, like, pick what order we want to go in ahead of time?
Val Kroll
Sometimes, Michael, I know which way we're gonna go. We're gonna wing it. That wasn't really a question.
Julie Hoyer
I feel like we've been doing this for 10 years. One's always been winging it, one's always over planning.
Val Kroll
It is being wong already. We are wonging it.
Tim Wilson
And that's number wing. Okay? Suffice it to say, it will be one.
Val Kroll
It will be one.
Mo Kiss
Well, let's put it this way. If we go in order and we don't get to all of them, I might not have anything to contribute except for. Are you serious?
Tim Wilson
Wow.
Mo Kiss
When other people say theirs so well.
Tim Wilson
I'm hoping, Val, you can get a few. Do you mean to tell me, Tim, you spent this long, and then.
Mo Kiss
People were still confused.
Julie Hoyer
That's right.
Josh Crowhurst
Well, fuck, guys, I tried. Was it really bad?
Tim Wilson
No.
Val Kroll
No. I thought Tim was gonna jump in, so I was being polite and waiting. Do you want me to go through?
Tim Wilson
No.
Julie Hoyer
Like, literally, I will always die. Like, I always have something to say, but I can't be like, I'm not gonna do it this time.
Val Kroll
All right, I'll do it. I'll do it. I'll do it. I'm up.
Mo Kiss
Rough start.
Josh Crowhurst
Poor Josh. He's gonna be like, this is the worst show to edit of my life.
Unknown
It's fine. Every show that I'm on is the worst show to edit because I have to listen to myself.
Julie Hoyer
That's an outtake right there.
Josh Crowhurst
That should be an outtake if it's not in the there.
Val Kroll
Well, no. Who?
Tim Wilson
Yeah.
Val Kroll
All right.
Tim Wilson
Rock Flag.
Julie Hoyer
And it's the Data Strategy Power Hour rebrand.
Mo Kiss
W.
Julie Hoyer
Good one.
The Analytics Power Hour: Episode #261 – 2024 Year in Review
Released on December 24, 2024
Hosts: Michael Helbling, Moe Kiss, Tim Wilson, Val Kroll, Julie Hoyer, and guest Josh Crowhurst.
The episode begins with Tim Wilson reminiscing about the podcast's journey since its inception on January 3, 2015. Celebrating 261 episodes and multiple live events across continents, Tim expresses gratitude to listeners, co-hosts, and guests for making the podcast a staple in the analytics community. Michael Helbling welcomes listeners, setting the stage for a comprehensive year-in-review discussion.
Notable Quote:
Snowflake Advancements: Josh Crowhurst highlights Snowflake's innovative features, particularly the Snowflake Feature Store and Snowpark. These tools have streamlined the creation of feature stores for machine learning models and allowed analytics engineering teams to program in multiple languages, significantly enhancing data pipeline efficiency.
Notable Quote:
Cross-Group Partnerships: Val Kroll discusses improved collaboration across different teams and practices within client engagements. This holistic approach has enhanced implementation processes, especially in integrating best practices from paid media with reporting tools like Adobe.
Consent Management Challenges: Tim Wilson shares insights into consent management issues faced by clients, emphasizing the importance of accurate data collection in compliance with evolving regulations. Highlighting Charlie Tice's expertise, Tim underscores the industry's ongoing struggle with effective consent management.
Consent Banner Optimization: Mo Kiss details a project focused on optimizing consent banners to enhance user interaction and brand trust. By experimenting with elements like button color and transparency, the team achieved significant improvements in user engagement without compromising ethical standards.
Notable Quote:
Sports Science Experimentation: Josh Crowhurst brings a unique perspective by exploring experimentation in sports science, particularly the benefits of low-intensity training for endurance athletes. He relates these findings to analytics by emphasizing the value of evidence-based approaches in performance optimization.
Generative AI in Experimentation: The discussion shifts to the integration of generative AI in ideation processes. Hosts commend the use of AI as a collaborative tool for hypothesis generation and brainstorming, enhancing the analytical workflow.
XMR Charts Adoption: Val Kroll introduces XMR charts, praising their effectiveness in visualizing data variation and helping clients understand KPI fluctuations. Lucy's implementation of these charts for a major client has been notably successful.
Notable Quote:
Joy of Business Reporting: Josh Crowhurst references Eric Sandosham's article, "The Joy of Business Reporting," which reframes reporting as an opportunity for analysts to gain deep business insights and build cross-departmental relationships, rather than viewing it as a mere obligatory task.
AI in Reporting: Tim Wilson and Josh Crowhurst discuss the practical applications of AI in report development and data analysis. From generating complex reports to summarizing lengthy discussions, AI tools like ChatGPT have become invaluable in enhancing efficiency and accuracy.
Notable Quote:
Generative AI for Hypothesis Generation: The hosts explore the role of generative AI in supporting analytical tasks such as hypothesis generation and data summarization. Josh Crowhurst shares experiences of using ChatGPT to distill complex data discussions into actionable summaries, highlighting the tool's potential in streamlining analytical workflows.
Challenges with AI-Driven Insights: Julie Hoyer voices concerns about the overhyping of AI for insights generation, cautioning against relying solely on AI without contextual understanding. She emphasizes the importance of using AI as an ideation companion rather than a standalone solution for data analysis.
Notable Quote:
Synthesis Over Knowledge Acquisition: Val Kroll shares an inspiring quote from Adam Grant that underscores the evolving nature of expertise in the age of information abundance. The focus shifts from mere knowledge acquisition to the ability to synthesize information and connect disparate data points.
Notable Quote:
Iterative vs. Traditional Strategy Execution: Tim Wilson advocates for an iterative approach to analytics strategy, emphasizing the need for flexibility and adaptability in a fast-paced environment. He contrasts this with the traditional, often rigid, long-term strategic planning, arguing that incremental steps can lead to more effective and responsive outcomes.
Conflict Between Strategy and Execution: The discussion highlights the tension between developing comprehensive data strategies and the practical challenges of executing them in dynamic business environments. The hosts agree on the necessity of aligning data strategy with overarching business goals while maintaining agility.
Notable Quote:
Deep Understanding of User Problems: Moe Kiss shares insights from Noam Levinsky’s presentation about deeply understanding and solving user problems. Using the analogy of navigation tools evolving from Thomas Guides to smartphones, Moe emphasizes the importance of continually advancing analytics solutions to truly address user needs.
Notable Quote:
Emphasis on Communication and Empathy: Julie Hoyer stresses the critical role of communication and empathy in analytics, cautioning against analysts becoming mere data order-takers. She advocates for meaningful partnerships where analysts actively engage with business partners to deliver contextually relevant insights.
Encouragement for Continued Learning: The hosts conclude with an optimistic outlook for 2025, expressing enthusiasm for ongoing growth and development in the analytics field. They encourage listeners to stay engaged, continue learning, and leverage the shared experiences to enhance their own analytical practices.
Closing Remarks: The episode wraps up with expressions of gratitude towards all contributors and listeners, reaffirming the podcast's commitment to fostering insightful and enjoyable discussions in the realm of digital analytics.
Notable Quote:
This episode of The Analytics Power Hour offers a comprehensive review of 2024's advancements, challenges, and learnings in the analytics landscape. From innovative data pipeline solutions and ethical consent management to the strategic integration of AI and the evolving nature of analytics strategy, the hosts provide valuable insights and practical takeaways for professionals aiming to enhance their analytical prowess in the coming year.