← The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
#654: Translating research into cross-functional strategic change with Adam Hagerman, Indeed
00:19:26
Loading summary
Transcript72 lines
- [00:02]
Doug
And Doug, there's nowhere I wouldn't go to help someone customize and save on car insurance with Liberty Mutual. Even if it means sitting front row at a comedy show.
- [00:12]
Host
Hey everyone, check out this guy and his bird.
- [00:14]
Advertiser
What is this, your first date?
- [00:15]
Adam Hagerman
Oh no.
- [00:16]
Doug
We help people customize and save on car insurance with Liberty Mutual together. We're married. Me to a human, him to a bird.
- [00:22]
Advertiser
Yeah, the bird looks out of your league.
- [00:24]
Doug
Anyways, get a quote@libertymutual.com or with your local agent.
- [00:27]
Advertiser
Liberty Liberty Liberty Liberty At Strayer University we help students like you go from Is it possible? To Anything is possible by offering access to up to 10 no cost gen Ed courses so you can reach your goals affordably and fast. Visit Strayer. Edu to learn more. No cost Gen Ed is provided by Strayer University affiliate sophia. Eligibility rules apply. Connect with us for details. Strayer University is certified to operate in Virginia by Chev and has many campuses including at 212150 in Arlington, Virginia the Agile brand.
- [01:13]
Greg Kilstrom
Welcome to Season seven of the Agile Brand where we discuss the trends and topics marketing leaders need to know. Stay curious, stay agile and join the top enterprise brands and martech platforms as we explore marketing technology, AI, e commerce and whatever's next for the Omnichannel customer experience. Together we'll discover what it takes to create an agile brand built for today and tomorrow and built for customers, employees and continued business growth. I'm your host Greg Kilstrom, advising Fortune 1000 brands on martech, AI and marketing operations. The Agile Brand podcast is brought to you by Tech Systems, an industry leader in full stack technology services, talent services and real world application. For more information go to teksystems.com to make sure you always get the latest episodes, please hit subscribe on the app you listen to podcasts on and leave us a rating so others can find us as well. Now onto the show.
- [02:10]
Host
We are recording live at Qualtrics X4 in Salt Lake City and seeing and hearing all about how to create and enable amazing customer and employee experiences. It's important to collect customer experience data, but if it's not driving change across your organization, is it really helping your business? Today we're going to talk about making meaningful cross functional change using CX research and data as a guide. I'm joined by Adam Hagerman, Director of UX Research for Employer Products at indeed. Adam has led transformative efforts at Indeed to turn customer experience research into cross functional strategic changes driving real improvements in both user satisfaction and product success. Adam, welcome to the Show.
- [02:51]
Adam Hagerman
Thanks for having me.
- [02:52]
Host
Yeah. Looking forward to diving in here. Before we do though, why don't you give us a little background on yourself and your role at Indeed.
- [02:59]
Adam Hagerman
Sure. I lead a team of UX researchers. We look over the employer products. We're trying to make sure that what we end up shipping for people to consume is solving relevant needs and helping them do what they need to do. Better, faster, cheaper, easier. Wonderful.
- [03:14]
Host
Great. So, yeah, let's dive in here. And so we're going to talk about a few things here, but I want to start by talking about transforming satisfaction measurement into strategic decision making. So you and your team at Indeed have transformed your approach to measuring user satisfaction. What led to the shift we needed to.
- [03:35]
Adam Hagerman
Yeah, measurement is satisfaction. Measurement is not new. We've been doing it since like phone surveys from ye olden days that you would get at dinner time. And the tool we were using was the same one, the Net Promoter Score. It's evolved, it's iterated over time. It's had improvements here or there, but at the end of the day it's a brand measurement and we have a product we need to work on. NPS is well known and my stakeholders were very excited. They're not anti user sentiment, it's just the tool they were using wasn't as helpful as it could have been. We ask the question, can we make this better? What can we do? Here are the shortcomings. Here's how it's preventing us from helping people do what they need to do better, faster, cheaper, easier. It's not giving us the insight we need. So let's find a new way to do it. And the intention of finding the new way to do it is to actually help our stakeholders build better, faster, cheaper, easier products.
- [04:27]
Host
Yeah.
- [04:27]
Adam Hagerman
Yeah.
- [04:28]
Host
So along those lines, I mean, probably, you know, many people listening out there are using nps, csot, you know, you name it. What were some of the telltale signs that it wasn't giving you everything that you needed?
- [04:40]
Adam Hagerman
If it tells us to push a lever and we push the lever but nothing happens, it's not actually telling us what lever to push.
- [04:48]
Host
Makes sense.
- [04:48]
Adam Hagerman
I guess that's the answer.
- [04:49]
Host
Yeah.
- [04:50]
Adam Hagerman
Yeah.
- [04:50]
Host
Hey, so you mentioned that data was harnessed not just to inform, but to quantify impact and to guide strategy. How did you approach turning research into something measurable and actionable for the business?
- [05:07]
Adam Hagerman
Research is the process of collecting information. The reason we collect information is because we need to make a decision. The product stakeholders need to make a decision. Do we do it this way? Do we do it that way they receive information from lots of resources, they get feedback from their go to market team, they get feedback from the engineering team, they get feedback from random person on the street, and they have to take all of that information and make a decision. What we bring to the table is kind of the collective baseline for what our users want. Our job is to advocate for users among that entire organism of ecosystem of information floating around. Data collection is a deliberate act. Just because something's been collected doesn't mean it's what you should be collecting. And we ask that question, are we collecting information that actually helps us advocate for users? Once we were able to do that and demonstrate, here's what we're doing and here's what it means for you, here's your return on investment, it was an easier case to make. Does that answer your question?
- [06:07]
Host
Yeah, I mean, I think so. I mean, so is it because there's lots of signals, right? So I mean again, there's some go to measurements that a lot of people use like NPS and others. And again to your point, nothing wrong with that, but if it's the sole measurement, there's just some questions, right? So yeah, So I guess how do you, how do you determine, is it an incremental, like in the advertising world, it's like media mix modeling or something like that. Is it a similar approach?
- [06:33]
Adam Hagerman
I actually used a very similar approach to media mix modeling. In media mix modeling, what you're really trying to do is of all the money that's floating around in brand advertisements, activation, whatever you're trying to see, where does an incremental dollar give me more benefit? So if I have $1 left to spend, am I going to put it in this thing or that thing? And what my measurement was trying to do was say of the experiences available where we could put our investments into improving user experiences. Where should you put your dollar? And we had to do a mathematical exercise, create an empirical argument for that's the right place to put the dollar. And then we had to hope that it worked, that we weren't lying.
- [07:18]
Host
Right?
- [07:18]
Adam Hagerman
Right. So we created systems of accountability for ourselves. In order for us to advocate for this new type of measurement, it needs to meet these criteria. And we set out a protocol for how we were going to check ourselves. And before we were ready to really roll full scale and say this is truth, listen to us, we wanted to make sure that we were actually representing the lived experience of our people, the customers that we have. So we took that medium mixed modeling approach. How can we model where Return on effort into fixing user experiences would give us that outcome.
- [07:55]
Host
So what was that process like then?
- [07:58]
Adam Hagerman
Lots of math.
- [07:59]
Host
Yeah, I would imagine. Right. What's the process then of convincing people to listen to. You know, again, people get really stuck in their ways. This is a change management thing as much as it is a measurement thing. Right.
- [08:13]
Adam Hagerman
You hit it on the nail or you hit the nail on the head. It's a change management thing. People are coming in with their own set of expectations, biases, baggage. This is what it means for me, either in good or bad terms. And the approach I like to take is just being brutally honest, direct. This is what's going on. Here's how moving forward with this, here's what you can expect as a consequence. I'm not going to tell you what to do, but if you're going to do this, here's your consequence. I'm giving you other options. And by the way, I have data to back it up.
- [08:46]
Host
Yeah, I mean, that's the key thing. Right. Instead of a feeling or a hunch or I did this at this other place. You've got the incremental improvements and that's
- [08:56]
Adam Hagerman
the media mix modeling that I talked about. Yes. It's working in a theoretical and. Theoretical and hypothetical space. But we adhere to all the rules of statistics that have come to us through the current philosophy of science, and that's how we construct our argument.
- [09:13]
Host
Yeah. Yeah. So how did that go? I guess at first, like, is that asymmetric?
- [09:19]
Adam Hagerman
Asymmetric. There were some people who. Who saw what we were trying to do and they were like, yeah, let's go. Yeah. There were other people who were more. More. They had been burned in the past, I guess is the best way to say it. We work with very smart, intelligent, experienced people and indeed, was not their first job. So they have the baggage from wherever they were and whatever research team did that thing. So when they hear Adam saying, this is what we're going to do now, or don't do that or whatever, I have to acknowledge that they're also a human being that have their own set of experiences, baggage, whatever. And the connection I make with them or I try to make is we're both here to do the same thing. We both want the same good outcomes. What questions do you have? If there are things that I can do to make you feel more comfortable, I'd like to know what it is. It may just be a matter of I didn't say it on that slide. So it's having frank conversations.
- [10:17]
Host
Well, and this is where it comes down to, you know, there's lots of talk about data driven decision making but this is the culture shift part of that. Right. Is again, I did this thing at this other place and it worked really well. So it must, you know, work again at this new place. In different circumstances our context is different
- [10:36]
Adam Hagerman
and sometimes that's part of the argument.
- [10:37]
Doug
Yeah.
- [10:38]
Adam Hagerman
When I'm convincing people it's like, yeah, over there it worked and I can see why it would or I can also see why it would fail miserably. It's that idea of no context is exactly the same and the success depends upon your context and bringing that out. Having empirical ardents.
- [11:02]
Host
So how do you look at making arguments or not even argument? Let's not make it arguments, let's make it friendly. The quick win versus the sustainable growth. A lot of this is there can be quick wins when you make any change sometimes or any good change. But how do you look at balancing between let's get some of those quick wins under our belt versus okay, this is going to achieve sustainable growth which is hard to project.
- [11:32]
Adam Hagerman
I had the long term vision in my head. I knew what I wanted to do, I knew where I needed more information. Like here's where it might fail, but I had to start with the quick wins. It's just like any other product that you push out, your first users are going to be your best sources of feedback. They're going to say, okay Adam, here's how you and your team maybe thought about this differently. Here's feedback I would have because here's how my context will change how you want to approach something. So the quick wins are necessary but you have to know where you're going. The corollary to your question is people who do a bunch of quick wins but don't know where they're going and that's how you get cruft like that's what it is. So thinking about your research? I try to think about my research programs as accretive. Whatever we're doing today is building on what we did yesterday and we'll build on it tomorrow.
- [12:25]
Host
So yeah, so the quick wins are. And I'm a huge fan of that approach but to your point, as long as there's a lie. But it's almost as much about. I mean there's benefit to the business ideally there's benefit to the customer as well with those but it's also about of winning hearts and minds. Right. Is that that's a big part of it.
- [12:46]
Adam Hagerman
You have to show that it works, that if Somebody's going to say, hey Adam, I want your stuff on my surface because I want to use your tool to make sure that our user experience is good. I want to deliver on that. I want to actually say, yes, I helped you do that.
- [13:04]
Host
So what would your advice be to leaders? Let's say they're not sitting in the research or the data component of the organization. But again, they read the same things I read about data driven decision. They know somewhere in their head that there's a lot of value here, but they're having a hard time kind of getting past whether it's biases, whether it's other loud voices in the organization. What's your advice to them to kind of just make the first step?
- [13:31]
Adam Hagerman
Well, that's a deep question.
- [13:33]
Host
Yeah, I'm sure it depends too.
- [13:35]
Adam Hagerman
Well, Kierkegaard would say take your leap of faith, stare into the abyss and go. And I do take kind of a similar approach. I talk about that, the series of experiences, bag of lessons learned that helps us decide how we should move forward. We may not have all of the information we need to make a decision, but we have a lot of information that can help us know whether we're making a bad decision. Check yourself before you wreck yourself. Have your internal system of accountability and then at some point you just need
- [14:05]
Host
to go, yeah, yeah. I remember when I was at the media briefing the other day and you had mentioned, you know, indeed is unique to some organizations in that being a technology company, access to data, things like that is a little more prevalent than in some or maybe more legacy companies and things. How much do you credit, you know, having that access to data? Like is, is it always a good thing or are there good and bad things about it?
- [14:38]
Adam Hagerman
I think it's good to have access to it. There's a barrier to entry. You have to know how to type in the right query. You have to know what are the signals that maybe you didn't do the query correctly. There's a lot of self awareness that goes into working with that. The people who primarily work in that space, this is what they do all day long. So they have those tips and tricks to be conversant in that kind of work. I viewed, or I continue to view democratization of information as inherently a good thing. My concern comes with, do we have the appropriate guardrails in place? Have we checked ourselves before we wreck ourselves?
- [15:23]
Host
Well, a lot of yeah, and I'm a huge fan of it as well. But also data literacy and you know, all the things that as well as the Ethical components, of course. How do you make sure that there's kind of a unified version of the truth then, with all that? Because, again, it's a great thing to have access, but everybody's asking their own questions, so on and so forth.
- [15:45]
Adam Hagerman
I'm not the only person participating in this conversation. Yeah. And there are times where I know my data is not the data somebody needs at that point. I should respect that, because the data I have to offer is not the data they need at that moment. I try to think instead, how do I participate in that conversation? How do I add an additional layer, let people know that this layer exists, that they can use it to ask questions of their own? One of the important aspects of the program I talked about was that it didn't just live in Qualtrics, it was actually married with all of our other data. So people could do their own investigations and they could use our data source as part of. As evidence. In what. I know you don't want to use the word argument, but in whatever empirical argument you can use it. Yeah, I feel like I'm not answering your question.
- [16:37]
Host
Well, no, it's. I mean, it's kind of a broad question. You know, I think it's. It is. And I think this is where platforms and kind of. We do need some kind of unified source of truth. At the end of the day, this is about happier customers and longtime customers. And so at the end of the day, that's the goal. Right. So if there's many different answers to that, maybe there could be some challenges there.
- [17:06]
Adam Hagerman
I guess where I'm getting at is I don't want to preclude somebody from having access to this, but I do want to make sure that when people are accessing it, they understand what they're accessing. We do that through documentation. We say, this is what you'll find in this database, in this data frame. We have lots of information about what is this program? How can you use this information? Oh, do you have questions about how we came up to it? Look over here. We try to make sure people are empowered to use the data in the correct way or in the intended way. Correct makes it sound like there is a right and wrong in the intended way.
- [17:42]
Host
Well, and I would imagine in that. And empowering them as well. You're also getting some amazing ideas that one person with access centrally would not have.
- [17:52]
Adam Hagerman
This goes back to that idea of a creed of understanding. We don't have to be the only source of truth, but what we're doing is we're adding to our knowledge.
- [18:02]
Host
Absolutely. Love it. Well, as we wrap up here, just a couple questions for you. I know we're almost wrapping up the event here, but I wanted to ask what's been a highlight so far for you at Qualtrics X4?
- [18:16]
Adam Hagerman
I usually enjoy the day two keynotes. Day one is very product forward. Like look at the cool stuff we're doing. Day two is more how should you think about doing your work? And I always enjoy those reframings because when we're just going about our day, we do what we do and sometimes we need to stop and listen to how do other people do it? How are they approaching this existential question? And I always enjoy those. Love it.
- [18:42]
Host
Yeah. So one last question for you. I like to ask everybody, how do you stay agile in your role and how do you find a way to do it consistently?
- [18:51]
Adam Hagerman
Change is inevitable. On my team, I don't have a lot of process. I do not have a lot of you must do this, then this, then this, then this. Because I know as soon as you build the process, something's going to change and it all just goes out the window. For myself and my team, I try to make sure that we're all oriented around what problem we're trying to solve and we understand that the way we do that will be flexible. Yeah, yeah.
- [19:15]
Host
Love it. Well, again, I'd like to thank Adam Hagerman, Director of UX Research for Employer Products at Indeed, for joining the show here at Qualtrics X4 in Salt Lake City. You can learn more about Adam and Indeed and Qualtrics by following the links in the show notes.
- [19:31]
Greg Kilstrom
Thanks again for listening to the Agile Brand brought to you by Tech Systems. If you enjoyed the show, please take a minute to subscribe and leave us a rating so that others can find the show as well. You can access more episodes of the show@theagile brand.com that's theagile brand.com and contact me. If you're interested in consulting or advisory services or are looking for a speaker for your next event, go to www.crowd greggkillstrom.com that's G R E G K I H L S T r o m.com the Agile brand is produced by Missing Link, a Latina owned, strategy driven, creatively fueled production co op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. Until next time, stay curious and stay agile.
- [20:24]
Advertiser
The agile brand.
- [20:33]
Doug
And Doug there's nowhere I wouldn't go to help someone customize and save on car insurance with Liberty Mutual. Even if it means sitting front row at a comedy show.
- [20:43]
Host
Hey everyone, check out this guy and his bird. What is this, your first date?
- [20:47]
Adam Hagerman
Oh no.
- [20:47]
Doug
We help people customize and save on car insurance with Liberty Mutual together. We're married. Me to a human, him to a bird.
- [20:53]
Advertiser
Yeah, the bird looks out of your league.
- [20:55]
Doug
Anyways, get a quote@libertymutual.com or with your local agent.
- [20:58]
Adam Hagerman
Liberty Liberty Liberty Liberty.
- [21:02]
Advertiser
At Strayer University we help students like you go from Is it possible To Anything is possible by offering access to up to 10 no cost gen Ed courses so you can reach your goals affordably and fast. Visit Strayer. Edu to learn more. No cost gen EDS provided by Strayer University Affiliate Sophia. Eligibility rules apply. Connect with us for details. Strayer University is certified to operate in Virginia by Chev and has many campuses including at 2121 15th Street north in Arlington, Virginia.