#654: Translating research into cross-functional strategic change with Adam Hagerman, Indeed

Summary

Episode #654: Translating Research into Cross-Functional Strategic Change with Adam Hagerman, Indeed

Release Date: March 24, 2025

Introduction

In episode #654 of The Agile Brand with Greg Kihlström®, host Greg Kihlström engages in a compelling conversation with Adam Hagerman, the Director of UX Research for Employer Products at Indeed. Recorded live at Qualtrics X4 in Salt Lake City, the discussion delves into the transformative journey of translating customer experience (CX) research into strategic, cross-functional changes within a major organization. The episode offers invaluable insights for marketing leaders aiming to harness data effectively to drive business growth and enhance customer satisfaction.

Transforming Satisfaction Measurement

The conversation begins with Adam Hagerman addressing the evolution of user satisfaction measurement at Indeed. Historically reliant on traditional tools like the Net Promoter Score (NPS), Adam and his team recognized the limitations of such metrics in providing actionable insights. At [03:33], Adam explains:

"Measurement is not new. We've been doing it since like phone surveys from ye olden days... NPS is well known and my stakeholders were very excited. They're not anti user sentiment, it's just the tool they were using wasn't as helpful as it could have been."

This realization prompted a shift towards more nuanced and actionable metrics that could better inform strategic decisions and product improvements.

Moving Beyond NPS: Challenges and Solutions

Greg Kihlström probes deeper into the shortcomings of traditional satisfaction metrics at [04:38], to which Adam responds by highlighting the inadequacy of NPS in guiding specific actions:

"If it tells us to push a lever and we push the lever but nothing happens, it's not actually telling us what lever to push makes sense." ([04:38])

Adam emphasizes the need for metrics that not only measure satisfaction but also provide clear guidance on where to invest resources for maximum impact.

Making Research Actionable

At [05:05], Adam elaborates on the process of turning research data into measurable and actionable business strategies:

"Research is the process of collecting information. The reason we collect information is because we need to make a decision... What we bring to the table is kind of the collective baseline for what our users want." ([05:05])

He underscores the importance of deliberate data collection that aligns with organizational goals, ensuring that the insights gathered genuinely advocate for user needs and drive product success.

Change Management in Measurement Approaches

Transitioning to a new measurement system involves significant change management. At [08:11], Adam discusses the challenges of persuading stakeholders to adopt new methodologies:

"It's a change management thing. People are coming in with their own set of expectations, biases, baggage... I'm giving you other options. And by the way, I have data to back it up." ([08:11])

His strategy involves being brutally honest and providing empirical evidence to support the proposed changes, thereby fostering trust and buy-in from diverse stakeholders.

Balancing Quick Wins and Sustainable Growth

The dialogue shifts to balancing immediate improvements with long-term sustainability. Adam shares his approach at [13:19]:

"I had the long term vision in my head... I knew where to start with the quick wins... thinking about your research? I try to think about my research programs as accretive." ([13:19])

He highlights the necessity of initial quick wins to build momentum and gather feedback, which then inform and support sustainable growth initiatives.

Advice for Leaders on Data-Driven Decision Making

When asked for advice for leaders grappling with biases and competing voices within their organizations, Adam offers a pragmatic perspective at [15:19]:

"We may not have all of the information we need to make a decision, but we have a lot of information that can help us know whether we're making a bad decision... And then at some point you just need to go." ([15:21])

He advocates for a balanced approach that leverages available data to make informed decisions while acknowledging and managing inherent uncertainties.

Data Democratization: Benefits and Guardrails

The conversation advances to the democratization of data, with Adam asserting its inherent benefits while cautioning about necessary safeguards at [16:26]:

"I think it's good to have access to it... My concern comes with, do we have the appropriate guardrails in place?" ([16:26])

He stresses the importance of empowering team members with data access while ensuring they possess the skills and frameworks to interpret and utilize data responsibly.

Maintaining a Unified Source of Truth

Addressing the challenge of multiple data interpretations, Adam emphasizes the role of documentation and clear data governance at [19:30]:

"We try to make sure people are empowered to use the data in the correct way or in the intended way." ([19:30])

By providing comprehensive documentation and defining intended use cases, Indeed ensures that data remains a reliable and unified source of truth across the organization.

Staying Agile in a Changing Environment

Concluding the main discussion, Adam shares his philosophy on maintaining agility within his team at [20:39]:

"Change is inevitable... I do not have a lot of process... We're all oriented around what problem we're trying to solve, and we understand that the way we do that will be flexible." ([20:39])

This flexible, problem-focused approach allows his team to adapt swiftly to evolving challenges and opportunities, embodying the agile mindset central to the podcast's theme.

Conclusion

Episode #654 offers a rich exploration of how Indeed leverages UX research to drive strategic, cross-functional changes. Adam Hagerman's insights into evolving measurement practices, fostering data-driven cultures, and maintaining agility provide a valuable roadmap for marketing leaders seeking to enhance customer experiences and business outcomes. Greg Kihlström adeptly navigates the conversation, ensuring that listeners gain a comprehensive understanding of the intricate balance between data utilization and organizational change.

Key Takeaways:

• Traditional metrics like NPS may not provide the actionable insights needed for strategic decisions.
• Transitioning to new measurement systems requires honest communication and empirical support.
• Balancing quick wins with sustainable growth is essential for long-term success.
• Data democratization should be paired with proper guardrails and data literacy initiatives.
• Maintaining agility involves focusing on problem-solving and remaining flexible in methods.

For those seeking to implement similar strategies, this episode underscores the importance of aligning research methodologies with organizational goals and fostering a culture that embraces data-driven decision-making.

Transcript

[Domo Representative]

We all know data is valuable. We use it to tell a story to make informed decisions for our businesses. But turning data into actionable insights can be a challenge. It's time to unlock the true potential of your business data with Domo's AI and Data Products platform. Domo lets you channel AI and data into innovative uses that deliver a measurable impact. Ask your data Anything at any time. Anyone on your team can use Domo to easily prepare, analyze, visualize, automate and distribute data, all amplified by AI. Domo goes beyond productivity. It's designed to transform your processes, helping you make smarter and faster decisions and drive real growth. All powered by Domo's trust, flexibility and years of expertise in data and AI innovation. Data is hard. Domo is easy. Make smarter decisions and propel your business forward with Domo. Learn more today@AI that's AI.domo.com the Agile.

[Greg Kilstrom]

Brand.

[Podcast Host]

Welcome to season seven of the Agile Brand where we discuss the trends and topics marketing leaders need to know. Stay curious, stay agile and join the top enterprise brands and Martech platforms as we explore marketing technology, AI, E commerce and whatever's next for the Omnichannel customer experience. Together we'll discover what it takes to create an agile brand built for today and tomorrow and built for customers, employees and continued business growth. I'm your host Greg Kilstrom, advising Fortune 1000 brands on martech, AI and marketing operations. The Agile Brand Podcast is brought to you by Tech Systems, an industry leader in full stack technology services, talent services and real world application. For more information go to teksystems.com to make sure you always get the latest episodes, please hit subscribe on the app you listen to podcasts on and leave us a rating so others can find us as well. Now onto the show.

[Greg Kilstrom]

We are recording live at Qualtrics X4 in Salt Lake City and seeing and hearing all about how to create and enable amazing customer and employee experiences. It's important to collect customer experience data, but if it's not driving change across your organization, is it really helping your business? Today we're going to talk about making meaningful cross functional change using CX research and data as a guide. I'm joined by Adam Hagerman, Director of UX Research for Employer Products at Indeed. Adam has led transformative efforts at Indeed to turn customer experience research into cross functional strategic change, driving real improvements in both user satisfaction and product success. Adam, welcome to the show.

[Adam Hagerman]

Thanks for having me.

[Greg Kilstrom]

Yeah, looking forward to diving in here. Before we do though, why don't you give us a Little background on yourself and your role at Indeed.

[Adam Hagerman]

Sure. I lead a team of UX researchers. We look over the employer products. We're trying to make sure that what we end up shipping for people to consume is solving relevant needs and helping them do what they need to do. Better, faster, cheaper, easier.

[Domo Representative]

Wonderful. Great.

[Greg Kilstrom]

So, yeah, let's dive in here. And so we're going to talk about a few things here, but I want to start by talking about transforming satisfaction measurement into strategic decision making. So you and your team at Indeed have transformed your approach to measuring user satisfaction. What led to the shift we needed to.

[Adam Hagerman]

Yeah, measurement is satisfaction. Measurement is not new. We've been doing it since like phone surveys from ye olden days that you would get at dinner time. And the tool we were using was the same one, the Net Promoter Score. It's evolved, it's iterated over time. It's had improvements here or there, but at the end of the day it's a brand measurement and we have a product we need to work on. NPS is well known and my stakeholders were very excited. They're not anti user sentiment, it's just the tool they were using wasn't as helpful as it could have been. We ask the question, can we make this better? What can we do? Here are the shortcomings, here's how it's preventing us from helping people do what they need to do better, faster, cheaper, easier. It's not giving us the insight we need. So let's find a new way to do it. And the intention of finding the new way to do it is to actually help our stakeholders build better, faster, cheaper, easier products.

[Greg Kilstrom]

Yeah, yeah. So along those lines, I mean, probably many people listening out there are using nps, csot, you name it. What were some of the telltale signs that it wasn't giving you everything that you needed?

[Adam Hagerman]

If it tells us to push a lever and we push the lever but nothing happens, it's not actually telling us what lever to push makes sense. I guess that's the answer.

[Greg Kilstrom]

Yeah.

[Adam Hagerman]

Yeah.

[Greg Kilstrom]

Hey, so you mentioned that data was harnessed not just to inform, but to quantify impact and to guide strategy. How did you approach turning research into something measurable and actionable for the business?

[Adam Hagerman]

Research is the process of collecting information. The reason we collect information is because we need to make a decision. The product stakeholders need to make a decision. Do we do it this way, do we do it that way? They receive information from lots of resources, they get feedback from their go to market team, they get feedback from the engineering team, they get Feedback from random person on the street, and they have to take all of that information and make a decision. What we bring to the table is kind of the collective baseline for what our users want. Our job is to advocate for users among that entire organism of ecosystem of information floating around. Data collection is a deliberate act. Just because something's been collected doesn't mean it's what you should be collecting. And we ask that question, are we collecting information that actually helps us advocate for users? Once we were able to do that and demonstrate, here's what we're doing and here's what it means for you, here's your return on investment, it was an easier case to make. Does that answer your question?

[Greg Kilstrom]

Yeah, I mean, I think so. I mean, so is it because there's lots of signals, right? So I mean, again, there's some go to measurements that a lot of people use, like NPS and others. And again, to your point, nothing wrong with that. But if it's the sole measurement, there's.

[Adam Hagerman]

Just ask some questions, right?

[Greg Kilstrom]

So yeah, So I guess how do you, how do you determine, is it an incremental, like in the advertising world, it's like media mix modeling or something like that. Is it a similar approach?

[Adam Hagerman]

I actually used a very similar approach to media mix modeling. In media mix modeling, what you're really trying to do is of all the money that's floating around in brand advertisements, activation, whatever you're trying to see, where does an incremental dollar give me more benefit? So if I have $1 left to spend, am I gonna put it in this thing or that thing? And what my measurement was trying to do was say of the experiences available where we could put our investments into improving user experiences, where should you put your dollar? And we had to do a mathematical exercise, create an empirical argument for that's the right place to put the dollar. And then we had to hope that it worked, that we weren't lying.

[Greg Kilstrom]

Right, Right.

[Adam Hagerman]

So we created systems of accountability for ourselves. And in order for us to advocate for this new type of measurement, it needs to meet these criteria. And we set out a protocol for how we were going to check ourselves. And before we were ready to really roll full scale and say, this is truth, listen to us, we wanted to make sure that we were actually representing the lived experience of our people, the customers that we have. So we took that medium mixed modeling approach. How can we model the where return on effort into fixing user experiences would give us that outcome.

[Greg Kilstrom]

So what was that process like then?

[Adam Hagerman]

Lots of math.

[Greg Kilstrom]

Yeah, I would imagine Right. What's the process then of convincing people to listen to. You know, again, people get really stuck in their ways. This is a change management thing as much as it is a measurement thing. Right.

[Adam Hagerman]

You hit it on the nail or you hit the nail on the head. It's a change management thing. People are coming in with their own set of expectations, biases, baggage. This is what it means for me, either in good or bad terms. And the approach I like to take is just being brutally honest, direct. This is what's going on. Here's how moving forward with this, here's what you can expect as a consequence. I'm not going to tell you what to do, but if you're going to do this, here's your consequence. I'm giving you other options. And by the way, I have data to back it up.

[Greg Kilstrom]

Yeah, I mean, that's the key thing. Right. Instead of a feeling or a hunch or I did this at this other place. You've got the incremental improvements and that's.

[Adam Hagerman]

The media mix modeling that I talked about. Yes. It's working in a theoretical and. Theoretical and hypothetical space. But we adhere to all the rules of statistics that have come to us through the current philosophy of science, and that's how we construct our argument.

[Greg Kilstrom]

Yeah, yeah. So how did that go?

[Adam Hagerman]

I guess at first, like, is that asymmetric? There were some people who saw what we were trying to do and they were like, yeah, let's go.

[Greg Kilstrom]

Yeah.

[Adam Hagerman]

There were other people who were more. More. They had been burned in the past, I guess is the best way to say it. We work with very smart, intelligent, experienced people and indeed was not their first job. So they have the baggage from wherever they were and whatever research team did that thing. So when they hear Adam saying, this is what we're going to do now, or don't do that or whatever, I have to acknowledge that they're also a human being that have their own set of experiences, baggage, whatever. And the connection I make with them or I try to make is we're both here to do the same thing where we both want the same good outcomes. What questions do you have? If there are things that I can do to make you feel more comfortable, I'd like to know what it is. It may just be a matter of I didn't say it on that slide. So it's having frank conversations.

[Greg Kilstrom]

Well, and this is where it comes down to. There's lots of talk about data driven decision making, but this is the culture shift part of that. Right. Again, I did this thing at this other place and it worked really well. So it must, you know, work again at this new place. In different circumstances, our context is different.

[Adam Hagerman]

And sometimes that's part of the argument. Yeah, when I'm convincing people, it's like, yeah, over there it worked and I can see why it would. Or I can also see why it would fail miserably. It's that idea of no context is exactly the same and the success depends upon your context and bringing that out.

[Domo Representative]

Having empirical ardents we all know data is valuable. We use it to tell a story, to make informed decisions for our businesses. But turning data into actionable insights can be a challenge. It's time to unlock the true potential of your business. Data with Domo's AI and Data Products platform Domo lets you channel AI and data into innovative uses that deliver a measurable impact. Ask your data anything at any time. Anyone on your team can use Domo to easily prepare, analyze, visualize, automate and distribute data, all amplified by AI. Domo goes beyond productivity. It's designed to transform your processes, helping you make smarter and faster decisions and drive real growth. All powered by domo's trust, flexibility and years of expertise in data and AI innovation. Data is hard. Domo is easy. Make smarter decisions and propel your business forward with Domo. Learn more [e-mail] that's AI.domo.com Want to learn more and join the discussion about marketing and AI. Attend the premier conference dedicated to marketing and AI that's Meacon, the Marketing Artificial Intelligence Conference from October 14 through 16 in Cleveland, Ohio. Meacon brings together the brightest minds and leading voices in AI. Don't miss this opportunity to connect with a dynamic community of experts, visionaries and enthusiasts. The Agile brand is proud to be the lead media sponsor of this important event. Register today@MarketingAI Institute.com that's MarketingAI Institute.com and use the code AGILE150 for $150 off your registration fee. I can't wait to see you there.

[Greg Kilstrom]

So how do you look at making arguments or not even argument? Let's not make it arguments, let's make it friendly. The quick win versus the sustainable growth and things A lot of this is There can be quick wins when you make any change sometimes or any good change. But how do you look at balancing between let's get some of those quick wins under our belt versus okay, this is going to achieve sustainable growth which is hard to project.

[Adam Hagerman]

I had the long term vision in my head. I knew what I wanted to do. I knew where I needed more information, like here's where it might fail. But I had to start with the quick wins. It's just like any other product that you push out. Your first users are going to be your best sources of feedback. They're going to say, okay Adam, here's how you and your team maybe thought about this differently. Here's feedback. I would have because here's how my context will change how you want to approach something. So the quick wins are necessary, but you have to know where you're going. The corollary to your question is people who do a bunch of quick wins but don't know where they're going and that's how you get cruft. Like that's what it is. So thinking about your research? I try to think about my research programs as accretive. Whatever we're doing today is building on what we did yesterday and we'll build on it tomorrow.

[Greg Kilstrom]

So yeah, so the quick wins are. And I'm a huge fan of that approach. But to your point, as long as there's a but it's almost as much about. I mean there's benefit to the business ideally there's benefit to the customer as well with those. But it's also about of winning hearts and minds. Right Is that's a big part of it.

[Adam Hagerman]

You have to show that it works. That if somebody's going to say, hey Adam, I want your stuff on my surface because I want to use your tool to make sure that our user experience is good, I want to deliver on that. I want to actually say, yes, I helped you do that.

[Greg Kilstrom]

So what would your advice be to leaders? Let's say they're not sitting in the research or the data component of the organization. But again, they read the same things I read about data driven decision. They know somewhere in their head that there's a lot of value here. But they're having a hard time kind of getting past whether it's biases, whether it's other loud voices in the organization. What's your advice to them to kind of just make the first step?

[Adam Hagerman]

Well, that's a deep question.

[Greg Kilstrom]

I'm sure it depends too.

[Adam Hagerman]

Well, Kierkegaard would say, take your leap of faith, stare into the abyss and go. And I do take kind of a similar approach. I talk about that the series of experiences, bag of lessons learned that helps us decide how we should move forward. We may not have all of the information we need to make a decision, but we have a lot of information that can help Us know whether we're making a bad decision. Check yourself before you wreck yourself. Have your internal system of accountability. And then at some point you just.

[Greg Kilstrom]

Need to go, yeah, yeah. I remember when I was at the media briefing the other day and you had mentioned, you know, indeed is unique to some organizations in that being a technology company, access to data, things like that is a little more prevalent than in some or maybe more legacy companies and things. How much do you credit, you know, having that access to data? Like is. Is it always a good thing? Are there good and bad things about it?

[Adam Hagerman]

I think it's good to have access to it. There's a barrier to entry. You have to know how to type in the right query. You have to know what are the signals that maybe you didn't do the query correctly. There's a lot of self awareness that goes into working with that. The people who primarily work in that space, this is what they do all day long. So they have those tips and tricks to be conversant in that kind of work. I viewed, or I continue to view democratization of information as inherently a good thing. My concern comes with, do we have the appropriate guardrails in place? Have we checked ourselves before we wreck ourselves?

[Greg Kilstrom]

Well, a lot of, yeah, and I'm a huge fan of it as well. But also data literacy and, you know, all the things that. As well as the ethical components, of course. How do you make sure that there's kind of a unified version of the truth then with all that? Because again, it's a great thing to have access, but everybody's asking their own questions, so on and so forth.

[Adam Hagerman]

I'm not the only person participating in this conversation.

[Greg Kilstrom]

Yeah.

[Adam Hagerman]

And there are times where I know my data is not the data somebody needs at that point. I should respect that because the data I have to offer is not the data they need at that moment. I try to think instead, how do I participate in that conversation? How do I add an additional layer, let people know that this layer exists, that they can use it to ask questions of their own. One of the important aspects of the program I talked about was that it didn't just live in Qualtrics, it was actually married with all of our other data. So people could do their own investigations and they could use our data source as part of, as evidence in what? I know you don't want to use the word argument, but in whatever empirical.

[Greg Kilstrom]

Argument they use, you can use it. Yeah.

[Adam Hagerman]

I feel like I'm not answering your question.

[Greg Kilstrom]

Well, no, I mean, it's kind of a Broad question. You know, I think it's. It is. And I think this is where platforms and kind of, we do need some kind of unified source of truth. At the end of the day, this is about happier customers and longtime customers. And so at the end of the day, that's the goal. Right. So if there's many different answers to that, maybe there could be some challenges there.

[Adam Hagerman]

I guess where I'm getting at is I don't want to preclude somebody from having access to this, but I do want to make sure that when people are accessing it, they understand what they're accessing. We do that through documentation. We say, this is what you'll find in this database, in this data frame. We have lots of information about what is this program? How can you use this information? Oh, do you have questions about how we came up to it? Look over here. We try to make sure people are empowered to use the data in the correct way or in the intended way. Correct makes it sound like there is a right and wrong in the intended way.

[Greg Kilstrom]

Well, and I would imagine in that. And empowering them as well. Well, you're also getting some amazing ideas that one person with access centrally would not have.

[Adam Hagerman]

This goes back to that idea of a creative understanding. We don't have to be the only source of truth, but what we're doing is we're adding to our knowledge.

[Greg Kilstrom]

Absolutely love it. Well, as we wrap up here, just a couple questions for you. I know we're almost wrapping up the event here, but I wanted to ask what's been a highlight so far for you at Qualtrics X4?

[Adam Hagerman]

I usually enjoy the day two keynotes. Day one is very product forward. Like, look at the cool stuff we're doing. Day two is more, how should you think about doing your work? And I always enjoy those reframings because when we're just going about our day, we do what we do. And sometimes we need to stop and listen to how do other people do it? How are they approaching this existential question? And I always enjoy those.

[Greg Kilstrom]

Love it. Yeah. So one last question for you. I like to ask everybody, how do you stay agile in your role and how do you find a way to do it consistently?

[Adam Hagerman]

Change is inevitable. On my team, I don't have a lot of process. I do not have a lot of, you must do this, then this, then this, then this. Because I know as soon as you build the process, something's going to change and it all just goes out the window. For myself and my team, I try to make sure that we're all oriented around what problem we're trying to solve, and we understand that the way we do that will be flexible.

[Greg Kilstrom]

Love it. Well, again, I'd like to thank Adam Hagerman, Director of UX Research for employer products at Indeed, for joining the show here at Qualtrics X4 in Salt Lake City. You can learn more about Adam and Indeed and Qualtrics by following the links in the show. Notes.

[Podcast Host]

Thanks again for listening to the Agile Brand brought to you by Tech Systems. If you enjoyed the show, please take a minute to subscribe and leave us a rating so that others can find the show as well. You can access more episodes of the [e-mail] that's theagile brand.com and contact me if you're interested in consulting or advisory services or are looking for a speaker for your next event, go to www.gregkilstrom.com that's G R E G K I H L S t r o m.com the Agile brand is produced by Missing Link, a Latina owned, strategy driven, creatively fueled production co op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. Until next time, stay curious and stay agile.

[Greg Kilstrom]

The Agile Brand.

[Domo Representative]

Ever heard of Farm to table? How about Farm to Home? That's how Costa Farm's plant business works with over 1500 plant varieties grown over 5200 acres. They're not just a company, they're your plant partners who've been perfecting their craft for 60 years. They deliver beautiful, high quality, easy to care for plants. They even offer virtual plant consultations and an insider club for rare plant access. Check out www.costafarms.com today and enter code worth knowing COSTAFARMS15 for a 15% discount on your first purchase. You can also purchase this unique plant brand at Lowe's, Walmart, Amazon and Home depot. Go to www.costacostafarms.com.

[Podcast Host]

Before we continue, I wanted to share a key strategic resource that a majority of the Fortune 500 are already aware of. Finding the best technology, business and talent solutions is not easy. With business demands and competitive pressures mounting, you need to be able to design, deploy and optimize your technology to provide leading customer experiences while driving business growth. Those of you that have been listening to this show for a while know that this podcast is brought to you by Tech Systems, a global provider of technology, business and talent solutions for more than 80% of the Fortune 500. TechSystems accelerates business transformation for their customers. Whether you're looking to maximize your technology roi, drive business growth, or elevate customer experiences. Tech Systems enables enterprises to capitalize on changes. Learn [e-mail] that's T E K systems. Com. Now let's get back to the show.