← Insights Unlocked
Insights Unlocked
How to stop wasting research and turn customer insights into action with Jake Burghardt
00:37:35
Learn how to reduce research waste and turn customer insights into action with Jake Burghardt, author of Stop Wasting Research.
Loading summary
Transcript31 lines
- [00:02]
A
Welcome back to the Insights Unlocked podcast. Today we're talking about a topic everyone struggles with. Getting more value out of the research you've already tackled. We've got the perfect guest to help us with that. Jake Burkhardt, the author of Stop Wasting Research. He'll walk us through how to rethink the way we prepare, share, and use research to make better decisions. We'll be covering everything from breaking down silos to exploring the role of AI. Get ready for a conversation packed with practical advice. Enjoy the show.
- [00:34]
B
Welcome to Insights Unlocked, an original podcast from User Testing, where we bring you candid conversations and stories with the thinkers, doers and builders behind some of the most successful digital products and experiences in the world, from concept to execution.
- [00:54]
A
Welcome to the Insights Unlocked podcast. I'm Nathan Isaacs, principal Content marketing manager at UserTesting. Joining us today as host is User Testing's Leah Hogan, principal for Experience Research Strategy. Welcome to the show, Leah.
- [01:08]
C
Thank you, Nathan.
- [01:10]
A
And our guest today is Jake Burkhart. Jake is the author of Stop Wasting Research and a leading voice in bridging the gap between customer insights and product decisions. With over 20 years of experience at companies like Amazon and Microsoft, Jake now helps organizations turn overlooked research into meaningful impact. Welcome to the show, Jake.
- [01:31]
D
Thank you. Very glad to be here.
- [01:35]
C
I am totally thrilled to have this conversation today. And as we were kind of chatting as we were leading up to the conversation today, this is a very practical and I think timely topic to be covering. But before we get into the meat of today's conversation, I actually wanted to learn a little bit more about your journey to kind of where it is that puts you in the place to write the book and what led you to have this focus.
- [02:07]
D
Well, again, thanks so much for having me. And you know, I got started in.com as a UX researcher and I've held a variety of roles over time. Always been a bit of a generalist in design, into roadmapping and product management, and increasingly an operational focus over time. And I did a lot of consulting, but I got tired of throwing things over the wall. I was really interested in how can we have more impact with research. So I wanted to go in house. Spent some time at a lab at Microsoft doing new productivity interactions that look pretty familiar in today's AI world, but were pretty experimental at that time and spent a good amount of time at Amazon running insight initiatives in the retail and Alexa organizations. But when I think back to sort of a couple of pivotal moments that kind of got me to where I wrote this new Book Stop Wasting Research from Rosenfeld Media When I interviewed for a principal UX researcher role at Amazon, I met with a hiring manager and just said, hey, I think that you have the research that you need. I imagine I just want to help you get more done with it. And it was a hunch I had and it kind of set me off in a direction I It's one of those moments you don't realize, but it kind of put me on a career tangent that I'm still on. And I'd say another key moment was a Covid detour doing some writing in the research repository space. Sort of the sea change that was happening about what can we do with more research knowledge to make it more actionable and useful and connecting in with the research ops community and just learning so much of what's going on. And folks were kind of really tool focused in all of it. And information structure focused is certainly central ingredients. And I found I had something a little bit different to say around what are the operations and change management aspects of this that make those tooling investments to get more done with research successful and not just another thing that researchers try and then say we didn't see the impact that we were hoping from that. So that's what drove me to write Stop Wasting Research.
- [04:17]
C
I have to say so many things that you just said resonated in my soul because I feel like a lot of us as researchers and research operations folks really are faced with this challenge of making sure that the culture piece evolves alongside what it is that we're doing from a tooling and training and governance perspective. And so I actually think that leads really well into our next question here, which is around more isn't always better. So we do, as researchers, we do a lot. What are some of those shifts that you think are really important to helping people? That change management piece that research teams really need to accomplish so that they can be successful in this, I would say quickly accelerating space.
- [05:19]
D
Yeah, I mean I think people have done so much to optimize their study processes, you know, and if we look at the books on how to conduct research, how to run the great workshop, great presentation, this is the things that people influencers are constantly talking about. They're focused on, you know, all those things that researchers excel at where they're honing in on those core unknowns in the team, really staying connected with partners around what assumptions and questions they have that could use new rounds of discovery, crafting great research processes in order to accomplish that and executing with, you know, a great Level of detail that's appropriate in industry and always that tension there. But bringing people along for the ride and delivering great outputs. And, you know, some of those outputs immediately have impact. Researchers careers are advancing, products are, you know, greatly improved. Customers see better outcomes. You don't write a book called Stop Wasting Research unless you've seen research have amazing impacts across the whole product development and delivery so spectrum. But, you know, I think what's the missing part and the change management piece that you're alluding to is what. What are the changes that we're looking for? And we kind of need to recognize that how much is left behind? In a lot of studies, you know, you talk privately to researchers, as I know you have, and you hear the frustration of the remainder, the cutting room floor. And if you take stock of that cutting room floor, as I've had the opportunity to do in large volumes of research, you see it's absolutely filled with wealth. You know, it is filled with gold. Not just in some abstract sense or like some idea that we need to march research forward because it's the right thing to do, but because it directly applies to what leaders are talking about, what they want to do tomorrow. So we're going out and we're discovering new things that we already know. And so a lot of the change management is on the research side, kind of building the sense that there is more value in their existing work, that sometimes being strategic is looking backwards and bringing the previous things forward and then increasing the visibility of research. We could talk a lot about that. But on the insight consumer side, on decision makers and owning teams, it's about continuous discovery and understanding that this existing research can have something to say about what's next. None of this happens overnight. So the book has these models and ideas. You can kind of choose actions and experiments to kind of move things forward, see what works, and build out an initiative to do more with research over time.
- [08:06]
C
Yeah, I really appreciated the workshops that you recommend, because I think people often want a cookbook that just says, do this, do this, do this, and then the output will be that. And truthfully, it is often the case that you actually need to do something that's a lot more nuanced, that takes into account what's happening now, because it's only when you find those specific gaps that you can prescribe that next step. And so that's another piece of, I think, the book that I really appreciated, which is just you may have seen these specific gaps, and then here's some recommendations around how to address those gaps. And I think that's the piece that people are really missing, like having some tangible ideas for how to get there. So, anyway, I feel like I'm giving you like an ad for the book, but I mean, again, because I feel like this is just coming a lot for me, it's like a really important thing to keep in mind. And actually, I think there's another piece in the book that is important to bring up, which is really around. So we're doing all this beautiful work, right. Maybe a third of it gets seed, maybe less. Right. And I think sometimes we're so focused on actually doing the work that we're not even paying attention to the outcomes that it's driving as much as we should have. And so kind of, to that point around, what are some of the gaps that you've seen? What are some of those ways that you've seen organizations thoughtlessly waste what it is that they're doing from a research perspective?
- [09:55]
D
Yeah, it's a great question. And a lot of the book gets, you know, you mentioned sort of the breadth of action ideas and things. There's this menu of things that people can choose from based on what makes sense for their environment. But first you have to sort of recognize the problem and start to see some of the root causes. And at an individual level, you know, researchers have this feeling of, do I have time to follow through on that study before I move on to the next one? And slowly changing perspectives about that over time to give researchers the space or allowing them to build the space to connect with teams over time about insights, are they finding the right audiences for insights? As decision making becomes increasingly fragmented, it can be hard to track down the right person to talk to about the right insight. Even when you were trying to stay in a narrow area, you may end up finding things can be useful by all sorts of folks. So there's an operational and visibility aspect to that. You know, it can be unclear that old insights have value or where to find them or what to do with them. So. And I think I took it up to some root causes. I, you know, and they end up being the structure of the book. If we look at the reasons why research gets wasted at a bigger picture level and kind of keep asking why, I came up with three, and there's preparation, motivation, and integration. And preparation is the idea that, you know, we deliver in a certain way, but it's not necessarily the way that's most useful for pulling that content back into new plans. So we deliver reports that are a summary of what we've learned for a set of questions and a time. But how can we prepare things to be ready to use for future decision making? And what accessibility and tooling and information structures can help us do that? The second one, motivation, is about, you know, research. You know, you alluded to it in your question. Researchers themselves may even see research as sort of an optional input. They're not following through on it as much as they could. Maybe they don't bias towards that side of the process. I'm arguing that researchers can lean into that space to get more impact, though it won't be for everybody. And decision makers and stakeholders may see research as more of an optional input rather than a long term product driver too. So that section of the book Motivation really gets into what are some of the things that we can kind of increase the desire and the visibility of research as a valuable tool for ongoing planning, not just a momentary spark that comes into teams. And then integration is you could have prepared your research for use, people could want to use it. They've seen other people use existing insights and achieve great results and they want to do it too. But unless it's present, unless you have the right touch points, unless it's there when decisions are made, all those things can just be good intentions. And so integration is about finding those right touch points, whether it's things that research communities push on their own cadence or finding the right cadences over time, like you would do with a study. But for an existing body, body of research, and the top learnings to date to say, how can we connect those into the right points and the right moments, that takes researchers to kind of step up from individualistic work slowly over time and see the value in connecting with each other and becoming more of a collective stakeholder is a lot of what the book talks about.
- [13:27]
C
Yeah, that's a really great point. And I think one of the important things to consider is just, and I love this point in the book too, just it's not for everybody. Right. Sometimes you have to let the power of the work exist for a little bit in order to get the skeptics aboard. And it gives people, I think, some permission to worry less a bit about whether or not everybody is along for the ride, but embrace the people who are and really prove the value through the activity that. With the people who are bought in already.
- [14:12]
D
Absolutely. Yeah. It's, you know, in some ways it's just taking a lot of change management best practices and kind of leaning them into this space and, you know, finding some folks need to see a lot of value demonstrated by others and championed by others before they come on board. Whereas a lot of people are lead users and just get this right away. So it's about leaning into those folks.
- [14:37]
C
Yeah, that's great, great point. And actually leads into that next question that I have because, you know, one of the other things that you talk about a lot is just the fact that the, the fact that people ask a lot of questions and do research across the Org is something that we know very well. Right. We are part of an ecosystem that includes people who have multiple responsibilities and job titles. And so there are those functions and then there are also those functions that may traditionally leverage insights from other disciplines across the organization. And here I'm really thinking about marketing teams or even customer success. And so I'm curious about how do you help translate what it is that we're doing in that usability UX research, maybe even CX research space into the settings where people may have less exposure to them, like in marketing and customer success without having to, you know, remake the wheel.
- [15:40]
D
Yeah, yeah, it's a great question. I mean I talk a lot about, I, there's some fundamental definitions and my, the definition of customer research I use, I think to your point is pretty expansive. It's not just UX and usability. It includes market data science, you know, and anything that kind of fits the mold where they're conducting what a researcher would recognize as research. You know, with a study plan and outputs and solid methods and things. And there's a surprising number of different streams of investigation that fit that mold. And you know, when you're connecting research to a new audience, there's always a bit of explaining to do so you can look at infrastructural solutions to that. You know, how can you educate more broadly, turn up visibility for different kinds of research? You know, those are kind of well, tread problems, I think, in, you know, building literacy over time is not something that happens immediately. But that being said, I think a well articulated insight in and of itself builds the kind of curiosity where people want to learn what that is. You know, where if you are pushing out more broadly a variety of insights from different sources under a collective umbrella, then suddenly when you see things that might resonate or apply to you, you're kind of digging in in a different sort of way. You're not learning in an abstract way. You're saying, oh, that applies to me. What is this? And those are the real moments to kind of understand. And that's where you hear people deflect sometimes. Oh, from the usability folks, it's how many customers, all those sorts of things. But oftentimes it's just they need time to. To your point about sort of early adopters versus laggards, sometimes ideas need to settle for a while. We're kind of building mindshare for a set of important insights over time. Not every last insight, but things that the community surfaces as top insights. And we're in doing that, we're sort of breaking down silos by kind of increasing the breadth. I think another way is, you know, if we do take this bigger term of our bigger definition of customer research, I've seen a lot of value in organizations of connecting the dots across different stripes of research evidence. So if you do have an insight that is supported by customer support and also found in usability studies, you know, how can you bring those together? In the book I talk about insight summaries as sort of each page, each insight summary is a one stop shop. Its own hyperlink, its own name, and then all the evidence that researchers can kind of include in there over time. And by mixing those kinds of research, you know, a customer support person might find it a little bit less alien to look at a usability finding because they're seeing their data in that context and vice versa as well. And you're making a more persuasive case for the insight over time by combining. Everybody talks about collaboration in abstract, but it's very hard to accomplish and you want to get more out of it than you put into it. And so what I pitch are certain information structures and kind of ways of turning up collective impact so that when an insight summary does deliver, all the folks that contributed to it can take some of the share of that win.
- [19:06]
C
Yeah, I just, as you were speaking like in my head, I'm just thinking about a conversation I had last week actually with a team that's about to democratize and one of the challenges that they haven't yet figured out is how do we even structure what it is that we're learning from all these folks who are going to become people who do research? They aren't yet. Right. They're just in the process of starting to think about democratizing research and how do you connect across all these disparate sources, the evidence and the findings and the people and the everything else. And to be able to say like there's this real, like, I don't know, keep thinking groundswell, but there's a lot of support across the org, like a lot of people actually know this, we might be Saying it in different ways, but like, we really know this now. What are we going to do about it and what have we done about it?
- [20:11]
D
Yeah, by collecting things together and by giving it a persuasive title and building common language. I think a problem with democratization is it's sort of point learning for a lot of folks, learning for themselves or a small audience and not sharing outward to your point. And a lot of the value of research is developing shared understanding of problems across teams among leadership so that things get prioritized. So there's some real tensions there.
- [20:42]
C
Yeah. Oh, boy. Yes. Again, it's like I feel it in my soul. And again, it leads really well to the next question because everybody wants to throw AI at problems because it helps us, I think, sometimes bring some structure to these very unstructured problems and questions. And so, you know, given the, the promise and the challenge of, of AI, do you think. Actually, I really dislike answering the. Do you. Do you. Have you. Because it's like yes or no. It's like the worst bias that you could ever introduce in a research interview. So I'm going to self edit and say, like, where do you see us going when it comes to the risk and the reward of integrating AI into research? Do you see us potentially again, do you. Could we see ourselves automating our ways to wasting even more, or is it a great opportunity for us to get better at not wasting research?
- [21:55]
D
Well, I'll start by saying it's such an exciting time. There's all sorts of things to explore. And I love how the research community is sharing what they're learning together at this time. It's a time where we're really figuring out the practical beyond the hype. And it's evolving so fast that it's dizzying. I find it dizzying. I'm also really enjoying it. So it's this interesting mixture. And I think when I think of your question, I think of marketing claims versus reality. There is one side of the technology community that's looking at very intense automation of research. And we have to ask ourselves, there's all sorts of tools, what's that useful for? And I think to this conversation of research waste, we can imagine a lot more content being generated. And is it, even if it's low cost to generate, how are we using it and is it valid? Of course. But I guess I get most excited about AI in sort of smaller practical applications, you know, in writing the book. It'd be very easy to write a book right now that is completely out of date in A second. Right. So I tried to keep it to some tenants and a few particular sidebars. That being said, you know, all those action ideas through the whole book, I can be applied to a lot of them in specific ways. And that's sort of what I'm exploring right now. So, you know, in the. There's a lot of people exploring it in the research process, for particular moments in the research process. Like I said, optimizing the study. Right. I think what is an interesting question that this book puts forward is what does it mean to optimize, you know, the body of knowledge and the activation of that body of knowledge? And it's. There's a tendency to think that we can kind of throw all sorts of research content into a bucket and put generative AI on top of it. And it certainly will get you somewhere, it'll always produce something. And then the question is, you know, in different environments, you know, how. How can we kind of learn from that and test it and do all those things? Is it going to build common language? I'm finding that the answer is no. You know, where to the extent that it's used as a discovery mechanism applied to research, not just research evidence, it could surface insights and human worded insights are the real currency that I'm pushing forward in this book. Of course, they may have used AI in the creation of those, but once it's a problem to solve that we're trying to get articulation for and mindshare for, you know, we're trying to move that forward. There are AI ways of doing that. I think a lot of the other scenarios that I'm looking at with AI is helping to author things like insight summaries, helping to find where research is being cited, where is it being linked to and how can we include those citations. You know, there's a lot of agentic things we can do in the information flow that is beyond just sort of discovering within the bucket of content. So a lot of practical applications, a lot of interesting explorations going on. I think some of it's overreach and some of it is just terribly exciting and it doesn't replace the parts that I'm particularly excited about now are things that researchers aren't even doing now that we could add value through new forms of automation and discovery.
- [25:28]
C
Well, actually kind of as a follow up, what are some of those things that you think researchers could. Are not yet doing that. Yeah, they might be.
- [25:38]
D
Well, in the book I talk about. So if you think about research being used in a study process, Researchers have sort of a tangible sense. It's relational. Right. Where you are meeting with stakeholders and talking them through insights and hearing their responses to them and maybe even seeing their planning links and things associated with it. As soon as you have a body of content that people are self serving or that you're broadcasting, maybe used in all sorts of ways, then how do you get visibility into it? If you can't turn up the visibility of how existing research is used, you can't normalize the use of existing research and turn it up over time. It's never going to be 100% that everything's going to have some research reference. But I talk about citations and it's not academic. It's literally just a link to research. This is the justification, this is what's informing this piece of work. And so I'll give you one concrete example of engentic stuff that I'm excited about is as you know, a lot of AI tools become an excuse to plug together all sorts of information sources. Suddenly we can have our repository of reports and our repository of insights in the same space as we have planning documents and design specifications and various things and we can look for the linkages across those things and maybe cross annotate across them. And that would give sort of unprecedented visibility into research impact. So that's one area that I think people aren't spending as much time today where they could do more to your earlier conversation about ROI to really be able to see visibly what's going on with research that's been created.
- [27:17]
C
I again, I just, I'm thinking in my head I have, I've made this spreadsheet and I've seen people spontaneously come up with it to track insights into spreadsheet. Right. And a spreadsheet and usually lives somewhere on some drive somewhere that only the researchers ever see. And then maybe on a quarterly cadence leadership gets to see some pretty slides that get built out that kind of talk to the best of the best. Yeah. And I'm now seeing in my head like you could actually come up with like a little one pager, like what was the impact and have all the citations in there. I'm like, I'm so excited. I have to go build this thing now. Great. I always love when I have conversations where I'm like, I need to go build something out of that. But actually I do have another question for you which is really around AI also I think inherently also brings up the tension of potentially losing that human part. Then how do we make sure that even in this space where we can have synthetic users who might be used in part to supplement or even create some of the data that we're using to make decisions. What can we do to make sure that we stay people and human and alive and recognizable in that space?
- [28:58]
D
Well, synthetic users, I'll address that first, then go into the rest of the question. I mean, it's another tool, and in a time where we're creating vast amounts of synthetic data for evaluating AI, I think it's. It seems like a natural fit to fit it forward into customer research for a lot of people because they generated a lot of data over here, so maybe we generate a lot of data over there. It's a tool that has some uses, and I'm not conversant on all of the uses that I'd be excited about. I think it's so granular down in the weeds. And it's been interesting to see the conversation online about it, but I don't have a verdict on it. I think I treat a lot of things like tools in the toolbox. And I don't know how often I'd be reaching for that, because when we talk about customer understanding in the human side, what we're talking about is kind of pulling from all of those research sources in an organization instead of reaching for synthetic. What do we already have that is direct from customers would be my first question. And so much of the book is about pulling that content together, pulling those people together, pulling those and that intelligence together, all those smart staff that you have to be more of a collective stakeholder. So you don't necessarily need to reach for fast data because you've synthetic data, because you've prepared real research to be easily used in research or in planning. And that's the easiest thing to reach out and grab. So, but how do you, in that content, how do you keep the human connection? You know, I mentioned great insight titles. You know, a compellingly written, you know, communities can come together and develop standards for how they write insights so that the. Even if they're generated in really different parts of an organization, they feel parallel. I talk about that in the book. But a compelling insight title speaks to a human problem. You know, we can look at it and, you know, immediately sort of grasp what's going on there and we may need to dig in and so research systems link into deeper content to be valuable and, you know, that may even be going to the report or even down to the underlying evidence, depending on the governance situation in your organization. So, you know, we can maintain human connection that way. I think another way is to say every time we're pushing research, existing research, and sort of continuous discovery of what's known rediscovery, I should say the researchers themselves that created that work are present in it. You know, you see their name, you could reach out and contact them, they can be involved in more of those conversations. Information, existing knowledge can be a bridge to researchers and they can answer the detailed questions and be involved and represent. But I think the last way I'd mention is just when we push existing research and we create more visibility for research knowledge, we can also provide more opportunities for people to connect into the next round of research. Whether that's UX research studies sitting in the lab, or being involved in analysis process and market research, or you know, data science or helping customer support make sense of a volume of content, you know, finding opportunities to let people get more directly connected to people and evidence while also sharing the knowledge, things we already know. So it doesn't have to be an either or, it can be an and.
- [32:23]
C
I really, I think that's an important point to keep in mind because you know, really by consciously just choosing to keep people at the center, it there's, it inflects, kind of pushes you in the direction of saying like, how do we deliver on that? Right, yeah, that's an important point. So a lot of the folks who are listening today lead teams and they lead teams that are in a lot of different spaces, marketing, ux, all those things. And if there's one thing that you could tell them that they should take away from today's conversation around reducing research waste, what would it be? And I mean, I think it's kind of like a dual reducing rate the waste and then also ensuring that we're using human centered evidence to make decisions.
- [33:23]
D
Absolutely, yeah. I think those root causes of preparation, motivation and integration are kind of a takeaway to say what could we be doing to prepare our research more, motivate more use of existing research, even if that just means celebrating the people that are doing it right and trying to build those lead users into a more common norm. And then how can we be integrating existing research better into decision making? So there's a whole bunch of ideas about that. But to pick the one thing, I mean to your point, I think you have to get a sense for that there is this problem. Personally, you have to feel it. And so I would say take a few existing reports that were really influential things that made a difference for you that you think made a big difference in the organization that help people think Differently. Maybe not a narrow a B test, but something a little bit broader. And go back and have a look at them, you know, something that's a year old, maybe even 2 years old, and take stock of them. You know, what changed things? We count this one as a win. But what was left behind to get a feel for the cutting room floor, what's in that executive summary or the top insights that, you know, you could look now and say, yeah, we found that five more times, or no, that's definitely. I can apply judgment and say that is still definitely a problem and get that personal sense for it and then think about taking a couple of those things and trying to move them forward. Go talk to those teams about those insights, talk to the researchers, work with designers to visualize solutions, whatever it might be. And by doing that process of recognizing the problem more concretely and the opportunity, there's gold there, and then trying to do something with it, you can start to think about, okay, when I talk to this community of researchers, when I talk to the community of insight users about getting more value from research, what are some things that some barriers, some opportunities, you know, a little experiment, I guess is what I would pitch.
- [35:31]
C
Yeah, yeah, that's a great point and a really great way to round out the conversation for today. And I just want to thank you for joining me for this conversation and want to also give you an opportunity to share how people can follow what it is that you're continuing to publish, your thought leadership and how they go about getting a copy of your book.
- [35:58]
D
Well, thank you again for having me. I've really enjoyed the conversation, appreciate the opportunity and I always welcome connections on LinkedIn. Jake Burkhart, you can find me there and I'm active sharing ideas from the book and kind of building it out over time. Additional ideas. The book is called Stop wasting research. Maximize the product impact of your organization's customer insights and it's available from Rosenfeld Media and a lot of places you could buy books online. And on my website, integratingresearch.com, you can find a link to my monthly newsletter if you wanted to keep in touch. That way I share sort of a summary of what I've been thinking about and some of the things I've published over the last month and would love to connect with you there. So really appreciate the opportunity.
- [36:45]
C
That's awesome. I just, I love the, the whole concept of just connecting with your team's brilliance already. So if there's anything that I would love for people to take away, that would be what it would be. So thanks again for the conversation. Hope to connect again soon.
- [37:01]
D
Thank you.
- [37:03]
B
Want to keep the conversation going? You can find the show notes@usertesting.com podcast if you haven't already. Don't forget to follow us on Apple Podcasts, Spotify, Overcast, or Google Play so you never miss an episode. And if you enjoyed today's show, please share it with a friend or leave us a rating and review on Apple Podcasts. And until next time, this is Insights Unlocked, an original podcast from User Testing.