← The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
#685: Using AI in UX research, design, and testing with Jason Bowman, The Office of Experience
00:25:54
Loading summary
Transcript49 lines
- [00:00]
Greg Kilstrom
Your brand may be staying on top of current trends, but are you agile enough to stay relevant, resilient and successful as customers, competition and the world continues to change at a breakneck pace? I'm thrilled to share the newly revised version of my first book, the Agile Brand. I'm calling it the Agile Brand Revisited. It's been updated to reflect our continually changing world and it provides seven principles that form the backbone of an agile brand, offering detailed insights and actionable steps for incorporating them into your business strategy. This is the book that started it all and I'm excited to share it with you. It's now available in print and digital formats and available everywhere. Learn more by going to the Agile Brand guide website at www.agilebrandguide.com. the Agile Brand welcome to season seven of the Agile Brand where we discuss the trend, trends and topics marketing leaders need to know. Stay curious, stay agile and join the top enterprise brands and Martech platforms as we explore marketing technology, AI, E commerce and whatever's next for the Omnichannel customer experience. Together we'll discover what it takes to create an agile brand built for today and tomorrow and built for customers, employees and continued business growth. I'm your host Greg Kilstrom, advising Fortune 1000 brands on martech, AI and marketing operations. The Agile Brand Podcast is brought to you by Tech Systems, an industry leader in full stack technology services, talent services and real world application. For more information, go to teksystems.com to make sure you always get the latest episodes, please hit subscribe on the app you listen to podcasts on and leave us a rating so others can find us as well. Now onto the show. This episode is brought to you by the Office of Experience, a design driven digital first vertically integrated and collaborative agency that believes in the power of ideas and the strength of people. Learn more@officeofexperience.com Agility requires finding ways to stay one step ahead of the competition as well as in anticipating customers needs. So how does a brand maintain the speed and agility in the area of UX design, where it has often taken a considerable amount of time, effort and testing to get to a better result. Today we're going to talk about using AI strategically in UX research, design and testing. Tell me to discuss this topic, I'd like to welcome Executive Director of Experience Strategy at the Office of Experience, Jason Bowman. Jason, welcome to the show.
- [02:37]
Jason Bowman
Thanks. Thanks for having me.
- [02:39]
Greg Kilstrom
Yeah, looking forward to and welcome back to the show I should say as well.
- [02:43]
Jason Bowman
Yes, yes, been here before.
- [02:45]
Greg Kilstrom
Looking forward to talking again for Those that missed when you were here before. Why don't we start with you giving a little background on yourself and your role at the Office of experience?
- [02:54]
Jason Bowman
Sure. So I've been in the digital making practice since the early.com 2.0.0 day, 1.0 days. I've worked everything from agency to in house as like a team lead to startups, you know, mobile, the whole thing basically. I like to say if it's been digital, I probably built a version of it at some point in my life and been at OX for about four years. Started leading the UX team and now I manage the content strategy and the business analyst team as well. Just kind of a nice way of taking that strategy all the way through from concept content and need, all the way through execution, through development. Just kind of keeping that thread tight.
- [03:38]
Greg Kilstrom
Yeah, love it. Well, yeah, let's dive in here. And so we're going to talk about a few things today, but I want to start by talking about how artificial intelligence is impacting the user experience practice in a few different ways. So first let's start with talking about how AI is impacting research. What are you seeing there?
- [03:59]
Jason Bowman
AI is huge in research. What I find is it's really speed up the process. So before teams would run to Google, they'd be searching through a lot of things. You'd be looking at best practice documents, comparing a couple of different people's best practices, reading up on things and what AI is helpful is it starts to either a get you started in one direction, summarize something, or just kind of like even it's getting so good that now it's even starting to analyze things. We started doing some things such as heuristics, where hey, here's a site, here's our framework. You know, it asks a few questions and it starts to spit out results that are about 80% of what we need. So it's really been fun as we've gotten deeper in the agency of training the engine and training the agent to be what we need by kind of creating scripts for it and going through. So that's great with heuristics Personas, it's great at giving you kind of rough ballpark Personas, but also by allowing it to feeding it Personas that you've already created and then reacting to some of the work or some of the ideas that you're presenting and how it might be better or worse. So that I think we talk a little bit of this later, but it starts to talk about how some of the testing some of the validation can happen early on. And, you know, even just overall content strategy, we found like, hey, we've got this kind of a site, or here's this kind of a page. What are some recommendations on versions of content strategy? Where might I lean? This way, it kind of allows you to test out ideas. And we tend to use Claude a lot at the Office of Experience. And with that, it can even start to code up a quick wireframe, give you a visual reference. It just kind of lets you play with some things quicker. We always, we, we encourage the use of Claude, the Internet, anything really, to help you create that thing. The only kind of rule we have with our team is make sure that whatever insight you get, make sure you agree with it, you validate it, and you can stand behind it.
- [06:05]
Greg Kilstrom
Yeah.
- [06:06]
Jason Bowman
Not just say like, because it's like, you know, if you, before AI, you might go to a competitor's website and you could pick up their landing page and design it just like that. And if I ask, why is it this way? Or the client asks, why is it this way? We simply say, well, because they had it right. That's not a good answer. And that doesn't help your case. The same rule applies to AI. It's like, well, why did you do that? Well, the AI gave it to you. Well, again, that doesn't really matter. It has to. You have to ingest it and apply it to your scenario and have it make sense.
- [06:39]
Greg Kilstrom
Yeah, yeah, yeah, it makes sense. And I mean, you know, even, you know, to your point about AI getting something 80% there or even 50% there, you know, what, whatever it is, it's, it's getting things further, quicker. So the, the key part as well is so that a human can validate it and make sure, you know, leveraging their expertise as well. So, like, I like that dynamic of, you know, let's use AI to get us further, quicker, faster, but let's use the humans to do what humans do best as well. Right, right.
- [07:13]
Jason Bowman
Because if you can get what would take a half day of research and you can kind of compile that into 15 minutes now, it gives you that kind of window or some pathways to do deeper research, maybe to ask, follow questions or dig in something deeper, but you've been able to consolidate and gather more information of much quicker time, it's super helpful in that way.
- [07:36]
Greg Kilstrom
Yeah, absolutely. So let's talk about the area of predictive validation. And maybe for those a little less familiar with that term, why don't you start with defining what predictive validation is, but also Talk a little bit about how AI can help there.
- [07:52]
Jason Bowman
Sure. So in user test, in user experience, one of the best ways to learn is actual users. We spend a lot of time creating, but then it's really about understanding how what you created actually applies. Because users are fascinating beasts of breaking things or doing what they're not told, or sometimes you just end up drinking your own kool Aid and you think that everything you're making is amazing and users tell you different. So what predictive validation does is it allows us before we go all the way to publishing or testing or maybe even before we run a full user test, you're able to just gut check things with that predictive validation. So you feed an artifact to AI, ask it some questions, it feeds it back to you. We talked a little bit about earlier, earlier about that with Personas, how you can say hey, here's this object based on these Personas. How do you think they're going to react to this content A or B or see this valuable or think this headline it can start to, to gut check. The funny thing about that was I was talking to one of the coworkers, my coworkers, like so what happens if you know, we need to start building in a little more time for those internal cycles and they're like, oh well see with AI you just change the script until you get the answer you want. So it can be good but as we, we can also manipulate it far easier.
- [09:15]
Greg Kilstrom
Right.
- [09:16]
Jason Bowman
But then with virtual audience testing, same thing with Personas. And then heat maps are really handy because you know, heat maps often tell scrolling and click eye tracking and you can some really easy to, to shoot up a comp, a wireframe or something and just kind of get a sense of what's where your contrast levels are, where the eye might be drawn. And it doesn't take the place of user testing, but it really does kind of help validate some things internally.
- [09:45]
Greg Kilstrom
Yeah, yeah.
- [09:46]
Jason Bowman
And then even when doing testing kind of in back to the research, it's great. And a lot of the testing sites already have this engine built in, but if you've got a bunch of recordings, you have a lot of surve great for data crunching, finding trends in large amounts of data, just spitting out roughly what it should do. So again really time saving and compiling and doing some of those things.
- [10:08]
Greg Kilstrom
Yeah. And I mean this is what I'm hearing as well. You know, when I talk with brands is, you know, it doesn't. They're still doing human research, but what they're doing, they're Getting better things to put in front of those humans by doing, you know, synthetic, by doing AI generated research and prototyping and all that stuff. So in other words, their, their results are better because they're, you know, they're 80% of the way there, you know, kind of, kind of to the earlier point. So it just, it not only, you know, all, all of the efficiencies of not having to take a human's time to do all the mockups and the research and all that stuff, but it's also enhancing that, that speed to getting the best solutions. Can you talk maybe a little bit more about that as well?
- [10:53]
Jason Bowman
Sure. I mean, AI like any kind of machine or something like that, or even just user groups, they're never going to be 100% right or perfect or anything like that. They're going to have kind of an inherent bias. But if we understand that inherent bias and we're using it to help generate content, we know there's a bias, then you can still trust it to test it with that same bias. So you take that in consideration. But it really is that way of, I mean, just for example, so let's just say that we have a disagreement internally or maybe even with the client, but, well, to say internally, it makes it easier. I think something should be one way, maybe the creative director thinks something should be another way. And there's a little bit of a debate and we don't really know who's right and no one really has the authority to overrule each other. So hey, why don't we just run this through this real quick and see and then you just kind of quickly what we used to do. Well, we'll set up an a B test. All right, let's get the prototypes. Let me spin it up. You could probably set a good a B test up in a couple hours. Then you wait a couple hours to get the results back. Then you got to read through it and you basically spend a day to validate something. This you can probably validate in five minutes. You know, if, you know, kind of, if you're good with prompts or you've prompted enough and you know, hey, here's this, here's this. What questions do you have for us in order to get some validation, respond that. And then it can kind of give you guidance and then give you some reasoning behind it. And it just takes some of the bias out of it. Right. It's just doing a general knowledge. Here's what this thing thinks can be wrong and you can override it just like any sort of test in your opinion, but it really is helpful in that way.
- [12:38]
Greg Kilstrom
What if your customer experience and culture strategy didn't just create value but actually funded your growth? I'm inviting you to join me in a powerful mastermind group I'm co leading with Journey Spark Consulting. It's called the CX and Culture Connection, Creating a self funding growth flywheel and it brings together CX and culture leaders who want to drive change, build stronger internal collaboration and actually reinvest efficiency savings into growth driving initiatives. This isn't just theory. You'll get monthly virtual sessions, one on one coaching, quarterly workshops and access to the value Accelerator tools focused on strategy, cultural alignment and voice of customer. Plus you'll connect with leaders across functions from marketing to product to ops who are facing the same challenges and pushing towards the same goals. Up to four team members can participate from your organization, so you're building alignment while building momentum. Want in? Learn more and sign up at journeysparkconsulting.com mastermind that's journeysparkconsulting.com Mastermind let's turn your CX investment into a growth engine. Want to learn more and join the discussion about marketing and AI? Attend a premier conference dedicated to marketing and AI. That's Meikon, the Marketing Artificial Intelligence Conference from October 14 through 16 in Cleveland, Ohio. Meacon brings together the brightest minds and leading voices in AI. Don't miss this opportunity to connect with a dynamic community of experts, visionaries and enthusiasts. The Agile brand is proud to be the lead media sponsor of this important event. Register today@MarketingAIInstitute.com that's MarketingAI Institute.com and use the code AGILE150 for $150 off your registration fee. I can't wait to see you there. Let's talk a little bit. You know now that just about everybody, if not everybody, has been using AI to some degree, I assume most people listening to the show have used something at least if not many things. You know, we've all seen the good and some of the not so good you mentioned some things like, you know, hallucinations, other things like that are possible certainly, but in the good scenarios, AI can be thought of as that collaboration partner, much as you've you've described so far. So let's talk a little bit about how UX needs to participate with AI to get the best results and maybe we'll we'll start with a negative and then we'll, we'll get a little more positive here, but start with the not so good. You Know, AI is prone to some odd behaviors, like things like hallucinations and other things. What should UX teams know when working with, with AI?
- [15:29]
Jason Bowman
I think we should just remember what the Internet taught us. Like.
- [15:33]
Greg Kilstrom
Right.
- [15:34]
Jason Bowman
You just can't believe it. Like you, you, it's a great starting point, but if it cites a study, you should make sure that it's really citing a study. It didn't just make up a study to cite. You know, you should double check if a fact seems really interesting, make sure you have the source. Right. Or you know, like if it's a study or research, you validate that with some other sources. So that's, that's just super important. I, you know, as we said, you could manipulate it. There's a chance it's feeding you too much what it wants with your prompt. We were joking. Sometimes, like if Claude was an actual person in the office, they, they'd do anything for you. Hey, Claude. Yeah, I love to do that. You know, let me go get you some coffee. And then they come back with like a milkshake. And you're like, right. And we're like, well, thank you. This is so handy. I didn't even know I needed this. So.
- [16:20]
Greg Kilstrom
Right, right.
- [16:21]
Jason Bowman
You know, really kind of just validating it and, and also discipline yourself in that it's not a magic genie. And, and just like all computers, it needs data and good data to run. You can't just say, tell me my five year projection and not give it any source material. It's going to be like, well, you know what, I don't know what you're saying. So.
- [16:42]
Greg Kilstrom
Right.
- [16:43]
Jason Bowman
The discipline of even creating a good prompt, asking, good question, having an idea what the results should be and just kind of guiding it down that path is a great way to get the most out of it. But you just, you have to validate, you have to know that it is still learning.
- [17:02]
Greg Kilstrom
Yeah.
- [17:03]
Jason Bowman
It's not a magic button.
- [17:04]
Greg Kilstrom
Yeah, yeah. And you know, in addition to, I mean, certainly, you know, a lot of leaders, managers, you know, just companies in general are under a lot of pressure to adopt AI for the, you know, the speed and efficiency things. They're also under a lot of pressure to innovate. Right. And so this, I think this is a, this is an interesting topic here with, with AI, you know, is, is it really coming up with visionary ideas or is it the humans that are coming up with ideas? You know, what, what role do, do humans and AI play in innovation?
- [17:37]
Jason Bowman
I think sometimes people use AI in place of innovation. Like we're using AI. And that's your innovation button. So there's that. I think, you know, there's also a lot of. We kind of call it the snake oil right now of everyone's like, ooh, well, I've got an AI engine that can take a creative campaign from concept, write the brief, build off of the brief, and deliver final copy. It's like, that sounds too good.
- [18:04]
Greg Kilstrom
Sounds great. Yeah.
- [18:06]
Jason Bowman
But I think innovation still happens. Like it. I think we. AI tends to take all the knowledge it can get, find the themes, and build off of that. But sometimes, much like best practices or much like groupthink or pop music, you could say, you know, it just becomes what people want. And sometimes that's great, but it's not always pushing the boundaries of a new. If we. If we take music, for example, like a new genre, it's not. Someone's not experimenting. AI is not going to necessarily experiment on its own based on your prompt.
- [18:43]
Greg Kilstrom
Yeah.
- [18:44]
Jason Bowman
You know, and so, you know, asking it, show me something new, it might not really do that. I think there's still that benefit of, of that we're able to do, because again, it builds off prompts, and in our heads and in our collective groups, we have hundreds of thousands of prompts that we don't even know exist. Things that we saw 10 years ago.
- [19:07]
Greg Kilstrom
Right.
- [19:08]
Jason Bowman
A movie I watched, that somehow there's an action in there that makes me think of this, and now I think of something different. And so I think that's where. Not that humans are better, but we are still necessary to innovate because I think we don't know all the prompts that we might have. So our brains, you know, we just can't get the AI to act like us because it doesn't have all the background that we have.
- [19:30]
Greg Kilstrom
Yeah, I think another. Another way of. Of looking at it too. You know, AI, like generative AI, is designed to predict the next most likely thing to be next to the. You know, so in the case of words, it's like, what's the next most likely word to show up? Which is not creative. I mean, because you can make the argument that, I mean, there's nothing new under the sun or, you know, whatever cliche you want to use. Like, humans keep reinventing things as well. But I think the. The important distinction is, is what you're saying as well is we draw from a much broader array of things, whereas AI is actually designed to predict the. The most likely and the most reasonable thing to show up next, because if it didn't, like, it would just be complete chaos and garbage if I did it. Right.
- [20:18]
Jason Bowman
Right. And it also sometimes will give the same answer to the same question over and over my.
- [20:24]
Greg Kilstrom
Right.
- [20:24]
Jason Bowman
My sister does some teaching and, you know, she gives a problem to students and she can tell the ones who copied her thing, put it in ChatGPT. And it comes up basically the same thing where even if two humans were reading the same material, they might write it differently.
- [20:38]
Greg Kilstrom
More.
- [20:39]
Jason Bowman
We would kind of diverge more from that. And as you said, AI is not made to do that.
- [20:45]
Greg Kilstrom
Yeah, yeah. Last thing I want to talk about here is, you know, just because you can doesn't mean you should. Right? So the same goes for AI Much, much as we've talked about a little bit here already. You know, when's the right time to use AI and when is it not the right time? And, you know, what should UX teams use to guide them here?
- [21:06]
Jason Bowman
I think AI is great. The beginning of. I mean, I think AI is good to use at any time in the process. I don't know that there's a right time or a wrong time. There's times to refine, there's times to look. But one of the things that, you know, we kind of talk about it, it lacks a bit of editing as well. So I can say, hey, I build me a homepage or build me, you know, a business analyst analytics tool, right? And it's like, I want these inputs, I want this, I want this output, I want this, I want this. And it can, you know, code me up a working prototype. But the problem is, is maybe I have too many numbers that don't intuitively make sense. Maybe it looks too much like a spreadsheet. Maybe it looks too much like this. So just because AI created an artifact doesn't mean that it's the right artifact to use. It doesn't mean that it can't be enhanced. It doesn't mean that it doesn't need user experience. Again, you know, there's a lot of brilliant people out there, but they may not be as skilled in one craft or the other. So they need that help editing, Right? You know, like, I can watch YouTube videos on how to build a house, but from actually doing it, someone who knows how to build a house has more tricks and tips and can guide me. It's like, I know it says that, but you don't need that. This is what you need for this state, for this instance or whatever, right? So the editing, the refinement, the making sure it makes sense, making sure it's not too verbose, rewriting checking headlines. I just think there's, there comes a time where it's easier to almost take it away from AI and just work it through than to continue to over iterate with AI because sometimes it doesn't have a history. Like we'll change that and then it changes five things. Like no, no, no, go back to the previous version and change that one thing down to the left, to the right. And sometimes it's just easier for us to do it ourselves.
- [22:57]
Greg Kilstrom
Yeah, yeah, totally agree. Well, Jason, thanks so much for joining today. One last question for you before we wrap up. What do you do to stay agile in your role and how do you find a way to do it consistently?
- [23:10]
Jason Bowman
I would say one of the ways we do it is I actually just follow my team a little bit. You give the team a little bit of a leash and they start moving in something and, and always kind of being open to new ideas that different people bring. I think that's when you start building ideas. On top of each new ways of working, new thoughts come. Something that you never would have considered suddenly comes into practice because someone else is like, well, we just tried this. It's like you tried that and it worked for you. They're like, yeah, it's like, then maybe we should try that again. And just being open to where ideas come from. From all sorts of different teams and disciplines or thoughts.
- [23:46]
Greg Kilstrom
Love it. Love it. Well, again I'd like to thank Jason Bowman, Executive Director of Experience Strategy at the Office of Experience for joining the show. You can learn more about Jason and the Office of Experience by following the links in the show notes. And thanks again to the Office of Experience for sponsoring this episode. Thanks again for listening to the Agile Brand brought to you by Tech Systems. If you enjoyed the show, please take a minute to subscribe and leave us a rating so that others can find the show as well. You can access more episodes of the show@theagilebrand.com that's the agile Brand and contact me if you're interested in consulting or advisory services or are looking for a speaker for your next event, go to www.gregkilstrom.com that's G R E G K I H L S t r o m.com the Agile brand is produced by Missing Link, a Latina owned, strategy driven, creatively fueled production co op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. Until next time, stay curious and stay agile. The Agile Brand. Ever heard of Farm to table? How about Farm to Home? That's how Costa Farms Plant Business works with over 1500 plant varieties grown over 5200 acres. They're not just a company, they're your plant partners who've been perfecting their craft for 60 years. They deliver beautiful, high quality, easy to care for plants. They even offer virtual plant consultations and an insider club for rare plant access. Check out www.costafarms.com today and enter code worthknowing costafarms15 for a 15% discount on your first purchase. You can also purchase this unique plant brand at Lowe's, Walmart, Amazon and Home depot. Go to www.costacostafarms.com today. Before we continue, I wanted to share a key strategic resource that a majority of the Fortune 500 are already aware of. Finding the best technology, business and and talent solutions is not easy. With business demands and competitive pressures mounting, you need to be able to design, deploy, and optimize your technology to provide leading customer experiences while driving business growth. Those of you that have been listening to this show for a while know that this podcast is brought to you by Tech Systems, a global provider of technology, business and talent solutions for more than 80% of the Fortune 500. Tech Systems accelerates business transformation for their customers. Whether you're looking to maximize your technology ROI, drive business growth, or elevate customer experiences, TechSystems enables enterprises to capitalize on change. Learn more at techsystems.com that's T E K systems.com now let's get back to the show.