← B2B Agility with Greg Kihlström™: MarTech, E-Commerce, & Customer Success
B2B Agility with Greg Kihlström™: MarTech, E-Commerce, & Customer Success
#47: The power of AI in UX research and design with Jason Bowman, The Office of Experience
00:21:35
How does a B2B brand maintain speed and agility in the area of UX design, where it has often taken a considerable amount of time, effort, and testing to get to a better result? Today we’re going to talk about using AI strategically in UX...
Loading summary
Transcript82 lines
- [00:00]
Zoe Saldana
Hi, Zoe Saldana. Welcome to T Mobile. Here's your new iPhone 16 Pro on us.
- [00:04]
Customer
Thanks. And here's my old phone to trade in.
- [00:07]
Zoe Saldana
You don't need a trade in. When you switch to T Mobile, we'll give you a new iPhone 16 Pro. Plus we'll help you pay off your old Phone up to 800 bucks and you still get to keep it.
- [00:15]
Customer
There's always a trade in.
- [00:17]
Zoe Saldana
Not right now. At T Mobile.
- [00:18]
Customer
I feel like I have to give you something in return for karma.
- [00:20]
Zoe Saldana
That's okay.
- [00:21]
Customer
I don't really have much in my purse. Oh, let's see. Hand sanitizer. It's lavender.
- [00:26]
Zoe Saldana
I'm good.
- [00:27]
Jason Bowman
Seriously.
- [00:28]
Customer
Let me check this pocket. Oh, mints.
- [00:30]
Zoe Saldana
Really, I'm fine.
- [00:31]
Customer
Oh, I have raisins. I'm a mom. Wait, wait one sec. I've got cupcakes in the car.
- [00:35]
T Mobile Representative
It's our best iPhone offer ever. Switch to T Mobile. Get a new iPhone 16 Pro with Apple intelligence on us. No trade in needed. We'll even pay off your phone up to 800 bucks with 24 monthly bill credits.
- [00:48]
T Mobile Ad Voice
New line $100 plus a month on experience beyond Finance Agreement 999.99 and qualify imported for well qualified plus tax and $10 connection charge. Payout via virtual prepaid card. Allow 15 days credits end and balance due if you pay off early or cancel.
- [00:59]
Ryan Reynolds
C mobile.com Ryan Reynolds here from Mint Mobile with a message for everyone paying big wireless way too much. Please, for the love of everything good in this world, stop with Mint. You can get premium wireless for just $15 a month. Of course, if you enjoy overpaying. No judgments. But that's weird. Okay, one judgment anyway, give it a try@mintmobile.com Switch upfront payment of $45 for.
- [01:23]
Customer
Three month plan equivalent to $15 per month required intro rate first three months only, then full price plan options available, taxes and fees extra. See full terms@mintmobile.com the Agile Brand.
- [01:38]
Greg Kilstrom
Welcome.
- [01:39]
Podcast Host
To the B2B Agility Podcast where we look at the factors that drive success in B2B marketing with a focus on the people, processes, data and platforms that make B2B brands stand out and thrive in a competitive marketplace. I'm your host Greg Kilstrom advising Fortune 1000 brands on MarTech marketing operations and CX best selling author and speaker. Now let's get on to the show.
- [02:04]
Greg Kilstrom
This episode is brought to you by the Office of Experience, a design driven digital first vertically integrated and collaborative agency that believes in the power of ideas and the strength of people. Learn more@officeofexperience.com Agility requires finding ways to stay one step ahead of the competition as well as in anticipating customers needs. So how does a brand maintain the speed and agility in the area of UX design where it has often taken a considerable amount of time, effort and testing to get to a better result? Today we're going to talk about using AI strategically in UX research, design and testing. To help me discuss this topic, I'd like to welcome Executive Director of Experience Strategy at the Office of Experience, Jason Bowman. Jason, welcome to the show.
- [02:49]
Jason Bowman
Thanks, thanks for having me.
- [02:51]
Greg Kilstrom
Yeah, looking forward to, and welcome back to the show. I should, I should say as well.
- [02:55]
Jason Bowman
Yeah. Yes.
- [02:56]
Greg Kilstrom
Been here before, looking forward to talking again. For those that missed when you, when you were here before, why don't we start with you giving a little background on yourself and your role at the Office of Experience.
- [03:06]
Jason Bowman
Sure. So I've been in the, you know, digital making practice since the early, you know, dot com 2.0 day, 1.0 days. I've worked everything from agency to in house as like a team lead to startups, you know, mobile, the whole thing. Basically. I like to say if it's been digital, I probably built a version of it at some point in my life and been at OX for about four years. Started leading the UX team and now I manage the content strategy and the business analyst team as well. Just kind of a nice way of, kind of taking that strategy all the way through from concept content and need all the way through execution, through development. Just kind of keeping that thread type.
- [03:50]
Greg Kilstrom
Yeah, yeah, love it. Well, yeah, let's, let's dive in here. And so we're going to talk about a few things today, but I want to start by talking about how artificial intelligence is impacting the user experience practice in a few different ways. So first let's, let's start with talking about how AI is impacting research. What are you seeing there?
- [04:11]
Jason Bowman
AI is huge in research. What I find is it's really speed up the process. So before teams would run to Google, they'd be searching through a lot of things. You'd be looking at best practice documents, comparing a couple of different people's best practices, reading up on things. And what AI is helpful is it starts to either a get you started in one direction, summarize something or just kind of like even it's getting so good that now it's even starting to analyze things. We started doing some things such as heuristics where hey, here's a site, here's our framework, it asks a few questions and it starts to spit out results that are about 80% of what we need. So it's really been fun as we've gotten deeper in the agency of training the engine and training the agent to be what we need by creating scripts for it and going through. So that's great with heuristics Personas. It's great at giving you kind of rough ballpark Personas, but also by allowing it to feeding it Personas that you've already created and then reacting to some of the work or some of the ideas that you're presenting and how it might be better or worse. So that I think we talk a little bit of this later. But it starts to talk about how some of the testing, some of the validation can happen early on. And, you know, even just overall content strategy, we found like, hey, we've got this kind of a site, or here's this kind of a page. What are some recommendations on versions of content strategy? What. Where might I lean? This way, it kind of allows you to test out ideas. And we tend to use Claude a lot at the Office of Experience. And with that, it can even start to code up a quick wireframe, give you a visual reference. It just kind of lets you play with some things quicker. We encourage the use of Claude, the Internet, anything really, to help you create that thing. The only kind of rule we have with our team is make sure that whatever insight you get, make sure you agree with it, you validate it, and you can stand behind it. Not just say like, because it's like, you know, if you before AI, you might go to a competitor's website and you could pick up their landing page and design it just like that. And if I ask, why is it this way? Or the client asks, why is it this way? And we simply say, well, because they had it right. That's not a good answer. And that doesn't help your case. The same rule applies to AI. It's like, well, why did you do that? Well, the AI gave it to you. Well, again, that doesn't really matter. It has to. You have to ingest it and apply it to your scenario and have it make sense.
- [06:51]
Greg Kilstrom
Yeah, yeah, yeah, it makes sense. And I mean, even, you know, to your point about AI getting something 80% there or even 50% there, you know, whatever it is, it's. It's getting things further quicker. So the key part as well is so that a human can validate it and make sure, you know, leveraging their expertise as well. So, like, I like that dynamic of, you know, let's use AI to get us further, quicker, faster, but let's use the humans to do what humans do best as well. Right, right.
- [07:25]
Jason Bowman
Because if you can get what would take a half day of research and you can kind of compile that into 15 minutes now, it gives you that kind of window or some pathways to do deeper research, maybe to ask follow up questions or dig in something deeper, but you've been able to consolidate and gather more information in a much quicker time. It's super helpful in that way.
- [07:48]
Greg Kilstrom
Yeah, absolutely. So let's talk about the area of predictive validation and maybe for those a little less familiar with that term, why don't you start with defining what predictive validation is, but also talk a little bit about how AI can help there.
- [08:03]
Jason Bowman
Sure. So in user test, in user experience, one of the best ways to learn is actual users. We spend a lot of time creating, but then it's really about understanding how what you created actually applies. Because users are fascinating beasts of breaking things or doing what they're not told, or sometimes you just end up drinking your own Kool Aid and you think that everything you're making is amazing and users tell you different.
- [08:26]
Greg Kilstrom
Right.
- [08:27]
Jason Bowman
So what predictive validation does meet is it allows us before we go all the way to publishing or testing, or maybe even before we run a full user test, you're able to just gut check things with that predictive validation. So you feed an artifact AI, ask it some questions, it feeds it back to you. We talked a little bit about earlier, earlier about that with Personas, how you can say, hey, here's this object based on these Personas, how do you think they're going to react to this content A or B? Or see this valuable or think this headline, it can start to gut check. The funny thing about that was I was talking to one of the coworkers, my coworkers, like, so what happens if, you know, we need to start building in a little more time for those internal cycles and they're like, oh, well see with AI you just change the script until you get the answer you want. So it can be good, but as we, we can also manipulate it far easier. But then with virtual audience testing, same thing with Personas. And then heat maps are really handy because you know, heat maps often tell scrolling and click and eye tracking and you can some really easy to, to shoot up a comp, a wireframe or something and just kind of get a sense of what's where your contrast levels are where the eye might be drawn and it doesn't take the place of user testing, but it really does kind of help validate some things internally.
- [09:57]
Greg Kilstrom
Yeah, yeah.
- [09:58]
Jason Bowman
And then even when doing testing kind of in back to the research, it's great. And a lot of the testing sites already have this engine built in, but if you've got a bunch of recordings, you have a lot of surveys great for data crunching, finding trends in large amounts of data, just spitting out roughly what it should do. So again really time saving and compiling and, and doing some of those things.
- [10:20]
Greg Kilstrom
Yeah, and I mean this is, this is what I'm, I'm hearing as well, you know, when I talk with, with brands is, you know, it doesn't, they're still doing human research but what they're doing, they're getting better things to put in front of those humans by doing, you know, synthetic, by doing AI generated research and prototyping and all that stuff. So in other words, their, their results are better because they're, you know, they're 80% of the way there, you know, kind of, kind of to the earlier point. So it just, it not only, you know, all, all of the efficiencies of not having to take a human's time to do all the mock ups and, and the research and all that stuff, but it's also enhancing that, that speed to, to getting the best solutions. Can you talk maybe a little bit more about that as well?
- [11:05]
Jason Bowman
Sure. I mean AI like, like any kind of machine or something like that or even just user groups, they're never going to be 100% right or perfect or anything like that. They're going to have kind of an inherent bias. But if we're used, if we understand that inherent bias and we're using it to help generate content, we know there's a bias, then you can still trust it to test it with that same bias. So you take that in consideration. But it really is that way of, I mean just for example, so let's just say that we have a disagreement internally or maybe even with the client. But well, to say internally it makes it easier. I think something should be one way, maybe the creative director thinks something should be another way and there's a little bit of a debate and we don't really know who's right and no one really has the authority to overrule each other. So hey, why don't we just run this through this real quick and see and then you just kind of quickly what we used to do. Well, we'll set up an a B test, all right? Let's get the prototypes. Let me spin it up. You could probably set a good a B test up in a couple hours. Then you wait a couple hours to get the results back. Then you gotta read through it and you basically spend a day to validate something. This you can probably validate in five minutes. You know, if, you know, kind of, if you're good with prompts or you prompted enough and you know, hey, here's this, here's this. What questions do you have for us in order to get some validation? Respond that. And then it can kind of give you guidance and then give you some reasoning behind it. And it just takes some of the bias out of it. Right? It's just doing a general knowledge. Here's what this thing thinks can be wrong and you can override it just like any sort of test, in your opinion. But it really is helpful in that way.
- [12:52]
Odoo Voice
So when I ask, what is Odoo, what comes to mind? Well, Odoo is a bit of everything. Odoo is a suite of business management software that some people say is like fertilizer because of the way it promotes growth. But you know, some people also say Odoo is like a magic beanstalk because it grows with your company and is also magically affordable. But then again, you could look at Odoo in terms of how its individual software programs are a lot like building blocks. I mean, whatever your business needs, manufacturing, accounting, HR programs, you can build a custom software suite that's perfect for your company. So what does Odoo? Well, I guess Odoo is a bit of everything. Odoo is a fertilizer, magic beanstalk, building blocks for business. Yeah, that's it. Which means that Odoo is exactly what every business needs. Learn more and sign up now@odoo.com. that's O D O O dot com.
- [13:54]
Greg Kilstrom
Let's talk a little bit. You know, now that just about everybody, if not everybody, has been using AI to some degree, I assume most people listening to the show have something, at least if not many things. You know, we've all seen the good and some of the not so good. You mentioned some things like, you know, hallucinations, other, other things like that are possible, certainly, but in the good scenarios, AI can be thought of as that collaboration partner, much as you've, you've described so far. So let's talk a little bit about how UX needs to participate with AI to get the best results. And maybe we'll, we'll start with the negative and then we'll, we'll get a little more positive here, but start with the not so good. You know, AI is prone to some odd behaviors, like things like hallucinations and other things. What should UX teams know when working with, with AI?
- [14:48]
Jason Bowman
I think we should just remember what the Internet taught us. Like.
- [14:52]
Greg Kilstrom
Right.
- [14:52]
Jason Bowman
You just can't believe it. Like you, you, it's a great starting point, but if it cites a study, you should make sure that it's really citing a study. It didn't just make up a study to cite. You know, you should double check if a fact seems really interesting, make sure you have the source. Right. Or you know, like if it's a study or research, you validate that with some other sources. So that's, that's just super important. I, you know, as we said, you could manipulate it. There's a chance it's feeding you too much what it wants with your prompt. We were joking. Sometimes, like if Claude was an actual person in the office, they, they do anything for you. Hey, Claude. Yeah, I love to do that. You know, let me go get you some coffee. And then they come back with like a milkshake and you're like, right. And we're like, well, thank you. This is so handy. I didn't even know I needed this. So.
- [15:39]
Greg Kilstrom
Right, right.
- [15:40]
Jason Bowman
You know, really kind of just validating it. And, and also discipline yourself in that it's not a magic genie. And, and just like all computers, it needs data and good data to run. You can't just say, tell me my five year projection and not give it any source material. It's going to be like, well, you know what, I don't know what you're saying. So.
- [16:01]
Greg Kilstrom
Right.
- [16:01]
Jason Bowman
The discipline of even creating a good prompt, asking, good question, having an idea what the result should be and just kind of guiding it down that path is a great way to get the most out of it. But you just, you have to validate, you have to know that it is still learning.
- [16:20]
Greg Kilstrom
Yeah.
- [16:21]
Jason Bowman
It's not a magic button.
- [16:23]
Greg Kilstrom
Yeah, yeah. And you know, in addition to, I mean, certainly, you know, a lot of leaders, managers, you know, just companies in general are under a lot of pressure to adopt AI for the, you know, the speed and efficiency things. They're also under a lot of pressure to innovate. Right. And so this, I think this is a, this is an interesting topic here with, with AI, you know, is, is it really coming up with visionary ideas or is it the humans that are coming up with ideas? You know, what, what role do, do humans and AI play in innovation?
- [16:56]
Jason Bowman
I Think sometimes people use AI in place of innovation. Like we're using AI, and that's your innovation button. So there's that. I think, you know, there's also a lot of. We kind of call it the snake oil right now of everyone's like, ooh, well, I've got an AI engine that can take a creative campaign from concept, write the brief, build off of the brief, and deliver final copy. It's like, that sounds too good.
- [17:23]
Greg Kilstrom
Sounds great. Yeah.
- [17:25]
Jason Bowman
But I think innovation still happens. Like, it. I think we. AI tends to take all the knowledge it can get, find the themes, and build off of that. But sometimes, much like best practices or much like groupthink or pop music, you could say, you know, it just becomes what people want. And sometimes that's great, but it's not always pushing the boundaries of a new. If we. If we take music, for example, like a new genre, it's not. Someone's not experimenting. AI is not going to necessarily experiment on its own based on your prompt.
- [18:02]
Greg Kilstrom
Yeah.
- [18:02]
Jason Bowman
You know, and so, you know, asking it, show me something new, it might not really do that. I think there's still that benefit of. Of that we're able to do, because again, it builds off prompts, and in our heads and in our collective groups, we have hundreds of thousands of prompts that we don't even know exist. Things that we saw 10 years ago.
- [18:26]
Greg Kilstrom
Right.
- [18:26]
Jason Bowman
A movie I watched, that somehow there's an action in there that makes me think of this, and now I think of something different. And so I think that's where. Not that humans are better, but we are still necessary to innovate because I think we don't know all the prompts that we might have. So our brains, you know, we just can't get the AI to act like us because it doesn't have all the background that we have.
- [18:49]
Greg Kilstrom
Yeah. I think another. Another way of. Of looking at it too. You know, AI, like generative AI, is designed to predict the next most likely to be next to the. You know, so in the case of words, it's like, what's the next most likely word to show up? Which is not creative? I mean, because you can make the argument that, I mean, there's nothing new under the sun or, you know, whatever cliche you want to use. Like, humans keep reinventing things as well. But I think the. The important distinction is, is what you're saying as well is we draw from a much broader array of things, whereas AI is actually designed to predict the most likely and the most reasonable thing to show up next, because if it didn't, like, it would just be complete chaos and garbage if AI did it. Right, right.
- [19:38]
Jason Bowman
And it also sometimes will give the same answer to the same question over and over. My sister does some teaching and she gives a problem to students, and she can tell the ones who copied her thing, put it in ChatGPT. And it comes up basically the same thing where even if two humans were reading the same material, they might write it differently.
- [19:57]
Greg Kilstrom
More.
- [19:57]
Jason Bowman
We would kind of diverge more from that. And as you said, AI is not made to do that.
- [20:03]
Greg Kilstrom
Yeah, yeah. Last thing I want to talk about here is, you know, just because you can doesn't mean you should. Right? So the same goes for AI Much, much as we've talked about a little bit here already. You know, when's the right time to use AI and when is it not the right time? And, you know, what should UX teams use to guide them here?
- [20:24]
Jason Bowman
I think AI is great. The beginning of. I mean, I think AI is good to use at any time in the process. I don't know that there's a right time or a wrong time. There's times to refine, there's times to look. But one of the things that, you know, we kind of talk about it, it lacks a bit of editing as well. So I can say, hey, I build me a homepage or build me, you know, a business analyst analytics tool, right? And it's like, I want these inputs, I want this, I want this output, I want this, I want this. And it can, you know, code me up a working prototype. But the problem is, is maybe I have too many numbers that don't intuitively make sense. Maybe it looks too much like a spreadsheet. Maybe it looks too much like this. So just because AI created an artifact doesn't mean that it's the right artifact to use. It doesn't mean that it can't be enhanced. It doesn't mean that it doesn't need user experience. Again, you know, there's a lot of brilliant people out there, but they may not be as skilled in one craft or the other. So they need that help editing, Right? You know, like, I can watch YouTube videos on how to build a house, but from actually doing it, someone who knows how to build a house has more tricks and tips and can guide me. It's like, I know it says that, but you don't need that. This is what you need for this state, for this instance or whatever, right? So the editing, the refinement, the making sure it makes sense, making sure it's not too verbose. Rewriting, checking headlines. I just think there's, there comes a time where it's easier to almost take it away from AI and just work it through than to continue to over iterate with AI because sometimes it doesn't have a history. You're like, well, change that. And then it changes five things. Like, no, no, no, go back to the previous version. Change that one thing down to the left, to the right. And sometimes it's just easier for us to do it ourselves.
- [22:16]
Greg Kilstrom
Yeah, yeah, totally agree. Well, Jason, thanks so much for joining today. One last question for you before we wrap up. What do you do to stay agile in your role and how do you find a way to do it consistently?
- [22:28]
Jason Bowman
I would say one of the ways we do it is I actually just follow my team a little bit. You give the team a little bit of a leash and they start moving in something and always kind of being open to new ideas that different people bring. I think that's when you start building ideas on top of each other. New ways of working, new thoughts come. Something that you never would have considered suddenly comes into practice because someone else is like, well, we just tried this. It's like, you try that and it worked for you. They're like, yeah, it's like, then maybe we should try that again. And just being open to where ideas come from, from all sorts of different teams and disciplines or thoughts.
- [23:04]
Greg Kilstrom
Love it. Love it. Well, again, I'd like to thank Jason Bowman, Executive Director of Experience Strategy at the Office of Experience, for joining the show. You can learn more about Jason and the Office of Experience by following the links in the show notes. And thanks again to the Office of Experience for sponsoring this episode.
- [23:25]
Podcast Host
Thanks again for listening to the B2B Agility podcast. If you enjoyed the show, please take a minute to subscribe and leave us a rating so that others can find the show more easily. You can access more episodes of the show at www.b2b agility.com. that's b2b agility.com. while you're there, check out my series of best selling agile brand guides covering a wide variety of marketing technology topics. Or you can search for Greg Kilstrom on Amazon. Until next time, stay focused and stay agile.
- [24:00]
Odoo Voice
The Agile Brand.
- [24:05]
Zoe Saldana
Hi Zoe Saldana. Welcome to T Mobile. Here's your new iPhone 16 Pro on us.
- [24:09]
Customer
Thanks. And here's my old phone to trade in.
- [24:11]
Zoe Saldana
You don't need a trade in when you switch to T Mobile. We'll give you a new iPhone 16 Pro. Plus we'll help you pay off your old Phone up to 800 bucks and you still get to keep it.
- [24:20]
Customer
There's always a trade in.
- [24:21]
Zoe Saldana
Not right now. @ T Mobile.
- [24:23]
Customer
I feel like I have to give you something in return for karma.
- [24:25]
Zoe Saldana
That's okay.
- [24:26]
Customer
I don't really have much in my purse. Oh, let's see. Hand sanitizer. It's lavender.
- [24:31]
Zoe Saldana
I'm good. Seriously.
- [24:32]
Customer
Let me check this pocket. Oh, mints.
- [24:35]
Zoe Saldana
Really, I'm fine.
- [24:36]
Customer
Oh, I have raisins. I'm a mom. Wait, wait one sec. I've got cupcakes in the car.
- [24:40]
T Mobile Representative
It's our best iPhone offer ever. Switch to T Mobile, get a new iPhone 16 Pro with Apple Intelligence on us. No trade in needed. We'll even pay off your Phone up.
- [24:49]
T Mobile Ad Voice
To 800 bucks with 24 monthly bill credits. New line $100 plus a month on experience beyond finance agreement. $999.99 and qualify. Imported for well qualified plus tax and $10 connection charge. Payout via virtual prepaid card. Allow 15 days credits and imbalance due if you pay off early or cancel.
- [25:04]
Libsyn Ad Voice
CT mobile.com Marketing is hard, but I'll tell you a little secret. It doesn't have to be. Let me point something out. You're listening to a podcast right now and it's great. You love the host. You seek it out and download it. You listen to it while driving, working out, cooking, even going to the bathroom. Podcasts are a pretty close companion. And this is a podcast ad. Did I get your attention? You can reach great listeners like yourself with podcast advertising from Libsyn Ads. Choose from hundreds of top podcasts offering host endorsements or run a pre produced ad like this one across thousands of shows. To reach your target audience in their favorite podcasts with Libsyn ads, go to Libsynads.com that's L I B S Y N ads.com today.