← Insights Unlocked
Insights Unlocked
Designing with AI: speed is up—are solutions?
00:37:44
Designing with AI: Louis Rosenfeld and Llewyn Paine explore how AI is reshaping UX design, workflows, and the future role of designers.
Loading summary
Transcript58 lines
- [00:02]
A
Welcome back to Insights Unlocked. In this episode, we're joined by Lou Rosenfield, Lee Nguyen Payne and Leah Hogan to explore how AI is reshaping the design process and what it means for designers navigating speed, craft and responsibility. We also dive into the upcoming Designing with AI 2026 conference. And listeners, listen up. You can use code INSIGHTS75 for $75 off your registration. Okay, now, enjoy the show.
- [00:32]
B
Welcome to Insights Unlocked, an original podcast from User Testing where we bring you candid conversations and stories with the thinkers, doers and builders behind some of the most successful digital products and experiences in the world, from concept to execution.
- [00:52]
A
Welcome to the Insights Unlocked podcast. I'm Nathan Isaacs, principal Content Marketing Manager at User Testing and and joining us today as host is Leah Hogan, principal Experience Research Consultant here at User Testing. Hello again, Leah.
- [01:06]
C
Hello.
- [01:07]
A
And our guests today are Louis Rosenfeld and Lee Win Payne. Lewis is the founder of Rosenfeld Media and a pioneer in information architecture known for shaping how we think about user experience for decades. He's joined by Liwyn, who is principal of Innovation Strategy at Lewin Payne Consulting. She helps organizations navigate emerging technologies and build better products through smarter design practices. We'll be talking about the upcoming Designing with AI Virtual conference being hosted by Rosenfeld Media and curated by Liwyn. Welcome to the show, Lewis and Liwyn.
- [01:44]
D
Great to be here.
- [01:45]
E
Thanks for having us.
- [01:46]
C
Awesome. I'm really excited for this conversation today because we're at, I think, a really interesting time in the evolution of technology and this is going to be, I think, a great conversation among peers. But to kick things off today, you've both been leaders in design and tech for quite some time and to just place yourself in time and space and technology. Can you share a little bit about your history in the space and what you find most exciting and or challenging where it is that we are right now?
- [02:28]
D
Well, I'll just jump in and say, first of all, thank you for having us on the podcast. It's great to see Aaliyah when you are at the University of Michigan and when I was there, which was into the mid-90s when I was a doctoral student there, we talk a lot about AI and the general consensus among those of us who thought we knew something was that it was never really going to come close to being something that was really useful for anything semantic. Now we've got genai, we've got LLMs, and it's really pretty impressive just historically. I'm pleasantly surprised. And all my faculty from the old days, if they're still around they're probably really shocked. It's much better than any of us thought it would be.
- [03:28]
E
Yeah. So I certainly agree with the surprise that AI has been. My whole career has been about bringing an evidence based lens to emerging technologies of all types. I started out with Disney and Microsoft looking at things like 3D television, if you remember that, biometrics, augmented reality, a whole lot of spatial computing. And with those technologies, it was always about cutting through the hype cycle, figuring out where the real value is going to be. But I think there's something different when it comes to AI because not only is it incredibly useful, but it also has this real problem of looking good at face value, but you have to really get your hands a lot dirtier to figure out where that's illusory and where it's really providing value. And I think that is the key value for the key challenge for designing with it, both in the past and it continues to be.
- [04:31]
D
Yeah, if I can add to that. I mean, like we're, we're in this sort of shock and awe mode right now. You know, it's like this is just technology. And technology's been shocking and aweing us, you know, since the start of the Industrial Revolution, but it just keeps getting more powerful and it just kind of comes at us faster and faster. And I think that's what's going on right now. And one of the things we're trying to do with this conference, but I think what our field is trying to do when it comes to AI is to slow things down a little bit. We can't slow the pace of the technology, but we can slow down or create space for slower, maybe more thoughtful consideration of what the technology can and should do and how humans should work with it, especially the humans who are designing and researching. And that's really what we're trying to get into with this conference. It's not just how can we use it ethically, it's just also like, how should we collaborate in new ways? Because it is new. How should we actually, how should the design process work? We now have AI robots that are part of our teams. How do we manage them? What do the humans do now? So we need space to ask these questions, to catch our breath and to slow things down, because the rest of the world is not doing that. And that's kind of our role as UX designers and researchers. We have to slow things down.
- [06:10]
C
Yeah, and I think that's a really interesting point that you're making, Lou, because one of the things that I think is a Challenge is that we are faced with this. Things are going really fast and things are changing. And then there are also fewer designers and researchers and people who can essentially call that audible and say, like, get intentional. And I feel like that's. If I'm reading through what it is that you're saying, it's not just about, like, slowing things down, but actually getting very intentional and starting to plan. And that creating space concept is, I think, a really great way to frame that. And I think, you know, related to that thought, just
- [06:59]
E
how do you create
- [06:59]
C
that balance between getting intentional, taking the time to do things while essentially observing the fact, and honoring the fact that things are going to continue to move as quickly as they are? Actually, I'd love to hear from both of you about that. So Liwa, maybe if you want to take that.
- [07:19]
E
I think that tension has really been at the forefront of the conversation all three years that we've done this conference. And I think the way that I have found most effective to handle this is to ask people to come armed with evidence. Because there are a lot of different goals in the industry right now. We're kind of speaking a different language when it comes to AI. Some people are talking a lot about productivity. Other people are talking about quality and craft. So I think we really need to take the. Take the pause, ground the conversation and ask people how they're. How they're proving their claims. And that's something that we explicitly ask for in the call for proposals, and it's something that we iteratively come back to in the speaker preparation process is it's okay to have whatever position you want on AI, but we are going to ask you why you think that and to come armed with your evidence.
- [08:26]
D
And it's interesting because, you know, we're still at a point where AI is often happening in large organizations simply by a very. I don't know what the word I would use would be. Flawless mandates from management, you know, everyone. You got to use it, everyone. You have to figure it out. And what does that mean? How do I do it? What should I be worried if I can't figure it out? Where do I begin? Those are all concerns that people in the trenches are dealing with and designers are. And researchers are no different. And that's why I think it's not just a matter of coming armed with evidence, but, like, I think a lot of organizations are going to learn from our field how to even think about this stuff from an evidence perspective. You know, I mean, I don't know that a lot of Other practitioners who are being like your corporate law department who's also under similar pressure thanks to AI. I don't know if they have the same kind of tools that people, let's say on the research side, have for validating the value of an investment in AI. So we kind of have that. We may take it for granted, but we have that. And we may be modeling things for other professions right here. And I like to think that this conference designed with AI is maybe the point of that spear.
- [10:00]
C
I think that's a really thoughtful point because I feel this. So you mentioned I was at the University of Michigan yesterday with our showcase and about half the projects there were AI focused and just all these different use cases that I never thought of were things that people were just like, let's see what we can do. And so that evidence and just seeing like information rich environments really need people who understand how it's put together and best managed. And you know, all those important pieces around making it accessible and usable and compelling are really critical. And with that in mind, I think there's a whole lot of power. There's also a ton of risk and opportunity and blind spots. And I'd love to hear from both of you your perspectives around what are some of the big things we should be out looking out for as practitioners in this space right now. Lee, when you want to take that
- [11:09]
E
one, first looking out for. I'm trying to ground that in my perspective. I'm coming from a research lens and that's my take on design. It's always going to be the lens through which I see things. I think the big concern for researchers with AI has been and continues to be accuracy. So I guess the key thing that I think researchers should be focusing on and thinking critically about is the form versus content of information output. So I think we're in this period where it's really easy to produce something that looks like a report. It's really easy to produce something I think from advancing research. Someone said it was insight shaped, but how do we go through and actually validate that that's driving the kinds of decisions we need to do with the confidence that we need for our organizations.
- [12:13]
D
And I'll add that I look at it through a very different lens of actually a publishing lens where we're constantly trying to figure out how AI can make writing a better process and improve writing. Not just the actual words, but essentially the creative process of coming up with ideas that goes on long before the words hit the hit the screen. So, you know, my perspective is, is Very much that you've got this, you know, certainly with Gen AI, like this great averaging machine. And, you know, some writing really benefits from that. I mean, I gotta tell you, the spam I get these days is really great. It's got no typos, no. No grammar errors. Finally they figured it out. So certain kinds of writing can be brought up to average. And that sounds bad, but like, think about the kind of writing that we do on a regular basis. We have to write for certain scenarios, but a lot of them are not books or they're not like, really important. It may have short half life. They may not be especially critical kinds of information. So I start thinking a lot about not just like, how does the process change, but how important is the output? Do I care if something was written by AI if it's only going to be read a few times for a few days? Is that okay? It might not be. It might be. But there's like, interesting decisions about quality that I don't think we really had to think about before.
- [14:04]
E
If I can go back to what you said, One of the things that you and I talked about when I was putting together my AI for UX Researchers workshop is the lens of jobs to be done. And that's something that I'm constantly coming back to. It's what I teach in the workshop. Design has a stakeholder and a job they need get to get done. Research has a stakeholder and a job they need to get done. And there are certain criteria that determine whether a solution is good enough. And I think we really need to be constantly thinking at that level these days of what is the actual need here? What are the actual criteria for success? And if AI is good enough on those criteria, it should be fine to use.
- [14:48]
D
Exactly. I don't want to bring authors down to average, but I certainly am happy to bring some of my own writing up to average. When I don't have time to spend an hour and a half writing a blurb for a free session that we're producing at Rosenfeld Media. It's going to be read a handful of times. So I'm so happy to reduce that time from an hour to 10 minutes.
- [15:15]
C
Great point. I think the other side of that is also a when do the chatbots just start talking to each other? So there's the reading that's, you know, really meant for human audience and immersion and understanding. And then there's the, I just need to communicate with folks. And, you know, I see like filtering happening with bots. Right. And so you're Writing for maybe a non human audience as well?
- [15:43]
D
Oh, yeah. I mean, you know, we're dealing with that right now. We're, we're like trying to. While we improve our own site's SEO, we're realizing that, well, maybe that's not even the right goal anymore. You know, instead of keywords, we need to be thinking about framing our content, at least our bubble content, in terms of questions and answers that seem to be more LLM friendly. So there's lots of interesting shifts going on right now.
- [16:13]
E
Agents are our new users.
- [16:15]
D
Yeah. Now we have to contend with running teams that have robots and with actually serving customers that are robots.
- [16:25]
C
Gosh. Actually, that brings me to a really great question, which is, so how did designers change how they do their work in this space where not everybody who's consuming the content is a human giving their full attention 100% of the time?
- [16:44]
E
I can speak to how we've been approaching that on the research side. And this again goes back to the AI for UX researchers workshop. One of the things that we teach because of that shift is that agents may very well be the new users of our interfaces. You know, for the past 30 years, making a human user's task easier has meant refining the human to computer interface. But now if you have users who are sending their agents to complete tasks for them, reducing friction for your users means streamlining that human to agent to computer interface. One of the things that we teach is a technique for agent experience testing. It's a lot like a usability test, but it also has some important differences. That's definitely a shift in how we, we think about design with AI, in addition to all of the questions that get raised about using agents as tools or staff members.
- [17:50]
D
And I think that, you know, at like a high level, we're kind of constrained by almost like industrial era thinking of how we collaborate, where it's very hierarchical and a lot of designers have been framed or see themselves as practitioners or maybe, you know, managers. Very hierarchical in that regard. Where I think like so many things, AI is accelerating a shift to a more networked or collaborative model where the human isn't necessarily the, the, the doer in a, in a kind of formal hierarchy, but is really more of an orchestrator of systems and people and robots. But that model of orchestration seems to be really resonating with a lot of the people that I think we come in contact with for the conference programming. But also just in general, that seems to be where a lot of UX design is heading. Less about being a practitioner and more about being an orchestrator.
- [18:59]
C
Oh.
- [19:00]
D
And it's hard to get that established in a, in a organization that doesn't really think that way. That is still very hierarchical.
- [19:08]
C
Yeah, yeah. Because humans are very hierarchical in their thinking. Just like naturally.
- [19:13]
E
That's.
- [19:14]
C
Yeah. How we work. Yeah.
- [19:16]
D
Can be. Well, we've been trained that way. I'm not sure we're naturally that way, but we are. You know, we certainly been taught for years and years that you go work in a workplace and it is a hierarchy and that's. You just got to get used to it. You gotta, you know, be prepared for that. But they may be changing.
- [19:35]
C
Interesting point. I. To that point, I'm curious, you know, across the conference programming this year, what are those things? What are the topic areas and points that are top of mind for the folks who are going to be meeting and discussing them?
- [19:59]
E
Well, Lou, I know that when you and I started talking about this, I think the key question on our minds is what happens in a world where our boundaries between our roles are suddenly blurring in this huge way. We talked a lot about designers, PMs and engineering suddenly all being able to use these same tools that produce the same kinds of output. I think that question is very prevalent across all of the different talks in different ways. The other thing that's really different this year is I think this is the first year we've been able to talk about AI augmented design in a truly end to end way. I've been involved with this conference for three years now. In the past two years, it was a lot of discussion about tools and skill kind of in a standalone way. But now it's starting to be a discussion about knowing where you fit, how to justify your value. And so while we still have those great talks on skills and tools, there's an entire day that's all about managing and supporting the end to end AI augmented design process. And I'm really excited about the talks we have for that day.
- [21:16]
C
That's awesome. Anything to add, Lou?
- [21:19]
D
No, other than that Lee Win is such an expert curator that I'm just really excited about this program. And just to make a couple points about it, you know, Lee Wen mentioned earlier that we go through this pretty extensive call for proposals and we. I don't know, Lee. When I think we got like 80 or 85 for about eight slots, it was a lot. And so that's typical for us with our conferences. And you know, we probably could have, we could probably have twice as many case studies as we. We ended up taking, but we have eight case studies. And the presenters, if you didn't know this about our conferences, they have. They work for about three months in cohorts with each other with Leeuwen's help, and then finally with the help of a professional speaker coach to develop their case studies not only as individual presentations, but as a collection where they reference each other. And there's a lot of coherence both for the individual days, each cohort has its own day, and then for the whole conference over two days. So they have been working already for a good month, month and a half. And we still have about a month and a half to go before the conference comes June 9th and 10th. We also have some interesting panels that we're finalizing right now. Leemon can go into a little depth on them. I'm not sure we can name names yet, but we're getting close and we have a couple of really fabulous featured speakers. Anil Dash, who is just like a giant in understanding how people and policy come together with technology. He's had a senior role, the, I think it was in the Clinton administration that was technology related. I mean, he's just been doing great stuff for many years. Then we have Paul Ford, used to be the webmaster, the original webmaster. Harper's, edited the Harper's Index, if you ever read Harper's Magazine. Just a really brilliant guy who is going to lighten the, the messaging a bit by covering a whole bunch of neologisms that he's inventing around this intersection of, of design and AI. And he's just a blast. So it's going to be a great m of fun and creativity and inspiration and really practical nuts and bolts. Case studies.
- [23:51]
C
Yeah, I think that's really, that's really helpful. And I think one of the things that I'm really curious about is, you know, girl told like should be skating towards the puck, like. And a lot of what it is that you are going to be speaking to is I think things that we're learning now. But as we're looking forward, what are the, the most important things that we should be paying attention to? Like, you know, you just mentioned neologisms, right? So there's a whole new language. And I even I, when I'm creating stuff these days, I'm like, I should create a glossary because I think, you know, I've heard three different ways that this one term that I am hearing people use is being interpreted by folks. So how can people, I think, stay current with what's happening but also be looking forward to what Are those trends that are going to most impact their everyday work?
- [25:00]
D
That's a tough one. Right. And you know, like we, we as we program this event the last few years, it's always a bit tough when you have people pitching talks through a call for proposals months in advance and then working on their presentations for, for three months before the conference when everything in AI seems to be changing every five minutes. And you know, is it like, is it a feature or a bug? You know. So one thing I think that's really important for not only our presenters, but for anyone working in this space is to focus on the journey. I know that sounds very kind of, you know, trite, but it is true. Like it's not so much here's what we did, but here's how we learned that's so important in this space. And again, like what we see with our program is like in a sense a microcosm and maybe even a model for what the, the field should be doing and how it should see, how to learn about this, how to work with this technology and, and not to feel so much pressure to have it figured out because I don't think that's a realistic outcome.
- [26:12]
E
Yeah, I think the point about the journey is such a good one. One of the things that, you know, definitely not trying to pick on other conferences, but one of the things you see at a lot of conferences is they just ask somebody to come in and talk from a big name organization. What I think is great about this conference is we are looking for storytelling and case studies and we really emphasize with each of the speakers, don't just talk about your hot takes on AI, really talk about what was the situation, what are the headwinds you faced? Because that's the stuff that doesn't get talked about really at all in our industries. Steve Yegg talked recently about the 18 month hiring freeze in our industry and how it makes it so much harder to tell how well you're doing with AI compared to other organizations because you don't have that new blood coming in. And this is an opportunity to see what other people are doing in real actual industry, in actual companies like yours. And I think that's so valuable.
- [27:25]
D
And Leah, you know, you were mentioning like this, this feeling of needing to come up with a taxonomy. And it's interesting. I was just talking with an old friend who's actually lives in Ann Arbor, one of your neighbors who is a content designer and she was feeling like at her job in large technical companies, she needed to come up with a glossary for all this, they, they've kind of charged her with that and you know, like the impulse to do that is good because we all need to do whatever sense making exercises we can. But as I talked to her with her, I was reminded that so often we're working on, on solutions without really good problem definition. And I, I think again this is something we have to do, especially in these settings where the boss says learn AI. Why? What for? To what end? Ask those questions, do the diagnostics before you move to the solutioneering. Especially right now we should all be playing with this stuff, but playing with it and viacoding is not the same as solving. And so our muscles for asking questions and doing true diagnostics are so important and we can't let them atrophy. And I like to think by giving speakers the time and space to be thoughtful, catch their breath, take a deeper look, we're helping them show our attendees how to develop those muscles and really feature them in a way that shows just how important this field is and even how more important it's going to be in the years to come.
- [29:16]
C
I hard agree with you. I think part of why, I think really the time for UX design and research has really come around again is just the fact that we've built out all this infrastructure without use cases in mind. We've got a ton of compute and we can solve a ton of problems. UX researchers and designers are I think, the people who are best positioned to be able to say now let's put all that compute to work. That 18 month hiring freeze has been both a time for of opportunity that would enable people, if they had the resources, to get out there and do some experimentation. But if they haven't, it's that catching up that folks need to do. And I think part of the value that I see potentially in this conference is giving people that moment to do some of the catch up, some of the learning, understand what's out there and also get some hope around where the opportunity really exists for them to demonstrate value.
- [30:24]
E
It's funny you say hope. That was one of the biggest responses we had last year was that UX folks can be feeling so lost and just kind of adrift in this current moment. And it's so great to have the grounding of other practitioners to see the problems that they're really solving and to help help us find our place again. And I think that's really one of the major goals for this year as well.
- [30:53]
C
Yeah, and I guess to that point, you know, I think we've talked about the fact that you're a design or a researcher, you come from the research perspective. But I'm curious about the people that you're expecting to attend and where are they in their careers, what backgrounds, you know, because I feel like advancing research. That's very clear. AI could be anybody. So who is the attendee who would benefit from having this conversation or being a part of this conversation?
- [31:24]
D
Well, I mean, first of all, we see as we see from like traditionally with our conferences, we're getting a lot of people at large organizations, you know, Adobe Infidelity and kind of at that level send a lot of people in force and you know, they are kind of at the sort of front lines of this whole conversation around what do we do now that they've told us we have to use AI? Those people tend to be typically three to five years into their careers. They're not looking for 101 content. They've probably experimented a bit. They might not have gotten as far as vibe coding, but some of them certainly have. And like, with any new technology you do have, like was it Bruce Sterling said the future's here. It's just not distributed evenly. There's just like a wide mix of expertise and experience right now in any kind of gathering that involves AI in our field. You know, there's. But I would say it's really across all the roles that have something to do with products. So we always see people who may have traditionally been researchers, maybe they don't have that title anymore, but that's how they've seen themselves. But also designers, especially the designers that are moving away from the UI into more conceptual or systems oriented areas like information architects, like service designers. I think we're seeing certainly a lot of people on the content design side, which is also not surprising. I don't know that we're seeing quite yet people coming from dev backgrounds, but I kind of assume that's going to happen because the lines are blurring anyway and everyone is trying to make sense of the same thing. We certainly see people who would associate themselves as product design folks.
- [33:32]
C
Great. All right, so I think a couple things first before we close out the conversation. Are there any things that you didn't cover today that you'd like to speak to? I know that's a big open question.
- [33:46]
E
I think we covered a lot of points. I think one thing that we didn't cover during the discussion of agents is there's the agents as new users model and that's certainly important. But I think there's also some really interesting Things going on with agents as tools or staff, which is not necessarily the term I like to use. But we're seeing some really interesting metaphors emerging around AI as your chief of staff, as your production crew. And one of our panels is going to be specifically about exploring those types of metaphors and the new models for how design can engage with those. So that's something that's very top of mind for a lot of folks with the big rise of claw agents. And I think it's going to be a great conversation.
- [34:36]
C
Lou, anything from you?
- [34:37]
D
I would just say, you know, just to put a very. Put my marketing hat on for a moment. It's a virtual conference. We do a lot of them. It's not new for us. It's hard sometimes to attend a virtual conference, but you. We do try to make it a lot more engaging than you might realize. And one of the things that we do at all of our virtual conferences is we give people an opportunity to participate in attendee cohorts, which are small groups of facilitated conversations, like 10 people randomly assigned. If you opt in, you will be one of around 10 people who, with the help of two volunteer facilitators, get to meet in advance and talk to each other, meet each other, figure out what you want to learn from the conference and hang out together. So it's like a watch party, and we have a number of those. We don't charge for that, but it does make things much more engaging. And then of course, you know, you will get access to the recordings and that if you can't make it, you know, all day for two straight days, watch the ones you can live, ask questions, and then go back and watch the things that you might have missed or rewatch something that really hit for you.
- [35:58]
C
Great recommendations. And the cohort's idea sounds really fascinating. I should try that. That's awesome. So thank you both for attending. Today's actually being a part of today's discussion. And I hope that this is an awesome event. I know I'm very intrigued and very curious about what other folks are doing. So looking forward to attending myself. So thanks for the conversation and I look forward to the virtual conference. What's the date again?
- [36:35]
D
June 9th and 10th. And then we've got some great workshops, including the one with Lee Wen on how researchers can benefit from AI.
- [36:44]
C
How can someone register?
- [36:46]
D
Come to rosenfeldmedia.com, go to the list of conferences. You'll see it listed there. Or you could probably just search Design with AI Conference 2026, and it'll come right up if we did our SEO right and looking forward to seeing you there.
- [37:05]
C
Sounds awesome. Looking forward to it myself. Thank you again for the conversation today and look forward to seeing you there.
- [37:13]
D
Thanks for the opportunity.
- [37:14]
E
Thanks so much.
- [37:16]
B
Want to keep the conversation going? You can find the show notes@usertesting.com podcast if you haven't already. Don't forget to follow us on Apple Podcasts, Spotify, Overcast, or Google Play so you never miss an episode. And if you enjoyed today's show, please share it with a friend or leave us a rating and review on Apple Podcasts. And until next time, this is Insights Unlocked, an original podcast from User Testing.