
Discover how UX must evolve in the AI era with Greg Nudelman. Learn to design intelligent, content-first experiences that truly serve users.
Loading summary
Nathan Isaacs
Welcome back to the Insights Unlocked podcast. In this episode, we're diving into the future of UX in an AI driven world with Greg Noodleman, a UX architect at Sumo Logic and author of UX for AI. Greg shares why traditional design tools might be holding us back and what UX teams should be doing instead to stay ahead. It's a conversation packed with insight and inspiration. Enjoy the show.
Sean Treizer
Welcome to Insights Unlocked, an original podcast from User Testing where we bring you candid conversations and stories with the thinkers, doers and builders behind some of the most successful digital products and experiences in the world, from concept to execution.
Nathan Isaacs
Welcome to the Insights Unlocked podcast. I'm Nathan Isaacs, Principal Content marketing manager at UserTesting, and joining us today as hosts is Sean Treizer, staff product strategist at UserTesting. Welcome, Shaun.
Greg Noodleman
Hey everyone.
Nathan Isaacs
And our guest today is Greg Noodleman. Greg is currently a distinguished designer UX architect at Sumo Logic, creating innovative AI and ML solutions for security, network and cloud monitoring. He's a UX strategist, AI product designer and the founder of UX for AI. With 20 plus years of experience, he he has helped brands like Cisco, IBM and Intuit. He's authored six UX books and 24 patents. His most recent book, UX for AI, was released in May. Welcome to the show, Greg.
Greg Noodleman
Thank you. Great to be here, Greg.
You've had a fascinating journey working at the intersection of UX and AI. Let's get started by taking us back to what first drew you into this space and how your career evolved to where it is today.
Yeah, it's a, it's a great question. I was a total nerd when I was young, right? So I read a lot of science fiction. I was the only child. So you know, that was like books were my friends. You know, the TV sucked in Ukraine back in the day when I was growing up. So I just spent a lot of time, you know, thinking about science fiction and robots and so forth and really fantasizing how would it be to live with real life robotic entities. So that's what we have today. So of course I jumped the chance to work on this every chance I got, both as a full stack developer and as a UXer for the last 20 years. So got a chance to work on 35 AI driven projects and learn what to do and most importantly, what not to do. Right. So lots and lots of ways that this stuff can go sideways. So that really has been my journey up to this point. And very excited about our new book. I know we're gonna talk about it, but here it is. So very, very excited. So that's, that's available now. Yeah, so that's me. Thank you.
Congrats on the release. Coming from such an early point of inspiration digging into science fiction, to your technical training, to then expanding that out, to making sure that those experiences land with the folks that they're designed for is quite a journey you've been on. So I'm really excited to dig into some of your practical expertise that you've shared on your blogs and in other spaces, starting with an article that you wrote titled Iceberg ux. You wrote that UX is not UI and compared to Figma, to the Titanic, heading straight for an AI iceberg. Love that metaphor. What did you mean by that? And how should UX teams reframe their role in this new era of AI driven experiences?
So we've been talking about kind of the challenges with Figma workflows for quite some time and partly because as a product, Figma has so far been really doing basically focusing on pictures and pictures are going away. Honestly they're fine. But you know, more and more what we need to be doing is going directly to code. You know, basically you draw something on a piece of paper, you feed it into a machine and you maybe give it some prompts and, and lo and behold, you actually get working code. We need to kind of get away from pictures, but that is not really the focus of the article. The focus of the article was bigger. So beyond that, what we've noticed a lot in our testing is is this people just aren't that sensitive to what it looks like. What they want to know is what does it cook like. So it's a little bit like UI being the plate and UX being the steak on the plate. Like, yeah, the plate's important, but the plate needs to get the hell out of the way. What we really want to know is what does the steak taste look like and look like, right? And that is largely beyond what figma can deliver. It's outside of figma's purview. The typical designer who has a Figma based workflow will just put Lorem Ipsum as the output or as the button name and then they'll just leave it alone. But the meat and potatoes, if you will, the actual dish, is all in the content. It means how is it summarized, how reliable it is, what are the suggestions that come after? What's the autocomplete look like? You know, all of those things are basically content and the content is outside of Purview of figma. And that is why, you know, any kind of graphical tool, I don't mean to pick on figma, it's just Figma happens to be the one that most people use. But you know, any kind of graphical tool is becoming less and less relevant because what we keep hearing from customers is it doesn't matter what the chat bot chat bubble is styled as. For example, I don't care, you know, maybe don't make it Comic Sans and don't make it, you know, purple unicorns, but other than that I don't really care. Is, you know, is the, is the text box on top or the bottom? Don't really care. Is the button say send or ask or maybe just has a little paper airplane on it? Don't care. Right. Like what I care about is the output. Is the output correct? Is the output summarized in the way that fits my workflow and where I am in, in the journey through the product that is critical and that again is outside of figma. That is what that does, why we need to change how we approach it. And that is what really it's in the book. Exactly that. Right. And one of the things we talk about is this new workflow and that workflow is all about incorporating AI into it. Now it's still very much user centered, that this is the important thing. But what it is, it also incorporates AI and data, which is typical, which is a typical piece. So you basically have lightweight prototype page, you have AI and you have data because those things are all interlinked and they need to be included in every test. And we have a very specific rag based workflow where you essentially set up your model in a way that would allow you to test basic approaches. And these things are very, very malleable. AI models are very accessible, malleable. You don't need any special prompts, you just tell it what you want and if it doesn't work, you tweak it. Right. It's, it's a kind of a collaborative process. The model is there to, to help you and to serve you and, and to there to become your perfect prototype. So yes, you want to have a prototype that is maybe a, you know, presentation of what it's actually going to look like. And then you also have the model as part of the test that you're doing with users. Now ask the question that you would normally ask and let's see the output. Is this what you want? Do you want it to be longer? Do you want to be shorter? Is this correct output? And by the way, where's the data coming from? Can we borrow your data to train it better for your use or should we be doing something different? Like those are the kinds of questions that UX teams need to increasingly be involved in. Not what does it look like, where do the buttons go? Like that's kind of not all that relevant.
Yeah, you bring up so many important points here. The value really lies in the content itself. And the challenge that we have to keep up with here is the dynamic interactions increasingly so that we're having to provide. It's not just one single path that we want our users to flow through. It is a dynamic exchange of what is their need in the form of a prompt and what is the output of that model. So it's quite a lot to keep up with. How are you encouraging UX teams to think more broadly about those exchanges, those interactions?
Well, I think it's all about our trademark empathy skills that, that we bring as UXers that, you know, a lot of us have, you know, a lot of folks have forgotten, I think, and, and kind of like really held themselves back, I think, in, in favor of doing more Figma based, playing around with the tool and learning all the layouts and all of that kind of stuff that's not going to really save your job going forward. That. This is exactly the kind of stuff that I call it robot monkey work. This is the kind of stuff that we can just turn over to AI and say, just do the layout for this. I don't care. And by the way, do it in react code. Don't make me a picture. I don't care. Right? Like don't, don't make pictures, just make running code that can actually interact with and that takes it to a whole other level. Right? So when you actually have working code, a lot of that energy that you've been spending animating Figma to show how it runs and whether this thing, you know, what's the transition and all of that kind of stuff like that is like massive mental energy that is no longer needed. Where should you be putting that mental energy and that effort into? Well, you should be putting it into a. Talking with your customers right away. If you have not talked to your customers in it's been a day for you, that's too long already, right? If it's been 24 hours since you talked to a customer, you better just rethink your workflow. If it's been a week, really, really time to rethink, right? Time to rethink your life's purpose. Maybe Right. Like, you need to be talking to customers on a daily basis and no joke, right? So the, the other part is your team, right? Like you need to be talking to data scientists, you need to be talking to AI engineers. You need to understand where they're coming from, where's coming from, what's the bias of the data. Let's chat about it, right? Let's really understand the outcome of the prediction, the value of the prediction. One of the things we talk about is value value matrix in the book. And there's a. It's a value matrix exercise where we go through and we say, well, if the, if the. If the. Let's say you ask a question and AI is making prediction, if the prediction is correct, what's the value? And if prediction is wrong, what is the impact of that? Right. And if you really consider that, that's a very deep, very valuable conversation that involves empathy, involves symphony, involves creativity, involves all those human things that AI just cannot replace. And that is where we need to be spending the time and effort and our energy. And that is why I put that down in the book, because I kind of got tired of talking about it. You know, I just wanted to write this down and say, here, you know, here's your kind of the blueprint for the things you should be doing. So anyway, that's what I would recommend is, is really having these deeper conversations about the content. Don't. If you're putting Lorem Ipsum anywhere, you should stop right now and like, just don't do that anymore. There's no. Should not be any more Lorem Ipsum anywhere. Let's just, let's call it the death of Lorem Ipsum. All right, sure.
Well, you have so much more assistance in creating content these days, and it's the value behind the content that we need to understand. These AI interfaces are really moving so much of the interaction behind the scenes. These can even be invisible interactions to the users. I'm really curious how then we need to shift our framing of how we measure that value. It sounds like you have tools like this value matrix and potentially others. So how can teams ensure that they're still designing intuitive and helpful valuable experiences when so much of the interaction is invisible and behind the scenes?
Yeah, I think partly what you may be talking about is agents, UI agents that may be executing majority of the workflow behind the scenes. I think with. With that in mind, it is critical that we again, put customer first. We put customer first and then our team also. Right. Like really understanding why the, you know, what Are the, the data scientists are putting in there because honestly, AI is too important to just be left to data scientists, which is why everyone needs to get involved. And UX in particular needs to bring that customer lens to, to the, to the conversation that real, that real value. Let me give you, let me give you one example. So. Well, I can't tell you too much about it, but so there's one project that, that I've been on where the AI would guess something automatically. And so one of the, one of the, versus having a customer select it, right? Let's imagine Netflix for example. So it's probably the closest analog would be Netflix where you know, in the Netflix when you open it up, there's a screen where you get to select a person that is interacting with it. And so that takes literally like a second, right? So we, we just decided to be generous. We said takes three seconds to select a Persona, right? Because you really want to look at all the little avatars and you know, maybe do a quick search, whatever. So, so once you select a Persona, the entire thing is, is customized to you. Now the proposal for that particular design was to remove that screen and then based on your search then to give you the Persona that's most likely this. So if you have kids, for example, if you were searching for blue clues, then you probably were a kid. And if you were searching for some, you know, orange is the new black, maybe you were the, the mother, you get what I'm saying? Like that kind of thing. So that was a multi year project that would eliminate this selection screen. And so one of the things that I sat down and they asked, you know, okay, I said, look, you know, what is the ROI of this? What is the. I know we're, we really are interested in this. Like this is a really juicy AI project, right? Like there's, there's just looks like a really fun problem to, to work on. But, but what's the real roi? Let's say they spend three seconds on this and let's say we, we guessed it wrong and, and the kid gets the orange and is the new black. Like you know, you have like the positive, you save three seconds. But the negative is you gave them completely inappropriate content, right? Some mature content that, that is not kid appropriate and vice vers, you know, the father likes science fiction and the mother likes the dramas and you're giving them sort of a mixture because you don't know who this person is. So again we sat down, we looked at the value matrix really, really quick. It was not a very Long exercise. And what we figured out is we need to get to a 99.7% accuracy with selection in order for this to really work. And that is better than Kaggle. Kaggle is the international data science competition. If they get to 98%, like that's, that's a blowout, right? Like that's, everybody celebrates. There's like confetti and balloons fall from the sky. 99.7 is basically unachievable with LLMs. Like at this point of technology, like, let me just be clear, it's, it's impossible because that, that means the model is so cold as to not be creative at all, right? So it just, it needs to be so locked down. So it was impossible. And a year's worth of work versus something that's just basically a select screen. So that is the conversation you need to be having as a team. Does that make sense? So, yes, a lot of this behind the scenes, but what we need to do is we need to understand it, we need to expose it, and we need to have a conversation about it. And if we're not sure, we need to, then ask the right question. And that is again why I feel like this book is really going to help you. Because in it I tell you how to, I teach you how to ask the right question, how to ask the right question of data science, how to have that really important conversation to expose what's behind the scene in the machinery. So don't just trust them that it's accurate, right? Like, in fact, accuracy is kind of bullshit to begin with. It doesn't, it doesn't matter because it doesn't match the real world thing, for example. So, sorry, do we have time for this? I'm going to wax poetic here. Let's say you have a really accurate model. So let's say you have a model that. This is from my mentor, actually, Arjit Singupta from Abel. So he came up with this example. So I'm gonna steal it from him today. So the example is this. Let's say you have a really accurate AI that's used by NSA to. Sorry, TSA is used by TSA to see who's the terrorist. So let's say you have a bunch of people walking the detector and then you know, AI is really smart, is looking at it and going, you know, this person's a potential terrorist. You should do a pad down. So the model that's going to be really, really accurate is not, is going to always return false. Because majority of people going through the TSA checkpoints are not terrorists. They don't have a knife, they don't have anything weapon. They just want to get through on the, on the, on their plane. So the model that's going to return false, going to be 99.9999 accurate is going to be completely useless because it's going to pull nobody aside because it's the model that is accurate, tries really, really hard not to be wrong. That means it, it's, if, if you think of it as like a baseball analogy, it never hits the ball, right? A really accurate model would literally go, oh, that's probably a strike. Oh yeah, it's probably strike, right? So it's never going to hit, it's never going to hit because, because it's trying not to be wrong, right? So that's a really, really accurate model. So when you come talk to your data scientists and they say, oh, don't worry about it, this model is really accurate, you got to call bullshit. You got to go, wait a minute, maybe it's too accurate, maybe it's leaving money on the table, maybe it needs to be a little more aggressive. That means train on recall, right? And in the book is what I teach you is how to have that conversation, how to understand what these things mean and how to really break that down and how to have that conversation. In other words, take what is hidden and expose it and really understand it. Because AI, at the, @ the end of the day, it's a tool, right? It's a, it's a medium, it's a tool. We need to learn how to use it, we need to learn how to ask the right question about it. Just like it was mobile, just like it was wearables, just like it was desktop before that. It's yet another iteration. Yes, it's moving fast. Yes, it's super scary. But you now have the tools to have that conversation you need, and for this you need the book. So there you go.
Well, I really appreciate these examples because in almost every scenario the common person who might not be working on these models, in my opinion, overestimates their capability and wouldn't distinguish the difference between accuracy versus practicality. Right? So there's years worth of work that have to go into tuning these models to make them practical. So I'd love to hear about any other common mistakes that you've seen that derail AI projects amongst the teams that are so focused on them. What, what are some traps that teams fall into and how can real world feedback be incorporated to catch those issues before it's too Late.
Yeah, I'd say. Yeah. By the way, the statistic that nobody wants to talk about in the industry is that 85% of these fail. This according to Gardner and the article was published in forbes. So literally 85% goes down the drain. Right. Fails to produce any ROI. And the reason is usually not technology, as you just said. I think that's a very, very good point. It is not technology. It is actually UX or UX kind of UX related reason, I would say. And I'd say the number one reason things fail is the use case. And that is kind of our bread and butter, guys. This, this is for ux. This is, this is it. Like if you picked one thing that we do, it's, it's use cases, right? Like we really need to, we need, you know, use cases run like a steel thread, like steel rail for everything we do. So picking the right use case that I can actually address is going to be really critical. And that is why in the book, we spend the whole first part of the book going over framing the problem. And that involves all of these different exercises. And one of them is storyboarding that you can approach that you basically draw up a storyboard even before you do any wireframes there. Again, Figma is doing us a disservice because it's like, oh yeah, go straight into height of wireframes and then they'll start talking about things like button placement and so forth. Well, we need to take a step back because, you know, if, if we just say, okay, is this even worth doing? Do we have the right data? Is the model answering the right question? All of these things are critical. This is where a lot of the stuff goes sideways because people get excited. And this is a very exciting. You know, if you have a hammer, everything looks like a dale, right? This is a really large hammer. AI is a huge hammer. It's like a sledgehammer, right? With, with like one of those rockets. If you, if you like Alita, Alita, the, the Battle Angel. Remember the, the, the doc's. Doc's hammer that he had a little button. He pushes the button and there's like a rocket engine at the, at the, at the end of the hammer. So it hits really, really hard. That's AI, but you need to aim it in the right direction because that hammer is, you know, in the wrong, in the wrong hands, destructive. And that is what happens. It's, it's, you know, in real life scenarios, for example, like Boeing, Boeing 737, Max is a perfect example of that it's, it's a poster child of mcast. Their AI system just guessed, guessed wrong. Right. And it was insisting that the nose was pointing up even though it wasn't. And it kept pushing the airplane down and it cost two crashes and killed 400 people. So, you know, this is again, and that's only one example. There are many, many others in the industry. The more these things are migrating into the physical world, the more critical it is for UX people to get involved, to understand the use case, to understand the model and its full abilities and its biases and the ethics and all of that stuff. Right. But it stems from use case. So use case, I would say is job one. If you take nothing else from this podcast, get my book, if you get two things out of this podcast is study the use case. Really understand where the use case is coming from and is it appropriate? Do you have the right model? Do you have the right data? Is it guessing in the right direction? Right. Like all of those things are really all wrapped up in the use case. So yeah, use case development, starting with the storyboard, then going into digital twin to really understand that and going into the value matrix. So those are the top three things I would say to focus on.
Thank you for that reminder, Greg. Because it's so easy to get swept up in the shiny new toy that we can forget to go back to basics. It always starts with solving a problem. Right. And that's only made more complex by the emergence of agentic AI. And so this is a topic you've written on about agentic ux where humans are collaborating with AI agents in complex multi step workflows. So what is the biggest mindset shift that teams need to make to design for that kind of relationship between humans and machines that are able to pick up on multiple tasks and drive themselves through them?
Yeah, I would say it's, it's something that's a lot of folks aren't talking about it, but it's coming very, very fast. Right now these things are learning. They're almost like interns or, you know, like little kids running around. Now in the article, what I, what I like to think about as a model is ants. It's like an anthill. So an individual ant may not be very smart, but there's a lot of ants. And you, the queen, maybe as a supervisor, that's the model that I like to think about. So the queen is talking maybe to the human and then it passes the queen, she passes the signals to her ant soldiers and workers and then they do the work. And then they come back to her and say, yeah, I did this and I did this and this is what I found. And she summarizes all of this stuff and then talking to, talking to the human. So right now these things are not that smart and they have trouble being in the world, if you will. For example, deploying code writing is working really, really well, actually. It's above and beyond anything that I thought. And it doubles every three months, apparently. So in other words, it's now writing, I think, about a week's worth of code and in a single prompt, which is mind boggling. So I would say a few more iterations and it will just write a year's worth of code and in one prompt and it's, you know, it's going to be game over. But so let me get, let me get to this point that I'm making. So, so the, the bigger point is that these things are not that smart and they have trouble being in the world. They have trouble sort of deploying things and, and you can look at it and go, you know, what uses that, right? Like, okay, it wrote the code, but it can't really deploy it and so on. And I would say what uses a little, is a little baby. What uses a newborn baby. I think that was one of the points about electricity that was made. It was like an electric generator was being demoed. I think it was Franklin who was doing that. And one of the investors said, what uses that, right, lights up a little, you know, moves a magnet or some, moves the arrow on an instrument. And he said, well, it uses a little baby. So these things are babies, but they're very, very quickly going to be the adult in a room. And that is, I would say the mind shift is right now they're learning. And the way they learn, they never forget anything. They don't forget anything that went right, anything that went wrong. Like these things are constantly eating the world. And as soon as there's going to be a shift very, very soon where these things are going to be the adult in the room, they are going to be the expert. They're going to be the expert on your system, they're going to be expert on your customer, they're going to be the expert on the industry. Most importantly, if you ever do any kind of marketing, you know that they're an expert in already the expert in kind of the marketing writing, the marketing content. One of the things I've done recently is shifted my marketing to be more AI assisted, if you will. And what that's done is got me my very first viral post on LinkedIn and that is just mind boggling. So in other words, the model understood the zeitgeist of the industry better than I did. I wrote all the content. In fact, I write every single thing by hand on ux4ai.com on my blog. But what I did is I asked for some help on marketing and that and it knocked it out of the park. Like, like I said, my very first viral post, very, very crazy to me. But what we're seeing inherently in this is maybe part of it is people clicking on it, but part of it is the algorithms promoting the post. Right? So it's in here. It's increasingly machines creating content that is interacting with a machine and that that machine recognizes and then promotes. So my GPT was trying to guess what the LinkedIn GPT would consider to be interesting and then boost that. So increasingly it's machines talking to machines and us kind of being in the middle as the creative force, as maybe the driving force in the symphony. But the machinery is being taken care of behind the scenes by these agents talking to one another. And like I said, you know, imagine for a second you have an intern on your team. If you have somebody like that and you're training this person and within a few months they know more about everything you're training them on than you ever will in a lifetime. Like that is the kind of thing that we need to be thinking of. These things are going to be part of our teams. They're here to stay. They're going to be kind of like our co workers and they may be the ultimate arbiter of what is right or wrong. And they'll be the experts very, very, very quickly. It's gonna be, it's gonna be months. It's not gonna be years, it's gonna be months. And then that is gonna cause a very significant shift in the workforce I think. Because if, if we're not right, we need to really start thinking of ourselves as what is our role, right? What is, what is that that we need to focus on? And that in my opinion again is empathy, is humanity, is, is being actionable in the world. Because AI models aren't necessarily walking around and interacting with, with real life things. And, and maybe we don't want them to because the, the, the agents tend to be very aggressive. Like an aggressive, you know, like an agent doctor may do a biopsy that is not needed just because they're like, yeah, well, I wanted to know if this guy has cancer of the brain or not? Well, you look like you might. And let's just tell him to drill into his head and check. Right. So that's the kind of thing I'm talking about. So agents are kind of inherently more aggressive. So we need to have new controls, we need to have new methods of interacting with these things because they're asynchronous. Everything we built so far as UXers have been a synchronous thing. In other words, you click a button, something happens. These things are. You click a button, something is launched, and then that something is operating in the world behind the scenes. You may not know what it's doing, and then it's occasionally checking in with you. So a completely different opportunity, a completely different way of kind of working with these things that needs, that needs to evolve, and we need to think of them as really the experts in a room. And how do you interact with, say, a contractor? That would be kind of a similar thing. So you essentially give a task to a contractor, they're doing it independently, they're checking in with you, let's say, in a weekly basis. How does that look like when it's a machine? And, you know, you know, what, what, what is that going to look like? So I'm actually really, really excited about it. I don't know if you can tell, maybe it's the coffee, but, you know, I'm super excited about this because to me, this is the first new thing to come in decades. It is so, so cool. And I hope you guys share my excitement because this is a brand new thing. It's super powerful. It's, it's really like. There's a book out that's called Super Agency. This is what I think of it. It's like, it's, it's making us cyborgs. It's, it's, it's a lot like sort of Iron man. Right? Like, it's the dream of Iron Man. But every single person is the Iron man or Iron Woman. Right? It's, it's the Jarvis, that smart suit of armor. And all this capability is at our fingertips. So, you know, I want to urge everybody to really embrace it, to, to study it, to understand it and to use it and, and to really be an active participant in shaping our future. Because if you're not going to be doing that, then it's going to be shaped by, by, you know, maybe people that don't have your best interest in mind. That's all. I'm going to leave it there.
Sure. Yeah. There are new forms of relationship that have taken shape. There were early examples of AI models once they started interacting with each other originally in English, at a certain point they moved beyond it because there are more efficient forms of communication than English, as these models discovered. And so we need to be really clear about how we're distinguishing machine to machine relationships, machine to human relationships, and ensure that we are maintaining human to human relationships. Right. And I know you believe strongly in this. You've built a community, you have a boot camp to help designers lead in the AI age and keep pace. So what are some of the essential skills and mindsets that you think teams need to thrive when the pace of innovation keeps accelerating as it has?
Yeah, I'm really glad you mentioned the bootcamp. So just a quick upsell, there's still a few tickets left. If you go to my site, you can get a 20% off coupon. It's right, it's right there on ux4ai.com it's coming up next week. So if you, if you can't attend it, it's online. We moved it online because so many people were interested and we couldn't accommodate the space. The space wise really highly recommend you do it. It's not just me, but there's three other people from IBM and Amazon, AWS and Klaviyo. So a fantastic lineup of instructors. So if you do, you know, if you take three things out of this podcast is go, go do, go do this thing if you can. I don't know when this is going to come out. It might be already too late for this boot. If it is, just feel free to cut this portion off. But the point is you need to find some training. You need to find your community. You need to find a way to talk to like minded folks and talk about this and strategize because machines are here, the shift is here and you need to upskill yourself through your fellow humans and for the use of AI to help thrive in this new normal. Because your med mastery of your figma auto layout is not going to be something that's going to feed you for the next five years. It's going to be very, very quick of a change and the cheese is already getting moved. So that's kind of one point I would say. The other, yeah, the other, the other mind shift is really start to define your moat. Like what are you good at? For example, for me, I'm a futurist. I like to bring different parts together and really imagine what that next phase of the product might look like and what that next phase of experience might look like for your customers. For you it might be something very different. It could be research, it could be, could be strategic, you know, communications, it could be whatever it is. And in the book one of the things that I do is just really break down like different roles that are today in the marketplace of UX as a big arena. And then I go through each one of the popular kind of job descriptions, if you will, and I say I give you an estimate, or at least my reading of the tea leaves, how much that will either change, go away or evolve and become even more important, even more relevant in the current marketplace. So I think that's something you need to do for yourself and really understand where your value is. Because I can tell you it ain't in Figma mockups. It's, you know, my realization came to me when I made a thousand mockups for a project and yet it fixed not one failing query. Like literally it's six months of work. It is massive amount of lift and to the point where I couldn't see straight anymore. And still it made zero impact. Okay, so this is what we're increasingly looking at is zero impact from traditional way of doing things and we need to really re examine that and, and look at what is going to move the needle. Where is your value to the enterprise? Increasingly the enterprise is going to be hybrid. It's not just people, but it's people agents, LLMs. Right. And your customers are using LLMs and like you mentioned Sean, they're talking to each other maybe not in the, not in the language we completely comprehend. And what does that look like for you? So that you know what your mission is and how your current set of skills is tied to your mission.
Well, thanks so much for sharing this. Thanks so much for being on the show, Greg. This has been incredibly interesting. You've expanded my mind and I'm sure our listeners minds on the possibilities of this leading frontier. Really enjoyed this conversation. So you've mentioned a few different resources. You have your UX for AI blog, of course, your new book out. Could you please just sum up for us where our listeners can learn more about you, your thought leadership and the work you're doing, whether at Sumo Logic or at UX for AI.
Yeah, I would say definitely check out Sumo Logic. We've got a lot of cool stuff happening. We got Copilot and Mobot and so on. So massive, massive AI driven effort for SOC and agentic presentations and so on, but for sort of the career development and so forth, just, just go to ux4ai.com and that's ux4ai.com for not, not the number and dot com and then you'll see all my resources there, including this awesome book that I highly recommend that's got kind of the best, the best of the blog and all and a bunch of new stuff in there that's all kind of packaged and wrapped up for you in a framework for designing AI driven products. It's really kind of your blueprint for next four or five years. Hopefully it gives you all the tools you need to move forward and practical exercises to actually learn it and master it. And speaking of mastery, on ux4ai.com There is a list of events so please check that out. You know, one that's coming up next week is May 13th through the 15th is our Bootcamp AI Bootcamp organized by Strat. Four people, four instructors teaching, you know, myself included. Very, very excited about it. We got a bunch of people, just a few tickets left. Check it out. We'd love to see you there. And looking forward to connecting on LinkedIn and all the social media. Thank you so much.
Fantastic. Thanks so much, Greg. I'm looking forward to reading the book myself and I'm sure many of our listeners are too. Hope you have a great day.
Thank you Sean. Take care.
Sean Treizer
Want to keep the conversation going? You can find the show notes@usertesting.com podcast if you haven't already. Don't forget to follow us on Apple Podcasts, Spotify, Overcast or Google Play so you never miss an episode. And if you enjoyed today's show, please share it with a friend or leave us a rating and review on Apple Podcasts. And until next time, this is Insights Unlocked, an original podcast from User Testing.
Insights Unlocked: UX for AI – Designing Intelligent Experiences with Greg Nudelman
Episode Release Date: June 9, 2025
In this enlightening episode of Insights Unlocked, hosts Nathan Isaacs and Sean Treizer engage in a deep conversation with Greg Nudelman, a distinguished UX architect at Sumo Logic and author of the recently released book, UX for AI. The discussion navigates the evolving landscape of user experience (UX) in an AI-driven world, uncovering the shifts UX professionals must embrace to design intelligent, customer-centric digital experiences.
Greg begins by sharing his early fascination with science fiction and robotics, which naturally led him to a career at the intersection of UX and artificial intelligence. Reflecting on his two-decade-long experience working on over 35 AI-driven projects, Greg emphasizes the importance of understanding both the potentials and pitfalls of integrating AI into user experiences.
"I spent a lot of time, you know, thinking about science fiction and robots and so forth and really fantasizing how would it be to live with real-life robotic entities. So that's what we have today."
— Greg Nudelman [01:55]
Diving into his article titled "Iceberg UX," Greg critiques the reliance on traditional design tools like Figma in the AI era. He argues that while tools like Figma focus on UI aesthetics, the true value lies in the underlying content and functionality.
"What we really want to know is what does the steak taste like and look like, right? And that is largely beyond what Figma can deliver."
— Greg Nudelman [03:55]
He uses the metaphor of a meal: UI is the plate, and UX is the steak. While the plate's appearance matters, the real satisfaction comes from the quality of the steak. Similarly, in AI-driven products, the content and interaction quality supersede mere visual design.
Greg emphasizes that in AI-driven interfaces, users prioritize the accuracy and relevance of content over visual elements. He illustrates this by stating that users might not care about the style of a chatbot's UI as long as the responses are accurate and helpful.
"Is the output correct? Is the output summarized in the way that fits my workflow? That is what matters, and that is outside of Figma's purview."
— Greg Nudelman [04:45]
Moving forward, Greg discusses the integration of AI and data into the UX workflow. He introduces the concept of a "rag-based workflow," where AI models are used collaboratively to refine prototypes based on user feedback.
"The model is there to help you and to serve you and, to become your perfect prototype."
— Greg Nudelman [05:35]
This approach shifts UX teams from static design iterations to dynamic, AI-assisted optimizations that respond in real-time to user interactions and data insights.
Despite the technological advancements, Greg underscores the irreplaceable value of human empathy in UX design. He warns against over-reliance on graphical tools and advocates for maintaining a customer-first mindset.
"Our trademark empathy skills that we bring as UXers... that AI just cannot replace."
— Greg Nudelman [09:50]
Greg introduces the concept of a "Value Matrix" to assess AI models' effectiveness beyond mere accuracy. This tool helps UX teams weigh the benefits and potential drawbacks of AI predictions, ensuring that models align with user needs and ethical standards.
"If the prediction is wrong, what is the impact of that? That's a very deep, very valuable conversation."
— Greg Nudelman [10:15]
He illustrates this with an example from TSA screening, highlighting how overly accurate models that rarely flag threats can become ineffective by missing critical instances.
Addressing the high failure rate of AI projects, Greg cites a statistic from Forbes indicating that 85% of such initiatives fail to deliver ROI, often due to flawed use cases rather than technological shortcomings.
"The number one reason things fail is the use case."
— Greg Nudelman [21:59]
He emphasizes the importance of selecting appropriate use cases and thoroughly understanding the problem space before deploying AI solutions.
Exploring the future of AI in UX, Greg discusses the emergence of agentic AI—AI systems capable of performing complex, multi-step tasks autonomously. He likens these systems to a colony of ants led by a queen, where the AI agents handle tasks and report back, facilitating a collaborative dynamic between humans and machines.
"These things are not that smart and they have trouble being in the world... they're like newborn babies quickly growing into experts."
— Greg Nudelman [26:42]
Greg highlights the need for a mindset shift in UX teams to accommodate and harness the capabilities of agentic AI, ensuring that human empathy and strategic oversight remain at the forefront.
Concluding the discussion, Greg shares essential strategies for UX teams to remain relevant and effective amidst rapid AI advancements:
Continuous Learning and Community Engagement: Participating in bootcamps and community forums to stay updated and collaborate with peers.
Defining Unique Value Propositions: Understanding and honing individual strengths that AI cannot replicate, such as strategic thinking and empathy.
Embracing Human-Centric Design: Prioritizing human needs and ethical considerations in every aspect of UX design.
"You need to upskill yourself through your fellow humans and for the use of AI to help thrive in this new normal."
— Greg Nudelman [36:09]
Greg promotes his book, UX for AI, as a comprehensive guide for UX professionals navigating the AI landscape. Additionally, he invites listeners to participate in his upcoming AI Bootcamp, hosted online to accommodate growing interest.
"My book is your blueprint for the next four or five years. It gives you all the tools you need to move forward and practical exercises to actually learn it and master it."
— Greg Nudelman [41:00]
Listeners are encouraged to visit ux4ai.com for more resources, event details, and to connect with Greg on social media.
Greg Nudelman's insights offer a compelling roadmap for UX professionals aiming to excel in an AI-driven world. By shifting focus from superficial UI design to meaningful content and interactions, embracing AI as a collaborative tool, and maintaining a steadfast commitment to empathy and user-centric design, UX teams can create intelligent experiences that not only resonate with users but also drive tangible business results.
For a deeper dive into designing AI-driven user experiences, listeners are encouraged to explore Greg’s book and participate in his educational bootcamps.
For more detailed show notes, curated clips, and supplementary materials, visit usertesting.com/podcast. Subscribe to Insights Unlocked on Apple Podcasts, Spotify, Overcast, or Google Play to stay updated with the latest episodes.