← Design Better
Design Better
Bonus Episode: Shani Sandy, VP, Experience Design at IBM, live in NYC for UserTesting's THiS Connect Tour
00:27:47
Loading summary
Transcript49 lines
- [00:02]
Shani Sandy
What's the human problem to be solved? That's the first question that does not go away. We're gonna go through waves of new technology. AI is the thing right now. Whether you think it's hyped or now, it's real, it's now. So our question still remains, what is the human problem to be solved? Dot, dot, dot. How may AI help solve that problem?
- [00:27]
Aaron Walter
Maybe you've already heard, but we're in the midst of a significant global technolog change. It'll be a struggle for many businesses to adapt, but IBM's a company that's no stranger to change. Time and again, they've reimagined, restructured and refactored their business to stay relevant. Our guest today, Shani Sandy, is playing an important role in IBM's current transformation as Vice President of Experience Design.
- [00:54]
Eli Woolery
Welcome to this episode of Design Better, recorded at this Connect, presented by User Testing in New York. We are talking with Shani today about how her team stays connected to users to design great products, design's role in the new era of AI and how IBM is evolving. This is Design Better, where we explore creativity at the intersection of design and technology. I'm Eli Woolery.
- [01:13]
Aaron Walter
And I'm Aaron Walter. Shani Sandy. Welcome to Design Better.
- [01:21]
Shani Sandy
Thank you, Aaron. Thank you, Eli. Thank you, User Testing. Thank you, everybody.
- [01:25]
Aaron Walter
So here's some irony for you, Shani. We met you at another live event in New York a year ago. And here we are again.
- [01:32]
Shani Sandy
And here we are again. As a.
- [01:33]
Eli Woolery
You're one of the brave people to ask a question, actually. So that's a lesson for you out here. If you ask a question, you might.
- [01:38]
Aaron Walter
It might be on the show.
- [01:40]
Shani Sandy
Watch it. I think it was an AI question, wasn't it? It may have been, sadly, yeah. I didn't mean to bring that up then, but it's back at it again.
- [01:46]
Aaron Walter
Oh, well, guess what? We've got some AI questions.
- [01:48]
Shani Sandy
Oh, goodness.
- [01:49]
Aaron Walter
Okay, talk to us a little bit about how your team's workflow is changing. So you lead a team at IBM, a User Experience team. You've kind of shifted roles recently. How is AI and the various tool sets that are emerging, how is that changing the way your team works?
- [02:04]
Shani Sandy
Yeah, you know, I'm of two minds with this space. I'm both optimistic about the tools and the ways in which it'll help us do our work better and yes, be more productive, also be more creative. I'm also mindful that each person has to kind of charter their own path with their comfort level and what agency they still have and what agency they give to these tools. Right. So I candidly haven't been pushing as much as I think some other leaders are around. You must adopt this AI tool. I've been more so pushing our critical thinking when it comes to the AI experiences we're designing. So what we're finding is that there's this design with AI, which is the former piece of like how we're incorporating our tools into our workflow and then there's design for AI and it's us as designers, as researchers thinking about what are those AI experiences look like in a way that still censors the user. And so that latter piece is really where we're spending more of our time. But I will say that I think in the second half of the year we'll probably get more rigorous around how we're incorporating the tool sets into our workflow. I just think from my point of view, in the kind of work we do, which I'm in the, I should mention I'm in the business where we do support and services of our hardware and our software. So this is a 24 7, 365. There's no rest. If something happens with one of our clients environments, we have to be there. There's no, oh, it's the weekend, that doesn't cut it. And so I think because of the type of work that we do, it's been extremely important to make sure that the uptime, availability, resiliency, all these factors that our clients count on us for are even better. So that means bringing designers into that work. So that's why I've been pushing more in that space as opposed to workflows and tools.
- [04:13]
Eli Woolery
Given that IBM actually has quite a long history with AI, with Watson and other projects like that.
- [04:19]
Shani Sandy
Yeah, Deep blue, the chests and. Yeah, yeah, absolutely.
- [04:22]
Eli Woolery
Jeopardy.
- [04:23]
Shani Sandy
Jeopardy. Yeah.
- [04:24]
Eli Woolery
So given that, I wonder if.
- [04:26]
Shani Sandy
You're back.
- [04:27]
Eli Woolery
You're back. That gives you sort of a longer term view of these technologies and both sort of their potential and limitations as you're designing products with and for them.
- [04:37]
Shani Sandy
It does, and it's great because it's a room full of researchers and designers. Right. You all know your stuff. And I think the difference now though, in the work that we're doing, we very much always focus on kind of the three areas of desirability. Is this something that our users actually want and doing the research around that usability, of course, understanding that what we're creating has to be usable and in a way that folks actually enjoy it. So there's a light piece so those three pieces don't go away. There's still dimensions we care about. I think what's different now is, especially with these AI experiences, can we create experiences that people trust? I think this is now the new paradigm that design is in, where what the outputs are that a user can get, frankly, shouldn't be trusted. Right. So designers have to figure out how to put into that experience, how to put into that engagement guidance or cues to help us build greater confidence around what we're getting from these AI assistants or agents or tools. One thing I've been very adamant about with my team is not just saying we're going to create an AI tool or an assistant, really asking first, like, what is the question? The question is, what's the human problem to be solved? That's the first question that does not go away. And so I think for all of us in the room, we're going to go through waves of new technology. AI is the thing right now. Whether you think it's hyped or now, it's real, it's now. So our question still remains, what is the human problem to be solved? Dot, dot, dot, how may AI help solve that problem? I think when we come to the table with that framework, it allows us to still tap very much into understanding our users, their challenges, their motivations, Getting very deep into that space. Before we start to slap on. Here's another AI thing. That's a space that I'm still deeply committed to, especially because of the type of work that we do where we're trying to help our teams, particularly our support agents, and then our clients be able to do their work better. AI can do that, but it's not the fundamental answer each time. Sometimes the process is broken. Sometimes you actually got to revisit the process because you realize the process steps are actually the foundational ways that we get things done. And if the process is broken, applying AI to that doesn't fix the problem, now does it? It might make it seem like it does temporarily, but it actually fundamentally doesn't. So for me, going back to the question, what is the human problem to be solved, for which AI may be the solution? Is a framing that I'd like to bring up anytime possible. Team is probably sick of me saying that.
- [07:27]
Aaron Walter
I like this idea of bringing it back to the human. Think about this relationship to human and the machine, and we've got to figure out what our relationship is and the idea of trust. Because trust, I think for a long time has been pretty significant issue. All of us here have lived through this very optimistic pre social media, kind of the ecstasy of we're creating great things that are going to change the world. And yes, it did change the world, but in a lot of ways that we didn't expect or intend. I'm optimistic and hopeful and been burned before. Don't want to revisit those bad outcomes. How are you thinking about trust? How are you thinking about designing for trust? How are you getting the insights about your customers to know where they are mistrustful or where things fall apart?
- [08:16]
Shani Sandy
Yeah, Aaron, I feel very much like you. I'm quite ambivalent in this space because I think, of course there are benefits to technology. In this case, we're speaking specifically about AI and oftentimes automation that's also a part of it. But I think that the downsides, frankly, are underplayed. And I think that for us, particularly those of us in design, I really think our work is to strip off a lot of the layers of the seemingly exponentially amazing technology that in certain rooms and spaces we all now have some form of access or familiarity with and really get at the deeper root of the consequences of the technology. So if we're supposed to be the ones that understand our users, if we're supposed to be the ones that advocate for our users, then that means that as we're designing these experiences for our users, we need to be thinking about what does that look like for them and what are the potential consequences and unintended consequences. I will definitely give a shout out because our IBM design team has done a fantastic job with actually creating these principles around how do you design for things like trust? Making sure that there's explainability involved when you're creating these AI experiences. So can you actually explain how the AI came up with output? Can you design that? Can I see as I'm interacting with an assistant, how did you actually come up with this? So that I know whether or not I can trust it or not, Especially with the types of clients that we deal with, we cannot afford to put out information that is erroneous. And I think probably anyone in this room could say the same thing. Things like having confidence ratings in the responses. So when a response is then given to a user, they're able to see okay. This particular response was scored about 85%. Right. So that also is a way to imbue these qualities of okay. As a user, I can have a bit more consideration that this might be a more accurate answer, but you shouldn't take it off the premise of it's right because it came out of the machine. And so I think these types of things, as designers, as researchers, we need to think about how do we do this for our users, and how do we also make sure that we continue to evolve those types of experience checkpoints, because it's going to change as the technology changes. I'm cautiously optimistic that many more designers and researchers are thinking this way. My concern is the technology is changing so fast and it's hard to keep up. And so that's why I keep saying, if we come back to like a very basic question of what is the human problem to be solved? I think that's a way to help calibrate us back to the humanity piece. But I do think that there are aspects of this work that we're just still learning, we're still figuring out, and we're trying to catch up sometimes. And so that means that designing for trust is going to mean literally every time we're putting out an experience that we have got to understand what that means for the user, whether it's ethnographic research, whether it's qualitative research, quantitative. We have to get literally in the shoes of our users to figure that out. And we've done that within our teams and really shadowed teams and are now putting in more behavioral analytics to figure that out. But I would absolutely say to each and every one of us who are in the room today, it is our responsibility to design for trust. It's not a nice to have in the kind of work that we're all likely now doing.
- [11:49]
Eli Woolery
I think the trust thing is huge and we run into it in our own work, maybe in a smaller scale way, and that occasionally we'll be writing an article and we want to pour through 200 pages of transcripts and extract some themes and time codes from a certain episode. But I have to go back and check did it hallucinate? And often it did. And then it tries to blackmail me. Then I tried to shut it down and it won't shut down. So we run into all these kinds of issues. But for a company operating at scale, where it's not just two people working on a podcast, but you're trying to stay connected to your users and sort of assess their level of trust. You mentioned ethnographic research, but are there ways or can you go deeper on that? Or are there other ways you stay connected to your users?
- [12:28]
Shani Sandy
Yeah. So last month our team had a gathering in person, an in person gathering. And as part of the gathering, it was really primarily around us as a team getting together and Forging certain principles around ways of working for our team and with our cross functional team members in development and in product. But it was also an opportunity to be on the ground with our users and do co creation sessions with them. And our users, they run the range. So they're internal agents who are dealing with support cases in volumes that are just unwieldy sometimes. And then there actually are clients who have issues and are coming to us to get their issues addressed. But we're also looking at doing things in a preventative way so they're not having to come to us for these issues. Nevertheless, one of the things we did is I was adamant that once we're there we have to carve out time to meet with our end users and conduct a session with them around their needs. So we did need statements, we did empathy mapping and we just like connected with our folks and understood what a day in the life looked like for them. I know that was the first time that ever happened, now it won't be the last, but it needed to happen. And so I think these types of things I'm definitely a fan of like hybrid ways of working and different approaches. However, you cannot replace the in person nuance look on people's faces the way they may reserve some of the comments they want to make because of the impact or the consequences it may have. Maybe the things that you see someone in the corner who's quiet and you curious to hear what they have to say. These types of things we're able to get at when we're in the room together with our users. And so I think those are ways in which bringing our users into the work and not in an extractive way. I have to say this, I think sometimes in our profession it's like, okay, I need to do some user research so I can validate XYZ that's extremely extractive and I don't care for that approach. And so whenever possible, I know it's not always possible, whenever possible we can do this in a co creative way. That's my preference. And I will say, you know, my first few months in the role that I'm in now, I, I shadowed people, I went on site visits with one of our agents who was in the field changing hard drives and fixing different parts of the client data center. And Jess was an observer. And so I think wherever we can do more of that, it builds a deeper appreciation of the work and then bringing our users in to do the co creation, there's no replacement for that.
- [15:03]
Aaron Walter
One fascinating thing that we're seeing as a theme as we talk to more and more teams is that there's this blurred line between disciplines. So the phenomenon of working on very complicated things like we all work on it requires a lot of specialties and deep knowledge and expertise. With that deep knowledge comes its own specialized language. As we have more specialized language, it is harder to communicate. And we have different value systems, engineers, designers, et cetera. There's different value systems and sometimes those collide in very uncomfortable interpersonal ways at work, or everyone here has some story like that. But one thing that's really interesting to us is that we keep hearing designers say, hey, I can do this engineering stuff now. I can vibe code some things to prototype very quickly. And we're hearing engineers say stuff like, hey, now I can design a UI and get pretty close pretty quickly. And I can't help but think that we are accidentally creating this brackish water between our teams where there's this opportunity for us to understand one another in new ways. That's something you've seen.
- [16:13]
Shani Sandy
It resonates. It resonates. I think that it's okay. I think it's okay that the lines are blurring because I think the type of work we're doing now is work that is so multimodal, it's moving at a clip that's so unwieldy that you need that cross functional, cross hybrid, multi hyphenate type of person. I think just to be in one lane, frankly, and just to have that specialty, it's limiting in the type of experiences that need to be created. So I think that it's uncomfortable. I think there's a feeling like, oh, that's my turf. And now a developer is doing something that they did in figma and now they think that they can design and et cetera, et cetera, right? Or vice versa. But I do think it's the way of the future where we're gonna have more folks who are just multi hyphenated in their professional role. And I actually think it's healthy because it reflects the multi hyphenated ways that we're living and working in the world. You wake up in the morning and you might look at your phone and then you look at your laptop and then you're off. And you might look at your watch on your wrist and that's like checking your biometrics and you're still in the world and you're still living. You're having environmental experiences in spaces and moving throughout space. And so I think that just the reality is that our modalities are just more and more evolved and will be more and more varied. And so to me, then professions should just reflect that. So it's gonna be uncomfortable, but I think that if you want to remain engaged, you should consider being a bit multi hyphenated.
- [17:50]
Aaron Walter
Yeah. You know, it's interesting, in the prior conversation, the panel before there was a comment about being digital native or not, feels like AI creates this opportunity for us to be native, almost any discipline, if we wanted to be.
- [18:04]
Shani Sandy
I'm probably going to screw this quote up, but the idea is that like the AI we are experiencing today is the worst AI we'll ever experience. I think that's exactly right. That we're kind of all in it. We're all kind of figuring it out as we go. And as the weeks and months progress, the technology is changing. Right. Just this past weekend, VO was released, which is the ability to use text to generate video.
- [18:29]
Eli Woolery
Yeah.
- [18:30]
Aaron Walter
Like really legit.
- [18:32]
Shani Sandy
Yeah, pretty legit video. I want to play around a little bit, but I'm not subscribing $150 a month for it. But these types of things are happening in real time. Right. And I say that a little bit facetiously too, about the cost, because in reality, not everyone is having access to these tools. And so there's another piece of this not suggesting that this has to be the space for designers, but I am suggesting that if you care about users and democratizing access, that, you know, that's another space we might want to consider playing in or understanding how can the access to these types of tools be made more accessible to more people? Because right now, frankly, if you want to play around with maybe a tool set of three, I won't name names with tool set of three, you're going to pay a couple hundred dollars a month for that on top of just anything else you have. Right. So I do think there's part of this that is a conversation that's privileged in your ability to access these types of tools.
- [19:32]
Eli Woolery
In addition to the kind of multidisciplinary changes that are happening and roles collapsing or the types of things you're expected to do expanding. It's a tough environment out there for folks that are maybe looking to change jobs or find a job. We do a survey every year and 41% of the respondents this year said that their biggest challenge was finding new work. Not necessarily maybe a job that they wanted more, maybe not. They're not out of work. But in any case, career is a top of mind. Companies are tightening Their belts. Folks are looking for jobs. In my case class, I teach graduating seniors and this is the hardest job market they faced in the 13 years I've been doing it. I'm just curious, how does this sort of shape the work at IBM and your team and how's it affecting you all?
- [20:12]
Shani Sandy
First, I want to recognize that it's tough times. I will add to that friends and colleagues that I have been talking to in the last few months, especially who are looking for work, have expressed how just darn hard it is. Like it's just tough right now. And it's not a reflection of people's skills. The reality is that places are not hiring as much anymore. The rub is you have these amazingly talented people in the marketplace that aren't able to find the right opportunities. And so I think about this in several ways. For me and my team, and also from an IBM point of view, there has been a lot of focus on gaining the skills, gaining particularly the skills around AI, understanding data and data structure and architecture, understanding the basics of what it means to build for AI. I think we all need to understand that it doesn't matter what role you're in. I think it's important to have an understanding of what that world looks like in terms of when people say AI, what they actually say versus what it actually means. And having enough ability to call BS when people say things is hugely empowering. So I think just for your own benefit, that's important. So I think yes, acquiring the skills is going to be important. What skills does that look like though? I think a lot of folks talk about prompt engineering and how you have to be the best prompt engineer and that's going to make you viable. I think there's validity to prompt engineering, but frankly like AIs can now create amazing prompts. So like, okay, so you learn how to prompt engineer and now like an AI can do it. So I think that to me, what's more foundational or maybe a return to is the so called like interpersonal skills. Sometimes people call them soft skills. I don't like that term, but I'll use it because I think when you say soft skills, people know what you mean. But I think these are the skills around being able to communicate in ways that are compelling, being able to convey your message, being able to negotiate, being able to collaborate with teams that are going to be more and more increasingly, increasingly diverse in role types. Right now we work with linguists and data scientists, not just developers and engineers and product folks. So my read is that what we've been doing within our teams is thinking about how to, yes, do the technical work of building these AI experiences, but also what are the soft skills we need to also be able to develop and advance in the world and the way it's changing right now. And I think for folks like, looking for work first off, wherever you can help folks, that's the place we should be in. That's the gap we should step into. Like, talk to folks who are looking for work, be there to say, you know, what if something comes up, I'm gonna put your name in first or I'm gonna make sure that, you know, I connect the dots to someone I might know. I think now is the time to, like, really dig deep into community and for each of us to help each other, especially when we're in situations where we're looking for work or wanting to make a career pivot. And I think the last two things I definitely want to say on this topic because it's such a big one and there's so much more to talk about because of how much it's impacting professionals across the board. I think, one, I would love to see more designers go into the space of building these AI companies. Pie was one. It was supposed to be like a personal assistant that had emotional intelligence. Right. I don't know how well it caught on because I stopped using it, but I would love to see see folks who are, again, focused on understanding users, understanding human behavior, and having those folks really get into worlds where they're founding these businesses because I think they're going to bring a different lens that's missing to the space. And I think the last thing is maybe it's a time for the overused word pivot. It might be time to say, you know what, I'm not really finding the thing for me, or I hear all about this AI space. Okay, cool. I know about it, but I don't want to go all in. So maybe it's time to pivot into something entirely different that you put on the back burner, but maybe right to do.
- [24:12]
Eli Woolery
Aaron's thinking about becoming a baker. I mean, I'm trying to not convince.
- [24:15]
Shani Sandy
Him not to, but listen, listen. Baked goods done.
- [24:19]
Aaron Walter
Everybody always wants that. Talk about putting the human back in the table.
- [24:22]
Shani Sandy
Yeah, yeah, you got it. You got a customer right here, built in.
- [24:25]
Aaron Walter
Yeah, I like what you're saying about helping people out to find work. And just a quick plug designbetter dot directory. There's a ton of talent there. Very talented, experienced people. If you're looking to hire and if you have job openings, same place, you can post your job openings for free.
- [24:43]
Eli Woolery
We have had people hired off.
- [24:44]
Aaron Walter
Yeah. So as we wrap up here, I'm curious, you've had an amazing career and you're doing a lot of interesting things at IBM. What are you most proud of what you and your team have done?
- [24:54]
Shani Sandy
I guess I'll just use the most recent example because to be fair, there's so many proud moments. But I had shared that last month our team was doing a gathering and it's a newly formed team, so there are some experienced folks, but some folks who are brand new. And we did it in Sofia, Sophia in Bulgaria. We had a three day session and the second day, guess who gets food poisoning? Oh, yeah, it was me.
- [25:21]
Aaron Walter
No baked goods.
- [25:22]
Shani Sandy
No baked goods. No baked goods. And so I was done. I didn't even know if I could make it through the rest of the trip. And so that Wednesday, I let my team know, you know, I'm not feeling so great. Let's touch and go. I'll see how I'm doing in the afternoon. But was like, I'm here, I can't not figure this out. Like, I gotta figure something out. So I was doing everything I could to get better. Eventually made into the office that afternoon. Again, nowhere near 100%, but did the best I could. And I got there and I was like, they didn't really need me. They did an amazing job. Second day was all about. It was actually I was leading the second day and, and revisiting our retrospective around areas that we wanted to improve on. What are some of the solutions we could create to address those problem areas. And had an entire program around what I wanted to make sure was accomplished. And I got there and they just did it. They led and they made it happen. I came in and they were already thinking about solutions and owners and, you know, how they would approach the challenge space. It might sound like, really like, oh, that's what you're most proud of. Like, that seems just like normal or small. But what you have to recognize is like, this was the first time meeting in person. I just hired half of the team. It's a very, very small team. I had just hired half of the team a few months ago. And so the very fact that they just assembled themselves and figured it out to me was reflective of them understanding that they have as much ownership in where the team goes than I do. And that was icing on the cake for me. So I think that would be it.
- [26:54]
Aaron Walter
Chani, thank you so much for being on design better.
- [26:57]
Shani Sandy
Thank you for having me. It was way too short. Next time we'll have to do a part two.
- [27:00]
Eli Woolery
We'll do a part two, okay?
- [27:08]
Aaron Walter
This episode was produced by Eli Woolery and me, Aaron Walter, with engineering and production support from Brian Paik of Pacific Audio. If you found this episode useful, we hope that you'll leave us a review on Apple Podcasts, Spotify, or wherever you listen to finer shows. Or simply drop a link to the show in your team's Slack channel, designbetterpodcast.com It'll really help others discover the show. Until next time.