← B2B Agility with Greg Kihlström™: MarTech, E-Commerce, & Customer Success
B2B Agility with Greg Kihlström™: MarTech, E-Commerce, & Customer Success
#66: Building a culture of AI innovation with Michael Domanic, UserTesting
00:21:21
Is your organization truly AI-powered, or are you just slapping a little GenAI onto existing processes? Agility requires embracing experimentation and empowering teams to rapidly iterate, especially when integrating transformative technologies like...
Loading summary
Transcript37 lines
- [00:01]
Craig
Is your organization truly AI powered or are you just slapping a little gen AI onto existing processes? Agility requires embracing experimentation and empowering teams to rapidly iterate, especially when integrating transformative technologies like AI. It also demands a culture that views failure as a learning opportunity, not a setback.
- [00:22]
Michael Dominic
The Agile brand.
- [00:30]
Greg Kilstrom
Welcome to the B2B Agility Podcast where we look at the factors that drive success in B2B marketing with a focus on the people, processes, data and platforms that make B2B brands stand out and thrive in a competitive marketplace. I'm your host, Greg Kilstrom, advising Fortune 1000 brands on martech, marketing operations and CX, best selling author and speaker. Now let's get onto the show.
- [00:56]
Craig
Today we're going to talk about how to build a culture of AI innovation from the bottom up, empowering every employee to leverage its power. To help me discuss this topic, I'd like to welcome Michael Dominic, Head of AI at User Testing. Michael, welcome to the show.
- [01:10]
Michael Dominic
Thanks, Craig. It's great to be here.
- [01:12]
Craig
Yeah. Looking forward to talking about this with you. Before we dive in though, why don't you give a little background on yourself and your role at User Testing?
- [01:19]
Michael Dominic
Yeah, so I've been at User Testing for over six years now and my role is head of AI at User Testing. So really what that means is I'm driving our AI transformation program. We want to make sure that all of our employees have the education, enablement, empowerment to use AI in interesting ways, to augment their work. Do that responsibly, do that ethically. That's my primary focus. I've been at User Testing for six years. I was a customer of User Testing for several years before joining User Testing. So funny enough, when I was a customer of User Testing, I was designing natural language processing chatbots and testing them in User Testing. That was like 10 years ago. So I feel like my work in AI now comes full circle.
- [01:58]
Craig
Nice. Nice. Yeah, it's always great when you're a customer and you end up working for the. I think that says something about the product. So that's great. So, yeah, let's dive in here. We're going to talk about a few things. I want to start with the concept that AI is everyone's job. And so User Testing has embraced this concept. What led to this decision and how did you overcome, you know, maybe initial. Any resistance or hesitance or things within the organization?
- [02:30]
Michael Dominic
Yeah, I think User Testing kind of recognized early on that this, you know, new wave of AI, you know, LLMs, generative AI, was really going to be Transformational for not just our business, but probably most businesses. So I think, again, like, we recognize that early on, and a group of folks here at User Testing started thinking about how that's going to impact us. How, how will that impact the way that we build our product, how will that impact features that we build into our product? But then also, how does that transform the entirety of our organization? So, because we recognize how really momentous this really felt, we decided to put a formal transformation program together that was going to help us do all the things that we wanted to do. And look, I, I think that there's two reasons why User Testing, and I think maybe like any company needs an AI transformation program. And the first is kind of what we talked about is there's a lot of opportunity that we all recognize to augment our work and to do more things by bringing generative AI into that work, and we want to take advantage of those opportunities. So I think the other reason is responsibility. And I think responsibility has two branches to it. The first is we want to make sure that if we're going to tell our employees we want you experimenting with AI, that they do that responsibly, they do that ethically. And the other branch to responsibility, I think, is the responsibility that companies like User Testing have to our workforce. We recognize that this may actually be one of the most transformational events that we're all going to go through in our lives, maybe in history. Right. So we want to make sure that everyone has the enablement to navigate through that transformation.
- [04:05]
Craig
Yeah, yeah, well, and it sounds like there's been quite a bit of work done within the organization. You've developed over 600 custom GPTs internally. Uh, so, you know, that's a, that's a significant amount of, of of work being done by those internal teams. So, you know, how have you managed to maintain quality control and avoid maybe, you know, 600 sounds really good and, and really empowering, but also that that number could be a potential, like, Wild west scenario, you know, so how do you, how do you avoid the, the latter? And, you know, with so many different AI tools being created simultaneously.
- [04:44]
Michael Dominic
Yeah, you're absolutely right. Like, it does feel like a Wild west scenario. And actually we're over 700 GPTs now, so it's becoming wilder. Right. So we do empower, enable everyone in our organization to build these custom GPTs. We built a very strong culture around it. Now, look, 700 GPTs across an organization of, you know, about 800 people. Not all those GPTs are being used widely or even Regularly. Some of them are just experiments. Some of them are GPTs that individuals will create for specific common workflows that they're in. Right. Like, they're the only person using them. But there are a lot of those GPTs that are being used either across teams, across functions, across the entire organization. So there is management involved in that. It's a little bit messy. Full transparency. We're in the kind of messy, weird, wild situation that we knew that we were going to be in, and we decided we would be comfortable with, because on the other side to that is we have all this experimentation. You know, as our employees create these GPTs, they're learning about the impact that AI has on their work. And look, even if the GPT that they end up creating is just an experiment, it's a good learning process to figure out, here's the power of bringing this into my work. So we're. So we're okay with that? Look, I do think that there comes a time where we need to maybe put a little bit of a fence around this, a little bit of, like, internal regulation around the creation of these custom GPTs. But I think we've been happy with the level of experimentation, and we kind of want to see that a little. That little bit of a messy scenario.
- [06:18]
Craig
Yeah, I mean, I think that's. That's really powerful to be able to empower employees to be able to do that. I mean, you know, I always say, like, experimentation, you know, experiments don't always yield the desired results. But that's not the fault, you know, that's. That's not the fault of the person. That's the point of an experiment. Right. So you've got to do some of this stuff to get great results. Right? So, yeah, I think that's just. That's good. That's teaching people to think and be creative and not worry that a hundred percent of what they do is going to be the perfect solution. Because it. It can't be. Right?
- [06:52]
Michael Dominic
Right. And look, I mean, like, not to belabor this point, but there is a time, like I said, that will come where we will exercise a greater kind of guidance and control over that. Everyone that's creating a GPT is a product manager. Right. They're taking on the role of a product manager, and they're building something that is going to have some relevance to them, you know, themselves, their team, or the wider organization.
- [07:13]
Craig
So you.
- [07:14]
Michael Dominic
There are certain levels of regulation that we're thinking about. Like, if you're building something for the wider organization, you're signing up to, you know, kind of fulfill these specific requirements. If you're building something just for yourself, like, okay, maybe there's not a lot of regulation.
- [07:28]
Craig
Yeah, yeah, makes sense. And so far you've achieved some pretty impressive internal results like, you know, increased productivity, reduced search time. Can you maybe share some examples of how some of these custom GPTs have delivered some tangible value?
- [07:46]
Michael Dominic
Yeah, it's something that we actually track pretty closely and attach ROI to. So an example of that is there are a lot of GPTs that have been built into the sales process. For example, our BDRs actually have a series of GPTs that help them target more relevant prospects with, you know, a higher cut through rate. In a SaaS company like ours, a really important event in our sales cycle is the discovery call. You know, that's where we actually have an opportunity to sit down with a prospect and have a deep conversation with them about some of the challenges that they're experiencing in their organization where user testing might be a solution to those challenges. And we do have like a relatively high conversion rate from our discovery calls. So we created a series of GPTs that would help us generate more of those discovery calls. Right. And generate more relevancy around those discovery calls. So some folks on that team created a series of GPTs. When we look at the before and after state, those the implementation of those GPTs contributed to 160% increase in our discovery call bookings. And that has again, pretty, pretty significant implications for, you know, user testing, being able to reach more organizations, have more targeted conversations with our prospects.
- [09:03]
Craig
Yeah, yeah, that's amazing. Your, your research in this also highlights the importance of user feedback in AI development. How do you incorporate some of those human insights into iterative development of some of these internal AI tools so they remain effective and aligned with business needs.
- [09:22]
Michael Dominic
Yeah, so we like to say that we drink our own champagne. Right. So user testing produces a, you know, sells a SaaS platform that helps our customers capture insights around all the things that they're building. We use that, that tool internally quite a lot. The way that I use that tool is I do a biannual survey across all of our employees that captures an understanding around how are they using AI, how often are they using it, what type of productivity is it leading to? Is it not leading to productivity? Right. What are generally people's fears about using AI? Because that's a real thing too. So we want to capture that. So generally biannual AI survey is going to help us understand the level of proficiency and adoption around AI. In between the Large survey that we do. We also do a lot of qualitative analysis and qualitative, you know, testing with our employees to understand more about why. Why have you been able to achieve, you know, the things that you've achieved with by bringing AI into your business or into your roles? Why are you not using AI? Like what do you have fears around this and what are those fears? What's like, what's driving that? Right. So we do a lot of kind of qualitative testing and analysis around the way that we're using AI in the business as well. And we're lucky that we have user testing as a tool to be.
- [10:42]
Craig
Right. Yeah, I know that must be a great. Yeah, that's a, that's a major benefit to be able to just learn more effectively and more quickly.
- [10:49]
Michael Dominic
Right?
- [10:49]
Craig
Yeah. So you know, what I've seen in my work and you know, I often work with organizations as they're adopting AI and you know, I think there's some people, you know, it takes all kinds, right? So there's those people that kind of like myself, that I'm not an engineer but I'll dive in and try anything and you know, sometimes I butt my head against the wall or whatever but you know, I' try anything and, and stuff. There's others that maybe are, for whatever reason are more reluctant. Some are maybe waiting for guidance or other things. So again, you know, kind of takes all kinds. But what training or resources did you find most effective in enabling? You know, whether it's the non technical employees, maybe those that had, you know, either skeptics or hesitants or things like that. And you know, maybe what approaches didn't work as well and you know, what do you learn from, from those experiences and enabling?
- [11:39]
Michael Dominic
Yeah, I think this starts with understanding that AI transformation, AI adoption is actually a creative challenge that's disguised as a technical one.
- [11:51]
Craig
Right.
- [11:51]
Michael Dominic
Like a lot of folks think, oh, like this is crazy technology, right. This is going to be handled by our technologists. I actually think that people that are the most creative are the ones that are making the most progress by figuring out interesting use cases and bringing that into their work. So I think for starters, it's recognizing what kind of challenge is this right now when it comes to enabling our employees? I mean that, that is a big focus, right? That's a big focus of transformation, is making sure that everyone has access to the level of enablement and the level of training. We do a lot there, right. So we do monthly AI lunch and learns where we do spotlights for like people who are doing interesting things with AI. They get an opportunity to talk in those. Yeah. To the entire organization around, what are, what am I doing? What could you learn by, you know, me bringing AI into this process? There's an AI center of Excellence at User Testing, which is a group of about two dozen ambassadors, cross functional people who sign up for the mission of saying, I've done a lot with bringing this into my work. Now I'm going to help my peers and teams adjacent to mine figure out how that means for similar roles, weekly office hours that are staffed by myself and by our ambassadors. So, I mean, there's just a lot that we're doing to help bring our organization this level of enablement education. There are also educational tools that we bring into user testing as well. There's an awesome tool called Section, which is like an edtech tool that gives a lot of lessons and content around how to bring AI into your work. We have really high adoption and engagement with that content. So, yeah, I mean, there's just a lot that we recognize needs to get done. We're trying to do as much of it as we can to help our employees recognize opportunities to experiment and bring AI into their work. We can't do it all, but if we could do at least some of that or a good amount of it, then a lot of our employees get an opportunity to engage with that knowledge.
- [13:46]
Craig
Actually, I had somebody from Section on the show a few months back, so definitely, yeah, really cool platform for, for learning there. And I mean, it definitely sounds like you're, you know, your, your teams are able to create a lot and learn a lot, and, you know, at the, at the pace, you know, not only, you know, the, the total number of GPTs, but it sounds like you're building about 40 GPTs per month. How have you, you know, in addition to teaching and educating, just making sure that everybody has access to the support that they need. Because I'm sure there's a lot of questions along the way and people trying new things and, and all that. You know, how do you, how do you enable all that along the way?
- [14:25]
Michael Dominic
Yeah, I mean, there's so, there's a Slack channel, right. That folks can come to and, you know, there's a lot of conversation that happens there. That's like a big center of gravity for our continuous conversation around AI. So, you know, folks can come there with questions if they have questions about resources, about ways to use AI. There's a very robust and hearty conversation happening there. Not to get too meta as well, but there's a GPT that folks can use that helps with their enablement and understanding. You know, what's happening with this AI transformation program. What are my resources? What do I have access to? Yeah, I think, like, what we're trying to do is meet people where they are. Right. There are existing tools that we use every day, and we're trying to bring that education and enablement where folks are working with within the business.
- [15:15]
Craig
So I want to know, we've talked quite a bit about the internal teams that are, that are using them. Let's call it the internal clients of some of these, these tools. I want to talk a little bit more about the end clients then. So, you know, the, the shift and the kind of embracing and embracing AI and empowering employees to be able to use AI. How has that, you know, influenced the products and services that you offer to your end clients?
- [15:42]
Michael Dominic
Yeah, so again, like, I mean, it's really a matter of bringing AI into all of the stuff that we're doing across the business to create better outcomes. Our product team is certainly a big focus of where we're doing that as well. Really what that's allowed us to do is work faster by building product that has greater relevancy for our customers doing that faster. There are really interesting things that we could do today with vibe coding. Right. That our product team can do to help build more robust workable prototypes that they give to our developers. And now our developers have a much stronger sense of the way that the product is supposed to work, the way that what they're building, how that's supposed to work. So there's just so much opportunity, Right. To, to enhance the things that you're doing across the business and product and engineering is certainly a big part of that as well. On the other side to that now is, you know, we have a number of robust AI features in our platform that our customers are using on a regular basis because, you know, we want to make, we want, we want to bring that efficiency to our customers as well. We want to help augment the work that they're doing. So when our customers are using our platform, they can invoke AI to help them find more, more interesting and relevant insights that they're capturing in the testing that they're doing in our platform. So, you know, we don't just do this internally. Right. We're trying to do this for our customers as well within our product.
- [17:11]
Craig
Yeah. And so looking ahead, you know, how do you see AI adoption in the enterprise? Like, how, how are you looking at that? And you Know, as well as this role of, of leaders, really empowering so a lot of those non technical employees to play, you know, shaping the future of the company using some of these AI tools.
- [17:32]
Michael Dominic
Yeah. So we've come a long way. We're roughly 12 months into our AI transformation journey and I think we're very happy with the level of progress that we've made there. I think the key themes right now that are relevant to us and probably most organizations. One is we want to find, find easier ways to bring all of our enterprise data into an AI environment to help bring more context to the things that we're doing in AI and we want to do that responsibly. Right. Like there are some challenges there, a lot of things to think about. We want to make sure that if we're going to bring connect an enterprise system to AI that that's not going to present any new, you know, legal or security challenges. So that's something that we talk about and think about quite a lot in the organization. I think the other key theme is all of the agentic AI opportunities, Right. That we have. Like I say this like a little bit of smile on my face because this is probably the most overused word in AI over the past six months.
- [18:25]
Craig
It is the buzzword of the day. Yeah.
- [18:28]
Michael Dominic
But there are actually real and very interesting things that we could be doing as these capabilities increase. So we're looking at ways to continuously augment the things that we're doing, create greater efficiencies by potentially designing autonomous agents to do, you know, some of like the grunt work, some of the kind of like more menial work that we do across the organization that's going to give our workforce the opportunity to do more of the strategic work that they're very, very good at. We're all knowledge workers, right. Like we're all hired into this company because we have, we're very good at strategic work. Right. Like we're very good at bringing our knowledge and our backgrounds and our experience into user testing. And the less, you know, menial work that we have to do, the less logging things in systems that we have to do, the more opportunity we have to do that strategic work. So we're looking at opportunities to do that.
- [19:21]
Craig
Yeah, yeah, absolutely. I definitely, yeah, I see with agentic a huge opportunity to buzzword that. It is like a huge opportunity to really elevate, you know, to your point, elevate what employees are doing and kind of leave some of the other stuff for the, for the machines that that's what they're best at. Right. So yeah. Well, Michael, thanks so much for sharing your ideas and insights today. One last question for you before we wrap up. What do you do to stay agile in your role and how do you find a way to do it consistently?
- [19:52]
Michael Dominic
That's actually a unique challenge right now, I think, for folks that are trying to wrap their arms around everything happening in AI. Look, I mean, like the what I try to do is I try to bring as much of the processes that I'm trying to get folks to do into my own work. So I bring AI into my work. I do have light agents that are helping me understand, like what are all the things that are happening right now. There's GPTs that I build to help increase efficiency around the knowledge stuff. And also it's kind of accepting the fact that we're in this wild west period, messy period of, you know, AI transformation and some of it is just kind of being okay with that. Yeah.
- [20:29]
Craig
Love it. Well, again, I'd like to thank Michael Dominic, Head of AI at User Testing, for joining the show. You can learn more about Michael and User Testing by following the links in the show notes.
- [20:41]
Greg Kilstrom
Thanks again for listening to the B2B Agility podcast. If you enjoyed the show, please take a minute to subscribe and leave us a rating so that others can find the show more easily. You can access more episodes of the show at www.b2b agility.com. that's b2b agility.com while you're there, check out my series of bestselling agile brand guides covering a wide variety of marketing technology topics. Or you can search for Greg Kilstrom on Amazon. Until next time, stay focused and stay agile.