← The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
#616: Synthetic personas in market research with Ali Henriques, Qualtrics
00:29:12
Loading summary
Transcript34 lines
- [00:01]
Greg Kilstrom
What's the state of your website? If you're looking for a new digital experience platform or DXP to run your digital experiences, I have the book for you. The Agile Brand Guide to Digital Experience Platforms is part of my best selling series of MarTech books. In this book I explore and demystify DXPs and look at the roles a digital experience platform should play, the different types on the market, as well as how to initially evaluate platforms, then how to best implement a DXP once you've selected it. The book also features a forward from Rupali Jain, Chief Product Officer at leading DXP provider Optimizely, as well as several other thought leaders in the industry. Learn more and get a copy in print or digital now by going to the Agile Brand guide website at www.agilebrandguide.com.
- [00:48]
Ali Enriquez
The Agile Brand.
- [00:56]
Greg Kilstrom
Welcome to season six of the Agile Brand where we discuss marketing technology and customer experience, trends, insights and ideas with enterprise and technology platform leaders. We focus on the people, processes, data and platforms that make brands successful, scalable, customer focused and sustainable. This is what makes an Agile brand. I'm your host Greg Kilstrom, advising Fortune 1000 brands on martech, marketing operations and CX best selling Author and speaker. The Agile Brand podcast is brought to you by Tech Systems, an industry leader in full stack technology services, talent services and real world application. For more information go to teksystems.com now let's get on to the show. What if the future of market research isn't human at all but relies on AI generated Personas to mimic real world consumers? Today we're joined by Ali Enriquez, Global Director of EDGE at Qualtrics. Ali is here to discuss how synthetic Personas, AI and digital qualitative tools are reshaping the future of market research and share some Insights from the 2025 Qualtrics Market Research Trends Report. Welcome to the show Ali.
- [02:07]
Ali Enriquez
Thanks for having me. Happy to be here.
- [02:09]
Greg Kilstrom
Yeah. Looking forward to talking about this topic. I have not talked with anyone about this so I'm really kind of curious myself. So look at looking forward to this. But before we dive in, why don't you start by giving us a little background on you and your role at Qualtrics.
- [02:25]
Ali Enriquez
Absolutely. Happy to. So I'm I call myself a career time Qualtrics user so started my career, I've always been in market research in the cruise line industry is where I started. I'm a certified moderator, quant research ninja all the things right. So started as an analyst at Royal Caribbean and got my hands on a Qualtrics license and I have never relinquished that. So I love my opportunity to kind of watch the Qualtrics journey from a couple of different spots. So as a client and then as a vendor before even joining full time. So was actually the first researcher hired into this team we now call Edge, formerly called Research Services. And now I was just telling you, I'm sitting here in Mexico City looking out upon a team of over 60 people here doing, doing what I, I was first hired to do. So our team really helps clients maximize the value of their license. And so we're supporting them in designing surveys, producing the insights and recommendations on the back end of that. But what we're here today to talk about is more of our innovation. So much like the rest of the industry, we are leveraging AI in really, really cool, creative ways, Synthetic being one. But yeah, so that's, our team is kind of split legacy market research agency type of capabilities as well as this new kind of innovation.
- [03:51]
Greg Kilstrom
Great, great. Yeah. So let's dive in here and yeah, the first thing I wanted to talk about was AI and synthetic Personas and market research. And so, you know, you touched on it very briefly there. But for those that are less familiar, I'll confess I'm learning myself here on some of this stuff. So let's start with some definitions. So maybe, you know, there's talk about synthetic responses, synthetic Personas, knowing all of this is tied to AI. Can you talk about, you know, what is some of those terms or what do those terms mean?
- [04:23]
Ali Enriquez
So hard to do that without jargon, right? So I think the simplest way that I've heard it described is like a digital twin, right? That's, that's becoming a bit more commonplace in vernacular, I think about how would I describe this to my mom? Right. So that's in, in its simplest form, it's taking human based data, operational data, whatever data you have, you know, in, in at your disposal, combining that with publicly available information like anything in Google or you know, available on the Internet and using that to model what a response would look like to a question. So it's hard. You know, we call it synthetic data, synthetic responses. But eventually synthetically simulated answers will, will be just that, there'll be answers to questions. But today what, what, what we're talking about, what we're building are, you know, for the researchers out there, record level, row level data. Right? So taking a survey and asking a Persona or a population what they'd answer to, to those questions with some prompting. So we say I want My Target audience is 18 to 24. They skew female, they live in urban areas. And what would they think of this new salad concept? Right. And the model is predicting, stimulating what that population would say, how they answer, what they react. So it's very much AI powered LLMs, large language models. We talk about it as simulated, modeled responses.
- [05:57]
Greg Kilstrom
Great, great. Yeah, thanks for those, those there. So let's. Another thing I mentioned, you know, at the top of the show, we're going to talk in a few different ways about the, the 2025 Qualtrics Market Research Trends Report. So, you know, first thing there, you know, it's highlighting that 71% of researchers believe that synthetic responses, as you just defined, will dominate within three years. Not time there. What's, what's driving the shift and, you know, how do synthetic Personas address challenges that we're hearing a lot? You know, do more with less. So budget constraints as well as other things like privacy concerns and even, you know, survey fatigue.
- [06:40]
Ali Enriquez
Yeah, it's all of those challenges. Right. I think often about the, you know, the pyramid of good, fast, cheap. Right, right. Honestly, synthetic helps us with all of those things. So, yeah, the researchers are struggling. I've not met a client who said, look, I'm, I'm increasing resources and I've got new budget.
- [06:59]
Greg Kilstrom
People said, no one ever. Right. Yeah.
- [07:04]
Ali Enriquez
So this is really, I think this. And then of course, we'll get into AI later, helping us do more with less. And that's a really, that's a tough spot to be in. So with synthetic, there are a couple of different ways that we're using and testing it. You can boost an audience. Right. It helps with accessing some of those harder to reach audiences where, you know, we might be staying in field longer to capture, you know, this very specific cohort or population. Well, synthetic can help with that. Right. So it gets us out of field faster budgetarily. Right. It, it's certainly more cost effective than a human response if we're comparing to that. And then when we're in a pinch. Right. It's certainly better than, than your gut. So testing a new concept with a simulated audience prevents risk. And so on the privacy aspect, that's also really appealing to clients. Oftentimes we're having, whether they're qualitative interview participants or survey respondents, we're asking them to sign an NDA before revealing a concept or an idea to them. So that's one kind of aspect of the privacy and confidentiality, you know, the other is you think about more regulated industries or you know, where privacy is much more predominant concern healthcare, you can, you can be testing and exploring concepts in a much more private and contained environment.
- [08:24]
Greg Kilstrom
Yeah. And so, you know, I know it's relatively early days still, but, you know, how do synthetic responses compare to traditional methods in terms of things like accuracy, applicability, you know, for, for a variety of tasks?
- [08:40]
Ali Enriquez
Oh, very well. So it, it depends on, on what we're looking at. I'll start with just out of the box LLMs there, there's certainly a bias for centrality and neutrality with, with any kind of, you know, GPT or, or, or, you know, open source LLM and that, that persists right. In this kind of validation work that we've seen. So it does very well when we're comparing just mean scores and certain, you know, basic validity measures. What we're challenging ourselves to do is we've got five or six really, really, really tough criteria that we're using as, as Qualtrics edge to determine if the model's ready for release. So we're looking at everything from hotbox score, mean deviation, we're looking at K means, so lots of different measures. And then, you know, our threshold for acceptance is really, really tight. And so the more we feed it, the more we learn. Where I think it's more challenged is niche audiences in harder to reach markets. I tell my team, please don't sell anything in Sri Lanka yet because we just, you know, we don't have confidence yet. It depends on the volume of data and the number of interactions that we're considering in the model. And so the more we feed it, the smarter it gets. And some, I guess a caution would be there are. So every single day my team is sending me a new provider and most of them are just wrappers on GPT. And so just be aware that that's great. You know, it's probably, if I had to put a number to it, 80% confident, you know, that's again, better than guts. But I really challenge folks to ask the hard questions about, you know, where exactly what are your sources, what else are you feeding it, if anything. Right. And then we'll start to see a lot more of these use cases publishing the graphs that we all want to see. Right. As researchers to feel good that we've looked under the hood and we know exactly how this is working.
- [10:38]
Greg Kilstrom
Yeah. And so, you know, along those lines, then there, I mean, it sounds like initial results very successful, certainly, certainly a lot to, to continue to do and you know, evaluate efficacy, all those kinds of things. But it Also sounds like as with any new technology there's going to be some hesitancy and maybe some skepticism and all that. And you know, rightfully so, like we shouldn't just charge headlong into this. Obviously you're doing the research to help us all out with this stuff, but you touched a little bit on things to look at as far as using the right tools but also you know, to build some confidence in these tools because they are working very well.
- [11:22]
Ali Enriquez
That's right. Yeah. To me it really boils down to published documentation on the validation and then at putting myself in any one of our customer or clients shoes, I want to hear from other clients. Right. I don't want to just take any provider at face value offering the opportunity to chat with and, or read, you know, a co published kind of again documentation and, or case studies. Right. Things like this would, would just really help boost, boost confidence. And I haven't seen many. Right. To be honest. And we haven't published many of our own. So I expect that in the next six months we'll be seeing a lot more of this. What is, what it's fantastic at and what to be careful with. And then we've come across plenty of organizations that called AI a four letter word. So you cannot include that in any of your, in any of your order forms or sows. Because we are, we're just not. Our organization's very, very, very weary and or it kicks off a totally different legal process. So it'll take time too for those who are really itching to, to get their hands on it for their organizations to come along. And so I've also heard a host of, of our clients developing their own in house models that way. It's, they're, they're testing the waters, they're doing so in a controlled environment and they're democratizing, you know, a little bit of this, this access. So that's pretty cool too. I don't, I don't see that as competition. Right. That's, it just helps build the case.
- [12:51]
Greg Kilstrom
Yeah, yeah, well. And you know, according to the report, it sounds like 89% of researchers are already using some kinds of AI tools. You know, maybe there's, there's specific things they're, they're using more than others there. But like what are, what are some of the most impactful ways that AI is already being integrated into research workflows?
- [13:13]
Ali Enriquez
Yeah, it's such an awesome stat. Right. I feel like this whole revolution came out of nowhere, but it's held flat year over year. So we asked the same question last year. And in a few ways I'm not surprised. So what we're calling it in the report is researchers have adopted AI as their newest team member. Right. So it's kind of pulled the seat. This is the task I need you to do for me. And you know, you also see in the report we ask about threat of this new technology and researchers are also saying that they're not concerned about their jobs at all. Job security is high and so how are they using it? It's a lot of the just again, publicly available AI tools to help with mundane menial tasks. You know, improving a headline, writing an exec summary, summarizing some of the data that we have and then of course doing the things that the researchers don't really want to be doing. Charting and stat testing and sifting through open ended answers and bringing forward feeding. So that's, that's really, we're seeing it more for operational efficiency right now than, and that's probably why we're not seeing job risk or threat. It is truly an extra pair of hands to help us do some of the things that we don't really enjoy spending our time doing.
- [14:34]
Greg Kilstrom
Well. Yeah. And also I think as consumers are increasingly omnichannel and the channels themselves are getting more complex, I think that adds another dimension to it as well. I mean, another thing. In addition to the synthetic research, digital qualitative tools are also becoming more capable as well as appealing to researchers. Things like eye tracking, intelligent video analysis. How do some of these methods replicate some of the more traditional qualitative research while you know, helping with, with some of the other things you've touched on?
- [15:11]
Ali Enriquez
Yeah, for sure. I, I saw a first wave of what I'll call AI powered tools. Some, some synthetic generation, but more for a lot of the things we were just talking about on summarization, segmentation, I saw that happen first with qualitative methods and, and I think that's only going to, to get bigger and stronger. It is incredibly expensive to recruit interview participants. Right. If you're looking at VPC level, you know, decision makers, you're talking $500. Right. And then the incentives. Right. The moderator's time. So I saw a wave of tools and solutions that were just fantastic at first, just helping you analyze and produce recommendations coming out of that type of qualitative work, but now to start to kind of blend and balance the human generated content and interview scripts with now synthetically generated. So I expect I will only continue because that type of dialogue, it's, that's what, that's what makes the insights pop, right? You want to hear from the voice of the customer, you want to hear from the prospect, you want to in their words, in their voice. So I don't think it will ever fully replace any of the things we're talking about. The human really is still at the center of all that we're doing. And we depend on that human data in both quantitative and qualitative methods to make the simulation even stronger, even better.
- [16:45]
Greg Kilstrom
And so, you know, we talked a bit about certainly budget cuts or budgets saying the same are pretty prevalent across, not just research across, you know, anywhere. And I deal mostly in marketing, so I'll just say in the marketing realm. Yet the 2025 research report says that some of those research teams that are identifying as like on the cutting edge of innovation are actually seeing some increased budgets and increased influence. What's the, what's the difference here? You know, what's, what's kind of setting those apart?
- [17:20]
Ali Enriquez
Yeah, that was another just awesome find. Right. And helpful because one of the first questions we get when we publish this type of work is what do I need to do differently? Right. How do I be more like them? And so some of this might feel, you know, as you'd expect. But when we look at the, those that are self identifying as being innovative on the cutting edge companies that are willing to try a new thing are also the ones that are earning more budget, as you said. And so they tend to also just have that political capital internally as well. So they are in such demand inside of their organizations. They are attached to some of the biggest, maybe even more confidential type of strategic research. They're not doing the just, you know, brand tracking and customer satisfaction studies. They're, they're there to help lead the charge and where are we going next? And they have a seat at that table. So I'd say kind of involvement in overall kind of company strategy. They are part of a dedicated market research organization. You often find research rolling into different parts of the business. But again that speaks to the companies. These organizations trust in these functions. And so peeling them kind of apart and creating an organization around them as well empowers them to do these things and test this new technology. It's really that and then more of what you'd expect, right. They are hands on in all the phases of the research. So they talk the talk and they walk the walk. So they're, they're very skilled in their discipline.
- [18:56]
Greg Kilstrom
And where would you say, you know, because those listening here, they're like, yeah, I Want more budget and I want to play with the toys. And obviously do do better research as well. Of course. But how does a company build this kind of culture of innovation? Is this, you know, is this something that, does it start at the top? Does it start at. On the team level? Is it. Probably, as with most things, it's a little of all. But like, how can meaningful change get made in that direction?
- [19:25]
Ali Enriquez
Yeah, it really does start at the ground level with a lot of the things that we just said. Start, you know, consulting. Leave that window open at all times. Right. Have it improve your headline, have it improve your email. Right, yeah. Ask for advice on, you know, how to, how to tackle a gnarly research question. And so, and like that, people start wondering, what's your secret weapon? Right. And then we find that that just kind of starts the. Yeah, starts the, the movement, if you will. And so.
- [19:55]
Greg Kilstrom
Yeah, yeah, yeah.
- [19:56]
Ali Enriquez
So some of, like, we've talked a lot about summarization, and then once you move on it, you know, automatic report generation and tagging data around different topics, whether quantitative or, you know, qualitative, you know, interview script type of stuff. And then what, what will happen next is that we'll start to have conversations with the data. Right. So the notion of conversational analytics will just help us do, do even more with the tools at our disposal.
- [20:27]
Greg Kilstrom
Yeah. Great. So one last stat from the research report. So, you know, 83% of organizations are planning to increase AI investment in 2025. That's probably not a shock to, to a lot of people out there because, you know, ideally it's strategic investments, but, you know, in looking at this, you know, 83% of organizations planning to do this, what areas of market research would you say are poised to benefit the most from this trend?
- [20:57]
Ali Enriquez
Yeah, we've talked about so many. Right. But from here I just touched on conversational analytics. But a more dynamic survey experience will be kind of our half step to this, this ability to just chat with my data. Right. And so I'll call it personalized survey experience, where we can just kind of tug on threads and dynamically route you without the researcher having to put much thought into that flow low in that padding. That's how it works right now, that, that will be beautiful because it improved the respondent experience and of course gets you more of the whys. Right. Because the survey takes you so far, if you didn't ask that next question, got to go back into field. So things like that and back to the kind of qualitative bundling, you know, pairing Human interview and, or focus group type of interaction with synthetically generated interview responses. I expect to see a lot more of that. You know, right now we're boosting base sizes with, with the synthetic response, so getting more data, but eventually we'll be focused a bit more on these quantities and everything. Right. But really quick, sharp quality answers to some of our tough research questions.
- [22:13]
Greg Kilstrom
Great, great. Yeah. And I, I guess, you know, not to ask too much of a leading question here, but to me, in my mind it's like we're marketers are awash in dashboards and charts and, and all those kinds of things. And you know, I think what you're touching on here is this idea of, you know, we spend so much time just getting to the point where we have these charts sitting in front of us and this is really, isn't this helping the researchers, the marketers, whoever, whoever's looking at this stuff or dealing with this actually do more what humans are good at doing?
- [22:49]
Ali Enriquez
Is that exactly right? I couldn't have said it better. I've been talking to my team a lot about the democratization of research right now. Guilty. Right. She gatekeep and feel like this is such a precious talent and discipline that it all has to come through us. But what I expect to start to see is, yeah, anybody can read a dashboard and interpret a chart. So putting this, this power into the user's hands, whoever they may be. Right. The, the brand manager, the operations manager, the product team to be able to interrogate their prospect, their customer, their user, you know, in a way that you don't have to follow all the bureaucracy and, and route things internally to different teams. That's, that's spot on. That's exactly it. And humans are, are the ones who really should be reasoning and deciding what action we should take, not charting and stat testing. So if you can have the machine do that for us. Yeah, we'll just be much better set up to take that next step, that action.
- [23:53]
Greg Kilstrom
Yeah, I love it. Well, thanks so much for joining today. One last question before we do wrap up. Something I asked to all my guests here. What do you do to stay agile in your role and how do you find a way to do it consistently?
- [24:09]
Ali Enriquez
That is such a great question. I told, I actually gave this advice to somebody here just the other day. Stay curious. Right. So I, I think seeking to understand and then doing whatever you need to kind of consume and, and, and reason with that. So reading a lot, you know, looking for diverse perspectives on topics just so that you understand all the different angles and doing things like this right? So just getting an opportunity to chat through and think about topics from different angles, different perspectives. I get the pleasure of doing that with such an awesome global team right daily. But then how do we think outside of our four walls? And what other voices and communities are we listening to? So lots of either written consumption or podcast. We've got very active Slack channels where we're sharing all sorts of content. You know, and it's a lot easier to listen if you haven't used Notebook LM to convert words into a podcast. It's awesome. You know, in your commutes or whatever, you can just be listening to what otherwise would have been a 50 page reports.
- [25:19]
Greg Kilstrom
Nice. Love it. Love it. Well again I'd like to thank Ali Enriquez, Global Director of EDGE at Qualtrics. To learn more about Ali Qualtrics and the Qualtrics 2025 market research trends report, you can follow the links in the show notes thanks again for listening to the Agile Brand brought to you by Tech Systems. If you enjoyed the show, please take a minute to subscribe and leave us a rating so that others can find the show more easily. You can access more episodes of the show at www.greggilstrom.com. that's G-R-E G K-I H L S T R O M.com While you're there, check out my series of best selling Agile brand guides covering a wide variety of marketing technology topics. Or you can search for Greg Kilstrom on Amazon. The Agile Brand is produced by Missing Link, a Latina owned, strategy driven, creatively fueled production co. Op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. Until next time, stay Agile.
- [26:25]
Ali Enriquez
The Agile Brand.