Loading summary

Transcript43 lines

[00:02]
Nathan Isaacs
Welcome back to Insights Unlocked. In this episode, Jason Giles sits down with cultural strategist and author Chewy Chewy Tan to explore why AI may be transforming customer research, but human judgment is still essential. Together, they unpack how cultural context shapes customer feedback, why common themes can hide very different motivations, and how teams can make smarter decisions by looking beyond AI generated summaries. Enjoy the show.
[00:32]
Podcast Narrator
Welcome to Insights Unlocked, an original podcast from User Testing where we bring you candid conversations and stories with the thinkers, doers and builders behind some of the most successful digital products and experiences in the world, from concept to execution.
[00:51]
Nathan Isaacs
Welcome to the Insights Unlocked podcast. I'm Nathan Isaacs, principal Content marketing manager at UserTesting, and joining us today as host is Jason Giles, User Testing's Vice President of Customer Intelligence, Customer insight, innovation and experience strategy. Hi there, Jason.
[01:08]
Jason Giles
Hello everyone. Good to see you again, Nathan.
[01:12]
Nathan Isaacs
And our guest today is Chuy Chewy Tan. Chuy Chewy is a cultural strategist and founder of Bayo Global, where she spent 17 years helping brands navigate cross cultural research across more than 50 markets. She's the author of Research for Global Growth and International User Research and she joins us today to talk about the cultural blind spots hiding inside AI powered customer insight. Welcome to the show, Chuy Chewy.
[01:38]
Chuy Chewy Tan
Thank you for having me.
[01:40]
Jason Giles
Chewy Tsuy. I'm so excited to have you on. We've been connected for a long time and we just, we've never got a chance to have this conversation. So we're like, let's, let's do it and let's record it. Okay, so before we get into all of this rich conversation around culture, can you just maybe take us back and give us a little context for how you got into this work in the first place? Was there like the, I don't know, particular moment or an experience that you had that really cemented this passion of yours?
[02:21]
Chuy Chewy Tan
Good questions. And it's like many things in life, I would say there's actually something I stumbled into rather than planned. So my background is human computer interaction. So I did a PhD in that and user research, of course. And in earlier in my earlier career, I actually had the opportunity to work with Marriott International Hotels, the hotels brand. And that was like 17 or 18 years ago. And it was one of the few companies at that time that recognized that there's a need to understand their customers in different countries, that they have different needs and they might need something different, we might need to design something different for them, which is quite impressive really, because at the time, like most people Just one design, anise it. So I had the opportunity to travel with them. I think the first country I went with them was Beijing. And I think probably because I was. I speak Mandarin, Mandarin is one of my mother tongue. So it's kind of like Cuichi, you can go and that is it. And that was great like because we were talking to Chinese and about how they go about booking hotels and so on. And then since then they invest a lot of money in understanding globally. So we will be going places like Qatar, Japan, Mexico and different places to not just talk to people but also observe how they are in the different hotels and in the lobby and how they use apps and mobile phones, mobile website and also desktop and so on. At that time it's more digital related because it's a digital teams I work with. And then after that I start to kind of write books about international research, talk about it in conferences, kind of like trying to own that space, space just purely because I was curious. And then I start working for myself. And so instead of focus much just on the digital side, but I was actually more about their global culturalizations and so on. So I don't look into just how people behave, but also look into the local culture first context. For example, I often think about the economy of a country, their infrastructures and a lot of other things sometimes even go back to their history because a lot of those elements could actually influence how we society behave because of colonizations or anything else. So those are actually very important because it's kind of influence what people value and also what whether a certain product fits into their life. So yeah, that's all the whole curiosity and everything kind of draw me into this work.
[05:11]
Jason Giles
I love that. And it's cleared by the way that you light up when you talk about it. The passion is still there. So I love you pursuing your curiosity and finding this kind of dimension perspective that has really been a great fit for you. So what prompted this conversation actually is you reached out to me and you were like, hey, there is a really important conversation that's really important to be having particularly right now. And so maybe can you talk a little bit about maybe what has AI fundamentally changed about customer insights?
[05:53]
Chuy Chewy Tan
Yeah, yeah, thanks for having me offering this platform for me to talk about it because I think it's very important topic to talk about since AI is kind of like influencing our lives so much. So when people ask me about what AI actually has changed, I think it changes the economics of customer insight. So a few years ago we all would agree one of the Bottlenecks of research is, was just simply making sense of all the data we get. Well, doing the research, but also making sense of the data. And researchers or teams might spend weeks to code the interviews or pulling everything together and summarizing and putting into a report. But today all of those could be done by AI in minutes. And they can analyze like hundreds or even thousands of customer input at scales in a speed that we never had in the past. So I do think AI actually do a lot of good things for the whole customer research industry. But I don't think the bottleneck has disappeared. The bottleneck that I was talking about, about spending a lot of time in and encoding the interviews and summarizing and so on, I just felt like the bottleneck has moved. So the bottleneck has used to be like synthesizing, right? But now I think the bottleneck is moved to the interpretations. Because we are very good now in producing summaries because AI could let us have good summaries in no time. But the questions now, or the harder questions now is are we the researcher or even the business people or product teams, are we interpreting the summaries correctly and accurately? What is interesting is this is not just me making it up on my observations, because there are a lot of recent studies and one of them is by, I think it's by Utrecht University and Western University as well. They compared thousands of AI generated summaries with the original research papers and they found that LLMs large language models were almost five times more likely than human written summaries to overgeneralizing the conclusions. So I'm not saying that AI is unreliable, right? Like it's much more reliable because we talk about hallucination in the past, but not as much anymore because things have improved. But to me, the problem is a beautifully written summary by AI. It doesn't mean it's a complete understanding or clear understanding, because if we think about it, if we only analyze one market, this itself already is something that we need to think about. But if you're analyzing feedback from six countries or 10 countries where people might use the same words for, for something, but it might be completely for different reasons and that become the challenges become multifold or become much bigger. So I don't think we should say that AI can AI actually provide customer feedback, because it clearly can. But it's more about how do we how to make sure the meaning survive the journey of customer input or customer voices into one strategic decisions that businesses have to do. So that's where researchers can create a lot of value.
[09:34]
Jason Giles
Yeah, for sure. And that, that moving of the bottleneck, it manifests itself. I see it firsthand with, with our research team. Right. Because there's. You still want to have the rigor one. And it takes time to ensure that like all the nuance that, like you say that over generalization that you, that you could miss, that still takes time. You know that it's just, it just is a different type of work.
[10:02]
Chuy Chewy Tan
And you can remove that completely because that you remove that you actually take like a very risky.
[10:10]
Jason Giles
That's exactly the risk. Right. Businesses making these decisions on a lot of data that may not have the rigor. And so that's the opportunity for research for now, certainly right now. Can you maybe give us. Your specialty is all around the differences between various markets. Could you talk us through an example of where, you know, maybe some of the terms that get used in, you know, particularly like things like, like trust and convenience, where they might actually being used that have a different underlying motivation or meaning something tangible that, you know, our audience could really understand.
[11:11]
Chuy Chewy Tan
Yeah, I'm sure. So in global research, I do think that we talk about differences a lot, right. Differences between countries. But I think similarities are often the beginning of analysis, not the end. So for example, I can give an example. You talk about trust or convenience. I'm going to use trust as an example. Trust is probably one of the most common themes that I ever heard about us here in user research in the last 17 or 18 years. Right. Because people, we often will hear people saying like you do interviews and in different countries and the AI might come back saying that trust is one of the biggest barriers of people not signing up on your subscriptions or not buying your products across all six markets. So now that is generally useful, right? It's a useful and valuable insight. And AI is doing exactly what we asked them to do. Right. It found a recurring pattern across a huge amount of customer input or customer feedback. But then to me, the more important question is what does trust actually mean in each market? Because in one market, trust might mean I have never heard about this brand, so it's awareness issues. Right. In another, it might mean I'm not confident putting my credit card details into your app or your website or some other countries might mean I'm worried that it's not easy or I forget to unsubscribe. So it's about that. Or it could be privacy or scams or so on. So in that instance, all of this is actually trust issues, but they are completely different business problem. And because there are different business problems, they need different business decisions. So for example, when I talk about the brand awareness earlier, if it is a brand awareness issue, since one business, then you might want to increase the awareness by improving your credibility of proof. But if it is like payment issues, then you might want to improve your payment methods or reassurance. If it's a subscription, then you might want to make cancellation easier and so on. So it's completely different even though it comes out as one theme, but it might require very different actions on that. So to me, a theme that AI might come up with is not the same as insight. Because a theme tell us what customers appear to have in common, but insight actually explain why they are saying it and what decisions we our business should do. In other words, I would say it's like what is driving certain behavior in certain markets. So I think that is very important in the AI world that we are living. Because AI help us find patterns and then now it's our job to understand what those patterns actually mean before we act. Because the same thing doesn't necessarily point to the same business problem, hence the solutions.
[14:26]
Jason Giles
Right. There's an example that you gave on your LinkedIn post around a sentiment that findings were well, it's too expensive. Right. And that's a very like. Okay, well similar to trust, right? Okay. This is a finding like trust is important. Well, hey, we're seeing a barrier to purchase and it's around price. Can you talk a little bit more about like what the story was behind that post?
[14:55]
Chuy Chewy Tan
Sure. That process actually comes from something that I encountered when I was working with Spotify for many different markets. It's not just one market, but throughout the years we go to different markets for different reasons and so on. But this expensiveness keep constantly come up one way or another. So we keep hearing participants or users from different countries say oh, it's too expensive. That's why I'm not subscribing, that's why I'm not doing this. And that's. And the natural reaction was to think, okay, this is a pricing issue, right? Like is affordability and so on. But throughout the whole my experience I know like it doesn't. It's not as simple as that is. So as the interviews went on, we sometimes will hear people will say the same people will say it's expensive. And then they will say, oh, I actually spend a lot of money. Like in Colombia they spend a lot of money on concerts, like lots a lot more than the subscriptions. But in concerts on things that will bring memory or have interactions with the local artists. Or in South Africa, they will spend a lot of money on data because that is their way to live because they have WhatsApp, they have businesses and so on. They need to have that continually. And Kenya, they will say that it's expensive, but then they will say that oh, I spent so much money for a party for my family because socially it's a collective society and family is really important and so on. So, so I would just say hang on, it's not about money. Then even you say it's expensive. So the more we look into it, the more we kind of say the real question we need to ask ourselves is expensive compared to what? Because sometimes it's honestly is generally affordability. But others might be just because your service was not worth me paying yet, or sometimes about the recurring subscriptions doesn't meet into their economic setting. Like oh, I get paid every day and it's up and down, so I can't have monthly subscriptions on. So it's the same sentence, right? It's extensive but then completely different reasons. So once you understand the research, you will end up actually have completely different business decisions to make and so on. So for example, you might want to improve your value propositions instead of lowering the price, or you want to redesign your onboarding to make it easier, or you might want to focus on the trust and so on. So you have to kind of look into that. What worries me sometimes with AI is that it's actually brilliant. Showing me that a lot of people, like thousands of people are saying the same thing. But it's up to us as a researcher, to products, teams or whoever to say, to start asking, are they actually talking about the same problem? Because we also, we have to remember customers are very good in describing their experience to such an extent, but they are not always very good in describing the underlying problem. Not because they do not want to tell you, but it's because sometimes they don't even know, right? Like what is the underlying that lead them to certain behavior or certain mentality. So when customers talk about trust or convenience or privacy or value, so we often will ask the questions that what is really driving them to say that? Because that is very important way to kind of guide us to the, the, the, the ultimate underlying insights that we are looking for.
[18:39]
Jason Giles
That is so true. I know I've had the pleasure for many years, you know, watching researchers and I'll do my own, you know, like I, I, I, I dabble and it's what, what I really find is that, so when I have a conversation, you know, maybe it's a moderate session, I, and I'll distill out and I'll get some findings and then I'll watch a researcher, a professional researcher basically do very ask similar questions. But then they like drill in and they really start teasing out. And when you look at like the findings that come out, mine are like, okay, yeah, you know, like they're good. But the insights that I'm getting from professional researchers always have just that extra little bit of like, okay, here's the answer to question, but here's the interesting angle that kind of turned up as part of this work. And interestingly, if I think about it sometimes where I'll be working with an AI, you know, and maybe we're doing some synthesis of a bunch of reports or data that I've got, I get that very surface level feeling from the AI as well, versus sitting down with a subject matter expert. And really they bring it to life in ways that I don't quite get from some of the technology right now. So I think it's really good.
[20:16]
Chuy Chewy Tan
We could say not yet, but definitely not at the moment. We don't know what it's going to be in the future, but at this moment, like, yeah, it's sometimes lack of the depth. Right, like we were looking for.
[20:31]
Jason Giles
No, totally. So I've had the opportunity to relocate a couple times now. You know, I spent a few years in the UK and in Spain. And I would say one of the things that, the best part about that experience was that it was so humbling because I thought that like everybody, you know, I didn't realize the biases that I had, you know, whether it was like the employee experience and my attitude towards work and it's been incredibly. Yeah, I guess I use the word humbling. I'm like, oh, wow. And it really has kind of driven an appreciation for understanding the different cultural implications of, of so many things that we can take for granted. And, and so I love this idea that there is more awareness and there's more specialists like yourself that are really helping companies understand how important those differences are and how it can be meaningful to their business. At the same time, folks that I've spoke to are like, well, you know, we've got AI now and this is great because, you know, we can, we can just get translation now. And I, I would like your take on why maybe understanding a market is so much more than just translation of words.
[22:13]
Chuy Chewy Tan
Yeah, I was trying to answer that question as using a personal story because I Think it will explain the problem a bit more. I actually posted that recently on LinkedIn as well. So my mom has this incredible skill to make a dish, a yum dish, right? So she once made it for for my sister's partner's parents. And afterward my sister said oh yeah, the dad have hinted that he will let mom to do more so he can have more because he enjoyed it so much. I just like I remember when I was home early this year while I'm from Malaysia and I said really? How did he hint it right? And she said well, he keeps saying that how delicious that is and how mom is so good at making it. I just like that is it. That is the hint that you are saying that is being so obvious because to me that sounds like just someone being very polite and say thank you. In the preachy culture, like even if something they don't want anymore, they never eat it. They will say oh that's really nice, you're really good at making it. And my sister said no, that is so obvious, it's a hint. So I kind of like, okay, we both are hearing the same thing, we didn't misunderstood things, but we are interpreting or we are responding to the very different culture rules about politeness, about gratitude or the whole indirectness. The thing is I grew up in that culture in Malaysia and I spent half of my first part of my life there and then the others half here in the uk. And so somehow I'm quite British and I'm quite Asian a long way. But somehow along the way my own wiring have shifted enough that I missed the signal completely in Asia. So that's why I kind of made me think like when researcher or businesses or teams treat AI research outputs as the full versions of customer reality, especially across different markets with different culture. And what would that mean then? Because even me as a human, I miss that international research that will be harder as well. So I remember I posted something about this similar things a few months ago and I heard a lot of AI moderator platforms that we have a lot now saying like oh, there's no problem, we have already done like hundreds of research in 10 countries recently or how many countries they were talking to. It's great that in any languages you want and you can do it while you're sleeping and so on because there's no tangible voices. But I think personally, I think translation is actually the easiest part of all for me is the harder question is like what did the cosmosis really mean? Right? And what's interesting is not just my experience as well. Because there are decades of cultural psychology research from like, I think Richard Nisbeck is one of them, is they have shown that people from different. We all know that as well. People from different cultures don't simply communicate or express them the same idea or express themselves the same way. And they will perceive situations differently and make different assumptions or make certain observations or conclusions very differently. So I see that in interviews as well. You probably have seen this as well yourself. If you say, if you have, if you. When you're doing the interview, because you might hear someone say, oh, that's interesting. So if you, if you look at. You know where I'm going, right? If you look at. Only look at transcript or ask AI to summarize, it will probably come as mildly positive, right? Interesting, positive words. But if you're sitting in interviews, you might notice something that they don't ask another question because they are not interested enough or their tone is flat or they hesitated or they move. So you might start thinking, actually, I don't think interesting is actually what they mean by interested. And a lot of AI moderator platforms now say they have emotional intelligence analysis. I'm yet to try it. Quite a few of them think, oh, we have this. So I can't comment on the accuracy and efficiency here, but that is emotional as one aspect. There's another aspect to us is interesting according to whom, right? In Britain, I use interesting a lot. It's a polite way to say not sure is actually, I don't like that much. And it's like, oh, it's a bit weird. But we don't want to sound confrontation. But in other cultures, the same word might generally means like, I'm interested or I'm curious. Tell me more. So the transcript is still the same, right? The translations hasn't changed. But what changed is a cultural context that give us the different meanings that those words carry. So AI is the same. They can translate beautifully, they can summarize beautifully. But it wouldn't automatically know if someone from different culture would most likely to hear what those words. So I wouldn't, I wouldn't say, like, ask yourself a question like, oh, what did they just say? Because that is not a valid question. When you read transcript, you probably want to think, what was someone from the same culture most likely to understand them in terms of what they meant? Because that is more important to get the right decisions, to get to the right to get the right insight, to get the right decisions to make.
[28:03]
Jason Giles
Yeah, I mean, that becomes my very next question. Right? Like with all of These like, like you don't know the assumptions that you're making. When I, when I first started working in the UK and someone was like, yeah, Jason, I really appreciated your perspective. It was very interesting. And I was thinking it was all great. And then I get the elbow in the side from my locals. They're like, yeah, that's not good.
[28:29]
Podcast Narrator
No.
[28:29]
Jason Giles
But I would never have known if it hadn't been for those who are from. From the culture of like, yeah, almost doing like that. The translation in real time. So for these teams then, that are taking their products into different markets or branding, whatever it is, where do you advise that they actually start? Like, how do you kind of crack this code and really have that understanding of what needs to be unique for the markets?
[29:06]
Chuy Chewy Tan
Yeah, I think is that a lot of ways that we can do that. Right. Because AI is good in finding cartoons as we talk about. Right. So we will tell us and what people say a thing that what people have in common. So. And it's good on that. So we should continue using that. But it doesn't tell us the why. Always doesn't always tell us the why. And the why is more important when we do the business decisions. So the way I really think about is really simple. So first of all, just try not to straight away jump from a theme to an action. So instead you cannot think about and stop in the middle and ask what is most likely driving this behavior in this market, like we talk about, because once we understand the driver, the decision will become clearer. So I started to kind of create a framework that I'm developing. It's called the Global Fit Loop. So it's very. At a very high level. It's answer three questions is that what is the signal? Why is driving it the signal in that market? And what decisions should change because of that. So AI can help us and very good in helping us on the first questions. The second question, what is driving in this market? Is where local context really matters. Because the local context is sometimes the culture or sometimes the regulations or sometimes the maturity in that specific category or industry or social norms, something. So the point is to assume the explanation. The point is not to assume the explanation before you have tested it. So that second one signal first, driving, why is the drivers second. And then the third one is where the decision should make should be the product marketing or growth team should work together to create that. So I'm not saying like choosing AI researchers. I do think like we should work together. The things that teams should do is not to skip the middle One about the drivers. Because that is the one step where you can move from pattern recognitions to customer understanding. And that's usually where you actually can making a local optimizations better and or adaptations better and make the right decision. So I would say follow this simple framework like signal driver and decision and
[31:41]
Jason Giles
what can you walk me through just like an example of what that conversation might look like where you stop and you're like okay, here's what we've seen, here's the signal. Now going through kind of your framework how that might play out.
[31:57]
Chuy Chewy Tan
Yeah, I'm trying to use the same one that we talked about earlier. You know, the trust things or the expensive things. Right, it's expensive. But then we needed to understand what actually drive that viewpoints of it expensive. It's kind of like either further asking that questions or actually looking into the transcript to say I'm going to stop and then go back to go back to my transcript and actually read back. Is there anything that is actually missing? So you know the point that we were talking about in Colombia, people say it's expensive, expensive. And the bottle is actually down to the quite to the end of the interviews that we're talking about other things, other areas because before going into Colombia I already look into one of the framework I created which is called cultural dimensions at 27 of them where it's possible related to Colombia and related to Spotify for example in this case. So I kind of like oh, there's elements of the proudness of connections with local artists or local. Local influences or subs and similar. And also the joy of having experience, physical experience. So we went in and we actually prompt a few more questions at the end and it's until at that point and they will start talking. Yeah, I spend a lot of money on concerts and I spend a lot of money on this and festival and things like that. So those actually didn't pick up by AI at all because it wasn't grouped as you know, it wasn't. The insight is the second part of the insight wasn't being picked up during the question we are asking why are you not subscribing? Or why is file value? So I didn't pick it up but we went back to look at the insights, the transcript and the notes that I normally take quite verbal Tim in and then we realized and then make a connection and that is ultimately one of the biggest insights that we ever get in that research. So yeah, sometimes it is kind of like the framework is like why is driving it. And then we went back and have a look. And then that kind of created a decision. It's not about pricing, it's not about that, but it's actually creating these kind of connections which change their propositions. In Colombia, they have a spot house, spot, Casa, Spotify. Is that something that they actually could bring local artists in and kind of interact with them for Colombia?
[34:39]
Jason Giles
What I love about that example too, is that we talk about all of our teams are driving for speed right now. Company is all about speed, speed, speed, speed. And yet, as design, research, even product leaders, we know that there's some places where it is important to slow down. And the example that you give is, that is such a great example of, look, we've got these signals, but this is the spot where maybe it is okay to just pause for a minute and really drill in. Particularly because the example you used was like, this was a big business decision. Right. So I think what I really like about this is, you know, being able to help encourage teams to identify those spots where slowing down is really important.
[35:36]
Chuy Chewy Tan
Yeah, the temptation is high, isn't it? Just like, everything is quick now in our world, AI helps it quick. So we need to show that everything is good can be run very quickly. But like you say, it's really important to just stop and say, hang on. Well, not only just question, is AI creating the right input or the right insight, but actually anything is missing that is important that the dots is not being connected. Well, I think that's really important. And unfortunately, because of the mentality of everything go fast and AI is so polished. The other thing is, I'm worried is, like, all the AI output is so polished, it sounds so perfect, and it's just so easy to say, oh, yeah, this is gold. This is gold value that we're getting.
[36:26]
Jason Giles
Yeah, well. And, you know, I was having a conversation with some other leaders recently, and one of the observations I've been having is the tools themselves make it so easy just to jump to the next step. They're like, okay, hey, we've done this thing for you. Do you want me to do this? Do you want me to do this?
[36:44]
Chuy Chewy Tan
Next?
[36:44]
Jason Giles
Do you want me to do this? And it's so easy just to be like, sure, yes, yes. And before you know it, you've gone so far down and you haven't.
[36:56]
Podcast Narrator
You.
[36:56]
Jason Giles
You haven't stopped to, like, check your assumptions and like, really, you've kind of slipstream past the important places where it makes sense. And. And the conversation that I think is important as leaders is where is that purposeful friction that we are applying to our teams in our workflows to ensure that just hitting the next button basically doesn't allow our teams to race past.
[37:30]
Chuy Chewy Tan
You're right about leaders, right? A lot of managers are C level. It's just like we pay for this platform, so you should get everything very quickly. And so they kind of encourage the speech to say, well I'm paying for this already. Like we're talking about either Claude or ChatGPT, Seamless LLMs or the AI moderation tools that now do the moderations and creating analysis and so on. So I heard a lot of C level is just like, well I'm paying for this using my budget so it should turn around quicker. I want days, not weeks. I want two days instead of seven days or few months. I was just like two days is not going to happen even if you use AI completely. So everything we talk about so far is actually applicable for both kind of AI, the LLMs generic and also the AI moderation tools as well.
[38:25]
Jason Giles
Yeah. Because speed is such a at a premium these days. Is there a simple thing that you encourage teams to do to build into their workflow before acting on maybe an insight that they've generated from AI?
[38:45]
Chuy Chewy Tan
I think one thing is always be very careful of the transcript you have. So. So there are two things, right? If you actually still lucky enough to be able to do in person interviews and then use AI to do the analysis. So you only have one part of analysis part that you have to be careful of. Or are we talking about even using AI to moderate? So that's one part that could miss an opportunity to dig deeper, right. To ask more questions, to observe, understand the culture. Say oh yeah, I know this culture is a bit like that, like local culture or local context. And I will prom it in a slightly different way when I do research in this country and that country. So if you are still doing the in person moderations, you could, you could reduce one layer of the possibility of risking missing information. But that allowed a lot of teams now is relying on AI one way or another to do analysis as well. So I think it's really important to kind of. I think I will keep saying like pause, not keep going like very easily. Just forgot and just rely everything on that. So always ask the same questions about the drivers and is that what. Don't take everything in the face value. If something looks too, too easy and too clean and too straightforward, then question it and go back to ask about the questions and to kind of look into that as well. I know sometimes like More managers will say like, I don't want to spend. This is going to slow down my team if I do this and that. But I don't think it actually needs to be a big process. But it's just stop and pause and ask some questions. Like for example, before we just take oh, what customers are telling us as insights, you just stop and say, okay, what are we assuming they mean? So you kind of challenge a bit that will make the team will go back and say, oh, let me check more or let me go into it and check more and so on. And it will tell a very different story. And once you have done that, maybe ask another question saying, what evidence make us confident that this is the right interpretations for this culture. If you need to, you might want to get some local experts to kind of like run through it or soundboarding and so on. So for me, it's not like stopping and do something differently, but I stop and ask a bit more question to challenge yourself, your team, but also challenge AI in some way. So yeah, I think that is really important to say, do we have enough evidence? If it is, yes, great. If it's not, we're not sure yet. That's okay. I always say I would much rather you spend the teams I'm coaching or working with. I would rather spending another 10 minutes or half an hour to challenge and interpretations than spending six months to build something that is wrong. So it's not about slowing things down, but it's just helping to move the. Move things in the right way faster.
[42:10]
Jason Giles
I would say 100%. And what I love too about your guidance is that it one, it's protecting the team's critical thinking. Right. Which I'm very conscious of as we use these tools. And the second thing, and you touched on this too, is the technology is changing so fast. And so by it's allowing you to see what is the current state like where. Oh, we're seeing that actually we don't see these hallucinations anymore or it's actually getting better than this. So I love your advice because there's multiple aspects to why that can be really powerful. We have been talking for a bit because I like to ask this question to any specialists and it's kind of just off the cuff. But what are some questions that you are getting? You know, you're a public speaker, you've written books. What are some questions that you like? Maybe a top question that you get from folks these days as it relates to global research or what you're seeing kind of emerge in this time I
[43:30]
Chuy Chewy Tan
used to have this question. Unfortunately I don't have the answer because this question is, is there any resources or anywhere that you can point me so I can understand different cultures? I just like where do we even start? Like are we talking about one culture or the other culture? Like one country or the other? We have like 196 countries nation or 192, I can't remember. But then within each country you have so many tribes and NCT and things like that. So it's impossible. There's one resources that point you to like just learn this or read this, you will be fine. Because even within Japan for example, like if you are in one industry than the others, the cultural aspect that you look into will be very different. Right? I used to go to Japan the same trip for Spotify and then for Asana, the work management tool. And even though it's in Japan, the same culture, we're doing different research for two different clients. But the things that we look into, the cultural elements we look into are completely different. One is about hierarchical, about like consensus and things like that. Nothing to do with Spotify in terms of their. What people care about. And it's, it's about identity, about the music, about having something physical so I can keep it and you know, I mean it's so, so it's very, it's. It's a huge in. It's a huge area on the main. But it's also making it very interesting. So I would say like keep being curious is whether with AI or without AI is always being curious, right? Like not making, keep asking the assumptions that you bring into to countries, even when you design a product, you always will have assumptions about that product, about maybe this is how people is going to buy something. This is how people is going to interact with my products. So just be curious and know that it's just assumptions and you want to go in and kind of find out, validate that as well. And then the curiosity in the AI world is another layer. It's like pausing. Like we talk about pausing, asking what might be missing. Because unfortunately there's no magic wand to say here you go, learn all about countries. You can just put it saying that I actually developing AI platforms that trying to. Which could actually hopefully be able to say okay, it's this country and this industry and domains you're in, you're in a marketing team of growth teams. You put it in and the system will actually look into the cultural aspect that I look into. Like using my 18 years of experience and so on. So hopefully that could be as close as a magic wand. But yeah, we will see where we get to and I'll announce that if I when I get that it's ready to test.
[46:33]
Jason Giles
Perfect. Perfect. And to that point, for those who would like to learn more about you and understand more about this entire big world, where would you recommend that the they connect with you or learn more?
[46:51]
Chuy Chewy Tan
LinkedIn is where I'm most active with. So if you hear about this podcast and want to connect, then just connect and mention that you hear from Insights Unlock and podcast and then happy to connect or comment in the comments on my LinkedIn. So yeah, and then I have my website, Bayo Global. I have a YouTube channel but I need to be more proactive in posting something. I go through phases as you know. Yeah, yeah. So yeah, those are places. But LinkedIn would be where you'll find me most amazing.
[47:26]
Jason Giles
And I did spend some time on your YouTube channel today and there's some really good stuff on there so I encourage folks to check it out. Chuy Chui, it is so nice to have spent some time with you. Thank you for reaching out. This has been a really fascinating conversation and so timely and yeah, just want to thank you for sharing your perspective with us today.
[47:50]
Chuy Chewy Tan
No, thank you. Thanks for having me.
[47:53]
Podcast Narrator
Want to keep the conversation going? You can find the show notes@usertesting.com podcast if you haven't already. Don't forget to follow us on Apple Podcasts, Spotify, Overcast or Google Play so you never miss an episode. And if you enjoyed today, please share it with a friend or leave us a rating and review on Apple Podcasts. And until next time, this is Insights Unlocked, an original podcast from User Testing.

Summary7 min read

Insights Unlocked Podcast – Episode Summary

Episode: How to avoid cultural blind spots in AI-powered customer research

Date: August 3, 2026
Host: Jason Giles (VP of Customer Intelligence, UserTesting)
Guest: Chuy Chewy Tan (Cultural Strategist & Founder, Bayo Global)
Producer: Nathan Isaacs

Overview: Navigating Culture in an AI World

In this insightful episode, Jason Giles sits down with cultural strategist and author Chuy Chewy Tan to explore the evolving landscape of AI-powered customer research. The conversation centers on how AI is revolutionizing data collection and synthesis, but emphasizes the crucial role that human judgment—and cultural awareness—plays in delivering meaningful, actionable insights.
The episode demystifies "cultural blind spots," revealing how themes that seem universal can mask vastly different motivations across markets, and guides teams on how to make smarter decisions when leveraging AI in global customer research.

Key Discussion Points and Insights

1. Chuy Chewy Tan’s Path to Cultural Research (02:21 – 05:11)

Background: Chewy describes her accidental journey into cultural research, starting from a PhD in human-computer interaction, leading to working with Marriott International on cross-country research—a rarity at the time.
- Quote: “It was one of the few companies at that time that recognized that there's a need to understand their customers in different countries…we might need to design something different for them.” (02:44 – Chuy Chewy Tan)
Key Insight: Cultural context goes far deeper than surface preferences; it is shaped by economy, infrastructure, and even history.
Personal Approach: Chewy was drawn to exploring not just behavior, but the underlying cultural conditions that shape perception and adoption of products.

2. How AI is Transforming (and Shifting Bottlenecks in) Customer Research (05:52 – 09:33)

AI’s Impact: The economics of insight generation have changed; AI can now process and summarize massive amounts of customer input in minutes.
- Quote: “Today all of those could be done by AI in minutes. And they can analyze like hundreds or even thousands of customer input at scales in a speed that we never had in the past.” (06:32 – Chuy Chewy Tan)
New Bottleneck: The challenge has shifted from data synthesis to interpretation—are teams interpreting AI-generated summaries accurately?
- Research Highlight: Large language models (LLMs) are five times more likely than humans to overgeneralize from data (07:50).
AI’s Limitations: “A beautifully written summary by AI—it doesn't mean it's a complete understanding or clear understanding.” (08:42 – Chuy Chewy Tan)
Actionable Thought: As research expands to more markets, identical themes (like “trust”) may mask different realities and require deeper interpretation.

3. The Danger of Overgeneralization: Trust Isn’t Universal (11:10 – 14:26)

Theme vs. Insight: AI often highlights common themes (e.g., trust), but the meaning—and business implication—of “trust” varies vastly by market.
- Example: In one country, “trust” might be about brand awareness; in another, payment concerns or fear of difficulty unsubscribing.
- Quote: “A theme that AI might come up with is not the same as insight…A theme tells us what customers appear to have in common, but insight explains why.” (13:47 – Chuy Chewy Tan)
Business Risk: Treating shared themes as universal can lead to one-size-fits-all solutions that miss deeper, market-specific drivers.

4. The “Expensive” Example: What’s Expensive to Whom? (14:26 – 18:39)

Spotify Case Study: Users in multiple countries reported Spotify was “too expensive.” Deeper probing uncovered that—
- In Colombia, high concert spend is valued over subscriptions due to live experience.
- In South Africa, high data costs are prioritized for essential connectivity.
- In Kenya, money is spent on family gatherings, reflecting collective values.
- Insight: “The real question we need to ask ourselves is: expensive compared to what?” (15:44 – Chuy Chewy Tan)
Takeaway: Identical feedback can stem from completely different motivations—understanding context is crucial.

5. Why Translation Isn’t Enough (22:13 – 28:02)

Personal Anecdote: Chewy describes misinterpreting a “hint” in family conversation—a common dish praised—due to missing subtle, culturally-coded cues.
- Quote: “We both are hearing the same thing…but we are interpreting or responding to very different culture rules about politeness.” (23:09 – Chuy Chewy Tan)
Translation ≠ Comprehension: Decades of research confirm people not only express ideas differently across cultures, but interpret situations through unique lenses, even when speaking the “same” language.
- Example: The word “interesting” means “I’m not impressed” in the UK, but “I want to know more” elsewhere.
AI Limitation: AI can translate and summarize, but may miss nuance and true intent without understanding cultural subtext.

6. Practical Guidance: Building Culturally-Smart, AI-Enabled Workflows (29:06 – 42:10)

Don’t Jump from Theme to Action: Pause between themes and decisions to ask, “What’s most likely driving this behavior in this market?”
The Global Fit Loop Framework:
1. Signal: What is the recurring feedback or theme? (e.g., "expensive")
2. Driver: What’s driving this sentiment in this market? (context/cultural reasoning)
3. Decision: What business action should be taken in response?
- Quote: “AI can help us and very good in helping us on the first question…The second question…is where local context really matters.” (30:27 – Chuy Chewy Tan)
Case Example Continued: Only after returning to transcripts did the Spotify team realize the true driver in Colombia was not cost itself, but value tied to local music events (32:00 – 34:10).
The Value of Slowing Down: Leaders must encourage teams to push for “purposeful friction,” slowing down in critical moments to interrogate assumptions (34:39 – 36:55).

7. Leadership, Speed, and Critical Thinking (36:55 – 42:10)

Temptations of Speed: C-level leaders may pressure teams to deliver insights at AI speed, but core, contextual understanding can’t be rushed.
Recommendations for Teams:
- Always question the assumptions behind AI-derived insights.
- Pause and drill deeper, even for 10–30 minutes, to challenge the first interpretation.
- Consider seeking feedback from local experts when needed.
- “I would much rather you spend…another 10 minutes or half an hour to challenge an interpretation than spending six months to build something that is wrong.” (41:05 – Chuy Chewy Tan)
Emphasis: Protect critical thinking as a team value—don’t let AI replace it.

8. Curiosity: The Cultural Researcher’s Superpower (43:29 – 46:32)

No Magic Bullet: There’s no single resource to learn all cultures; even within a single country, context matters by tribe, industry, and domain.
- Example: Doing research for Spotify and Asana in Japan revealed entirely different relevant cultural dimensions.
Final Advice: “Keep being curious,” always question assumptions, and pause to ensure you’re not operating on false equivalence.

Notable Quotes & Memorable Moments

On the risk of polished AI outputs:
“All the AI output is so polished, it sounds so perfect, and it's just so easy to say, ‘Oh yeah, this is gold.’”
– Chuy Chewy Tan (36:00)
On surface-level insights:
“What I really like about this is being able to help encourage teams to identify those spots where slowing down is really important.”
– Jason Giles (34:39)
On human vs. AI insight depth:
“I'll watch a researcher…they drill in, and…when you look at the findings…the insights…have just that extra little bit…that I don’t quite get from some of the technology right now.”
– Jason Giles (18:39)
On critical thinking:
“It's not about slowing things down, but it's just helping to move things in the right way faster.”
– Chuy Chewy Tan (41:48)

Key Timestamps

[02:21 – 05:11] Chewy’s origin story: from academia to cross-cultural research
[05:52 – 09:33] The real shift AI brings: from synthesis to interpretation
[11:10 – 14:26] Trust as a case study: why themes can't be taken at face value
[14:26 – 18:39] “Expensive” for Spotify: different drivers in different markets
[22:13 – 28:02] Translation vs. cultural comprehension: personal anecdotes and research findings
[29:06 – 34:10] The “Global Fit Loop” and practical steps to avoid cultural blind spots
[34:39 – 36:55] The business value of pausing and purposeful friction
[41:05 – 41:48] Invest a little time in challenging interpretations—save months of potential misdirection
[43:29 – 46:32] Embracing curiosity: there’s no one-stop resource for understanding culture

Resources & How to Connect

Chewy Chewy Tan’s LinkedIn: Most active platform—listeners invited to connect
Bayo Global website and Chewy’s YouTube Channel: Additional resources and videos on global research

This episode is a must-listen for anyone navigating global customer insight, especially as AI transforms how teams gather and act on data. The message is clear: AI can amplify your reach, but only human insight can bridge the cultural gaps that matter most.

Loading summary

Transcript43 lines

[00:02]
Nathan Isaacs
Welcome back to Insights Unlocked. In this episode, Jason Giles sits down with cultural strategist and author Chewy Chewy Tan to explore why AI may be transforming customer research, but human judgment is still essential. Together, they unpack how cultural context shapes customer feedback, why common themes can hide very different motivations, and how teams can make smarter decisions by looking beyond AI generated summaries. Enjoy the show.
[00:32]
Podcast Narrator
Welcome to Insights Unlocked, an original podcast from User Testing where we bring you candid conversations and stories with the thinkers, doers and builders behind some of the most successful digital products and experiences in the world, from concept to execution.
[00:51]
Nathan Isaacs
Welcome to the Insights Unlocked podcast. I'm Nathan Isaacs, principal Content marketing manager at UserTesting, and joining us today as host is Jason Giles, User Testing's Vice President of Customer Intelligence, Customer insight, innovation and experience strategy. Hi there, Jason.
[01:08]
Jason Giles
Hello everyone. Good to see you again, Nathan.
[01:12]
Nathan Isaacs
And our guest today is Chuy Chewy Tan. Chuy Chewy is a cultural strategist and founder of Bayo Global, where she spent 17 years helping brands navigate cross cultural research across more than 50 markets. She's the author of Research for Global Growth and International User Research and she joins us today to talk about the cultural blind spots hiding inside AI powered customer insight. Welcome to the show, Chuy Chewy.
[01:38]
Chuy Chewy Tan
Thank you for having me.
[01:40]
Jason Giles
Chewy Tsuy. I'm so excited to have you on. We've been connected for a long time and we just, we've never got a chance to have this conversation. So we're like, let's, let's do it and let's record it. Okay, so before we get into all of this rich conversation around culture, can you just maybe take us back and give us a little context for how you got into this work in the first place? Was there like the, I don't know, particular moment or an experience that you had that really cemented this passion of yours?
[02:21]
Chuy Chewy Tan
Good questions. And it's like many things in life, I would say there's actually something I stumbled into rather than planned. So my background is human computer interaction. So I did a PhD in that and user research, of course. And in earlier in my earlier career, I actually had the opportunity to work with Marriott International Hotels, the hotels brand. And that was like 17 or 18 years ago. And it was one of the few companies at that time that recognized that there's a need to understand their customers in different countries, that they have different needs and they might need something different, we might need to design something different for them, which is quite impressive really, because at the time, like most people Just one design, anise it. So I had the opportunity to travel with them. I think the first country I went with them was Beijing. And I think probably because I was. I speak Mandarin, Mandarin is one of my mother tongue. So it's kind of like Cuichi, you can go and that is it. And that was great like because we were talking to Chinese and about how they go about booking hotels and so on. And then since then they invest a lot of money in understanding globally. So we will be going places like Qatar, Japan, Mexico and different places to not just talk to people but also observe how they are in the different hotels and in the lobby and how they use apps and mobile phones, mobile website and also desktop and so on. At that time it's more digital related because it's a digital teams I work with. And then after that I start to kind of write books about international research, talk about it in conferences, kind of like trying to own that space, space just purely because I was curious. And then I start working for myself. And so instead of focus much just on the digital side, but I was actually more about their global culturalizations and so on. So I don't look into just how people behave, but also look into the local culture first context. For example, I often think about the economy of a country, their infrastructures and a lot of other things sometimes even go back to their history because a lot of those elements could actually influence how we society behave because of colonizations or anything else. So those are actually very important because it's kind of influence what people value and also what whether a certain product fits into their life. So yeah, that's all the whole curiosity and everything kind of draw me into this work.
[05:11]
Jason Giles
I love that. And it's cleared by the way that you light up when you talk about it. The passion is still there. So I love you pursuing your curiosity and finding this kind of dimension perspective that has really been a great fit for you. So what prompted this conversation actually is you reached out to me and you were like, hey, there is a really important conversation that's really important to be having particularly right now. And so maybe can you talk a little bit about maybe what has AI fundamentally changed about customer insights?
[05:53]
Chuy Chewy Tan
Yeah, yeah, thanks for having me offering this platform for me to talk about it because I think it's very important topic to talk about since AI is kind of like influencing our lives so much. So when people ask me about what AI actually has changed, I think it changes the economics of customer insight. So a few years ago we all would agree one of the Bottlenecks of research is, was just simply making sense of all the data we get. Well, doing the research, but also making sense of the data. And researchers or teams might spend weeks to code the interviews or pulling everything together and summarizing and putting into a report. But today all of those could be done by AI in minutes. And they can analyze like hundreds or even thousands of customer input at scales in a speed that we never had in the past. So I do think AI actually do a lot of good things for the whole customer research industry. But I don't think the bottleneck has disappeared. The bottleneck that I was talking about, about spending a lot of time in and encoding the interviews and summarizing and so on, I just felt like the bottleneck has moved. So the bottleneck has used to be like synthesizing, right? But now I think the bottleneck is moved to the interpretations. Because we are very good now in producing summaries because AI could let us have good summaries in no time. But the questions now, or the harder questions now is are we the researcher or even the business people or product teams, are we interpreting the summaries correctly and accurately? What is interesting is this is not just me making it up on my observations, because there are a lot of recent studies and one of them is by, I think it's by Utrecht University and Western University as well. They compared thousands of AI generated summaries with the original research papers and they found that LLMs large language models were almost five times more likely than human written summaries to overgeneralizing the conclusions. So I'm not saying that AI is unreliable, right? Like it's much more reliable because we talk about hallucination in the past, but not as much anymore because things have improved. But to me, the problem is a beautifully written summary by AI. It doesn't mean it's a complete understanding or clear understanding, because if we think about it, if we only analyze one market, this itself already is something that we need to think about. But if you're analyzing feedback from six countries or 10 countries where people might use the same words for, for something, but it might be completely for different reasons and that become the challenges become multifold or become much bigger. So I don't think we should say that AI can AI actually provide customer feedback, because it clearly can. But it's more about how do we how to make sure the meaning survive the journey of customer input or customer voices into one strategic decisions that businesses have to do. So that's where researchers can create a lot of value.
[09:34]
Jason Giles
Yeah, for sure. And that, that moving of the bottleneck, it manifests itself. I see it firsthand with, with our research team. Right. Because there's. You still want to have the rigor one. And it takes time to ensure that like all the nuance that, like you say that over generalization that you, that you could miss, that still takes time. You know that it's just, it just is a different type of work.
[10:02]
Chuy Chewy Tan
And you can remove that completely because that you remove that you actually take like a very risky.
[10:10]
Jason Giles
That's exactly the risk. Right. Businesses making these decisions on a lot of data that may not have the rigor. And so that's the opportunity for research for now, certainly right now. Can you maybe give us. Your specialty is all around the differences between various markets. Could you talk us through an example of where, you know, maybe some of the terms that get used in, you know, particularly like things like, like trust and convenience, where they might actually being used that have a different underlying motivation or meaning something tangible that, you know, our audience could really understand.
[11:11]
Chuy Chewy Tan
Yeah, I'm sure. So in global research, I do think that we talk about differences a lot, right. Differences between countries. But I think similarities are often the beginning of analysis, not the end. So for example, I can give an example. You talk about trust or convenience. I'm going to use trust as an example. Trust is probably one of the most common themes that I ever heard about us here in user research in the last 17 or 18 years. Right. Because people, we often will hear people saying like you do interviews and in different countries and the AI might come back saying that trust is one of the biggest barriers of people not signing up on your subscriptions or not buying your products across all six markets. So now that is generally useful, right? It's a useful and valuable insight. And AI is doing exactly what we asked them to do. Right. It found a recurring pattern across a huge amount of customer input or customer feedback. But then to me, the more important question is what does trust actually mean in each market? Because in one market, trust might mean I have never heard about this brand, so it's awareness issues. Right. In another, it might mean I'm not confident putting my credit card details into your app or your website or some other countries might mean I'm worried that it's not easy or I forget to unsubscribe. So it's about that. Or it could be privacy or scams or so on. So in that instance, all of this is actually trust issues, but they are completely different business problem. And because there are different business problems, they need different business decisions. So for example, when I talk about the brand awareness earlier, if it is a brand awareness issue, since one business, then you might want to increase the awareness by improving your credibility of proof. But if it is like payment issues, then you might want to improve your payment methods or reassurance. If it's a subscription, then you might want to make cancellation easier and so on. So it's completely different even though it comes out as one theme, but it might require very different actions on that. So to me, a theme that AI might come up with is not the same as insight. Because a theme tell us what customers appear to have in common, but insight actually explain why they are saying it and what decisions we our business should do. In other words, I would say it's like what is driving certain behavior in certain markets. So I think that is very important in the AI world that we are living. Because AI help us find patterns and then now it's our job to understand what those patterns actually mean before we act. Because the same thing doesn't necessarily point to the same business problem, hence the solutions.
[14:26]
Jason Giles
Right. There's an example that you gave on your LinkedIn post around a sentiment that findings were well, it's too expensive. Right. And that's a very like. Okay, well similar to trust, right? Okay. This is a finding like trust is important. Well, hey, we're seeing a barrier to purchase and it's around price. Can you talk a little bit more about like what the story was behind that post?
[14:55]
Chuy Chewy Tan
Sure. That process actually comes from something that I encountered when I was working with Spotify for many different markets. It's not just one market, but throughout the years we go to different markets for different reasons and so on. But this expensiveness keep constantly come up one way or another. So we keep hearing participants or users from different countries say oh, it's too expensive. That's why I'm not subscribing, that's why I'm not doing this. And that's. And the natural reaction was to think, okay, this is a pricing issue, right? Like is affordability and so on. But throughout the whole my experience I know like it doesn't. It's not as simple as that is. So as the interviews went on, we sometimes will hear people will say the same people will say it's expensive. And then they will say, oh, I actually spend a lot of money. Like in Colombia they spend a lot of money on concerts, like lots a lot more than the subscriptions. But in concerts on things that will bring memory or have interactions with the local artists. Or in South Africa, they will spend a lot of money on data because that is their way to live because they have WhatsApp, they have businesses and so on. They need to have that continually. And Kenya, they will say that it's expensive, but then they will say that oh, I spent so much money for a party for my family because socially it's a collective society and family is really important and so on. So, so I would just say hang on, it's not about money. Then even you say it's expensive. So the more we look into it, the more we kind of say the real question we need to ask ourselves is expensive compared to what? Because sometimes it's honestly is generally affordability. But others might be just because your service was not worth me paying yet, or sometimes about the recurring subscriptions doesn't meet into their economic setting. Like oh, I get paid every day and it's up and down, so I can't have monthly subscriptions on. So it's the same sentence, right? It's extensive but then completely different reasons. So once you understand the research, you will end up actually have completely different business decisions to make and so on. So for example, you might want to improve your value propositions instead of lowering the price, or you want to redesign your onboarding to make it easier, or you might want to focus on the trust and so on. So you have to kind of look into that. What worries me sometimes with AI is that it's actually brilliant. Showing me that a lot of people, like thousands of people are saying the same thing. But it's up to us as a researcher, to products, teams or whoever to say, to start asking, are they actually talking about the same problem? Because we also, we have to remember customers are very good in describing their experience to such an extent, but they are not always very good in describing the underlying problem. Not because they do not want to tell you, but it's because sometimes they don't even know, right? Like what is the underlying that lead them to certain behavior or certain mentality. So when customers talk about trust or convenience or privacy or value, so we often will ask the questions that what is really driving them to say that? Because that is very important way to kind of guide us to the, the, the, the ultimate underlying insights that we are looking for.
[18:39]
Jason Giles
That is so true. I know I've had the pleasure for many years, you know, watching researchers and I'll do my own, you know, like I, I, I, I dabble and it's what, what I really find is that, so when I have a conversation, you know, maybe it's a moderate session, I, and I'll distill out and I'll get some findings and then I'll watch a researcher, a professional researcher basically do very ask similar questions. But then they like drill in and they really start teasing out. And when you look at like the findings that come out, mine are like, okay, yeah, you know, like they're good. But the insights that I'm getting from professional researchers always have just that extra little bit of like, okay, here's the answer to question, but here's the interesting angle that kind of turned up as part of this work. And interestingly, if I think about it sometimes where I'll be working with an AI, you know, and maybe we're doing some synthesis of a bunch of reports or data that I've got, I get that very surface level feeling from the AI as well, versus sitting down with a subject matter expert. And really they bring it to life in ways that I don't quite get from some of the technology right now. So I think it's really good.
[20:16]
Chuy Chewy Tan
We could say not yet, but definitely not at the moment. We don't know what it's going to be in the future, but at this moment, like, yeah, it's sometimes lack of the depth. Right, like we were looking for.
[20:31]
Jason Giles
No, totally. So I've had the opportunity to relocate a couple times now. You know, I spent a few years in the UK and in Spain. And I would say one of the things that, the best part about that experience was that it was so humbling because I thought that like everybody, you know, I didn't realize the biases that I had, you know, whether it was like the employee experience and my attitude towards work and it's been incredibly. Yeah, I guess I use the word humbling. I'm like, oh, wow. And it really has kind of driven an appreciation for understanding the different cultural implications of, of so many things that we can take for granted. And, and so I love this idea that there is more awareness and there's more specialists like yourself that are really helping companies understand how important those differences are and how it can be meaningful to their business. At the same time, folks that I've spoke to are like, well, you know, we've got AI now and this is great because, you know, we can, we can just get translation now. And I, I would like your take on why maybe understanding a market is so much more than just translation of words.
[22:13]
Chuy Chewy Tan
Yeah, I was trying to answer that question as using a personal story because I Think it will explain the problem a bit more. I actually posted that recently on LinkedIn as well. So my mom has this incredible skill to make a dish, a yum dish, right? So she once made it for for my sister's partner's parents. And afterward my sister said oh yeah, the dad have hinted that he will let mom to do more so he can have more because he enjoyed it so much. I just like I remember when I was home early this year while I'm from Malaysia and I said really? How did he hint it right? And she said well, he keeps saying that how delicious that is and how mom is so good at making it. I just like that is it. That is the hint that you are saying that is being so obvious because to me that sounds like just someone being very polite and say thank you. In the preachy culture, like even if something they don't want anymore, they never eat it. They will say oh that's really nice, you're really good at making it. And my sister said no, that is so obvious, it's a hint. So I kind of like, okay, we both are hearing the same thing, we didn't misunderstood things, but we are interpreting or we are responding to the very different culture rules about politeness, about gratitude or the whole indirectness. The thing is I grew up in that culture in Malaysia and I spent half of my first part of my life there and then the others half here in the uk. And so somehow I'm quite British and I'm quite Asian a long way. But somehow along the way my own wiring have shifted enough that I missed the signal completely in Asia. So that's why I kind of made me think like when researcher or businesses or teams treat AI research outputs as the full versions of customer reality, especially across different markets with different culture. And what would that mean then? Because even me as a human, I miss that international research that will be harder as well. So I remember I posted something about this similar things a few months ago and I heard a lot of AI moderator platforms that we have a lot now saying like oh, there's no problem, we have already done like hundreds of research in 10 countries recently or how many countries they were talking to. It's great that in any languages you want and you can do it while you're sleeping and so on because there's no tangible voices. But I think personally, I think translation is actually the easiest part of all for me is the harder question is like what did the cosmosis really mean? Right? And what's interesting is not just my experience as well. Because there are decades of cultural psychology research from like, I think Richard Nisbeck is one of them, is they have shown that people from different. We all know that as well. People from different cultures don't simply communicate or express them the same idea or express themselves the same way. And they will perceive situations differently and make different assumptions or make certain observations or conclusions very differently. So I see that in interviews as well. You probably have seen this as well yourself. If you say, if you have, if you. When you're doing the interview, because you might hear someone say, oh, that's interesting. So if you, if you look at. You know where I'm going, right? If you look at. Only look at transcript or ask AI to summarize, it will probably come as mildly positive, right? Interesting, positive words. But if you're sitting in interviews, you might notice something that they don't ask another question because they are not interested enough or their tone is flat or they hesitated or they move. So you might start thinking, actually, I don't think interesting is actually what they mean by interested. And a lot of AI moderator platforms now say they have emotional intelligence analysis. I'm yet to try it. Quite a few of them think, oh, we have this. So I can't comment on the accuracy and efficiency here, but that is emotional as one aspect. There's another aspect to us is interesting according to whom, right? In Britain, I use interesting a lot. It's a polite way to say not sure is actually, I don't like that much. And it's like, oh, it's a bit weird. But we don't want to sound confrontation. But in other cultures, the same word might generally means like, I'm interested or I'm curious. Tell me more. So the transcript is still the same, right? The translations hasn't changed. But what changed is a cultural context that give us the different meanings that those words carry. So AI is the same. They can translate beautifully, they can summarize beautifully. But it wouldn't automatically know if someone from different culture would most likely to hear what those words. So I wouldn't, I wouldn't say, like, ask yourself a question like, oh, what did they just say? Because that is not a valid question. When you read transcript, you probably want to think, what was someone from the same culture most likely to understand them in terms of what they meant? Because that is more important to get the right decisions, to get to the right to get the right insight, to get the right decisions to make.
[28:03]
Jason Giles
Yeah, I mean, that becomes my very next question. Right? Like with all of These like, like you don't know the assumptions that you're making. When I, when I first started working in the UK and someone was like, yeah, Jason, I really appreciated your perspective. It was very interesting. And I was thinking it was all great. And then I get the elbow in the side from my locals. They're like, yeah, that's not good.
[28:29]
Podcast Narrator
No.
[28:29]
Jason Giles
But I would never have known if it hadn't been for those who are from. From the culture of like, yeah, almost doing like that. The translation in real time. So for these teams then, that are taking their products into different markets or branding, whatever it is, where do you advise that they actually start? Like, how do you kind of crack this code and really have that understanding of what needs to be unique for the markets?
[29:06]
Chuy Chewy Tan
Yeah, I think is that a lot of ways that we can do that. Right. Because AI is good in finding cartoons as we talk about. Right. So we will tell us and what people say a thing that what people have in common. So. And it's good on that. So we should continue using that. But it doesn't tell us the why. Always doesn't always tell us the why. And the why is more important when we do the business decisions. So the way I really think about is really simple. So first of all, just try not to straight away jump from a theme to an action. So instead you cannot think about and stop in the middle and ask what is most likely driving this behavior in this market, like we talk about, because once we understand the driver, the decision will become clearer. So I started to kind of create a framework that I'm developing. It's called the Global Fit Loop. So it's very. At a very high level. It's answer three questions is that what is the signal? Why is driving it the signal in that market? And what decisions should change because of that. So AI can help us and very good in helping us on the first questions. The second question, what is driving in this market? Is where local context really matters. Because the local context is sometimes the culture or sometimes the regulations or sometimes the maturity in that specific category or industry or social norms, something. So the point is to assume the explanation. The point is not to assume the explanation before you have tested it. So that second one signal first, driving, why is the drivers second. And then the third one is where the decision should make should be the product marketing or growth team should work together to create that. So I'm not saying like choosing AI researchers. I do think like we should work together. The things that teams should do is not to skip the middle One about the drivers. Because that is the one step where you can move from pattern recognitions to customer understanding. And that's usually where you actually can making a local optimizations better and or adaptations better and make the right decision. So I would say follow this simple framework like signal driver and decision and
[31:41]
Jason Giles
what can you walk me through just like an example of what that conversation might look like where you stop and you're like okay, here's what we've seen, here's the signal. Now going through kind of your framework how that might play out.
[31:57]
Chuy Chewy Tan
Yeah, I'm trying to use the same one that we talked about earlier. You know, the trust things or the expensive things. Right, it's expensive. But then we needed to understand what actually drive that viewpoints of it expensive. It's kind of like either further asking that questions or actually looking into the transcript to say I'm going to stop and then go back to go back to my transcript and actually read back. Is there anything that is actually missing? So you know the point that we were talking about in Colombia, people say it's expensive, expensive. And the bottle is actually down to the quite to the end of the interviews that we're talking about other things, other areas because before going into Colombia I already look into one of the framework I created which is called cultural dimensions at 27 of them where it's possible related to Colombia and related to Spotify for example in this case. So I kind of like oh, there's elements of the proudness of connections with local artists or local. Local influences or subs and similar. And also the joy of having experience, physical experience. So we went in and we actually prompt a few more questions at the end and it's until at that point and they will start talking. Yeah, I spend a lot of money on concerts and I spend a lot of money on this and festival and things like that. So those actually didn't pick up by AI at all because it wasn't grouped as you know, it wasn't. The insight is the second part of the insight wasn't being picked up during the question we are asking why are you not subscribing? Or why is file value? So I didn't pick it up but we went back to look at the insights, the transcript and the notes that I normally take quite verbal Tim in and then we realized and then make a connection and that is ultimately one of the biggest insights that we ever get in that research. So yeah, sometimes it is kind of like the framework is like why is driving it. And then we went back and have a look. And then that kind of created a decision. It's not about pricing, it's not about that, but it's actually creating these kind of connections which change their propositions. In Colombia, they have a spot house, spot, Casa, Spotify. Is that something that they actually could bring local artists in and kind of interact with them for Colombia?
[34:39]
Jason Giles
What I love about that example too, is that we talk about all of our teams are driving for speed right now. Company is all about speed, speed, speed, speed. And yet, as design, research, even product leaders, we know that there's some places where it is important to slow down. And the example that you give is, that is such a great example of, look, we've got these signals, but this is the spot where maybe it is okay to just pause for a minute and really drill in. Particularly because the example you used was like, this was a big business decision. Right. So I think what I really like about this is, you know, being able to help encourage teams to identify those spots where slowing down is really important.
[35:36]
Chuy Chewy Tan
Yeah, the temptation is high, isn't it? Just like, everything is quick now in our world, AI helps it quick. So we need to show that everything is good can be run very quickly. But like you say, it's really important to just stop and say, hang on. Well, not only just question, is AI creating the right input or the right insight, but actually anything is missing that is important that the dots is not being connected. Well, I think that's really important. And unfortunately, because of the mentality of everything go fast and AI is so polished. The other thing is, I'm worried is, like, all the AI output is so polished, it sounds so perfect, and it's just so easy to say, oh, yeah, this is gold. This is gold value that we're getting.
[36:26]
Jason Giles
Yeah, well. And, you know, I was having a conversation with some other leaders recently, and one of the observations I've been having is the tools themselves make it so easy just to jump to the next step. They're like, okay, hey, we've done this thing for you. Do you want me to do this? Do you want me to do this?
[36:44]
Chuy Chewy Tan
Next?
[36:44]
Jason Giles
Do you want me to do this? And it's so easy just to be like, sure, yes, yes. And before you know it, you've gone so far down and you haven't.
[36:56]
Podcast Narrator
You.
[36:56]
Jason Giles
You haven't stopped to, like, check your assumptions and like, really, you've kind of slipstream past the important places where it makes sense. And. And the conversation that I think is important as leaders is where is that purposeful friction that we are applying to our teams in our workflows to ensure that just hitting the next button basically doesn't allow our teams to race past.
[37:30]
Chuy Chewy Tan
You're right about leaders, right? A lot of managers are C level. It's just like we pay for this platform, so you should get everything very quickly. And so they kind of encourage the speech to say, well I'm paying for this already. Like we're talking about either Claude or ChatGPT, Seamless LLMs or the AI moderation tools that now do the moderations and creating analysis and so on. So I heard a lot of C level is just like, well I'm paying for this using my budget so it should turn around quicker. I want days, not weeks. I want two days instead of seven days or few months. I was just like two days is not going to happen even if you use AI completely. So everything we talk about so far is actually applicable for both kind of AI, the LLMs generic and also the AI moderation tools as well.
[38:25]
Jason Giles
Yeah. Because speed is such a at a premium these days. Is there a simple thing that you encourage teams to do to build into their workflow before acting on maybe an insight that they've generated from AI?
[38:45]
Chuy Chewy Tan
I think one thing is always be very careful of the transcript you have. So. So there are two things, right? If you actually still lucky enough to be able to do in person interviews and then use AI to do the analysis. So you only have one part of analysis part that you have to be careful of. Or are we talking about even using AI to moderate? So that's one part that could miss an opportunity to dig deeper, right. To ask more questions, to observe, understand the culture. Say oh yeah, I know this culture is a bit like that, like local culture or local context. And I will prom it in a slightly different way when I do research in this country and that country. So if you are still doing the in person moderations, you could, you could reduce one layer of the possibility of risking missing information. But that allowed a lot of teams now is relying on AI one way or another to do analysis as well. So I think it's really important to kind of. I think I will keep saying like pause, not keep going like very easily. Just forgot and just rely everything on that. So always ask the same questions about the drivers and is that what. Don't take everything in the face value. If something looks too, too easy and too clean and too straightforward, then question it and go back to ask about the questions and to kind of look into that as well. I know sometimes like More managers will say like, I don't want to spend. This is going to slow down my team if I do this and that. But I don't think it actually needs to be a big process. But it's just stop and pause and ask some questions. Like for example, before we just take oh, what customers are telling us as insights, you just stop and say, okay, what are we assuming they mean? So you kind of challenge a bit that will make the team will go back and say, oh, let me check more or let me go into it and check more and so on. And it will tell a very different story. And once you have done that, maybe ask another question saying, what evidence make us confident that this is the right interpretations for this culture. If you need to, you might want to get some local experts to kind of like run through it or soundboarding and so on. So for me, it's not like stopping and do something differently, but I stop and ask a bit more question to challenge yourself, your team, but also challenge AI in some way. So yeah, I think that is really important to say, do we have enough evidence? If it is, yes, great. If it's not, we're not sure yet. That's okay. I always say I would much rather you spend the teams I'm coaching or working with. I would rather spending another 10 minutes or half an hour to challenge and interpretations than spending six months to build something that is wrong. So it's not about slowing things down, but it's just helping to move the. Move things in the right way faster.
[42:10]
Jason Giles
I would say 100%. And what I love too about your guidance is that it one, it's protecting the team's critical thinking. Right. Which I'm very conscious of as we use these tools. And the second thing, and you touched on this too, is the technology is changing so fast. And so by it's allowing you to see what is the current state like where. Oh, we're seeing that actually we don't see these hallucinations anymore or it's actually getting better than this. So I love your advice because there's multiple aspects to why that can be really powerful. We have been talking for a bit because I like to ask this question to any specialists and it's kind of just off the cuff. But what are some questions that you are getting? You know, you're a public speaker, you've written books. What are some questions that you like? Maybe a top question that you get from folks these days as it relates to global research or what you're seeing kind of emerge in this time I
[43:30]
Chuy Chewy Tan
used to have this question. Unfortunately I don't have the answer because this question is, is there any resources or anywhere that you can point me so I can understand different cultures? I just like where do we even start? Like are we talking about one culture or the other culture? Like one country or the other? We have like 196 countries nation or 192, I can't remember. But then within each country you have so many tribes and NCT and things like that. So it's impossible. There's one resources that point you to like just learn this or read this, you will be fine. Because even within Japan for example, like if you are in one industry than the others, the cultural aspect that you look into will be very different. Right? I used to go to Japan the same trip for Spotify and then for Asana, the work management tool. And even though it's in Japan, the same culture, we're doing different research for two different clients. But the things that we look into, the cultural elements we look into are completely different. One is about hierarchical, about like consensus and things like that. Nothing to do with Spotify in terms of their. What people care about. And it's, it's about identity, about the music, about having something physical so I can keep it and you know, I mean it's so, so it's very, it's. It's a huge in. It's a huge area on the main. But it's also making it very interesting. So I would say like keep being curious is whether with AI or without AI is always being curious, right? Like not making, keep asking the assumptions that you bring into to countries, even when you design a product, you always will have assumptions about that product, about maybe this is how people is going to buy something. This is how people is going to interact with my products. So just be curious and know that it's just assumptions and you want to go in and kind of find out, validate that as well. And then the curiosity in the AI world is another layer. It's like pausing. Like we talk about pausing, asking what might be missing. Because unfortunately there's no magic wand to say here you go, learn all about countries. You can just put it saying that I actually developing AI platforms that trying to. Which could actually hopefully be able to say okay, it's this country and this industry and domains you're in, you're in a marketing team of growth teams. You put it in and the system will actually look into the cultural aspect that I look into. Like using my 18 years of experience and so on. So hopefully that could be as close as a magic wand. But yeah, we will see where we get to and I'll announce that if I when I get that it's ready to test.
[46:33]
Jason Giles
Perfect. Perfect. And to that point, for those who would like to learn more about you and understand more about this entire big world, where would you recommend that the they connect with you or learn more?
[46:51]
Chuy Chewy Tan
LinkedIn is where I'm most active with. So if you hear about this podcast and want to connect, then just connect and mention that you hear from Insights Unlock and podcast and then happy to connect or comment in the comments on my LinkedIn. So yeah, and then I have my website, Bayo Global. I have a YouTube channel but I need to be more proactive in posting something. I go through phases as you know. Yeah, yeah. So yeah, those are places. But LinkedIn would be where you'll find me most amazing.
[47:26]
Jason Giles
And I did spend some time on your YouTube channel today and there's some really good stuff on there so I encourage folks to check it out. Chuy Chui, it is so nice to have spent some time with you. Thank you for reaching out. This has been a really fascinating conversation and so timely and yeah, just want to thank you for sharing your perspective with us today.
[47:50]
Chuy Chewy Tan
No, thank you. Thanks for having me.
[47:53]
Podcast Narrator
Want to keep the conversation going? You can find the show notes@usertesting.com podcast if you haven't already. Don't forget to follow us on Apple Podcasts, Spotify, Overcast or Google Play so you never miss an episode. And if you enjoyed today, please share it with a friend or leave us a rating and review on Apple Podcasts. And until next time, this is Insights Unlocked, an original podcast from User Testing.