Podcast Summary: The Agile Brand with Greg Kihlström®
Episode #685: Using AI in UX Research, Design, and Testing with Jason Bowman
Release Date: June 4, 2025

Introduction

In Episode #685 of The Agile Brand with Greg Kihlström®, host Greg Kihlström delves into the transformative role of Artificial Intelligence (AI) in User Experience (UX) research, design, and testing. Featuring Jason Bowman, Executive Director of Experience Strategy at the Office of Experience, the discussion explores how AI enhances efficiency, fosters innovation, and integrates seamlessly with human expertise to build agile and resilient brands.

Guest Introduction: Jason Bowman

Jason Bowman brings a wealth of experience to the conversation, tracing his roots in digital practices back to the early days of the internet. With a diverse background spanning agency roles, in-house teams, and startups, Jason has honed his expertise in UX design, content strategy, and business analysis. At the Office of Experience, he leads the UX and content strategy teams, ensuring that strategic concepts are effectively executed through development.

AI's Impact on UX Research

Speed and Efficiency in Research

Jason Bowman highlights the significant acceleration AI brings to UX research. Traditionally, research tasks—such as gathering best practices or analyzing competitor strategies—were time-consuming. AI streamlines these processes by providing quick summaries and initial analyses, allowing teams to bypass extensive manual searches.

“AI starts to either get you started in one direction, summarize something, or even start to analyze things... it’s really been fun as we’ve gotten deeper in training the engine to what we need.”
- Jason Bowman [03:58]

Enhanced Heuristic Analysis and Persona Development

AI tools like Claude are instrumental in generating heuristics and personas. By inputting existing personas, AI can predict how different user segments might interact with content or design elements, facilitating early-stage testing and validation.

“We started doing some things such as heuristics... it starts to spit out results that are about 80% of what we need.”
- Jason Bowman [05:00]

Predictive Validation in UX Design

Defining Predictive Validation

Predictive validation refers to the use of AI to assess UX elements before full-scale user testing. This approach allows designers to gauge potential user reactions and iterate on designs swiftly.

“Predictive validation allows us before we go all the way to publishing or testing... to just gut check things with that predictive validation.”
- Jason Bowman [07:51]

Benefits and Considerations

By utilizing AI for predictive validation, teams can save time and resources. AI-driven tools can perform rapid A/B testing simulations, providing quick feedback that helps in refining designs before human testing.

“You set up an A/B test... What used to take a day can now be validated in five minutes.”
- Jason Bowman [11:00]

However, Jason cautions about the potential biases inherent in AI systems. It's essential to validate AI-generated insights with human judgment to ensure accuracy and relevance.

“If you can get what would take a half day of research and you can compile that into 15 minutes, it gives you that kind of window to do deeper research... super helpful in that way.”
- Jason Bowman [07:36]

AI and Human Collaboration in UX

Balancing AI Efficiency with Human Expertise

Greg Kihlström emphasizes the symbiotic relationship between AI and human input. While AI can handle repetitive and data-intensive tasks, human creativity and critical thinking remain irreplaceable.

“Let’s use AI to get us further, quicker, but let’s use the humans to do what humans do best as well.”
- Greg Kihlström [06:43]

Case Study: Content Strategy and Prototyping

Jason illustrates how AI assists in content strategy by generating multiple content versions, which can then be refined and validated by UX teams. Additionally, AI can produce wireframes and visual references, expediting the prototyping phase.

“With Claude, it can even start to code up a quick wireframe, give you a visual reference. It just lets you play with some things quicker.”
- Jason Bowman [06:35]

Innovation: AI vs. Human Creativity

AI's Role in Innovation

The discussion pivots to the capacity of AI to drive innovation. Jason asserts that while AI can process vast amounts of data and identify trends, true innovation—a leap beyond existing paradigms—still heavily relies on human intuition and creativity.

“We are still necessary to innovate because we don't know all the prompts that we might have. Our brains just can't get the AI to act like us...”
- Jason Bowman [19:07]

Limitations of AI in Creative Processes

AI’s predictive nature means it often generates solutions based on existing data, potentially stifling groundbreaking ideas. Human creativity involves drawing from diverse experiences and making unexpected connections, areas where AI currently falls short.

“AI tends to take all the knowledge it can get, find the themes, and build off of that... it's not always pushing the boundaries of a new.”
- Jason Bowman [18:05]

Greg adds that while AI can suggest likely outcomes, it lacks the ability to produce truly novel ideas without human prompting and guidance.

“AI is designed to predict the next most likely thing... it’s not creative in the human sense.”
- Greg Kihlström [19:30]

Best Practices for Using AI in UX

Validating AI Outputs

Jason underscores the importance of verifying AI-generated content. Just as one wouldn't accept every piece of information from the internet without scrutiny, AI outputs should be cross-checked for accuracy and relevance.

“You should validate it with some other sources. [...] You have to know that it is still learning. It’s not a magic button.”
- Jason Bowman [15:29]

Disciplined Prompting and Iteration

Effective use of AI requires crafting precise prompts and iteratively refining them to guide the AI towards desired outcomes. This disciplined approach ensures that the AI serves as a valuable tool rather than a source of unchecked responses.

“The discipline of even creating a good prompt, asking good questions, having an idea what the results should be... is a great way to get the most out of it.”
- Jason Bowman [16:42]

Recognizing When to Rely on Human Input

There are scenarios where AI-generated solutions may not meet the nuanced needs of a project. In such cases, relying on human expertise to refine or overhaul AI outputs is crucial for maintaining quality and relevance.

“Just because AI created an artifact doesn't mean that it's the right artifact to use. It needs user experience... editing and refinement.”
- Jason Bowman [21:05]

Staying Agile in a Rapidly Evolving Landscape

Fostering Team Autonomy and Openness

Jason shares his strategy for maintaining agility within his team. By granting team members autonomy and encouraging openness to new ideas, a culture of continuous improvement and adaptability is cultivated.

“Give the team a little bit of a leash and they start moving in something... being open to where ideas come from all sorts of different teams and disciplines.”
- Jason Bowman [22:57]

Embracing Continuous Learning and Experimentation

Staying agile requires embracing new methodologies and tools, including AI. Encouraging experimentation and learning from each iteration helps teams stay ahead of industry trends and maintain relevance.

Conclusion

Episode #685 of The Agile Brand offers a comprehensive exploration of AI's role in enhancing UX research, design, and testing. Jason Bowman's insights reveal that while AI significantly boosts efficiency and provides valuable preliminary analyses, human expertise remains essential for validation, innovation, and nuanced decision-making. The dialogue underscores the importance of a balanced approach, leveraging AI's strengths while acknowledging and addressing its limitations to build agile, resilient brands poised for long-term success.

Notable Quotes

• Jason Bowman [03:58]: “AI starts to either get you started in one direction, summarize something, or even start to analyze things... it’s really been fun as we’ve gotten deeper in training the engine to what we need.”

• Greg Kihlström [06:43]: “Let’s use AI to get us further, quicker, but let’s use the humans to do what humans do best as well.”

• Jason Bowman [07:51]: “Predictive validation allows us before we go all the way to publishing or testing... to just gut check things with that predictive validation.”

• Jason Bowman [19:07]: “We are still necessary to innovate because we don't know all the prompts that we might have. Our brains just can't get the AI to act like us...”

• Jason Bowman [15:29]: “You should validate it with some other sources. [...] You have to know that it is still learning. It’s not a magic button.”

Further Information

To learn more about Jason Bowman and the Office of Experience, visit their website. For additional insights and episodes, subscribe to The Agile Brand at theagilebrand.com and explore consulting and advisory services at gregkilstrom.com.