Design Better: Bonus Episode Featuring Jason Giles, VP of Product Design at UserTesting

Hosted by Eli Woolery and Aaron Walter

Introduction

In this bonus episode of Design Better, hosts Eli Woolery and Aaron Walter sit down with Jason Giles, Vice President of Product Design at UserTesting. Recorded live at the UserTesting Human Insights Summit in Austin, Texas, the conversation delves deep into effective collaboration models, democratizing research, the integration of AI in user testing, and maintaining a human-centric approach in design.

Collaboration Models: Orchestra vs. Jazz Band

Jason Giles introduces the analogy of collaboration models by comparing traditional team structures to orchestras and his preferred method to jazz bands.

Jason Giles [02:16]: "There are two major approaches. There's what I call the orchestra model... The other end of the spectrum is the jazz band. And that's good for environments that might have a culture of learn by doing, maybe have a higher appetite for a little bit of risk, maybe sometimes make some mistakes."

Giles explains that while the orchestra model suits larger, risk-averse organizations requiring highly trained designers and product managers, the jazz band approach fosters flexibility and improvisation, encouraging teams to learn through experimentation.

Cultural Shifts in Design Teams

Aaron Walter highlights the cultural implications of adopting a jazz band approach, emphasizing generosity and collaboration within the team.

Aaron Walter [03:52]: "Presumably, like if you're taking this less structured approach where more people can be involved in the research process and the design conversation, that's going to have cultural implications."

Giles responds by discussing the challenges of implementing flexibility, such as resistance within teams and the importance of structural support during hiring and onboarding to instill a collaborative mindset.

Jason Giles [03:52]: "For the designers and the PMs in their career model, part of what is expected as a competency is to be able to get their own customer feedback."

Democratizing Research: Accessibility and Actionability

The conversation shifts to the benefits of centralizing research efforts, making insights accessible across the organization.

Jason Giles [08:43]: "Knowing what's available... the idea that it's right at your fingertips, I think that really could change the behavior of how companies are making decisions."

Giles emphasizes that reducing friction in accessing research results in more informed decision-making and fosters a customer-centric culture throughout the company.

Balancing Quality and Inclusivity in Broader Team Involvement

Eli Woolery raises concerns about balancing broader team involvement with maintaining the rigor and certainty of research findings.

Eli Woolery [05:06]: "Maybe you could talk us through a little bit of the risks and the benefits of this improv approach."

Giles shares his experiences at UserTesting, highlighting the challenges of ensuring quality while empowering non-researchers to contribute. He notes that striking the right balance involves setting guardrails and providing oversight to maintain the integrity of the research.

Jason Giles [05:22]: "We kind of pulled that back and now we've gotten back into this place where there's enough guardrails and we're able to have oversight and see the results."

The Role of AI in User Testing and Maintaining Human-Centered Design

The discussion explores the potential impact of artificial intelligence on user testing, questioning whether AI can replace the nuanced insights derived from human interactions.

Eli Woolery [11:28]: "One thing that often comes up is this idea that maybe eventually the AI will be smart enough that you can throw a design at it and it can give you more qualitative feedback."

Giles advocates for a hybrid approach where AI serves as a co-pilot, enhancing the capabilities of human researchers rather than replacing them. He underscores the importance of human discernment and creativity in the design process.

Jason Giles [12:34]: "You can't take the human out of that either with testing with real people or by having the discernment, the creativity that a human's going to have in ideating solutions."

Managing Research Team Workload and Strategic Focus

Aaron Walter addresses the challenge of research teams being overwhelmed with ad-hoc requests, potentially diverting focus from strategic, long-term projects.

Aaron Walter [14:38]: "How do you think about balancing those? Like, let's work on refinement and sanding off the edges of a product, but let's also keep our eyes to the horizon of changes in culture, changes in devices that could totally disrupt a business."

Giles shares UserTesting's approach to mitigating this issue by enabling 80% of evaluative research to be conducted by non-researchers, allowing the core research team to concentrate on high-impact, strategic initiatives.

Jason Giles [15:49]: "We set a goal that 80% of any evaluative research is done by a non-researcher... our researchers are spending so much of their time on the stuff that really is impacting the business at a larger scale."

Training and Quality Control for Non-Researchers

Ensuring the quality of insights from a democratized research process is crucial. Eli Woolery inquires about training methodologies that maintain high standards.

Eli Woolery [16:14]: "How do you train people that are outside your research team to contribute without diluting the quality of the insights?"

Giles explains that UserTesting employs a high-touch, personalized training approach akin to teaching music in a jazz band. Regular bi-weekly meetings between design leaders and product leaders help triage research questions, ensuring that only valuable and relevant insights are pursued.

Jason Giles [16:14]: "We do tend to focus on micro learnings because we're not a huge team, so we can personalize it a little bit... We have a really strong relationship between our header design and one of our product leaders."

Personal Insights and Wrap-Up

In the concluding segment, Jason shares personal interests outside of work, highlighting his love for music and how relocating to Edinburgh has influenced his lifestyle.

Jason Giles [18:23]: "I've got vinyl now. So that's what I do in cold Scottish weather is I come home, pour a whiskey, put some tool on, and that's my life."

The hosts wrap up the episode, encouraging listeners to engage with the show and share their experiences.

Conclusion

This insightful conversation with Jason Giles offers a deep dive into modern approaches to product design and research. By embracing a jazz band model, democratizing research access, integrating AI thoughtfully, and maintaining a strategic focus, organizations can foster a more innovative and customer-centric culture. Giles’s experiences at UserTesting provide valuable lessons for design professionals aiming to enhance collaboration, creativity, and efficiency within their teams.

Notable Quotes:

• Jason Giles [02:16]: "There are two major approaches. There's what I call the orchestra model... The other end of the spectrum is the jazz band."

• Aaron Walter [03:52]: "Presumably, like if you're taking this less structured approach where more people can be involved in the research process and the design conversation, that's going to have cultural implications."

• Jason Giles [08:43]: "Knowing what's available... the idea that it's right at your fingertips, I think that really could change the behavior of how companies are making decisions."

• Jason Giles [12:34]: "You can't take the human out of that either with testing with real people or by having the discernment, the creativity that a human's going to have in ideating solutions."

• Jason Giles [15:49]: "We set a goal that 80% of any evaluative research is done by a non-researcher... our researchers are spending so much of their time on the stuff that really is impacting the business at a larger scale."

This comprehensive summary captures the essence of the conversation, highlighting key discussions and insights shared by Jason Giles. It serves as a valuable resource for anyone interested in enhancing their design and research practices.