Insights Unlocked Podcast: Building a Customer Research Panel with BECU

Release Date: May 19, 2025

Introduction

In this episode of Insights Unlocked, host Nathan Isaacs and co-host Becky Wright engage in a comprehensive discussion with Jessica Schultz, Senior Manager of Experience Research at BECU. Jessica shares her insightful journey of building a successful customer research panel from the ground up, highlighting the strategies, challenges, and lessons learned along the way.

Jessica Schultz’s Journey into Experience Research

Jessica begins by recounting her humble beginnings at BECU as a teller, where her passion for problem-solving and process efficiency first took root. About 15 years ago, attending a user conference introduced her to the concepts of Personas and Journeys, sparking her interest in UX and user research.

[01:46] Jessica Schultz: "I felt kind of like a hobbit where I was like, wait, this comes in pints. There's like a career in UX. This is a thing."

Determined to pursue this newfound passion, Jessica obtained a master’s degree in Human Centered Design and Engineering from the University of Washington. This academic foundation enabled her to implement Human Centered Design principles at BECU, fostering a user-centric approach in product design and experience research.

[03:10] Jessica Schultz: "Empathy, that deep understanding and then the constant curiosity. That's probably my biggest weakness is I am so curious that any kind of interesting problem is like, oh, I'd like to find out about that."

Inception of the BECU Customer Research Panel

Motivation Behind Building a Panel

Jessica explains that BECU's decision to create their own customer research panel stemmed from the need for more efficient and direct member feedback mechanisms. Existing recruitment methods were time-consuming, taking weeks to identify suitable participants.

[05:06] Jessica Schultz: "We realized that we'd have a pretty big efficiency play to be able to take that recruiting from, you know, two to three weeks down to, you know, two to three hours by investing in creating a panel."

Early Decisions and Planning

The initial phase involved understanding the mechanics of UserTesting's custom network, ensuring legal and security compliance, and designing the member journey. Jessica emphasizes the importance of thorough documentation and journey mapping to create a seamless recruitment and participation experience.

[06:31] Jessica Schultz: "We created mural journey walkthroughs, journey maps of what was this recruitment process going to look like, what was that experience going to look like."

Recruitment Strategies and Member Engagement

Diverse Recruitment Channels

BECU employed a multi-faceted recruitment approach, including:

• Direct links in member newsletters
• Homepage announcements during calendar campaigns
• Social media marketing
• Training frontline staff to invite members during interactions

[07:45] Jessica Schultz: "We worked with our frontline staff team members in both our branch locations and our contact center to let members know about the panel."

Maintaining Engagement Over Time

To keep the panel members engaged, BECU consistently launched studies relevant to members' interests. Clear instructions and coaching were provided to ensure members understood the process, especially for first-time participants.

[12:16] Jessica Schultz: "We provide super clear instructions... And then in every single question, like please speak your answer, please write your answer."

Jessica also highlighted the unexpected benefit of improved study protocols, which enhanced the quality of feedback even from non-BECU panel participants.

[16:14] Jessica Schultz: "Changing the way we write our protocols to be more instructive... has actually improved the output of the studies."

Ensuring a Representative Panel

Broad and Inclusive Recruitment

BECU aimed for a diverse and representative panel by keeping the membership open to all and continuously inviting new members. Demographic data collection was crucial to balance the panel's composition.

[17:12] Jessica Schultz: "Our biggest strategy is to keep inviting, keep that invite up and running... the more we can increase the panel, the larger the representation."

Balancing Demographics

When certain demographics were overrepresented, BECU supplemented with prospects from the UserTesting panel to achieve balance.

[17:55] Jessica Schultz: "If we're weighted towards younger participants than older participants, we can fill in with prospects to balance it out."

Balancing Transparency and Privacy

Data Management and Consent

BECU prioritized privacy by ensuring that Personally Identifiable Information (PII) was not accessible to their team. Clear communication about data usage and security policies was paramount.

[19:26] Jessica Schultz: "We chose to not have the member emails visible. We're very careful in how we word our studies... protocols in place to delete any sensitive information inadvertently shared."

Clear Communication

The partnership with UserTesting was transparently communicated on BECU's landing page, ensuring members understood the collaboration and data handling practices.

[19:50] Jessica Schultz: "It's very clear that members understand how that works."

Utilizing Panel Insights for Decision Making

Informing Product Roadmaps

Insights from the panel directly influenced BECU's product development and prioritization. Generative moderated interviews allowed product owners to observe user interactions and gather actionable feedback.

[21:26] Jessica Schultz: "Product owners are able to take that away and inform the roadmap... it's always a prioritization exercise."

Enhancing Digital Platforms

Usability studies conducted with panel members provided critical feedback for improving BECU's digital banking platforms, ensuring changes positively impacted user experience.

[22:20] Jessica Schultz: "It's really critical. We can run a basic usability study with a participant who just banks... to make sure that our changes are positive."

Overcoming Internal Challenges

Securing Organizational Buy-In

Given BECU's member-obsessed culture, internal buy-in was minimal. The organization's focus on member needs facilitated smooth approval processes, reinforced by detailed documentation and clear process mapping.

[24:05] Jessica Schultz: "The idea of what do members think, what do members want, what do members need? That desire to have that deep empathy is pervasive throughout the organization."

Addressing Fill Rate Concerns

Initial concerns about study fill rates were mitigated by the panel's responsiveness. BECU achieved rapid recruitment times, often filling studies within 24 hours, thus alleviating timing-related apprehensions.

[28:14] Jessica Schultz: "Our studies have filled pretty quickly. Typically 24 hours."

Metrics for Evaluating Panel Success

BECU tracks several key metrics to assess the effectiveness of their customer research panel:

1. Number of Members Signed Up
2. Active Participation Rates
3. Study Fill Times
4. Demographic Representation

[30:18] Jessica Schultz: "Our first metric is just number of members that have signed up... how well it represents our member base."

Quarterly reports provide updates on panel growth and health, ensuring continuous monitoring and improvement.

[31:29] Jessica Schultz: "We report out on the member panel once a quarter... to see the overall growth and health of the panel."

Recommendations for Building a Customer Research Panel

Must-Do Strategies:

1. Thorough Understanding of the Custom Panel Mechanics

   • Map out the entire recruitment and participation process.

   [32:23] Jessica Schultz: "Make sure you really understand how the custom panel works. Do the work to map it out."

2. Strategic Demographic Data Collection

   • Collect comprehensive demographic information upfront to minimize screener-dependent filtering.

   [32:50] Jessica Schultz: "We spent quite a bit of time on the demographics... to ensure a positive participant experience."

3. Provide a Positive Initial Experience

   • Launch general studies accessible to all panel members to foster engagement and reduce entry barriers.

   [34:20] Jessica Schultz: "We have a general study that pretty much anybody who signed up for the panel would be able to take and that one is kind of always on."

Things to Avoid:

• Launching Highly Niche Studies Early On
  • Avoid overly specific studies that can alienate new panel members and reduce engagement.

[34:50] Jessica Schultz: "Avoid having super niche studies at the beginning... have that general study to create a positive experience."

Closing Remarks

Jessica encourages listeners interested in learning more or collaborating with BECU to reach out via her LinkedIn profile or visit BECU's website. Her insights provide a valuable blueprint for organizations aiming to establish their own customer research panels, emphasizing the importance of empathy, strategic planning, and continuous engagement.

[36:55] Jessica Schultz: "We are a growing team, growing practice... especially if that idea of being member obsessed and really looking for ways that we can focus on the financial well-being of our members in our communities."

Becky Wright and Nathan Isaacs wrap up the episode by thanking Jessica for her valuable contributions and encouraging listeners to explore more through UserTesting's podcast channels.

Takeaways:

• Strategic Planning: Establishing a customer research panel requires meticulous planning, understanding of tools, and cross-departmental collaboration.
• Member-Centric Approach: Prioritizing the member experience fosters engagement and yields more actionable insights.
• Continuous Improvement: Regularly launching studies and adjusting strategies based on feedback ensures the panel remains active and representative.
• Transparency and Privacy: Balancing openness with stringent privacy measures builds trust and encourages participation.

For more detailed insights and actionable strategies, listen to the full episode of Insights Unlocked on your preferred podcast platform or visit usertesting.com/podcast.