How AI + user feedback transformed puppy training with Zigzag's Jack Mitchell

Summary

Summary of "How AI + User Feedback Transformed Puppy Training with Zigzag's Jack Mitchell"

Insights Unlocked episode titled "How AI + User Feedback Transformed Puppy Training with Zigzag's Jack Mitchell" delves into the innovative integration of artificial intelligence (AI) in puppy training through the Zigzag app. Hosted by Nathan Isaacs and Michael Dominic of UserTesting, the episode features an in-depth conversation with Jack Mitchell, Head of Product and Operations at Zigzag. The discussion explores the challenges, strategies, and successes Zigzag experienced while developing and implementing their AI-powered virtual puppy trainer, Ziggy.

Introduction

The episode opens with Nathan Isaacs introducing the topic: the transformation of puppy training through AI and user feedback. He sets the stage by highlighting Zigzag's mission to provide expert, science-backed puppy training at scale while maintaining engagement and personalization.

[00:02] Nathan Isaacs: "Today we're exploring how AI is revolutionizing puppy training with Jack Mitchell, Head of Product and Operations at Zigzag, the number one puppy training app."

Host and Guest Introductions

Michael Dominic welcomes listeners to the podcast, emphasizing the show's focus on candid conversations with leaders behind successful digital products. Nathan Isaacs introduces himself and Michael Dominic before welcoming Jack Mitchell to the show.

[01:25] Nathan Isaacs: "Welcome to the Insights Unlocked podcast. I'm Nathan Isaacs, Senior Manager for Content Production at UserTesting, and joining us today is Michael Dominic, UserTesting's Head of AI."

Jack Mitchell provides a brief introduction, sharing his extensive experience in B2C startups and his role at Zigzag.

[01:59] Jack Mitchell: "I've spent over 10 years in B2C startups... I was like, this would have been great during those early years."

Jack Mitchell's Background and Zigzag Overview

Jack elaborates on Zigzag's inception and mission, likening it to "Duolingo for dogs." He describes Zigzag as a B Corp with a 20-person team, focused on delivering daily bite-sized lessons and AI-driven puppy training advice.

[01:59] Jack Mitchell: "We offer users daily bite-sized lessons and AI-driven puppy answers."

Jack also shares a personal anecdote about his dog, Tina, underscoring the practical applications of Zigzag's solutions.

[02:59] Jack Mitchell: "My dog's name is Tina. She's a four-year-old Labrador... I can get a little bit of help with our training coaches and the app."

Implementing AI in Zigzag: Overview of Ziggy

The conversation shifts to Zigzag's pioneering feature, Ziggy—a generative AI chat integrated into the app to provide personalized training advice. Jack discusses the strategic and technical challenges faced during Ziggy's development.

[03:50] Jack Mitchell: "The biggest strategic challenge was that we were in growth mode... we needed to find a way to provide science-backed advice through a scalable method."

Ziggy serves as a virtual puppy trainer trained by real dog trainers, ensuring that the AI responses are trustworthy and tailored to each dog's profile.

[03:50] Jack Mitchell: "Ziggy is a cartoon AI coach trained by real dog trainers and is personalized to each dog's profile when they sign up to Zigzag."

Challenges in AI Integration

Jack outlines the dual challenges of maintaining personalization while ensuring consistency and accuracy in AI responses. Collaborating with multidisciplinary teams, including data specialists, dog coaches, engineers, and product designers, was crucial to overcoming these obstacles.

[04:30] Jack Mitchell: "We had data people, we had non-technical dog coaches, we have engineers and product designers. It's really about bringing all those people together."

User Feedback and Its Role in Developing Ziggy

Continuous user interviews and feedback were pivotal in shaping Ziggy. Jack highlights how understanding user needs and hesitations informed the design and functionality of the AI feature.

[06:59] Jack Mitchell: "We interview users here on a continuous basis... people didn't realize it was there in the first place, but two, people just felt silly interacting with training coaches."

Building Trust with AI Features

A significant focus of the discussion is building user trust in AI-driven training. Jack explains that unlike generic AI models like ChatGPT, Ziggy delivers curated, science-backed responses tailored to dog training.

[06:02] Jack Mitchell: "What we did was get our training coaches to ultimately train the model we provide. So users know they can trust the answers they're getting."

Impact on User Engagement and Retention

The integration of Ziggy led to increased user engagement and retention. By offering an accessible and non-judgmental platform for users to ask questions, Zigzag saw a substantial lift in overall retention rates.

[11:24] Jack Mitchell: "We saw an increase in overall retention rates... users know that a problem can be solved, it makes them more likely to return."

Measuring Success and ROI

Jack emphasizes the importance of tracking metrics to assess the ROI of AI features. Zigzag employed A/B testing to compare subscription rates and average revenue per user between versions with and without the AI chat.

[16:12] Jack Mitchell: "We were looking at reducing our load on the training coaches and increasing subscription rates... we knew categorically what impact that had on our core metrics."

He advises other businesses to start small, iterate quickly, and maintain continuous feedback loops to ensure AI features deliver genuine value.

[15:30] Jack Mitchell: "Start small, iterating quickly and tracking that usage, getting those feedback loops from the outset."

Internal Use of AI at Zigzag

Beyond the customer-facing AI, Zigzag leverages AI internally to enhance product development processes. Jack discusses using AI for design reviews, copy editing, handling qualitative user feedback, and ensuring legal compliance, thereby speeding up iteration cycles and fostering innovation.

[19:06] Jack Mitchell: "We're using AI a lot for things like design reviews, copy editing, and handling qualitative user feedback... it massively speeds up our iteration cycles."

Future Plans and AI Refinement

Looking ahead, Jack outlines Zigzag's plans to continuously refine Ziggy based on real user interactions and expand into new problem areas. The focus remains on data-informed, user-centric approaches to adapt to the evolving AI landscape.

[22:22] Jack Mitchell: "We're only refining Ziggy based on real interactions... solving new problems as well."

Conclusion and How to Connect

The episode concludes with Jack sharing how listeners can connect with him and learn more about Zigzag. He encourages dog owners to download the app and provide feedback to help improve the service.

[24:47] Jack Mitchell: "Feel free to connect with me on LinkedIn... please download Zigzag if you're a dog owner or especially a puppy owner."

Key Takeaways

1. User-Centric AI Development: Continuous user feedback is essential in creating AI solutions that genuinely meet user needs.
2. Building Trust in AI: Providing curated, science-backed responses fosters user trust and enhances engagement.
3. Measuring ROI: Tracking specific metrics through A/B testing is crucial to assess the value and impact of AI features.
4. Internal AI Utilization: Leveraging AI internally can streamline processes, enhance product development, and foster innovation.
5. Iterative Refinement: Continuous refinement based on real interactions ensures that AI features remain relevant and effective.

Notable Quotes

• Jack Mitchell [03:50]: "Ziggy is a cartoon AI coach trained by real dog trainers and is personalized to each dog's profile when they sign up to Zigzag."
• Jack Mitchell [06:02]: "What we did was get our training coaches to ultimately train the model we provide. So users know they can trust the answers they're getting."
• Jack Mitchell [10:15]: "We have a 96% positive rating with our responses because we capture feedback every time we give a response."
• Jack Mitchell [16:12]: "We knew categorically what impact that had on our core metrics."

Final Thoughts

This episode of Insights Unlocked provides valuable insights into the strategic integration of AI in a consumer-focused app. Jack Mitchell's experiences with Zigzag illustrate the importance of user feedback, trust-building, and meticulous measurement in successfully leveraging AI to enhance user experiences and drive business growth.

For more information and to listen to the full episode, visit usertesting.com/podcast.

Transcript

[Nathan Isaacs]

Welcome back to the Insights Unlocked podcast. Today we're exploring how AI is revolutionizing puppy training with Jack Mitchell, Head of Product and operations at Zigzag, the number one puppy training app. Jack and his team set out to solve a big challenge. How do you provide expert science backed puppy training at scale while keeping it engaging and personalized? The answer, Ziggy, Zigzag's AI powered virtual puppy trainer. But building trust in AI driven training wasn't easy. Jack will share how user feedback played a crucial role in shaping Zige, helping the team understand what pet owners needed most, where they hesitated to ask for help and how AI could make training more accessible. We'll also dive into how Zigzag measured AI's impact with a B testing, tackled user engagement challenges and ensured their AI solutions delivered real business value. It's a fascinating look at the intersection of AI, UX and pet care. So let's get started. Enjoy the show.

[Michael Dominic]

Welcome to Insights Unlocked, an original podcast from User Testing where we bring you candid conversations and stories with the thinkers, viewers and builders behind some of the most successful digital products and experiences in the world. From concept to execution.

[Nathan Isaacs]

Welcome to the Insights Unlocked podcast. I'm Nathan Isaacs, Senior manager for content production and user testing and joining us today is host is Michael Dominic, User Testing's Head of AI. Welcome Michael.

[Jack Mitchell]

Hey everyone.

[Nathan Isaacs]

And our guest today is Jack Mitchell, Head of Product and operations at Zig Zag. Zigzag is the number one puppy training app. Welcome to the show, Jack.

[Jack Mitchell]

Thanks Nathan.

Jack, thank you so much for joining us today. Look, as we get started I'd love for you to tell us a bit about your career journey. Tell us about Zigzag, tell us about the work that you do there.

Of course. So I've spent over 10 years in B2C startups. So I started out at a scaling fintech credit report company and during that my eight years there, like pretty much anyone on the planet, I got a dog in lockdown and had the opportunity to join zigzag as a second employee. Zigzag is a 20 person B corp and we're a corporate backed startup. I use it to describe Zigzag as Duolingo for dogs. So we offer users daily bite sized lessons and AI driven puppy answers. So you know what, what does this mean? I've gained deep expertise in user focused growth and rapid scaling and now applying that know how to create engaging science based training solutions here at digvac.

So Jack, before we unpack all of that, what's your dog's name? What kind of dog do you have?

My dog's name is Tina. She's a four year old Labrador. I didn't have Zigzag during puppyhood, so you know, when I found Zigzag I was like, this would have been great during those early years. You've really got to get started early. So yes, still have a few, few underlying issues to resolve. But yeah, I can get a little, little bit of help with our training coaches and the app.

Yeah, fantastic. So putting the focus back on Zigzag a little bit here. So Zigzag was among the first dog training apps to integrate a generative AI chat. So can you tell us a bit about that feature? Also it'd be great to hear what were some of the biggest technical and strategical challenges that you faced in developing that AI powered solution. How did you overcome those challenges? So tell us a little bit more about that.

Cool. So Ziggy is a cartoon AI coach trained by real dog trainers and is personalized to each dog's profile when they sign up to Zigzag. So when we built it, the biggest strategic challenge was that we were in growth mode and still are. So at the time we had a WhatsApp chat support for all real life training coaches. And as you can imagine, that's not really scalable as we, as we grow as a business. So we needed to find a way to provide science backed advice through a scalable method while still incorporating that core value proposition for our users. The technical challenge there was balancing personalization with a need for consistent answers that were validated through proof of concept testing from a technical side, but also getting trainers to review the answers to ensure the responses given were suitable to our training methods, but also science backed. So in terms of how we overcame these challenges, first it was all about continuous user interviews, really knowing what problems our users were coming to solve, looking at the data that we already had around that. And we found that out through things like chat tagging. As you can imagine, collaboration was key throughout that process. So we had data people, we had non technical dog coaches, we have engineers and product designers. So it's really about bringing all those people together to understand that problem and deliver a relevant solution. And then on top of that we needed to deliver an AI solution which users trust. So why wouldn't they just go to chat GBT rather than us? So that solution testing was really important throughout that process to make that not only trusted but engaging and valuable.

Yeah, can you unpack that a little bit because I think I often have the same question when using a product's AI features is why am I using this as opposed to just going to one of the frontier models directly? So why would someone use your feature as opposed to going to ChatGPT?

Well, the thing that's really hard about in ChatGPT, you're getting very generic answers that are sourced from the wider Internet. The what we did through our responses was get our training coaches to ultimately train the model we provide. So it means users get know that they can trust that the answers that they're getting and they're getting training methods that are positive rather than perhaps negative for, for, for training their dog or, or puppy so that, you know, they really can trust those answers and get those answers quickly.

I'd be curious to go back to the process around interviewing folks to get feedback on your prototype tests and really all the stages of development. What might have been some of the biggest surprises that you gained from testing and how did those insights ultimately impact the final version of what you rolled out?

We interview users here on a continuous basis, so we're doing regular user interviews. And we recognize this problem because our training coaches was a real star feature for our products. And we found a couple of things that, you know, one, that people didn't realize it was there in the first place, but two, people just felt silly interacting with training coaches. They didn't want to bother them with like, stupid questions like, you know, maybe they just thought of something and then they're out and about and, you know, didn't want to bother a human even though they're, you know, paying a subscription. So that was probably one of the big ones. And we knew that there was an opportunity there. And on top of that problem, we, when we got to solution testing with, with the AI feature, when we matched that solution to the problem, one of the biggest insights that. Yes, like as you expect to. And it was very early, early days with AI, not everyone was using ChatGPT at that stage, so people knew about the concept. But there's just as there is now, low initial trust. People just don't want to engage with an AI bot because they don't. Yeah, but they don't know what response they're getting. Also they don't know how to engage with it in the first place. So that solution testing was really important for us. What we did throughout that process, one of the key insights was keeping a human in the loop to validate any kind of critical advice, especially answers we couldn't give, but also letting Users know. And in fact, we separated our experience between human and AI just to make that really clear to our users what that value proposition was. Second from that, we incorporated the Ziggy character, which is very personable, and it helps build up that user confidence and get people to engage with it in the first place. And finally, one of the biggest things that really drove engagement was personalized suggested questions. So getting users to engage early on was really important to us. And those personalized questions really encouraged users to engage with the feature early on in their customer journey. So those are the things we tackled early on to build on that. And since then, we've been continuing to refine prompts and the model itself. And that's really through continuous data gathering, whether that's through our user interviews and qual research or the quants research research we get on a continuous basis through things like ratings.

Yeah, that's. That's fantastic.

Yeah, you.

You touched on something that I've been thinking about through all of this, and I think about this. Wherever companies are integrating AI features is that trust thing. Right. A lot of folks approach AI not knowing what to trust and what level of trust that they should be putting into it. Are you noticing a change in the level of trust that people have in the product as they continue to use.

It in terms of the feature itself or just generally the product?

The AI feature itself, I think people.

Just learn more and more how to engage with it. So once they've used it, once we see high retention, what we found is that users really trust our responses and we get really good positive ratings. We have a 96% positive rating with our responses because we capture feedback every time we give a response. That's great. So, yeah, yeah, the. The tractors certainly there hasn't diminished.

Yeah. I think another thing that you touched on that was really interesting to me as well is, I guess, you know, for the traditional kind of chat feature where folks are engaging with an actual human trainer, they feel a little bit of hesitation to reach out to those individuals for silly questions. So, yeah, there's just kind of like a whole, you know, can of worms, I guess, that can be opened up with the way that users engage with your product. If they just feel like they're not actually bothering someone, they're just asking an AI chat. So that's interesting. I mean, are you. Are you seeing a lift in engagement from folks because they know that they can just engage with a AI chat feature as opposed to a person?

Yeah, definitely. It's to that point that when we a b Tested it, we saw an increase in overall retention rates. We saw, you know, obviously it was a new feature. So it was, you know, we, it was a feature that, for people to come back to. So as, as soon as they users know that a problem can be solved, it makes them more likely to return. Now ultimately, at our core, our products and unique value proposition is a training journey, which is more of a proactive training journey. Here's personalized to you as a user and your puppy or dog and it's there for you to follow. This is more of, oh crap, the, the dog is doing this thing that I never expected it to do. Maybe it's peeing on my shoes the whole time. And you can as a user go in and ask that really stupid question and then if your answer isn't solved. Yeah, you can go to a training coach as well who can give you a bit more of a, an even more bespoke response to that.

Yeah, that's fantastic. So Jack, switching gears here a little bit, I think what we see are, especially in the digital product landscape is there are a lot of companies that are figuring out, are trying to figure out how do we integrate AI into our product. I think it's safe to say that most product teams are pretty focused on this right now. So I think a lot of companies are struggling to ensure that those AI features are driving real value rather than just being some novelty or just being AI cool. So how did Zigzag determine that an AI chat feature would genuinely improve the user experience for puppy parents?

Yeah, so I think, I think it's in terms of how we work here. We have a relentless focus on business value. You know, we always have our on growth targets and we're look or that's always, always our North Star and we map that to that user value. So I touched on it earlier. Those continuous user interviews are really important to us to really know what our problem space looks like. So instantly we knew that very early on, this is 18 months ago now, that that solution was a great match for problems our users have. When we go back to the business side. We knew we were going to a B test this feature right from the start. So we needed to know that it would drive real business value for us. And obviously that maps to solving real pain points for users like training confusion and things like a lack of guidance or coming in to solve a biting issue, stuff like that. So that relentless focus on those results and that sort of qualitative insight that maps to that is really important. And having clear success metrics from the outset. So whether that's subscription rates use, just general usage and feedback rating. So when we write up a design brief that is front of mind for a product designer to really design with that in mind in it. In terms of the key takeaways, I'd say I think every founder, what most founders around will go, yeah, I need like an AI roadmap. And we're gonna just like nail that roadmap. So yeah, it's not something that you should do just for the sake of it as a business, but yeah, if you are going to try anything out, yeah, make sure it solves those problems. I'd advise starting small, iterating quickly and tracking that usage, getting those feedback loops from the outset. And yeah, use that to continuously refine any AI features that you build. Because yes, it can be relatively straightforward to set these things up, but to those points you need to get usage. There's no point in building a feature that is dead in the water. And no one uses this from the outset.

Yeah, Jack, that's great that you guys are tracking some metrics around success. So there's this conversation that happens across the AI landscape, the AI product landscape, and that's this conversation around roi. Right. Like every company, again, they're trying to figure out how do we integrate AI into our products, into our services, into our companies.

Right.

To help us do things better, do things faster, more efficient. Um, and then there's this conversation around, like, what is the ROI of this? So can you just double click on what those success metrics are that you're tracking? And is there any kind of like tight ROI calculation that you attach to those metrics as much as you're comfortable sharing?

Yeah, yeah, of course. The key, the key, there's two key metrics we were looking at on top of things like usage, but primarily one was about reducing our load on the training coaches. So we needed to create a solution that enables us to scale and maintain some of our core feature sets as a key selling point. And the second was when we a B test. We're a B testing around things like subscription rates and average revenue per user. Yeah, ultimately lifetime value. So we a B tested the version with AI chat versus the version that had nothing. So we knew categorically what impact that had to subscription rates, our average revenue per user and our other core metrics. So yeah, having having, you know, that's not as easy, easily done for every single business of every size. But I think certainly, you know, when you're talking about smaller businesses and B2C apps where you're getting a lot of. A lot of data or enough data to run those. A B test, like, setting up these features with that in mind is crucial to know what your ROI is.

Yeah, I'm really glad that you all are focused on that. So I think when I look at the way people respond to that question right now across the landscape, what I'm seeing are three different answers. The first answer is people just kind of shrugging their shoulders and they're like, I don't know AI, right? It's valuable. There's another set of individuals who are AI enthusiasts and they basically say, well, we don't need to track ROI because we know that AI is just massively improving efficiency and everything that we're doing. Right. Like, you don't measure the ROI of electricity, you don't measure the ROI of email, which I'm sympathetic to that. But, you know, like, there are business stakeholders that actually need to see that this is driving value. So, yeah, if folks that are listening to this podcast, like, if I. You've made a couple really good points. And one of those really good points is this, right? Like, you should be tracking this, you should be attaching value to it. You should be reporting that to your business stakeholders. Because, yes, we know that AI does improve experience, it improves efficiency. Right. It makes things better. But we need to be tracking that. So that's fantastic that you're doing that. So I want to switch gears again a little bit here and just look at generally, you know, how your team is using AI to like, enhance your internal process. So a lot of companies are doing that as well. Right. So they're using AI to enhance their product development process or just like, just any general process. So how does Zigzag leverage AI tools to enhance your process and build and refine product?

Yep. So Zigzag is a really small team. As I said, I was the second employee. So, yeah, yes, we're corporate backed, so we have the funds to be able to grow, but we need to employ people at the right time. So we've been very careful with our growth to make sure we're growing sustainably. Currently, the team is myself as head of products and operations, working with a senior product designer. So we have very limited resources and that just generally means we are collaborating with lots of people in the business and get them involved early on and throughout our development process. Specifically, we're using AI a lot for things like design reviews, copy editing, loads to do with qualitative user feedback, also things like legal checks to now, obviously in the where we are, we have GDPR and other things to keep up with. So knowing what impact we are adding within legal and compliance in terms of what that means, it just, it massively speeds up our iteration cycles. So, you know, it lets us refine our product strategy, get some input on maybe it might be things like legal very early on in that process. So that time between, you know, where you're developing something to getting it signed off is you're minimizing those gaps between things getting done and it going to different people. So I think that's one of the main things that it really helps with, of getting that level of input throughout that process, which ultimately speeds things along another side of it. It really helps explore new ideas quickly, get different perspectives and do a lot of the groundwork where ultimately, you know, in a business art size, we can't hire loads of junior people like right now. So it just enables us to do those things a lot quicker and a lot of the groundwork basically in terms of how I my general advice to this kind of stuff is any business leader should be making it as easy as possible to play around with AI tools and making that as easy as possible to do it with things like enterprise subscriptions to anything so they're comfortable sharing data. And that's especially in small businesses, it's a lot more possible to do this stuff. And just working out what you can do as an individual within a business is, is probably one of the main things because the technology is advanced the whole time, so you can do more and more stuff. And there's loads of cool new tools like things like Craftful, which you can use product teams throughout the development process.

So looking forward into the future, obviously we're looking at a very quickly evolving space. Right. So as AI continues to evolve, how do you plan to keep refining your AI chat experience to further enhance the experience that your users receive from it?

Yeah, so continuous metric tracking is key to what I'm doing and what we're doing as a team. We're only refining Ziggy based on real interactions. So where we see opportunities and we're looking at what responses people are giving our AI in terms of both the questions they're asking, but also the responses the AI gives. That's getting reviewed the whole time to enable us to improve the model, but also looking at expanding into things like new problem spaces where we can add a lot of value for our users and even exploring about how it can fundamentally define how we're building the product experience as well, you're going to see. I think we're going to see startups who just build things in completely different ways because the way that incumbents are built, it just don't make. Make any sense. It's a lot easier to build and build a lot of interesting stuff. I generally say it's our approaches. We're very data informed and combining that with. With user focus that enables us to keep track on an ongoing basis of what how users are engaging with specifically Ziggy, our AI feature, but also doing it in a way that generally helps dog owners and know that in a IT in our, our stage of business that isn't necessarily all, you know, a part of that would be about improving what we have, but another side of it is solving new problems as well.

Yeah. So I think what I'm hearing is a lot of generative research for generative AI features.

Yeah. Yeah, it's really. Yeah. Massively important for us. I think. Yeah. That it's marrying those two things up is, you know, it enables you to jump when it makes sense to jump and act quickly when having that generative backing really allows you to play around with all this cool new stuff that's coming out the whole time.

That's great. So, Jack, thank you so much for being on the show today. I really enjoyed this conversation. So how does someone learn more about you, your thought leadership and all the work that you do at Zigzag?

Yep. So feel free to connect with me on LinkedIn search Jack Mitchell. There's thousands of us I think, but Jack Mitchell and Zigzag, you'll find me. I'm more than happy to chat about products, in particular B2C. I specialize in apps, but also have done a bit of web stuff before, so more than happy to chat about anything, products, love meeting product people. And yeah, please download Zigzag if you're a dog owner or especially a puppy owner. And DM me with some feedback. Let me know how it's going, anything you'd like to improve.

Fantastic. Thanks again, Jag.

Cheers, Michael.

[Michael Dominic]

Want to keep the conversation going? You can find the show [e-mail] podcast if you haven't already, don't forget to follow us on Apple Podcast, Spotify, Overcast or Google Play so you never miss an episode. And if you enjoyed today's show, please share it with a friend or leave us a rating and review on Apple Podcasts. And until next time, this is Insights Unlocked, an original podcast from User Testing.