← Design Better
Design Better
Nate Koechly and Matthew Darby: YouTube's UX Director and Director of PM on redesigning one of the world's most-used apps
00:43:22
In our conversation, Nate and Matt share how they developed predictive measurement tools to gauge user perception, why they pair visual updates with quality-of-life features like comment threading and improved video controls, and how their research process has evolved from measuring clicks to understanding satisfied watch time.
Loading summary
Transcript76 lines
- [00:00]
Nate Keckley
I talk sometimes about wanting to put the best frame possible around every piece of content. You go to a museum, every piece of art has a custom frame around it. And so what's our version of that? How can we make each individual piece of content, each individual user experience in the best possible light? Then for the users, how do you give them the most agency, the most empowerment within that place? When you mesh those things together, there's just lots of stuff that can emerge. It's just endlessly interesting.
- [00:29]
Aaron Walter
Redesigning one of the world's most used apps is no small feat, especially when that app is also the second largest search engine in the world. Of course I'm describing YouTube. Over the last four years, Nate Keshley, UX director at YouTube, and Matthew Darby, Director of Product Management, have been leading an ambitious effort to balance Google's metrics driven culture with the subjective challenge of making an app feel more modern. In our conversation, Nate and Matt share how they develop predictive measurement tools to gauge user perception, why they pair visual updates with quality of life features like comment threading and improved video controls, and how their research process has evolved from measuring clicks to understanding satisfied watch time.
- [01:17]
Eli Wilrick
We also dig into one of YouTube's most complex challenges, the algorithm. As Nate and Matt explain, what users say they want doesn't always match what actually makes them happy on the platform. They also discuss their work, exploring ways to give viewers more agency and control, including the possibility of using natural language to tune your feed. Both guests have a genuine passion for how YouTube enables deep expertise and niche interest to find their audiences. From 3D models of the Golden Gate Bridge to forest fire education from Northern California lookouts behind the algorithms and design updates is a platform where, as Nate puts it, when you give people a voice, the things they say are just Inspiring. This is DesignBetter, where we explore creativity at the intersection of design and technology. I'm Eli Wilrick.
- [02:01]
Aaron Walter
And I'm Aaron Walter. At DesignBetter, our primary mission is to produce work that helps people like you refine your craft, improve your collaboration skills, and get inspired by the creative process of others. If you enjoy what we do here, the best way to support us is to become a Premium subscriber@designbetterpodcast.com subscribe. We'll return to the conversation after this quick break.
- [02:38]
Sponsor/Announcer
DesignBetter is brought to you by WIX Studio, the platform built for all web creators to design, develop and manage exceptional web projects at scale. Learn more@wix.com studio.
- [02:54]
Aaron Walter
And now back to the show. Nate Keckley and Matt Darby, welcome to Design Better.
- [03:04]
Matt Darby
Thanks for having us.
- [03:05]
Aaron Walter
You guys have been working on YouTube for quite a while, both of you. YouTube is one of my personal favorite platforms. It's kind of changed my life in many ways. I learned so much from it. I probably spent too much time there like most people. And you've been working on a lot of new release stuff as of late. Could you give us just like a quick walkthrough of some of the ideas that have recently come to pass?
- [03:30]
Matt Darby
Yeah, Nate and I have been working for four years now actually. We've been doing these bundles, we call them internally and they have like a city namespace so we name them alphabetically. So the first one we did was Amsterdam in 2021 and we just launched Delhi this year. The sort of genesis of that program was the app had just got out of date. I think everybody in the company sort of recognized that the design was lagging competitors. We'd also heard that feedback from viewers as well. We did like a big UXR study that sort of got into all the details of how we were lagging behind. And so we went on this four year journey and the goal was to modernize the app. So like a pretty sort of subjective goal really. And Google doesn't always do well with subjective goals. And so each year we've got together like a collection of visual improvements that we predict are going to really improve users perception of modernity. And we have like a predictive measurement score that we developed for that purpose and we've released them in a bundle together because we think it makes sense for visual design to go out as a cohesive whole. And we've paired them with what we call value features as well, like sort of quality of life improvements for viewers. So that when we go to market we can say, hey, there's a redesign and you can zoom into the video and you can, you know, hold down on the video and it plays at double speed. And so that's always been really nice for a sort of just a way to tell viewers how the app is changing. People are not always thrilled when it's just visual design, you know, and there's no sort of functional change. So as I say, we've done four of these now. The latest one that went out just two weeks ago actually, we updated all the iconography so you'll see like a whole fresh set of icons with some nice little branding cues in there. There's some sort of angles that pick up the triangle that you see in the word mark. So they're sort of nicely subtly branded to be YouTube. We overhauled the player design so the player is much brighter than it is now. The controls obscure the content a lot less. We did some nice color sampling throughout the app, so if you open panels like the description, you'll see some color sampling based on the video. One quality of life value feature was comment threading, so you can see comments presented in threads, which is pretty cool if you're a commenter and a comment reader. There's a new saving flow. We updated some of the motion in the app, we created some new search cards for musicians, so a nice sort of bundle of changes. We did predictive measurement for all of these features before we launched them and then we do a confirmation measurement when we've actually launched them in the app to make sure that we did indeed increase viewer perception of modernity. And so that's what we've done last four years.
- [05:58]
Aaron Walter
So Google being Google, I imagine that there's a fair bit of research quantitatively that goes on behind the scenes and there's probably some qualitative research as well. Maybe Nate and Bat, you can both speak to your research process behind the redesign.
- [06:13]
Nate Keckley
I mean, I think first was to understand a bunch of objectives. What does it mean to be more modern, what does it mean to be more simple, to create a more beautiful stage for the content, to help content shine, to increase efficiency and a bunch of other objectives, and those are themselves research derived. Understanding how users perceive all those things, how users sort of comparatively value all those things. From there, just lots and lots of different methods. So predictive, sort of side by sides, this versus that against a bunch of sort of structured scales. Once things start getting into users hands, diary studies, it depends on the project, if we're putting things in the right place. Lots of information architecture methodologies to where users expect features to be. And then as we start to scale up, we look at a lot of those constituent characteristics. How organized are things, how clean are things, how appealing are things across the world, around the world, in different markets for different user segments and try to make sense of all that together. At the end of the day we're always focused on creating a better user experience, helping users have more long term joy, long term satisfaction. And so everything does start with understanding the user and we just keep striving to find new methodologies to do that. We have a lot of metrics which are proxies for how users are engaging with things and valuing things, but we're always trying to get better proxies and get closer to true User intent, that true user feeling. And so continuing to innovate on the research methodologies that give us that is a big part of it.
- [07:45]
Matt Darby
I think the UXR methodology that we created here was actually the most crucial part of this four year project. The subjective part of design. Whether something is beautiful, whether something is modern, those are the kinds of qualities of design that Google just culturally doesn't do all that well with because it's very difficult to really know what success looks like or how you would measure something like that. And so we put together this program where we measure each of the proposed changes before we build them and we get a scorecard. And the scorecard's pretty sophisticated, actually. You get like different demographic breakdowns and sort of gender breakdowns and so on. And we can also iterate on designs. It's very useful for being able to get qualitative feedback on like, well, I didn't like this. And we really like that other part of the design that was really useful to get a slate of features and then this confirmation once we'd launched to see actually, you know, once we put them all together in the product, we measure this in a holdback. And that was really essential for convincing the organization that this was a thing that's worth doing. In a culture where it's very metrics based, it's very engineering based, having that level of rigor, rigor that was based in asking viewers about how they felt about these design changes was like super essential for us to get the sort of organizational Runway to run the program.
- [08:58]
Aaron Walter
I think this is interesting, like if it can be quantified, then it's true. And if it cannot be quantified, then it is not true, which to me seems like that's just false because we go through life every day and there's so much that's qualitative and we know those qualitative things to be wonderful and true. Can we quantify the quality of walking through a forest that that is good for our human experience? Probably not. We could try. But is it useful? Still good and important to have. I want to return back to that original premise that you had, Matt. You said there was some feedback from the user base that said the app feels out of date. That's something that cannot be quantified. But yet that does boil down to business metrics. How long are people staying on YouTube? Are they watching multiple videos, are they staying with videos, et cetera? How do you frame it? Or do you frame these sorts of qualitative things in a competitive way to say like, hey, we're looking at these competitors and here's what they have. We don't have their metrics necessarily, but we know that this is an important thing for us to consider because YouTube, it's a consumer facing product, it's a consumer facing platform. How do you frame that with competitive metrics?
- [10:17]
Nate Keckley
We do consider that. I mean, first and foremost, you know, we're interested in letting viewers decide what's working for them, what content they want to watch, how they want to watch it, which features they want to use. We keep going back to our roots of being user first, viewer first, creator first, but then it's also interesting to learn from the industry. Of course, these users are using many products also. And so that's part of the experience. And so when we do a lot of research, we do assess competitors also and ask users about those differences and see what they associate with us and what they associate with others and strategically how we think about that. And then maybe more meta or zooming out a little bit. The tech is such, and these apps and media and content is such a part of culture that we aren't just competing on raw technology. You're competing on the role you play in a user's life and what they turn to you for. And so you are competing on the quality of the experience in addition to the quality of the content and how well the tools fit into people's lives. And then one interesting thing we did hear from users in a bunch of that research was they do expect the products to be progressive and to keep changing. You know, you look at cars, they redesign every year. Fashion has two Runway seasons a year. Change is part of the expectation. And we really heard users telling us that too, to keep up, to be progressive, you know, the latest and greatest. So it's both. We look internally at our users first. We're inspired by a lot of the great things happening, you know, across the industry and viewers lives and users lives in different ways. And then we try to listen to what they're telling us.
- [11:46]
Aaron Walter
Matt and Nate, you've both been at Google a long time. I think almost 30 years combined maybe, if I'm doing the math right from your profiles. And I realize not all your time has been spent on YouTube, but YouTube itself has been around for 20 years. Maybe you could each give your personal perspectives on how the product has evolved over the years. The cultural impact, I mean, it's kind of come about in unexpected ways. It's become a music platform, it's becoming the most popular podcast platform, it's becoming an education platform. So yeah, just curious, just given the time that you've spent at Google and with YouTube. Each of your thoughts there?
- [12:22]
Matt Darby
Yeah, no, I've spent almost my entire Google career at YouTube. Actually I joined not so long after the acquisition. In retrospect, I'm sure at the time the people that were acquired, it was like a year or two. So yeah, it's changed at all. Recognition I think, I mean both the product and the organization as well. The organization is completely unrecognizable from what it was back then. It was, I think there were like about 300 people maybe working on YouTube when I first joined, you know, and of course the technology as well. Yeah, so it's very different. I don't think anybody really exactly foresaw what it was going to become as well. I mean it's a very organic process. The ecosystem in particular has sort of grown in this organic way that was difficult, maybe impossible to predict looking back. It was just a lot of sort of throwing ideas at the wall and seeing which one stuck to the wall really. I wish I could tell you that there was some grand plan or that we had like a much greater vision for the platform and the products and it was just a website. Back in 2008, 2009, my first few years as a product manager I was like mostly pretty embarrassed about the product. It looked pretty low rent for a long time as well. So it's really been a journey to see it evolve from that into now. I think an absolutely world class organization. A product that is important in the world and I think is a force for good in the world without getting too misty eyed about it. So seeing that transformation has been a real treat.
- [13:36]
Nate Keckley
I think it's been fun to have our 20th birthday and to think about that journey and to think about sort of what's next. And you know, I think I've always loved Google's mission and I feel it's pretty timeless And I think YouTube's is too and I think they dovetail nicely. So you know, if Google is making all the world's knowledge useful and accessible and YouTube is about giving everybody a voice and showing them the world and you know, I think that mission continues to sort of inspire us. But it's been wild to see something that started out me at the zoo turn into something that is redefining how the world connects and consumes and learns and watches and listens and all the other things that we've sort of grown into. We talk a surprising amount about the mission and trying to do that. And again, it's been nice to reflect on 20 years and to think about the next 20 ahead and stay energized to keep facilitating that evolution of the world.
- [14:30]
Aaron Walter
It's been fun to watch YouTube evolve in unexpected ways. It's unique in that it's a product that has such a dedicated user base that they use the product in unexpected ways. You probably didn't expect that people would use YouTube as their music platform and then that spins off another product, which is YouTube Music, and it's now the biggest podcast platform as well. How do you find these kind of user generated what seem like adjacencies or adjacent use cases and then respond to those and bring that back into the product strategy?
- [15:11]
Nate Keckley
Well, I mean, it starts with creators and the things that they're doing and the way that they're engaging with their audiences. And so Neil talks about just wanting to build the best stage for that creativity and for that connection. And I think that's where all the creativity comes from and where a lot of the insights come from. There's tons of just media formats that wouldn't have been greenlit in traditional contexts. But a creator's onto something and a viewer resonates with it. And now you've got all these new formats unboxing or this or that, that people sort of didn't see. And so really we're just trying to let that happen and then facilitate it. And so we focus on just sort of the core user journey, which is find content, immerse in content, dive deeper into the content of the community around it. On the one hand, it's quite straightforward, but that gives rise to all these specialized experiences and these different user journeys and different sort of sub segments within that. You know, we did some really interesting research a few years ago on what users do while they're watching and trying to think through. Okay, well, what does it mean when you're watching a video and you're cooking and your hands are dirty? What features do you need at that moment? When you're jogging, listening to a podcast, what features do you need? And so that same sort of core user journey exists in all these places. I talk sometimes about wanting to put the best frame possible around every piece of content. You go to a museum, every piece of art has a custom frame around it. And so what's our version of that? How can we make each individual piece of content, each individual user experience in the best possible light with the most capability? And then for the users, how do you give them the most agency, the most empowerment within that place, when you mesh those things together, there's just lots of stuff that can emerge. But it's tricky to allow all that specialization to emerge while still being tractable and clear and simple and easy to use. It's just endlessly interesting.
- [17:05]
Aaron Walter
Matt, your role as a product manager. I'm curious how you think about a product that's been around for so long. And products naturally tend to accrete features and grow and spread and occasionally you have to assess and say, do we need to cut this or rethink this? What's that process been like from your side?
- [17:22]
Matt Darby
That is a challenge. Yeah. I mean, it's particularly a challenge because Google famously has this very bottom up culture. And so we have lots of teams that are broadly empowered to launch stuff in the apps. And part of my job actually is to build the frameworks and the patterns and the reusable components to allow teams to do that easily. But that does mean that you end up with a lot of stuff in the app. There's lots and lots of features and some of those features end up taking off and some of them don't. I think we could definitely be a lot better at identifying features that are not doing so well. We were just discussing this last week actually, and what do we do about that? The conceptual model just keeps getting more and more and more complicated unless you do something about it. And I think that does create the opposite of what Nate was talking about earlier, like the idea that it can be a simple app and a beautiful app, and the more craft that there is and the more complexity there is in the conceptual model, the harder that is to achieve. It's really a lot of work. It's not easy usually to deprecate features because there's usually a community that loves that feature and will howl mightily at the suggestion of its removal. But it is a necessary thing to do over time and we need to get better at it, frankly as well.
- [18:28]
Nate Keckley
I think there's the removal and there's also just trying to increase relevance for individual users. And so when we thought about years ago, we tried to understand modernity and what users wanted in this era, they wanted clean and distraction free, largely. But because there are so many different user journeys, it's not so much about what you have and what you don't have, it's how do you have the right thing at the right time in an efficient, understandable, sort of repeatable way. You know, more personalization, more tailoring, more on demand is also part of it. In addition to tending the garden Sort of rigorously over time.
- [19:05]
Aaron Walter
Google has this term cuj customer user journeys. Is that something that guides most of the work that the design team is doing?
- [19:14]
Nate Keckley
Yes, yeah. Critical user journeys. Customer user journeys. Yeah. Like what's the flow through the app? Where's Main street, where are the cul de sacs? A bunch of different metaphors you can layer on. But yeah, if you're trying to design for the user, first and foremost you have to understand what journey they're on, what their motivations are, what their context is. And I think that's excitingly so much more powerful. There's so much more potential there sort of emerging now. It feels like as we can continue to tailor the experience for individuals. But yeah, we do focus a lot on that. And you know, I think the fundamental viewer user journey is like I said before, immerse, dive deeper, move on, find content and then there's subsets of those. The list gets really long, really quickly. But it also can get that clear. But yeah, we talk a lot about user journeys and the intent of the user at a given moment and how to help them achieve their goals.
- [20:03]
Matt Darby
Yeah, we spend a lot of time thinking about those journeys. These three goals of watching. You want to be immersed in the video, then you want to dive deeper and find out more stuff about the video. We spent several years investing in the video player and we actually launched like a ton of features like Video Zoom, where you can zoom into the player chapters, a feature we call Speedmaster, where you can hold down on the player and have it play at double speed, like extra speeds for premium users. That whole suite of features that work together as a system. We spent, you know, many years like sweating the details on those. And that's really in service of this be immersed user journey that you really want to find exactly the right point in the video. Heatseeker, which is like the little heat map that you get as you sort of scroll around and you can find interesting points in the video that other people are watching. And that was a multi year journey that was just to make that one critical user journey that much better. My favorite feature in that suite is we call it internally double tap to seek where you like double tap on either side of the player to like skip forwards and backwards 5 seconds. We launched that several years ago now actually and it's been copied a lot. I think most good video players have it a. I think it's a killer feature because it was really difficult to seek inside the video before. It's super tedious to use the scrubber Bar, which is the little bar at the bottom. And B, it was actually a nice demonstration of when the bottom up culture is a real benefit to doing product development because it was one of our designers, she came up with the original idea for using the double tap on either side. And the insight was that it was on different sides at different speeds. Seems obvious in retrospect, but we were actually very constrained. We didn't want to use Swipe because we had another idea for Swipe, but he already had tap. So she had these kind of leftover gestures and managed to repurpose them. And then somebody else came up. There's another designer on the team that came up with, well, there's a product manager actually came up with the idea that you should be able to mash and so that you can just keep tapping and it would just advance a long way. And then another designer came up with the idea that you'd have to have the indicator to show you how many seconds you were advancing. And then there was an engineer that reached out to me as we were getting near launch and he said, how close does it have to be? Does it have to be exactly 10 seconds? And I was like, well, not necessarily. It's more, you know, you're trying to get back to what was just said, you know, like, I missed something. What was that moment that we just said? And so he introduced this idea that we would snap to the nearest keyframe. And this is a detail about how video encoding works. And every 10 seconds we send a keyframe. And the keyframes are just a full representation of the video frame. And they're much faster for the client to decode and put on the screen, whereas the intermediate frames, you have to do some math to figure out what the differences are. And so it just speeded it up by a few milliseconds. It just made it a little tiny bit snappier by snapping to the keyframe. And actually, if you use the feature, you'll find that you sometimes jump five seconds, sometimes you'll jump 15 seconds. And so anyway, it was a nice synthesis of all of these different disciplines and different ideas from people across the team. And it went on to be a gangbusters feature. It gets used a ton in the app.
- [22:54]
Aaron Walter
What are the challenges of redesigning and launching new features on a platform that is servicing billions of people every day?
- [23:03]
Matt Darby
It's an enormous challenge. Every feature that we launch, we launch in an A B test. I mean, it's the only way that you can launch at scale. So even the smallest thing Even if it's a bug fix, sometimes we'll just test it in an A B test to see what happens. Sometimes we even do things like we have the code available for the feature and we'll just launch the code without even switching the feature and run an A B test on that just to see what effect putting the code in the client had. We're always trying to measure the effect on the ecosystem. And also there's a lot of tragedy of the commons dynamics that can work out if you don't pay attention to them. Like crash, you can slowly, slowly make the crash rate get worse and worse and worse with lots and lots of features. So lots of system health metrics, for instance, we track in the client to make sure that the client's still in a healthy state. And yes, there's like thousands and thousands of metrics, way, way too many metrics. And it's putting a new, something brand new into that dynamic nature of the ecosystem and seeing how things change inside the ecosystem. Quite often there are these zero sum trades. If you get some engagement here, it comes at the expense of some other surface over here. And so it's almost always a pretty delicate balance. And there's a lot of analysis that we do and a lot of metrics that we look at.
- [24:18]
Aaron Walter
We'll return to the conversation after this quick break.
- [24:24]
Sponsor/Announcer
Design Better is supported by Masterclass. I am a lifelong learner. It's been decades since I've been in school. I remain in life's classroom curious about design, technology, science, history, filmmaking, architecture, cooking, AI and so much more. Masterclass is an essential part of my learning. I've gone deeper into each of these subjects by taking masterclasses with people like Herbie Hancock, Frank Gehry, the late David lynch and Anna Wintour, who taught a wonderful class on leadership and creativity. And I'm excited to jump into Elaine Winteroth's Masterclass on designing your career, Kim Scott on having hard conversation and Melanie Hodgkin, strategic decision making.
- [25:08]
Aaron Walter
Masterclass.
- [25:10]
Sponsor/Announcer
The brightest minds are making my mind.
- [25:12]
Aaron Walter
Brighter and I love it.
- [25:13]
Sponsor/Announcer
Masterclass has in depth expertise that you can tap into anytime and any place. You can access Masterclass on your phone, your computer, your smart tv, or even just listen in audio mode. I love Masterclass and I think you will too. Right now, our listeners can get an additional 15% off any annual membership@masterclass.com designbetter that's 50% off masterclass.com designbetter.
- [25:45]
Aaron Walter
And now back to the show.
- [25:52]
So we're just about half an hour in and we have not yet mentioned AI, which is probably refreshing for a lot of people. But I'll break our streak there and just ask curious mostly about how AI is affecting search. And since YouTube's the second largest search platform at the moment, it's obviously disrupting all kinds of search. But I wonder how you're thinking about it from a product standpoint and a discovery standpoint for your users. And what do you kind of see that trajectory like?
- [26:20]
Nate Keckley
Yeah, it's super interesting times. I zoom out a little bit first and think about the progression of sort of the Internet eras through them all. It's about getting more capabilities for users and about technology that gets you closer to true user intent and true understanding of what a user's trying to accomplish. And so specifically, language really reduces that gap. You know, in the web 1.0 days, everything was done through form elements. And Then in web 2.0 you could directly manipulate things. And then in mobile you were freed from tables and grid based layouts and you could have sort of bespoke designs that more closely matched the user's expectations, mental models. And I think now with language and with presumably reduced cost of sort of interface development over time, it enables us to have a better picture of what the user's actually trying to accomplish, and then hopefully the product design capabilities to meet that even more closely. The world learned how to talk in a pretty optimized way with short queries. And now that language is more accessible to more people, we're seeing more expressive queries and people bringing new questions forward to us. So I think that's really interesting. What we can do with that is we'll see. But you know, I think when users have higher expectations of us, it just gives us new moments to meet. And so maybe when you're trying to find the best thing to do on a trip or the best way to fix this thing, it just lets us be more efficient, we hope, where we can identify more clearly what the users want and then find that information in new places. So it might not just be from a single video. We might be able to bring a set of videos to a user in a new way and explain it. So that's one thought. Second thought, maybe that goes with it is what is video good at and how can we make that work better? Sometimes I think about the history of text and all the tools that have developed around text and all the things you can do with text. You can snip it and you can skim it and you can morph it in different ways that are more challenging today with video. And so what is a good snippet, what is a good narration? How do you help explain the context around the videos to a user in new ways? And so I think it's pretty early days, but to have so many new raw materials available I think is pretty exciting. I think as a result, we also do see users bringing more queries to YouTube and more query diversity. And users come to YouTube for entertainment, but they also come for a lot of information and a lot of connection. And we're starting to see that. And I think with more expressive design and with the ability to sort of craft a better response to each query, we're super excited about it.
- [28:57]
Aaron Walter
How are shorts changing the way you think about the product?
- [29:02]
Nate Keckley
Well, one thing they've told us is that users value for some journeys, efficiency is really important and getting to the essence of the answer or getting to the essence of the joke or the story or the insight. And so I think shorts and short form video in general have increased our thinking around content efficiency in getting people to the heart of what they're looking for even faster. So that's one dimension. I think another dimension is the amount of participants. We continue to lower the bar to be a creator, to be creative, to put something out in the world. It can be a big project to put together a long form video. And so more people are getting to experiment with new ways to be creative and to communicate thanks to short form video. So I think that's like a dynamic that will play out for a long time.
- [29:52]
Aaron Walter
Presumably right now they're sort of squished together where you get like a block of shorts and then a block of long form videos. Do you think about those as different? Like is that a fork in the road where maybe there's two different pathways for users?
- [30:06]
Nate Keckley
Well, I think there's a lot of different user intents. And so sometimes you're looking for a specific video and that's sort of one path and other times you're really looking for a set of videos or just for, you know, a scrolling sort of content experience itself more than an individual piece of content particularly. So I think that is a pretty distinct, or it can be a pretty distinct sort of user intention. But it's not the only one. I mean, sometimes people want to lean in, sometimes they want to lean out, sometimes they're looking to be really active and guide like a research session and poke around and curate and find a lot of things. And other times, you know, they're looking for a different experience. So I don't think that's the only fork in the road, but I do think that is a distinct user journey at times.
- [30:48]
Matt Darby
We have done quite a bit of experimentation with mixed feeds as well, where we mix what we call VODs, non shorts and shorts together, and they can actually in some circumstances, perform quite well. It's definitely something we're looking at. It's not completely straightforward, but definitely something that we're working on and we're aware of.
- [31:07]
Aaron Walter
How do you think about prioritizing the different user groups that make use of the platform? Obviously, there's probably some, definitely some business decisions that are made there. But also just from your own perspective as product and design leaders, what does that prioritization look like?
- [31:22]
Matt Darby
I was thinking about this, actually, when Aaron was asking earlier about all of the different niche uses and niche user journeys. And Nate and my teams, we really think of the broad set of users, so we're trying to build features that are just generally useful across the platform. And most of the teams that are adjacent to us are trying to keep the health of the ecosystem at large in really good shape, and that allows these different communities to flourish. As Nate was saying, there's a team that just looks at podcasts and they're building podcast features. And yes, we have to figure out how to sort of fit that in such that it feels connected and integrated into the app. That can be a challenge. Same with music features, same with premium. Members have their own sets of features. So that's an ongoing discussion. It's not the most straightforward thing, actually, to sort of bring those in at the right times, but that's what we aspire to do.
- [32:09]
Nate Keckley
I guess I would just add that. Well, I mean, there's a lot of people on YouTube and so we look at lots of slices of the user group and we do, you know, focused user research into different segments, you know, from time to time. And a lot of that is just to track the health and to try to identify any blind spots or any areas where we're not serving people as well as we could. It is important, I think, to be diligent and look at different segments and look at different sort of usage behaviors and really use that as a way to sort of triangulate what's working and what isn't and where sort of the opportunity is. It's less about prioritizing specific groups and it's more about, I think, prioritizing the aggregate health. It's also interesting to see different segments teach us different things. And so, you know, the way podcast consumers consume content is a little bit different than the way sports people do. And all of that could be inspiring. Maybe one more interesting dynamic I think is looking around the world and the Internet and YouTube plays a different role in different markets in the same way. We saw sort of mobile have different characteristics on different continents and different. It's sort of when different markets matured and what technology was sort of most prevalent right then. So it is interesting. We see interesting information seeking journeys in India that we don't see as much in some other countries, for example. And so that just reminds us just to keep looking and to keep trying to understand more at this scale. There's just so much diversity that on some sense it aggregates out or averages out, but that's a risk. And so it's important to keep drilling in everywhere and understanding.
- [33:41]
Matt Darby
Yeah. One thing I'd say as well that might be surprising is in general, like Nate and my teams, I mean we measure the business impact, but that's not a goal for our teams. Like we don't have a revenue goal. We're just thinking about how can we deliver value to viewers. And that's very intentional. There's a separation of church and state now. We work with the ads teams quite closely and mostly our interests are like totally aligned. Right. We want to do the right thing for viewers, but in general we want to just deliver as much value for viewers that creates a healthy ecosystem from the viewer point of view and that makes a bigger opportunity for the teams to monetize in different ways.
- [34:15]
Aaron Walter
I'm curious how you think about the algorithm, because the algorithm in YouTube is quite good. I discover so many wonderful things through the algorithm. But there are also times where it's like I'm stuck in a loop and I want to kind of break out of that. There's a button up at the top new to you. And I've often thought, like, I wish there was like I could dial it up to 11 where I could really just scramble the whole thing. How do you think about the algorithm and the experience it creates for people?
- [34:46]
Nate Keckley
One way to think about it is that the algorithm sort of is the audience. Its job, when it does it well, is to represent what that user wants. And, you know, I think we believe that there's content on YouTube for everybody and it's our job to help find it. So at its basic level, it's that. But it's interesting to try to design for these probabilistic and things that aren't sort of deterministic. It's not just one in one out. And so one, what are all the right signals and what are we sort of optimizing for? We want long term satisfaction, long term value, long term satisfaction. And so our understanding of that has changed a lot over the years. Early on we were measuring clicks and then watch time, and then satisfied watch time as a subset of overall watch time. And we keep trying to figure that out, and then we try to figure out the things that detract from that. So content that might be totally fine and beloved by one user might be off putting for another user. So we have to figure out how to model things like that. And then users have other characteristics. You know, they want freshness, they want novelty, they want tried and true. And so how do you manage all of those things together when you really just have a flat list to present by default at the end?
- [36:00]
Aaron Walter
Can I give you a use case?
- [36:01]
Nate Keckley
Yeah, please.
- [36:02]
Aaron Walter
So lots of people use YouTube to do research, especially on products. Researching a car that you might buy or some other thing, and then you've made that purchase, you are no longer researching, and yet you just get the endless feed of here are all the cars. Those sorts of scenarios where what I wanted has changed and now it's totally different. Are those parts of your conversation around.
- [36:26]
Nate Keckley
The algorithm that's definitely part of the conversation and the time of day and the device you're on. You probably want a slightly different piece of content on Friday night on your couch in the living room than you do on the road. And it's really tricky to identify those sessions. And so it's an interesting blend between explicit and implicit. How much can we just predict and how carefully can we pay attention and respond versus when is the right answer just to ask the user and to listen to that. We also see the dynamic where what people say isn't always what they do. And so you have to balance those two things out. But in general, we're trying to give more agency and more responsiveness based on increasingly nuanced understandings of what satisfies a particular journey. And so in the case of music, maybe the priority is to give you five songs in a row without you having to change the channel, whereas that wouldn't be the goal at all on a research journey where we want to give you more diversity and let you sort of hone in on it. And so it's tricky to come up with UIs that are simple enough to get out of the way, but still enable you to sort of get what you want. And we have a long way to go that's an endless Quest, that's something.
- [37:37]
Matt Darby
We hear from users all the time actually, is that they would like the ability to tune their algorithm more precisely. It is something that we're working on in sort of two ways, giving viewers insight into what we know about them. The inputs into the algorithm, the things that we know that they're interested in. We are also exploring ways where you could tune the inputs. It's really not straightforward. As Nate said, quite often viewers, they want one thing and actually you can show empirically that that's actually bad for their usage on the platform. So that's a big dynamic that we have to navigate. But it's definitely something that we hear. It's definitely something that we're interested in and working on.
- [38:13]
Nate Keckley
And it's an area, I think, where we're like, have renewed interest. The sensitivity to the signals, like the things that we can measure is increasing. But more importantly, I hope that we're starting to see some early signs that user expectations are changing. And so the ability to tell us how to change your feed with natural language instead of with a bunch of bells and whistles is interesting, you know, certainly for some users at least. And so it does feel like the tools we have to approach this problem are changing really rapidly right now.
- [38:43]
Aaron Walter
Do you guys remember that episode of Seinfeld where Kramer is pretending to be Moviefone? And George and Jerry, they would call in and try to get answers and finally it was just like, why don't you just tell me what you want?
- [38:58]
Nate Keckley
Exactly.
- [38:58]
Aaron Walter
That's basically what you're describing.
- [39:00]
Matt Darby
That's right, yeah, yeah.
- [39:01]
Nate Keckley
And it's interesting, you know, users do so much on YouTube that it does reflect them in some ways or can be a nice reflection. And so I think that's an interesting dynamic.
- [39:10]
Eli Wilrick
Also, what's your favorite YouTube channel that.
- [39:14]
Aaron Walter
You'Ve run into lately? And if it's not a full channel, it can just be a video or podcast or music too.
- [39:19]
Matt Darby
Mark Lewis, actually, it's very obscure. He's like a middle aged guy doing fitness stuff and building his home gym out in the country in England. And so, I don't know, has a lot of resonances for a middle aged guy who was also working on his fitness. And he has a very dry sense of humor, so I enjoy his content.
- [39:35]
Nate Keckley
Number one is animographs. This guy does 3D modeling, like he just did the Golden Gate Bridge. He does like three or four videos a year, works on them for months and months and months and then models it and then flies around and describes it all. And so I Drive across the Golden Gate Bridge a lot. I learned a ton. It's like an hour and a half video, and then he did another hour behind the scenes. And at the end of it, he just looks at the camera and just thanks the audience for, like, enabling him to be so passionate about something and find these. So many people that love the same thing. So to have a platform where that level of passion and craft and nicheness can find such success. And that's a different way of YouTubing than you think of, you know, the weekly Vault vlog or whatever. And so I love that one. One other one is called the Lookout for those in Northern California. It's a person in the forest service that knows everything about, like, forest fires. And it's an example to me of, again, somebody's, like, depth of expertise, being there for the world, and also just the quality of the information. Like, I've learned more from them about the climate around here than I have from any more traditional source. Case after case, when you give people a voice, the things they say are just inspiring. And those are two, I think, just sort of only on YouTube. Examples that I watch quite a bit.
- [40:50]
Matt Darby
Man. Yours sounds so much more wholesome and educational than mine. You're gonna have to give me those channels, Nate.
- [40:55]
Nate Keckley
There's more where that came from. There's something out there for everybody.
- [40:58]
Aaron Walter
I want to add one to your list. Have you guys seen kurzgesagt?
- [41:02]
Matt Darby
Yeah. Yeah, That's a favorite. Yeah.
- [41:03]
Aaron Walter
Yeah, I love that one. So good.
- [41:05]
Matt Darby
Yeah. So well done.
- [41:06]
Aaron Walter
Yeah.
- [41:06]
Matt Darby
That's the difficult thing, you know, having cast my mind back to sort of 2008 and 2009, to imagine that a channel like that could exist with that level of polish and production value, it's astonishing, really.
- [41:18]
Aaron Walter
Well, Nate and Matt, congratulations on the release. Thank you for making YouTube. It's just a wonderful resource. I love it so much.
- [41:25]
Matt Darby
That's fine. It was just me and Nate, by the way, I hasten to add. So you're definitely thanking the right guys. Yeah. But it's nice when people say that. Actually, one of the things that has made me stick around at YouTube, as well as it being a consumer product that I love, and it's such a privilege to work on a consumer product that you love yourself. But, yes, it's common when people ask you what you do, it's very common for people to say, I love YouTube and get a lot out of it. And that's pretty special, actually.
- [41:49]
Nate Keckley
That's great.
- [41:50]
Aaron Walter
Thanks for being on design. Better.
- [41:51]
Nate Keckley
Thanks for having us.
- [41:56]
Aaron Walter
This episode was produced by Eli Woolery and me, Aaron Walter, with engineering and production support from Brian Paik of Pacific Audio. If you found this episode useful, we hope that you'll leave us a review on Apple Podcasts, Spotify, or wherever you listen to finer shows. Or simply drop a link to the show in your team's Slack channel designbetterpodcast.com It'll really help others discover the show. Until next time.