20Product: How Duolingo Build Product 10x Faster with AI | Duolingo's Biggest Lessons on Paywalls, Push Notifications and In-App Purchases | Why Small Teams are the Future of Product | Why PMs Will Become Extinct with Cem Kansu, CPO @ Duolingo

Summary

Episode Summary: The Twenty Minute VC (20VC) – "20Product: How Duolingo Built Product 10x Faster with AI"

Release Date: June 20, 2025

In this insightful episode of The Twenty Minute VC (20VC), host Harry Stebbings engages in an in-depth conversation with Cem Kansu, Chief Product Officer (CPO) at Duolingo. The episode delves into Duolingo's innovative product strategies, especially focusing on how the integration of Artificial Intelligence (AI) has accelerated product development, enhanced user engagement, and driven revenue growth. Below is a comprehensive summary capturing the key discussions, insights, and conclusions from the episode.

---

1. Introduction to Cem Kansu and Duolingo’s Product Strategy

Harry Stebbings opens the conversation by introducing Cem Kansu, highlighting his pivotal role in shaping Duolingo’s product strategy, leading features like Duolingo Math and Duolingo Music, and steering the company to become the top-ranked education app globally.

2. Evolution of Duolingo’s Product Team and Monetization

Cem Kansu shares his journey at Duolingo, emphasizing the evolution of the product team from 50 to 850 employees and the shift from a pre-revenue to a revenue-generating model.

"[05:55] Cem Kansu: It’s been a great journey."

He discusses the initial challenges in monetizing Duolingo without compromising its mission of providing free education. Initially resistant to ads, in-app purchases, and subscriptions, Kansu highlights the strategy to introduce these revenue streams without hindering user experience.

3. Design Process and Prototyping at Duolingo

The discussion transitions to Duolingo's design processes, where Kansu outlines the iterative approach to product development, leveraging tools like Figma for design and rapid prototyping to test and refine features.

"[09:53] Cem Kansu: So depending on the size of the change, we might do extensive prototyping."

He illustrates this with the development of the "Video Call with Lily" feature, showcasing Duolingo's commitment to pixel-perfect design and user-centric features.

4. Leveraging AI for Accelerated Product Development

A significant portion of the conversation revolves around the integration of AI in Duolingo’s product development. Kansu explains how AI tools have enabled the team to prototype ideas swiftly, iterate based on feedback, and launch new features like the Chess course in a fraction of the traditional timeline.

"[13:12] Cem Kansu: It’s way shorter and way less people."

He emphasizes the importance of having a skilled team adept at using AI tools, which allows for smaller, more agile teams that can execute innovative ideas rapidly.

5. Future of Product Teams and the Role of AI

Kansu shares his vision for the future of product teams, predicting a convergence of roles where engineers, designers, and product managers become hybrid roles empowered by AI.

"[16:17] Cem Kansu: They will converge in the sense that they'll be a lot more capable of taking over each other's jobs."

He advises aspiring product professionals to become proficient in AI tools, advocating for an "AI-first" approach to stay ahead in the rapidly evolving tech landscape.

6. Monetization Strategies: Ads, Subscriptions, and In-App Purchases

The conversation delves deeper into Duolingo's monetization strategies. Kansu discusses the cautious introduction of ads, ensuring they are tastefully integrated to avoid disrupting the user experience. He explains the transition from free access to a freemium model, where core learning remains free while premium features like "Duolingo Max" offer enhanced functionalities.

"[29:43] Cem Kansu: That's lesson one: things change."

He shares lessons learned from monetization, emphasizing the importance of maintaining a high-quality free experience to drive organic growth and user loyalty.

7. In-App Purchases and User Engagement

Kansu elaborates on Duolingo's in-app purchase mechanics, such as the virtual currency "Gems," which allows users to buy features like "Streak Freeze." He underscores the importance of offering purchases that enhance, rather than detract from, the user experience.

"[50:29] Cem Kansu: I think people do really aggressive things and that ends up hurting the experience."

He advises against aggressive monetization tactics that can erode trust and harm retention, advocating for user-friendly and value-driven purchases.

8. Measuring Success: Retention Metrics and User Engagement

Retention emerges as a critical metric for Duolingo's success. Kansu explains how daily active users (DAU) and retention rates are pivotal in assessing user engagement and product effectiveness. He highlights features like streaks and leaderboards that foster daily habits and competitive spirit, thereby boosting retention.

"[51:56] Cem Kansu: Retention is everything when you think about building product."

Kansu discusses strategies to enhance retention, such as personalized push notifications and engaging feature designs that reinforce daily usage habits.

9. Impact of AI Tools on Engineering and Productivity

The episode touches upon the realistic impact of AI tools on engineering productivity. Kansu provides a balanced view, suggesting that while AI tools like Cursor enhance average engineer productivity by approximately 10-20%, they are not yet transformative enough to create "10x engineers."

"[22:23] Cem Kansu: I think it's maybe in the realm of like 10 to 20% more productivity of an average engineering capacity."

He cautions against the hype surrounding AI, emphasizing the need for practical adoption to realize tangible benefits.

10. Future Revenue Streams and Brand Expansion

Looking ahead, Kansu discusses potential future revenue streams for Duolingo, such as merchandise, leveraging the brand's recognizable characters like Duo. He aspires for Duolingo to evolve into a multifaceted entertainment and education brand, akin to Disney.

"[45:11] Cem Kansu: Why not? Exactly. We would love to become the Disney of the next generation."

He acknowledges the challenges but remains optimistic about expanding Duolingo's brand presence and intellectual property offerings.

11. Lessons Learned and Best Practices

Throughout the episode, Kansu imparts several valuable lessons:

• Freemium Models: Maintain a robust free version to drive growth while offering premium features for revenue.

• Iterative Prototyping: Utilize AI and rapid prototyping to test and refine features quickly.

• User-Centric Monetization: Ensure that monetization strategies enhance rather than detract from the user experience.

• Retention Focus: Prioritize retention metrics as indicators of product value and user satisfaction.

• AI Integration: Embrace AI tools to empower hybrid roles and accelerate product development.

12. Conclusion

The episode concludes with Kansu expressing excitement about the future of product management in an AI-driven landscape. He envisions a world where AI tools democratize product development, enabling more innovative ideas to flourish and enhancing the overall quality of consumer products.

"[25:02] Cem Kansu: It's a 10x productivity gain where the best ideas will eventually win."

Harry Stebbings wraps up the conversation by acknowledging the transformative strategies employed by Duolingo under Kansu’s leadership, offering listeners a masterclass in building a beloved consumer product through thoughtful design, innovative monetization, and strategic use of AI.

---

Notable Quotes:

• "[06:05] Jem Kansu: You know, my favorite dude on product is Gustav Soderstrom from Spotify, and he always told me that the details are not the details, they are the product."

• "[16:53] Jem Kansu: Honestly, if you were advising like a young product person today… what would you advise?"

• "[29:43] Cem Kansu: That's lesson one: things change."

• "[51:56] Cem Kansu: Retention is everything when you think about building product."

• "[25:02] Cem Kansu: It's a 10x productivity gain where the best ideas will eventually win."

---

Key Takeaways:

1. AI as a Catalyst: AI tools significantly expedite product development cycles, enabling rapid prototyping and iterative improvements.

2. User-Centric Monetization: Successful monetization hinges on integrating revenue streams without compromising user experience, fostering trust and loyalty.

3. Retention is Paramount: Focused efforts on enhancing retention through habit-building features and engaging user interactions drive long-term success.

4. Future of Product Teams: The convergence of roles empowered by AI will lead to more versatile and efficient product teams.

5. Balanced Growth Strategies: Maintaining a strong free offering alongside premium features ensures sustainable growth and competitive advantage.

This episode serves as an invaluable resource for startup founders, product managers, and venture capitalists interested in understanding the intricacies of building and monetizing a successful consumer product in the age of AI.

Transcript

[Harry Stebbings]

This is 20 product with me, Harry Stebbings. Now 20 product is the monthly show where we sit down with the best CPOs in the world to discuss how they build their product teams. Today we've got one of the best joining us in Jem Kansu, CPO at Duolingo. One of my favorites. At Duolingo, Jem leads product Strategy for over 90 million monthly active learners. Since joining, Jem has played a pivotal role in driving user engagement, revenue growth and product innovation. He's launched Duolingo Math and the wildly successful Duolingo Music. Under leadership, the company is consistently ranked as the number one education app globally. This is a masterclass in how Duolingo has created a product of true consumer delight. But before we dive into the show today, are you struggling to beat model benchmarks or implement Gen AI in your product? If so, you need Turing. Turing is an AGI infrastructure company backed by incredible investors like Foundation Capital and Westbridge Capital. And they do two things. Number one, they help leading companies in AI labs like Salesforce, Anthropic and Meta enhance their LLMs with advanced reasoning, coding, multilinguality, multimodality and more. Number two, they combine human and artificial intelligence expertise to deploy cutting edge AI systems for awesome companies like Rivian and Reddit. Right now, Turing offers a free 5 minute self assessment to help you pinpoint your place in the Gen AI journey, get tailored next steps to optimize your model strategy and then finally learn how Turing can refine and implement your model for better performance. Take the guesswork out of Gen AI. Visit turing.com 20vc to start your free assessment today. And once your Gen AI strategy is on point with Turing. Now let's switch gears to the future of mobile and how one company is turning your screen into real returns. We spend nearly half our waking hours glued to our phones, upwards of 50 hours every week. Recently, one company transforming this reality stood out so much, I personally became a shareholder. Mode Mobile Mode Mobile created the EarnPhone, the smartphone that pays you for daily activities. Instead of big tech profiting billions from our attention, Mode returns over $325 million directly to users. Through earnings and savings, Mode's revenues surged an incredible 32,481% in three years. Recognized by Deloitte as 2023's fastest growing software company in North America. And here's why I'm excited. MOAD's equity offerings have raised over $30 million from 20 retail investors, one of 2025's standout public raises. And you can now join me as a shareholder with as little as $1,000 at invest.modemobile.com 20VC for a limited time unlock up to 100% bonus shares and a free earn phone. Email us for the investor brief at 20vcodemobile.com or check out invest.modemobile.com20VC. We've talked about making money from your phone. Now let's talk about making your product exactly what you want it to. Whether it's the software you build or the software you buy. Your tech stack should be creating results, not creating roadblocks. Well, Pendo's no code Software experience management platform makes your software better with tools to see where users get stuck. Guide them with in app messaging and constantly improving your UI. It's so easy that over 14,000 businesses use Pendo to increase revenue, lower costs and reduce risks. Businesses love the control, engineers love the freedom. Everyone wins. Start for free today at Pendo IO forward slash 20 product. You have now arrived at your destination gem.

[Jem Kansu]

Dude, I am so excited for this. I just mentioned it, but I frickin love the Duolingo product.

[Unnamed Participant]

I told you I love that you love it.

[Jem Kansu]

It's so nice for me because I interviewed lots of CPOs and I don't love their product. And you kind of have to pretend, you know, it's like you're on a date where you're like, oh really? That's what you do. Oh, fascinating. And here it's like I love it. And you know what I love is the attention to detail.

[Unnamed Participant]

Yes, yes.

[Harry Stebbings]

So special.

[Unnamed Participant]

Yes. And you're learning Spanish.

[Jem Kansu]

I'm learning Spanish, yes. Bruno S. Americano attention to detail for you. Thank you so much.

[Unnamed Participant]

No, we take pride a lot in the pixel perfect details. Like I think you'll see that in the product. Right. It's like every screen will have a very specific design, language, characters will be very specific, etc. So like I'm glad you noticed it totally.

[Jem Kansu]

Let's have some chronology because otherwise I'll just go off on a tangent. Can you take me to the moment you actually joined Duolingo?

[Unnamed Participant]

Yes.

[Jem Kansu]

How did that come come to be?

[Unnamed Participant]

Yes. So I joined in 2016. It was kind of a dark rainy day in September. I'll skip the funny part of the story anyway.

[Jem Kansu]

No, I like that.

[Unnamed Participant]

Join for the story.

[Jem Kansu]

Oh yes.

[Unnamed Participant]

I'm feeling 2016 in Pittsburgh, Pennsylvania. That's where our headquarters is and I was the first PM that was handled the baton for. We were pre revenue. We would like to have revenue and can you help us figure out how to monetize? Obviously I was an early career PM and I knew ads. I had worked Google before. My first experiments. My claim to fame at Duolingo was putting ads on Duolingo. As you can imagine, this is good. Good fame. Yeah, exactly. Good fame and bad fame at the same time. And then I worked on monetization for about six years. So I built all of our revenue features, ads, subscriptions in app purchases, and everything that's under our subscription umbrella. Subscriptions obviously became the lion's share of Duolingo's revenue. And then I kept on building our teams that work on revenue and building our features that work on revenue. And in the past three, now I've moved on to being our head of product. So I lead the product management team and now I dabble with all of our product efforts.

[Jem Kansu]

I mean, my word, how exciting.

[Unnamed Participant]

It's been a great journey.

[Jem Kansu]

So you're the guy who puts ads in my Duolingo.

[Unnamed Participant]

I did that, yeah. Yeah.

[Jem Kansu]

Brilliant.

[Unnamed Participant]

You're welcome.

[Jem Kansu]

You know, I think it's well done. I am just jumping around to be fair, but my favorite dude on product is Gustav Soderstrom from Spotify, and he always told me that the details are not the details, they are the product.

[Unnamed Participant]

Yes.

[Jem Kansu]

And I'm just intrigued. When you think about product building, how do you think about the details being the product versus, for Christ's sake, we actually need to build product and move on.

[Unnamed Participant]

I think I have a very similar saying, I repeat a lot. Consumer products live and die in the pixels. Meaning if you are working on a consumer product, you have to figure out how to make your change pixel perfect for the user. So, for example, product managers are sometimes viewed as these. Like, I'm the mini CEO, I decide strategy and then people execute. That is like the worst definition of a product manager that I've ever heard. Why?

[Jem Kansu]

Because everyone says that on the show.

[Unnamed Participant]

They love. They love saying that. First of all, the CEO is the CEO of the product, not the product manager. The product manager has to understand what the user will respond to when you kind of give them what they're going to see in the app. So they have to understand user design really well. They also have to be able to stretch into business strategy, etc. That said, this is why consumer products live and die in the pixels and attention to detail matters. If a button that's like the dark shade of green versus the light shade of green will make a difference in user behavior, a product manager or a product designer for that record, like people working on the product should understand what that means and I think this is also how we build product teams at Duolingo, where everyone should understand pixel perfect design and why certain specifically what the user is going to see, how that's going to change user behavior.

[Jem Kansu]

Can we start at the start of the process, which is the design process?

[Unnamed Participant]

Yeah.

[Jem Kansu]

What does the design process look like for you today at Duolingo? And has that changed over time?

[Unnamed Participant]

It's evolved. It's evolved quite a bit. I joined Duolingo when it was 50 employees. Now we're about 850. So things have changed and we've gotten a little bit more process. But at its core, the way it starts is you start with all of us use the app heavily. So most of the ideas that we work on come from our own usage of the app. Not all of them, but a lot of them. So, for example, if we feel like, and this isn't maybe this is the example, we can kind of highlight. We felt like Duolingo didn't teach speaking enough. So when we were testing GPT3, we're like, okay, this is a good way. We've tried many different things to teach speaking. And then GPT3 was this, like, light bulb moment where, like, we can actually teach speaking better if we use LLMs. So the design process got started with, like, you have the technology and you have Duolingo. How do you bring them together through various iterations, back and forth? We prototyped a thing where Lily talks to you. We really liked it. We made a feature out of it. Now it's called Video Call with Lily. Now, a lot of our users have access to it and are using it.

[Jem Kansu]

So I was walking through Belgravia, which is a very smart part of London, the other day, and I was on Duolingo and I was going, voy a la playa. And this guy comes up to me and he's like, I'm sorry to interrupt your Spanish call, but I love your show. And I'm like, you've literally been standing there listening to me go, bruno es Americano and voya la playa thinking, this guy is a fucking idiot. So, yes, I agree, and it's fantastic integrating it. But that design process, then, is it fast? And who needs the approvals? Don't laugh. Do you do figma? Do you just go straight to prototyping? Now, how does that work?

[Unnamed Participant]

Yeah, great question. So we use Figma for a lot of our designs. That said, for something that is an experience, you got experience. Figma is good for us to say, this isn't headed in the right direction. Let's go to prototype. We have a product review process where our CEO Luis is involved, I'm involved, our head of design is involved, and our product leaders are involved. Where teams come, they show their change and we give feedback. They refine it, and they either go to prototype or they go straight to an AB experiment.

[Jem Kansu]

Wow.

[Unnamed Participant]

So depending on the size of the change, we might do extensive prototyping. In the case of video call, we prototype for, like, maybe two, three months. We did one change, we tested it, we didn't like it, we iterated, and then it was ready to go out to our users.

[Jem Kansu]

What does extensive prototyping entail?

[Unnamed Participant]

Extensive prototyping? I think you. Let's say we have an idea. We think calling Lilly is going to be a good way to practice our Spanish. The way that the first prototype is going to look is, like, it's not going to be well animated. The illustrations are going to be janky. Maybe there's going to be, like, error codes in, like, some part of the ui. But we'll test the core interaction of can she give us smart answers? We'll try that. And then we'll be like, okay, she can. Let's move to problem number two. It needs to look engaging. We solve that. So I think extensive prototyping is you prototype test and, like, refine what the spec is.

[Jem Kansu]

So it's like a step one utility. Step two, design.

[Unnamed Participant]

Yes.

[Jem Kansu]

Step three, perfection.

[Unnamed Participant]

Pretty much. Exactly. And I think for smaller stuff, like, if it's not an entire new feature, you can do it all in one go. But if the extensive iterative prototyping is good, because you don't have to invest, like, months of time before you see if it works or not. So you validate if it works and then you kind of do the other steps.

[Jem Kansu]

Is there subjectivity to if it works and, like, a user behavior like Lilly, you could be like, well, I thought that was really engaging. And another person would be like, I did not.

[Unnamed Participant]

Yes, 100%.

[Jem Kansu]

What do you do then?

[Unnamed Participant]

Well, I think we bring. We bring a lot of smart people in the company together. We have them all tested and see if we can align. For the most part, we don't all have to align, but generally, if enough high judgment, people are like, yeah, this kind of works. Like, I can actually practice speaking better than everything else I could do on Duolingo, which was the case. We're like, we have something good on our hands. Let's make it better.

[Jem Kansu]

You mentioned Lily there. I obviously spoke to Severin the other day. Wonderful, elegant, Man. And he told me about like two of the. I think it was PMs who came up with the idea for Chess.

[Unnamed Participant]

Yeah.

[Jem Kansu]

And brought that forward. And they weren't really sure. How do you think about the ideas you choose to pursue versus not choose to pursue?

[Unnamed Participant]

This is a great question. You know, I would love to say we have this like super ironed out process. You know, you follow ABC and you have the perfect answer to the question of what to invest in. The reality is it's a lot messier than that. But Chess is a really good example of bottoms up innovation in the company where it's not an executive that had a great idea and said, ye, we should go in this direction. So as a product manager and a product designer, they didn't have an engineer on the team. They went and prototyped. Basically they're like, we think Chess will be a good subject for dueling and to do. We're like kind of weird, doesn't exactly fit, but we're like, yeah, cool idea. They went and prototyped and almost like a full Chess lesson experience with no engineer. So like our PM Vibe coded basically the whole thing. Our designer created excellent looking designs. We tested it and we're like, this is a pretty good duolingo lesson. So like they kept iterating actually like on this Vibe coded prototype. And then until a point where we're like, this is very much headed in the right direction. We really enjoy using this prototype. And then we staffed a team with engineers and now it launched this week. So in a span of eight months, it went from like, I don't know, this is kind of an interesting idea to like a full launch of a new duolingo course.

[Jem Kansu]

How does that differ in terms of timeline from traditional idea to launch?

[Unnamed Participant]

It's way shorter and way less people. Our experience with doing this is, was with math and music. Before it was triple the amount of people and double the amount of time it takes to get a course out. The big unlock there is, I would say, maybe two things. One of them, AI coding certainly speeds up because you can prototype quickly, get feedback, iterate rather than if you couldn't do that, you have to have an engineering team that's like building prototypes and that's a much slower process. The second one is honestly, I think having great people. I mean we had a great PM and a great designer and they know how duolingo works, what we think is good, so they can independently make really quick decisions. So I think staffing a great team with great tools was the big unlock.

[Jem Kansu]

When you look at that process and you look at the team that went into it. What does the structure of the product team look like today and how will that change over time?

[Unnamed Participant]

This is a great question. I think today the traditional product team is. You're going to have somewhere between five to 15 people. That's the traditional product team that has engineers, designers and product managers at its core. And it might have other functions like data science, marketing, depending on what they're doing. What I expect is going to happen, which is maybe the chess example is a good example, is these teams are going to be smaller because you might not need as many people to do the task at hand, which might be just like, we just need to create a prototype of this thing. We just need to figure out if this is the right design. We're probably not going to need as many people. So we'll have smaller number of people on each team. But honestly, I'm hoping by doing that we'll be working on much more innovative bets quicker. Because again, like, this chess example is a good one. If we can do that, like 10x more, well, Duolingo will hopefully be 10x better over time.

[Jem Kansu]

Will it be 10x better or will it just be 10x broader? Because there's a question of, like, you can go for breadth of subject or depth of engagement, and that is a challenge. I love a Matthew McConaughey talk and this is why his show is so successful, because I bring in very wise leaders. But he says, you know, options make an enemy of us all and actually, constraints are sometimes good.

[Unnamed Participant]

Yeah.

[Jem Kansu]

How do you think about ensuring feature simplicity without creep?

[Unnamed Participant]

I think focus matters a lot in product. I think you can really distract yourself by going too broad. So our goal is not to 10x the number of subjects Duolingo teaches today. Making Duolingo 10x better in everything we teach is the main goal. That said, I think if we find a great subject that would be additive to Duolingo and we believe we can do it really well, I think you'll see us doing that. Right now we teach four subjects like language, math, music, chess. The goal is not to get to 40 subjects and do them all equally the same. I think make what we have 10x better is the primary goal.

[Jem Kansu]

Can I ask, when you think about design and when you think about product and Eng does what it takes to be 10x as a product person change. Now, like you said, that they vibe coded. Do we have just hybrid people?

[Unnamed Participant]

I think so. I think what is likely to happen is there is clear work definitions of what an engineer does today, what a product manager does, and what a designer does. I think these are going to converge way more than what's happening today in technology organizations. So I don't know if they'll converge fully. I think that's maybe a little far. But they will converge in the sense that they'll be a lot more. All of them will be capable of taking over each other's jobs in multiple ways because AI tools will enable them to design faster or PMs to start designing more, or PMs to vibe code more, or designers to do the same, or engineers to stretch into design or pm. So I think these roles will converge more. That's already happening.

[Jem Kansu]

Honestly, if you were advising like a young product person today. 20, I'm your little bro. Okay, I'm 23. I know we both have the good looks to carry off, but what would you advise me as a product person entering this very different world?

[Unnamed Participant]

Learn AI tools in and out today, and they will look different almost in, like, four weeks. So I think you should be experimenting with how AI tools build product, like, very heavily. This is what I admire my own team. Like, we have basically told and aligned with our team. We will be AI first as a product team. This is very similar to when companies went mobile first in, like, 2012. It felt really artificial and weird because, like, tech companies looked at their user base and said, said, oh, 1% of our users are on mobile, 99% is on web. Going mobile first is kind of stupid. Like, we're investing in 1%, but, like, quickly, in, like, three years, they realized that was the right bet to take, but it was quite artificial. Anyone that's going into products should bet on the same thing, which is you should build AI first. Even though when you use these AI tools and say, all right, create me a great app, you end up with, like, a dinky web app. In my opinion, that will get solved in, I don't know, maybe 12 months. But I think if you know how to use these tools to create product, build product, and you can kind of harness their power to build great things, you'll come out ahead.

[Jem Kansu]

There's a difference between idea and action. And I would like to say that with my team here, obviously we have a big media company, and I would like them to be bluntly trying tools. And I'm like, should I implement an hour a day testing time?

[Unnamed Participant]

Like, jokes aside, I think you 100% should.

[Jem Kansu]

What do you do deliberately to make sure people actually are? Because people are lazy, bored, and tired.

[Unnamed Participant]

There's going to be inertia. Right? I think it's like just saying, yeah, we should all use AI. I think that doesn't actually get groups to be like, yeah, like I'll go do this tomorrow. I mean some people will because they're, they're excited. I think what tends to work is I think in one, incentives, you should incentivize people to automate repetitive tasks. I think you should award and highlight people doing that. That's what we do. I think that helps.

[Jem Kansu]

How do you reward people?

[Unnamed Participant]

For one is recognition. I think that's what we do right now a lot. So if one of our product managers automates the task, we encourage them to share it with others and we cl. I think that is one way to recognize someone that has done something great.

[Jem Kansu]

I'd rather have 25k. Well, I think you give me a clap.

[Unnamed Participant]

I mean money, money is another incentive. So I think you can reward people with compensation. That, that's another obvious one. And I think the third is in, in an organization. I think it's also in a tech company or like Duolingo. We bake it into how we think about performance. So if someone is actually creating more output with AI, well, they'll probably get promoted faster.

[Jem Kansu]

So what do we do in product today that we definitely won't do in five years?

[Unnamed Participant]

Oh, there's a lot, I think specs, like, you know, companies write product specs that like go from someone's idea that turns into a product spec and then that goes into design, that goes into engineering. This process is like dead. Like why would you write specs if you can go from your idea to an exact prototype? So this whole middle assembly line of creating product is a dead process.

[Jem Kansu]

Do you lose the design process? Does Figma become obsolete in the world of replica? Say?

[Unnamed Participant]

This is a really good question. I don't think it becomes obsolete, but the way we use these tools and maybe it'll be Figma, maybe it'll be a different tool, is going to change significantly. I think we'll go from basically text of an idea to a prototype of an idea rather than everything in between. Maybe Figma is the tool that we use to do that, maybe it's something else. But then now there's like three, four steps to get there. Or actually even more like there's basically text to design and then design to code is kind of how things go. Actually. There's text to spec, spec to design, design to code. This is gonna all merge is my guess. So this process is changing already.

[Jem Kansu]

Does this favor you as a big company because traditionally I like that you.

[Unnamed Participant]

Call us a big company so.

[Jem Kansu]

Well, you're a public company Worth like $20 billion, Jim. I think it's just like we don't.

[Unnamed Participant]

See ourselves as such as you can see. But yeah, sure, yeah.

[Jem Kansu]

But like it's $20 billion. $20 billion. Well done. And so my point being there, like traditionally that would have been a bad disadvantage that you have. That process respectfully is cumbersome and my teams seed don't need to do that now. You don't need to do that.

[Unnamed Participant]

It's to our advantage. I mean we can cut process. We're not doing process because we love process. I think if we can speed up all of our product teams, to us that's an advantage. Like we would love to disrupt our own process as soon as we can. So that's like every individual can ship product faster.

[Jem Kansu]

Will you have more or less engineers in five years time?

[Unnamed Participant]

Oh, this is a great question. Honestly, I don't know.

[Jem Kansu]

Well, that's not a question. That's not a good answer. Give me a yes or a no.

[Unnamed Participant]

Yeah, I'll give you my take. We will have more engineers. However, the definition of an engineer might look different in five years. Meaning this hybrid, I'm going to call it the prototype role, where it's like a hybrid of engineer, designer, PM will be way more common and I think that will be a lot of the headcount growth that organizations experience rather than I think right now when you say engineers, like software engineering is like a very defined discipline. I think we'll end up with these like undefined builders that can use AI tools really well. And I think companies will employ a large number of those.

[Jem Kansu]

When we look at your cursors or your wind surfs or any of these tools that our engineering teams are using today, do they make 10x engineers 100x or do they make average engineers 10x?

[Unnamed Participant]

I think my read right now is more of the latter. And also the magnitude of multiplication is way lower than like 10xing things. I think it's maybe in the realm of like 10 to 20% more productivity of an average engineering capacity.

[Jem Kansu]

That's not that exciting.

[Unnamed Participant]

No, I don't. So my read is like, it's not that exciting. And I think there's a lot of hype generated around like, you know, we're now replacing like I think some companies are putting out statements are like, okay, like we're not going to hire engineers anymore. I've seen really hot takes like HubSpot.

[Jem Kansu]

Said the other day that they now produce too much code to ship.

[Unnamed Participant]

That sounds insane. I hadn't heard that.

[Jem Kansu]

Yeah, yeah, Amazing. And then Salesforce, 20% of their code is AI generated. Microsoft is 30%.

[Unnamed Participant]

I think these stats are very much of the autocomplete variety. Like what curse is a lot of times good at is you start writing code and then you can autocomplete with AI. That's great. That's awesome. I mean, it's not a 10x unlock if you can do that. Like meaning if you're writing an article, if you can complete 20% of it with autocomplete is the same. It's like, yeah, it's good, but it's not a massive life changing accomplishment.

[Jem Kansu]

What is a massive life changing accomplishment in that product paradigm then? Or productivity?

[Unnamed Participant]

The two things that really become 10x unlocks is when we figure out text to excellent looking, pixel perfect design with just prompting. I think that is not there, but I think it will get there. So for example, we think that Bruno es Americano is kind of the, the experience we should build. So we build a charact like Bruno that you can talk to and he's American, he teaches you American English, let's say that's what we want to build without touching a single piece of code. Can we create the best animated Bruno character that talks to you in American English? If we can do that, that is a massive unlock. And I think this is not super far. So that's like text to design and then design to code. We have the design that we can build a prototype of that thing that is good enough to go into duolingo and go out to real users. That is a 10x unlock.

[Jem Kansu]

Hablo mucho espanol. Muy bien, Gracias. I feel this is going well. The Joyce of also end of day is you're just like, ah, fuck it.

[Unnamed Participant]

Why not?

[Jem Kansu]

Why not? It's not 10am I can't be asked to be formal. When you have that unlock, the amazing thing is it means that bluntly, everyone can create product to an extent.

[Unnamed Participant]

I think we're about to enter a wild time with this. Yes.

[Jem Kansu]

Does that excite you?

[Unnamed Participant]

It's. I'm so excited.

[Jem Kansu]

Or worry because it now means that you have infinite supply of product.

[Unnamed Participant]

Yes.

[Jem Kansu]

And that means that discovery becomes even more challenging.

[Unnamed Participant]

Yes. On a personal level, I'm super excited and you know, for the discipline of product management, I'm also super excited. Maybe you've. You've had multiple folks on this show and I've Heard this come up actually. It's like PMs do have imposter syndrome because they're like when they're interacting with engineers, they're not technical enough. When they're working with designers, they're not creative enough. These tools level up everybody to have much. Maybe not at the same level but to a similar leveling field where we're all operating with like strong tools. So it helps PMs be a lot more productive and independent for themselves. So I think you up level product managers, which is good like for my function, I love that. But you up level everybody to be way more productive. This unlocks much easier product building and I think this is very similar to just like a 10x productivity gain where the best ideas will eventually win. So, so creating 10x more prototypes doesn't. I mean it creates maybe chaos at first, but then the users eventually get the best of hundred ideas rather than best of five ideas we can ship today. So I think just like creation goes up and when creation goes up, the best things that go out to users are just going to be better.

[Jem Kansu]

Agree completely. When you look at the tools again, who wins, cursor or Winsurf?

[Unnamed Participant]

I'm going to say cursor.

[Jem Kansu]

Why?

[Unnamed Participant]

Because internally at Duolingo I'm seeing way more usage and way more creation come out of cursor. That said, maybe we haven't experimented enough with Windsurf.

[Jem Kansu]

Can I ask you, Everyone says to me, I look at these tools and you know that neither of them have lock in or defensibility. And as we discovered from my apt due diligence on some companies we mentioned before, I naturally go, that sounds very interesting, tell me more. But I don't actually understand why. Why did they not have defensibility and is that wrong?

[Unnamed Participant]

I feel like these statements come out of is there a data lock in? So like if you're feeding it more data, is the tool getting better or is there like a social lock in where I don't know, you're collabor with folks and you know, switching is painful. For example, a lot of enterprise tools have heavy data lock in. Like we use Jira for e.g. task management in the company. It's very hard to switch out of because all of our history has been now fed in there and switching out means we kind of have to leave that data and go. We use JIRA for a la poop.

[Jem Kansu]

I'm going to introduce you to Carrie at Linear after this.

[Unnamed Participant]

Okay, that sounds good. Yeah.

[Jem Kansu]

And he is amazing. Amazing. And this is called Venture Value Add here a week and honestly, they're brilliant and much better than Jira.

[Unnamed Participant]

I mean, I've seen their designs. It looks awesome. I'll say the hardest thing is to switch out of the thing we have. So even if we love the product, switching out is going to be a big debate because there's a huge cost. And I think people bring this up because you can switch from cursor to Windsurf, you don't lose anything. Right? If the Windsurf is better tomorrow morning, well, we'll start using that. Maybe these will change. Like, meaning the more code you build with one will just make it so much better. For example, now ChatGPT has memory. I think that is an amazing lock in for ChatGPT because now since it remembers our chats, it can now give me smarter answers. That is a big lock in for me to not use a different product.

[Jem Kansu]

I do have to ask because you mentioned at the beginning, like being central to the monetization and I spoke to Severin and he said that bluntly, it was your fault that they waited so long. Jokes aside, why did it take so long, do you think? And how do you reflect on that decision?

[Unnamed Participant]

Hindsight bias is easy to apply to our past and say, oh, we waited too long. But I think at the time a few things were, I would say maybe headwinds to starting to monetize. One of them is Duolingo obviously has a very strong mission today. And that certainly was the case then when it first came out. Now, Luis, you can find Luis's talks. This was a lot easier to find on YouTube and I think it was on our corporate pages. I think we removed it now because we moved away from that. But the monetization plan for Duolingo was going to be, everything was going to be free. So as users went through, did language exercises, they would do translation and then we would crowdsource those translations as a service to anyone who wanted to buy it. So if CNN wanted to translate an article from English to Spanish, they would feed into Duolingo system system. That article would be broken up into chunks, fed into users and exercise. And as crowds translated that as an exercise, it would populate back a translation. That was the original plan. And as Luis announced Duolingo, he also said, we will never have ads, never in app purchases and never subscriptions. And funnily enough, that's those the exact three things I did as part of my job at Duolingo.

[Jem Kansu]

And your lesson from that is never say never.

[Unnamed Participant]

That's lesson one that things Change and your view on. On these change. But I think two, the reason there was a lot resistance in the company for monetization is we launched with the view that if you did these things, you would screw up your amazing, great consumer product. I think what honestly my job eventually turned into is obviously building these things, but also changing the organization's mind that there is a way to do these without going against our mission of providing access to free education and doing them well. The big unlock there was at the time, and honestly, mostly today, all education apps would lock all of their content behind the paywall because. Because, you know, that's so straightforward. Like, why did you come to this app to learn? Well, what are we going to charge for? Let's charge for the learning content. We did the exact opposite where like, learning content is completely free, but we are going to monetize the bells and whistles. We're going to do freemium. So most of the stuff will be free, but the premium features could be part of the subscription. Even our investors at the time was like, that's not a good idea. I've never seen that work.

[Jem Kansu]

Well, if I was one of your investors, I'd say, hey, growth is everything. Engagement is everything. Retention is, is the fucking holy grail. Don't stick in ads when we don't have a massive company yet.

[Unnamed Participant]

I think that was also the common wisdom at the time. There was the Facebook model, just grow, doesn't matter, ignore monetization. And honestly, it was good advice. But I think what we ignored is you can still have high retention. You can keep growing your product, but also monetize at the same time. I think it felt like it would kill things and it didn't. It was fine. Like, I mean, putting ads on duolingo, if you do it tastefully, doesn't kill your retention. So you can still have very high retention and monetize the bells and whistles, but have high growth.

[Jem Kansu]

Can you just take me to the moment where you were like, okay, we're going to move from a zero revenue company to trying to make money?

[Unnamed Participant]

Yeah.

[Jem Kansu]

Was there a meeting? Can you take me to that moment?

[Unnamed Participant]

It probably wasn't one meeting, but there was a few key moments where I remember being like, how am I going to figure this out? One of them was sitting with Luis in, honestly, probably one of my first weeks, and I'm a very early product manager trying to figure stuff out, out. And I'm like, all right, Luis, like, here's our ads roadmap. I calculated how much money this is going to bake. Here are the experiments I'm going to run. And he's like, all right, what is it going to look like? And I show him like the prototype or like we look at it. He's been looking at it. He's like, these ads look like shit is his first response. He's like, I'm not sure if we should do it. And I'm like, okay, where do we hear you? I understand that this doesn't look like the pristine every other page of the app. It's different, it's another advertiser's property. But we can improve the look, we can change the design anyway. It was an iterative process from there to be like. He's like, let's test, let's see what happens. And the other worry was like, oh, this is going to kill retention. We test it, it's fine. It's like neutral on retention on the first experiment. And we're like, okay, our users are not running away. We are improving how the ads look and when they're surfaced. And then I think over time it started snowballing to be like, okay, we can do this, we can do ads. Once we did ads, the second phase was, was, well, no ads was like a request people started sending us being like, can I pay you guys to remove ads? We're like, well, let's try that. That's a good start.

[Jem Kansu]

Before we move then into the paid then do you accept that you cannot do ads without hindering user experience a little bit?

[Unnamed Participant]

There's certainly a cost.

[Jem Kansu]

I had it the other day and you have the timer in the top left and I'm like, listen, it's annoying. I stay of course. And I love duolingo just the same.

[Unnamed Participant]

I can't say I love ads as a user either. So I think you sacrifice because it is not a property. You meaning if I bring an advertiser, they have to show their asset and you know that asset comes with a five second timer on, I don't know, a video for selling you shorts, whatever it might be, it's a cost. I think that you take. That said, there are many different ways to do ads. So I think treating ads as this like black box carrier of like it's only bad and it can only hurt is a bit too simplistic.

[Jem Kansu]

What's the lessons on what it takes then to do ads? Well, if it's too simplistic to just.

[Unnamed Participant]

Black box, I think ad load is the key thing there. We stick to one. You can, you will only see one ad at the End of a lesson. And that's not mid lesson, that's not started lesson. You finish your lesson and that's you take a breather. That's the only time we show you an ad is what we aligned on versus you can put a banner ad on top of. You know there's many tasteless ways to.

[Jem Kansu]

Do it which optimize for cash though.

[Unnamed Participant]

Correct.

[Jem Kansu]

Because you can charge more for in play versus post.

[Unnamed Participant]

Correct. But I think that is short term thinking at the end because you, you maximize a lot of maybe revenue now because you're putting a lot of ad clicks the app. But the moment you ruin your free user experience, you kill your growth engine.

[Jem Kansu]

Can I ask you, do you get paid on a views or on a clicks?

[Unnamed Participant]

We get paid by impressions, so views I guess. So the rate that these ad networks pay you depend on how clickable these impressions are. It kind of both touch eventually the ad revenue you make if you're running in programmatic ads on your products.

[Jem Kansu]

Okay, yeah. And so I then say, you know what, fuck this, I want no ads. And you're like, what happens then?

[Unnamed Participant]

A lot of users obviously were like, I love Duolingo but you know, I'll pay you guys whatever. Can I remove the ads that no surprise because that's a common model. That was the idea for our subscription. So it's like we had ads, we had a no ad subscription. That was the start. And what we saw also at the time was there was a lot of user love for Duolingo. It sounds like you also love Duolingo. And we saw users just buying the subscription because a lot of our users was. I was using Duolingo for five years. I just wanted to contribute something back. So I bought the subscription even though I didn't know what was really in it.

[Jem Kansu]

No, I'm not that Christian. Definitely not. The $20 billion market cap made me feel. I was like.

[Unnamed Participant]

But yeah, so. But basically the no ads was a starting point and it was feature light. Honestly at the time, the subscription was kind of a nice to have more than anything else.

[Jem Kansu]

What was? If you were advising me as a founder on doing that, like premium subscription.

[Unnamed Participant]

Yeah.

[Jem Kansu]

What would you say? Having been through that, it really depends.

[Unnamed Participant]

On what business you're building. But if you are building a consumer app business, I would very, very highly advise you to build a freemium model. Meaning you should have a very great free user experience without asking anybody to pay. The reason for that is if you have a premium product, let's say it's all paid, you can Only use the app if you subscribe or it's very limited. What you have to build is also a paid marketing engine. So you have to spend a lot of performance marketing to acquire subscribers revenue from them and spend it back on performance marketing. That's how paid products kind of have to grow. Free products. If you build an amazing free product, it takes way longer. But what you end up building is an organic growth flywheel. Meaning you tell your friends about Duolingo. I don't know if you do. I tell my friends about do. Let's go off of me. They can go experience the full thing for free. Whether they pay or not. Doesn't matter if they like it. They will also tell their friends. It just expands the entire pie of people who can use your product product without having to pay. And we have a lot of users that use the app for free for five years. Eventually the first of January, they're like, oh, I'm gonna go ham on my French this year. I'll subscribe. It doesn't matter. But they use Duolingo. So to build basically a large user base. And if you want to build a large business, it's way more defensible to build a great free user experience and do it freemium rather than full premium. The other problem with premium is competition. So, like, the reason Duolingo has grown quite a bit in the past 10 years or even gained initial success success was because it had an amazing free user experience. If you created the exact equivalent of Duolingo today, what you would probably do is, okay, like, you want to make some money. Apps have tried this. Actually. You kind of copy Duolingo exactly as is. You make it fully paid. You're like, I'm going to go fully paid and spend on paid performance marketing. Well, what happens is the users who you can acquire will land on your app and say, okay, you asked me to pay $80, $100, whatever it is, or $10, or I can just go to Duolingo and do it for free. So your paid costs go really high. So it makes it really hard to compete with a free, amazing product. So this is a very roundabout way to say, if you're building a consumer app business, I think you want to make sure you have an amazing free growth Flywheel makes you defensible and it kind of spreads your product without you having to spend marketing money.

[Jem Kansu]

How do you think about the right time to hit them with a pair of paywall? You want to build the trust, the love, the loyalty without being like, okay, level one paywall Exactly.

[Unnamed Participant]

So I think it's twofold. One of them is honestly like hitting with a paywall is kind of the wrong approach in the sense that I think you want people to discover the paywall by experiencing features. That is the best way to honestly like convince someone to buy something. Meaning, let's say I would like to convince our users to buy Duolingo Max and its key feature is Video Call. The best way to do it is have them try Video Call, actually. So a lot of our ways to do that is maybe show them a video of how Video Call works and then say, if you're interested here is Duolingo Max. Or have them go into the experience, have a call themselves and then at the end say this is part of Duolingo Max. That is kind of the best way. When it's like contextual and users discover that there is something to be paid to like experience. I think just like having people showing kind of what I would call like generic ads works way less better and it looks spammy in the app. So we try to do everything contextual. For example, if we show you an ad, we will say right after there's an ad free version if you'd like to upgrade. And that is in a way like you've discovered an ad. Okay, maybe you didn't love it, okay, but you can also find an ad free upgrade.

[Jem Kansu]

I just wasn't expecting Pornhub Premium to be an advertiser.

[Unnamed Participant]

Aussie for a second. I was like, oh, the ad network.

[Jem Kansu]

What surprised us, it was the first useful ad I've had on Duolingo.

[Unnamed Participant]

It's the only ad you click on.

[Jem Kansu]

Sign up, sign up, sign up. The joys of doing the show is also like, you hear the show and you're like, God, it's so serious. And then behind the scenes you're like, what a ride.

[Unnamed Participant]

Behind the scenes. Pornhub preview.

[Jem Kansu]

Yeah, it's much more fun that way. Can I ask you, when you look at the different paywall pages that you've experimented with with, are there lessons on how to bucket different offerings?

[Unnamed Participant]

Oh yeah.

[Jem Kansu]

What, what are those lessons?

[Unnamed Participant]

There's a lot. The biggest one I will say is what you say in your copy in Strings makes a massive difference. And obviously you don't want to kind of sell something that you don't have. But how you talk about entering that experience makes a massive difference. I'll give you an example. We have tested many ways to on the button that is, that is going to start your subscription. We tested probably endless number of things Turns out if you say try for 0.00, which is not a normal human, like I wouldn't talk to you and like would you like to try something for 0.00 pounds? Like I wouldn't say that. But in the string, if you write multiple zeros and you hint at the fact that there's a free trial with multiple zeros, conversion goes up a significant amount. Over start your free trial over anything else, start my free week, start my free trial, start your free trial or come up with any other com and we've tested probably many different things. Turns out if you put multiple zeros, I think it like lights something up in the brain that you're like, oh yeah, I get it, it's very free. It's because it's like 0.00. So copy matters. That's one. Because there's with subscription there's natural aversion to entering a thing that might be hard to get out. We also have seen talking about how easy it is to cancel is like an unlock in conversion.

[Jem Kansu]

Do you guys send reminders?

[Unnamed Participant]

We also send reminders because I signed.

[Jem Kansu]

Up to Max and I signed up because of the reminder.

[Unnamed Participant]

There you go. So that, that actually, that actually makes a big difference. And honestly this is one of those changes I love because it's user friendly. If you don't like the experience, I want to remind you to cancel. We don't want people who forget and pay because of it. But that relieves the anxiety of entering a subscription free trial.

[Jem Kansu]

And the thing about the different packages like give people just one option, paid versus not paid, max versus premium versus free. And they're like choice. I'm always fascinated by the paradox of choice.

[Unnamed Participant]

I think compari comparison is super important. What we've also seen is if you give one it's for example, we have multiple packages now. There's super, there's family plan, there's Max. We think you are a great fit for super individual. If we only show you one package and you have no anchor for different packages, it converts way less. Because it feels like if that's not for you, you're just going to decline. If we give you more than three packages on one page, it's too much like too much choice. You also convert way less. I think two to three is we've seen as a sweet spot. So you want some anchors to compare and most of the time if you give a choice of three, you want the middle option to be the option that you really recommend because that makes choice very easy. Humans love I mean, on a menu too, if you think about it, if you have the lowest price, middle price and highest price, middle price feels like the safe choice. If you don't know what to choose. Very similar to if you're kind of offering someone a subscription choice. I think you want to make the choice easy by recommending one and recommended one being the middle one feels safe to choose.

[Jem Kansu]

What percentage of people take the recommended?

[Unnamed Participant]

Very high. I don't. I don't remember the exact percentage, but certainly more than. It's the majority. More than 50%. Wow. Yeah.

[Jem Kansu]

Any other big lessons on copy matters? I'm sure you've been through many different iterations.

[Unnamed Participant]

Oh, my God. We've tested so many things. Yeah, Copy matters. You know how every company eventually puts like a checklist in front of you where like, oh, if you do this package, you get like two check marks, but if you this other package, you get like 10 check marks. This has become like a very common way for subscription purchase flows weirdly. That also works because it's the psychological difference you can see visually on like, oh, what I have now has one check mark. What I don't have has like 10 check marks.

[Jem Kansu]

Well, 10 check marks, 24 7. Customer support.

[Unnamed Participant]

I was better. It feels like you're getting a lot, and I think that that feeling of getting a lot feels better. I think the other thing that we've learned is there's a lot of emotion on purchasing anything. Certainly something that is an upgrade to a consumer experience like Duolingo. Going from free Duolingo to super Duolingo or Duolingo Max. And I think you want to convey that emotion in your design. Meaning Duolingo takes pride in like a fun being fun and animated while our flows being animated on duo, doing funny stuff through the flow. It works well because it like, conveys that this is not just a boring subscription as a service purchase. It's like part of your experience and it's an upgrade. So anyway, you give illustrations, animation and things like that that make it a fun flow that moves metrics at the end and increases conversion.

[Jem Kansu]

What percent of revenue is subscription today?

[Unnamed Participant]

Today it's more than 80%.

[Jem Kansu]

What revenue line does Duolingo not have today that will be meaningful in five years?

[Unnamed Participant]

We do have this revenue line, but it's incredibly small today. But I think it will be way bigger is merchandise. We have a store you can buy Duolingo merch. Our most selling item is the dual plushie. It's like a very funny plushie of duo. I think we are One of those brands where our characters are quite duo is quite recognizable at this point and we're trying to build more IP around our characters. There's a lot we can do on creating better merch and expanding our merch offering.

[Jem Kansu]

Surely content is it. You want to become the Disney Disney of the next generation?

[Unnamed Participant]

Why not? Exactly. We would love to become the Disney of the next generation. I think that is hard for sure, but that would be awesome.

[Jem Kansu]

Well, I'm actually the founder of this idea and now company. That was my idea. So thank you, Louis. I will take over. Would you say that you're an education company, an entertainment company, a content company? I know it's all semantics.

[Unnamed Participant]

Honestly. Neither is the reality. We have maybe somewhat of an identity crisis is also the reality. Reality. We don't identify as an education company, but we also don't identify as a game or entertainment. So we're kind of in our weird own league and own market that we've created. The reality is what we compete for. If you're like, okay, but what are you? Who do you compete with? We eventually compete for screen time. We would like to make your screen time very useful for you to learn something and get better. The reason we don't fit into education very much is if you look at the education space base, honestly, this is a bit of a hot take. But it's all boring ass products. So I think we would like to not be a boring ass product. As you can see, we try to do everything super fun. And I think if you say, well, you guys do a lot of fun, then are you a game? Well, we're not a game either. We also want to teach you something. So we're kind of in this zone. League of our own is how we see it. But what we eventually compete for is your screen time. So everything that takes up your screen time, that might be social media, that might be gaming, that might be streaming. Whatever it is at the end is what Duolingo competes for.

[Jem Kansu]

It's also a hard one on multiples. Gaming multiples are shocking, education tough.

[Unnamed Participant]

And also that's not the product we're building that we're not building just for entertainment, but we're also not building just to educate and ignoring the fun element of it. So that's why it's like right in the middle.

[Jem Kansu]

Okay. The final element of the store, we've got the ads, we've got the premium and then we have in app purchases. How did we think about what worked there? And the biggest lessons to doing that so successful successfully so in app purchases.

[Unnamed Participant]

For us was very similar to how games have designed in app purchase mechanics. So we have a virtual currency. It's called Gems. It's not named after me. It's the actual actual diamond gems.

[Jem Kansu]

Sure, sure.

[Unnamed Participant]

Yeah, but, yeah, but you were the.

[Jem Kansu]

One who came up with monetization, so.

[Unnamed Participant]

Yeah, yes. Gems was one of the first features I built actually. So there's some tie in there.

[Jem Kansu]

In 20 VC, we hand out Harry Stars.

[Harry Stebbings]

It's the same.

[Jem Kansu]

I didn't come up with it actually.

[Unnamed Participant]

Damned no connection.

[Jem Kansu]

No connection at all. But it just happens to be that.

[Unnamed Participant]

And, and you can.

[Jem Kansu]

You're going to listen back to this and go, this guy was a joker.

[Unnamed Participant]

Yeah. I am curious what you. What you cut and what you keep. So it's going to end up really being interesting.

[Jem Kansu]

It's gonna be fun. It's a prize for you and me. But.

[Unnamed Participant]

So you give gems, we give gems. And you can either buy gems or earn gems and you can spend those gems in virtual features. So you can. For example, one of the most popular features features is a streak freeze. So streak, as you might know, is a very popular feature. People take pride in their streak. They don't want to lose it. But if for one reason or other you can't do duolingo that day, you can use a streak freeze. This is a very valuable item. People love it. And if you earn gems, you can spend your earned gems on it. If you're out of gems, you can buy gems to purchase a streak freeze. Items like that exist and that's part of our gem economy.

[Jem Kansu]

Have you ever had someone. Sorry to interrupt you. Have you ever had someone lose a streak and then contact you and be like, listen, I was tracking up the like Everest and you know, I dropped my phone and. And so I couldn't do my streak all the time. What's your favorite story?

[Unnamed Participant]

High volume request. I think that most, most of the stories are time zone switches. And honestly, this has happened to me, so I actually have full empathy for this where I actually did my lesson. But the reason is I flew to Australia. I was on the plane for 14 hours and when I landed, the day had already moved forward, so I lost my streak.

[Jem Kansu]

So you had to call up the head of product completely.

[Unnamed Participant]

Get it? And I. We fix those. So like if you actually reach out to customer support and they were like, I have this legitimate reason. We're like lenient. We're like, of course, like, we'll fix your streak.

[Jem Kansu]

Don't laugh. Do you have to prove it?

[Unnamed Participant]

Honestly, we really don't ask that many questions. But and, and, and we're lenient. But yes, there's, there's like time switches and there's like I was on a mountain with no connection. What was I supposed to do? And of course get off the mountain. We understand. But also yeah, like use a street freeze maybe and then get, get breeze.

[Jem Kansu]

That's why they're there. But okay, so lessons on how to do that effectively. Is it just create addictive apps that people don't want?

[Unnamed Participant]

Well like I think the core experience of course matters a lot. Addictive sounds like the bad version of this. But habit building, like you want to build good habits and the reason we, we use that term is like if you want to learn the language, you have to practice if you want to be fluent. Thousands of hours and to get to thousands of hours you need a daily habit. You can't like binge your way to fluency. So that's why at Duolingo is built around a daily habit. Once you've done an experience that people want to use daily, you have good daily retention. Then you kind of layer on these benefits that are kind of extra like streak and a streak freeze. You don't need it, but it is really core to your daily habit. And then freeze is this extra benefit that you can monetize. That's how we've thought about it.

[Jem Kansu]

Totally. Yeah. That's fascinating. What do you think people do wrong with in app purchases you see so many, you're a great product market mind. What do people do wrong?

[Unnamed Participant]

I think people do really aggressive things and that ends up hurting the experience. So for example, street freeze is so extra, right? It's, we're talking about it because it's like if you don't buy it, your duonger experience is going to be exactly the same. But it's this vanity metric that you can choose to freeze and if you don't, nothing really changes. It's all up to you. Like your French is going to be the same either way. I think what sometimes apps do is they make really core functionality and in app purchase and I think then you start killing the fun the retention producing elements of, of your app. So imagine if Duolingo was every new unit of a lesson was in that purchase. It was like a dollar per unit. You would get pretty sick of it pretty quickly. It's like buying a chapter of a book for a dollar every time you could do it. That's a monetization model but it's a pretty stupid and bad one. So I Think you want to find things that people find joy in buying that doesn't hurt the free user experience.

[Jem Kansu]

I think it actually goes to trust, which is when you do that, I actually lose trust in you and it makes it a transactional relationship that I don't even feel those I'm a premium. Like the one hit pain of like I'm a premium, fine, you pay the toll and now you enjoy the holiday.

[Unnamed Participant]

And you feel good.

[Jem Kansu]

But if it's like consistent pain inflicted, fuck that.

[Unnamed Participant]

No, it's very annoying. Like we talked about with ads. The same way tasteful monetization really matters. It builds user trust. And I think user trust goes a long way for the business because you tell your friends they download the app.

[Jem Kansu]

What are the metrics that matter most to you?

[Unnamed Participant]

There's two metrics that drive the business. So if you really squint at a business like Duolingo, you have to look at 2 metric. One of them is daily active users and the second one is revenue. But if you just want to spreadsheet a business, you have to look at DA use and revenue. But I think for me what matters is retention. So.

[Jem Kansu]

Well, because that's like company metrics.

[Unnamed Participant]

Correct.

[Jem Kansu]

But if you think about product matrix, the inputs that drive the outputs, retention.

[Unnamed Participant]

Retention is everything when you think about building product because it is the best metric to distill down if people are finding value or like liking a feature or a new app, whatever it is. What is your daily retention? I mean if it's a daily app, daily retention. If it's a weekly app, you look at weekly retention. For us, everything we build is, is around daily usage, so it is daily retention. So if we say for example Chess, where we just launched Chess, the thing we're looking at day in, day out is what is the daily retention? Because if it's really high, well, it's going to compound to high daus. We know it's going to. For example, you can look at daus. But DU has many inputs that are muddying the waters of Is this a sticky habit building product? So daily retention is the king of all metrics for consumer products.

[Jem Kansu]

And just so I understand, daily retention is. Do I come back tomorrow?

[Unnamed Participant]

Correct.

[Jem Kansu]

High bar.

[Unnamed Participant]

High bar, exactly. With mobile products, that's why it's so important because you have many different things that can steal your time on your. You don't. Maybe I didn't have anything. You're like, I don't know what apps are.

[Jem Kansu]

Okay, so we daily active users are like cool, cool. What is good?

[Unnamed Participant]

I think it's going to really depend on your product. There's no easy benchmark you can put. And also it really depends on what your user base looks like. If it's all new users, you have a different metric than if you have a bunch of users that are already settled users. I'm going to call them like Duolingo or they've already been using for like three years or above.

[Jem Kansu]

When they're settled users, do those cohorts in terms of usage look quite like Candy Crush users? Like the very, very settled gaming users?

[Unnamed Participant]

I think so. I knowing Candy Crushes metrics myself at all. But what we intend to see is if you've reached a 30 day streak, so you've built up a 30 day streak, your retention is very close to 100%. It's not 100%, but it's very high because you've built a habit. Like it's not crazy because you've done it for 30 days without breaking your streak. You have built a habit, so you're very likely to continue with that habit. But if you look at retention of users or streak less than seven days, it's way lower because they actually haven't built a habit. Or maybe they're unsure of doing or they're just playing with it. One way or another, they haven't built a habit.

[Jem Kansu]

Do you see wildly different retention numbers in paid versus free?

[Unnamed Participant]

We do. However, the difference is a little hard to parse on its causation versus correlation. Do they retain better because they are a subscriber? Or they were likely to retain because they were more engaged and that's why they decided to subscribe? It's hard to parse in the data, but if you look at just pure users who choose to subscribe, their retention and users who choose not to subscribe, retention is much higher.

[Jem Kansu]

What caused the biggest change in retention from a product perspective?

[Unnamed Participant]

Three things.

[Jem Kansu]

Wow, that was very.

[Unnamed Participant]

This is something I've thought about a lot.

[Jem Kansu]

You had that answer ready to go, didn't you? Bloody hell. I was like, that's a good question. You're like, I got that one.

[Unnamed Participant]

So one of them is the streak that no surprise having the streak and making the strength streak exciting. So when you finish your first lesson, there's this flame animation that lights up, there's a haptic, there's a sound. We make the streak like one of the most important things on Duolingo, the.

[Jem Kansu]

Details being the details, it's. It's exceptional really.

[Unnamed Participant]

The details matter a lot. So every tiny thing that might not even make sense to talk out loud. But when you see it, that made the streak more exciting. Has helped improve retention, actually. So the streak is this incredibly important mechanic that instills daily habit, helps you learn a language or learn a subject. That is one of the biggest things for retention, I would say second as a feature was leaderboards. Turns out, no surprise in hindsight, but if you see your name ranked on a leaderboard, there's this human tendency to be not at be at the bottom of that leaderboard, but instead move towards the top. So a lot of our learners, even though when they came to duolingo, they didn't come to compete in French, we have a leaderboard feature where you see your name among a cohort of similar learners winners. You want to rise to the top. A lot of users want to at least. So that drives a lot of usage, a lot of retention.

[Jem Kansu]

Who's like, no, I'm happy languishing at the bottom.

[Unnamed Participant]

I think some people are like, I don't want to compete. But a lot of users, when you know you see your name, you're like, I would rather move to the top, not to the bottom. I'm one of them for sure.

[Jem Kansu]

For sure. Do not understand the people. I'm happy just being there. Like, well, you're not getting hired, are you? Say, the worst tiring trait ever. Thoroughly uncompassitive. Happy losing.

[Unnamed Participant]

Exactly. Exactly.

[Jem Kansu]

Okay, that's Love that.

[Unnamed Participant]

So that's the second one. Third is push notifications, actually for every app. It is an incredible mechanic that you have to figure out how to make amazing. Still, still, maybe 10 years ago it was email was a heavy channel. Email is a dead channel for most products at this point. Dead is maybe an overstatement. Like we obviously invest a good amount of time in making our email.

[Jem Kansu]

The email team. We still love you. Don't worry.

[Unnamed Participant]

Yes, we do love you.

[Jem Kansu]

I hope you enjoy the dark basement with the le in the corner. It's fine. Lewis will let you out next year.

[Unnamed Participant]

Yeah. Where the push notification team is like popping champagne.

[Jem Kansu]

They're at the Belmont in Portofina celebrating 20B.

[Harry Stebbings]

So, yes.

[Jem Kansu]

What does it take to do push?

[Unnamed Participant]

Well, you can get incredibly complicated and this was kind of news to me over my career. Incredibly complicated with the limited real estate and copy you write in a tiny appreciation notification. For example, timing is a key factor. Content is a key factor. Personalization of that content is a key factor. Emoji usage is a key factor that moves metrics. So if you get into the full details of really optimizing push interviews, which we have, you can do a lot with it. And every year we're like, okay, we invested a lot. Are you going to keep improving? There's a lot to do on improving push notifications. I'll give you one insight that was kind of like tested in and out and then this turns out to be the winner we schedule. So let's say you did your duolingo Lesson Today at 1pm we schedule a notification almost exactly at 23 and a half hours later from that time. And we've tested many different timing. And the notification will remind you to practice your Spanish because we. You just did a Spanish lesson. We've tested many different times, but the 23 and a half hour works way better for retention because turns out humans do lessons. If you're a morning person, you do in the morning. And if we schedule it almost 30 minutes before you're ready to do your morning lesson, you're very likely to do a duolingo lesson at the same time. So based on your behavior, we time the next notification that has done really well for retention. So when you get into this type of experiment.