← Marketing Against The Grain
Marketing Against The Grain
How a $1B+ Crypto Company Really Uses AI in Marketing | ft. Kraken CMO
00:47:28
Loading summary
Transcript132 lines
- [00:01]
Kieran Flanagan
On today's show, we're covering all the AI questions you've been too afraid to ask. What we're doing is breaking down with Maya Gupta, the CMO at Kraken, one of the best crypto exchanges in the world. We're going to go through how do you use AI, which model is best for which task, and what's the best strategy for AI adoption in your company? You're going to leave this episode with a whole new framing for how to move forward with AI. Let's get to today's show. We right back to the show. But first, a quick word from our sponsor. Remember when marketing was fun? When you had time to be creative and connect with your customers? With HubSpot, marketing can be fun again. Turn one piece of content into everything you need, know which prospects are ready to buy and see all of your campaign results in one place. Plus, it's easy to use helping HubSpot customers double their leads in just 12 months, which means you have more time to, you know, enjoy marketing again. Visit HubSpot.com to get started for free.
- [01:16]
Kip Bodnar
Hey Meyer, welcome back to the show. We are happy to have a second time attendee on Marketing against the Green.
- [01:23]
Maya Gupta
Thanks for having me, Kieran. Thanks both you and Kip. Great to be here.
- [01:27]
Kieran Flanagan
Thanks for being here.
- [01:28]
Kip Bodnar
So kind of reintroduce you to our audience. I think you are our very first guest. So you've had a pretty incredible career. You're currently the CMO and chief growth officer over at Kraken, one of the most impressive companies in the crypto space, but just a very impressive company in general. And we wanted to get you on because you are a forward thinker. You were hidden up growth in Spotify. You've led marketing and growth and freshly just a ton of incredible roles usually in companies that are actually trailblazers in their industry. So of course we wanted to get you on to talk about AI because we want to know how smart leaders and just smart people in general are using AI. So maybe we could just kick off with you giving our audience a little bit of context on where you are seeing AI get used across your marketing and growth teams. What are some of the use cases that you particularly find pretty interesting?
- [02:15]
Maya Gupta
Yeah. Well, let me maybe if it's helpful, just give context on what the growth theme at Kraken looks like. We are obviously like many teams still experimenting. Some areas are more mature in using AI. Some we are still learning. But the growth theme at Kraken, now in its current stage, which I call our third era, In a way is pretty much end to end growth. So it has marketing, we have a pretty decent sized growth, analytics and research team, product design, product engineering, obviously product led growth as well. And to be honest, the two areas where we started using AI and experimenting was around creative both from a brand and storytelling standpoint as well as performance. A lot easier on performance of course, because that's where you need a lot of velocity and variety and AI does a great job in giving you those variations. We're also a global organization, huge footprint in Europe where there are so many different languages. So know translation has been a huge area for us both in product and off product. But I've been really inspired by our research team which is a pretty small unit, but they've been leveraging AI quite a bit for both corn and quant. You know, using platforms like Boltai for some of the call work. They've been using Pollfish Again I personally haven't dug in a lot on these platforms yet, but I know there's a lot of internal inertia to see. How do we bring more efficiency, more velocity in doing stuff that was taking much longer in the past and also increase the level of quality that we get in some of these functions.
- [03:43]
Kip Bodnar
What are you using it for actually on brand? Because we do get a lot of folks on here when Kip and I have guests on. It is a lot of performance marketing. It is a lot of informational content. I would love to hear because Kraken do a ton of smart stuff on brand campaigns. Think you guys have a formula one or part of the formula one as well. So like what you. What are some smart things you do on brand?
- [04:05]
Maya Gupta
Yeah, it's harder, you know what we feel and we have this discussion at least three or four times a week. We have a channel on Slack where we are brainstorming, looking at all the new tools that are coming in and brand, I think it always is where AI is helping us get smarter and come up with more ideas, idea generation versus actually coming up with the end asset. Also because a lot of our marketing right now has pivoted to being very product focused, we are highlighting the value proposition and the rtbs for our core product. The interfaces, you know, we are trying to show how customizable and flexible our dynamic interfaces are. For our Kraken Pro product for instance, which is one of the best in the category and purely with AI, is very hard to focus in on a graph, focus in on dashboards. So what we are learning is, you know, it's great to give US ideas, it's great to give us different variations, but when we are zooming in in certain angles, that's where you still need, you know, your more traditional way of creating content. And the other place where the team is really moving fast is sometimes you have long form content. Like we have a lot of influencers and kols that we work with, but the team is using different types of AI platforms to create shorter versions of that content to drive faster distribution across social. So there are a lot of clip anything type of platforms that you use to create 100 different assets from one long form asset that you may have with an influencer, which then you can distribute at much higher velocity.
- [05:32]
Kieran Flanagan
Maya, I like to add in on that. I think what's missed in the brand creative, product marketing side of things is that historically there's just been a lot of time and money spent trying to guess is this idea match what the person who I'm trying to communicate with? Like, does it match what they actually want? And I think what you're saying and what we found at HubSpot is like, AI is a fast and cheap shortcut to that problem. We use Claude internally on the HubSpot marketing team. Everybody has a license to Claude. We have cloud projects and so we have a whole project just all around our core buyer Persona. And so anytime we're writing a product page or building a brand campaign, doesn't matter what it is, we can just ask basically a fictional version of our customer what they think, give us feedback, what resonates, what doesn't. And it used to have to spend a lot of time and money on focus groups and market research and stuff. And you now the cycle times can just be so much faster. It feels like. Are you guys seeing that?
- [06:26]
Maya Gupta
Yes.
- [06:27]
Kip Bodnar
Yes.
- [06:27]
Maya Gupta
You know, obviously this conversation is lighting me up to think about what other things we could actually do in our world almost right away. You know, one element of what you mentioned, Kip, which I think is different, that AI brings different from a traditional model is see when you're trying to figure out how your customer will respond, a creative team or one of your sub growth teams, they're engaging with an internal research team to understand, okay, tell me how you know what the feedback would be. They go into market, they run the study, they run call and con, and they come back not only speed and velocity, but this is a direct hook that now the growth teams have where they don't have a dependency on another layer. So it also helps them get a deeper understanding of what the customers may respond to and Also bringing more agility because you are now doing that not once you've had an MVP of an asset, but you're actually doing it while you're incepting in much early on in the phase.
- [07:21]
Kip Bodnar
Yeah, I think research in general, I think you mentioned it. There is one of the most interesting use cases, the latest releases from deepthink. We covered O3, their research capabilities. They actually are displaced in most research roles. But we have covered it before that. There is a really interesting way for these AI models to replicate your customer, which we were talking about, like how it can be the voice of your customer because it's basically trained on the data of the Internet. But some of the recent agents, like the operator agent or another one that we're going to cover soon on this channel called Proxy can actually help you laser in on parts of the Internet. So you could say like go to G2 crowd, go to these other things and use that data to replicate my customer. And so then you're able to actually have a back and forth. There's a company that I'm an investor in called Hyperband and, and they actually do this for companies where they'll build an agent to replicate your customer in a bunch of different scenarios. And then rampant sales reps can call that customer and have a sales conversation and train on calling that customer. So I do think it's a pretty interesting use case in the future which is, you know, we've had multiple shows where today you have this billion dollar, I think it's multi billion dollar industry. I don't know if you know what it is because it's like $3 billion. This market research industry.
- [08:38]
Kieran Flanagan
Oh, we looked it up before. It's. It's more than that. It's like 8 to 10 billion. It's massive, massive. The market research business.
- [08:43]
Kip Bodnar
And like can you pay a thousand people to like come back in your data? But the ability for AI to just do that en masse I think is going to displace a lot of that need as well. And so I do think there's a future where you'll be able to test your creative, test your brand campaigns and get pretty instantaneous feedback. But you'll be able to pair that with like internal data as well to be able to make sure a marketer can get things right more than they get things wrong.
- [09:11]
Maya Gupta
Yes, I think on the research side we are absolutely seeing that the pace at which we're getting feedback is dramatically different from when we are using a traditional model. Now one area where we've taken A certain path where we've created local instances, let's say of ChatGPT, because there's confidential data. We are obviously for the nature of the business that we are in. How much are you guys seeing that being done versus most brands and businesses leaning on all the data being in the cloud. So one of the things that we are doing at Kraken is we've created our local instances where all the unstructured data is actually being pushed into that local instance and is being trained. And then you're just asking a lot of questions and queries, even just to understand user behavior, because over a period of time, historically these have been PDFs, decks and docs with tons of data. But I think creating a local incent just give that access to thousands of crack nights pretty instantly.
- [10:07]
Kip Bodnar
Yeah, this is actually a super important point. I think Kip and I have talked about this. The number one way to make this very impactful within your company is to be able to provide the ability to collapse all the unstructured data into a repository that then you can just have the employees easily access. Like so when they build things, it can access that unstructured data. Obviously, Kip and I are kind of maniacs, so we have.
- [10:28]
Kieran Flanagan
That's the nice way to say it.
- [10:30]
Kip Bodnar
We have personal pay plans, I think, to every AI tool and we're fortunate enough to have enterprise seats to every AI tool. But we do have local instance as well. Very similar to the way that you guys sound like you're building it.
- [10:39]
Kieran Flanagan
Yeah, I think there's a couple of things here that I would share. The unstructured data unlock is crazy, right? And if you look at some of the reports with Google 2.0 Flash, its ability to extract data and insights from PDFs for essentially 40, 50 cents is like unreal and kind of unmatched. There's two things we've decided recently to do on our team, Kieran. Right. Which is one, we're in the process of having technical writers document all of the workflow so that we can then be very clear on what we're going to automate and what we're going to automate now, what we're then going to automate in the future. And then the second thing is every meeting is recorded. Yeah, every meeting is recorded. And we are creating like a whole nomenclature for how those recordings and transcripts are stored. So it's just like everything's recorded. We are just going to unlock the creation of unstructured data. And it was sweet. Like I had a big off site last week with a few people. Kieran, you were there for a little bit and we recorded everything. Then you dumped all that in Claude and we could go and basically build follow ups and everything from that. It's really great. So I think those are two things that we're doing. Kieran, I think the other thing we could talk about for everyone that I don't think has been talked about yet that Meyer kind of brings up is when do you like buy a business seat of one of these frontier models? When do you maybe build a app on top of their APIs that's specific to your company? Or when do you maybe like run an open source model locally? I imagine you, Kieran, have some opinions on that idea. I want to kind of hear what you think and hear what Maya thinks there.
- [12:10]
Kip Bodnar
Okay, that is actually a pretty good question. Right, thanks. The one that I thought about much more deeply is I will make sure is this the same thing which is when you are trying to stand up AI use cases within your company, do you build that custom? Do you use a vendor and I guess build a split between open source and non open source? Like do you commit to a certain model? I think that's one of the harder things to figure out right now because that's like one of the things that I'm trying to figure out all the time across our go to market which is when you are trying to stand up things. In AI, the most important thing is to get signal as quick as possible because it is really hard to know what the capabilities are of the model at scale. And so there's two things there. I still think in a lot of cases you do have to customize it a lot to your needs because I think AI works best when it replicates workflows that are unique to how you do things. Right. The best AI companies actually are shipping daily because they're sitting with design partners looking to see how these people work and then ship into their needs to integrate AI seamlessly into the workflow. And so you actually in a lot of cases have to build custom off the shelf software is even really hard to get signal. I would still always want to start with off the shelf because I would want to try to prove the use case as quick as possible versus committing to building something internally. The open source one and the closed source one is actually a really interesting question. So I've always believed open source is going to be a big part of how companies use models internally. But to really use open source you need to have a layer on top that can actually automatically fine tune the open source models and then orchestrate between them based upon the use case you want to do. So you can imagine you're trying to do something in the future and you're using some sort of interface and that interface basically is able to capture the intent like the use case you want to do and then in behind the scenes it can orchestrate and send to the right model dependent upon your needs. Right. And that does not exist. Like I actually try to invest in some companies doing that, but it is actually a way harder problem to solve than I first thought. So I don't know Maya, if you have opinions on that, but that would be my like off the top opinions.
- [14:15]
Maya Gupta
Yeah, this is a great point. I think it's a multidimensional framework, to be honest, because there are so many different layers to figure out what your strategy should be. On one hand I think it's a crawl, walk and run. I think the lowest hanging fruit for any business and any team is leverage a SaaS platform and experiment it and prove incrementality and then you figure out the path forward. I think the path forward is determined, I feel, based on a couple of things. One is, is that core to your core product offering? If it is core to your core product offering, you want to own the ip, you want to build it.
- [14:44]
Kip Bodnar
Yeah.
- [14:45]
Maya Gupta
And that is core within the product, so that plays a key role. Second is, you know, the security and confidentiality of that data. So depending on the use case, if it's creative and if you're playing around with that, no brainer. I think Most of the SaaS platforms, most of the cloud based solutions today will fulfill most of your needs. And if there is a very unique edge case, then it's a big question. Kiran, I do feel you have to ask is, hey, is a cost benefit analysis strong enough where you go down that path where you customize or you build an adaptive layer on top of it, or is it reasonable enough that it can solve 80% of your efficiency and then 20% are edge cases. But the local instance point that I made earlier, it is because there is certain type of data that we just would not want to leave our four boundaries and we felt that, okay, we can create a local instance, continue to train it over time with all our unstructured data. So I think there are three or four different layers and I feel either which way you got to start with experimenting with what's out there and absolutely see some of the early signals.
- [15:46]
Kip Bodnar
Yeah, I agree.
- [15:48]
Kieran Flanagan
Look, I think to chime in to try to help everybody watching and listening to today's show. What's going to happen is a lot of companies are going to be like, oh, I'm behind, I want to get into this AI game. And they're going to go by business seats of ChatGPT, of Claude, of Gemini, pick a core frontier model, and that is a great place to start. Really, really good place to start. There's nothing wrong with doing that. What you will find is if you have very specific use cases, building small applications on top of the API is like orders of magnitude cheaper. Yeah, right. Like way, way, way, way cheaper. And so if you are very focused in what you're trying to do, I think you're going to be better off to build custom. If you are general use cases trying to figure that out, then probably going and buying those seats is probably not a bad place to start. When you buy the seats, you get a bunch of features and those features continue to improve as they roll out and make the product better. But if you have like, very focused use cases where there's a couple of very specific things you want to do and use AI for, you are far better to build off of an API like a basic web app or agent, because it's going to be way, way, way cheaper. And then I suspect if you're hosting things locally and you have specific use cases or data privacy concerns, that's probably where open source is going to come in. You're going to take an open source model, you're going to run it locally, you're probably going to pick one that's best for the very specific problem or small set of problems you have. Right. And that's, I think, probably the three sets of choices people are making today.
- [17:13]
Kip Bodnar
Right?
- [17:13]
Maya Gupta
Yeah. I do have a question from you guys, because you, of course, have been digging so deep into AI and talking to so many leaders from an operational standpoint, what have you seen being most effective when AI is assessed and adopted and evaluated within each team? Or are you saying, okay, actually find a person across the board, across teams, because, look, there's so much innovation happening. It's also noise. Right, right. Because you're also running your core business. You're trying to hit those KPIs, you're trying to hit outcomes, but this is all relevant. So operationally, what have you seen being most effective in terms of experimenting and identifying what are the right tools to play around for what type of use cases?
- [17:57]
Kip Bodnar
This is a great question. Kip and I spent a bunch of time on this last week actually. So this is really timely. I kind of divide it up into three parts in terms of AI adoption within the company. So we have these top down goals from the company and it's basically just a stake in the ground. It's saying like we believe these places across the company should be transformed by AI because we know where AI capabilities are today. But the thing that I, Kip, kind of build for is we build the use cases that we believe AI is going to be transformational for in the future, regardless of where the model capabilities are today. And what we mean by that is, I think a year ago we talked about this. Kip, which is the model capabilities is a solved problem.
- [18:37]
Kieran Flanagan
It's time and money.
- [18:38]
Kip Bodnar
Yeah. The model capability, don't worry that it cannot do what you want to do today. Build the infrastructure and the setup to do that thing and the model will solve itself. That was 12 months ago. If you look what's happened in the last 12 months, that was pretty accurate. Right. The model capabilities have been transformative. It's speeding up, not slowing down. So I think putting the stake in the ground and having large, big bets at the company level and then having a large pod that can go after those bets and they should be run. The big thing that I think and what I've seen in other companies is it should be run like a growth project, not an IT project.
- [19:08]
Maya Gupta
Yeah.
- [19:08]
Kip Bodnar
You have to have a growth methodology to how you approach those. Because AI in and of itself is an iterative technology. It is not a deploy the technology like regular software. And there you go. Right. It is a very like learn as you go. Then the second thing is team enablement. And then the third thing is employee in general enablement. So I'll give you a somewhat take on employee enablement. That might not go down well with folks. Right. I think on the employee side of things, the job of the team and the company is to provide the tools and the kind of permission to go and play with AI, integrate AI into your workflow, figure out what works, what doesn't work, like play around with it. Here's the tools, here's some courses that you might want to go take. Nothing is mandated. And if you don't understand that this is a paradigm shift in how you do work and you don't want to integrate it into your work and you are not curious about it, that's on you, not the company. You should not have to force people to understand that this is one of the most important things that you can do. The second one is the hardest one actually. And that's the one I want to pitch over the kip, which is what do you do at the team level? So should teams understand how AI can transform what they do, or should there be a central AI team that goes into those teams, spends time with them, looks at their workflows and it is the AI specialist team and it understands how it can transform what they do. And I think actually either or of those could work. I don't know if there's a right option.
- [20:30]
Kieran Flanagan
Here's my take because this is a really good discussion and a bunch of marketing leaders and VCs, you know, listen to this on RSS and so I'm sure we'll kick off some debate. I think you have to have both of what Kieran just said is the honest answer. I think the number one way to get AI to take off in a team or within an organization is to centralize some valuable unstructured data use case like the second. Our product marketing leader had a couple projects in Claude that was like, here's everything we know about our Persona and you can just get feedback from that Persona because I've taken all the time to train it, upload all of our decks and docs and everything there, and I've tailored the output and everything for you. She did another one about like removing any like, business jargon and stuff from your copywriting, like basic things that like everybody can use. That is how you get real adoption. And so I think the individual teams are required to be the experts to find and gather the unstructured data, put it in the frontier models, Claude, ChatGPT, whatever you're using, and make that accessible to their team and the broader team. I think you then need a team of specialists who are experts in AI and automation to say we can now automate 90% of localization. This is a solved problem. We got DeepML, we got a bunch of different tools where we can make localization 10 times better. And we're going to go run and build very specific workflows, custom software, whatever it may be, to go and solve that problem that like, your core team isn't going to have enough knowledge and expertise to go into. We can go and train that team and do everything we need, but I think both is the only possible outcome. Kieran.
- [22:08]
Maya Gupta
Yeah, yeah.
- [22:15]
Kieran Flanagan
Let me tell you about a great podcast. It's called Creators of Brands. It's hosted by Tom Boyd. It's brought to you by the HubSpot Podcast Network. Creators of Brands explores how storytellers are building brands online from the mindsets to the tactics, to the business side. They break down what's working so you can apply that to your own goals. Tom just did a great episode about social media growth called 3K to 45K on Instagram in one year, selling digital products and quitting his job to go full time creator with Gannon Mayer. Listen to creators are brands wherever you get your podcast.
- [22:52]
Maya Gupta
I totally agree. I think it's a hybrid model and I was thinking through how are we running it and what are the teams that are running it successfully? The big outlier or the big difference in the scenarios is anytime the actual business team and that could be marketing growth product. Anytime they are the ones generating the inertia, they are the ones pushing the use case. We are seeing better results and outcome. Now of course in some cases they'll have a dependency on somebody in a global data team or engineering to figure out a local instance of something or create customization. But anytime they are the ones who are driving and pushing for the outcome because they are closest to the efficiency opportunity in what they're doing, we are actually seeing success.
- [23:36]
Kip Bodnar
Yeah, I think that makes a ton of sense. I think curiosity has never been more valuable than it is today. I was talking to a friend about this earlier on. Your time has never been more valuable than it is today because the opportunity to apply it to AI is so huge. Like, I think about this all the time, which is I have a really high bar for what I should spend my time on. If it's not AI, if I'm doing something during the day and it's not AI, I'm like, why am I doing this? Like, it has to be a really high bar for this to be important. I don't think that's healthy, by the way. I'm not saying that's what you should do. I actually don't think it's healthy because it forces me into a bunch of like tailspin.
- [24:11]
Kieran Flanagan
Well, the best quote I've heard on this is from Dan Schipper, founder of Every friend of the pod. He was reviewing OpenAI 03 mini and high and he was like, these advanced reasoning models are a bazooka for the curious mind, right? And that is like the perfect metaphor of like, geez, if you are curious. Just like you take out a whole big problem really quickly and learn something. And I'm somebody who's, for better or worse, addicted to learning. And so like that just becomes insatiable. And you're right, Keir. You just stop being like, why am I doing anything that is like a bottom 20% thing ever.
- [24:48]
Maya Gupta
Yeah, that is so spot on. And I almost feel it's like social, you know, when social became a thing, whatever, 10, you know, 14 years back, you could not just assume that you would understand the nuance of social and building communities if you yourself were not in it.
- [25:03]
Kip Bodnar
Exactly.
- [25:04]
Kieran Flanagan
Yes.
- [25:04]
Maya Gupta
You have to live and breathe it to then say, oh, I have a chance at building a brand that is distributed through social and build communities. So, believe it or not, of course I follow both of you and I read a lot of your content. Late last year, as I was getting into the break, I actually dove really deep in Claude myself with my financial data. Personal, personal financial data. Me and my wife, we always end up overspending and, you know, like, it's a cluster F. When you actually look at your statements, there are like thousands of rows and your Excel eventually breaks, so you can't really do anything. And then you look at all kinds of pie charts that typical financial systems give you, but it doesn't really tell you anything. So I read a lot about Claude and I saw one of the posts from Kiran where you had shared. Here are the five or six platforms and tools you're using. So, you know, I use the three or four weeks to get into Grok and Claude and Gemini. So what I did was I cleaned up all my data, I removed all the PI, man, I dumped all the CSV files into Claude, and I went crazy asking all kinds of questions because why are we overspending? Where are we spending? You know, and what are the repeatable spends? My wife thought that she's overspending on Amazon and I corrected that because, you know, this clearly said, no, this is how much you're spending on Amazon versus, you know, all of the restaurants. And we were surprised that despite having our own car, we were spending so much on Uber in a span of three months. And then we. So I think the only way you unlock AI in your professional life is when you're actually living and breathing it and finding incrementality and finding those use cases in your personal life as well.
- [26:35]
Kip Bodnar
100%. I think personal life is a big one. What I do now is everything I do, I start with AI and the financial ones. That's a pretty interesting one. I do something similar where I plugged in my portfolio to OpenAI and then asked it, how can I diversify this more? I made my first investment two months ago in a clean energy fund. I had no idea what this clean energy fund was. Nothing. Like, it was all recommended by AI, broke down By I added to my portfolio for diversification in climate change. And I just said I'm going to like, just invest in this and see what happens. Right. Because I'm so committed to it. I'm like literally putting my money into. And so I think that is the start. There's actually a use case I wanted to show you, Meyer, because I think you're the best person in the world to show this to, which is one of the things I've been fascinated by is how the reasoning models have become better with strategy.
- [27:27]
Kieran Flanagan
While you're pulling that up, my son has like medical stuff and like if I get a complex medical report. Yeah. Like I just upload that and get like the full, real, like summary of it. Yeah. Before, like waiting for a specialist is.
- [27:38]
Kip Bodnar
Like magic is exactly. The health ones. I have all of my health data now in a project.
- [27:44]
Kieran Flanagan
Right?
- [27:44]
Kip Bodnar
Yeah. In one project.
- [27:46]
Kieran Flanagan
That's super smart.
- [27:47]
Kip Bodnar
Just to give an example of how incredible AI is. Not that I was not listening to our conversation was not dialed in, but I wanted to set this up because I wanted to show you. Keep in mind when I'm going through this, I did this whilst we were talking on the pod. Wow.
- [27:58]
Maya Gupta
Okay.
- [27:59]
Kip Bodnar
And. And the reason I'm saying that is because I think it would take a growth team a week or so to be able to do this. Now, one caveat is this is with synthetic data, because I can't show real data on this show. I've been trying to get good at trying to create synthetic data to show use cases. The synthetic data is just not complex. So what I've given it is like a dashboard for a growing SaaS company at 30 million in ARR that wants to double its ARR over the coming years. And so the first prompt is. And if you're subscribed to the podcast and you're watching this, we'll put the sheet for the prompts in the YouTube comments. You don't have to try to look at the prompts, but the first prompt is basically just saying, okay, well I need to get to that number in 12 months. Of all the metrics that you see here, what are the three to five I should focus on? And I'm going to skip past this a little bit because it's not that interesting because again, it's synthetic data. So the data was pretty obvious that it picks out. Churn rate gives me examples of why it picked out that tactics and trade offs picked out. ARPU picked that conversion rate free to paid. Again, this is a PLG company, so it has premium tier, starter tier, pro tier and enterprise activation rate and then referral rate. Cool. And so then I ask it to put together this and this is where it started to get much better. Now this is O1, this is not O3. The reason it's not O3 and I don't think this is European thing, I think this is just an everyone thing. I can't upload files or even graphics. I can't upload anything to O3 yet. So then it will take those metrics. Right. All I'm saying is like create a growth plan and I'm going to go through the incremental improvements I've made. But it starts to get really interesting. Right. Basically I don't have to tell it to do hypothesis. Right. It does its own hypothesis. It starts to give you pretty great experiments. When I first looked at this, which is just like as we were going through things on the podcast, I think it's as good as an average growth team, if not like a pretty competent growth team where it will give you the kind of experiments you can run to improve churn rate. There's one here that is actually pretty interesting. So it started to do things now that I found it never used to be able to do, which is give me something that I hadn't thought of. So this one here, early warning outreach, it's telling us to like basically trigger usage signals and so when you see someone drop in daily active users. So you take your daily active user cohort and then you trigger emails to cohorts where you see the daily active usage drop off over time. Yeah, which is actually a pretty smart thing to do.
- [30:15]
Maya Gupta
One question, Kiran. Yeah, my mind's buzzing. Two questions actually. One is, are you also uploading your UX and design flow for it to understand where the opportunities could be?
- [30:26]
Kieran Flanagan
That's a good question.
- [30:27]
Maya Gupta
That's one question. Let's stop there because I'm very, very curious about how to leverage this.
- [30:32]
Kip Bodnar
This was going to be my ending point, but I'm glad you brought this up, which is if I actually went through this and so I'll go away to the end because I want to cover your point because it's part of what I was going to cover at the end. All the way at the end. I've asked it to continually think more deeply. If you ask it to think more deeply about these problems and do another version, you'll get better results, better results, better results each time. Now, there's been a release lately on how deep seek and others and these reasoning models were Managing to get better results. And one of the key things was just every time it thought it was finished giving you the answer, they would just give it the word wait, that's it, wait. And they're doing that to say think more. And what it's doing in the background is it's looking through all of the things it's giving you and stack ranking them and trying to give a better one, a better one and better one. So that's the reason the reasoning models in the background are trying to reason out what is the best answer they can give you. Not too dissimilar from search, but like I think somewhat more sophisticated. And then it's given me a big swing. But your point is really important because I'm pretty sure and I know because if I was showing you the internal use case, the more context you give it, the better. So the way to actually make this incredible is you give it your actual real data, which for Kraken I assume is very complex. Right? Which would be better because then the reason the model would be able to decipher the first question of like really tell me the metrics that matter in this complex model. That is actually really important. The ones that's picked here are pretty self explanatory. They're just like best in class PLG metrics. It would be interesting for a business like HubSpot or Kraken it would pick more interesting things. The second point is okay, we'll turn that into a growth plan. But actually here's all of the experiments we've run. So you have a library of all previous experiments which again speaks to the fact the most important thing to do for AI to be impactful is documentation. Boring thing you could ever think of. It's really documentation and you load in all your experiments and then to your point, which is the next version I want to do internally, which I think is a great idea. We do have a ton of wire mapping and flows from our customer journey and so I would actually just add all of that into the context window as well. And I suspect you're going to get a first version of a growth plan that needs to be edited but does not in any way need to be rebuilt.
- [32:42]
Maya Gupta
Yeah, okay, here's what I would love to do, man, I would love to come back in four weeks because I'm not giving you guys much other than some abstract. But all I have to figure out today and I'm gonna tell you the type of growth questions we are trying to answer that any business that is global is trying to answer. And I wanna bring some of those use cases back. But the only caveat is I need to figure out the local incident because we won't push that into cloud. I think that is too scary for me to even think that I'm gonna do it. Doesn't matter what enterprise account, but I get it. Here are some no freaking brainer questions that I can assure you this guy would or any of these platforms will help us get much faster. One is in a global company, I'm always looking at growth rates, difference between geographies. Yes, no brainer. Just tell me hands down, why is activation rate stronger here and worse there? Now they're all just looking at all the variables, all the inputs in the activation rate and telling me what the differences are. 2 We always look at, okay, who's a high value cohort and who's a lowest value cohort, but trying to figure out session analysis to understand, okay, what is it the high value cohort doing beyond the aha moment, you know, that is driving maximized ltv. Now all that is doing is data crunching. It's just trying to figure out patterns, which is what AI can do a lot faster than a data scientist would. Running different queries. And then obviously we focus so much on payback period because we challenge ourselves to have very strong fully loaded payback periods and then starting to look at even just basic stuff. Okay, what is the IRR on your cac? You know, stuff where we are spending a lot of energy which is rather mechanical or you are searching for nuggets. If I can go back and figure out a way to dump all my raw data, I'll come back with some very interesting insights that we are generally trying to solve every single day, but it's taking us long.
- [34:32]
Kip Bodnar
Yeah, and I'll just give one quick tip there. When you get the internal data and you have those internal trends and you can anonymize it because I'm going to say use deep research here and you might not be able to use that locally, I don't know. But you can anonymize it to make you feel better. Just give it the trends, pair it with external trends in those geographies. This is exactly what I was going to say. Yeah, okay. Okay.
- [34:54]
Kieran Flanagan
It's deep research plus the similar web API.
- [34:57]
Kip Bodnar
Yes. Yeah, exactly.
- [34:58]
Kieran Flanagan
What, what happens is you get all this data internally in any kind of company of scale. If you're a 10 person or a hundred person or a thousand person company, you get some and you're trying to contextualize it. Right. But you're trying to contextualize it largely on like anecdotal information. And once AI helps you synthesize it and find the key trends and timing on those trends, you can just basically bring in deep research, similar web API and a couple external sources and like it'll correlate very perfectly for you. But I promise you, you guys have.
- [35:27]
Maya Gupta
Totally changed my mindset in 30 minutes.
- [35:31]
Kip Bodnar
That's what we're trying to do.
- [35:32]
Maya Gupta
I mean, look, obviously I've always had this hurdle to think about. How am I going to apply it? It's always in the back of the mind. We are spending so much energy right now. We know exactly what questions we're trying to answer. We know where we are, we know how we're going to drive growth. But the latency in getting those insights is just insane. Right now what I want to go back and do is pick somebody from my team to figure out how we're going to launch. And then maybe you're right. You know, there is elements of anonymized data which we can actually upload and put patterns. And interesting thing is in crypto, market behavior is easily capturable because the biggest market driver is a global bitcoin price and we have a lot of price indexes. So that can eliminate that variable and really understand what else is happening, you know, in a particular geography different from another one.
- [36:19]
Kip Bodnar
Yeah, I would say you have actually incredible external trends as well. Like I would say there's some like really interesting patterns across geographies in your external trends because, you know, it's such a vibrant and popular space.
- [36:33]
Maya Gupta
Yeah, yeah.
- [36:34]
Kip Bodnar
I think that's a great place to leave it. The last one I'll leave you with that I think is like pretty interesting is I have all these different project assistants. I did one for growth. Now a project assistant is basically we have an ea. She's incredible. But like to be across every single thing you do that is hard to scale.
- [36:50]
Kieran Flanagan
It's impossible. Yeah.
- [36:50]
Kip Bodnar
I've started to build these project assistants for different projects and there is a good one for growth which is if you have a Google Jam, OpenAI, custom ChatGPT or cloud projects, you can just literally not care about the structure of anything that people are giving you as long as it's specific to this project. Just say like, hey, put all the things that are related to this project, your updates, your experiments, whatever they may be in a single decks, doesn't matter. Everything in a single folder. Right. Named project, which is usually a goal like increase activation rate by 50% in 2025, everything goes in a folder, then everything that folder goes into Project Assistant. It is pretty incredible. You can get everything you need. If you're saying like, tell me about the three experiments in January that we did, the one that basically led to the biggest impact and why the other two failed. You never have to go on this like conundrum of slacks and emails to try to figure out stuff and that. And again, these mundane use cases are the ones where there's huge upside because if you're managing hundreds of people, all of that stuff takes hours and hours of your day. And that one I have found to be incredibly valuable. And the last thing I was showing KIP earlier is like, it can build out interfaces dynamically for you in the actual console. So example would be anytime I do a meeting, I get the meeting transcript. And I said, add all the follow ups into a table. And so it will keep updating the table in the AI instance. And I say remove this and I add these ones. I've done those. It's just for me, you can't share it with people easily because it doesn't write back into a file. But it's unbelievable. Like I can't describe how much of a game changer these assistants are. The key is to attach them to a singular project.
- [38:25]
Kieran Flanagan
Yes.
- [38:25]
Maya Gupta
Yeah.
- [38:26]
Kip Bodnar
And that's your repository for the data for them. And then they are your assistant for a specific project. And then you don't have to care about all of this crazy structure. You just say, please put all of your updates into this folder.
- [38:38]
Kieran Flanagan
You're like, why do I need project management software?
- [38:40]
Kip Bodnar
Yeah. I don't think you do in the future. You definitely don't in the future.
- [38:42]
Kieran Flanagan
It's kind of wild.
- [38:43]
Maya Gupta
Yeah. And actually, even if you do, even if you're using it, it's not effective because as a human brain, you're not actually mapping out all the dependencies.
- [38:51]
Kip Bodnar
Yeah.
- [38:51]
Maya Gupta
You're not thinking through. And also the worst thing, the point you made about, okay, what are the last three experiments you did and which one worked and why? The one that didn't work. The challenge we have today in the mechanical operations is we are not going back. We are not looking at what we learned from those experiments. Because.
- [39:08]
Kip Bodnar
Because it's too hard.
- [39:09]
Maya Gupta
It's too hard.
- [39:09]
Kip Bodnar
It's too hard. It's so hard. Exactly.
- [39:12]
Maya Gupta
Yes. So it's not only that we are inefficient because it's so manual, but then we become ineffective because we are not technically applying the learning from experiments that were done and imagine if you were not the one who did it.
- [39:24]
Kip Bodnar
Exactly.
- [39:25]
Maya Gupta
Then it's even harder.
- [39:26]
Kip Bodnar
Right.
- [39:27]
Kieran Flanagan
Literally one of the best things Kieran, you and I did in the last six months was like we were going through planning at the end of last year and we just took some of the raw decks and data from just planning and we were just put them in ChatGPT and we were just like, what are all the dependencies we're not accounting for? What's the stuff that's going to go wrong that we're not thinking about? And you just like the overlap. Historically that's just been really, really hard. And we got like a great readout. We changed a couple things. It was awesome. And you're like, it took like 20 minutes and you moved on. It was great.
- [39:57]
Kip Bodnar
Yeah.
- [39:58]
Maya Gupta
You know, with all the content that you guys are sharing, like what's the best way to get the portfolio of the platforms or different type of use cases is like how do you feel if the team's now going into the second gear? For example, the gear one is okay. Hey, we've tasted blood, we've seen it works. Now it's about expanding the horizons and really pushing the limits. Now do you guys have a bit of a Gear 2 playbook? Like exactly what we just discussed. There are a lot of names. I was trying to capture some of them and I'll dig in. But any thoughts on that? Is that something that you guys are helping guiding the industry with?
- [40:31]
Kip Bodnar
Like going from AI foundational use cases to then really scale the AI use cases?
- [40:36]
Kieran Flanagan
Is that or like what model to use for what thing? Is that what you're asking?
- [40:39]
Maya Gupta
Yes, yes.
- [40:40]
Kieran Flanagan
What's it, what model to use for what thing. And that's what's tricky.
- [40:43]
Kip Bodnar
Yeah. I think Claude for anything.
- [40:45]
Kieran Flanagan
Ryden, if you're a marketing, just buy Claude seats for everybody.
- [40:48]
Maya Gupta
Yeah.
- [40:49]
Kip Bodnar
Right now any write in is like Claude. Even internal stuff for execs and stuff. I still run through Claude. Claude is a better creative model than OpenAI and Google.
- [40:57]
Maya Gupta
Yeah.
- [40:57]
Kieran Flanagan
And a very good coder still, by.
- [40:58]
Kip Bodnar
The way, from what everyone tells me because I can't delineate between all three. They all seem pretty great at coding. Claude is like the preferred model for coding and people don't know why. I don't know if you've seen that. People are like, I don't know why it's so good but it's just like it's way better. Right. Claude is also this weird thing where we had a Scott on who leads product for Entropic and he was asking like, why do you love Claude so much? I was like, well, it's like, why do I like one friend better than the other? I don't know. I just like gel better with Claude. Right. It just, it gets me, it understands me, it just knows me.
- [41:27]
Kieran Flanagan
Yeah.
- [41:28]
Kip Bodnar
The O3 OpenAI models are definitely better for strategy. And so if you take a growth that example, I showed a growth dashboard and took the data and then asked for a strategic doc. I still find OpenAI in particular the recent models and you can see on the benchmarks, it's just better at strategy. Google Gemini is not far behind and I need to actually get deeper into the Gemini 2.0 releases. The beauty of Gemini is just the integration with Google's platforms.
- [41:52]
Kieran Flanagan
It's huge.
- [41:53]
Maya Gupta
Yeah, yeah.
- [41:54]
Kip Bodnar
That is their key advantage. And I think the thing that could still mean that they win because just the integration into G Drive is you have all the context. Right.
- [42:02]
Kieran Flanagan
I got a rough draft of everybody's performance review just from all the decks in G Drive and Gemini is so much faster, so much better.
- [42:10]
Maya Gupta
Oh, man. Yeah. This is so amazing, man. I mean, obviously we've been using AI at different places, but I'll have so much more to share in four weeks because I'm taking a lot of energy away from here.
- [42:23]
Kieran Flanagan
Sweet. Let's do it.
- [42:24]
Maya Gupta
Yeah.
- [42:24]
Kip Bodnar
I think this is a great episode because we got some real insights into we're a company like Kraken and you are how you think about AI and then just riffing on these things of like, what are things you're thinking about in the future and it'll be interesting for you to come back on and tell us like, did you find it good? Did you not find it good? Has it been impactful or not? Because AI is such a broad thing, like you can use it anywhere and it's really missed more than it's hit. And so I think everyone's trying to figure out like, well, where is this actually worth my time? Because everyone is busy. Right. We don't have time to be spending iterate on AI and having a bunch of misses when we actually have real goals and results to drive.
- [42:57]
Maya Gupta
That's the biggest part that comes back to that operational question. But I think this is where leaders have to just take it on upon themselves and just get the feet in hand. And I think the key is what strategy you use to filter out from the clutter.
- [43:11]
Kip Bodnar
Right. 100%.
- [43:12]
Maya Gupta
It is very easy to get distracted.
- [43:14]
Kieran Flanagan
Exactly.
- [43:14]
Maya Gupta
So that's why your point at the end is important. Look, we've Shared a lot of names, a lot of platforms, but just focus yourself on two or three. That's it. And those are the big ones. And start there. Then you can start to become niche. Oh, here is this one thing. But otherwise it can get really noisy and then you get overboard and then you go back to your old habits.
- [43:34]
Kip Bodnar
Yeah, yeah, yeah. I actually think this is the most interesting test of how quickly users change behavior of all time.
- [43:41]
Maya Gupta
Yeah.
- [43:42]
Kip Bodnar
Because to your point, the natural inclination will always be just, you know, it's a bit hard. I've tried seven times. It's missed six times. I'm going to go back to you doing the thing I used to do.
- [43:52]
Kieran Flanagan
Well, the other thing, Kieran, you and I have talked about a lot is that, like, I truly believe that, like, the what model is best for what thing question is probably not the right question. I think the right thing to say is all of these things are transformatively powerful and we are underusing them.
- [44:06]
Maya Gupta
Yeah.
- [44:07]
Kieran Flanagan
And so if I just took one.
- [44:09]
Kip Bodnar
Yeah.
- [44:09]
Kieran Flanagan
And just obsessed about becoming deeper in my adoption fluency expertise in just one, I'm probably far better than everybody else.
- [44:20]
Kip Bodnar
Yeah.
- [44:20]
Kieran Flanagan
Right. Because it's like there's so much of all of these models that we're under using because we are flipping back and forth between all of them.
- [44:27]
Maya Gupta
Yes.
- [44:27]
Kip Bodnar
Yep. Cool.
- [44:29]
Kieran Flanagan
All right.
- [44:29]
Kip Bodnar
I think this was a great episode. We really appreciate you coming on. As always, you're an incredible guest, and I think your insights here are great for our audience. So I appreciate it.
- [44:37]
Kieran Flanagan
And I think we just. We're trying to go through the process everybody has been going through. Right. And like, everybody is kind of living in the same world over the last six to 12 months, and hopefully we shared some perspective that's helpful. And then Meyer's gonna come back and give us around two, which will be awesome.
- [44:52]
Maya Gupta
Yeah.
- [44:52]
Kip Bodnar
Cool.
- [44:53]
Kieran Flanagan
Awesome. Thanks, man.