← The Growth Podcast
Loading summary
Transcript125 lines
- [00:00]
Hannah Stahlberg
I have spent now like 1500 hours in Claude and I am still iterating on my setup and improving it literally every single day.
- [00:07]
Akash
So we've talked about Claude. There's Claude, there's ChatGPT, there's Cursor, there's Cowork. When should PMs be using which there's
- [00:14]
Hannah Stahlberg
not like a right or a wrong answer. Although for most advanced PM work, you should be using some type of a coding agent.
- [00:20]
Akash
Hannah Stahlberg is a PM at Doordash and former Google APM. She spent over 1500 hours in Claude code, wrote the viral Claude code for everything process, and uses Claude code all day at her work. Han, what's the biggest mistake PMs make when using Claude code for product work?
- [00:36]
Hannah Stahlberg
I think the biggest mistake is gift that people give up too early.
- [00:40]
Akash
What's underhyped versus overhyped in AI for pm?
- [00:44]
Hannah Stahlberg
I think that underhyped is following your curiosity.
- [00:46]
Akash
I for one had never heard of this. I think it's a genius idea. Can you just unwrap the covers and show us exactly what this looks like?
- [00:54]
Hannah Stahlberg
This is what I call Team OS or Team Operating system, which is your team's knowledge base that helps everybody on the team move faster.
- [01:03]
Akash
You said Claude code is the most misleading name in AI. Why?
- [01:08]
Hannah Stahlberg
Because it's not just for.
- [01:14]
Akash
Before we go any further, do me a favor and check that you are subscribed on YouTube and following on Apple and Spotify podcasts. And if you want to get access to amazing AI tools, check out my bundle where if you become an anal subscriber to my newsletter, you get a full year free of the paid plans of Maubin, Arise, Relay App, Dovetail, Linear Magic Patterns, Deep Sky, Reforge, Build, Descript, and Speechify. So be sure to check that out@buildle.akashg.com and now into today's episode. Hannah, welcome to the podcast.
- [01:47]
Hannah Stahlberg
Thank you so much for having me here. I'm super excited.
- [01:49]
Akash
So I've been thinking a lot about the future of product management, and one thing I'm noticing across companies in every geography is that PMs are supporting more and more people. One PM might have supported three or four engineers in the past. Now they're often being asked to support 10 engineers. And on top of that, where they might have just mainly concerned themselves with design and engineering in the past, now they have to interface with everybody. Sales, marketing, support, the list goes on and on. How are you dealing with this and what does the future look like for PMs in this world?
- [02:20]
Hannah Stahlberg
Yeah, So I think the future looks like exactly what you're saying, which is that a singular PM is going to be supporting a broader set of functional roles and at a much higher number than in like the previous team sizes. At the same time, I think we're seeing in the industry that roles are starting to merge together. So it used to be that generally all product decisions were made by the pm and then engineers were just writing code and design was just doing design. But now what we're seeing is, you know, engineers are building products and maybe even like deploying them without a pm. Designers are prototyping and building product and the whole everyone on the team is starting to make product decisions. Similarly, like as PMs, we're starting to do a lot more data analysis, also prototyping, making designs. And so we're starting to see a lot more blended functions where everyone kind of needs to have the best context that used to be kind of isolated within different roles on the team.
- [03:12]
Akash
So your answer to this is to create a well organized, high context repo. I for one had never heard of this until I saw you writing about it and talking about it. I think it's a genius idea. Can you just unwrap the covers and show us exactly what this looks like?
- [03:28]
Hannah Stahlberg
I can. Okay, cool. So this is what I call like Team OS or team operating system, which is your team's knowledge base and storing all of your team's shared context in one place. That helps everybody on the team move faster and do their job to the best of their abilities, especially being able to get context across many different types of of functional roles. So what we see here is there's three main parts of the Team os. You have the Claude folder. This is the folder where you might put shared agents, commands and skills that are shared by everyone on your team, which we're going to talk about a little bit more. Then I have the product development folder. And here you're going to see a lot of different subfolders across different functions. And then we have a team folder where you might have like team level documents like onboarding guides or retros. And at the top we have the Claude MD at the root level. This is the guiding route for Claude throughout your repository. And this has a few different key components. So the first is it has what's called a Doc index, and this tells Claude how to navigate the repository. So Claude needs to know where to go to look up different types of information so that you can do natural language queries in the repo and get the answers that you need. The other two things that I like to have at the root level are who is on my team along with their handles in key products and then like key Slack channels or DM groups. And the reason for this is you really, when you're kind of doing all of your work in this repository, you want Claude. And also to be clear, this works with any type of coding agent. It doesn't have to be Claude. You could do this with Codex, you can do this in cursor. You want Claude to know who's on your team so that you can just right click queries like Slack Alex about the bug that came up on the customer call today. And because the Claude MD file is loaded every single time, it's going to load Alex's Slack ID and it's going to be able to use the Slack MCP to send Alex to Slack. It's also really nice if you're talking about feedback or like meeting notes, because then when you're like, oh, I got feedback from Taylor, Claude knows that Taylor is your design partner and is going to be able to better contextualize that feedback without every single time you're writing. Yes, like Taylor, the designer on my team gave me this feedback. And similarly for the Slack channels, by knowing all the channels and what the purposes are, you can write natural language queries like, hey, send this in the product channel, send this in the ENG channel, and Claude will know exactly what to do.
- [05:51]
Akash
So how has your Cloud MD file changed over time? What have you Learned in the 1500 hours you've put in through trial and error and making mistakes.
- [05:59]
Hannah Stahlberg
So I think this is actually what a lot of people get wrong about CloudMD, which is you don't want very much in your CloudMD file. So Claude MDS should be very, very, very lean, especially in a team team repository like this. And generally, if we start looking through the repository, what we see is that there's actually multiple levels of Claude MD files. And so we'll see here, here's the root level. This one loads every session. The remaining files are going to start to load progressively as you type natural language queries, which is something that we're going to walk through. And this is really important because the way that this repo is structured is around the theory of context management. So I like to call this like context 101. But there's four like key concepts that you need to know about context. One is what is context? Context is the amount is the information that is in a given session with an LLM like what is the information that the LLM can access at a given point in time? The next is the context window, which is how much information can the LLM hold? And there's all the Frontier Labs recently upped this to a million tokens, which sounds a little bit jargony, but basically means that it can hold seven to eight novels worth of text, which, if you start to think about it, is a lot. But the amount of docs that are produced by a given team and company is a lot more than that. The next part is compaction. So when your context window gets full, all that information needs to get compressed down so that the LLM can keep going either in the conversation or in the work that it's going to do. But when that happens, you lose a lot of fidelity. And so you have kind of a compressed summary of the information, which is much less useful. And then the fourth concept is thinking room. And this is really important because thinking room is basically the difference between how much information you have in the conversation and how big the context window is. And so that is where the model can think and reason. And if you have a lot of information, the more and more information you have, just like a person, the less and less room there is to think and reason. And so the whole repository, and so the whole repository is structured around helping Claude read and use the the right information at the right time in order to effectively work on the task at hand. A lot of people don't know this, but I like. So I have this little bar in my status line and it's actually going to monitor how much context I'm using as we write queries throughout the repo. And this is going to help us see how much context is being consumed by the conversation.
- [08:27]
Akash
Awesome. I hadn't actually seen the nested CloudMD files before, so what do those look like inside there?
- [08:33]
Hannah Stahlberg
Yeah, so. So what we're going to see in the repo is that the Claude MDs are generally just doc indexes. So this is just telling Claude, like what is in this folder and what is the purpose of it. And the reason for that is if you didn't have these doc indexes, Claude would actually need to be running Explore agents, like search the repository for any queries that you write. So I actually want to do a couple examples to kind of show how this works. So in the product folder, one of the things we might have is who our customers are and what's really cool about how you structured this repo is I'm going to say who are my top Customers. And what we're going to see here is we're actually going to be able to see how Claude navigates it. So it's loading these Claude MD files and using the DOC indexes to navigate throughout the repository and find the exact information needed to answer my query. And now it's starting to read, okay, in the customers folder, who are my customers? What is the account context that I've stored on them? And what you can see here is that we've only used 3% of the context window and Claude's not looking in the wrong places. Right? Claude didn't go into the analytics folder, it didn't go into the Data Engineering folder, it didn't read a single unnecessary piece of information to answer my query, which means we still have ton of room to think.
- [09:51]
Akash
So the art of having good Claude MD files is actually minimizing the amount of context that Claude needs in order to answer a given question.
- [09:59]
Hannah Stahlberg
It's minimizing the amount of context that's consumed and making sure that you're only consuming context that's relevant to what you are actually trying to do. Right? Like, if I'm asking about my customers, I don't need to go read a bunch of SQL queries.
- [10:12]
Akash
Very cool.
- [10:14]
Hannah Stahlberg
And we can also start to do, like, more interesting things here. So, you know, I can say, what, who did I meet with over the last two weeks and what did I learn? Because I store all my customer information inside the repository in a really structured manner. So if we go under my Customers folder, I have a file for each of my customers. And in here I actually have a cloud MD on the customer. And in here, this is where I'm saying, okay, what's the key? Like, what you want In a cloud MD is things that you need on 80% of sessions. And so generally for like a customer, you might want to know, okay, who are the key contexts of that customer and what do they do, what's their segment, what are. And then again, I have a DOC index here. This is how to find key key resources on this customer account. Now, Claude is like primed to know where to go for everything about this customer. And so now Claude's doing an analysis and saying, okay, today's March 25th. What was the last two weeks? There's a lot of calls. It's going to read them, but actually look here at what it's reading. It's only reading the summary files. It's not going into like every single transcript, which I don't know about you, but like, my customer calls can be like more than an hour, right? Claude would not be able to go and quickly and easily synthesize, you know, 50 transcripts really quickly at High Fidelity. Which is why the repo set up that it only needs to go into the transcripts if the summary doesn't have what it needs to like. Answer my question.
- [11:36]
Akash
Here's the dirty secret about prototyping. You spend two weeks building a prototype, you validate your assumptions. Engineering loves the direction. Then what happens? You throw the whole thing away. Bolt changes this completely. When you prototype in Bolt, you're not building throwaway mockup, you're building real front end code that integrates with your existing design systems. So when you hand it to engineering, they don't throw it away, they ship on top of what you've built. I use Bolt every single day. I host my land PM job cohort on it and honestly I'm up till 2am some days just vibing in the toll, having fun and building. That's when you know a product is good. When you're using it past midnight. Not because you need to, but because you want to. Check out Bolt at Bolt New Akash that's b o l t.n e w a a K a S h link
- [12:21]
Sponsor/Advertisement Voice
in the show notes Today's episode is brought to you by Jira Product Discovery if you're like most product managers, you're probably in Jira tracking tickets and managing the backlog. But what about everything that happens before delivery? Jira Product Discovery helps you move your discovery, prioritization and even roadmapping work out of spreadsheets and into a purpose built tool designed for product teams, capture insights, prioritize what matters and create roadmaps you can easily tailor for any audience. And because it's built to work with Jira, everything stays connected it from idea to delivery. Used by product teams at Canva, Deliveroo and even the Economist. Check out why and try it for free today at atlassian.com product-discovery that's a T L A S S I-A N.com product-discovery Jira product discovery Build the Right
- [13:16]
Akash
Thing so some adding some more files in the form of summaries is actually going to help it collect the relevant information faster, correct?
- [13:24]
Hannah Stahlberg
So you basically are always wanting to think about how to structure information so that Claude can quickly and easily find what it needs to know in a format that's really easy to use, which kind of goes to another topic within the repository which is your shared agents, commands and skills. So when if you want struct while LLMs can work really well with unstructured information, it is obviously easier if information shares a common structure. And so teams that are using this system ideally should try to organize information in a structured way for Claude, so that for example, all customer call summaries follow the same format. That means that when Claude needs to do a synthesis on like hundreds of different customer calls, they're all following the same format, which makes it much easier for Claude to work with. And so that's why, for example, you might have a customer call skill and then you would have everyone on your team who's summarizing a customer call, summarize it in exactly the same way, put it in exactly the same place. Then when you need to do cross customer analysis, even though maybe you had 10, 15, 20 different people taking those different calls with different customers, everything is organized in a very consistent fashion that Claude can work with super easily and super quickly.
- [14:39]
Akash
So you're multiplying leverage by creating these skill files to take unstructured inputs. Maybe people have different ways of interviewing, but then structure the summaries in a similar way.
- [14:48]
Hannah Stahlberg
Exactly. So then all of your summaries come out with the exact same format, even if maybe your company has a ton of different account managers who would all synthesize things differently.
- [14:58]
Akash
Very cool.
- [14:59]
Hannah Stahlberg
And so now we see, okay, we got like a pretty detailed analysis of the last few weeks. So it's going to tell me, okay, who I met with, who did, what was the customer, when did I meet with them, who was in the meeting, what happened on each of them super fast. And then it also gave me like a quick cross cutting theme analysis. And all of that was just off of a natural language query. I just said, you know, what happened in the last two weeks? What did I learn?
- [15:22]
Akash
Hmm, super powerful.
- [15:23]
Hannah Stahlberg
So kind of continuing to walk through the repository. So as you can see, there's a lot of different parts that you might have in a product folder. You might have some competitive research. Maybe you have your launch emails, meeting summaries, your PRDs, processes that you need to run, context about how your product works, things for sales enablement, strategy docs. Here you might keep like business context about the company or the landscape that you're operating in, who your users are and what they need to be doing. Your roadmaps, vision docs, and then workflows is actually a concept from Carl, who has also been on your podcast. These are like multi step processes, like preparing for a bi weekly update that you might want to run on. Like that you might need to run Sorry, let me try that again. A workflow is like a multi step repeatable process that you need to do often on some cadence. And Carl has like a great system for getting this done that I've actually applied in a lot of my own workflows. Well, as, as well.
- [16:19]
Akash
Got it. And what is that system briefly?
- [16:20]
Hannah Stahlberg
Yeah, so basically when you need to do a complex multi step operation, the way to do the operation is stored in the Claude MD of the folder and it's kind of similar to having a command and then there's different files that says how to execute each part of the process to then synthesize something into a final document. And so I find this to work really well for meetings where you might need to pull a bunch of different information together and then put it in a document in a repeatable workflow that you want to run on some cadence.
- [16:54]
Akash
Got it. What else do we need to know about this repo? If we were trying to create our own.
- [16:58]
Hannah Stahlberg
Yeah, so we talked about the product portion and we can go deeper into like what each of the different folders are and like what you might use in them. But I think it's actually also really important to talk about the other functions as well because this is how everyone scales themselves and get the most out of the repository. I don't view the team OS as like something to help everyone become a better PM or to be better at doing product. I really view it as a way for everyone on the team to scale themselves and help everyone to leverage what's best about all of their teammates. So for example, in your analytics folder you might have links to all your dashboards, all of your experiment analyses and the results of them investigations that were done. And then I think the most important part is metrics, playbooks, queries and specifications. So this is how you scale data analysis across the team. So I like to generally organize by topic area and then product area. So in this dummy repo, I made up a company called Forge Labs, they're bringing another AI prototyping product to market. And so these are different parts of the Forge Labs product. And here we start to see. Okay, so here we've outlined all of the metrics for the billing part of the Forge Labs, but product, and then here, and what you'll see here is that here we've linked all of the dashboards that are relevant to this and then we also have a link into where the queries are for these metrics. And then if you go under queries, under billing, then you would have all of the SQL queries for how to query the metrics related to billing. And then here in the schemas we would have all of the table schemas that actually back these metrics. And so if I wanted to do data analysis, I would have all the references that I need as a PM to do correct analysis and have all of like, I basically get to have access to like the analyst on my team's brain and everything that she's so amazing at and set me up for success in doing analysis correctly. And so we can do some pretty cool stuff stuff here as well. For example, I might say I'm just going to use whisper flow. How do we calculate generation, success rate? Show me the metric definition, the SQL query and the table schema. Now I get to know everything about how to calculate this metric, what data tables to use, how it's defined, what tables it comes from, and it's going to be able to give me all of this. I think this is really important because especially if you're. Once you have anything that's more than like a very, very early stage product, your data tables can get really complex. The right way to query different metrics is not always obvious. And if you don't have the right guidance for Claude and you just point it at a database and say, hey, pull this stuff, it might not do it correctly. And so now here we see, okay, let's see if we can expand this. Yeah, so here, and this kind of goes to how the repository is structured. When I put that query in, it knew exactly what files to reference. So it went in the SQL, it went in the table schemas, it went in the metrics, and then it started reading everything about this part of the product. And then it actually was able to tell me, okay, how is this metric defined? And then what is the way to query for this? And then what is the schema that backs all of this? And if this was not like a dummy repo and like a demo instance, like you can actually have this hooked up to Snowflake MCP or another analytics MCP and you could actually start having Claude do the analysis for you. Similarly, we could say, okay, what do we know about why users drop off during custom domain setup? And because in the repo I have a playbook that would be an example of what an analyst might add into the repo for how to do the funnel analysis? And so CLAUDE is going to be able to know, okay, like how do we do this funnel analysis? Maybe this has already been investigated. It's going to find all the information in the repository and then use that to answer my question and answer it correctly. And so we become a lot less concerned with like hallucinations, inaccurate data, inaccurate analysis. Because everything that you're using as a PM to do the analysis is something that's actually been checked into the repository by an analyst or a data scientist on your team. And you know that you're using like verified approaches to get the metrics.
- [21:19]
Akash
Very cool. So you'd have like an analyst or data scientist really audit some of these playbooks and some of the table descriptions and join keys that we saw earlier.
- [21:28]
Hannah Stahlberg
Exactly. So the repo is something that like the whole team should be building together. So on my team at work, like the data scientist that I work with owns our analytics folder and every time that we're building a new feature, her and I are aligning on, okay, what are the metrics for this feature? What are the backing queries for the metrics? What are the tables? And then we're making sure that all of it's documented in the repository so that when we, when we roll out a feature, I can actually check how it's doing without being reliant on a data scientist. Or our engineers can also check on how it's doing because they're empowered with all of the queries that they need to also do, like validation of the feature in production. And the organization here is really important. And there's a reason to split out the metrics, queries and schemas. And again, it goes to this theory of context management. So you might just have a question about metrics, right? You might just want to say, what are the metrics for the billing feature? And you don't want Claude to then go pull all the queries and all the schemas to answer that question for you. You just want it to know what the metrics are. And I find this really helpful for when I'm writing a prd. For example, I can have Claude easily look up every metric I've ever defined for my product, figure out, okay, which metrics might we want to update? Which ones do we need to add in? Because I have all that context, like really well structured by feature in the repository.
- [22:45]
Akash
Love it. This is really speaking to kind of the future of how engineers don't want to have to be reliant on a BM to get a data report Here. You're the PM is actually becoming the glue, making sure that analytics has agreed on these are the right things, put the right things into the repository before a feature starts. But Then once a feature is launched, if an engineer is on call and the whole team is asleep, they can just query it all by themselves.
- [23:11]
Hannah Stahlberg
Exactly. And I think, and we actually make this part of our feature launch process. So when we're rolling out a new feature, the feature is not rolled out until the repository is updated. Because that's how we know that we're continuing to create that shared context so that everyone has what they need to do their job effectively as our product. As our product grows more and more complex.
- [23:34]
Akash
Awesome. Are there any other nooks and crannies people should know about of this repo?
- [23:38]
Hannah Stahlberg
So I think the repo really benefits every function. So I just put an example of like what you might have in an engineering folder here. So I put some bug investigations and then we use the term rfcs for like technical design documents. And again, this goes to helping everyone to do their best work, helping everyone to have shared context and historical context. So you might store all the bug investigations across your product in the repository. Because unfortunately we usually have bugs more than once and oftentimes in like the same part of the product. And so it's really helpful for the person investigating that bug to be able to see, okay, what are all the bugs that have happened here? What was the approach or so here? Like, I saved a bug investigation plan and we can see, okay, like when did this happen, what were we investigating, what was the scope, what parts of the infrastructure did it touch, how was this analyzed, what was the root cause, how was it fixed, all the data examples and all of the queries. And so then if someone has to go investigate another bug, here they have all the context of how every bug was ever investigated. And this helps them and helps them to work with a CLAUDE or another type of coding agent to really effectively investigate another bug.
- [24:47]
Akash
Very powerful. And who owns basically the engineering folder, like your tech lead.
- [24:53]
Hannah Stahlberg
So on my team, everyone is an owner of our knowledge repository. So each of the functional leads kind of takes ownership of like, okay, how do we want certain things done within this area? But the team as a whole needs to agree on the way to structure the information, because LLMs don't work as well with unstructured information. And then it becomes the onus on everyone to be updating the repository and making the team share context. And even better. And I think that this is how teams become really high performing in an AI native era is everyone is working to improve the repository to make the team faster. Right? Everyone should be writing shared skills that benefit the team, shared commands that benefit the team agents that help the team, we as PMs but also everyone on the team can also set up shared automations, which I kind of think of as the third pillar of this. This could be for example, using the information in the repository to run a weekly report that synthesizes all the customer research and what you learned. And if with the way that this repo is structured, you can actually have that just be an automation that runs every week, synthesize all the research, post a message in your Slack channel so that everyone stays up to date on customer learnings.
- [26:06]
Akash
So I think we got the high level, the 8020 of the setup of the repo. You check in all your day to day product work into the repo. What does that look like tactically?
- [26:17]
Hannah Stahlberg
So tactically I only work in cloud code these days, so I write every single doc first in CLAUDE and then I check it into the repo for and my team to review. And so, and that's how everyone on my team works as well. So like my whole design team works in claude, the engineers work in claude, we're all working in the shared repository. My data scientist is also working this way even like and I think, sorry, people think that this is only for like technical roles and I think that that is a very, very wrong assumption. So actually all of like my business operations product, operations strategy and operations partners are also participating and sharing in this shared context repository, putting up PRs, adding their context into the repo. We're all collaborating together in this space every day.
- [27:07]
Akash
What would that look like? So where would I, let's say I just completed the first draft of my strategy document for next quarter. What, where would I put that and how would I check that in?
- [27:17]
Hannah Stahlberg
Yeah, so you would likely, if you or me, have actually written that whole doc in claude and if not you could pull it in from a Google Doc using the mcp. And then under you'd probably have a strategy docs folder here or I might call it like Vision and you can start organizing it by quarter. So I would have different folders under here and I would say okay, you know Q2 20, 26 vision and I would like have all the docs for
- [27:41]
Akash
that under there and then when you check it in or something like that, how do I put this up into GitHub so that other people can see it?
- [27:47]
Hannah Stahlberg
Yeah, so you would just put up what's called a pull request or a pr and that is how you. So first you would commit your work when you're ready, when all the work is ready. To review, you would put up a pull request and then you would put that up for your team to review. Generally you'll have certain people that you would want to review certain types of work. So for example, if I'm writing a PRD that I know a certain engineer on my team is going to implement, I would put up the PRD within a pull request and I would put them as the reviewer on the pr. And something that's really nice about doing all this in Claude is that you can have the GitHub command line interface or MCP hooked up. And because Claude knows who everyone on my team is, I would literally write a query that's like, put up a PR for Morgan to review this PRD with the name of the PRD and everything would just work, never leaving Claude at all.
- [28:35]
Akash
Wow. Actually awesome.
- [28:38]
Hannah Stahlberg
And then because I also have like for my team configured that we have shared commands for creating PRs and our shared commands actually auto post a slack message to our team's channel with like certain structures, depending on who put up the pr, what is the contents of the pr, who should review it. So we actually like automated a lot of this.
- [28:59]
Akash
This is pretty mind blowing. So you're basically saying that you don't just have a code base, you essentially have a code base of your team's context for Claude and you are like push pulling and sending PRs for that context. And it's not just PMs doing it, it's analysts doing it. Designers, engineers go to market partners. Everybody is participating in that overall product team. So os.
- [29:25]
Hannah Stahlberg
Exactly. So like for example, my strategy partner takes even more customer calls than I do and she checks every customer call that she takes into the repository so that I can review what we learned and that helps us to work super, super well together. And she's completely non technical. Like she had never opened GitHub in her life two months ago and now she is putting out PRs every single day. I think that's something that I feel like very passionately about. I feel like I see a lot of chatter online like, oh, this way of working is only for PMs or it's only for engineers, or it's only for technical people. And I think that's very incorrect. I think this is something that anyone can learn how to do and that when everyone does learn how to do this, we can all work so much better together.
- [30:07]
Akash
And just so people who are scared of GitHub, like the command, at least how I do it, I'm not sure how you do it is I would literally just tell it like, am I logged into GitHub? Do you have access under my name? It'll say yes. And then I'll say commit this pull request for this issue and tag this reviewer. All in natural language. That's all you basically have to do, right?
- [30:27]
Hannah Stahlberg
Yeah, basically. I would also say I have an amazing GitHub 101 guide on my substack that has helped thousands of people learn how to use GitHub confidently. So I would also probably wrap you to that.
- [30:37]
Akash
Cool. What's the 62nd summary of that that
- [30:40]
Hannah Stahlberg
I didn't cover of how to use GitHub?
- [30:42]
Akash
Yep.
- [30:43]
Hannah Stahlberg
So the 60 second summary is all of your work should be on a branch. When and the process is basically you put all of your work onto a branch as you're working. Every time you finish a certain milestone, you're going to commit it. And this means you're saying, I reached a stopping point and I want to save all my work. When a given item is done, let's say, you know, you might write your PRD in chunks and when the whole PRD is done, you're going to open a pull request and you are going to then open a pull request and ask for review. This is where you would tag a reviewer. They might give you some feedback that you might address. And then when everything is good said and done, you will merge it into the main branch. Which means that now everyone in all of their local repositories has access to your work.
- [31:28]
Sponsor/Advertisement Voice
Today's episode is brought to you by
- [31:29]
Akash
the experimentation platform Kameleon. Nine out of 10 companies that see themselves as industry leaders and expect to grow this year say experimentation is critical to their business. But most companies still fail at it. Why? Because most experiments require too much developer involvement. Chameleon handles experimentation differently. It enables product and growth teams to create unique and test prototypes in minutes. With prompt based experimentation, you describe what you want. Chameleon builds a variation of your webpage, lets you target a cohort of users, choose KPIs and runs the experiment for you. Prompt based experimentation makes what used to take days of developer time turn into minutes. Try prompt based experimentation on your own Web apps. Visit kameleon.com prompt to join the waitlist. That's K A M E L E O O N
- [32:19]
Sponsor/Advertisement Voice
Today's episode is brought to you by amplitude replays of mobile. User engagement are critical to building better products and experiences. But many session replay tools don't capture the full picture. Some tools take screenshots every second, leading to choppy replays and high storage costs from enormous capture sizes. Others use wireframes, but key moments go missing, creating gaps in your understanding. Neither approach gives you a truly mobile experience. Amplitude does things differently. Their mobile replays capture the full experience. Every tap, every scroll and every gesture with no lag and no performance hit. It's the most accurate way to understand mobile behavior. See the full story with Amplitude.
- [32:59]
Akash
I hope you're enjoying today's episode. Are you interested in becoming an AI product manager? Making hundreds of thousands of dollars more joining OpenAI and Anthropic then you might want to do a course that I've taken myself. The AIPM Certificate ran by OpenAI product leader Mikdad Jaffer. If you use my COD and my link, you get a special discount on this course. It is a course that I highly recommend. We have done a lot of collaborations together on things like AI product strategy, so check out our newsletter articles if you want to see the quality of the type of thinking you'll get. One of my frequent collaborators, Pavel Hearn, is the Build Labs leader. So you're going to live build an AI product with Pavel's feedback if you take this AIPM certificate. So be sure to check that out. Be sure to use my code and my link in order to get a special discount. And now back into today's episode. Awesome. That covers the main elements of the repo. I want to talk a little bit about creating high quality documents. How does someone use this repo to create a 10x PRD or product strategy document?
- [34:00]
Hannah Stahlberg
Yes, so I think this type of I think having a shared context repository helps with writing really high quality docs. But that's not the only piece of the puzzle. The other piece of the puzzle is knowing how to plan effectively. Claude. Okay, cool. So let's talk about plan mode. I'm just going to clear this. For those who don't know, Clear is a way to wipe the context of your current session. And it's really important to do this when you're switching tasks because again, Claude is going to use the information in that conversation to guide its work. And so if you're starting a completely fresh task, you either want to open a different terminal window or you want to clear and wipe the context so that you're really focused on only on the task at hand.
- [34:43]
Akash
Got it.
- [34:44]
Hannah Stahlberg
I want to show kind of the difference between not using plan mode and using plan mode. So I'm going to split my terminals here and because we're kind of doing this like we have this imaginary AI prototyping company. I'm going to use, like, Google Stitch as an example, since they just had, like, a major release this month. So what I'm going to put in here is I'm going to put a basic prompt. I'm going to say, you know, research the most recent Google Stitch release and tell me about what happened. Okay? And then in here, I'm going to say the same thing. Research the most recent Google Stitch release, tell me about what happened, and give me a proposal for what you're going to do. And now we're going to kick both of these off at the same time. And so here, this is just a basic prompt. And when you're doing this, you don't really know, like, what Claude is going to do, right? Claude is making all the decisions here, and you're going to get back something. And I really like the junior employee metaphor for Claude. So I think of working with Claude, like, working with, like, a really, really eager and highly talented junior employee. And if you don't give that. When you have, like, a junior employee and they're not trained and they don't know what you like and how you like to work, then you don't totally know, like, what you're going to get back from that employee when you don't give them any guidance. And that's what basic prompting happens. So you're going to get back something, but you don't know if that something is going to be useful for you, if it's going to be in the format that you want. And so that's why I don't generally recommend this approach. So here. So there's a more lightweight version of prompting where before Claude does anything, you just say, like, what is your plan? Right. You're kind of asking your employee, okay, like, what are you planning to do? And you're going to assess. Are you, like, generally aligned on the direction? So here, Claude did a little bit of research, told me what happened, and it's giving me a proposal, right? And it's actually using the context from the repo to generate the proposal. Remember, this was a completely fresh session. I didn't load in any context. But here it's saying this is a significant shift in the design to code tooling landscape. Here's what I'd suggest for the Forge team, right? We should evaluate Stitch for rapid prototyping. We should see how it hooks up with our workflow. We should see if this is, like, competitive to what we're doing. And we should share the findings in these two slack channels, right? And it did this. It came to all these conclusions, it itself, using the context from the repository against this natural language query that I had asked. Now, if I was writing a doc, this is still probably not what I would have wanted. And so that's where we go into plan mode. So to get into plan mode, you press Shift tab twice and then you're going to see that plan mode is on. And so now I would say, okay, so now let's pretend, right, that I need to write some, maybe like a strategy document around this most recent Google Stitch release. So again, I'm going to start and I'm going to say, okay, research the most recent Google Stitch release. And what I'm going to want to do is write a strategy doc about how this impacts us as a company and what we should do. Now the difference between you might be wondering, well, you know, in that other terminal, Claude gave me a proposal of what it was going to do. We could like go back and forth and keep chatting in the terminal. And that is like a totally valid thing to think. The difference is that in. When you're using think of LLMs are trained to have a bias for action in order to be helpful. And so the goal of the LLM is to get into action to help you as the user. But it's kind of like a horse that's like chomping at the bit and it's like, please let me just go run and help you. And that's not very good when you want to be in planning mode. And so what plan mode does is it takes away that bias for action. It's kind of like taking away the keys and saying, hey, we're not going into action right now. We are going to only plan and you're going to get a lot better results. But people still don't like totally use plan mode effectively, which is what we're going to talk about. So now what it's saying, it's going to research Google Stitch and my product context in parallel so that it can plan the strategy doc effectively. And so here it's actually researching the release and it's researching my code base at the same time. And this is how you can start to see that, okay, when you have all this context in your repository, then you're already set up to load a lot of that context into your documents. Now we're going to wait for this to run.
- [39:01]
Akash
It's interesting. It also shows how much potential tokens are there, like 25k and 98k. But it's not like burning through 130k tokens. It knows where to look within those?
- [39:12]
Hannah Stahlberg
Yes. And I think that's something that's really important because I talk to people a lot who were like, you know, I put some queries into Claude and I burned like hundreds of thousands of tokens and I hit my usage limit within 30 minutes. And that's generally because people's work is not organized and optimized or Claude to easily traverse. And so you're very, very inefficient in your conversations. And look again, Claude is only loading the stock stuff that's relevant to my query. It didn't go into the analytics folder, it didn't go into the engineering folder. It went into my competitive research folder where it has access to my competitors. It looks like I have some information on Google Stitch now. It's checking my writing guide and my existing vision docs to understand the expected tone and format. And so we're going. You can see that Claude is already having a big head start in probably getting to what I would want. And it's also identified that my knowledge, like what I have in the repository is outdated because of this release. It's probably going to suggest that I make some updates. Cool. So now Claude is actually using something called Ask User Question tool, where it's asking me some questions about what I want to do here. So who is the primary audience for this doc? Let's just say it's for leadership. And what's the focus of the doc? I think that we should maybe do a competitive analysis. I think that would be pretty interesting. Or maybe we should do both. So let's just do a full block and then yes, we also want to update our existing files. So this is a very important practice, which is you need to keep the context repository updated. Otherwise you have what's called context wrong, which means Plaud is going to use context that's outdated. So we're going to update those. And now it's starting to form the plan.
- [40:51]
Akash
And so when you probably checking in this pr, it'll have two components, the strategy doc and the update to the competitive Intel.
- [40:59]
Hannah Stahlberg
Exactly. The PR is going to contain every single thing that I've changed in the repo as it relates to this task. So now it's going to read these and then we're going to kind of go into the next part of planning. And I also think you actually raised a good point there, which is it is in order for your work to be reviewable, you want to generally put like chunks of work together so you know something that contains 85 file changes, is going to be really hard for anyone to review. And so generally I like to segment my work for the person who's reviewing it. So let's say at the same time I might be working on a design brief for my designer, I might be working on a PRD for an engineer, and I might be updating a metrics file for the analyst on my team. I would actually open three separate PRs for those and I would tag each different person as the reviewer on that work. That makes it much easier for the person to review and approve and then me to merge all that information back into our central repository.
- [41:53]
Akash
Fascinating.
- [41:54]
Hannah Stahlberg
Cool. It's almost done now it's writing the
- [41:56]
Akash
plan and we're thinking with high effort here you can change that. But I generally try to keep it at like the max settings. Looks like you do too.
- [42:03]
Hannah Stahlberg
Yeah, I pretty much am always thinking with high effort especially, I mean, you can use medium for some stuff, but generally if I found for anything that involves like writing or reasoning, you're going to get the best results with high effort. And so it's worth waiting. For the purposes of this demo, I might put us down to a lower effort, depending on how long something sick. But I also know we can cut this down.
- [42:24]
Akash
And then I think like, if you're feeling frustrated about that. What I usually do in this downtime is I spin up my second agent.
- [42:30]
Hannah Stahlberg
So usually I wouldn't have this downtime because I would go start working on something else. Oh, the other thing, I'll cover this afterwards, actually. Okay, so here again is where I see people not go right with plan mode. So these files can be pretty long. It's pretty hard to read them in the terminal. But what you can actually do is open it up and read it. And the most important part of having a good plan for Claude to execute execute is actually reading the plan right. If you're going to send, which I have found that not a lot of people actually do. And if you're going to, you know, send someone off to burn a bunch of your tokens and have really high usage, you again probably want to know what your employee is going to do. And so I will always actually read through the plan and make sure that I am aligned on it. And now we're going to talk about some how we get into some more advanced planning techniques. So here. Okay, so here it's having a structure for my strategy doc and then it's going to update some competitive intel files. Well, you know, if I'm writing a strategy Doc, I might not just want to research this most recent release. And so what I might do is I might want to say, okay, actually what I want to do is I want to research anything that any of my competitors have shipped in the last three months. And I want you to do a sentiment analysis on the news coverage that these launches received and help me understand, like, which publications cover these releases. I then want to get a deeper comparison of basically how the landscape has changed in order to inform the strategy Doc. And I want this research to be parallelized. Once the research is done, let's have a check in to review it and then we'll write the doc afterwards. So what I'm doing there is a couple of different things. So one, Claude does not naturally parallelize plans. And I'm creating phases of work that I'm going to have Claude start to work through. And I'm kind of broadening the scope of what I'm going to do here so that I can get more work done at once. The other thing is for this plan, after the research comes through, I might want to actually review that research to shape what goes into the strategy Doc. So sometimes I'll create a checkpoint in the plan where I like to check in with Claude before we continue to the next phase. While we're talking through this, there's some other parts in the plan that I think are very important. So another part is verification. So verification is how Claude knows that the work was done and done well. I wish that it was not doing the search now. I should have prompted that differently. Verification is how Claude knows that the work is done and done well. So here, what we can see here is there's actually like no verification on how the research is being performed. So if I was like doing this in real life, I might actually talk to Claude about how I want that research to be verified. Right. If it's making claims, I might ask for the sources to be cited. I might ask for URLs to like the news releases or something like that. It's important that I know how to check Claude's work and that Claude knows how to self verify its own work before giving it back to me. Another example of like self verification is if you are building a front end feature, for example, you can have Claude use something like Playwright MCP to go in a loop and actually check the front end and validate its own work before telling you that the work is done. So the ability to tell Claude like what good work looks like and how to know that the work is done is a very key part of planning. I want to cancel this and it's not going to let me do that. So we can see here that Claude is actually using six agents at once to do this research.
- [46:08]
Akash
I'm so happy they created like the automatic parallel sub agents. It's so useful.
- [46:13]
Hannah Stahlberg
It is useful when this didn't happen every time I practice this. But we will get there. So other things that like I would do if this was a real plan that I was executing is I would actually go through the strategy doc structure. So, you know, Claude is proposing a structure for this doc, but that might not actually be the way that I want the doc structured. And so I would continue to give feedback to Claude about, okay, what are the sections that I want in the doc? What is the narrative of those sections in order to make sure that what I ultimately get back is what I what I actually desired. The other thing, while we are waiting for this to run, the other concept that I'm going to talk about is actually storing plan files within the repository. So what we're going to kind of see is like when you're doing like a longer and more complex plan, you're going to be putting time and effort into writing that plan and what teams are starting to doing. What teams are starting to do is actually store the plan file itself itself in the repository. So what you'll see in the repository is I actually have folders for plans and I have these in like most of the core folders, like in, like in my strategy folder. I also have a file for plans for other strategy docs that I've written. And the reason to store these in the repository is again, to help speed up everyone's work and have that historical context. If you're going to spend a couple hours with Quad figuring out how to do something you might want, you might need to do something similar again in like a month or in two months. And you don't want to start from zero again. You would like to have a previous plan to reference for how you did that to save yourself time in the future. Or maybe somebody on your team needs to do something similar and then they can go off of your plan. When you're doing agentic coding, this is also a way to track like the work that's in progress as coding agents are working through different parts of the plan. So OpenAI actually published this recently in an article they wrote on harness engineering where they talked about how they made the plan file first class artifacts of the shared repository.
- [48:20]
Akash
Very cool. So you and your plans not just the final strategy docs. When you're ship, when you're again checking in that pr, you'd include the plan document in there. And do plan documents get summaries like customer calls?
- [48:33]
Hannah Stahlberg
No, you don't want to summarize the plan because you want another session to be able to build off of that plan in its entirety of every single thing that you did. And when this finishes, I'm going to show you. I might actually just pull up. I have a different, I have a pre built plan for writing a strategy doc. Another concept that I want to talk about as it relates to planning is actually asking Claude to show you the agent prompts that it's going to use. So think about it. We're saying, okay, Claude is like a junior employee. You want to be able to check its work and understand are you aligned on the outcome that you guys are going to achieve together. And so when Claude, as part of a plan is kicking off other agents, now you have a junior employee kicking off even more junior employees. And again, you might want to know how this employee is going to direct the work of the other employees, because again, you might be misaligned or you might not have the same goal. And so in my plan files, I actually like on complex plans, I don't do this on every plan. I like to have Claude write out what it is going to prompt every single agent with. And so here I see the agent prompt in the plan file. What is that agent writing? It's going to use my writing guide. Okay, what's the context that this agent is going to get? What files is that agent going to read? And then how is it going to write that section? And this is really important because especially on writing tasks, if not all agents get the same context, or if you don't know what files that agent is going to read, you don't know that they're going to have the right context to effectively write that part of the document. And the reason to split long doc writing across multiple agents is again, because of the size of the context window. Writing is actually a pretty expensive operation. And when you're writing a very, very long form doc, with all the thinking and reasoning and all the docs that you need to synthesize in order to write the doc, you generally cannot have one singular agent read 40 context files and go write a great doc. And so I like to be pretty directive of, okay, what are the different sections of my doc? What is the context that you need in each section? And then who's going to write that section and then synthesize them all together with the orchestrating agent. Another big planning tip for this type of a plan is that you want all of the agents to write their output to temporary files, which you usually have to prompt POD into. Claude will not do this always automatically. And so it's something that I always check for as I'm planning with claude. And the reason for this is that two things. So again, going back to the theory of context, CLAUDE can only hold a certain amount of information. If you have maybe 10 agents running at the same time, and they all, at the same time return their work to the parent agent, everything is going to crash and you will lose all the work that you just did. And so it is very important that you have each agent actually store its work in temporary files and then have your parent orchestrating agent work off of those files to like compile the final synthesis, for example.
- [51:32]
Akash
Got it.
- [51:32]
Hannah Stahlberg
Okay, so we finished some research. It's asking me more questions. And then the last thing that I wanted to quickly show, yes, we're going to save all of these, is how I like to invite Claude as a thinking partner. So you saw throughout the planning process that Claude was using the Ask User Question tool to ask me questions that it felt like it needed to ask in order for us to build out this plan together. But what I like to do is actually invite Claude to push my thinking and help me to be like a better PM or consider things from different angles. And I'm going to show you how I do that if it will finish rewriting this. Another technique that I like to use, because I find that having a lot of terminal windows open is very confusing, is I actually name all of my terminals. So I would call this strategy Doc. And then like, I would actually, you know, I would maybe name this, like, prompt example. And I like my work to be really pretty. So I will also usually like, change the color of them. And I also might like set an icon as well. You can set custom icons too. I usually have like 20 or more terminals open at a given point in time. And so if they're not like named and color coded, then I usually can't find my work. And I found that a lot of people, like, don't know that you can do this.
- [52:40]
Akash
I don't color code it or add a custom icon. I think I needed to get to that level.
- [52:44]
Hannah Stahlberg
Yeah, I really like picking the different icons and sometimes I use it to have. Maybe this is where I'm opening all my PR. So I put GitHub icon or something like that.
- [52:55]
Akash
Yeah,
- [52:57]
Hannah Stahlberg
I like to color code to be able to keep track of things and then also name everything. Now I'm going to open up the plan file again. Here is where I'd really like to invite Claude to be a thinking partner for me. So now what I'm going to say is use Ask User Question Tool to push me on my thinking and help me consider other angles that we might want to pull into this document, different sections that we might want to add to the final doc or other questions that you need to clarify my goals for why we're doing this.
- [53:29]
Akash
This is by far the most comprehensive planning process I've ever seen.
- [53:36]
Hannah Stahlberg
And so now Claude is going to ask me questions. And you can have Claude interview you, like, pretty in depth. Like, I might say, I didn't do it for this demo, but I might say, take as long as you need, ask me as many questions as you need. And we're going to. Now it's going to start to ask me a lot more questions about what I'm trying to do here, which is going to help us to further refine the plan. And by the time all of this is said and done, we're going to have a really, really robust plan. Okay. So, yeah, now it's saying, you know, what's driving the timing of this document. It's going to change the framing depending on why we're doing this. I'm just saying this is a strategic inflection point. And then it's saying there's some hard questions missing. What are other angles that we should have this doc address? Maybe there's fights that we want to walk away from. There's a lot of pushes here, so let's just cover some of those. And then it's also calling out, like, other areas that we might want to add in. Okay, yeah, I'm just going to say yes to this. And so, yeah, now it's like pushing my thinking. Right. It's catching gaps in my reasoning. It's pushing me to consider, you know, did I intentionally leave something out? Did I not? And now Claude and I are getting very crisp on what is going to be done here. And now it's asking me more questions. So I'm just hitting enter here for, like, the point for the purposes of this demo. But usually I would read them, I would think about it. I would usually give more feedback on these questions and, like, usually different, dictate my thoughts until Claude really understands how I'm thinking about this problem.
- [54:57]
Akash
So most people, they're rushing in. They're just letting it write the first draft, the strategy document, then they're yelling at it and saying, you got this wrong, you forgot this frame. Your approach is let me spend two to three hours. It's not all getting the plan right and then iterate on it.
- [55:12]
Hannah Stahlberg
Exactly. Yeah. So now it's asking me more questions. And I think the other thing which we touched on very briefly is that I have different. So I have in this repo, like an example of your user level Claude folder. And here I have different like writing guides for different types of docs that I would write so that Claude can better write in my voice. These are just dummy guides. And so whenever I'm doing writing, I make sure always that in the plan the agents are given my writing guide because they might not auto invoke the skill. Skills only have like a 70% or so like auto invoke rate. And when you're going to let something run for a long time, I don't see why you would leave anything to chance. And so I will always, usually explicitly in the plan if I want a certain command run or a certain skill called, I will make sure that that is specified in the plan document.
- [56:01]
Akash
Yeah, I always say use X skill and triple check that that's there. The skill is like a lot of the alpha, if you guys haven't realized quite yet. So those dummy ones that she has right now aren't actually going to be super useful. It's like the value is when she has that skill, then she goes to a product review, sees some other PM at DoorDash, wrote an amazing strategy doc. She tells her skill, hey, I need you to go improve. Here's another good example. And then she runs a whole process using this skill and she says it fell apart at this point. Go iterate and improve. And then your skill itself improves over tons of iterations.
- [56:35]
Hannah Stahlberg
Exactly. And so yeah, now we're going to have a much more detailed plan. I don't want anybody scared. You don't generally need to spend two to three hours planning, but I think generally people are under planning and that's why you're not getting the output that you want because you left a lot of room for interpretation about what you were hoping to get. And so now you know, we have a much more detailed plan file and we're not going to read the whole thing here. But this is where I would keep going. I would read through all the changes, I would make sure I'm aligned with them. And then kind of a last important technique is. So this should oh, it already did the research. That's why if a plan has phases, which this one doesn't totally because we didn't fully set up having it do the full research, I would actually outline the different phases in the plan and have Claude track when each phase is complete. This way, if you need to have a long running plan over multiple days and you have to compact or stop in the middle, Claude knows exactly how
- [57:41]
Akash
to pick up writing those progress markdown files that you talked about earlier.
- [57:47]
Hannah Stahlberg
Exactly. I usually keep this all in the plan file because the plan file gets reloaded after compaction. And what we'll notice here. So the natural plan files, they have these really cute names from Anthropic. They all have like three words, but they are ephemeral. So they're stored in your cloud folder at the user level. You can actually open it up and see them. But they are wiped every 24 to 72 hours. So again, if you spend a lot of time on a plan and you want to save it, I will actually, as part of that plan, usually save the plan file down so that I can reference it in the future, even after the plan has been completed, because I'm likely going to be doing similar work.
- [58:24]
Akash
Does everything get a plan? Every strategy doc, every feature results, write up every prd, our student, Some things well enough defined via skills in other contexts that you don't need a plan.
- [58:34]
Hannah Stahlberg
I don't think that there is a right or a wrong technique. I have generally found that anything that is relatively complex and requires synthesis or deep thinking where you want a certain output and you don't benefit from having a plan.
- [58:50]
Akash
Yep, makes sense. And it sounds like things that you're maybe a little bit less certain about. Like we kind of approach. This is like we're not exactly sure even like what's relevant about Stitch and the other competitors last three month launches. So it has to go out and do that first phase. There's more undefined. So that's where planning comes a little bit more useful. Sometimes you have a PRD where basically the team has already agreed on the whole feature and you have a meeting transcript you can feed it. Then you might not need as much in depth.
- [59:17]
Hannah Stahlberg
Exactly. If it's like a pretty straightforward task you probably don't need. And also this is a very in depth the plan file. I don't do this for like all of my work. I try to tailor the level of planning to exactly what you were saying. What is the level of complexity and ambiguity and then the higher, the more ambiguous and the Higher complexity that something is, the more time and effort you should be investing in the plan. And that's also why I showed the technique here of just asking for a lightweight alignment proposal. I have still found that just doing this, you will get a lot better results because you can still be misaligned, even on very, like, straightforward tasks. And so I really like Claude to just quickly tell me if there's any level of, like, vagueness. I like Claude to tell me what it's going to do. Usually I'll give a very quick correction and then sign off on it. And I feel like you get much better and much more consistent results with this methodology.
- [60:08]
Akash
One of the practices you have that I think is really genius is how you are applying a beginner's mind mindset to learning and improving in cloud code. Can you walk us through that?
- [60:18]
Hannah Stahlberg
Yeah. So I think, honestly, getting started in these areas can be like, it can feel very overwhelming and you can feel like, wow, I'm so behind and like, I don't know anything. But I think it's really important to have that beginner's mindset where you just feel really comfortable asking questions. And so to do that, like, I will usually ask Claude about anything I don't understand and ask it to teach it to, to me. So, for example, I literally might say, you know, explain to me the benefits of why this repository is structured the way that it is, and also things that could be improved about the structure. And I would, like, do this and then Claude is going to, like, explain things to me. This is also how I improve what I'm doing. So Claude is like, now going to analyze the repository and it might tell me some things that we didn't actually cover on this podcast about what could be improved. But I think that's the point. Like, well, I, you know, I've spent a lot of time using Claude code. I'm still learning every single day. And I like to use Claude to help me learn and make sure that I understand why things are working, why things are not working.
- [61:17]
Akash
So I use this similar prompt every day. And I will say I have a slight tweak on hers, although hers is great. So what I do is I tell it first. I want you to go research everything that Anthropic has shipped in the last 90 days and create a calendar of all the. The features. Then I want you to go read the top cloud code influencers and the top posts that they've had in the last 90 days. Then I want you to compare my setup to the latest features. And what influencers are recommending and tell me how I can 10x my setup. And that prompt has just been huge for me because it's taking some of what we were doing with the planning on the strategy doc, where it's going out and doing the research. And we know that a lot of Claude's data training data is quite stale. It's like from 2024 at this point. And so it doesn't even know about its own latest features sometimes I find. So I think that that can also help it.
- [62:05]
Hannah Stahlberg
Yeah, and I think here's like another good example. So what something we didn't cover in the earlier walkthrough is that I have a feature index in the repo and it's actually a YAML file. And so, you know, if someone was opening this repository and they didn't understand like why this was a YAML file, I would actually ask Claude to explain to me like the benefits of why this is a YAML file and why this structure is used and also what a YAML file is. I would use this to learn along the way. So I think oftentimes I see people right now online are sharing their skills and commands and agents, which is amazing. But then people are downloading them and using them and not knowing why these things work. And so I always, whenever I'm using anything, I always start by having Claude teach me why is this thing good or not good. And then that also makes me more comfortable iterating on the thing because if you're just downloading and copying things and not understanding them, when it doesn't work the way that you expect, you won't know how to iterate on it or update it or improve it. And so now Claude gave me a really detailed explanation of what YAML is, why YAML is the right format for a feature index in your repository and help me to understand this topic and why this matters within the structure of the context of the repository.
- [63:26]
Akash
This is how you all should be approaching these things. You're going to find more files that Hannah and I upload, other people upload that you're downloading. You're going to start cloning repos on GitHub, make sure you do this process of learning about it. And that's why we included this segment in the episode, because the beginner's mindset is really important. Now we're going to cover a couple of the hot topics to make sure that you have a really well rounded understanding of this. So Hannah, what's the biggest mistake PMs make when using Claude code for product
- [63:56]
Hannah Stahlberg
work, I think the biggest mistake is that people give up too early. I think like learning anything new, it takes some time to learn how to use Claude effectively, to get really good at using Claude code. And like you saw, building out this type of a context repository is not something that you're going to be able to do over overnight. And so I think people try it for like a day, they don't get good results and they're like, oh, like this isn't for me. You know, I've, I have spent now like 1500 hours in Claude and I'm still iterating on my setup and improving it literally every single day.
- [64:30]
Akash
And with the pace that this team ships at, they're constantly adding in new features, new things, and so just staying on the top of it. There's a lot to be done in terms of that constant iteration. So we've talked about Claude, There's Claude, there's ChatGPT, there's Cursor, there's Cowork, when should PMs be using?
- [64:50]
Hannah Stahlberg
Which I think there's not like a right or a wrong answer. Although for most advanced PM work, you should be using some type of a coding agent. I think for chat it's generally just if you need a quick answer that doesn't need like super high context. Otherwise ideally you're really building a context repository that you can use that you can leverage with any coding agent.
- [65:13]
Akash
If a PM only has two hours this weekend, what steps should they take to set up cloud code?
- [65:18]
Hannah Stahlberg
So what I like to recommend for people is that the single biggest piece of leverage you have is freeing up your time to learn. Like, I think especially right now, the most important thing that we can be doing is learning. And so if I had two hours this weekend, my question would be, what can I do to create six hours for myself next week? And I would find something to automate so that I can free up 6 hours to go learn things. And so generally that's what I recommend for people, is you should be trying to carve out at least an hour a day to just play with AI. And in order to have that hour a day, you need to automate work in order to free up your time so that you can learn and also help to up level your teams.
- [65:58]
Akash
Amazing. What's underhyped versus overhyped in AI for PMs.
- [66:03]
Hannah Stahlberg
I think that under hyped is following your curiosity. So I feel like there is a lot of pressure to always be on top of the exact latest news, like the exact latest release and, you know, get really good at whatever is being posted online on a given day. I think that we're all going to have a lot more fun and learn better if you're following the things that you're curious about. And so, you know, if evals don't get you out of bed in the morning, like, don't start there. Like, start with automating, something that frees up your time. Or start with, if you love design, like, start playing with prototyping and then. Yeah, I think that the other thing that I think is underhyped is building expertise in one area. So I think right now a lot of people are very shallow at, like, many different things, but it actually does take time and investment to, like, really learn a topic. And so I would say the other underhyped thing is spending the time to go deep, even if that means maybe you're not learning some of these other areas right now.
- [67:00]
Akash
You said Claude code is the most misleading name in AI. Why?
- [67:06]
Hannah Stahlberg
I think because it's not just for coding, which I hope is what everyone took away from this, from this podcast episode. While I do code in Claude code, most of my time is not spent coding. It's spent writing docs or doing analysis or building local HTML prototypes for my team or other types of prototypes, which actually we didn't get to show on this episode. I had a really fun one, but I think it is not just for coding and it's not just for people who are technical anyone. Like, again, my operations partners are also spending all day in quad. They're contributing to our repository. It's really the best tool, I think, right now for doing mobile.
- [67:47]
Akash
What should they have called it?
- [67:48]
Hannah Stahlberg
Well, that's why I called my series Claude Code for Everything, because I don't know what. I don't. I don't know what they should have called it. I do like the name Cowork. I think that was like a really great branding on their part. But yeah, I don't know if I have a snappy name for it.
- [68:05]
Akash
What would you tell a PM who's scared of the terminal, scared of IDEs?
- [68:10]
Hannah Stahlberg
I would tell them, like, don't be afraid to be. Be a beginner again. And there's. I mean, I hope what folks saw on here is. In my mind, there's not a big difference between typing into a chatbot and like typing into the terminal. Once you've done it for like an hour or two, you'll probably start to feel pretty comfortable.
- [68:27]
Akash
What mcps do you need to hook your team OS up into in order to make it effective.
- [68:33]
Hannah Stahlberg
Every single MCP that you can access. The limit does not exist exist. I am adding like a new MCP every couple days at this point. But generally right most companies are going to operate on like a certain tech stack, right? You're going to have a certain set of software vendors that hopefully have either MCPs or what's often under discussed are command line interfaces or CLI tools. Claude works really well with both of them, but the goal is any core piece of software that you use in your day to day work should be hooked up.
- [69:05]
Akash
This has been a masterclass I have done. I think it's seven or eight Claude code episodes and I was learning every single minute of this one. We covered how to create a team os, how to set up that repo, how to write really amazing documents by creating a comprehensive planning process, and how to have a beginner's mindset with Claude code. Hannah, thank you so much. If people want to find you online, where should they go?
- [69:30]
Hannah Stahlberg
They should go read my substack, which is called in the Weeds. You can find it at hannastolberg.substack.com Awesome.
- [69:37]
Akash
I hope you enjoyed that episode. If you could take a moment to double check that you have followed on Apple and Spotify podcasts, subscribed on YouTube, left a rating or review on Apple or Spotify and commented on YouTube. All these things will help the algorithm distribute the show to more and more people. As we distribute the show to more people, we can grow the show, improve the quality of the content and the production to get you better insights to stay ahead in career. Finally, do check out my bundle@bundle.akashg.com to get access to nine AI products for an entire year for free. This includes Dovetail, Mobin, Linear, Reforge, Build, Descript, and many other amazing tools that will help you as an AI product manager or builder succeed. I'll see you in the next episode.