← Behind the Craft
Loading summary
Transcript87 lines
- [00:00]
Rohan
Aim for something that seems 10x more ridiculous than you ever think it could. And it probably can do 90% of it. You cannot walk from point A to point B in the OpenAI office without hearing codecs get mentioned over the course of the day. I'm probably stewing 3 or 4x more than I would have without codecs. Codex actually knows how to set up its own automation. It just comes back to you. You didn't have to think about what it was doing while it was doing it. It's literally not a question of what is possible. It's just everything is possible. It's just what should.
- [00:27]
Podcast Host
All right, so why don't we start by talking at a high level? You know, I've been a PM for over a decade, too, and like, a lot of the work, dude, is just like, you know, editing Google Docs, making plans, doing alignment meetings, like, all this kind of stuff. Right. But, like, how has Codex fundamentally transformed how you work as a pm?
- [00:43]
Rohan
I would say that there's two levers by which it's changing how we work as product managers. One is the actual how we do the specifics of the job. But then secondarily is that now with the entire team, you know, engineers, product managers, designers, all using codecs, it's just the role that everyone's playing is slightly different. And so it's not just doing the old stuff differently, but even just the way we operate is very different. I think that. I mean, fundamentally, I would say a lot of. If you look at what product managers were historically doing, what we've been up to, it's a lot of information synthesis and kind of building ideas around lots of different interactions with different customers. And I think that, at least at OpenAI, we are pretty light on the product side. You know, on Codex, we only have a few PMs working across a lot of different areas. And so fundamentally, this is only possible because of Codex at a baseline level. You know, we get thousands of. Every day, there's probably hundreds of different problems or data points or bugs that we're getting from enterprise customers or, you know, obviously customers on Twitter and things like this. And so I'd say at the front end of this, it's. Codex is extremely valuable for just synthesizing tons of information and kind of enabling someone, myself, to operate with way more context than I would have been able to previously. I'll be added last minute onto an entirely new project that I, in 20 minutes, can immediately just have all of the context of stuff that happened before I joined, you know, because Codex can go into all of our tools connected into things like Slack and Notion and Linear and email and Google Drive. And so I think that everybody, especially the product managers, just are able to be way more leveraged and quickly jump into different things. And then I would say the actual, you know, beyond just being able to make decisions with that information, it's all of the ways that we generate artifacts have fundamentally changed. Right. Whether it's slides, you know, prototypes, docs, PRDs, all of these things I think are now either a lot of these things that previously required a lot of effort now can actually be done via automation.
- [02:39]
Podcast Host
Yeah.
- [02:39]
Rohan
So then it allows us to actually just spend a lot more time focusing on decision making and kind of thinking through things and kind of, we're talking to customers more importantly. So yeah, I mean, happy to, we can dig up. There's a lot of different ways that it's changed, but I think fundamentally I think we're just spending a lot more time doing the more important stuff, which is talking to customers and working with our teams to build product and a lot less time doing a lot of the more manual stuff.
- [03:01]
Podcast Host
Yeah, dude, there's like a meme going around that like the plans are for agents, they're not even for humans anymore. So. And you know, dude, like there's like so much time wasted on just like writing these like in the old days writing 10 page PRDs or like long term strategy or whatever. Like I'm really curious because I know the Codex team works really hard and moves very fast. So like how much planning do you guys do and like how, how is Codex involved in the process?
- [03:25]
Rohan
I mean we're definitely, I'd say in General being at OpenAI, we are all, we operate very much a research lab, even how we develop product. And so we're constantly, you know, trying things and so we're not planning things out six months in advance. I think I've, I've, I don't know what the. Right. I've always, I've been thinking we need a new, a new word for roadmap because roadmap kind of implies this 12 month out thing. I think it's more what's the four to eight week plan? We know what those priorities are and let's execute against those and then four weeks from now we'll see what the new priorities are. I think that the way that Codex really influences a lot of this stuff is basically one is we can just build things pretty pretty much instantly. And so oftentimes I'd actually say that we spend A lot of time looking at solutions, deciding if we want to actually ship it. I think that a lot of the work, you know, there'll be an idea. For example, actually the in app browser is a really good example of this that we added a couple months ago. Which one of the engineers on our team, Ed Fraser, he's awesome. He basically one morning he I show up to work and he's look, I built the browser in this because it was really annoying doing front end iteration. What do you think? And I was like, that's awesome. We should definitely figure out how to ship this. And so then that was almost a starting point was having an actual in product branch that we could actually develop off of. And I think that's a good kind of mental model is there's just a lot. It's like the product development life cycle is kind of inverted in a lot of ways because before it was very much spend all this upfront time planning and really making sure that engineers are only doing the most important things. And now it's actually flipped and it's hey, do everything. And then let's figure out what actually should go through and in what ways it should. Got it.
- [04:56]
Podcast Host
That's actually really amazing for our profession, I guess because you're saying basically the engineer basically built the MVP already and he can gain that iteration loop internally of giving feedback and making it better and just working on the actual product instead of some document.
- [05:09]
Rohan
Right? Totally. And in general, I would say we try to have work. Everyone within OpenAI is extremely AGI pilled and very much at the frontier of thinking about curiously, how can we leverage AI better, how can we improve our products in this way. But also our engineers are someone that is very product minded and so which I think is actually really important with AI with tools like Codex. Because human to human collaboration becomes a big bottleneck. If you think about each of our projects, we really are just have one or two engineers working on it and it's entire, you know, product lines. And so in a mode where there's so many engineers, there's actually a ton of human, you know, collaboration overhead. But then also you're moving slower. And so then the product decisions might happen a couple of times over a week versus when an engineer can move extremely quickly, there's a ton of micro decisions that ideally they could just make themselves and not need to work with me. And so often when we start a project we really, what I try to do is just think through what are the most important questions that need kind of decisions that we need to make about this thing, to feel cohesive to our overall strategy is successful. All of this stuff, almost the guardrails of what we're doing. But then beyond that, I think our engineers and our designers move pretty autonomously, and product is more kind of what the other the two ends of the process taking it to market, but also kind of the more strategic planning up front.
- [06:25]
Podcast Host
Got it. You're just trying to keep track of, like, a lot of stuff the engineers are working on, right? Kind of like.
- [06:30]
Rohan
Well, I mean, honestly, there's no way to keep track of all of it. Codex is my only way. Every day. There's probably something, a term I hear, and this is one of the ways I use Codex, which is whenever I'm in meetings or literally anytime I don't understand something or I hear something I didn't know about, I just have Codex go and gather all the context. And I would say a couple times a day, there's something that someone's doing where I'm. It's incredible that they're doing it. I had no idea it was happening. And that's amazing.
- [06:51]
Podcast Host
That's amazing, dude. So, yeah, it really has flip, right? It's not like the PM is trying to come up with all the ideas. It's like the opposite.
- [06:58]
Rohan
No, I mean. And I mean, we're super lucky. I mean, this was also true at cursor. Obviously, we're building a tool for ourselves. You know, fundamentally, it's funny, because you cannot walk from point A to point B in the OpenAI office without hearing Codex get mentioned and someone talk to someone else and be like, oh, I asked Codex to do this. Or I really think that if Codex went down or we didn't have access to it at this point, our company would be operating. It would be a pretty tough time. And I think that that was originally why we built it in the first place, is we want to use it to. You know, years ago, we started Codex as this project to accelerate our own research. Now, I mean, it turns out that that can just help us accelerate everything. And so then we can, you know, we have the intuition over how it should work and what should be better about it.
- [07:40]
Podcast Host
This episode is brought to you by Ocean's. I hired someone through oceanz for podcast post production a few months back and can't imagine running the podcast without his help. He's proactive, picks up new tools fast, and uses AI to compound everything that he ships. Oceanz doesn't just place assistants, they place operators. The talent is AI fluent and delivers the same Output as a senior year as hire at 3 to 5x less cost. They reject 99% of applicants. So the person who lands on your team is already operating from day one. If you're scaling and need marketing, ops, finance or EA help, I highly recommend giving Ocean's a try. Check it out@ocean.com Peter now back to our episode. I imagine it must be incredible working there because it's kind of an interesting spot because like yeah, I switched completely to Codex and like pretty much anything you want to automate or streamline you can just ask Codex to figure it out and like build a scale or something around it.
- [08:33]
Rohan
Right.
- [08:33]
Podcast Host
At the same time I feel like OpenAI has like what like a built in chatgpt users. So how do you get the normies to get Codex pilled without overwhelming them? That's kind of like the key problem, right?
- [08:46]
Rohan
Yeah, I mean it's really interesting because I think, I mean I've worked in AI coding for a while and now it's turns out these agents are super useful for everybody. But I think developers even haven't fully warmed up to the level of ambition you can have with codecs. You know, if you go within to any large enterprise, maybe the top 5 to 10% most kind of AI pilled frontier engineers are maximally leveraging codecs. But then there's a long tail of people that are still, you know, maybe pair programming, doing less delegation, things like that. But it's interesting because engineers have kind of seen AI incrementally expand its capabilities over the last three years. But then if you look to a non developer, it's kind of been just chatgpt this whole time. And so then the type of things that they're asking chat are very different than what you can actually delegate to Codex.
- [09:33]
Podcast Host
Yeah.
- [09:33]
Rohan
And so honestly I think that is the big opportunity for us is to figure out, you know, what are the ways to just make people feel more ambitious with what they can ask. Because ultimately I was, it's. There's actually nothing. Almost everything you should, you need to do professionally should almost always just immediately have codec start doing it. So I think that that is an interesting kind of both product and kind of enablement overall. Just mentality challenge.
- [09:59]
Podcast Host
Yeah, I've gotten more ambitious with my requests for it over time. Like I have a skill that it posts like 7 Social media platforms at once and a bunch of them don't even have APIs. So it just uses like the browser used totally.
- [10:10]
Rohan
I mean the browser and that's what I think is so Cool is the approach we've taken very much has been build these really useful primitives for codecs to interact with your computer. And then the magic is when they all kind of combined into this thing where you don't even know, you don't even need to know how anything works. All you do is just ask Codex for what you need. And under the hood, Codex actually knows how to use itself to then actually do the things you need. Right. So the most magical experiences I've had with browser use are not when I ask it to use browser or ask it to use my computer, it's just when it was trying to do something. Once I was, I, I built, I wrote a notion doc and I basically wanted to create a site off of the notion doc to kind of, you know, prototype something. And so I asked Codex, can you turn this notion doc into a site which cites is kind of the hosted websites we launched recently. And it was funny because then it did it and then I went back and looked at the trace and it basically notion MCP didn't expose getting the actual images in the doc. And so Codex opened up a browser and then just went into the DOM and got the actual file images and then used that in the website. And I was like, that's amazing. I'm glad it did that. So it's kind of, that type of stuff feels really magical to me.
- [11:20]
Podcast Host
Yeah, dude, it's like it's a lot more persons than humans in some ways. It just keeps trying to figure it out.
- [11:25]
Rohan
Totally. And I think that's, I mean, it's funny like, I mean, I think that's why our goal feature is, you know, feels really useful because there is generally our goal with OpenAI and with Codex and generally with our agents is we want it to just keep getting better at doing harder and harder, more ambitious, longer time horizon tasks. And often some of that's like the harness level, but some of that's also just at the scope of, you know, continuing to prompt it. Before we had goal, I think often what people would do is just add a bunch of steers where it's okay, keep going and then keep going and then keep going.
- [11:53]
Podcast Host
Yeah.
- [11:54]
Rohan
And I think now it's, you know, it'll just, it'll just keep going now until it actually meets the criteria, which I think is awesome.
- [11:59]
Podcast Host
Got it. I think you prepare like a few demos for us. Do you want to kind of share some PM workflows that you use?
- [12:05]
Rohan
Yeah, I'm going to share kind of in a dummy environment because we're not able to share the things I'm doing in OpenAI directly, but at all, hopefully be able to illustrate some of this stuff. So there's a few different ways that I. I mean, there's a lot of different things that I do with Codex. I think one thing actually, and so just orient everyone. This is the Codex app. This is kind of, you know, an, an agent development environment that we've built. It's kind of an agent control plane. And so there's a lot of different things. The most fundamental thing is, you know, you can start a chat, the model, all this stuff, it can work within a, a project or just on its own. But so there's a few different ways that I use codecs that are really valuable to me. So one is that at OpenAI, you know, we have a ton of channels, we use Slack very heavily. And so we have a ton of Slack channels that we get feedback from customers, our internal team send feedback that we discuss ideas. And so one of the things I use Codex for is actually to just synthesize feedback, kind of highlight important things, and then actually upload it into a linear board that I can review later. And so this is kind of illustrative of what I did, which is essentially there's a thread that I wrote in Codex where I asked it for feedback. I actually, you know, also spent a lot of time asking it to figure out how to correctly input it into linear and what are the ways that make sense. And then often, and this is very true for a lot of other workflows, I'll then just ask Codex to set up an automation to do this continuously. And so what's great is Codex actually knows how to set up its own automations. And so Codex was actually able to go in and say, great, now do this every day or do this every week. And then, you know, things that I would often also do is say, you know, also send a Slack message to me every time it finishes. And then Codex can go in and update the automation. And so there's probably five or six different automations this, that I've set up where it's a manual process that I then ask Codex to do once and then I just ask it to, you know, go and set up the automation. And so this is probably a big, I would say this is a huge unlock for a lot of things. And there's also really interesting creative ways that you can use automation. Something that I'll do often is, you know, basically tell Codex, you know, there's some, there was some sort of message that got sent or where I'll have sent a message and then based on the result of that message, I'll ask a question to one of my coworkers. So I'll be like, hey, when Alex responds to this message please. Or to the last DM I sent him, draft a response email to the client. And you know, so basically what Codex will do is actually use automations under the hood to kind of keep checking the Slack message that I had with Alex and then essentially once it triggers, it'll draft an email and then actually delete the automation. And so I think that the things that's really cool is using Codex knows how to use itself and so you can just ask it to do pretty much random things that aren't very specified and it'll just use its own primitives to do that really well.
- [14:57]
Podcast Host
Oh, that's interesting. Okay. Yeah, I've set up like daily jobs and weekly jobs before, but not this kind of trigger stuff. Just like Alex responding and trigger something. So it'll actually do it and it'll actually remove it afterwards.
- [15:08]
Rohan
Exactly. Because it can update the automation, can have. It'll set up the automation to actually tell itself to delete the automation when it meets the acceptance criteria. And so you can do really basically anything that's time based or requires kind of a trigger. You can actually Codex will set up automations to do those things and you don't even need to kind of be very specific, which is cool.
- [15:30]
Podcast Host
You can do like an automation where like if someone takes roll hand in slack on long thread just like pretend to be rollhand and respond to this.
- [15:36]
Rohan
Exactly. Yeah. So actually it's funny, Alex also, he was just sharing this with a couple of us where he'll like do Odex, but it's not a Slack app. It'll just be an automation running every few minutes to see if he tagged Codex and then it'll actually just respond. There's a lot of. Yeah, I think automations are honestly kind of not under appreciated but kind of under leveraged I think feature that we have because it's so flexible because of the fact that codecs can intelligently kind of update it. But there's one thing, or sorry, real
- [16:07]
Podcast Host
quick, my one problem with automations is like, like a lot of them are local. Right. So they only run when the computer's on. So sometimes if I do like a Friday morning thing and my computer's not on, it doesn't work, you know.
- [16:17]
Rohan
Totally.
- [16:18]
Podcast Host
So I guess I have to put it on my Mac Mini or something.
- [16:21]
Rohan
Yeah, that is one option. I mean we did launch support for mobile and kind of you can also control remote instances of codecs via in a cloud. Yeah, but we are going to. So right now we don't have native cloud on Automations but that is something that you know, is in the pipeline. So. But I think it's, it's kind of interesting because I know there's a meme of the laptop kind of, you know, slightly ajar but I mean to be honest I originally when I joined I was kind of oh, you know, the automations being local feels limiting but I think that in the, if you kind of are just using them, I mean for the most part the whole day my laptop's open so it's, it actually hasn't been a huge problem but I do think it's, you know, we are going to have some solutions there for sure. So the other, the other thing that I think is really interesting that I think has really fundamentally changed the way that we work or at least the way that I work is ImageGen. So you can actually use ImageGen directly in Codex. If you just tag image gen the skill it'll actually trigger using the model. And so it's really interesting because I think this is. If you think about the way that we were thinking about using AI to kind of do quick design iterations it often was writing code to prototype things. But I actually think ImageGen is a much faster way to prototype ideas. And so this is just to illustrate the example. So I took a screenshot of the composer where you can type in a new message into Codex and I basically said hey, when you launch Codex you can select the project. And so I asked Imagegen or asked Codex using ImageGen to basically prototype four or five ideas to improve the way to select the project. And so what it did is actually, you know, generated a bunch of different ideas, you know, nice immediately. And I think what's, I mean to me actually this was kind of an AGI moment for me where I was. Because I think we always think of ImageGen as this thing which is useful for making, I don't know, turn my hair blue or something. But the fact that it can create such good digital mocks essentially really allows for super quick iteration. Right. This feels a much quicker iteration to me, iteration than kind of generating five different react dummy websites. And then what I'll often do is say, you know, cool, that was awesome. Now take the first one and make a prototype of it on a, on a site. And so then something that it creates, something that can be shared and then, you know, we can kind of reason over it. So I'd say ImageGen was definitely been a huge unlock for prototyping and just generally expressing ideas in product.
- [18:57]
Podcast Host
Like do you just give it a reference screenshot or do you have to give it some sort of like design system thing or. No, just a reference screen shot.
- [19:02]
Rohan
Yeah, for this one, I mean, so just gave the screenshot and it was. It's able to, you know, do a pretty good job of staying within the design language. We've also built skills internally at OpenAI that basically kind of enforce our design language or it enables Codex to kind of access it when it is generating design assets. The other thing that's interesting is you can integrate with plugins, the Figma plugin, and so then that'll actually be able to pull things, your design tokens and things like that.
- [19:31]
Podcast Host
But like, for the audience, just like a really basic skill is just like, here's the colors, here's the fonts, and here's like, you know, that's it, right? Like that kind of stuff.
- [19:39]
Rohan
And I mean, what's great with things, you know, an often thing might happen is you do this iteration, you look at this and you say, hey, this wasn't right quite what I wanted. And so you keep seeing it until it is. And then what you can do is actually use. We have something called a skill creator, which is actually a skill to create skills. And what you can say is, hey, Codex, look at this thread and create a skill to ensure you are consistent moving forward. And this is, I'd say, a pretty common thing that we do, which is you have some sort of interaction with Codex and you just help Codex to kind of at the meta level, do something, make a skill, make a plugin to ensure that this thing gets essentially templatized. Moving forward.
- [20:19]
Podcast Host
Yeah, that's what I do too. I have skills for doing stuff and then it doesn't quite get it right in one shot sometimes. So I have to give it feedback and I'm like, hey, can you update a skill to account for this feedback? But dude, one thing I worry about is because I don't actually read the updates that it makes. So I'm worried that I'll just slopify the skill, you know, if I keep making it do stuff.
- [20:41]
Rohan
Do you think about that too?
- [20:42]
Podcast Host
Or do you have a. Yeah, no,
- [20:44]
Rohan
I. I do think about that because I feel a lot of times, I mean, and this is kind of a general thing that we're kind of thinking about, which is a Lot of things. Skills and plugins do introduce a lot of things to the prompt. And so we're starting to think about, you know, especially as you have more autonomous agents running, are there ways to kind of eval the output when you make changes to it or, you know, build confidence? Right now what I'll do is I'll just test it and basically, cool. Looks good and kind of keep going.
- [21:10]
Podcast Host
Okay, cool.
- [21:13]
Rohan
The last thing that I'll share on the just things that I do that I think are super leveraged is essentially oftentimes and this kind of goes to this idea of kind of disposable software. It's so easy to create software and tools. And so using Codex. And so often I'll just create one off things that aren't necessarily reproducible. But for example, we use a lot. And so often there'll be just a ton of messages that I need to respond to. And so I'll actually just on the fly, ask Codex to generate little apps for me to work through things that are happening in other tools. So for example, a common thing I'll be is, hey, look at all my Slack messages, figure out the most important ones to respond to and then actually just show it to me visually within a local website. And so then it'll kind of generate, you know, different versions of things like this. Um, something I actually have set up also is when it generates something that feels useful. Oh, I can actually use this repeatably. Then I'll actually just like set up an automation to tell it to update this every, let's say, hour. And so then I can actually just work out of this a little bit. Um, so I, I think that there's a lot of. And I would say this is where there's a lot of overhang of capabilities where I think we're used to thinking about software, these little apps taking a lot of effort, but then now it's so easy. So then you can actually just create extremely custom versions of everything that you need for the workflows that make sense for you.
- [22:34]
Podcast Host
Got it. So this one's about kind of like a prioritized list of things you need to respond to or actions you have to take.
- [22:40]
Rohan
Yeah. Another version of this that I'll often do is look at the last day of messages and find all the commitments that I made and then kind of give me a prioritized list of that. And so, uh, you know, I think that, you know, I'm, I would say I'm not the most organized person in general, but I think with Codex it enables me to just basically delegate organization and persistence to the tool, because it can. And this. I actually, I think I just tweeted about this, where it's like, there's so many things where you have so much to do. Um, and often it just. The to do list adds up and you kind of feel, you know, there's some anxiety there. And I think what's really relieving is actually you can just ask Codex to do these things, and ultimately, probably 80% of the time, it'll actually just knock the thing off the list that you need it to do. And so for me, that's basically frees up a ton of mental space to be able to focus on more important things.
- [23:28]
Podcast Host
Yeah, I feel like almost all these work streams, including strategy and everything, is like, the machine can do, like, 80% of it, and then the last 20%, you kind of apply your touch to it. Maybe you'll get fewer and fewer over time.
- [23:41]
Rohan
Oh, yeah. And I think also there's. Yeah, it's 80% of the time, and maybe some of the time the machine can do it on 10. But it's like, I think also there's so much time where, especially as product managers, where we just spend gathering context. You get added to a Slack thread and it has 50 messages. You would normally have to sit there and spend 15 minutes just reading them. Often what I'll do is just go into Slack, copy a thread, and then ask codec to just give me the TLDR of this, and then I can immediately kind of respond. And so I think you just. Over the course of the day, I'm probably doing 3 or 4x more than I would have without codecs, which is pretty crazy.
- [24:18]
Podcast Host
And there's probably a ton of. Just, like, outside of the product development process, I found AI super helpful for interviewing candidates, like summarizing interview notes or writing my own performance review. Because it has memory. Right. So you can probably just get it to write a performance review.
- [24:33]
Rohan
Oh, totally. Yeah. Because I'm also worst at actually documenting anything that I've actually done. I mean, I've been a founder most of my career, so I usually just do stuff. And it's usually. But it's funny because, yeah, you can just tell Codex, go look at what's all the things I've contributed in the last three months, and it'll, you know, spend a couple hours and really just put together the definitive list.
- [24:53]
Podcast Host
Yeah, Imagine meetings are more fun too. Right. Because probably a lot of it spent just, like, putting up prototypes and like, playing the actual Product, right? That's my guess. Yeah.
- [25:02]
Rohan
Yeah, I mean, I think, yeah, especially, I mean, honestly, a lot of. Because we work in a lot of. We work very in person as well. So there's a lot of just serendipitous conversations that we have and that's someone showing the thing that they built and then it's, oh my gosh, you know, let's actually a little bit and kind of go from there, dude.
- [25:19]
Podcast Host
So like. So like, you know. So do you think your schedule is filled with meetings still or do you have an actual build time?
- [25:27]
Rohan
Like, yeah, I mean, I'm so. I mean, at OpenAI, one of the things I, I focus a lot on our enterprise motion and so, okay, I do spend. I mean, I do actually probably have a pretty large number of meetings relative to what would be optimal. But a lot of that's actually just with customers. So. But yeah, I would say my build time is from 5pm onwards, evening time. But. But no, I think that it's funny because actually I'll even use Codex as my personal chief of staff. So in the beginning of the week, I'll have it look at all my meetings, figure out, you know, consolidate the ones where it can be more give me focus blocks and figure out which ones could be canceled as a DM or something. So it's definitely helping me find the focus time. But I think in general, what we do in the free time is actually just spend way more time with our customers.
- [26:14]
Podcast Host
I mean, yeah, I think any meeting with customers definitely worth taking. So. Yeah, no doubt about that. And let me just ask some questions because, like, I kind of become a Codex power user too. Like, one thing I struggle with is like, I don't know how people like Peter Steinberger can do it, but like, I can pretty much only work on maybe like three or four Codex threads at the same time, you know, like, how many active conversations do you have going on the panel?
- [26:39]
Rohan
Yeah, I think. I mean, Peter's definitely on another level. So I would say maybe don't necessarily immediately start try that at home. But I think for me, I would say online probably and in general five or six kind of threads. But what's interesting is it's not. It's very delegated, so I'm often not waiting for it. It's just I'll fire it off and then at the end of a meeting or I have a free second, I go back to Codex, I see all the unreads and I actually then have. Oh, actually that was great. It did make the slide deck that I was preparing for for the next some meeting tomorrow. And then I can kind of iterate on it. So I kind of. It's kind of the way that I think about the paradigm is much more, you know, let's say that you have someone that you work with that you manage. If everything they were doing you had to maintain constant state on it, it wouldn't be a very effective delegation. And I think with Codex, that's the goal is how do we move much more to that fully delegated state where it's. It just comes back to you. You didn't have to think about what it was doing while it was doing it. And that I think kind of starts to address that. Just the mental overhead required. And a good example of this is with, let's say you're pushing a feature, you want to get a prod, get a PR landed. Some of the stuff that we've built is you can actually ask Codex to nurse that pr, wait for the CI to finish, then fix the CI and keep going and respond to the human feedback on the pr. And then once it's fully ready to go, then ping me on Slack and let me know that I can merge it. And so it's like now you would have previously had to spend 10 different interaction points thinking about that PR, but now it's just at the very end, okay, cool, look, it's good to go. Um, and then you can kind of go.
- [28:12]
Podcast Host
So have you managed to build a system to manage the system? Like, people talk about loops and stuff on Twitter. Have you. Are you kind of trending in that direction too, or. Or like.
- [28:20]
Rohan
Yeah, I think, honestly I haven't. I'm. I feel not at the next level that Peter's talking about, but it's. So I'm still out here prompting, I think the. But the thing that, for me, I think. I mean, honestly, I would say slash goal is a good version of this where it'll just keep iterating on itself. The thing that I'm trying to optimize more for is basically. Basically the general optimization function for me right now is every time I do anything, I basically think, could Codex have done this? But then there's a whole class of. So that's a whole class of things. Then there's a whole separate class of things that are things that we don't even think to do today because they would just be way too painful. And so I think that that's actually a really interesting bucket of opportunity for automation. A. An example that I've kind of started Doing these days is we have project channels, you know, for different products that we're building and, or you know, partnership things or whatever. And so I'll actually, what I've started doing is actually set up for each of these channels a site that essentially has the full state. It's almost a, A project overview. And then I have codecs basically every couple hours for each of each of these channels go and essentially get all possible, you know, things that have changed or conversation that's happened and essentially integrated into the overall site. And so then it becomes this context source for anybody that needs to onboard or on ramp or even for me if I want to see updates. So there's that word. We were never doing anything on this level before, but it's actually very useful.
- [29:45]
Podcast Host
Yeah, that's actually really smart. So that becomes the source of truth. Right. Because somebody has like going back and forth in Slack and manually updating a PRD is such a pain in the ass. Just like just get AI to do it.
- [29:57]
Rohan
I feel it was just a known thing among everybody that any document is effectively immediately out of date the moment you ship it. And it's. But now we actually can create fairly up to date documents that are updating themselves. Suddenly they actually become a useful artifact.
- [30:11]
Podcast Host
Oh, that's super cool. Yeah. What kind of things do you use go for? Do you like go like hit my okr. No, what kind of things do you go for?
- [30:20]
Rohan
Let me think. What's the most out there thing? I think for me it's honestly I just always use it because I think that I'm fairly latency insensitive most tasks. For the most part, the tasks that I'm asking Codex to do, it's. I just care about it completing it less than I do doing it in 10 minutes or something. Um, and so I'd say it's a pretty standard thing that I'm using. But I'd say the big ones are often when I'm generating from scratch an artifact where I actually don't have as much opinion. I mean often, you know, I'll go in and say this is what I want. Let's say I'm generating a side deck or some sort of document. It's. I generally know what I want and so then it's just using voice to communicate, you know, transcription. I'll ask it to do it and brain dump. But then sometimes I'll ask it to do things that I don't even know and it's very open ended. Then I'll use goal especially for that because it's okay. Go really figure out what's going on everywhere, then come up with an idea. Maybe ask me before you build it, then go build it. Got it.
- [31:14]
Podcast Host
Okay. All right, man. Just a few more questions. Like, do you have a personal set of skills or like, do you mostly use OpenAI skills but like, do you have like a person of Rohan skills to apply your own taste and stuff like that? Or.
- [31:24]
Rohan
Yeah, I, Yeah, I actually don't. So I typically am using. So I mean we've actually built out a ton of interesting plugins internally and actually some of them that we've shipped externally for specific roles, e.g. design or finance and things like this. But for me, I actually am pretty light on the skills. I think part of that for me is I often find that from. It's really useful for me to see where does codecs fall over when it doesn't have any specific stuff. Um, and I'm, I mean, honestly, maybe I'm just like too, you know, small brain on the things I'm trying to do with it, but it's generally pretty good at doing most things. Um, there are a few times though where I, I think what I use skills for is defining workflows is maybe one way to think about it. Um, and often I would say it's either in a skill or an automation, but it's like, okay, cool, we just did this thing in a chat for like 20 minutes. I want to do this again probably a couple times a week now. Turn that into a skill or turn that into an automation or something like that.
- [32:20]
Podcast Host
Got it, got it. Okay, got it. And then do you have any just like high level tips for using codecs for some of the more Codex pale people? I remember Jason published an article about how he uses codecs and the first one was like, just like keep talking to it in a long running thread, which I didn't really do before this. Do you have any personal tips or anything?
- [32:40]
Rohan
Yeah, well, one thing that was a huge unlock is we recently shipped Codex can now control threads. So you can kind of have a high level orchestrator thread that's actually spotting other threads and things like that. So that's definitely, I think, a great way to use the tool. For me, I would say my biggest advice would be or the biggest unlock is one make sure that Codex is integrated into all your tools. So sometimes that might require, you know, building custom clis and things like this, but often it's just, you know, as simple as we have a plugin for the tool and just go integrate it um, and then I would say, and then what's great, though, is even if it doesn't have the tool connected, it can just use computer use often. The other thing, I mean, is honestly just thinking through what are all the stuff that you're doing and why can't Codex do it? Or like, every time you run into an issue, or either it's like, did you ask it to do it? If not, then great, just try. And then if you tried and it didn't work, then I think some folks, it's easy to see it not do something correctly and then kind of just be great. That's. I'll just do it myself. But it's. I think if you always ask the question of what was missing almost as a human, how do we know that it didn't do the right thing? Probably we have some context or some information that Codex didn't have. And then it's just answering the question of what did Codex not have? And then figure out, how can I give it to it so that in moving forward it has it Then it kind of creates this acceleration towards just automating more and more of the things that you're doing.
- [34:03]
Podcast Host
Yeah. You can also string a bunch of skills together into like a whole workflow. Just have it keep running. I found that super helpful. Yeah.
- [34:12]
Rohan
I'm curious for you, what changed the shift when you were. What made the shift for you? Or the aha moment of this feels really useful. Or Codex specifically can really help me with things.
- [34:21]
Podcast Host
I think what blew me away was like, browser use. I really think. I don't know who worked on it, but I really think it's like the number one feature Codex has. So it totally blew me away because, for example, even for this podcast to turn this transcript that we're talking about into. I've turned into a bunch of assets, right? Like newsletter posts, like social posts, like, all this kind of stuff. I got to post to all these different platforms. And then I built this whole automation where it strings together a bunch of different skills. It turns this transcript into a newsletter post based on my examples. There's like a remove AI slop skill that I run a lot of slop stuff. And then there's another skill to post it onto social media platforms. A lot of it, like I said, doesn't have APIs, so it uses browse views. You just figure it out. Recently I made a nerd skill, this. This thing called Hyper Frames. So you can actually make video assets based on, like, text. Right. I have another skill to make slides like HTML Slides. It's just crazy, man. You can string all this shit two together. Oh yeah. I mean, like, you know, I don't, I don't just like auto, automate the stuff. Like there's a bunch of creators out there who just automate everything. Like they just pump out tons of slop all the time. So I actually check the stuff. But like it saves a lot of busy work, you know. Totally.
- [35:24]
Rohan
I mean it's cool because I think even for product, it's funny because we talk about the bottlenecks. You write code more fast with codecs. There's all these SDLC bottlenecks. But then it's funny because I think there's then becomes even bottlenecks in other parts of the org when it comes to shipping so much software. Right. For example, we ship so much product at a pace that it's hard to create marketing assets for them. Really high quality videos and demos and things like this. And so our Devex team, you know, Vermont's team, they developed some skills to actually use codecs to generate nice kind of visual graphic videos. So now they can actually ship demos much faster and things like this. So I mean, I think it's like every time there's a bottleneck, it's, you know, the solution is just find a way for codecs to help unlock it a little bit.
- [36:07]
Podcast Host
Yeah, yeah. I feel like the models are already beyond kind of like most people's ambition of what they should be doing. I think that's probably the number one tip. It's just like ask Codex to try to do it. Let's be purpose.
- [36:21]
Rohan
Yeah. You should aim for something that seems 10x more ridiculous than you ever think it could. And it probably can do 90% of it and it's okay, great. Now I need a reset to be 10x more than that.
- [36:31]
Podcast Host
So what do you think, dude? Do you think. Because I feel like the PM profession has gone through a troll where people just glorify cross functional alignment secretaries for a lot of roles. But I actually feel pretty optimistic about PM now because of all these new tools that you guys are shipping. What do you think? Do you think it's going to become a lot more fun as a job or. It probably already is for you, right? It's just.
- [36:53]
Rohan
Yeah, I mean, I think, I mean, yeah, I'm having fun.
- [36:59]
Podcast Host
Yeah.
- [36:59]
Rohan
A good time. But I mean, I think, yeah, I feel everything that's happening only makes pretty much every role more leveraged. Not even just pm, but every role. It's kind of like, I mean, really what it Unlocks is being able to just do way more and own the end to end way more and work on way more things. I think what. And it's kind of interesting, I would say AI, you can feel AI leverage, meaning you can do the same thing faster. But I actually think where we see what we've been able to do and the most AI built companies I've worked at, OpenAI and Cursor, it's the way that we operate is not doing the same thing, small thing faster. It's actually just doing way more things and doing all of them faster relative to the number of people you have. And so I think as someone who just loves building useful tools and working with customers and just, I don't know, just delivering useful experiences to people that delight them, it's cool to be able to just do way more of that and you're so unconstrained. You know, I think there's literally. I can't remember the last time where it was. It's literally not a question of like, what is possible. It's just everything is possible. It's just like, what should we do? Pretty exciting. Yeah.
- [38:01]
Podcast Host
You just gotta set up a system to do it. Yeah, but, but like, but like, I think, I think maybe a downside or maybe a plus is like you don't, you don't actually end up with working less. You probably end up working more because there's so much stuff that you can do.
- [38:12]
Rohan
Right. It's like, yeah, for sure. I mean, I think, yeah. I mean, I would say for me it wasn't the acceptance criteria was to work less. I mean, I'm definitely just, you know, always been someone who works a lot. But I think, yeah, I mean, I don't know, I think that we're going to see some interesting shifts as these tools get better and better at automating more and more. For sure.
- [38:29]
Podcast Host
All right, Rohan. Well, I mean, kudos to you and team man. You guys really turn codecs around. It's like pro been my number one tool. And where can people find you online and keep following the journey?
- [38:41]
Rohan
Yeah, I mean, I'm on Twitter and LinkedIn just. ROHAN. Yeah, I'm posting every now and then, trying to just let people know what we're shipping. But yeah, I really appreciate you having me on and it's been fun to see all the stuff you're posting and I don't know, it's been a fun journey.
- [38:56]
Podcast Host
Yeah, dude, I love the memes, like people resetting limits all the time and also some of the snark too. I love the. I love it. Yeah.
- [39:03]
Rohan
It's funny because I feel often we're just actually communicating on Slack or. Sorry. On Twitter, apparently someone posts feedback, they'll be like, someone will tag me. Yo, can you look at this on Twitter? Yeah, I got you.
- [39:12]
Podcast Host
Yeah. That's what the best problems teams do, they're all over Twitter, dude, so it makes sense.
- [39:16]
Rohan
Yeah, vibes are great. So.
- [39:18]
Podcast Host
All right, dude, chatting, man.
- [39:19]
Rohan
Yeah, appreciate it. Cheers.