← Behind the Craft
Loading summary
Transcript139 lines
- [00:00]
A
The only job left is to understand what you don't like about something and put into words and like ldai. So much of my work now can just be done through the Codex app. Whether that's the browser, whether that's like drafting messages, sending messages, turn this thread into a heartbeat. I want you to check my emails, my slack, my linear. Every as I've used this over time it's been able to do things like pre draft Slack responses or pre draft emails. The work shouldn't have to stop because you close your laptop. Once that's the case, I'm even more excited for what's possible.
- [00:31]
B
Hey everyone. My guest today is Jason from the codex team at OpenAI and really excited to get Jason to show us how he uses the new ChatGPT work and codex app. From managing threads to setting up skills, to schedule tasks and long running loops. And Jason is one of the best Codex maxers that I know. So welcome sir.
- [00:48]
A
Thank you so much for having me. Excited to be here. Yeah.
- [00:52]
B
So why don't we start with the basics? Can you give us a quick tour of the new ChatGPT app and maybe talk about when I should use ChatGPT work versus Codex?
- [01:01]
A
Yeah, I mean, for the most part, ChatGPT work is sort of a different view of the codecs experience. Right. So it turns out that if you want to build a very good work productivity tool, a really good coding agent is a place to start. And so Codex will be able to show you things like your git history, your pull requests, code changes, whereas work kind of just hides that behind the scenes. You know, if I'm making slides potentially, I don't really, really need to see that. I've written 20 lines of Python to do something. Right. And so work is a way of sort of just redressing those that UX experience.
- [01:34]
B
So it's primarily a UX and maybe a default prompt kind of change.
- [01:37]
A
Exactly. It's. I would say it's mostly the ux. I actually do a lot of my work in work now because I don't have to review. I don't really review most of the code I write.
- [01:45]
B
Yeah, yeah. I think a lot of us don't review code anymore. For better, for worse. Yeah. Great. Okay. And then. Okay, can you share your screen? And I have another question for you, which is, you know, there's GPT 5.6, Soul, Terra and Luna, and each of them have different effort levels. So can you give us some advice on which model and effort level to use, you know, or what do you Default to.
- [02:07]
A
For the most part, I mostly just use Soul Medium. Right. A lot of the work that I do these days has been just organizing my life at work. Reading Slack, you know, setting my calendar, scheduling all these meetings. These are all things that don't need the highest intelligence.
- [02:20]
B
Right.
- [02:21]
A
So most of my work generally is in Medium. And then when it comes to actually building out potential prototype applications. So for example, I'm trying to learn the drums. I've been working on a small little app that just teaches me how to play the drums there I'll set things on extra high or ultra. I'll build out a pretty complicated goal and then I'll just have Codex work for a couple hours to build that out.
- [02:41]
B
Oh, that's interesting. Okay, I'm sure we'll talk about that later. Maybe you can share your Codex app. And let's talk about maybe first how you kind of organize all your threads and projects.
- [02:51]
A
So basically in the Codex app you basically have work in Codex. The only real difference here is pull requests, for example. Right. So maybe we can just go back to work. A lot of my work on the developer Experience team is operationally very heavy. And so half of the work that I do is mostly just managing what's going on externally. And so you'll see here I have a bunch of different automations that are doing things just reading Reddit, reading Twitter, reading LinkedIn to figure out where the feedback is coming from, what are some issues people face. Then I also have chief of staff threads that tell me what I need to focus on on a daily basis or a weekly basis. And you can see that these run at a different automation loop. And then I have things for how I'm planning Dev day. I'm working on a mission video. How do we think about the agents API? These are some of my drumming drumming projects, my personal projects. I'm working on some slides. But you'll see that almost everything is kind of just a pinned thread. Right. I don't really have hundreds of threads working in the background. Every pin thread is a workspace and with Compaction all that's just kind of managed for me.
- [03:55]
B
I see. So everything is kind of a long running paint thread because codecs has really good compaction. It doesn't lose previous context. Right?
- [04:04]
A
Yeah. And everything else I just write down. Right. So for example, if I think something is important, I'll just tell the model to save that in the notes directory if there's actual work that needs to be done instead of telling the main thread to do this task, I'LL tell it to spin up a sub agent. And this is something that can be done automatically with Ultra Mode, but I tend to be a little bit more cautious on when I spin up subagents.
- [04:27]
B
Okay. And I noticed that just real quick that you have a project called Vault. Do you have multiple projects or you keep to like one big project for everything?
- [04:35]
A
Yeah, so because I have so much of my context written down in an Obsidian Vault. Vault is basically the only project. The other one is Animate codecs. That's kind of a side project I'm working on where I can just, you know, put codecs in remotion and just move some stuff around. But everything's in the vault. I have other code bases, like a Monorepo or I have other code bases, different demo projects, and everything starts from the Vault and that way if it's at GitHub when I move to the cloud or anything else, I can just pull that repo and start from there.
- [05:05]
B
I see. Okay, so. And why do you use Obsidian instead of just having a bunch of markdown files or, you know, just better organize things?
- [05:13]
A
Yeah, sometimes it's just nice to open up Obsidian and just see, you know, the images rendered to be able to go through all these cross links. But generally Obsidian is just the structure that I use. It's very rare that I actually open things in Obsidian. It's mostly just in context for my agent. But from time to time it's definitely beneficial to set that up and just
- [05:33]
B
to confirm, this is just a bunch of markdown files. It's not a database or anything.
- [05:36]
A
No, yeah, it's basically all just markdown files. I have a directory for different projects, I have a directory for people, I have a directory for different notes, I have my daily notes and my preferences, and these are all just things that I saved down.
- [05:50]
B
Okay, got it. All right, dude. Well, let's talk about the chief of staff. So basically the Chief of staff, you have a daily one and a weekly one, and they work based on scheduled tasks, right?
- [06:00]
A
Yeah.
- [06:01]
B
So how does that work every day? Let's take the daily one. It schedules. What. What does it do?
- [06:06]
A
Yeah, so I have a automation that runs at 9am and at I think 1pm and 5pm and basically all it does is it has to read all of my slack, it reads all of my Twitter DMs using computer use, and then it will read all of my emails that I have not responded to and my linear board and it will just give me a glimpse of what's happening right on linear if it finds that something is needs to get cleaned up, maybe a status needs to be updated, codiff will just do that for me. And so I just always get a clean view as to what needs to happen. So for example, I'm going to Brazil really soon and it's been able to just show me that I need to go figure that out. It's also telling me I need to go pick up a package.
- [06:48]
B
Okay.
- [06:48]
A
One other thing I do, for example, that the Kachiva staff is really, really helpful on, that I want to share is just checking the flights now. So usually because I have a lot of travel, if Codex finds that there is a booking ticket and a reference number, it will check in the flight for me and text me the boarding pass to my personal phone. This is my work laptop. These are just small things that I just have set up over time. This is not something that just comes for free in any way. But as you start realizing how you can use Codex now, if I'm in New York, I'll just get a text message when I realize that my Codex has just sent me the boarding pass. But see, you know, it's telling me about the artifact. I had some notes prepared for our meeting today, right? There was a linear ticket that was made. This is the meeting I have afterwards with the marketing team. You know, this is just someone from the company giving some feedback, right? Some projects, logistics and then some other projects that I'm working on. And then what you'll see here is someone told me that there's a production thing that's happening in Toronto. I just take an app shot and just make sure they're in my calendar and that's basically how I use these things.
- [07:50]
B
This episode is brought to you by Google AI Studio. The easiest and fastest way to build and share your idea with real users. For example, I use AI Studio to create three different landing pages for this bakery using images generated by Google's built in landing nano banana model. One highlights fresh bread, another highlights awesome cakes and the last highlights the bakery story. I can publish the website easily and get a custom URL in this case corgiconfections AI Studio to share all three variations with potential customers for feedback in one tap. When I was a product manager, Google AI Studio was my go to tool for building prototypes like this. And it really is an easy and free way to to share your ideas with your team, customers and everyone else. Check it out@aistudio.google.com now back to our episode. Let's say I'm just a Normal person trying to set this up myself. I don't, you know, let's say I have email, calendar and maybe Slack, you know, so what kind of prompt do I voice dictate to Codex to get this thing going?
- [08:55]
A
Yeah, I mean in the beginning I would say turn this thread into a heartbeat. I want you to check my emails, my Slack, my linear every whenever it makes sense, 9am, 1pm, 5pm and just tell me what I need to prioritize. Just by doing that you'll get a pretty good overview of what you need to do and then over time as you tell the model its different preferences. So for example, when it first did this, it didn't include links. I would just say great, well now I want you to make sure that everything has a link. And then over time as things got more complicated, I said, you know what, I want you to start using linear. Right, got it. Now as I've used this, over time it's been able to do things pre draft Slack responses or pre draft emails. And so instead of just telling me that an email came in, it'll give me a link to that opens the email draft pre written and then I can just clean some things up and then respond. And those things have been super helpful.
- [09:50]
B
Okay, so you never actually get to just automatically send a Slack message for you. Do you do that?
- [09:54]
A
I do that sometimes as well, but it really depends on the context. Right. For example, if I'm working on some code I might just say great, make a pull request and then check in on this pull request. And once all the tests are green, I want you to do DM Andrew on Slack with the link to the pull request and the preview URL. It can do that quite well. But the cheapest app thread because it reads 40 emails and 100 Slack messages. I want to be a little bit more surgical on how things are handled.
- [10:22]
B
And you can try to also learn your voice over time. Read my sent emails or read my messages and of draft stuff in my voice.
- [10:29]
A
Yeah, so one of my favorite skills is basically I have a write me skill. And so if you open up Codex and say, hey, I want you to use a Slack connector and read the past week of Slack messages I've written and make a skill to figure out how to talk me, it does a pretty good job. And then I might say, but I also want you to recognize that I want you to talk differently when you talk to external users versus people on my team. And I want you to talk differently whether you're messaging an executive or a coworker. And you start investing in these skills over time. So the first version just talks about you and then as you go over time, you learn these things.
- [11:06]
B
Right.
- [11:07]
A
You're always editing the skills. I do the same thing with email.
- [11:10]
B
Okay. Are you able to share this rightly scale?
- [11:12]
A
So I have my personal, you know, repo of all the skills that I use for my personal website. Sorry, my personal codec. And so yeah, I have an email me skill. I have a tweet me skill. I have a skill that turns my voice transcripts into blog posts. I have skills that turn videos of me into video essays. A bunch of different stuff here that has been very helpful for me.
- [11:36]
B
Why don't we. Can you open one of them? Tweet me or something? Okay.
- [11:42]
A
Yeah. So it's okay. There's a couple of different voices that you have, Right. Sometimes you're teaching different things in codec, sometimes you're sharing demos, sometimes you're just replying to people who are giving feedback and collecting feedback. Right. It's being able to find all these things and figure out the kind of language I use. And I just small examples of what
- [12:00]
B
this might look like and how much of this does you actually type out versus just get Codex to write it out.
- [12:06]
A
Most of this is written by Codex because all I really need is sort of the ability to do these kinds of references.
- [12:12]
B
Yeah, I have a lot of these skills too. And I think you'll agree with me because codes can do all this stuff. I feel it's become the hub of everything. I kind of read my emails and send emails through Codex and write social media posts and post them through Codex too. So it's kind of like this hub that I actually used to do everything else.
- [12:28]
A
Yeah, totally. I feel so much of my work now can just be done through the Codex app, whether that's the browser. Right. Whether that's drafting messages, sending messages. And then now with computer use, even the small things, booking a flight, I just get to do that with Codex. And it's been pretty funny to sort of watch these things get better over time.
- [12:44]
B
Do you worry about that as the skills proliferate and all this stuff proliferates, that there's just a lot of bloat inside everything that you use, try to clean up periodically or no?
- [12:57]
A
Yeah, I clean them up tons. So one of the cool things with Codex is all the sessions are saved on a database somewhere. I don't even know where it is at this point, but if you just tell Codex and say, hey, I want you to look at the past 400 sessions, which skills were never used. Can you clean things up, interview me and go through this process? It generally does a pretty good job. Recently it said, hey, you don't really use these data analysis skills. You can just delete them and reinstall them when you need to. You use a bunch of different Slack skills. Should we consolidate them? Or even the fact that, hey, you have a write skill, a slack skill, and an email skill, should we have merged them into one thing and then it just starts cleaning things over time? Right. It's no different than hiring someone day one. You have to tell them a lot of information and you give them feedback and they change over time. And that's how I think of using Codex. It's not the fact that I'm going to have a skill that just runs forever perfectly right. These are always things that are going to be improved over time.
- [13:52]
B
Okay, so maybe you have a scheduled task thing that just runs this cleanup job and then it pings you on a regular basis.
- [13:57]
A
Yeah, I mean, I used to have that. I have that less and less these days. I really just sort of decide at some point maybe I should do some cleanup sometimes clean up the work trees and stuff like that.
- [14:08]
B
And of course, the skills need to improve over time because, for example, all the writing skills that I use, they never get anything perfect in one shot. I have to go back and forth with it a lot. So do you just say, hey, look at our past conversation and try to update the skill, or do you have
- [14:22]
A
some sort of trick to this? That's basically it. Yeah. It's kind of incredible. I feel one of the disappointing things I always tell people so much of it is just, you just tell Codex. Why are you giving me this feedback? Just tell Codex. Yeah, so oftentimes when I'm doing writing, I'll just say, hey, it's been a couple weeks since this skill has been updated. Go read the last 200 Slack messages. What has changed? Go read the last 200 tweets. Go read the last couple of blog posts I wrote. Something else I do is every once in a while I'll just say, hey, read all the sessions that invoke skills. And what was the feedback I gave to those skills? Four months ago, I built a skill called self improve, which I'm not updated on my personal repo yet, but it's probably changed a lot. But all it does is it says, okay, look at these skills that are the most called. Read the sessions of those skills and tell me if there's been Any consistent feedback about them. A recent example of this was actually the Slack skill, where basically what it realized was every time I do something from Slack, I ask Codex to then reply back into Slack. For example, someone might say, jason, can you make sure that this person is included in the program? Then I'll tell Codex this question. Codex will say, yes, they're in the program. And I keep telling, can you post back into Slack, let them know that it's been done over time? Codex will now just decide, oh, you asked me to handle something on behalf of a request internally. Once I'm done, I'll reply back, these are some small details that just reduce the friction and at some point you just get more time back.
- [15:53]
B
That's really smart. Okay, so I've done the self improvement thing by just kind of manually asking each thread to improve. But I haven't done something where you've done where you build a skill to look at all the past threads. So it actually saves all the past threads in the database somewhere.
- [16:05]
A
Yeah, they're all just in session JSON. Yeah.
- [16:08]
B
Okay. Then you kind of review the feedback and you kind of go do one and three and skip two or something. You kind of review it.
- [16:14]
A
Right? Yeah. Yeah. I do agree that I do ask you to. I feel this pain. I do ask you to close the loop every single day. Please just do this automatically. And if I told it to close a loop day one, I think I would be too nervous. But now, because I know it's something I asked for and it does it correctly, I feel much more confident about updating these skills to just Slack on my behalf.
- [16:38]
B
Got it. Okay, so it sounds like we both run our kind of operating system with codecs using a bunch of skills. Can you talk about what the difference between a skill and a plugin is? Should we be building plugins too or.
- [16:48]
A
Yeah, I think generally a skill is sort of one component of a plugin. Right. So a plugin is a couple of MCP servers. Potentially it's a couple of skills and it's a couple of different resources and scripts.
- [17:03]
B
Right.
- [17:03]
A
So, for example, I right now have the write me skill, the Slack me skill, the email skill. Generally I think the right thing to do is if I wanted to share this with my team, I would build a single plugin called Better Writing. It contains all these skills and maybe some other MCP servers. And that's the thing I can share with the rest of the team. So I would say a skill is one piece of a plugin.
- [17:25]
B
Okay, so plugin is maybe a Collection of skills with a bunch of scripts and other files that you can share more easily.
- [17:31]
A
Yeah, I would have a content creation skill that might have all these different sort of content creation plugin that has all these skills combined.
- [17:38]
B
Okay. And I should notice, but let's say I actually do have a content creation skill. So let's say I want to make it into a plugin and publish it to everybody who uses codecs. I can do that.
- [17:49]
A
Yeah. So since last week we just had the ability to have folks be able to submit their own plugins into the Codex plugin directory. And so anyone can now just submit these things. If it's just skills, it gets reviewed pretty quickly. If it's mcps, we'll do a little bit more of a process to audit the security. But generally you can just get that installed in anytime you want now.
- [18:11]
B
Nice, dude. Okay. All right, dude. Well, let's talk about browser and computer use, which in my opinion is probably my favorite feature that codecs has. It's pretty incredible. Let me ask you this. What is the most incredible browser and computer use thing that you've seen?
- [18:27]
A
I think the example that I always talk about was this time when I was going on a bike ride. I'm going on a bike ride and I check my phone and. And at work someone says, hey, Jason, can you re export this video for me? Something about the subtitles is not working correctly and I need to GitHub access to review the code myself. Can you help? So I connect remote control to my phone to my MacBook and I text it. Can you please use computer use to figure out where this video is, what tool was used to export this video and just edit the video and post it back on slack. And then 20 minutes later there's another video that has been shared on the Slack thread that has all these fixes changed. And then I basically said, Great. Every 30 minutes, check the Slack thread, look for any feedback. And with any feedback, just export v2, v3, v4 of this video.
- [19:15]
B
All right.
- [19:16]
A
And then when I got home, it was greenlit and it was a launch video for migrate to Codex. That was pretty awesome.
- [19:23]
B
And did the other person know that your Codex was replying to him or her?
- [19:30]
A
I mean, I think if you look at the time, I don't think they would have been able to know. Right. It was just the fact that every time they gave some feedback, 40 minutes later there would be another new video with the wording changed or something made a little bit faster, a little bit slower. Got it.
- [19:48]
B
Do you have any Tips for getting the most out of browser use or they're just asking more and more ridiculous things. Do you have any tips?
- [19:55]
A
I mean, I think the biggest one is understanding that now browsers. The browser inside Codex has the ability to do auth and bring in your cookies so you can use it in a login state, which means you just have access to way more information. The second thing is just understanding that you can also control multiple tabs. And so if you want to do things research or comparisons and preparing your browser for some review, Codex is really, really good at that. A really simple example. I wanted to go shopping. So one thing I could have done is I could have said, okay, I want you to do some research and give me a markdown file of the things I should buy. But instead what I said was I want you to do this research. Once you're done, give me a recommendation. But I want you to open every product in its own tab. And I have to get my coffee. And when I came back I had four tabs open. I read the review, I looked at all four and I just pressed checkout on the one that I really agreed with.
- [20:52]
B
Got it.
- [20:53]
A
Maybe in the future I'll feel confident in letting it check out for me. But just presenting the information as four tabs was very useful.
- [21:00]
B
Yeah, some people are using, I think browsers trade stocks or something. I don't think I trust it enough to do that kind of stuff.
- [21:06]
A
Yeah, I mean I know people who've been buying stuff with computer use for quite a while. So maybe I'm not there yet, but maybe I will be there one day.
- [21:12]
B
And so that's actually something that maybe I have some feedback on. So. So do you primarily use the browser use now in the in App browser? In Codex browser. So you have to sign into everything?
- [21:22]
A
Yeah. So generally I just didn't know that my AI has its own browser. And then Chrome is the thing I use. That's how I usually think about it. And then computer use is basically on the application level and so it can do things if I. If it needs to control settings, it'll use computer use. But if it's any website, I generally try to keep it within the in
- [21:42]
B
app browser computer use. I've only used it to clean up my downloads folder. I don't know if you have any kind of fancy things use cases for computer use.
- [21:53]
A
Nothing too much. I mean most of what I care about is never having to fill out a form again, never having to do some automation and figure out some API. So for example, our Open source program is a form that goes into airtable, and to get the data back out, I have to download the CSV file. And so part of my automation just opens it, presses download, moves it into some other drive, and then does this automation. And at this point, it just works. Other than that, most of it is going to be testing apps.
- [22:27]
B
Testing apps. Okay, got it. Yeah. Yeah, that makes sense. Yeah. Okay. That's actually a really good segue. I love to now talk about how you build apps. We can top up your drumming app, or is that a website or is that an app?
- [22:39]
A
Same thing, Right? So I think, yeah, basically a couple. Couple of weeks ago, I ordered a drum kit. It's on my way. And I learned that I need to learn things like rudiments, which is, you know, how do you do your sticking? I learned that the best way of learning is doing call and response, which basically says, you know what, someone will play a drum beat and you copy them. So I said, great. Codex. Make me an app that has a library of different rudiments, and I want to be able to change the tempo. And then as I learn a new concept, I add it into the app. And now I'm able to upload any YouTube video of music, take the drums out, play drums along with it, put the drums back in, play with the drums, and I can just make a custom learning app based on my own progression. And that has been really, really fun.
- [23:25]
B
Yeah. Can you show us how you built this app? Maybe from scratch?
- [23:28]
A
I mean, it's been like a week of work, so I don't know if it's from scratch.
- [23:33]
B
Okay. Okay.
- [23:34]
A
Again, one of the things I want to really call out, it's cool to have these demos where it's just. Yeah, I ran a goal for six hours. But a lot of it is the fact that you can grow these things over time.
- [23:43]
B
Yeah. So what I really want to understand is as you talk about running the goal for six hours. Right. But to do that, you need to have a really good plan or a spec or something. Right. Initially. So how did you come up with that?
- [23:53]
A
Yeah, so basically I just had a long conversation with it. Right. I basically record and say, hey, you know what? I want to learn how to play the drums. A bunch of YouTube videos talk to me, telling me about things, practicing rudiments and doing hand and feet coordination. And look at this. It's like, you know, 300 sub agents. I want to learn how to do all these coordination. What are the. What are the core skills I need? And then with that I basically built out a plan MD file, a goal MD file and a work log. And this is part of that skills library. It's just called Ultra Goal. And by doing that, it's able to work on different applications. So here, this is an example of me being able to upload a song and play that. Let me see if I can just tell it to serve the app.
- [24:32]
B
So what I want to understand is you wanted to learn the drums and you wanted to build an app. Right. But it sounds like the first step was to get it to just talk to it and figure out what you're trying to build.
- [24:41]
A
Yeah.
- [24:42]
B
And then you said you generated some sort of a plan MD or go MD or something. What's the difference between those two?
- [24:49]
A
Or. Yeah, I mean, the goal is sort of like the success criteria. Right. So, for example, a goal would be at some point, I want you to be able to paste in a YouTube URL and actually extract the different parts of the drum set. Right. That's a goal. And until you can do that, you have to keep working. And the plan is like the implementation detail. I want you to write this in React. I want you to have two different tabs. I want you to use these technologies, and if you don't know how to use these technologies, read the documentation and basically go from there. And the worklog is just a way that I can just review the code and say, oh, yeah, it's really getting stuck on this permissioning issue. The work really is, for me, as just someone on the DX team, to understand what are the limitations of these tools?
- [25:40]
B
Oh, the worklog is kind of updated as you go along during the app.
- [25:43]
A
Yeah, it's kind of like the log itself, because through compaction, I'm not going to read every message.
- [25:48]
B
Right. And the plan and the goal, you got Codex to generate it, or is it.
- [25:51]
A
Yeah, I would almost say if you are trying to build good goals, don't write the goals yourself. Right. Like, I think the right thing to do is to basically tell Codex what you want, describe what the success criteria is, and then just say, great, now set your own goal to accomplish this task. It is generally much better. And the reason I have the goal as a goal MD file is that I can just edit that file every once in a while and the goal will still keep running. So if my scope expands, I can just edit the goal without having to.
- [26:26]
B
Do you mind just sharing the goal to md or can you open it?
- [26:29]
A
Basically, I want you to verify the ability that to transcribe a video from a YouTube video. Yeah. This is the test song. This is the video id.
- [26:39]
B
I see. So goal is not just build me a really cool drumming app to do xyz you gotta people to verify.
- [26:46]
A
Exactly. Yeah.
- [26:48]
B
Verify some sort of core functionality.
- [26:51]
A
Yeah. Right. So in the example it's I need you to be able to open the app using computer use, upload this YouTube video and get the data back out.
- [26:58]
B
Okay, got it. And before you start building this thing, do you have, I guess it's a personal app, but do you get ChatGPT images to generate some designs or do you. Do you know that I think that
- [27:08]
A
was needed more in the 55 world for 5 6. I think the front end is good enough that I don't really need to worry about these things. Usually I just tell it to use certain tools. So I say I want you to use Tailwind and shadcn.
- [27:21]
B
Okay.
- [27:21]
A
And it has become much better at doing front end without using ImageGen.
- [27:25]
B
Got it. Okay. So in the beginning of every project you have a plan and a goal and well, we can't end this without you actually showing the drum app. So you can show it real quick.
- [27:33]
A
Yeah. So I now just have all these different tools to tell me how to play these things. Right. So this is a four limb grid where left hand, left foot, right foot, right hand.
- [27:44]
B
Oh, wow.
- [27:44]
A
And it counts me in and I'm just trying to, you know, learn these little patterns by myself.
- [27:51]
B
Oh, you're just going to follow along on the drum.
- [27:53]
A
Exactly, exactly.
- [27:55]
B
Got it.
- [27:56]
A
And then it just says, okay, now I want to learn like polyrhythms. How do I clap five beats in one hand and two beats in the other? Yeah, right.
- [28:05]
B
That's pretty cool, dude. So normally to learn this stuff you watch some YouTube videos, maybe take a course or something, but basically you build an interactive app to teach yourself.
- [28:14]
A
Exactly. Right? Because now I can add anything that I want. So one thing is to be able to do this, which I think is the first step in just learning the coordination. But I would love to be able to then at some point say, you know what, I want to give it a YouTube video. I want to learn this drum solo codex. Break this drum solo into small parts and put these parts in the same components and change the tempo so I can practice this these things. And yeah, this is just a good way of warming up, but also understanding how we can use AI to build these tools myself.
- [28:41]
B
So you have an engineering background, but what if someone doesn't know, for example, to say, hey, use tailwind or this is how you convert the YouTube video into this drum thing. Should you get codecs to give you some trade offs?
- [28:52]
A
I mean, I think, again, this is one of the things where a lot of people in college, for example, ask, okay, now that coding is solved, what is the thing that people should really figure out? Right. And the example I always give is, hey, if you want to. If you want taste, you have to eat. I think our job now is to come up with that language and learn more about what you want. And so a part of it. Yeah, it makes a lot of sense to know all these things ahead of time, but it's also just really valuable to be dissatisfied with how good something is and learn the vocabulary and the words you need to figure out what it takes to make things better. And so I would just rather say tell Codex to make something and just think hard about what you don't like about it. Right. And learn more as you go. Right. The benefit of even learning to code has always been just go build something, and as you face problems, you'll learn to face these problems. And maybe four years ago, the problems were just, how do I write the code? But now it's like, oh, no, I want you to. Oh, actually, I don't have a good experience. Why is that the case? Oh, I don't have the right keyboard shortcuts to change the tempo of the song. Okay, let's add a keyboard shortcut system. Oh, I don't like the fact that I can't switch between these two panels. Why does it flicker so much? And you'll ask codecs, and codecs will say, oh, the reason this flickers is because we're not eagerly loading this page. Great. Fix that. I think as long as you're dissatisfied or as long as you're conscious of what you don't about something, you can make progress towards making it better. That's the real skill. You don't need to know everything ahead of time.
- [30:19]
B
Okay, got it. Yeah. So it's kind of like you iterate and you prove it and you learn stuff over time.
- [30:23]
A
Yeah, yeah, right. It's the same with skills. It's the same with hiring somebody. My expectation is never that AI will solve my problem at the first step, because I don't expect humans to know what I want unless I really knew what I want.
- [30:34]
B
I think you kind of have to go the extra mile because Codex will just build something for you and be go fix the drum or go fix this thing and they will fix it. Right. I think maybe there's some way to add Some instructions. Hey, can you actually explain how you fixed it or what kind of trade offs they should make to actually understand what's happening behind the scenes?
- [30:51]
A
That's the same thing as saying that go fix it is. I think that's the issue is that people keep saying go fix it without actually thinking about what is the thing that they don't like. I think the things I'm talking about are more. Oh, I really want to think about how we should make better keyboard shortcuts. Could we color code things better to make sense of it a little bit more? Instead of having circles, can I have right hand, left hand, Right. It's actually about being really conscious about the things you want it to improve. If you just say make it better 20 times, it's going to be hard to figure out whether that's actually going to give you good results. Right?
- [31:24]
B
Yeah. So I guess leveling up, building of Codex is just learning to be more specific in the feedback that you give the AI. Right?
- [31:30]
A
Yeah. And I think that's the only job left is to understand what you don't care about something and put into words and tell the AI.
- [31:37]
B
Sometimes I feel just getting a little bit philosophical as the last question. I kind of feel like these days we're all managers, Right. We're giving feedback to our AIs, but sometimes I kind of miss the craft of just getting there and getting my hands dirty, whether it's changing some code or manually tweaking some tweet or something.
- [31:53]
A
Yeah.
- [31:54]
B
Do you kind of have that feeling
- [31:55]
A
or not too much. Right. I don't think of myself as a manager. I don't really think about myself as being heavily rewarded for working hard. Everyone works hard at OpenAI. But what makes you stand out? You stand out because you care about how good something is. You care about the results that something's going to deliver. Right. And for me, this app is a good app. If I actually learn to play the drums better, this app is a good app because I've made it shareable and other people can use this tool and learn something about maybe how it's made or how to play music. This app is successful if I can build a small community around that. How the code looks to me, I think is secondary to the outcomes I want to drive, whether it's for myself or for others.
- [32:38]
B
Okay. Basically the user experience and the user outcomes.
- [32:40]
A
Yeah, yeah.
- [32:41]
B
Makes sense. All right, dude. Well, I mean, you guys are doing an amazing job. I'm curious. Just over this past weekend, there were so many updates from Thibault and everyone else. Do you guys have any kind of principles in the team that just. Or is this. Everybody you hire cares a lot about the community and the users? How do you guys.
- [32:57]
A
Yeah, I mean, I think so much of it comes top down from Tebow. What I said before.
- [33:02]
B
Right.
- [33:02]
A
Everyone works hard. Like Tebow works hard, I work hard. But I think just Tebow cares a lot. Right. And Tibo engages with the community in a really great way and I think that just trickles down to the rest of the team. And the rest of the team cares a lot as a consequence of that.
- [33:15]
B
Yeah. I think it's important just to be transparent and talk human because I think it's very easy to just become very corporate and do press releases and stuff. I've seen this in my whole career and it never works out. People just want to talk to the human face.
- [33:27]
A
Yeah. I think Tebow's been doing a great job of just representing the company and also representing the users towards the company as well.
- [33:33]
B
Awesome, dude. Well, where can people follow you online and maybe give us some hints about what's coming next for Codex, how it's going to take over even more?
- [33:41]
A
Yeah. So you can find me on Twitter @jxnlco if you want bigger updates on the product. You can also check out OpenAI devs. And I think the biggest thing for me is just like so far Codex has been on the desktop and we're moving a lot of that into the ChatGPT work experience in the cloud. Right. And so some of the automations now can be running in ChatGPT work rather than Codex. And I think I'm really excited to just make more progress in the cloud experience overall. Right. The work should have to. The work shouldn't have to stop because you close your laptop and you know, once that's. Once that's the case, I'm even more excited for what's possible.
- [34:14]
B
Yeah, we talked online about that live calling feature too. You got to let me call my Codex and give us some directions.
- [34:21]
A
Yeah, I think definitely voice of the future. Like I use dictation a lot and I think we're going to make a lot of progress. There's.
- [34:26]
B
All right, Jason, well, thanks so much for your time, man. I definitely learned a lot in this conversation.
- [34:30]
A
Awesome. Take care. Thanks for having me.