← How I AI
How I AI
How to design AI agent loops: schedules, goals, and subagents in Claude Code and Codex
00:29:06
Loading summary
Transcript1 lines
- [00:00]
A
Prompts are out and loops are in. If your agent isn't able to prompt itself through an automation, what are you even doing? In today's episode, I'm going to teach you what a prompt is in normal person speak, how to write one when it's useful, and some pitfalls to watch out for. We will be doing this in Codex and in cloud code, and at the end of this episode, you'll be one of the cool kids whose agents prompt itself. Let's get to it. This episode is brought to you by workos. AI has already changed how we work. Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically. But there's a catch. These tools only work well when they have deep access to company systems. Your copilot needs to see your entire code base. Your chatbot needs to search across internal docs, and for enterprise buyers, that raises serious security concerns. That's why these apps face intense IT scrutiny from day one to pass. They need secure authentication, access controls, audit logs, the whole suite of enterprise features. Building all that from scratch, it's a massive lift. That's where work OS comes in. WorkOS gives you drop in APIs for enterprise features so your app can become enterprise ready and scale upmarket faster. Think of it like Stripe for enterprise features. OpenAI perplexity and cursor are already using work OS to move faster and meet enterprise demands. Join them and hundreds of other industry leaders@workos.com start building today. Okay, so why are we all prompt maxing? Of course, it's Pete at openclaw who told us we are old news if we are prompting and we really need to be designing loops where our agents can prompt themselves. Now this one tweet spun off tons of content about what is a loop, how to use a loop, and to be honest, I don't think any of them explained it very well. So I am here to answer your safe space questions about what is a loop? How do I get one set up? Is it really that useful and should I really be letting my agents prompt itself? I think the answer is yes and yes. There are tons of great use cases for loops and we're going to talk about how you can use those and how they can be beneficial, especially with software engineering. But there are some reasons why you wouldn't want to use loops and honestly I still do a little prompting. So don't worry if you are not loop maxing, you are in good company and you can still get a lot done with AI. So to answer what a loop is I'm just going to make this super simple for you all. And this goes back to one of the earliest articles I wrote on openclaw, which was this article about why Open Claw feels alive even though it's not. And the core of this article was explaining that there are many ways you can prompt an AI agent. And often we only think about one way to prompt an agent. But actually there are many ways an agent like Claude Code, like Codex, like ChatGPT, like name your favorite agent here can be prompted. And I want to go over what those ways are. First. There are messages. This is a human triggered input. This is probably how most of us are prompting our agents. We are going to a chatbot and we are typing in some sort of prompt, waiting for a response and then typing another response. That is a message turn based prompting strategy. I still think there's use for this kind of prompting. I use it all the time. But that is not what we're talking about when we're talking about loops. Instead, when we're talking about loops, we're talking about automated prompting of an agent. And there are a couple form factors that can take and I just want to remind you what those are. And I'm using open Cloud because I think it demonstrates these types of prompt loops but is not the only system that does them. So the first one is a heartbeat. You can set a schedule like every 30 minutes, every hour, every five minutes. And on that schedule it's going to kick off a task. And so you're going to say every five minutes, check if I have a new JIRA ticket and if so, start a coding agent to triage and fix that JIRA agent. That's sort of like on a heartbeat every five minutes. I want it to do that. Then there is a cron. A cron is at this time or on this schedule. Do this. So it can be at 9am, it can be at a specific time, it can be every Sunday night. These crons are a little bit more scheduled. A heartbeat is kind of on a regular basis. Crons are more on a set defined schedule. And then the last thing that I've talked about are hooks. So you can prompt an agent based on an internal life cycle. Like a tool was called, a session was started, a session was reset, or an external hook, like a web hook from an external session. Every time I receive an email I want to get a web hook and kick off some sort of agent. And I only remind you of these things because these are common ways to do automations outside of AI. So we were doing automations on heartbeats, on crons, on hooks, way before AI even happened. But now you can do that in order to prompt your AI. And so I think this whole concept of a loop is really just reminding people you do not have to use your human fingers to type in a prompt in order for your agent to do work on your behalf. Now what's different between when I wrote this article and now is a new type of loop has been shipped as a first class citizen of both cloud code and Codex, which is a goal. A goal is a type of loop that sets an outcome and, and runs an agent against that outcome until the outcome can be measured and validated or the agent is blocked. And so I would say there's one more loop type that's becoming pretty common in AI coding in particular, although I think there's lots of use cases for it. But again, pretty simply a loop is a scheduled or kind of semi autonomous automation that allows an agent to instruct itself what to do, prompt itself and get that work done. Now what do you need to write an effective loop? I like this article by Addie Osmani about loop engineering. I think it's really good. It does break this down pretty well. You can see it's fairly recent, it's from this month. But my favorite part about this article is what you need to write a good loop. To write an effective loop you need these five things. I like how this is written out in this block, in that it's, it tells you what the thing is. It's an automation kind of what its job is. So it's like triage of a task to be done or a prompt to be set on a schedule. And then it shows you how Codex and claude code do this. And so for Codex your automations can come out of the automations tab and you can actually define your automation in the schedule there. And then in Claude code you have scheduled tasks, both of them have slash, goal and then they all have different hooks and integrations. Cloud code has the benefit of GitHub Actions, which I think is nice for engineers, but both of them are basically at parity in terms of the types of automations that you can run. And then a couple other foundational things that I think are helpful when you're running loops. And why are these things helpful? Before we get into what they are, they just keep the work clean. If you are going to be yoloing loops all over the place, you're going to want some consistency in execution, you're going to want clean Workspaces, you're going to want conflicts resolved and avoided. And so all these things are really to make those loops effective. And so what? What are those things? They are work trees. I feel like this entire podcast could be get one on one. But work trees are just basically a way to isolate the work, especially the coding work of an agent away from other agents. Work in a sandbox. There are skills, repeated ways to do common tasks. We have a full episode on what skills are from earlier last year when they came out. Plugins and connectors. These are just the tools that your agent has access to. And so those can be like GitHub connectors, connectors to Google Docs and Google Calendar, and plus plugins, which are some instructions on how to use those tools. Sub agents, both codecs and clog code allow you to kick off sub agents. This is just a way to federate out work from the main thread so that sub agents can do specific tasks, especially validation. And then there's some way to track state. And essentially just think of this as like a to do list, so you can save it in a markdown to do list. You could use Linear as a task tracker. Both Claude Code and Codex use this. And so if you put all this together, basically what you have is a way to kick off an automatic prompt, a way to keep that prompt going until the job is done. And the way you keep it going is you can keep it scheduled or. Or you can give it a goal. And it can't exit the loop until it's hit that goal. And then you empower this agent that has been kicked off autonomously with the isolation it needs not to get in each other's way, and the tools it needs to get the job done, including its little army of sub agents. That's it. So again, I promise you, I'd explain it to you very basically what a loop is. A loop is a way to autonomously kick off an agent with a prompt or set of prompts on a schedule or on kind of a recurring basis until it's done. It could be done because the time's up, or it could be done because the job's done. Now people are gonna ask, what should I use a loop for? And when you're designing loops or designing agents, I say, this is the time for the manager. You are designing a job. And so just imagine that you're onboarding an employee. That employee could be an executive assistant. That employee could be a customer service agent, that employee could be a software engineer. You're onboarding this person and you're going to say, you know what, every Friday ea, I would like you to review my calendar, see who canceled on me, where I could have used my time more effectively. If there are any follow ups and send me a slack to get this done. And I want you to do that every Friday. Guess what? You've just designed a loop for your executive assistant. If you have a software engineer and you say, you know what, every hour I want you to check if there's a GitHub issue that needs to be addressed and if there is, triage it, write some code, put it up for code review. Congratulations, you just wrote a loop. And if you have another software engineer and you say, every time you get a PR to review, I want to make sure that you iterate over it until it meets our defined code standards. All, all the lints and checks are clean and it's ready to deploy and you just work on it until all the checks in the GitHub PR are clean. Guess what? That is a goal loop. So I really like to think about loops as designing workflows and designing jobs to be done for people. It just happens to be that you can put this intelligent agent against the loop and then it's ready to go. So that's it, that's a loop. And you know, I think people get intimidated by these complex like dynamic workflow diagrams and these hype boy posts about how they're running thousands of loops all over the place and they never prompt anything themselves. And again, I just want to make this really accessible for folks. If there is something where you feel like every day or every hour a set job can be done, that's a good time for a loop. And don't set your alarm and wake up and type into Codex or Claude code the prompt that would kick it off. Instead set it up to do that itself. And the one magical thing I would let people know is you can create loops to have your agent prompt other loops. So again, you can think about a human with a team of agents who all have their own team of agents and you can start to get really creative about what these loops are. But let's go ahead and build one. So I'm going to build one in Claude code and one in Codex and I'm going to do one sort of non technical one and one technical one. And then we will end with some warnings about using loops. Okay, I wanted to start with a non technical example and I pulled up Claude cowork instead of Claude code just to really break this down for the people who are not technical. And what makes me laugh is there is a loop front and center here in Claude Cowork, which is my morning briefing. And so if you have used a scheduled task in Cloud Cowork, guess what, babe, you have written a loop. It's a loop that kicks off on a regular basis. It prompts itself on what it needs to do, it gets the job done. And when I have my morning briefing, I am finished. And so scheduled tasks are a loop. Now, it's not a goal style loop, but it is a loop itself. So if you are wanting to tiptoe your way into loops, I would say Cowork. Scheduled tasks or in Claude code, what we call routines are perfect ways to get started. So the morning briefing is a perfect loop 101 for you all to start. But let's actually make this a little bit more intelligent. I'm going to go into Claude code and I am going to create a loop that's a little bit more technical, but for the product folks that are watching, okay, to start in Claude code, I'm going to write a loop or a routine that's a little helpful for the product managers, engineering teams out there. It's not a super technical one, but it's one I think is really useful. And so what I'm going to do is I'm going to create a routine. It can run locally, which means it's going to run on my computer, which means I got to keep the laptop screen open. But don't worry, I do that anyways. Or it can work on the cloud. I'm just going to have it run locally and I'm going to call it Daily Aging PR Review. And so if you can guess from the description, this is going to be a loop or a routine that looks for open PRs that have been open more than 12 hours and babysits them till they are ready for merge or alerts the team about aging. This episode is brought to you by Runway, A new kind of creative platform that has everything you need to generate any image, video or piece of content you want all in one place. With Runway, it's now possible to go from initial idea to a finished deliverable in a matter of minutes. From turning low fidelity product shots into campaign ready imagery, all the way through putting together big brand films, Runway can help your team scale your creative ambitions while keeping your budgets and timelines from doing the same. Runway brings together the world's most advanced AI models, which is why enterprises like Microsoft, Robinhood, Amazon and Adobe, along with studios like Lionsgate And Legendary all use Runway to ship real work every day. Try it yourself@runwayml.com How I AI promo code How I AI okay, so the problem I'm having is that we ship a lot of PRs because we do agents everywhere and loops everywhere. And then I kind of get bored of babysitting them and we forget to merge them in a timely basis. And so we probably have like, I don't know, 40 PRs that we need to go through. And the ones that I'm most worried about are the ones that like we kind of moved on from, from and are letting age. And so I'm going to give it some instructions and I am going to say look at the prs open on the chat Purity app app. If There are any PRs open more than 12 hours, please review their merge readiness. If there is anything that you can babysit, spin up a thread to babysit that PR until all merge checks are green. Otherwise send a slack in the product channel to the team about the open PRs that are ready for approval and merge. Be mean when you send the slack. Okay, so I am putting in that, those instructions and I'm going to say I want this to run, let's see, daily at 9:00am that's fine. Actually, let's have it daily at 10:15am because it's about to be 10:15. Okay. And then I am going to have it work in my chat PRD branch on base is fine and I'm going to create that automation. Now there's a loop and a couple things I want to call out about this loop. It happens every day at 10:15, so it happens on a schedule. I don't have to come in and say what PRs do I need to review. And it's going to tell me the next time it's going to run. And then one thing that I want to call out is you remember I said your agent can have agents. I called out here that if There are any PRs that need to be babysat, you can spin up a thread to babysit that PR until all merge checks are green. So not all the work has to happen in the one master thread. It can actually kick off sub agents or other threads to watch the work. And so I'm going to go ahead and not wait the four minutes and run this now. Okay, so once this is kicked off, yes, it's going to prompt with that original prompt that I put in the routine or automation, but then it's going to be pretty autonomous and work itself. And you know, I'VE given it basically two outcomes it needs to go after. It needs to identify anything over 12 hours that it can watch and actually monitor and make sure all the merge checks are green itself. That's success criteria one. And then success criteria two is it would use our Slack connector to send us a message. Again, I'm not going to make you watch this, but you can see here it's going to work all by itself. I am not going to have to monitor it. And all I'm going to get at the end of the day is a good set of PRs that are ready to merge and some mean messages about how we're ignoring good PRs and not putting awesome product in the hands of our customers. So again, I want to demystify what a loop is. It is just something that happens on a schedule. Now this is a very simple loop and it has access to a bunch of connectors, it has access to GitHub, it's going to have access to Slack that's already set up. So I feel like this agent or this like pseudo employee with a job is well set up to be successful. But this is a perfect use case for a loop and a very, very simple time based one. Okay, and it says no Slack MCP surfaced. I'm going to make sure that Slack is turned on. There we go. Now it should be fine. Okay, now let's talk about a more advanced loop. So I wrote that one in Claude code. It is a scheduled routine. It is pretty simple. But I'm going to also pull up Codex and show you another loop that I think is really interesting that's a little bit more complicated and a little bit more technical. Before I go into writing a more complicated loop, I wanted to call out some of the things that I like in Codex when you're thinking about or learning how to write loops. So in cloud code they're called routines. In Codex they're called automations. And what I like about what Codex has done is they have these templates and so they actually have given you a couple good ideas of quote unquote loops, automations routines that you can run. So if you're looking for inspiration, I would really look at these automation templates and I'm actually going to use one. I think it's this from recent PRs and reviews suggest next skills to deepen. And so this is sort of a meta tool that I'm going to use which is look at all the code we shipped, look at all the code commits and comments and then come up with skills that our coding team, including agents, could use to deepen the work. And so I'm going to select that one. It's going to happen Fridays at 10am it's going to happen weekly. I think weekly is right. Again, you want enough data in these loops for it to do a good long job. But I'm going to give it a little bit more information. So the prompt is out of the box from recent PRs and reviews. Suggest next skills to deepen grounding rules. Anchor each suggestion to concrete evidence. Avoid generic advice. Make each recommendation actionable and specific. I'm actually going to be more specific. If we have developed any tools for agents or developers to automatically validate their work, ensure that we have a skill for those tools. Specifically, command line tools or mcps, where an agent or a software engineer can run a test suite or smoke test against a specific use case, are very important to build skills around. I'm going to add one more thing to the skill just to show the power of subagents and automations. If you identify a skill, spin up its own thread and use that skill validated against the base branch of the repo. We want to confirm that the skill actually works and outputs high quality. Okay, so this is like a loop with sub agents that is probably going to generate its own loop. I'm actually going to force it to generate its own loop by saying you should use a goal when validating the skill. So when you prompt the sub agent, make sure you prompt it with a very specific goal it can use to validate against. You know, basically when you write a loop or a goal or an agent, you just say, validate loop goal, validate loop goal and you're good to go. But this basic prompt is saying, okay, every Friday I want you to look at all the code I merged. I want you to identify skills that are missing. There are specific types of skills that I think are very important, which is skills to use some of the internal tools we've developed. If you see a new skill, I want you to spin up a sub thread, another chat, I want you to validate that skill with a goal loop. So not only are we setting a loop at the schedule basis, we are setting up sub agents to work on specific things and then we're using a goal in those sub agents, which is a different type of loop, to validate the work. So this is like a very meta task, but I think one that illustrates the power of loop based prompting. It doesn't just have to be on a schedule, it can be on a schedule set up a team that does work on a schedule or on a schedule, set up a team that does work on a loop until it's done. And so I'm going to go ahead and create that and then again I'm going to just run this now and we're going to see here that this agent is going to spin up, the automation is going to start. One of the things that Codex does that's kind of interesting is it sets up its own memory. So you can see here a little bit of the scaffolding of what an automation looks like. And then it just gives its own prompt. Now again, it's going to go ahead and search the code, run its own commands, it's going to look at GitHub and it's hopefully going to create those new skills. And then what I ideally we're going to see in the left hand side in these all chats is new threads being kicked off to test the skills that it's identified it needs to run. And so it found one strong automation candidate. Let's see if it actually kicks off a thread to validate it. Okay, so it did, it identified a chat smoke CLI skill. Basically this is a command line tool I built to sort of test chats without having to use the UI in chat prd. And it basically spawned a dedicated sub agent to test the skill with a goal to test it against the base branch and tell us whether its instructions actually hold up in practice. So look, it spun up this agent, you can see agent, it's got a little key name and it's given it a goal. So you can see here, it's pursuing this goal which is validate the local repo chat Smoke CLI skill on the base branch. And it's basically a loop until that validation is done. So what we're going to see is more and more sub agents being kicked off. You can click them here by clicking this little dropdown. So I see Gauss, which is working on my Smoke CLI skill. And then let's see, Galileo is working on a different skill. It's working on the GitHub address comment skill. So it's basically like a babysitter PR skill. And so this automation that I've set up happens on a Friday. It's going to look at our repo, it's going to create skills and then it's going to create sub agents that are on again a goal which is a type of loop to validate that those skill works. And it's just going to do it over and over Again, until it has done as many skills as it thinks is appropriate for the last week. And so that is like my mega loop that actually I, I did not think to do until live on this episode. And it's going to be really useful for me on a regular basis. So I'm going to let that run. But before I get you out of here, I just want to talk about a couple warning signals around loops. This is amazing. We all want our agents to work for us on a schedule whenever we want. Doing work that we don't want to do. It's great. What are some of the problems? One, Loops can get expensive. So I just kicked off an automation that happens on a regular basis. It does wide ranging work, it decides when to spin off sub agents and it does loop based validation which means it's burning tokens until it hits a threshold that it decides is successful. If you do not write that loop well or your validation criteria is too thin, guess what? Your agent is going to burn tokens. I think we've seen this with OpenClaw in particular or some of these like agent harnesses is they're really good at loops, they're very diligent, they get interesting work done, but man do they love to burn tokens. And so I think these loops are a great way to spend money. So just be thoughtful about where you apply it and then make sure you're monitoring it for both cost and efficiency of the setup. So I think that's thing number one. Thing number two is I wrote actually pretty, pretty poor loop prompts. I would not recommend people follow my prompting. It did fine. But I think loops, especially goal based loops, are ones where writing the prompt really precisely is super, super important. OpenAI has a great guide to writing goals for Codex. I use that all the time. And in fact what I often have codecs do is is write its own goals. Loop based prompting is just its own thing. Goal based prompting in specific is just its own thing because you have to be very precise about evaluation and success criteria. If you are not, you will be very disappointed and use a lot of tokens for not a lot of output. So again, I would be much more careful with loop based prompting than I would be prompts or conversations where I am actually monitoring it myself. Other than that, I think there are lots of places where you can use loops. I gave some examples that are really more about product and engineering, but you can use goals and loops for all sorts of things. We talked about the morning brief. That's a scheduled loop. I used a goal loop in another episode to clean out my Gmail inbox. That's another great example. You can have agents that prompt themselves to do effective research and spawn off sub agents if a specific topic seems interesting to you personally or your business. And so I just think there are tons of ways for you to think about how could I put my little agent worker on a schedule? Or how could I give my little diligent agent a job that can be done and validated against a goal, and then how can I leave it alone until that work's done? That is a loop. That is my summary. I cannot wait to hear in the comments what you're using looping for, or if you think this is just totally overbuilt stuff that's wasting tokens. I've found it useful, but I'd love to hear what you think. Thanks for joining. How I AI thanks so much for watching. If you enjoyed this show, please like and subscribe here on YouTube or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review which will help others find the show. You can see all our episodes and learn more about the show@howiaipod.com See you next time.