Context engineering with Dex Horthy

Summary

---

Podcast Summary: The Pragmatic Engineer – “Context Engineering with Dex Horthy”

Host: Gergely Orosz
Guest: Dex Horthy (Founder of HumanLayer)
Date: July 15, 2026

---

Episode Overview

This episode dives into the concept of context engineering—a crucial but often misunderstood practice in building reliable AI-driven software. Dex Horthy, founder of HumanLayer and early proponent of context engineering, shares two years’ worth of insights from experimenting with AI agents, building “lights off” software factories, and engaging with hundreds of AI engineers. The conversation spans the evolution of software factories from their 1968 origins to modern agent-driven workflows, the physics of LLM context windows (“smart” and “dumb” zones), loop engineering, managing codebase slop, spec-driven development, and the role of human judgment in the AI software future.

---

Dex’s Journey Into Tech and Early Influences

• Physics to Programming: Dex’s early academic path was in physics, but he preferred practical problem-solving over academia.

  • “As software engineers, we’re lazy. We want to do the most high leverage thing that makes our job easier.” (05:49)

• First Real-World Project: At 17, writing a pathfinding algorithm for lunar rovers with NASA’s Jet Propulsion Lab unlocked his love for programming and problem abstraction.

  • Led to professional stints at Sprout Social (platform engineering), Aspiration (acting CTO), and Replicator, with a steady focus on building software factories and platform infra.

---

Defining Software Factories: Past and Present

[Approx. 45:49–51:56]

The Pre-AI Factory

• Origins at a 1968 NATO conference; the concept likened software development to an assembly line with defined stages: planning, coding, code review, CI, testing, deployment, user feedback, and support loops.
• Continuous automation was pursued to free engineers from manual, repetitive tasks (e.g., Chef, Nagios, CI/CD, DevOps wave).

The Agentic Factory

• With AI, software factories evolve as agents automate more stages—writing code, reviewing PRs, triaging support tickets, and even integrating user feedback loops without manual involvement.
• Bottlenecks move from manual labor to code review and verifying agent output quality.
• Fully automated “dark factories” are feasible, but not without severe trade-offs.

“At the core of a software factory, you have a source of work… Pre-AI, people would take tickets and build them. Now, you take that person and you replace it with an agent.” (49:10)
“If you remove humans from code review, you can push more tokens through the system.” (74:12)

---

Context Engineering: What, Why, and How

[19:38–24:27]

• Definition: Context engineering is the discipline of carefully composing the inputs (context) fed into LLMs/AI agents—spanning system prompts, retrieval-augmented generation (RAG), memory/history, and business data—to boost output quality, traceability, and reliability.

  • “At the end of the day, they’re all just different ways to pass tokens into a model and ask it to produce usually some structured output.” (20:26)
  • “Owning your context window” emerged as a core principle—maximizing leverage from what an LLM can actually “see” and act on.

• Why Now: Larger context windows in LLMs surfaced the need to understand and manage what data and instructions matter. Good context management enables higher-quality code/products.

• Engineering Trade-offs: Early focus should be on using the best available model. Only after value is proven do you “context engineer” for scale and cost by splitting workloads, optimizing prompts, and introducing tiers (powerful models vs. cheaper ones per subtask).

“Your job as an engineer is … what tokens do I need to put in to maximize the chance that the tokens out are going to be good.” (14:20)

---

The Physics and Limits of Context Windows

[27:25–31:58]

• Smart vs. Dumb Zones: Attention is quadratic in transformers. Only the first (say) 100–200k tokens of a million-token window are truly “smart”—later content is poorly attended, leading to rapidly degrading model performance.

  • “The longer context windows are good… But at the end of the day, the intelligence of the model drives its ability to attend to tokens…” (28:06)

• Cost and Latency: More context = higher compute and slower response; context bloat introduces noise and can confuse the agent, making compaction and distillation essential.

---

Loop Engineering – From Prompting to Autonomous Iteration

[31:58–36:29]

• Early inspiration: Ralph Wiggum technique (continuous, hands-off AI loops verifying own code).

• Loops are best when the task can be automatically and deterministically verified (compilers, tests, linters).

• “Slow loops” – Dex’s team runs nightly agentic jobs that address one issue at a time (e.g., code linting), opening small, reviewable PRs.

  • “We have a cron job. The structure of the loop is really easy—run this linter, fix one thing, commit and push.” (36:18)
  • Avoid flooding humans with unreviewable, massive code changes.

• Risks of Loop-Driven Automation:

  • Quality degrades if humans stop reading code entirely; autonomy at scale accumulates architectural “slop” that’s hard to remediate.
  • Dex’s own “lights off” factory had to be shut down after 3–6 months: “By November, we had shut it down… It’s easier to start over than to fix it.” (40:46)

---

Spec-Driven Development, RPI, and Compression

[60:14–65:56]

• Research-Plan-Implement (RPI):

  • Research: LLMs read and summarize codebase state into a doc.
  • Plan: Human/agent plans implementation in detail for reasoning consistency.
  • Implement: LLM produces code from plan, with PR for review.

• Pitfalls:

  • Early plans were too granular, making reviews double the work.
  • Keeping specs/plans/code in sync is rarely worth the effort, even with AI help; always treat code as the “single source of truth.”

• Intentional (Frequent) Compaction:

  • “Frequent intentional compaction is the building block … Which means doing as much work as possible in the smart zone.” (65:56)
  • Collapse sprawling context into verified, brief artifacts (Markdown summaries, design docs) between sessions—compact, verify, start fresh (“smart zone”).
  • Frequent resets help LLMs stay on track.

---

Harness Engineering

[24:58–27:25]

• Parallel to context engineering, harness engineering is optimizing both the “inner harness” (tooling/integration points LLMs interact with) and “outer harness” (your custom dev environment).
  • Definition is still fluid; clarity is evolving via community discussion.

---

Key Observations and Cautions

• AI Slop:

  • “Slop in, slop out.” If you outsource all architectural and design thinking to agents (“lights off factory”), you’ll quickly reach a state where rewriting is easier than fixing:
  • “If you stop reading the code, you have about three to six months before your code base becomes easier to rewrite than to fix.” (91:04, Host summary)

• Token Harder vs. Token Smarter:

  • Token Harder: Max out LLM usage to generate output as fast as possible—shortsighted, unsustainable.
  • Token Smarter: Seek leverage by mixing human judgment, careful checkpoints, and “lights on” reviews.

• Spec Drift:

  • Attempts to keep specs/design docs totally in sync with code run into diminishing returns; best to create tactical docs, use them during the current build, then discard.

• Dark Factories:

  • Fully automated, zero-human-in-the-loop software factories (dark factories) tempt with productivity but fail the test of maintainability and reliability—leading to catastrophes months down the line.

---

Dex’s Recommended Engineering Practices and Principles

12 Factor Agents (17:23)
Highlighted principles for reliable AI-driven engineering:

• Own your prompts and context window
• Treat tool calls as structured outputs
• Compact errors and context frequently
• Use small, focused agents (triggered anywhere)
• Agents as stateless reducers (or “transducers”)
• Maintain the right human checkpoints, especially at planning and review

Memorable Quotes & Timestamps

• On context engineering:

  “The only way you can impact the quality of your output from AI is by caring a lot about what the inputs and crafting them.” (18:03)

• On slop:

  “If you outsource your thinking, you’re going to get garbage.” (78:08)

• On the still-unsolved challenge:

  “The cost function of bad architecture and bad program design can’t be evaluated by running the unit test… it hits you three to six months later.” (40:46)

---

Where Next? The Role of Human Layer and the Future IDE

[80:24–84:36]

• HumanLayer is Dex’s company, building a collaborative, cloud-first “AI IDE” aimed at experienced engineers working on complex, high-stakes systems.
• The tool emphasizes agent management, real-time collaboration, and in-context design/spec refinement—possibly challenging the centrality of pull requests in the engineering loop.
  • “The IDE of the future needs to be rethought from the ground up for agents.” (81:31)

---

The Human Factor in Hiring & Networks

[86:39–89:02]

• SF/Silicon Valley brings unique critical mass and “pay-it-forward” tech community—a high density of like-minded, ambitious problem-solvers.
• Fundamental CS skills matter most in hiring; LLM/AI engineering intuition can be learned much faster than foundational engineering.

---

Parting Wisdom & Book Recommendation

[90:22–90:56]

• Classic texts (Refactoring by Martin Fowler, Clean Code, Pragmatic Programmer) continue to matter—arguably more than ever—in the age of AI-driven engineering.

---

Notable Timestamps and Topics

| Timestamp | Topic | |---|---| | 03:01 | Dex’s early days: Physics to programming, NASA rover project | | 05:49 | Importance of developer experience/platform engineering | | 14:20 | “12 Factor Agents Manifesto” & key principles | | 19:38 | Origin and definition of “context engineering” | | 24:58 | Harness engineering explained | | 27:25 | Physics of context windows: smart vs. dumb zones | | 31:58 | Loop engineering: Ralph Wiggum, backpressure, and slow loops | | 40:46 | The “lights off” factory fails—codebase maintainability crisis | | 45:49 | Software factories: Origin, evolution, and today’s AI architecture | | 60:14 | Research-Plan-Implement (RPI) and its lessons | | 65:56 | Frequent intentional compaction—keeping agents in the smart zone | | 74:09 | “Dark factories”—risks of zero-human software factories | | 78:08 | “Slop in, slop out”: Quality and cognitive leverage in agentic teams | | 80:24 | HumanLayer and rethinking the AI IDE | | 90:22 | Closing book rec: Refactoring by Martin Fowler |

---

Final Thought

This episode is a goldmine for software engineers and leaders grappling with the shift to agent-driven workflows. While AI accelerates the loop, the critical leverage remains: smart engineering judgment, surgical use of checkpoints, and a relentless focus on maintainability.

“Context engineering has been so long lived because it’s grounded in the fundamentals of how transformer attention works… Context engineering will be interesting and important to anyone building on AI.” (27:25, Dex Horthy)

---

[For more deep dives, visit newsletter.pragmaticengineer.com]

Transcript

[Host]

What happens when you let AI agents ship code for months and no developer reads a single line? Today's guest tried exactly that. He built a lights off software factory and four months later he had no choice but to shut it down as things just stopped working. Dex Horthy is the founder of humanlayer and the person who coined the term context engineering days before Andrej Karpathy and Toby Ludka made it famous. He spent the last two years talking to hundreds of AI engineers about what actually works when you build with LLMs and is testing the most extreme ideas his own team in today's conversation we discuss context engineering, what it is and the physics of context windows, including what the dump zone is. Loop engineering from the RALPH W technique to the slow loop that Dex's team runs every night to wake up to Code cleanup prs, the rise of software factories from a NATO conference in 1968 through DevOps to today's agentic factories, Spec driven development and why specs always drift from the code itself and many more items if you want to understand increasingly important concepts like concept engineering and harness engineering, or want to know how far you can push the let agents build everything idea from someone who pushed it further than almost anyone, then this episode is for you. This episode is presented by antithesis. If you work with agents, your job is no longer just writing code, it's specifying and testing it. And antithesis is the most effective method of verifying agentic code today. Today's episode is brought to you by Buildkite, the CI orchestration platform trusted by OpenAI, Anthropic, Cursor, Nvidia, Uber, Canva, and more. Today we're talking about push the right context into models so that they write better code. Right after that starts working, your agents will write more code a lot more trusting. That code avalanche is where many teams face a challenge today. Every change that an agent makes still has to be built, tested and proven safe before it ships. Worked on my machine is not enough, so you obviously need CI. But when agents are pushing 5, 10 or 50 times the commit volume to your pipelines, faster CI runners won't save you. Shaving 30 seconds off a single build is meaningless when a queue is 100 jobs deep. What you really want is a CI system that gets faster as the volume grows and CI that offers instant parallelization to give you unlimited concurrency and to intelligently route changes at runtime. This is what buildkite does and why global software leaders continue to rely on it. The same architecture that observed the scale of Shopify And Uber a decade ago now runs about 1.4 billion job minutes a week across Cursor, Meta, Reddit and Snowflake. While the rest of the CI world are cracking under the weight of re architecting their platform build, Kite continues to reliably grow. Agents running on your infrastructure or buildkite. Any cloud, any chip, your secrets, your scale, every artifact and log is captured. So when something fails, either you or your agents have immediate insight for why. As you're entering the context you'll give to your agents, think about how you'll verify what they hand back. If your system is buckling under the increased volume, head to buildkite.compragmatic 30 day all access trial, no credit card and an actual human NGO on standby. His name's Ola and he's very helpful. So Dex, welcome to the podcast.

[Dex Horthy]

Super stoked to be here, dude.

[Interviewer]

Before we get into some of the context engineering and some of some of the more spicy stuff as well, how did you get into tech? How did you fall in love with computers?

[Dex Horthy]

Oh man. So I was doing undergrad as a physics major and I realized that I didn't like academia. And there's basically two or three paths out of physics is basically you go get a PhD or you go into finance or you go do programming. At that time this was, you know, 2012, 2011, when it was like in the middle of undergrad and deciding what to do. And I had done an internship when I was in high school. I was working with NASA researchers to a jet propulsion lab in California. They had just gotten this really high fidelity, like the most, you know, fine grained data set of altitudes, like the heights of very, at very like topographical map of the south pole of the moon. And the south pole of the moon is really interesting because some of the craters there are so deep. Because of the angle it has, it got hit by meteor storms like no other part of the moon. So there's very deep craters that have never seen sunlight. And so there's frozen liquid water in there from the formation of the moon. And so scientists were really interested in getting down there and exploring. And so we had this really fine grained map and it was like, okay, cool, let's build software so that I have point A to point B. I know the limitations of my rover can, you know, max incline up is this max incline down is that find a path from point A to point B that doesn't like break those rules of the incline. So I was 17, I had never cracked a CS textbook. So I basically wrote a really naive, bad version of Dijkstra's algorithm for pathfinding. So I was in college, I was like, I don't know if I want to do the academics thing, but I really enjoyed programming back in the day. And so I decided to go. I got like half of a CS minor and then started working on a API platform team at a software company in Chicago.

[Interviewer]

And Sprout Social, right? Yes.

[Dex Horthy]

And basically never went back.

[Interviewer]

Yeah. And then where did you go from there? Where did you pick up, like, the parts of the trade? Because very early on, your first job, that's not really common. You were doing platform engineering back in, you know, more than a decade ago.

[Dex Horthy]

From that point, it took me about two or three months to notice that, like, the most valuable work that was being done in the company was being done by, like, of course, it's obvious. Like, the first couple engineers who know everything and understand where everything was. And like, you spend a day on a support ticket from a customer and they solve it in five minutes. But, like, you have to solve it, so you learn and whatever. And I realized, like, the most valuable people in the company were the people that were building the developer platform. Cicd, sandbox environments, preview stuff. And so I kind of like, that was my first step into the journey. And I've basically been obsessed with software factories since that, like, three or six months into my first job.

[Interviewer]

We talk about software factories now, but you're, you're talking about software factories back then. So, like, you were starting to already think that this is how we can produce better software inside. This is pre AI world. Right.

[Dex Horthy]

Well, and I'm always surprised, like, there's a huge class of developers that say, I don't want to work on cicd. I hate cicd. I'm like, really? Because building the thing that builds the thing and building the thing that builds the thing that builds the thing is like, as software engineers, we're lazy. We want to do the most high leverage thing that makes our job easier. So how do we. If we can build a thing that helps us build a thing that helps us move faster, then that's the best use of my time as a lazy engineer.

[Interviewer]

And then you went to another startup. As aspiration.

[Dex Horthy]

Aspiration, yeah, aspiration.

[Interviewer]

Also platform engineering.

[Dex Horthy]

Yeah. I was brought in and then like three, three months into the job, the VP of engineering who hired me quit or got fired. I don't know. There was some drama about it. I probably shouldn't talk about it. And then I was there for about a year and was kind of like acting CTO for a while, like, hired a couple of people, helped hire the new VP of engineering, but I was out of there. I don't think I'll ever do consult consumer again. I think I'm actually a B2B guy.

[Interviewer]

Good to know. And then you went to Replicator, where you spent like a good, solid, like four years and went from engineer for the deployed engineer to product manager.

[Dex Horthy]

Yeah, I did core engineering for, like two years. We were building a container orchestrator. Like before Kubernetes, before Docker, Swarm was really a thing. We built our own Orchestrator. The founders had this vision that, like, oh, Docker is going to make it much easier to ship on PREM software. And when I say on prem, I don't mean literally like a rack in a colo. It's more like, hey, look, bring the app to where the data is rather than sending the data up to some cloud vendor. And Docker makes it much more, much easier to package up apps and move them around. And so they had this thesis that, like, basically you could build a platform that the experience that you get when you use GitHub Enterprise, which is like, you install it and it has this admin panel, but then you just get GitHub running in your data center, and your code never has to leave your data center. Suddenly you could build a generic SaaS where everybody could have that. So I did two years an engineer there, and then our head of sales, we parted ways with our head of sales. And honestly, I was having a lot of arguments about the software factory with our cto. And it's kind of like almost like a too many cooks in the kitchen kind of thing. I'm sure many listeners. Listeners have had this experience of like, well, yeah, I know I have these tickets to build, but, like, CI sucks. I gotta fix CI because it's too slow or it's like there's too many different builds and it's always breaking. I was like, I'm gonna fix that and then I'm gonna do that. And it's just like, dex, I need you to stop fixing the build pipeline and, like, do the tickets I gave you. I'm sure you've had this experience, perhaps. Yeah. And then.

[Interviewer]

And was this what led you to either forward deployed engineering?

[Dex Horthy]

Yeah. So I, like, I really loved our customers. Our customer. Our customers are Hashicorp, Datastax, Puppet, all these really cool engineering brands. Travis, CI, Circle, CI. I was like, yeah, I actually love working with our customers. Our customers are awesome. And it was a great Way to like get in the trenches. A lot of really good engineers who were solving the hardest problem at the company, which is like how do we take this three to five year old SaaS platform and package it all up so that someone who knows nothing about our architecture can run it reliably in their own AWS vpc, in their own on prem data center, whatever it was. And so I spent, I was our first kind of customer facing engineer and it was about three months I we close. I met with like every company customer that was like kind of in the pipeline but wasn't moving sales wise. And we closed like 12 deals in three months. And the CEO was like, holy crap Dex. Like the, the investors are taking my calls again. Like I don't, I know you want to get back to coding but like I need you to go hire three people and like build this team out because I think you might have been like born for this.

[Interviewer]

Wow.

[Dex Horthy]

Yeah, so I did that for about four years. Built that org to like 25 people. And then Zurb happened and it got a lot smaller and we kind of realized like, hey, we have a product that's like pretty good. And we've been solving what lots of early startups do is like, okay, there's some usability issues. We'll throw, we'll get a bunch of smart people, throw them in the trenches with our customers. Great for sales, great for retention, all this stuff. And it was like, oh, we actually like the margins on that aren't, aren't good enough. And so we basically were like cool. We actually just need to make the product way more usable, do a more PLG shaped thing.

[Interviewer]

PLG means product LED growth.

[Dex Horthy]

Product LED growth. Make a little more self service so you don't need an expert to teach you how to use it. I was like, cool. If that's the most important thing then I want to go be a product manager because I have tons of opinions. I've now spent four years in the trenches with our customers. I have a laundry list of roadmap things that I think would make the product way easier to use and adopt and implement and deploy.

[Interviewer]

And now you went the full arc. You went towards the dark side.

[Dex Horthy]

Exactly. Yeah I did. I was like, this is going to kill my street cred, isn't it? But I was really glad. You know, I think a lot of engineers are afraid that if they go do a customer facing thing they lose all their credibility. And like, yes, I wasn't coding for 10 hours a day. I was coding for like three or four hours on a Saturday for fun. Not. But I mean we were helping people build YAML, we were building clis. We owned a lot of the tooling that customers use, but it was like the last mile delivery side of it, not the core platform. And like, on a more personal note, I had spent the last like most of my 20s feeling like, okay, a little bit introverted, a little bit like socially awkward. What a lot of engineers, I'm sure, experience. And I had talked to, my uncle's a music producer, so he used to work with like Randy Newman and a bunch of like really famous musicians. Oh, wow. Yeah, this guy, Mitchell Froome and he. I was sitting with dinner with him at some point and when I was, I think it was when I was still in undergrad, but he gave me this lecture. He was basically like, if you want to be really good at something, you have to make it the only thing you do. The guy playing guitar nights and weekends trying to get his band off the ground will probably never achieve greatness. The people who become great are the people who basically make it. Like, if I don't play guitar, I don't eat. And you go and you sit on the street all day and you play for 14 hours a day or whatever it is. That's the only way to become great. So I said okay. Instead of trying to like read self help books about how to be less introverted and less socially awkward, like, what if I just made it my fricking job to just talk to people and make friends and like help people and solve their problems? And I think it worked out. I recommend it. I think everyone should spend a year or two at least doing something really like customer facing.

[Interviewer]

Did you do this because you felt that it was holding you back, being introverted or like, what, what, what? And I know you got the motivation from the whole musician motivation. I get it on one part. But what was it that you said, like, is the customer facing thing that I'm going to be doing it? Because clearly you were pretty great at like writing code by that point. You could argue you were doing it night and day. So where, where did you find that? Like, I actually, I think like customer facing or like getting this introvert off of me. Did you feel that I was holding you back or you just wanted to be good at it?

[Dex Horthy]

It was just kind of a thing that was like interfering with my like general life satisfaction. And it was also like, I'm not a very type a person. I'm very disorganized. I don't know people Call it like, okay, I'm like ADHD now. That's why I can run 30 clots in parallel or whatever it is. But I was like, I was really bad at email and calendars and spreadsheets. I just like, didn't care about these. Didn'. And so, like, another side effect of this was like, it just forced me to be organized and keep a lot of things going. And so, like, I don't know, there's like, weird benefits you get from like, stepping outside your comfort zone and learning, like industrial disciplines that are separate from what you've been doing. And so the opportunity presented itself and I was like, oh, I like working. I'll try this for a little bit. Started going really well. I'm like, cool, let's keep, let's see, let's see how far this thread goes.

[Interviewer]

And then afterwards, you're now in your second startup. You, you became a founder and you also got involved in an AI pretty early as, as it was even before it was so obvious that it would change how it would change how we develop software. Right?

[Dex Horthy]

Well, I would say I was, I was later than I could have been because we started the company, me and a buddy in Chicago started a company in the data engineering space in about 2020, November 20th. We decided in like August of 2020.

[Interviewer]

This is Metalytics.

[Dex Horthy]

Metalytics technically still the same company as Human Layer. We just like pivoted the, the, the mission. But yeah, the advice I got from every angel investor that, you know, people who just knew ctos I'd worked for before and stuff, they were just like, look, hitting a lot of heads, wins. I don't know if you know, like the whole DBT data engineering 5 tran, that whole arc where it was like this huge party and tons of investor money going into all these different companies. And then within, by like 2021, 2022, there was kind of the ZIRP thing and just this general realization that the TAM for those sorts of tools is not as big as Total addressable market. Yes, the total addressable market for those sort of tools was, was not as big, quite as big as, as we all thought it was. So it was, it was a hard place to raise money. It was a hard place to get customers.

[Interviewer]

Yeah. And then I met you at, while you were at Human Layer NSF at an event. You actually talked and we chatted afterwards. But by, by that, this was about a year ago, you were already. You started to have some really strong opinions on using AI. And one of them was this now famous 12 factor agents manifesto is.

[Dex Horthy]

Are we calling it a manifesto now?

[Interviewer]

I'm calling it a manifesto. It's a manifesto. I'm calling it. Let's talk about this. This was 12 engineering principles to build reliable production ready apps. How did you come up with this? And maybe we can also talk about some of them.

[Dex Horthy]

Yeah, so I'll, I'll kind of like go to like around August, the co founder I was working with the kind of burned out and left and it was very, we were on good terms, it was very mutual and I decided to just start messing with AI stuff and I was building AI agents and what was really in vogue right then was like the LangChain, the Crewai, these like agent frameworks. And it seemed like there was a ton of you go on, you go in the crewai Discord, there's 10,000 people. It's like, okay, this feels like the right shape and there's clearly this eco. You go in every single one of those projects, they have a Chroma DB plugin, they have like a Composio plugin. There's like clearly like this is the, this is the shared interface that everybody is building for. I said, okay, what's missing from all of this? The agents can call tools, but it's really hard to like control which tools they call. And if it's a chat bot, obviously you can show approved deny in, in the UI of your application. But I kind of was obsessed with what I would call like outer loop agents or proactive agents that would run in the background, get triggered by events. I mean openclaw is basically like the biggest manifestation of this of like you have a heartbeat, it wakes up, it sees if there's any work to do, it tries to do stuff. And my thought was like, I'm not going to trust that agent to do anything meaningful if I can't get like a slack message or an imessage or something. When it wants to do something and kind of guarantee deterministically that I can approve or deny that or deny it with feedback and say actually no, do it like this. So we played in that space for a while and talked to a lot of founders and founding engineers and builders. We came and did YC in the fall of 2024 with this idea. We're building out this API platform and it was sort of like pager duty, but like it wasn't who's on call to fix the servers, it was like who's on call to this like routing mechanism for like who needs to approve this agent and can they like escalate it or delegate it or defer it, all this stuff. And we built it for this ecosystem career AI linkchain fi. There's so many grip tape. There was so many in that, in that time. And then I talked to tons of AI engineers who were actually building really interesting things and like actually making money doing six figure contracts shipping AI to the enterprise. And all of them had tried that stuff for like a month or two and then they had thrown it out and they were just writing all the API calls by hand and they were building more things that look more like pipelines and workflows than these sort of like hands off call tools in a loop kind of thing. And so I talked to a hundred people and I spent a lot of time, a lot, a lot of time hanging out with one of my best friends, vaibhav from Boundary. So they build a programming, they built this proto buffs for AI thing and I think they're about to launch their full fat programming language, Turing complete thing. But he had this way of thinking about agents and building with models and building with inference where it was a lot more about understanding what structured output really is under the hood. And every single step in your AI workflow is just tokens in, tokens out. And your job as an engineer is figure out, okay, what tokens do I need to put in to maximize the chance that the tokens out are going to be good. And kind of distilled all these ideas into about 12 principles and wrote about it on GitHub. Posted just like this like 12 page GitHub repo threw it on Hacker News, got like 500, it was on the front page for like 2 days and I think it really resonated with a lot of people.

[Host]

Yeah.

[Interviewer]

So I'll just quickly read the 12 principles and then let's talk about like one or two that resonate. So the 12 are natural language to tool calls. Own your prompts, own your context window. Tools are just structured outputs. Unify execution state and business state. Launch, pause, resume with simple APIs, contact humans with tool calls. Own your control flow, compact errors into context window. Small focused agents trigger from anywhere, meet user where they are. Make your agent a stateless reducer.

[Dex Horthy]

Haha. The stateless reducer. Yeah, the stateless reducer one was a little. Actually someone hit me up on Twitter and corrected me. It's actually, it's actually a transducer because there's technically multiple steps in the workflow.

[Interviewer]

But there we go. But, but, but of this one, this, this was a Year ago. So like which is like forever and, and, and how the tooling is evolving. Which ones still stick with you or if you're like, all right, these were good. That that still seemed to hold off.

[Dex Horthy]

Yeah, I think I spent most of March writing it. Published this in April. And then Swix hit me up from AI Engineer and he said, hey, can you come. You want to come talk about this? So I gave this talk. 12 Factor agents in like June 6th, I think and small room. Maybe like it was packed but it was like maybe a hundred people. That was the year at AI Engineer where like the lower physically like on, on the, on the second basement floor was all the super corporate stuff and you go up a level, it's a little bit more. And then like on the top floor is all the like weird cutting edge like startup stuff that like you probably shouldn't care about yet kind of thing. So we were up there on the top of this like weird way of thinking about agents. And then about a week later, two weeks later, Toby Lucky from Shopify says, I really like this idea of like context engineering. And I'm like, I wrote about this two months ago. This is great. Toby gets it. And then a week later, Andrej Karpathy is like, well, I really like. I think what we should think about is not prompt engineering, but context engineering. I was like, yes, that's my. Anyways, I don't know if you ask Gemini depends what day it is, they will tell you. Either me or Toby or Andre came up with context engineering. You can't really own a word. Like I don't. No one remembers who invented the word prompt engineering. But of all the factors, factor 3 of own your context window. And basically the only way you can, whether it's agentic or a single step at a pipeline, the only way you can impact the quality of your output from AI is by caring a lot about what the inputs and crafting them.

[Interviewer]

So let's talk about context engineering, which I am going to credit you that you coined it. I did some research and like I think you were earlier but a few days. So there you go. You. You coined it. We're adding, we're adding to the. We're adding to SEO juice. We'll have it in transcript. Dex Coin Context engineering.

[Dex Horthy]

Well and, and like asterisks on that is basically like I learned about context engineering from talking to these hundred engineers and founders. I just kind of like what was the same about what they were all doing. And I put a name on it. So like I didn't invent doing it. I was just like, I think, I think there's this thing and like, vocabulary and names are really important and having, like, clean ways to talk about the problem, especially when, like, a lot of the content about AI right now is so much hype and jargon that is, like, meaningless. I was like, okay, I think there's a word here that is useful to builders that explains how they should be thinking about building their software.

[Interviewer]

So what is context engineering?

[Dex Horthy]

It's kind of like de abstracting a lot of the abstractions that have been layered on top. So you have a rag, you have memory, you have agentic history, you have structured output, you have all these things that are like different ideas in the frame of agentic programming. And at the end of the day, they're all like different ways to pass tokens into a model and ask it to produce usually some structured output. And understanding that is a lot more powerful than trying to learn memory and trying to take some agent framework off the shelf and some memory framework off the shelf. I mean, those are. These things are all really good. If you want to get to like, 80%, you want to get a really good demo. But when you have to go from 80% to 95% or 99%, you need to go down a level and think about what's everything we're putting into the context window, what order is it going in depending on which model we're doing? And all of this stuff matters. You have all of these levers that you can pull. And it just felt like the right abstraction for thinking about how do I get AI to do the thing I want as accurately as possible?

[Interviewer]

Why is context engineering started to become more talked about? It was about a year ago. Did it have to do with the. The context window that we could pass onto LLMs? Pretty much. Did it start to expand or did we just start to realize that we can do a lot more by passing on from. You know, the easiest one is, of course, system prompts. But of course, whenever you build an LLM behind the scenes, you will pass additional context as well. Not just to prompt the user. You will add a bunch of stuff. That's, I guess, a dirty secret of any LLM. But why do you think the focus is moving on to, like, all right, context is important.

[Dex Horthy]

I think it always was important. I think what had to happen is a ton of smart people, again, like all these builders I talked to a ton of smart people, had to, like, focus really hard on producing. Like, I want to make software that I can sell I want to make something that's accurate enough that I'm proud of and I can sell to an enterprise and they're going to be happy with it. And there's just like the, the, the, the easiest way to get to really high quality AI applications is by thinking at that token level. Thinking about a string of different LLM calls, like rather than just tools in the loop. And it's kind of open ended and very flexible, but not that reliable. Thinking of agents as workflows, as pipelines, as some mix between maybe a couple tools in a loop versus just hey, I have my tools and I have my model and I have my system prompt and these are the only levers I have. And it's actually, no, you have way more levers. It's going to take more work and you're going to have to like understand the LLM with a deeper intuition. But it was a thing that we always needed and it just took time for people to build with this technology to figure out that this is the layer of abstraction that allows you to break through the quality ceiling.

[Interviewer]

And how are cost and context engineering connected?

[Dex Horthy]

Yeah, I don't know. I was talking about this with someone this morning about when you're working with LLMs. One of the things I like to say is kind of like make it run, make it write, make it fast. See if the world's best lm. At the time, I think we did a podcast episode that at the time it was like, oh, three, see if O3 can solve your problem and then give it to people and see if they want that. And then if people want it and you use it a lot, then go do a bunch of context engineering. Because your engineering time is always the bottleneck. Like humans trying to figure out and solve problems and build evals and improve and try different dimensions or set up JEPA or whatever it is is always going to be more expensive than just using a smarter model until you have millions of requests a day. And then it's like, okay, we're going to do a bunch of context engineering, break this up into three calls and get it to work on GPT 4.0. And then we're going to take two of those and make those two work on GPT4.0. I'm using old model names. But the point is like for a certain task in your workflow, can you get GPT OSS120B, which is like 1/1000th of the cost of opus. Can you get it to solve parts of the problem so that the tokens and the things you're using the smartest frontier models for are just the things that you really need, that level of intelligence. But you shouldn't go build all of that and over engineer it until you've proved that you need it, that it's valuable, that it's like, okay, this is. Now I'm gonna get to Eli Goldratt. And like, what is the.

[Interviewer]

The.

[Dex Horthy]

He had this book, the Goal, right? It was about how to model your factory. And I'm sure we'll get to that when we talk about software factories. It was like, what is the bottleneck in your system? And one day it will be latency and cost. But it's probably not that. When you first start out. And context engineering is how you move from the. You add human effort to the equation to improve the efficiency, the speed, the cost efficiency of your system.

[Interviewer]

Interesting. And then one thing that came up more recently and a lot later recently is harness engineering. What is harness engineering?

[Dex Horthy]

So I made a post in like October, I think about or maybe November, of like, hey, there's this new thing that I see is like, I'm calling it harness engineering. My definition that I had at the time is not what. Actually, this guy Viv, who's at LangChain now, does a lot of really good writing on agents and how to think about harness. He had written something called harness engineering, like a couple weeks before me, but I hadn't read it at that point. And my take was basically like, okay, when you build an agent, you use context engineering. When you use an agent. Because we gave this talk in August of 2025 about, like, how to apply context engineering to how you use coding agents. And that kind of evolved into this idea of like, how do you take a harness, like Claude code, like codecs. How do you engineer against the integration points of that harness? So commands MCPS skills, how you organize your code base. How do you kind of optimize the environment that the coding agent runs in to, like, get the best results? The same way with context engineer, how do you optimize the inputs to every single prompt? Well, harness engineering just is like, how do I raise the floor so that every single turn of this thing, the results are as good as possible? And the term got super blurry. And some people think harness engineering means building a harness, and some people think harness engineering means building around a harness. I actually like what Martin Fowler came up with. As usual, he's very good at naming things. And he kind of defined the. You have the LLM and then you have the inner Harness, which is like the thing that the tool definitions and the integration points that like say like a cloud code or a codex or amp actually exposes, that's your inner harness. And then you have the outer harness, which is the stuff that you, the human do to customize that for your specific needs, your code base, your languages, etc. That's the best definition I think we have for harness engineering.

[Interviewer]

It's interesting how naming is still so important, isn't it? Well, it's like.

[Dex Horthy]

And as soon as you name anything, people are most people. I'm actually surprised that context engineering still means the same thing to most people that it did a year ago and that it's even still relevant. Like that's honestly the craziest thing to me is like you wrote how many things were written about AI 15 months ago still matter or are still interesting or are still like have good advice baked into them. Stuff changes a lot. I think context engineering has been so long lived because it's grounded in the fundamentals of how transformer attention works. And until we have post transformer models or linear attention or whatever it is, which, who knows when that's going to happen, context engineering will be interesting and important to anyone building on AI.

[Interviewer]

Can we talk about the physics of context? You, you had a tweet. This one, the context reality check. This is a graph of as you get to 1 million contexts, just the quality just drops, it goes down. What do we need to know about the context? Again, we now have models that do have a 1 million context window. Maybe we'll have even longer ones, but when you start to just put in more stuff into the context, it starts to become less efficient. What do we know so far in terms of from the practical perspective of someone who is using the context window to add on a bunch of stuff, may that be mcp, may that be tools, may that be skills, may that be all of these things?

[Dex Horthy]

Yeah, I mean, so the longer context windows are good. You can talk to it for longer, like they're doing a good job. But at the end of the day, like, especially when you had like opus, it was like Opus 4.5 and then Opus 4.51 mil or 4.6 and 4.61 mil. You're not actually getting a like smarter model. Like the intelligence of the model is, is what drives its ability to attend to all of the tokens in context window to figure out on the next turn which parts of this 100k or 200k context window are the most relevant to making the decision of like, what is the next tool we call? And doing that over and over again in a loop. So I don't know. There was some study that came out in 2025 which found that. And again, these are old models. So like inflate your numbers. But it was like Frontier. LLMs can follow about 150 to 250 instructions before it starts to drop off. Their ability to follow all the instructions just like drops off pretty quickly. And I think Lori Voss had arise. I haven't actually looked at the data, but they did a study with like the next generation models a year later and it looks like it's like much better the number of instructions you can get in. But in any case, you have like, I split context engineering into like two categories. You have like the, the most people think about like the information budget of like, okay, I can do rag and I can pull out chunks of this document. Rather than putting the entire book into my context window, I can just go grab the pages that matter. But it's also, your instruction budget is like if you give the model too many instructions and especially too many conflicting instructions, and that's in your initial prompt. And also, like, if you have a conversation, you start going down a path and then you change your mind and you start going down a different. You're like, actually, I don't want to do any of that. I want to do this. It's like, it's a lot of computation the model has to do to notice that it has to ignore that whole thing. And when both of those things are kind of far back enough in the context window that they're only half getting attended to, your likelihood that it's like actually going to like, remember the exact instructions you gave it a hundred thousand tokens ago is like, it goes down quite significantly.

[Interviewer]

This is all very interesting because as engineers, we are expected when, you know, when we're AI engineers, which now a lot of software engineers are meaning you just like use LLMs to build software. Like underneath there's an LLM layer somewhere. You're an AI engineer. Congratulations. But it sounds like the expectation is to be a good software engineer pre AI, you need to understand, you know how to write good code and it helps when you understand a little bit of the underlying. We didn't need to do that that much over time, but it never hurts. But sounds like right now we're in this phase that to be an engineer who can write an efficient AI systems that use LLMs, you need to understand the dynamics of the context. You need to understand why? Stuffing your context one way or the other can be, computer can introduce latency and all of these. It sounds like it's kind of more of an intuition and of course there's some understanding. But from talking to, you're like, well it, it, it does this computation. Like I know, you know because you've tried it out, right?

[Dex Horthy]

Like I'm not, I'm not a PhD in machine learning. Like I couldn't actually go like draw a mathematical proof of how this works. But we know attention is quadratic and the more stuff you put in, the more it has to spread this attention out over everything.

[Interviewer]

This just feels like an absolute new area and like a little bit very different to like what we're used to like software engineering, which is like pretty kind of like black and white, right? The compiles or doesn't compile. That's true.

[Dex Horthy]

I mean there's a different kind of intuition. I was talking about this earlier as well. It's like there's a different kind of intuition that you, that you develop over years as a software engineer. And there's many categories of it, but the one I'll, I'll call attention to that is like a thing that you cannot teach, you cannot do, you cannot learn in a textbook. The only way to learn it is like I know bad patterns in software because I have debugged them at three in the morning. This is my buddy Jake from Netflix said this in his talk at AI Engineer Code. It's just like there's no better way to learn what is good and what is bad and what works and what doesn't than suffering through the thing that doesn't work.

[Interviewer]

Well, speaking of suffering through the things that doesn't work, a new paradigm that is spreading up is loops loop engineering. The idea that instead of writing prompts, just write loops, set up your loops. And this all started with the Ralph Wiggum technique where it will just. Well, I guess that's an early version of loops that were just loops around. And now we're hearing with some of the biggest slabs talking about that they're actually just doing loop engineering. What is your take on, have you done some loop engineering yourself? Have you set up some loops and what do you think is good about it and what do you think is bad about it?

[Dex Horthy]

Yeah, so I think of loops as, I mean I could ramble on this for 10 minutes. This is an entire talk. But I'll try to lay out some high level stuff and then we can dig in wherever you think is most interesting. We had Ralph Wiggum. It's actually a year and four days ago was the first time I saw the Ralph Wiggum demo. Like, Jeff Huntley was just like visiting SF and he just like came through and like dropped everybody's jaws with his like, yeah, I just ran Sonnet around the clock and spent six grand in six weeks. And like I built an entire Gen Z programming language. Look at it compiles and it has a stage two compiler where the compiler for the language is written in the language itself. And all the insane. And the core lesson from all of that, I think was the idea of back pressure, which is basically, and I think a lot of people were doing this for a very, have been doing this for a long time, which is how do I let the model check its own work? How do I automate the process of getting feedback into the model? And there's lots and lots of different flavors of this. You can have deterministic linters, you can have unit tests. Like, part of what made the programming language easy to build with RALPH is a programming language can be infinitely verified. You write, you write the code in the language, you compile it. If the compiler fails, go fix the compiler, you run the program. If the program fails, you go fix the compiler. Like, it's like, it's very, very verifiable. And I think the lesson in loops engineering is like, if you can make a problem very verifiable, you can kind of like treat it like a black

[Interviewer]

box and then have that loop because it will keep improving itself because of the verification loop is already there.

[Dex Horthy]

Exactly. And so like you can do this with CICD is like, I do this every time I'm doing a release. I'm like, I'm tired. The CICD is slow. Cool. Go research the code base, make a change, make a pull request, run the test, see if it's faster, try again, run the pull, run, run the test, push, push to the branch, check again, see if it's faster. And so it's like if it can verify its own work in a loop. Instead of design, instead of saying let's try this approach or let's try that approach or suggestion and being really back and forth, you just say like, my goal is to make the CI faster. And you tell the model, here's the steps, here's the five, here's the five steps. You're going to write some code, you're going to commit it, you're going to push it, you're going to launch a sub agent to watch the job until it's finished. It's going to tell you what happened, then you're going to decide what to do next. And so that's like the very simplest example I have of like designing loops.

[Interviewer]

And you just said the goal, which is cloud code. And I think codecs have both chip goal, which is you just set the goal and it iterates until it reaches it or as long as it makes progress towards it.

[Dex Horthy]

Exactly. And so it's like if it's verifiable, if you can measure, this is auto research too. Auto research is like, hey, go make this model twice as fast. And like, it's just a prompt that tells the model, like, go to it over and over again and try things until it actually has good results. So that's what I think of loops engineering. I don't know. We, we do a very interesting kind of loops engineering where like the, the challenge is like, I think it's very easy to get very excited about building the thing that builds the thing or building the thing that builds the thing that builds the thing we talked about. And so people say, oh, we need to like redo everything as this big, like agentic first factory, maybe even a dark factory. And they're like redesigning their entire thing to be their infrastructure for the next five years. And I'm sure one thing we know of in engineering, and especially pragmatic engineering, is how can you make this more incremental? How can you make it more continuous? And a lot of people don't have the option to just, hey, I ran a RALPH loop for three days and it fixed every linter error in our code base. Here's a 60,000 line PR who wants to review it and who wants to sign off on merging and deploying it and that there's not gonna be any bugs. Nobody. So I think the thing I'm most excited is actually like what we call like iterated loops or like slow loops, where we basically have a cron job. We have the loop. The structure of the loop is really easy. It's like, run this linter, fix one thing, commit and push. And then we run that every night in our GitHub Actions and we wake up every morning to one PR that makes the code base a little bit better.

[Interviewer]

I like the stall loops.

[Dex Horthy]

Yeah. And it has two dimensions, so you can add. Now we have a blueprint for it. And actually Kyle just shipped a skill so that you can build these yourself. You can add more like feedback mechanisms. So we have react doctor for the front end. We have another anti pattern that has no deterministic tooling. But Kyle's just like, here's what good looks like, here's what bad looks like. Go fix one thing and bring it. It was like prop narrowing. Basically we have a bunch of optional props and most of them don't need to be optional. It's like, here's how to make the prop not optional so that you know that the code just is like cleaner and easier to reason about. And so you can add more conditions, more things of like, fix one thing. I wanna wake up to a PR. So now we wake up to like four PRs. Cause there's four separate things. And then the other dimension you can do here is as you gain confidence, you can increase the scope. Instead of fixing one thing, fix four things. And so these are like other ways to think about loops where it's like something that's not a human triggers it to start. Whether it's, you know, an alert from Sentry, whether it's a user feedback, like support ticket, whether it's PM writes a ticket, whether it's a test is failing any of, or it's a cron. It runs on a schedule. But it's like the trigger should be something that you don't have to like press a button on and there's a defined workflow and it makes everything a little bit better.

[Host]

Dex just described letting agents fix things without a human pressing a button. But what if a bug is too difficult not just for an agent, but also for human to reproduce, let alone fix? This is where presenting sponsor Antithesis comes in. I was recently pairing with the Antithesis team where we did a walkthrough of how they helped fix a nasty bug in etcd, the open source key value store used by Kubernetes. This is the bug that actually happened in etcd. The team noticed that the linearization validation assertion failed during the regular antithesis runs. This is not good because the linearization guarantees strong consistency, so this needs to be fixed. So what the ETCD team did was run a casualty analysis inside Antithesis. This generates this graph, which is a bug probability graph. Here the X axis is virtual time and the Y axis is probability. Now we see that something happened just before virtual time 24 that caused a huge jump in the probability that the bug would occur. Going deeper, we can look at the entire set of timelines. Vertical lines going down represent events branching off from the same state. And the purple dots are where the bug happens. If we look closely enough, we see that all of the failures come from one parent branch. Gotcha. This is such a useful debugging tool. In the end, the team was able to figure out that process pauses were causing the bug using all these antithesis debugging tools. This non deterministic bug was diagnosed in a deterministic way. How cool is that? Oh, and this is an actual bug that then got fixed in Etcd. You can see the bug and the fix in ETCD's GitHub repo. Honestly, the tools that Antithesis built for debugging feel pretty darn futuristic, but they are also really powerful. Head over to antithesis.compragmatic to learn more. I'd also like to talk about our season sponsor Sentry. Sentry is a tool I use for application monitoring on all of my projects, including their Pragmatic engine backend. I've used it for 10 years now, starting with when I worked at Uber. An 8 Sentry feature I'm liking is their SEER AI agent which helps investigate production errors. For example, here's an actual error I had in my application. I can just ask Seer what might be the root cause and it brings context and it can also make a plan to fix it right from the web interface. And the nice thing is how Seer also works great from Slack as well, not just from the web. One place I find even more handy to use Sentry is from Codex or Clock code using Sentry mcp. Also you can set up neat automations like when a resolve Sentry issue resurfaces. You can kick off a Cursor agent or GitHub Copilot agent to investigate their aggression, read their relevant code and open a PR with the suggested fix. I'm not a fan for using AI tools just for the sake of it, but I really like the practical integrations where I can fix errors faster and with more context. Check out Sentry@Sentry IO Pragmatic and start monitoring and fixing regressions today. And with this let's get back to Dex and to agentic loops that trigger themselves.

[Interviewer]

Now you said we can get more ambitious and we can add more things to it, but I'm going to quote you with one of your tweets which says this may surprise you that this is coming from me, but I think we're in for a one to three year period where stop might break at 3am and you're relying on loops to fix it and nobody understands what's under the hood and you're looking at it as existential threat to your company.

[Dex Horthy]

Yes. Yeah, that one was great. That one did a lot of numbers.

[Interviewer]

It Resonated.

[Dex Horthy]

Here's the other side of it is like, I think that the today, with today's models, today's programming languages, today's infrastructure, you might get away with not reading the code. Problem with loops is like, at a certain point you're going to generate so much code that you can't read it anymore. This is the Strong VM Dark factory. This is like Ryan Lapopolo's, like, harness engineering. Just spend as many tokens as possible. We tried this. We built a Lightsoft software factory in July of 2025, and by November we had shut it down. I think it takes about three to six months of you shipping all the time with nobody reading the code before you realize, like, wow, this is getting way worse and it's easier to start over than it is to fix it. Like, the models have made the code base so bad that it is actually going to be easier to just, like, rethink this from scratch. And maybe that's okay because we have AI and it's easier to rebuild things from nothing. And like, usually when engineers say, like, oh, we can't fix this, we have to rebuild it, the feedback is like, no, just refactor in place. Just constantly keep the code base getting better. You mentioned what I said. You'll notice what I said was not use loops to ship the features that users want. We use loops to actually improve the code base quality. And we read all the code because we care about how it's architected and we care not just about the system architecture, but what I would call the program design, which I think is something people are going to. Where are the interfaces, where are the seams, how are we doing Dependency injection? All of these things that, like, make your code base more maintainable over time and keep you from falling into this trap of like, okay, well, now if I change something over here, it broke something over here. This is the classic problem of software engineering that, like, software engineering was invented in the 1970s because we realized we needed techniques for avoiding that problem of like, this giant ball of spaghetti. And I don't think the models are smart enough. And I don't think we actually have the training and the benchmarking and the eval techniques to get models to write code that is more maintainable over time, versus they're all trained on Sweebench and Sweebench looking things right. All of the benchmarks are basically like, here's a commit in Django, here's an issue that was filed around that time. See if you can create the fix that the human created and it's Django and it's Apache and it's. There's a hundred repos in Go and C and Typescript and Java and all these different languages. But they're all. It's like the problem with training models on maintainability is like the cost function of bad architecture and bad program design can't be evaluated by running the unit test because it hits you three to six months later when you're like, holy crap, like no one can make. The software has become so hard to change.

[Interviewer]

Is this not similar to how senior software engineers. Why it took years for someone to become a senior? Because typically, and in some environments you can become a senior faster. Typically fast moving, where there's a bunch of issues and you have to keep fixing it. Sometimes, you know, some people are working in the same place for 10 years and they're still not that level. The point was it just takes time for you to understand the small mistake that you make right now that snowballs into like something disastrous later and you get hit by it and you realize like, okay, things like, you know, like testing matters, architecture matters. Tech depth can actually be a killer. You know, we don't talk about it anymore, but we used to talk about how tech depth kills or slows down companies so badly pre AI that their competitors can overtake them or they're just like stuck with a two year refactor not shipping any new features and the competition, you know, ships a bunch bunch of other stuff and now they're ahead.

[Dex Horthy]

And I will say like, it is possible that GPT7 will fix this. But if you are turning the lights off in your software factory and you're saying like, hey, you know what, like we're not going to read the code, it's fine, the models are smart enough. If we give it the right feedback and just throw enough tokens at the problem, it will keep getting better. This is what led to this tweet. Like, that might work, but if nobody read the code in three months and you replace all of your, all of your like code review with loops of like, hey, if a user complains, we give it to an agent. If something crashes, we give it to an agent. If a PM writes a ticket, we give it to an agent. If a CEO writes an obnoxious essay about what we should be building in Slack, we give it to an agent. And then you stop reading the code because that's going to produce way too much, like no one can read it. And the PR reviews become the bottleneck. So you Replace that with agentic testing and agentic code review. But none of these things have intuition for software architecture because we haven't trained it in yet. And so you're going to wake up one day and you're going to have an issue with this happened to us and we got through it. And at the time it was still worth it. It was like spent three weeks onboarding back into the code base that we had stopped reading three months ago. Because no matter how much sophisticating expert prompting, we could not get opus. I think it was Opus 4.1 at the time. We could not get Opus 4.1 to actually find the root cause. We had to go spend several days digging through the code and figuring out like, oh, there's just actually a primary key that's being routed through this whole thing that needs to be changed to a different type of object and it needs its own thing.

[Interviewer]

Oh, so this actually happened to you?

[Dex Horthy]

This happened to us, yeah. And when it happened, I was like, you know what? That sucked. That was terrible. But we did it, we solved it. And it's still worth. It's still worth not reading the code for most of the time at the cost of every once in a while I'm going to have to spend two weeks fixing an issue by hand. And I don't believe that anymore because I think the amount of code we're able to write now is actually like 10x or a hundredx. And I think the problem's just getting worse.

[Interviewer]

So let's talk about software factories.

[Host]

Yeah.

[Interviewer]

In your mind, because I feel it's an overloaded word, but what do you think of a software factory before AI and now post AI?

[Dex Horthy]

Do you know what the first definition of software factory, the first time it was used? No, it was a NATO conference in 1968.

[Interviewer]

Oh, Grady, which would know about this?

[Dex Horthy]

Yeah, exactly. Yeah, great. You should ask Grady about it. They talked about the idea of like, okay, you actually need to build a system of steps and like, just like a factory floor. You have like the coding part and the testing part and the validation part and the integration part. We had no cicd. We barely had version control. Like, but you needed a factory. And then it was adopted by like Toshiba and a bunch of companies. And then the, the next moment was like DevOps and you have like this idea of like, okay, we're going to do cicd, we're going to automate, we're going to use chef and ansible puppet, whatever. All these technologies is like, instead of having dudes running around data centers. Like resizing disks and stuff or clicking around the AWS console. Yeah, exactly. It was like cool. We build loops. The server hits 90% disk space. That sends an alert to nagios. Nagios triggers a chef run. Chefs makes the disk space, the disk bigger. Feedback loops. Right. This has been around for a while and in 2018, I want to say this guy, Nick Chelain, who was, he was like the CTO or chief software officer of the Air Force, he wrote this 100 page essay of, hey, the DoD needs a software factory.

[Interviewer]

Department of Defense.

[Dex Horthy]

Yeah, the Department of Defense and the Air Force. And he called it DevSecOps factory. And he said, we need all the things that all of the good enterprises are using. We need Jenkins, we need like code quality scanning, we need security scanning, we need cicd. We need to be able to ship. We're shipping once every three months or once a year. We need to be able to ship every day like all these other companies. And the way we do that is we actually embrace all these automations and technologies so that engineers are. 90% of the issues are caught by automations instead of people actually like manually checking it or manually reading the code or manually integrating modules together.

[Interviewer]

Wow, Talk about forward thinking in, in the government.

[Dex Horthy]

I know. No, as I was surprised, like, oh, nice. Like this is, I mean, and that was a part of it is like, hey look, we're falling behind in like, you know, I don't know exactly all the reason, but I imagine also about like attracting really good talent is like, hey look, if we have like the modern software stack and we're building things fast and we care about efficiency and we care about people's using people's time. Well, we care about them spending time on the hard parts of the job, not manually looking for SQL injections. Like you could automate that.

[Interviewer]

So this was software factories. Pre AI. Pre AI. Now I've heard the term loft more because of AI.

[Dex Horthy]

Yeah.

[Interviewer]

Is it the same? Is it different?

[Dex Horthy]

So this is really hard to say without a drawing, but I'll try to draw it out. At the core of a software factory, you have like a source of work. Most you can imagine, a linear, a jira, a state source of truth. Your object, whether it's a spreadsheet or whatever. You have like what stages is the work in.

[Interviewer]

Yep.

[Dex Horthy]

And pre AI, you would take, you know, you would maybe do some architecture review planning, you maybe do some Sprint planning. And then people would take tickets off the queue and they would go build them and then you would make A pull request and people would review it and you would run CI checks and then you would send it to prod and then it would make contact with your users and your users would complain about stuff and that would go to your support team and back into your work tracker and it would crash and you would have issues and that would go into your monitoring stack and that would go into your tracker and that was your loop. And then people would take stuff off the tracker based on priorities. Product managers, engineering managers, engineers prioritizing work. And they would go and do that. And the first change is like this

[Interviewer]

long wind, lots of phases. And this is also why when like a developer ships a bug, but by the time it comes back to you, it might be two or three months or even longer. And by the time I get fixed, it might be a year or two. And you know, this is why when you're using a piece of software, it's like that annoying bug and you talk with customer support, but it's just a very like long latencies at each part of the factory, if you will.

[Dex Horthy]

Yeah, and the step where someone pulls a work item off a queue and starts working on it is, you know, couple hours to a couple days before it actually gets integrated into everything else and touches user in of weeks.

[Interviewer]

A couple.

[Dex Horthy]

And that's in a great world, right? Sometimes you go build it and then you merge it and then it actually gets released three months later. But we're going to assume we're in a fairly modern like we're somewhere like the, a Netflix or a Meta where engineers are capable of shipping a hundred times a day or a thousand times a day, but it still takes 2, 3 hours to do the work. And now with an identic factory, what you do is you take out that person building the thing and you replace it with an agent building the thing. And so you have orchestration to trigger things. You have a sandbox, you have an LLM, you have an inner harness, you have an outer harness which is like the dev environment you build for the agent. And maybe you give it a browser, you give it a video recorder. If you use like things like cursor background agents, they've kind of built this outer harness around the inner harness that is the coding agent. And then you make PRs with that problem there is that like, okay, now, now it takes 10 minutes to do a build instead of two hours or two days. And so now the bottleneck is code review. So okay, let's throw a bunch of AI agents at code review and let's do agentic testing so that, like, we can basically catch a lot of the easy stuff. And humans are only focused on the most, like, important, critical core parts of the code base. And then the next level up of your agentic factory is you do the top is like, okay, and then it gets deployed, it goes to prod, and a user complains. You just hook your support queue right up to the agent. Someone complains about something, agent tries to fix it, and instead of looking at a ticket and then saying, okay, go send, you just close that loop. And instead, every time something goes wrong, you just get a pr. And then every time something crashes in sentry or Datadog or whatever, it goes into the tracker, it gets picked up by an agent and you get a pr. This is the ramp inspect thing. This is the. The only difference is, like, then you have so much code to review and people say, well, let's try turning the lights off. Let's just take all the human testing and review steps out and we'll say, okay, cool. If users complain that it's broken, and if users don't complain, then it's working. And we're not going to read the code we're going to use. We're going to treat the whole system as a black box.

[Interviewer]

So you said you tried this out when it was like Opus 4.1.

[Dex Horthy]

You.

[Interviewer]

You built the software factory. It was running beautifully until it just blew up on your faces. How do you think of this model? Because I can see an ideal world where it works, but clearly we're not in an ideal world. Where do you think we are? And could some of this actually work at some point? Or what progress are you seeing right now? And what is the today, the situation? How much of this do you believe we can automate? Or should we automate?

[Dex Horthy]

Yep. So if you know me, you follow my stuff. You know, I stand for three things. Number one is like, cutting through the hype and the jargon and going, trying things and talking to people who are using things and figuring out which parts of this actually work and are valuable. Number two, we talked about words. I try to find and protect useful bits of language because I think it helps us all move forward. And when you take a useful word like agents, or you take a useful word like software factory, and then you semantically diffuse it. This is another Martin Fowler word. You make it mean everybody likes the word and it all becomes hype. And everyone starts with and agents means nothing anymore. Agents could be a chatbot, it could be a slack bot, it could be a coding Agent. It could be tools in a loop, whatever it is. So I like to protect important, useful words and like, help help us all, like, elevate the conversation out of that hype and jargon. And then I care a lot about going one level down beneath where I'm generally working. I think there's always. This is the same thing with context. Engineering is like, I was rarely actually going and like building LLMs or understanding or training LLMs, but knowing how they're trained and how Transformers works informs how you build at one layer up. And for the software factory, my version of that is I spent the last couple weeks going really deep on reinforcement learning with verifiable rewards. Rlvr, which is like this very productionized. Like, it's not like RLHF is still like fairly academic and pure. RLVR is this like, it's a machine in these labs of how we train these models. And I'm studying like the benchmarks for coding agents and the techniques for training them and how we like, give it a small problem, have it solve it, delete the test changes it made, revert them, apply a test patch, see if it passed. And then even the frontier this year we have like, we can get into this later, but like Frontier Code and Swimarathon, these new benchmarks that are supposed to be like, better at evaluating models ability to maintain a code base over time and write maintainable code. And they are better, but I don't think they're sufficient. But it was basically this idea that, like, the only thing that made Claude code good was reinforcement learning. And the dimension along which it got good was like, we made a model. We trained the model and the harness together. And so the model got really good at calling the specific tools in that harness, really good at reading files, writing files, searching for files, all this stuff through doing these problems. And that was what made it feel so much better than all the other CLI coding agents that came before it. And so people are like, okay, that was so much better and they're just going to keep getting better. But it's like, it got really good in one dimension and the dimension that they're not getting better in because it's hard, expensive, maybe we need to like get a lot more creative with how we design these, these verifiers and benchmarks is in how do I make code that in three months is going to like improve the productivity of humans and agents, mostly agents, but humans and agents in the code base instead of making it worse over time.

[Interviewer]

And so you think that part is just missing. We haven't seen too much improvement.

[Dex Horthy]

I haven't seen. Obviously no one knows what the labs are doing internally because it's all very secret. But I think if we. Looking at where the bench, the benchmarks tend to reflect where the labs are, right? If the. There is no benchmark that can convey to me, did this model write code that is going to make my code base better or worse? The best we have is, I think frontier code from the cognition team is really interesting. They have like, did the test pass? And then they have like two layers of model review. So they have a judge model that checks. Okay, is the patch the model made similar to the patch that is like the golden answer set. So even if the model didn't write the exact code that the benchmark was expecting did, was it functionally equivalent? And the next one is like a like, code quality review from another judge model and like, that's better, but it's not, it's not sufficient. And this is why I also think agentic code review is like, yes, it will catch things and it will raise your floor. But I don't believe, like the model writing the code is the same model reading the code. And if you ask a model, hey, is this code good? It's going to be like, oh, yeah, it's great, comprehensive. It's got unit tests. You've tried this, I'm sure. And you say, okay, review this PR that my coworker wrote and tell me everything that's wrong with it. I was like, oh, it has this problem and this problem and this is sycophantic and they want to tell you what you want to hear. And so, like, it's really hard for me to trust a model to evaluate the quality of, of code that's written. And so I have some ideas on, like, okay, can you build a benchmark where the model builds 20 features in a row and maintains the code base the whole time and it doesn't know what features are coming? You treat it like a real product team where you don't know what you're going to build next week until you get there and you find out what's most important. And then can we try to evaluate, like, can we build a problem like that? That's hard enough that most frontier models fail by issue 6 or 7.

[Interviewer]

Is it fair to say that, you know, like, we've had the software factory, like, before AI, it was just like lots of loop. It was like the PM giving the tickets to dev, the dev building it, deploying to production customers Using IT customer support, getting tickets and then creating PM triaging. And it just kind of goes around in this loop. Is it fair to say that the software factory of how a company, a team builds and maintains software, that is changing because now everyone's replacing some parts of it. Maybe the least advanced teams will just be. Devs are starting to use Claude code or codecs to write faster. They're not spending as much time on there. Some others are also having the deployment, the feedback. Some actually have the agents already one shotting bug. So is it fair to say that the software actor is just changing everywhere? Maybe at different speeds. But everyone, I think every team who is building production software, they're frantically experimenting, trying, and everyone's at a different pace. You'll have the AI native starters where most of this will have agents in them and you'll have the laggards who are more cautious ones. They have agents in a few places, but not in the others.

[Dex Horthy]

Well, and I think that's the key is like if you want to do loops engineering, you should build one loop at a time and you should keep them small and contained. Basically. I think everything except stop reading the code is really good advice. Take support tickets and turn them into tickets in your system and then maybe turn those into PRs. Great. The advice that I have and like what we kind of like are chasing at human layer is like, how can I add another checkpoint in that factory? So instead of having one human viewpoint where you're reviewing PRs and sometimes there are a hundred lines and sometimes they're a thousand lines, but it's quite a lot of effort for, especially if it's bad, especially if it needs rework. It's quite a lot of effort for a human to be like, okay, this is wrong, go change it in this way. And then you loop back to the agent and then you come with another one and like doing a lot of loops on there, once, once the direction has been committed to, it's hard to steer off. Like you're better off just kind of restarting from scratch. How do you build like controls and mechanisms around that? And then my take is like, if you do a little bit of human agent planning and like discussion before you hand it to the implementer, whether it's, I mean, planning and specs, whatever you want to call it. Again, this is spec driven development is another word that has become kind of very like muddled as far as what it means. But basically how can we spend an hour before we start building so that the PR, when we read it only takes 20 minutes because the code is perfect. Instead of not touching and just literally saying every user reported issue becomes a PR through the loop. And then we read that PR and it takes six hours because there's back and forth and we have to make changes and things like that. It's all I'm all about, like, let's find leverage. And so you basically, you have three options in the software factory world. If you're going to go all in on Agentix software factories, you can turn the lights off and just let everything flow and pray that you don't create too much slop and pray that the next generation of models comes fast enough before you create a giant pile of ash. You can slow way down and read every PR and read every line of code and then you're only going to really get modest benefits from AI because that becomes, I think you should expect maybe 30 to 50% lift in productivity is kind of what I see when we go into teams. Or you can find the right leverage points where humans can actually an hour spent over here in planning can save you four hours in implementation in terms of fixing and going back and getting the design right. And that's what I call like seeking leverage. You can find the right leverage points for the agents to guide the work. Then you can actually move like two to three times faster while maintaining a like 99% like accuracy to like, if the humans were carefully writing this code by hand, how would it come out?

[Interviewer]

Now jumping a little bit back to ideas. I will come back to this. This was earlier, maybe it was last year, but you had the research plan implement. Can we talk about the original research plan implement framework? And then also what you've learned about this approach, what, what you got wrong about it.

[Dex Horthy]

Yeah, sure. Yeah. So I mean the first time we talked about RPI was in August of 2025. And it was basically like the research was this thing of like, hey, before you go build anything, go read lots and lots of code, use a bunch of sub agent subagents in parallel, understand all the code. It was this technique that like worked really well for hard problems in complex code bases. You just asked Claude to do a thing that that's. It would read three files and make a change. It would have no context. So you start the research, you don't even tell it what you're working on. You just tell it, hey, can you tell me how this system works and this system and how they connect together? And then you get a markdown doc out. And this was the context engineering part is like, that would take a hundred thousand tokens of context, but you would get a 10k token doc out of it. That summarized it. Then you would start a new context window and you would do planning. And the planning would be. And I actually realized, like, the plans that we were building last summer were actually terrible, but it would basically be this long. You would say, okay, now here's what we're building. Here's the research doc. Build a plan to implement it. And in retrospect, now that we see, like, everyone is obsessed with how do I get agents to work for longer. I think the reason why in like May, June, July, August of 2025, that a lot of people became really interested in planning was it was a very powerful lever to get agents to work for longer. If you said build me a B2B SaaS for burrito delivery, you'd get like a homepage and that's it. But if you said build me a plan, it would build out this big plan. And then in the next context window, you'd say, hey, here's the plan. Here's all the changes we're going to make. Go implement. It would actually keep going until the plan was done. So the plan was a really good way to anchor an agent and remind it that like, hey, you're not done until this is all finished. So that was the original RPI and the plan doc. What was bad about it is it didn't give you leverage. The plan was every single line of code that was gonna change, like in diff blocks and like all the new stuff to write. And so like people would review these plans. We recommended this. We told people to read the plans. We read all our plans. And then eventually I found myself like, I just kind of skimmed the plans. And so you're not really using it as a way to resteer the agent. It's just kind of there. And then you go write the code and there's a crap. Some people would review the plans and the code and it's like, okay, well the plan was. Took you 20 minutes to read and then the pull request takes you 20 minutes to read. And they're different. And so you actually doubled the amount of time you're spending reading code instead of like doing less of it.

[Interviewer]

You've anti leverage and hang on, was spec different development, not related to this. The. The one that Amazon Kiro for example, and. And GitHub workflows again a year ago did, which was it also it first generated a plan and it had the human review it. And then it started to and you could edit it as well. And then it went off and implement this part and it looked beautifully on the surface. It should have worked great. But it's tossed into the garbage. Outside of some maintenance projects, I think it just didn't work. Like all the feedback I got, people just stopped using it because it just didn't really work that well. It just rhymes to the RPI framework a little bit. The original one. Right.

[Dex Horthy]

Well, so our thing too, like the biggest difference between RPI and spec driven development. And some people refer to RPI as spec driven dev because for some people sdd, all it means is I use a bunch of markdown files while I'm coding and forget what's in them. I just. Spec driven dev, those are my specs and I'm using them to drive development. There was this OpenAI researcher who talked about spec driven dev and like, hey, stop reading the code, just write the specs and treat like the coding part is compiling specs into code. That part never really materialized. Maybe with GPT7, you know, but the challenge. I'm on a GitHub issue in spec kit that has been open for a year and every couple weeks I get. There's a new email on the thread of people complaining about this problem. Like, okay, I edit my specs and then I edit the code and the code drifts and the specs. How do I keep the specs up to date as the code is changing? And it's basically like you now have two sources of truth and it's. It stops being useful. And so like, that's why when rpi. The idea of the docs is they're all. For a while we kept them around, but after two or three months we're like, like, oh, these are actually like tactical execution docs. I do the research, I do the plan, I do the implementation, I throw the docs out. And the next time I need research, I just do it from scratch because tokens are cheap and my time is expensive and the amount of time I might waste if I reuse a research that is no longer in sync with the real state of the code base. So we just create it live every time. This is why it's like context engineering still matters. Creating artifacts that compress the state of the code base and compress the intent of the builder into small things that can be reused in the future for the scope of a task is like a very powerful like tactical approach, but it's not a thing. Like, I have very few opinions on like what sorts of docs that you should leave lying around your code base that are like evergreen. I've seen people try to maintain parity between documentation or specs and the code itself, and I don't think anyone actually like found it very useful. Like you can do it and it works, but it's like the ratio of the effort it takes to keep them up to date and trivially. You could do this with AI probably, but I've never known anyone who was like, yeah, this is great and we're glad we have it. Like you could do it and it might help, but I don't think anyone found it useful enough to maintain a system to keep the specs in the code in sync versus just using the code as the source of truth. Always.

[Interviewer]

Now, you mentioned something interesting, which is with context engineering you need to sometimes compact. And you've previously talked about intentional compaction that when context is noisy, deliberately compress the useful part into clear like markdown artifact, verify it and then start a fresh conversation. Can we talk about this kind of compaction and why it's important? And it sounds like it's going to be a building block where it already is for context engineering, right?

[Dex Horthy]

Yeah, Frequent intentional compaction is the building block. It completely comes from context engineering. Context engineering is like how do we get the most out of today's models? How do we change what we're putting into the model, into the context window, into the agentic chat? How do we control that in such a way that we get the best results possible? Which means doing as much work as possible in the smart zone, the, you know, first hundred thousand tokens of the context window. And this intentional, frequent intentional compaction is basically like, okay, the research step, we're going to go read a bunch of code and turn it into a doc. That's our compaction. We take that forward. In the next session we're going to read, we're going to read the ticket and the intent and turn that into a design document that we call it. It's like, okay, here's the high level spec of what we want to do. Here's the high level like current state, desired, end state and then a bunch of design questions. The model has kind of like a very thorough, maybe even over engineered like plan mode. And then you take the research and the design and you do a new session, new context. When you're like cool, you, you've compressed the intent and you've compressed the state of the code base so that you can then do your planning of like, okay, we know what the end state Looks like we know where we're going now. Let's break down how we're going to get there. All of these different steps of the process exist because models have shortcomings in each of these phases. So the research is pretty hands off. I don't read the research docs. It's just like, go read a bunch of code and then like, make a doc out of it. Models are pretty damn good at that. If you ask it to find a bug and have opinions about the code base, that's different. But if you just ask it, what is the intent and how does this stuff fit together, that's usually pretty straightforward. But designing the end state of the, of the software, the architecture and the program design models are not great at. They make a lot of, like, they make decisions and sometimes they're right and sometimes they're wrong. So we want to have a human in the loop. There's. And then the steps to get there. I, we talked about this before, but models love making what I call, like, horizontal plans. If you ask a model, like, build a plan of steps to go build this app, it's like, cool, we're doing the database and then we're going to do the services layer. Then we're going to do the API and then we're going to do the front end. It's like, well, that actually kind of sucks because we're going to be on the other side of 2000 lines of code. And let's imagine this is an existing code base, right? We're going to make changes to all these different parts of the system. I can't test it till the end. And so what I would do is like, okay, how would I have built this if I were building by hand? Well, okay, I would probably create a mock API endpoint with fake data. And then I would go kind of get the front end kind of how I want it to look. And then I would actually go, like, build a services layer and actually wire the data through. And then I would make a database migration and make my new table. And then I would actually add a lot of business logic. And then I would add a bunch of error handling. And it's completely orthogonal to how model, like, models would write the database layer and all the error handling without ever, like, anyone's ever touched or seen the code or whatever it is. And so this is another place where we're like, we like to have humans involved because humans have really good taste and judgment. And like, like, I would rather read five separate little mini diffs of, like, things that I can manually verify and Explore than read 2000 lines of code and be like, well, it's not working. I don't know where. You don't know where because you wrote the code, you were supposed to get it right. We talk about compaction context engineers, like, how can you stay in the smart zone of the context window, which is again, the dumb zone. I will say disclaimer, it's really good training wheels if you don't have intuition about this.

[Interviewer]

So, so let's just define these things. What is a smart zone and what is a dumb zone?

[Dex Horthy]

So it's a little bit blurrier than I would like it to be. I think in November we talked and said, oh, it's about the first 40% of the context window. But then we had million smart zone. Yeah. Then we had million token context windows. So then I changed it to the first hundred thousand tokens. If it's a really like 4.8, I usually will go up to like 200k. But basically the thing Jeff Huntley had and Ralph Wiggum was like, the less context window you use, the better outcomes you'll get.

[Interviewer]

And basically the smart zone mean meaning if you have context in that first part, it should work a lot better. And then like the dumb zone is like, once you have stuff there, it's kind of forget about it. Like, it'll be confused. It's not going to do much. Like it'll degrade.

[Dex Horthy]

Yeah. And there are times, and this is an intuition thing, like I will often go up to three 400k tokens. Four is rare, but I will go up to 250, 300k tokens for certain types of work. Or my intuition tells me that I can keep working without, without degrading the performance. But if you don't have good LLM intuition, like 100k for smaller models, 200k for these, like, really beefy like, codecs and opus. 4.8 models is usually a good, like training wheel guideline of like, if you pass there, your quality of results may be degrading. The biggest tell I see for this is often the model's trying to get the test to pass. And your 200k token. Well, let me try this. Okay, let me try that. And it's like trying a bunch of stuff and it's getting more and more extreme. And it's like, thing, oh, let me delete your dot end file and try again. Like, this is where things get really, really weird. And so it's like if you start to see certain types of. If I'm like, oh, we're at 300k tokens and I need to like fix the unit test. I'm like, cool, write everything we did to a file or even I'll just do like a built in compaction depending on the model. And then I'm starting a new session at 30k or 50k tokens. And I'm like, cool, we're going to do a hard thing, which is you're going to get this fricking test to pass and you're not going to be stupid about it.

[Interviewer]

By the way, one thing that you said like about the model being dumb is you said that if the model ever tells you you are absolutely right, you should start over. And we've all had that. When it tells me like, oh, you know, you didn't, you're absolutely right. And I'm like, we just get annoyed, but why should we start over? What's happening there in your observations?

[Dex Horthy]

Yeah, that's great. Yeah. And the new, the new, you're absolutely right, I think is you're right to push back on that, right?

[Interviewer]

Yes. That's Opus, right?

[Dex Horthy]

Yeah. Opus is like, you didn't run the test, did you? You're right to push back on that. I totally did it. But no, for me, you're absolutely right was always what the model would respond if you were like, like, that's totally wrong. You did it. Like you, if you were, if you said something where you were angry or frustrated or just wanted to point out that it's done something wrong, it would respond with, you're absolutely right. And most of us have had the experience of it says that and then it continues to do the wrong thing. So it's like once it starts doing dumb things because there's, there's four things in your context window that matter. There's like the size of it, how many tokens. There's like the quality of the information is like, is there any incorrect information? Like if the model had some thinking trace where it decided the wrong thing was true, is there missing information? Does this like have context missing that it should have. And then there's the trajectory. And the trajectory is very subtle. But you may have had sessions. The trajectory, meaning you're prompting the actual history of everything. I call it trajectory is like the actual history of like what the agent has done in the past.

[Host]

Yeah.

[Dex Horthy]

And so if I say, hey, make this change and the agent makes the change and then it runs the test and then they're broken and then it fixes the test, I have very high confidence the Next change I asked it to make, it's going to follow that path again. Because it's like, okay, here's a conversation. And the last time the user asked me to do a thing, I made the change, I ran the test, test broke and fixed the test. And then I told the user. But if I say make a change and it makes a change, it doesn't run the tests, then I'm on a different trajectory. And if I say, okay, make another change, it's like basically the, they're autoregressive, so they're, they're predicting the neck. What's the next message in this conversation? And so the example we talked about in no Vibes Allowed was of course the like, hey, the model makes a mistake and then you yelled at it and then it made another mistake and then you yelled at it and then it's like, cool. What's the next message in this conversation? Well look, if I read the history, I should probably make another mistake. So the human can yell at me. So the human can yell at me. So I was like, okay, that's a great, that is a great example of like time to start over.

[Interviewer]

Let's talk about some observations on how software engineering is changing. One thing you talked about recently on the evolution of the coding meta is going from token harder to token smarter. Can we talk about what you mean by token harder and token smarter?

[Dex Horthy]

Yeah, so token harder is, I mean I'm in a, I'm in a group chat called Hyper Engineering and it's all like people trying to max out their cloud subs.

[Interviewer]

Oh wow.

[Dex Horthy]

Okay. It's just like, that sounds like a

[Interviewer]

fun, is it a fun, fun place?

[Dex Horthy]

It's a fun place, but it's like all token harder. It's like look at all the side projects I built. It's look at everything that I've gotten my Claude token. I've got six, six Claude code accounts. I've gotten all of them maxed out every five hour period. I've timed it out so I always use all the tokens and it starts up immediately when the limit resets. And so it's like, I mean getting into Eli Goldratt and the goal is like optimizing for utilization and efficiency of one node in your factory rather than the end to end goal of like how do we ship value and things that people like that are stable and like will last a long time. But that's my idea of token harder. And it's the same thing with the dark factory thing is like, hey, if you, if you, if you remove humans from code review, you can push more tokens through the system.

[Interviewer]

So we talk about software factories, but what is the dark, dark factory?

[Dex Horthy]

Ah, so the dark factory is. This comes from this idea of like there are factories where everything is automated by robotics. So you can imagine like a car factory where it's all robots building the cars and they don't have lights because there's no humans.

[Interviewer]

Oh, so that's where it comes from, dark factory.

[Dex Horthy]

Yeah. You walk in there, there's no lights, there's not even light switches.

[Interviewer]

So it will be the fully automated software factory where it, it will be like no human input.

[Dex Horthy]

Basically no human input. But raw materials go in, cars come out.

[Interviewer]

Yep.

[Dex Horthy]

And I think in a micro, like you can have mini loops that are dark in your, in your thing of like, hey, if, if the code review agent comes back with a problem, you loop that back to the builder agent, it fixes it and comes back and that's dark. You don't need a human loop for that. But the full dark factory where you don't read any code. Yeah, it's a good way to maximize your token utilization. It's like if your belief is like, my job is to extract as much intelligence out of the machine God as I can because that's how I get the most value and the most leverage on my time than token harder. And my take is basically what we talked about before token smarter is like, okay, how do I move faster? How do I get as much value out of AI as I can without having to turn the lights off while still maintaining control and taste and judgment and understanding the system architecture and having a lot of like, applying my hard won opinions through 10 years of software engineering to the design of the program so that I can feel confident that the code's going to get better and more maintainable over time. The same thing of like you look at like the SRE team inside Google, they brought out this book, SRE site Reliability Engineering. And the whole take was like, hey, we're going to go from one data center to five data centers and we need the same six person team to be able to manage five data centers and we need the Same six person team to be able to Manage 50 data centers next year. And it's basically how do we apply software to this problem so that instead of scaling linearly of like, okay, every Data center needs five DevOps people, so we need to scale the people with these things. How do we continually automate the parts that we don't need? So a little Bit orthogonal and maybe even, like, contradictory to what I just said. But this idea of, like, how do you find leverage? And the way, the way.

[Interviewer]

Well, I think what you were saying there is, like, in Google that never seek to remove those SREs from the process at all. They just said, like, look, can we think ahead and scale yourselves? And they actually grew the team. It wasn't actually six people. It was more like, I think Google specifically said, okay, we have five data centers. Next year we'll have 50. There's six of you. We do not want to have 60 people. We don't want. And then management layer and all that is like, how can we do it with like 12 or like, or like 10? And then one will have 500. And now actually their SRE has grown, but of course, yeah, but they never, you know, I think as engineers, like, we feel pretty threatened when someone says, like, all right, we just want to have zero engineers. Like, I mean, that's not a fun place to work at. But it sounds like token.

[Dex Horthy]

It's not a possible place to work at if they have zero engineers. Neither of us can work there. Right.

[Interviewer]

But do I understand the token? Smarter is like, let's keep humans in the loop. Let's keep adding value and figure out what are the parts which are not as relevant, boring, where we don't need it. And so, like, one developer can probably do more than before. But you are built to, like, be part of this whole thing. And the lights are on in a factory.

[Host]

Yeah.

[Dex Horthy]

And it's like, basically, I think what I'm trying to get to is, like, the connection here is like, SRE built a thing where, like, headcount scales at, like, a square root function or a logarithmic function, whereas their output scales, like, linearly. And you want to say that the way you do that is with good architecture and good program design. And so in order to, like, avoid this problem where you have to throw more people or more tokens at the problem, if you design good software in such a way that it gets more maintainable and more scalable over time. And like, just today, it doesn't feel like, like, basically you need humans in a loop to be able to do that.

[Interviewer]

Let's talk about AI slop. At one point you wrote, yeah, AI can write your code, but it can also write your specs and PRDs. But the same, the same rule is always slop in, slop out. If you outsource your thinking, you're going to get garbage. Yep.

[Dex Horthy]

So, yeah, that's basically the idea is like the way we think about like getting high quality outputs is like, yeah, you could write the code by hand or you could sit with a model and work back and forth and go maybe a little bit faster. And you have control. And every time it makes a change, you go read the change. And if it's bad, you tell it. Nope, we want it like this. And you kind of incrementally, slowly. This is like kind of the stage 2 or stage 3 version of working with agents, where like the agent's writing all your code, but you're kind of very much in the loop. And this will make you go faster, but it won't make you go that much faster. It won't make you go anywhere near. There's like, there's like that level and then there's like the maximum speed you can go while still caring about the code. And then there's like the maximum speed you can go if you've turned the lights off. And so we always think about it as like, in terms of leverage is like, okay, let me take. Everything starts with like a sentence or a voice note, ramble like, I want to build this thing and it's going to work like this, whatever it is. Let's say like on average like two sentences. I got to fix this thing or there's a support ticket. I got to fix this thing. If you can turn that with AI into a one pager and then turn that one page and make sure that's correct, and then turn that one pager into a three pager and make sure that's correct and then turn that three pager into a ten page like detailed outline. Then you can write a hundred pages worth of code and it's maybe not perfect. You shouldn't like sweat over these documents and make sure they're perfect. But you're increasing the chance that like you're decreasing the uncertainty of the outputs. It's like you can think of like, you have like a line of like where it's going and then you have like the probabilities of where like it might go in that range. If you are kind of reviewing along the way, as you get more and more detailed into how, what you're building and how you want it to be built, you kind of collapse the uncertainty and the set of end states that you could land in. That's me doing the physics thing of like, you gotta superimpose all these probabilities and like, I don't know, I have this thing that like, I think people who really like playing real time strategy Games are probably gonna be really good with AI because you kind of have to like, I don't know, Matt Pocock was just talking about Fog of War and like, things that are at the frontier of like, there's stuff we don't know about this problem yet. How can we find that out? And how can I make the best decision now, knowing what I have seen? There's a. I've seen a couple pieces of information, and so there's a 30% chance it's this, and there's a 40% chance that it's this. How could I get more information? So in my head I can like, recalculate those probabilities and decide what's the most likely path that's going to lead us to success.

[Interviewer]

Speaking of the most likely path that leads you to success, let's talk about your company. That's you've. You've just come out of stealth. Human Layer. What is Human Layer and what is the probability that you're setting up for success?

[Dex Horthy]

That's a good question. 100%, 100% probability? Maybe 110, but no. So human layer is an AI IDE. It's a collaboration platform and it is building blocks for your software factory. And the basic pitch is like, engineers solving hard problems in complex code bases. Basically, there's two categories of builders. There's like vibe coders building side projects, and then there's people building production software where the stakes are high. And if something breaks, we're going to get fined millions of dollars or, you know, we're going to lose millions of dollars of money for the company. And there's a whole spectrum in between there. But it's like if you are kind of in the left half of that spectrum, you're building software that matters and it has to last and be around for a while. Then we are helping people like that solve problems two to three times faster without descending into slop. It's like, how do you maintain that near human level of quality and move two to three times faster?

[Interviewer]

And what were the ideas that you, you built and that you came with?

[Dex Horthy]

One idea that we're really excited about right now. I mean, it all comes from this RPI and this, like, using specs to like. I mean, I've kind of been hinting at it this whole time, right, of like, okay, cool, like start really high level and zoom in layer by layer and resteer and like, find. Find that leverage that helps you move faster and increase the chance that your agent's going to build exactly what you want. Or something that's really high quality. The other thing I think that's really interesting that where I just posted yesterday, I said, hey, chat, should we kill the pull request? And that's something I can't talk too much about. But basically the idea is like the IDE of the future needs to be rethought from the ground up for agents. And it might not even be a, like, I don't know, a lot of editors kind of started with the text field and bolted on an agents tab. And then eventually you've seen like cursor three. I can't even find the text editor. I know it exists. People have told me you can get to a text view of files, but it's also very agent first. And so we started from the ground up of like, what is an IDE design for helping a developer interact with and manage the work of agents? And then we zoomed down and said, how do we make this collaborative and build in a sync engine and durable streams and all of these like pieces of tech that enable me to get human input and feedback on what I'm doing with agents in real time rather than waiting for the pull request time. And great engineering teams have been doing this for decades of like, hey, we're going to have a design review where we're going to talk about how we're going to build the thing as like a two page Google Doc or whatever, ten page, whatever.

[Interviewer]

P R D, E R D. Yeah,

[Dex Horthy]

your architecture requirements document. And then you go to Sprint Planning and you break it down into little tickets and you decide who's going to do what. It's like, AI can help with all of this. You should. If you're just using AI to write the code, you're missing out on a lot of the benefits that AI can bring to your sdlc. And a lot of people say like, well, we don't need any of those meetings anymore because we have the loop, we have the dark factory. Things just fly around the loop. But it's like, okay, but if you want to actually move faster and maintain quality, then like you should have these checkpoints before you go to actually write the code. And you should use AI to help with that. So we built this like cloud platform that's kind of has like a Google Doc style component where you can comment and the agent can surface like mockups and mermaid diagrams and HTML and all these things. So basically, how do we make agents like Figma style? Every. Everything's in the cloud, everything's collaborative. I see all my coworker Sessions, they see all of mine. It's almost like the benefit that Slack had over email was that you didn't have to be in every conversation to know what was happening. You could maintain, you could see all these channels light up, you could check on them. You go, okay, I don't care about any of that. But if you saw a conversation that you cared about, you could jump in on that. And it's like, how do we do that for engineering work versus like we really had these like, very strict. Even when we called it agile, it was very waterfally, like PRD Arden tickets. Everyone goes and builds for a day and then you get the PR back and then one person reviews it. How do you create this more just like soup and like, what is the data model for that world where you have like agentic traces, you have documents, you have tasks and projects that group these things. You have actual git diffs being streamed everywhere, where it's like, why would I review all the code at once when I can just always every. Everybody's work lives in a shared environment that anyone can go interact with.

[Interviewer]

I mean, what it reminds me is like what GitHub did to software teams before GitHub and its competitors. You might have a tracker somewhere, but most teams were just kind of like inside the company. You didn't know what one team was. I remember pre GitHub, like, you know, you had individual teams. Some of them had like a board with stickers, but no one else in the company knew what they were doing. They were all working in isolation. And now when you have GitHub or even the internal version of GitHub inside a company, you can always see when you go to a team, you see the pull request flying. You can join in, you have history, it is all kind of connected and it came together and now it's like, you know, for a very long time I was like, duh, you're going to use GitHub or people will copy it. So do I sense that you're trying to build something like this workflow for like when you have the software factories, which are like dark factories and loops at a bunch of places. How can we have this, this new way of working which will feel natural, but like coming up with it is hard work and it's counterintuitive.

[Dex Horthy]

How can we do something that accomplishes what GitHub did? But like 10x better, like more specifically, like more continuous and more real time and more collaborative than like these discrete units of work that is like the pull request.

[Interviewer]

Well, I now I'M starting to understand why you're saying maybe you should kill the pull request because pull request was invented by GitHub, right? Like it is not part of Git, but they did it as a way for you to do a code review, merge before it goes in and be able to modify it, or like just reject it, et cetera.

[Dex Horthy]

And it's probably a lot better than whatever we had before, which I guess was like emailing your git patch to Linus and ask him to merge it into the kernel or whatever.

[Interviewer]

They still do it. It works for them. That's the point. But it only works for them.

[Dex Horthy]

Yeah, I don't know anybody else who does that. I mean, I'm sure even before GitHub for you, you guys had what, like

[Interviewer]

CVS or CVS cfs if you had a lot of money for Microsoft, they

[Dex Horthy]

made us use Subversion at in undergrad because the guy who invented subversion was a UChicago guy. The year after I graduated, they switched everybody to Git and I was like, damn, I learned a useless thing just for somebody's ego.

[Interviewer]

Specifically for AI startups or startups building on top of AI or building AI products, how important do you think location and network is? Especially you are based in the Valley. We see research that AI startups are more frequently funded from here than normal startups as well. Do you see this advantage? And also do you see some disadvantages of being a specific, may that be Silicon Valley or elsewhere?

[Dex Horthy]

I don't have really strong opinions on this actually. Like, Paul Graham gave a talk in Sweden about why SF is cool. Rather than just regurgitate that, I will forward people onto that one. We can put in the show notes or whatever. But he talks about all of the dynamics of Silicon Valley and the pay it forward culture and the people take you way more seriously just because you're based here. I lived in Chicago for a long time. I have a lot of really good friends from high school, from college, from growing up in la. And never before have I felt like so locked in with like my people more. Never have I felt more, seen more connected. Like there's just so many people here again, talking about the founder thing. People who care deeply, who are incredibly competent, who like we have all the same types of problems, we love all the same types of things. Like I don't do land parties where we play video games, but all my buddies will come over and we'll sit in the office till 11. We'll just do co working and like hack on cool, fun, fun projects. And stuff. And like, you can't do that anywhere else. There's not enough, like, critical mass for that to just happen organically everywhere you go. And I absolutely love it. I wouldn't trade it for anything.

[Interviewer]

Yeah, I think a critical mass nails it on the head when it comes to hiring. What types of folks are you hiring for, specifically? Because I'm interested in how hiring changes and what, what a standout engineer means for you and how you are trying to, you know, confirm that those traits exist.

[Dex Horthy]

In general, we are looking for people who have really strong software fundamentals. So understand distributed systems. Understand like the core fundamentals of CS and operating systems and these kind of things. I mean, you don't have to be a PhD in fricking kernel design or whatever, but it's a lot easier. We can teach somebody, I think, to be a really good AI developer in a few months. You can build enough intuition where you are, you know, accelerated off the ground and you can go like, keep growing there. It's really hard to teach someone a CS undergrad program in three months.

[Interviewer]

And what's the problem space that you're excited about in software engineering or even product engineering or building products that you think in the next few years is going to be one of the interesting things that you're going to be attacking.

[Dex Horthy]

My co founder could talk more about this, but like, there's a lot of interesting things happening in real time, in cloud and sandboxes, in sync, and kind of like using these new building blocks that have gotten really solid in the last couple years. We're big fans of the electric SQL team. We're users of durable streams. It's like, how can you build systems that kind of are a lot more spread out and distributed and almost like decentralized? This is really interesting for coding because you want to be able to run coding agents anywhere. You want to be able to run them for a short time, for a long time, on demand, on a schedule, all these things and have them all be part of this kind of like, like brain. So I don't know, parts of what we're doing are really boring. Like all our data's in postgres. And then parts of what we're doing is really interesting, but there's a lot of distributed systems problems, there's a lot of infrastructure problems. Like we are building tools for AI, but there's a lot of problems in building collaboration platforms that are really, really hard. And there's a lot of new tech that makes it easier and more interesting, but it's still by far from an easy problem.

[Interviewer]

It sounds like what you're saying is like the infra layers to some extent, a new infra layer is being built and it'll take some time, but it'll be like just new blocks and it will eventually become the primitive. Like for cloud, we have primitives already, but it took frigging decades to get those together or more.

[Dex Horthy]

Yeah, you had AWS in what, like 2008, 2006. Yeah. And then you got kubernetes a decade later.

[Interviewer]

Yep. And as closing, what's a book or a reading that you would recommend something that you personally enjoyed?

[Dex Horthy]

Nowadays we talk a lot about Refactoring by Martin Fowler. Classic. I think it's because we spent a lot of time improving the design of existing code and trying to figure out how to get models to build code that is easy to maintain and like easy to read and easy to understand and easy to build on. I feel like I probably have a better answer than that, but that's what's top of mind these days. We're reading a lot of classics of software engineering, Refactoring, clean code, the Pragmatic Programmer. All that stuff I think is more relevant now than it has ever been.

[Interviewer]

Love it. Well, Dex, thanks so much. This was fun.

[Dex Horthy]

This was a blast, dude. Thanks for. Thanks for having me on. This is great. I had a lot of fun.

[Interviewer]

I don't know about you, but I

[Host]

really enjoyed this conversation. Dex is such a big believer in gender coding, yet he's the one warning us that if you stop reading the code, you have about three to six months before your code base becomes easier to rewrite than to fix. And this comes from Firth has experience. His team built a light soft software factory, ran it, and then had to shut it down. I also like the idea of the slow loop. Loop engineering feels like a somewhat meaningless term to me. What Dex's team does is actually pretty boring. A cron job runs every night, fixes one issue or one anti pattern and opens one small pull request. The team wakes up to a code base that's a little bit better every morning and dev still needs to review and prove it. This is a practice that honestly, any engineering team could just adopt today. Finally, I really enjoyed the history lesson. The term software factory comes from a NATO conference in 1968. The idea of software used to build software with analogies to a factory is more than 60 years old. And every generation of our industry has tried to automate more of the loop of building software. AI agents are just yet one more attempt although probably the most successful one. Do check out show Notes below for the related the Pragmatic engineering deep dives that go even deeper into AI engineering and other related topics Topics. If you enjoy this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating on the show. Thanks and see you in the next one.