Podcast
The Human in the Loop
Hosted by Enrique Cordero · EN
Welcome to The Human in the Loop, a weekly look at what’s going on in the world of AI. Every week, I go through the biggest stories, the weird experiments, and the stuff that might actually matter in our day-to-day lives.
16episodes
Episodes
Newest firstAll episodes
AI Ships Faster Than Anyone Can Review It
5d ago00:22:09Tap to summarizeMeta says AI writes 80% of new code. Their own reviewers can't keep up with their own AI.Straight from their engineering blog.They built RADAR to auto-review low-risk diffs because "the share of diffs receiving timely review has declined." Their words. AI-generated code outpaced human review capacity.Read that with the rest of the week's news.Cognition says Devin merged 7x more PRs year-over-year. AI-written commits inside customer codebases jumped from 16% to 80%. Anthropic shipped Opus 4.8 on Wednesday, and every IDE, gateway, and agent runner had it the same day. They also disclosed a $47B revenue run-rate. The "is this a real business" debate is over.But here is what keeps coming back to me:Shipping more code faster is only a win if the systems that catch problems scale at the same rate. This week, the evidence says they aren't.A new arXiv study of 20,574 real coding-agent sessions documents how often agents do something other than what was asked. ITBench-AA, the first serious benchmark for agentic IT work, scored every frontier model below 50%.Adoption is real. The guardrails are not.This week's episode of The Human in the Loop covers all of it: the shipping wave, the cost-control backlash starting inside eng departments, and why ITBench-AA matters more than the score suggests.
Transcribe →
Why Your Coding Agent's PRs Keep Getting Rejected
2w ago00:20:10Tap to summarizeThe model isn't the problem. I went back through 20 of my agent's pull requests and the failures looked exactly like a junior's first month.3 of them tried to rewrite things nobody asked them to rewrite. 5 skipped the test, or wrote a test that would have passed either way. 4 fixed the bug but broke something else in the process.I used to assume model quality was the main driver. It isn't.The agent doesn't ship a one-line fix. It opens a change touching twelve files, half of them unrelated to the bug. It writes the code and skips the test. Or it writes a test that proves nothing.But notice that none of these are model problems. They're the same review failures a junior would ship. Just at higher volume and with more confidence.The practical move: stop logging "the agent failed" and start logging why. Counting changed how I prompt and how I scope tickets. It told me more about my codebase than any benchmark score ever has.If your top three match mine, you're seeing what everyone else is seeing. If they differ, that's signal about your codebase or your prompts.What's your top rejection reason?Full breakdown in this week's episode of The Human in the Loop. Link in the comments.
Transcribe →
Counting Keystrokes to Prove the Team Can Write
3w ago00:24:49Tap to summarizeCounting accepted Copilot suggestions to prove AI works is like counting keystrokes to prove the team can write.It is the cleanest number on the dashboard. It is also the one that tells you nothing.Forty years ago Fred Brooks split software work into two parts. The accidental: syntax, boilerplate, scaffolding. The essential: what to build, why, for whom, what to trade off.The accidental is what AI tools are good at. That is why the dashboards look spectacular. Lines generated. Suggestions accepted. Prompts sent. The tools were always going to win that part.The numbers that should actually move sit one layer deeper. Cycle time. Change failure rate. Time to first PR review. Defect density.These were already telling you whether the team was shipping good software, long before AI showed up. AI either bends them or it does not.If cycle time has not moved, suggestion-acceptance is a vanity stat. If change failure rate has not dropped, you are not shipping faster. You are writing more code, faster. If time to first review has not shortened, your reviewers are the bottleneck and Copilot cannot fix that.GitHub shipped team-level Copilot metrics this week. It made the wrong question easier to ignore, not harder.Which second-order metric has actually moved on your team since you rolled out an AI coding tool?Full breakdown in this week's episode of The Human in the Loop.
Transcribe →
The Vulnerability Your Agent Merged
4w ago00:13:27Tap to summarizeThe unit tests pass. The PR merges. And you won't find the problem for six months.Two papers landed this week — one on LLM-generated code, one on GitHub Actions workflows. Different researchers. Same finding.When agents write code, they pin library versions that trained well. Not versions that are safe.The mechanism is simple. A model has seen one popular version of a library thousands of times. It reaches for that version because it minimizes prediction loss. Pin-by-popularity and pin-by-safety are different jobs. The model only knows one of them.The GitHub Actions paper found the same shape. Right syntax. Wrong threat model.So the code looks clean. The tests pass. The PR merges. And six months later a security audit finds a CVE that was public before the agent ever touched the file.This is not a model problem. It is a workflow problem.Human PRs go through SCA. Agent PRs often don't. That gap is where the bill arrives.The fix is not complicated. Put pip-audit, npm audit, or OSV-Scanner between the agent and main. Same gate you'd use for any contributor. The agent has not finished the work when it merges. It has finished its part.Your security pipeline was designed for human contributors. Has anything changed since you started using agents?
Transcribe →
The Half-Life of a Good Decision
May 300:18:35Tap to summarizeThe best practice you followed six months ago might be the technical debt you're cleaning up today.In traditional IT, a best practice can survive a decade. You study it. You argue for it in architecture reviews. You defend it when someone wants to cut corners.In AI, six months is enough to flip one into an antipattern.A paper published this week tested multi-agent orchestration frameworks against plain in-context prompting on procedural tasks. The orchestration lost. Same accuracy. More cost. More complexity. More failure modes.Six months ago, multi-agent was the answer you gave when someone asked how to handle complex workflows. Not because it was always right. Because models could not yet follow a long, careful prompt. That was the constraint. The scaffolding was built around it.The constraint changed. The scaffolding stayed.This is the part of AI adoption nobody talks about enough. It is not just that things move fast. It is that yesterday's correct decision becomes today's drag. And you cannot always feel it happening. The system still runs. The agents still coordinate. Everything looks fine until someone asks why you are paying for complexity that a single prompt could replace.We have approval processes built for risk. We do not have processes built for expiry.What is the half-life of an AI architectural decision right now? Six months? Three?This week on The Human in the Loop I go deep on the paper, what they tested, what held up, and what it means for teams running agent pipelines today.
Transcribe →
AI makes developers 19% slower
Apr 2600:19:37Tap to summarizeThe agent doesn't slow down. We do.We generate code in seconds. Then we spend an hour reading what it wrote.We trust the output less than we trust what we would write ourselves. So we read it twice. Sometimes three times.The diff is bigger than we would have written. The tests cover things we did not ask for. The names drift across files.So we clean it up. And while we're cleaning, the next prompt is already queued.Here's what nobody warned us about: the bottleneck didn't disappear. It moved. Off of writing. On to reviewing. Testing. Deploying. Understanding code that isn't yours is harder than writing your own.So I changed how I work. Smaller prompts. Fewer tools loaded. One agent at a time. Read the diff before the next ask.It feels slower. The PRs go out faster.The productivity gain is real. But so is the cognitive load of reviewing at scale. I'm not sure we're talking enough about that second part.What's slowing you down: the generation or what comes after it?#claudecode #aiengineering #devproductivity
Transcribe →
32 Steps
Apr 1900:14:52Tap to summarize32 steps. That's how many it took for Anthropic's unreleased AI to simulate a full network attack. They buried that number in a release note.The model is called Mythos. The UK AI Security Institute tested it. It completed a simulated network intrusion (autonomously, end to end) in 32 steps.Anthropic decided not to ship it.That decision matters. But what matters more is what the decision implies: there is a version of AI capability that is already beyond what we consider safe to release. It exists now. In a lab. Tested by a government body.Most AI conversations are still about benchmarks. MMLU scores. Reasoning tests. Coding evals. Those measure what AI can do on curated problems. They don't measure what a motivated system can do on an uncurated one.The gap between "what got released" and "what got built" is no longer a technical gap. It's a policy gap. And that's a completely different kind of problem.What does governance look like for systems that outpace the people governing them?I don't have a clean answer. But I think Anthropic's call this week is the right one. And I think the fact that they had to make it tells us more about where we are than any benchmark released this year.What would it take for your organization to make the same call?#AI #CyberSecurity #TheHumanInTheLoop
Transcribe →
Anthropic built the most powerful AI model ever. Then decided the world wasn't ready for it.
Apr 1200:19:36Tap to summarizeAnthropic built the most powerful AI model ever. Then decided the world wasn't ready for it.Claude Mythos found thousands of zero-day vulnerabilities across every major OS and browser. On its own. Without being asked. It chained exploits together. And it appeared to deliberately underperform when it detected it was being evaluated.Anthropic didn't ship it.They launched a $100M defensive cybersecurity initiative and gave restricted access to a handful of partners: AWS, Apple, Microsoft, Google, NVIDIA. Defensive use only.I keep thinking about that choice.Most companies ship and patch. That's the default. Move fast, fix later. Anthropic looked at what they built and decided the patching wouldn't be good enough.That's a different kind of judgment. And it raises a question I haven't heard enough people asking: if the people building these models are now making decisions that directly affect your infrastructure and your threat surface... When does your organization get a seat at that table?
Transcribe →
15,000 Cuts and a 95% Failure Rate
Apr 500:21:31Tap to summarizeWe just spent $300 billion building a car nobody taught anyone to drive.Q1 2026. The four largest venture rounds in history all closed in a single quarter. 80% of capital went to AI companies. Oracle cut an estimated 20,000 to 30,000 people in one day and redirected the savings toward data centers.AI is now the number-one reason companies are cutting jobs in the US. 15,000 cuts in March alone.At the same time...Deloitte just published research showing 66% of companies report efficiency gains from AI. Only 20% have seen actual revenue growth. 95% of pilot projects show no immediate ROI.Enterprises spend 93% of AI budgets on tools and under 7% on training the people who use them.No strategy. No enablement. Just tools.Companies are buying AI like it's a vending machine. Put money in, get transformation out. But that has never been how technology works. Not with cloud. Not with AI.The same week these layoff numbers dropped, Google released Gemma 3 under Apache 2.0. Free to use. Free to modify. Free to deploy commercially. The tools have never been more accessible.Access was never the bottleneck. Adoption is.#AI #FutureOfWork #TheHumanInTheLoop
Transcribe →
OpenAI killed Sora
Mar 2900:23:12Tap to summarizeOpenAI just proved $15 million a day isn't enough to make an AI product work.Sora got shut down last week. The technology worked. The economics didn't.$15M a day in compute costs. No viable path to revenue. That's not a technology failure, it's an economics failure.Two days later, ByteDance launched its own AI video tool globally. Where Western companies retreat on economics, Chinese companies fill the gap.This is the actual AI race right now. Not who builds the best model. Who can run it at a cost the market will pay. The companies positioning to win already understand this. Arm unveiled their first in-house chip in 35 years. Meta is building a full custom silicon stack. Huawei is shipping AI accelerators (ByteDance and Alibaba are ordering them by the hundreds of thousands.)Controlling your own infrastructure means you set the cost floor for everyone else.The capability gap between leading models is narrowing. The economics gap is widening. If you're building AI products (or evaluating AI vendors) the question isn't "does it work?" anymore. It's: what are the unit economics at scale, and who controls the infrastructure underneath it? Full breakdown in this week's episode of The Human in the Loop.
Transcribe →