The MAD Podcast with Matt Turck: "Everything Gets Rebuilt: The New AI Agent Stack"

Guest: Harrison Chase, Co-founder & CEO, LangChain
Host: Matt Turck
Date: March 12, 2026

Episode Overview

In this high-energy, illuminating episode, Matt Turck and Harrison Chase explore the evolution and current frontier of AI agents—how modern infrastructure is supporting their rise and what this means for developers, enterprises, and the wider AI ecosystem. From LangChain’s open-source roots to cutting-edge architectures like DeepAgents and the dynamics between models and frameworks, they break down technical concepts into accessible insights, peppered with inside stories and practical advice.

Key Themes & Discussion Points

1. The Evolution of AI Agents

• Early Limitations:
  • First agent frameworks (like the original LangChain) implemented the idea of running LLMs in a loop to call tools—a concept proposed in early papers (e.g., React)—but initial models weren’t reliable in real-world use (01:58).
• Breakthrough Moment:
  • "I think two things basically happened. The models got better, but then also we started to discover these primitives of a harness that would really let the models do their best work. And we saw an explosion of people building agents."
    — Harrison Chase [00:00, repeated at 03:48]
• Agent Types:
  • Two broad categories have arisen:
    • Conversational Agents: For customer support/chat use cases, requiring low latency and minimal tool use.
    • Long Horizon (Coding) Agents: For planning, code execution, and managing complex workflows—now more feasible due to better models and infrastructure [04:04].

2. Coding Agents Dominate

• Why Coding?
  • "Code is really useful. You can use it to parse text files, do things programmatically… all the big model labs have been RL-ing code into those models. That is the stuff that works the best."
    — Harrison Chase [04:04]
• Convergence:
  • Conversational and coding agents may merge as systems allow synchronous conversational agents to kick off and manage background task agents [05:26].

3. The Harness: Core of Agent Architectures

• Framework vs. Model – Who Wins?
  • "I think the harness is the most important thing... it was the secret sauce of what made [end-user agent products like Manus] work."
    — Harrison Chase [06:51]
• Defining 'Harness':
  • "How the model kind of interacts with its environment is what I would say. It’s the set of tools that it has. Other things that the harness does is like take advantage of prompt caching, context compression..."
    — Harrison Chase [08:29]

4. Anatomy of a Modern AI Agent (DeepAgents & Peers)

• Core Components:
  • System Prompt: Procedures/instructions for the agent’s overall behavior [10:18]
  • Planning Tool: Letting agents break work into lists of tasks, track status (though plans tend not to be rigidly enforced anymore) [11:18]
  • Subagents: Isolated context windows for parallel/independent tasks—introduces benefits but also communication complexity [13:13]
  • File System: Mechanism for LLMs to manage/read/write long-term or large context without blowing up token windows—used for summarization, tool results, etc. Can be a real or virtual (e.g., DB) file system [15:31, 17:22]
  • Skills: Files (often with markdown instructions/scripts) that agents can load on-demand—'progressive disclosure' [19:13]

5. Memory, Context, and Compaction

• Context Engineering:
  • Use of file systems and summarization for managing what the agent "remembers" on a task [15:36, 20:28]
• Types of Agent Memory:
  • Short-term: Active context in current threads/conversations.
  • Long-term:
    • Semantic: Information/facts, akin to retrieval-augmented generation (RAG).
    • Episodic: Past interactions/conversations.
    • Procedural: How-to instructions or agent configurations—often embodied as files the agent can even update itself [23:15].
• Context Compaction:
  • Summarizing history to compact large contexts, with the innovation of agents being able to trigger their own compaction [20:18, 22:44].

  "Compaction happens when you basically build up a bunch of context and you want to condense it. Models can’t handle infinite context."
  — Harrison Chase [20:28]

6. Agent Ecosystem, Observability & Enterprise Challenges

• Stability & Investment:
  • Low-level infrastructure (observability, evals, sandboxes, deployment) is where investment is safest; harness architectures are still volatile and evolving [28:01].
• Observability:
  • "You don’t really know what the agent will do until you run it... Observability becomes more important and more different than compared to software."
    — Harrison Chase [40:25]
• Memory, Evals, and Prompt Optimization:
  • Tightly linked, as agents leverage feedback and memory to improve over time [42:14].

7. Sandboxes & Security in Agent Execution

• Why Sandboxes?
  • For safe code execution—especially crucial as agents increasingly write and run their own code, often with access to sensitive credentials/APIs [29:54].
• Security Practices:
  • Isolating API keys and agent access to prevent prompt injection and credential leaks [32:47].
• Deployment Strategies:
  • Some run the agent inside a sandbox, some call the sandbox as a tool; both approaches are common [31:23].

8. LangChain’s Journey & Products

• Origins:
  • Harrison’s experience at Kensho (world-class engineering culture and alumni) → Robust Intelligence → Early meetups seeing LLM patterns → Launched LangChain, seeing quick adoption [33:57].
• Evolution:
  • From simple abstractions and runbooks (v0) to orchestration (Langgraph) and now robust harness frameworks (DeepAgents) with production-grade agent runtime features [37:42].
• Langsmith (Commercial Product):
  • "The main thing in there is what we call observability... The biggest part of Langsmith is what we call observability."
    — Harrison Chase [40:25]
• No-Code Platform:
  • Recent addition enables anyone (even non-coders) to assemble agents by configuring prompts, tools, and skills [43:43].

Notable Quotes & Timestamps

• On Harness Primitives:

  "We started to discover these primitives of a harness that would really let the models do their best work."
  — Harrison Chase [00:00, 03:48]

• On the Importance of Coding Agents:

  "A lot of them end up looking like coding agents. Code is really useful. The models are trained on code... so that's the stuff that works best."
  — Harrison Chase [04:04]

• Harness vs. Model:

  "Manus was an end user product, but their harness was so good... that was the secret sauce of what made it work."
  — Harrison Chase [06:51]

• On File Systems:

  "We use file systems to offload large tool call results... summarize context, manage LLM context. It lets the LLM manage its own context window.”
  — Harrison Chase [15:36]

• On Memory:

  "Memory is super important... Semantic, episodic, procedural — procedural is literally the configuration of the agent."
  — Harrison Chase [23:15]

• Advice to Enterprises:

  "The most important thing is building up the instructions and the tools themselves. Those are always going to be valuable, no matter how you expose them."
  — Harrison Chase [25:33]

• On Platform Vision / Roadmap:

  "We want to build the platform for agent engineering... but observability will be the core pillar of it that we're going to be best in class at."
  — Harrison Chase [44:56]

• Where Differentiation Lies:

  "A lot of the differentiation is in the instructions and the tools and the skills... for your domain. That’s the stuff that won't change."
  — Harrison Chase [45:52]

Timeline of Key Segments

• 00:00-03:48 — The evolution of agents: from reactive loops to sophisticated harnesses.
• 04:04-06:26 — Agent types: conversation vs. coding/long horizon; convergence trends.
• 06:51-10:00 — Framework/tools vs. models; importance and nature of 'the harness.'
• 10:00-19:13 — Deep dive into agent ingredients: prompts, planning tools, subagents, filesystems, skills.
• 20:18-23:01 — Managing and compacting context; agent-driven compaction.
• 23:15-25:33 — Memory types and implications for agent design/enterprise deployment.
• 28:01-29:54 — Stable infra layers: observability, sandboxes, evals, deployment.
• 29:54-33:32 — Sandboxes: use cases, deployment approaches, security concerns.
• 33:57-37:24 — Harrison’s backstory, origins and mission of LangChain.
• 37:42-43:43 — The journey from LangChain v0 to Langgraph and DeepAgents, launch of LangSmith and no-code platform.
• 44:56-46:32 — Future vision, doubling down on observability, where differentiation in AI agents is lasting.

Tone and Language

• Candid, insightful, and technical but accessible.
• Frequent use of analogies and real-world developer experiences.
• Emphasis on the “bleeding edge” nature of the space, with honest uncertainty about future architectures but clear practical lessons.
• Respectful, collaborative, energetic rapport.

Summary for New Listeners

If you want to understand what it takes to build, deploy, and scale modern AI agents—and why nearly everything in the stack is actively being rebuilt—this episode delivers a blueprint rooted in field experience. Harrison Chase, whose work at LangChain is at the heart of the agent infrastructure revolution, deconstructs both technical architectures and the shifting landscape of tools, memory, observability, and deployment. The takeaways are not just about tools, but about the knowledge, processes, and bespoke skills that make differentiated, future-proof agents possible.

Recommended Next Steps:

• For technical founders and enterprise builders: Invest in developing domain-specific instructions, tools, and skills—they’re the real long-term differentiators.
• For those new to agent infra: Explore open-source frameworks like LangChain; try no-code harnesses for rapid experimentation; keep a close eye on observability and security best practices.
• For everyone: Prepare for rapid evolution. The agent stack may keep changing, but the core competencies of agent memory, context and workflow orchestration are here to stay.