AI + a16z Podcast Summary

Episode: From Code Search to AI Agents: Inside Sourcegraph's Transformation with CTO Beyang Liu
Published: November 25, 2025
Host: a16z (Sam Altman, Guido Appenzeller, Martin Casado)
Guest: Beyang Liu (Co-founder and CTO, Sourcegraph)

Overview

This episode dives into the rapid transformation of software engineering at the hands of AI, focusing on Sourcegraph's evolution from code search to full-fledged AI agents. Beyang Liu offers front-line insights into this shift, why the U.S. AI ecosystem is at risk of dependency on Chinese open source models, the real impact of large language models (LLMs) on developers and the economics of AI-powered coding, the future of developer tools, and the policy landscape shaping global AI competitiveness.

Key Discussion Points

1. The New Role of AI in Software Engineering

• Efficiency vs. Enjoyment: AI boosts developer productivity in coding, but may diminish some of the "fun" and creative satisfaction of programming.

  • Quote: “You talk to some devs and they're like, you know, I've never been more productive. But coding isn't fun anymore. That's one of the things that we're trying to solve for. It's like amazing new technology. It feels like magic. Never experienced anything like this before in my life.” — Beyang Liu [00:16]

• Changing Nature of Code Creation:

  • Most development time used to be spent simply understanding large codebases rather than writing code.
  • LLMs and agents allow engineers to work at a much higher level of abstraction, acting as orchestrators rather than line-by-line authors.
  • Quote: “Now when I'm doing stuff it's more at the level of like telling the agent to make the specific edits or execute like a specific plan. And I'm really playing the role more of like an orchestrator now.” — Beyang Liu [29:10]

2. Sourcegraph’s Evolution: From Code Search to Coding Agents

• Initial Mission: Built to make sense of massive enterprise codebases via code search and navigation.

• LLM Turn: Recognized early potential to leverage LLMs for improving code search with semantic understanding and, eventually, building agentic coding tools.

• Amp — The Coding Agent:

  • Designed from scratch to capitalize on robust new LLM capabilities—especially "agentic tool use" where models can call tools and reason iteratively.
  • Quote: "When LLM sort of matured... there was a big opportunity to combine LLMs... and then our latest product is this coding agent called amp." — Beyang Liu [04:38]

• Benchmarks & Results: Amp has hit #1 on merged pull request benchmarks, often besting other solutions by focusing on agent design rather than chasing the latest model hype.

3. On Models: Open Source, Closed Source, and Chinese Origin

• Model Agnosticism: Sourcegraph’s philosophy is "agent-centric, not model-centric." The behavior, toolset, and prompts matter as much as the underlying model.

  • Quote: "We view the model as an implementation detail." — Beyang Liu [13:04]

• Open Source Model Dominance:

  • Lately, the best open-weight models for coding agents (especially for certain workflows) are of Chinese origin, not American or European.
  • Quote: “Frankly like the ones that we find most effective at agentic workloads, they're almost all, I would say they are all of Chinese origin right now.” — Beyang Liu [35:24]

• Dependency & National Strategy:

  • Concern that U.S. policy/attitude toward AI safety and open sourcing is causing a dangerous drift toward reliance on Chinese core technology.
  • Quote: “If the US open weight ecosystem doesn't catch up, we're kind of in danger of the world migrating to a world where, where most systems are heavily dependent on models of Chinese origin.” — Beyang Liu [35:24]

4. Agents, Composition & Evaluation

• From Functions to Agents: The “atomic unit” of modern software is no longer a deterministic function, but a stochastic agent—less predictable but flexible.

  • Quote: “The agent is really the analog of the function, but just updated or generalized to AI.” — Beyang Liu [15:23]
  • Quote: “It's a stochastic subroutine.” — Beyang Liu [27:58]

• Abdicating Logic & Correctness:

  • Traditional systems were all about determinism; now, developers "abdicate" full control and correctness to agents/LLMs.
  • Memorable moment: Sam Altman pressing on this philosophical discomfort with unpredictability, and Beyang’s pragmatic acceptance.

• Evaluation & Metrics:

  • Evaluations (evals) are good for regression/unit tests but problematic as optimization targets; agents can “game” whatever metric is chosen.
  • Quote: “Any eval set, like, what are you trying to capture... by definition, that means your eval set is always like lagging...” — Beyang Liu [17:30]

5. The Coding Frontier: Fast vs. Smart Agents and Product Strategy

• Agent Diversity:

  • AMP ships both a “smart” (intelligence-optimized) agent (usage-priced) and a “fast” (latency-optimized, ad-supported) agent (free).
  • Market demand is more nuanced than just “cheap vs. expensive”; developers segment across the frontier of speed, intelligence, and cost.

• Product Model Innovation:

  • Including ad-supported agent access to expand reach.
  • Quote: “We realized that hey, actually the inference costs are significantly lower... Maybe there's like a model here... We launched it and it's been growing very quickly since then.” — Beyang Liu [08:49]

• Specialization & Model Size:

  • Smaller models (billions, rather than hundreds of billions) can handle targeted tasks and edits, increasing speed and efficiency.

6. The Policy and Regulatory Landscape

• Policy’s Chilling Effect:

  • Uncertainty and regulatory patchwork (GDPR, copyright, liability) are making American companies gun-shy about releasing open models, giving global edge to China.
  • Quote: “All of the rhetoric around developer liability... My guess is... a lot of these folks are gunshots shy.” — Sam Altman [41:36]

• Strategic Mistake—The ‘Terminator’ Narrative:

  • Early US policy & discourse overly fixated on existential AI risk (AGI ‘killer robots’) rather than capability and global competition, stalling innovation and open source movement.
  • Quote: “That narrative, I think, is largely been dispelled within our circles, but I think that it's sort of taken on a life of its own in other circles, and it's made its way to some of the halls of policymaking in the us.” — Beyang Liu [38:31]

• Call for National Policy Shift:

  • Uniform, clear, pro-competition federal regulation, not a state “patchwork” or anti-competitive lock-in likely to favor giants and stall startups.
  • Quote: “The best thing we can do is to take a step back and let the free market function... avoid any sort of like anti competitive behavior.” — Beyang Liu [44:18]

Notable Quotes & Memorable Moments

• On the creative challenge AI brings:
  “My job has changed so much in the past year... Now when I'm doing stuff it's more at the level of like telling the agent to make the specific edits or execute like a specific plan. And I'm really playing the role more of like an orchestrator now.” — Beyang Liu [29:10]

• On unpredictability in software infrastructure:
  “This is the first time in computer science I can think of where we've actually abdicated correctness and logic to us... But now we're like, figure out this problem for me.” — Sam Altman [00:00, 15:20]

• On the U.S.’s paradoxical position in AI:
  "We were the first with open source models, we had Llama three and now we like, like listen, you're using Chinese models. Yeah, like where are the US models?” — Sam Altman [41:36]

• On policy stagnation and risk tolerance:
  “If you've been sold on this sort of Terminator style narrative, that's going to put you in a very different mindset... in your tolerance for open sourcing model weights.” — Beyang Liu [39:13]

• On agents as the new unit of composition:
  “The agent is really the analog of the function, but just updated or generalized to AI... It's a stochastic subroutine.” — Beyang Liu [15:23, 27:58]

Timestamps for Key Segments

• Opening thoughts on AI’s impact on correctness and developer enjoyment: [00:00–00:45]
• Beyang Liu’s background; Sourcegraph’s origin story: [02:54–04:38]
• Transition from code search to coding agents (AMP): [04:38–08:24]
• The smart/fast agent strategy, ad-support economics: [08:32–10:17]
• Agent-centric vs. model-centric philosophy: [13:04–14:52]
• Unpredictability and software as stochastic agents: [15:20–17:22]
• Market segmentation across intelligence, speed, and cost: [19:25–21:15]
• Open source models and the global AI race: [21:29–24:28, 35:24–36:09]
• Strategic policy mistakes: U.S. AI narrative and regulation: [37:29–39:06, 44:06–45:18]
• The future of software engineering and human roles: [28:14–33:10]

Conclusion: The Takeaways

Sourcegraph’s journey embodies both the promise and peril of AI in software development: more powerful tools make developers superhumanly productive but risk eroding the very act of creation they once enjoyed. The U.S., having led the AI revolution, now risks ceding open model leadership to China due to self-imposed regulatory headwinds and lingering doomer narratives. Beyang Liu urges policymakers to re-embrace openness and competition, and invites developers and founders to prepare for a world where agents, not functions, are the new building blocks of software—and human creativity remains the essential bottleneck.