Who's Coding Now? AI and the Future of Software Development

Summary

---

Podcast Summary: Who's Coding Now? AI and the Future of Software Development

AI + a16z Podcast | May 16, 2025
Host: Derek
Guests: Guido Appenzeller, Matt Borenstein, Yoko Lee

---

Episode Overview

This episode of AI + a16z delves into how generative AI is reshaping software development—far beyond simple code autocompletion or “Stack Overflow on steroids.” The discussion features a16z partners Guido Appenzeller, Matt Borenstein, and Yoko Lee, who examine the current and future impact of AI coding tools, the transformation of developer workflows, market opportunities, the future of computer science education, the evolution of programming abstractions, and the bifurcation between “vibe” coding and enterprise software.

---

Key Discussion Points & Insights

1. AI Coding as a Major Use Case for Generative AI

• Market Size & Adoption

  • Coding with AI is plausibly the second-largest AI market, possibly even surpassing general chatbot uses, considering its homogeneity and size ([01:59]-[02:26]).
  • “If you think about this, we have 30 million developers worldwide...That's $3 trillion worth of value.” — Guido Appenzeller [05:04]

• Why Developers Lead AI Adoption

  • Developers “solve their own problems first” and are early adopters, leveraging AI to boost productivity and build their own tooling.

• Verification and Market Fitness

  • Coding is a “verifiable problem” with clear inputs and outputs, making it well suited for automation and AI assistance.
  • Yoko Lee highlights: “You can verify a coding function, input, output are very clear compared to user preference and all the other problems.” [04:04]

2. Shift in Developer Workflows

• Evolution of AI Coding Tools

  • From copy-pasting code from ChatGPT as a replacement for Stack Overflow ([07:52]-[08:28]) to more advanced setups:
    • IDE-integrated tools (e.g., GitHub Copilot, Cursor)
    • Autocomplete at line/paragraph level and interactive chat interfaces
    • AI agents that can leverage external documentation and world knowledge (e.g., automated doc fetching with Firecrawl) ([10:43]-[12:57])
  • “It has changed a lot over the last even six months how I use these models.” — Guido [07:52]
  • “There’s more integration to the real world now.” — Yoko [11:42]

• Collaborative Coding as Dialogue

  • Back-and-forth between developer and model to refine specifications and clarify requirements: “The model is almost a sparing partner to think through the process.” — Guido [09:26]
  • Contextual coding includes personal coding guidelines, preferred paradigms, etc.

• Pain Points and Errors

  • AI tends to “hallucinate” APIs or functions that don’t exist, and adapts slowly when corrected ([17:06]-[17:26]).
  • Agent-to-agent communication can be unpredictable, sometimes ignoring or replacing returned results.

3. Who Benefits: From Juniors to “Neckbeards”

• Democratization vs. Depth

  • AI tools help junior or “vibe” coders get up to speed and can enable non-developers to create software.
  • Senior/architect-level engineers may be less impressed, especially when solving novel or deeply specialized problems requiring extensive custom context ([15:26]-[17:06]).

• Limits at the Cutting Edge

  • "If you are the first person on the planet to write code that solves a specific problem, there's just zero training data out there. I think it'll always be very hard… That is 0.01% of all software development." — Guido [17:45]

4. The Changing Face of Coding Education

• Prompt Engineering vs. Fundamentals

  • Will prompt management replace deep technical knowledge? The consensus: “To truly optimize, troubleshoot and scale out even the simplest application, you must be able to look deeper than a surface level abstraction.” — Derek [18:26]
  • The analogy: “Everybody can build a shack, but you cannot build a skyscraper.” — Guido [19:27]

• Historical Cycles

  • Comparison to the rise of WordPress/blogging and Excel/accounting—higher-level abstractions create new jobs and change skill sets but don’t erase foundational knowledge.
  • "Explaining the problem statement, algorithmic foundations, architecture, and data flow is getting more important... The nitty gritty coding, what's the clever way to unroll the for loop? That's a niche now." — Guido [21:26]

• Stratification of Skill Levels

  • New “vibe” coders can build useful tools; mastering underlying abstractions remains important for optimization, scale, and deeper system-level work ([26:46]-[28:05]).
  • "If you are someone who aspire[s] to serve something at scale... it's really hard to get away without knowing the underlying knobs." — Yoko [26:46]

5. The Future of Programming Languages & Abstractions

• AI as a New (or Not So New) Abstraction

  • Could AI itself become the next higher-level programming language or paradigm? ([24:25]-[25:06])
  • “If I can basically define certain things in human language in an efficient way...that could change a lot of things.” — Guido [00:26/24:25]

• Persistence of Formal Languages

  • “Formal languages exist for a reason...there has to be some high-bandwidth expressive way for a person to design software. I have a hard time seeing Python going away.” — Matt [25:47]

• Hybrid Models and Meta-Specs

  • Possibility of mixing natural and formal specifications; AI might bridge the gap but precision, reliability, and optimization require deeper knowledge ([28:05]-[29:56]).

• Gaps for “Vibe Coders”

  • Users see the code AI generates but may not know how to modify or extend it, reflecting a need for more transparent interfaces ([30:07]-[31:05]).

6. AI and Legacy Code Migration

• Translating vs. Understanding
  • AI can help port old code (COBOL, mainframe) but context and intent are often lost. The best results come from extracting specs first, then using AI to reimplement ([33:07]-[34:17]).
  • “If you just try to translate [legacy code], you pick up many of the idiosyncrasies of that language...The most efficient way...is to actually go first and try to create a spec. Use the AI to create a spec from that code.” — Guido [33:19]

7. Reliability, Uncertainty, and Software “Certainty”

• Non-deterministic Behavior & Risk

  • Adding AI as a core primitive increases unpredictability—small input changes can yield huge output changes (chaotic systems). ([37:49]-[38:07])
  • “It's a chaotic system.” — Matt [37:49]
  • AI introduces new failure modes (e.g., unpredictable text input exploitation), requiring new design patterns and mindset.

• Changing Trust Metrics

  • Sometimes, software can't guarantee never making certain errors—so standards shift from never failing to “acceptably rare,” such as aiming for error rates below that of trained humans ([38:25]-[39:13]).

8. Is There a “Narrow Waist” (Standard) in AI?

• Prompt as the New Narrow Waist
  • In the internet stack, the protocol (IP) formed a universal abstraction layer. In AI, could the “prompt” play that role? ([39:33]-[41:57])
  • “Big tech cycles are built on abstractions... You can now express and steer the model with a prompt. A fairly mediocre Python programmer can leverage a very powerful LLM just by prompting it.” — Guido [40:02]
  • Prompting is not a formal language; emerging practices include structured prompts, JSON modes, and possible future formalization ([41:19]-[42:30]).

9. Bifurcation: Vibe Coding vs. Enterprise Coding

• Blurring the Lines
  • The distinction between “vibe” coding (high-level, outcome-focused, unconcerned with implementation) and classical coding (careful choice of stack/tools) may persist, but the tools and mindsets are overlapping.
  • “I can totally see enterprise users doing Vibe coding. And that's a compliment.” — Yoko [44:02]
  • Enterprise solutions may still demand more formality and reliability, but utility is driving even serious users to embrace more ambitious AI-assisted workflows.

---

Notable Quotes and Memorable Moments

On AI as a Language/Abstraction:

"If I can basically define certain things in human language in an efficient way and...use this directly as input for a compiler, that could change a lot of things."
— Guido Appenzeller [00:26]

On Productivity Uplift:

"If we can double the productivity of a developer for all developers in the world, that's $3 trillion worth of value. That's the value of Apple Computer or something like that."
— Guido Appenzeller [05:04]

On the Real Impact of AI Coding:

"This is so much bigger than Stack Overflow on steroids."
— Derek [00:40]

On Who Can Now Code:

"We're not like priests of the computer...ordinary people should be able to control computers in direct ways, not just in pre-baked programs."
— Matt Borenstein [19:07]

On the Limits of AI:

"If you are the first person on the planet to write code that solves a specific problem, there's just zero training data out there."
— Guido Appenzeller [17:45]

On Education and Abstractions:

"If you're operating on one abstraction, you need to learn the abstraction lower than where you're operating from."
— Paraphrased from Martin via Yoko [20:21]

On Uncertainty and Chaotic Systems:

"It's a chaotic system."
— Matt Borenstein [37:49]

On the Future of Prompts:

"Will we ever have a formal prompting language? Maybe. I think there are some overpriced Stanford PhDs working on that problem."
— Matt Borenstein [41:22]

On Vibe Coding in the Enterprise:

"I can totally see enterprise users doing Vibe coding. And that's a compliment."
— Yoko Lee [44:02]

---

Timestamps for Important Segments

• Biggest AI Markets & Coding’s Size: [01:59]-[05:04]
• Developer Workflows & Stack Overflow Replacement: [07:52]-[10:43]
• IDE Integration and World Knowledge: [10:43]-[12:57]
• Senior Engineers, Limits, and Creativity: [15:26]-[17:45]
• The Role of Education & Abstractions: [18:49]-[22:46]
• Programming Languages and AI as Abstraction: [24:25]-[28:05]
• AI Coding for Non-developers ("Vibe" Coding): [29:56]-[31:05]
• Legacy Code and AI Migration: [33:07]-[34:42]
• Certainty, Non-determinism, and Design Patterns: [35:56]-[39:13]
• Prompt as the Narrow Waist / Formalization: [39:33]-[42:30]
• Vibe Coding vs. Enterprise Coding: [43:13]-[44:09]

---

Closing Insights

While AI is unlocking vast new opportunities in software development—from boosting productivity and lowering barriers to entry, to reshaping how old software is rebuilt—it is also surfacing new challenges: ensuring reliability, managing abstraction layers, and redefining essential technical skills. Rather than replacing foundational computer science, AI is prompting a shift toward higher-level problem formulation, systemic thinking, and new forms of human-computer collaboration, setting the stage for both “vibe” and large-scale, enterprise-grade code to thrive in parallel.

---

Transcript

[Guido Appenzeller]

Today, I think AI is not just a higher level language abstraction or something like that, but could it become one over time?

[Matt Borenstein]

That's the question. Do you think?

[Guido Appenzeller]

Yes, I think it could. If I look at a classic, say, compiler design or in programming languages, if I would have LLMs as a tool, I would probably think very differently about how I would build a compiler.

[Yoko Lee]

Yeah.

[Guido Appenzeller]

And I don't think we've seen that work its way through yet. But if I can basically define certain things in human language in an efficient way and maybe in a sort of tight enough way that I can use this directly as input for a compiler, that could change a lot of things.

[Derek]

Thanks for listening to the A16Z AI podcast. I'm Derek and today we have another great episode digging into how generative AI already is and will continue to impact our digital lives. A few weeks ago we featured a 16C partners Guido Appenzeller, Matt Borenstein and Yoko Lee discussing what's up with AI agents, and we brought them back for this episode to discuss what AI coding tools mean for the future of software development. It is, as you'll hear, a topic that entails so much more than just five coding. The three of them discuss everything from the relative market size for AI generated code, presumably quite large for a capability that could juice developer productivity to the tune of trillions of dollars, to how prompt based programming might affect the value of classical computer science education. This is so much bigger than stack overflow on steroids. So get ready for the full discussion, starting with the question of why coding has become such a big use case for generative AI after these disclosures. As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. For more details, please see hey16z.com disclosures.

[Matt Borenstein]

We'Re pretty sure it's the second biggest AI market right now. Correct me guys if I'm wrong, but you know, consumer, Pure Chatbot, I think is number one, and I think coding is number two. Just purely looking at the numbers.

[Yoko Lee]

But consumer is the aggregation of a lot of.

[Matt Borenstein]

Exactly right.

[Guido Appenzeller]

That's how I define market. I think there's. You can make an argument it's the number one, actually. If you look at really homogenous markets.

[Yoko Lee]

Is coding bigger than companions? Yes, I think so, yeah.

[Guido Appenzeller]

Yes, at this point.

[Matt Borenstein]

You think so? That'd be interesting.

[Guido Appenzeller]

It probably depends how you classify something like ChatGPT, which to some degree is used for companionship.

[Matt Borenstein]

So a Large portion of ChatGPT usage I think now is companionship.

[Guido Appenzeller]

I think that's right, yeah.

[Matt Borenstein]

Well, in the end, is a person's motivation greater to build something or to find love? I think it may be neck and neck. One thing that's very unique about AI coding that's sometimes underappreciated is this was actually an existing behavior in a couple of ways. First, people were already going somewhere to look for help, which we mentioned earlier, is Stack Overflow, for the most part. So there was already sort of this muscle people were exercising when they hit a problem they couldn't solve to go find the information on the Internet. And this is really just a much better form of that. You know, there are all these jokes that Stack Overflow is actually writing most of the code for the last, you know, X number of years. But a lot of that may just shift to AI models, right?

[Guido Appenzeller]

It's not clear if that was a joke or not.

[Matt Borenstein]

Yeah, yeah, maybe, maybe not a joke. Also, there was this thing called GitHub Copilot, right? They did this really foundational work to start to transition people off of that sort of Stack overflow use case to using AI models. And I think companies like Cursor have just done a much better job with that now. And so you have this fairly unique thing that you're actually taking advantage of an existing user behavior, an existing market, and like selling a great product into it.

[Guido Appenzeller]

I think there's one other aspect which is, look, if you're a developer and you have access to latest AI technology, the first problem you solve are your own problems, right? So I think it's developers just, that's the problems they understand best, that's the problems they face every day. And so they build infrastructure for themselves to use.

[Yoko Lee]

And developers are always early adopters for new technologies just because, like naturally they like to tinker, they like to configure new tools and they're lazy. So anything that actually increase the productivity, they will adopt. But I also think coding market is doing so well also because it's somewhat verifiable problem. You can verify a coding function, input, output are very clear compared to user preference and all the other problems. And you can also reframe a lot of different problems into coding. So I would even argue that some of the art generation is a coding problem. People will not like it. But historically we always use machine learning before we call it AI in Adobe Photoshop, and that's somewhat coding. You can really map the trajectory of brushes. That's coding. Vector generation is also coding. So I think the beauty of code is that you can really remodel a lot of the real world problems and make it into very machine consumable formats.

[Guido Appenzeller]

I think there's one other aspect which is it's a massive market. If you think about this, we have 30 million developers worldwide. Let's say average value created by a developer is $100,000 a year. That's $3 trillion. I think if I look at the data we've seen from some of the large financial institutions, they're estimating that the increase in developer productivity from just a vanilla copilot deployment is something like 15%. My gut feeling is we can get that substantially higher. Right. Let's assume we can double the productivity of a developer for all developers in the world. That's $3 trillion worth of value. That's the value of Apple Computer or something like that. It's an incredible amount of value that we unlock there. So it's a massive market. And I think it was when there was a good blog debate on if we're overinvested in AI and I think back then the number was $200 billion annual investment, something would ever recuperate. Here we have a way to recuperate $3 trillion. So that makes the 200 billion look like peanuts. I think what might also be how to work is it's an easy market to capture because developers understand it and it's a very, very big market. Something potentially it might be the first really large market for AI in terms of fabric.

[Matt Borenstein]

Yeah, no, that's a great point. Software and software developers create a huge AM value at every company and every organization around the planet now. And this is sort of a shortcut into, into this sort of core capability. So, so that, that makes a lot of sense. It there's almost a bootstrapping effect to your point too guido. Because if you count not only the productivity gains but like the brand new things that are being created with these models, like you kind of, I think can see a cycle starting where you're getting better and better AI coding models which allow you to create better, better software, you know, better new Net, new AI applications also.

[Derek]

Okay, so AI generated code will generate a lot of value going forward, but how much has it already changed the day to day experience of programming once.

[Guido Appenzeller]

The AI revolution has run its course? Do we have any idea how the job of a software developer will look like it'll look different from today? I think we're seeing that when I'm writing code today, I'm writing specifications, I'm having discussions with a model about how to implement something for easy features. I can actually ask it to implement a feature and just reviewing it. How will the process be different? Will there still be the same stages? Will we all turn into basically product managers that write specifications and then the AI writes the code and you just step in occasionally to debug or what's the end state here? Do we have any idea at this point?

[Yoko Lee]

Or we all become QA engineers and we test if it's to the spec.

[Guido Appenzeller]

There we go.

[Matt Borenstein]

That is kind of ironic, right? We all got into this to avoid being QA engineers. I like what you're saying, Guido. Maybe we could each just talk a little bit about how we use AI models in coding right now. Like, can you share a couple of stories about how this has changed your coding workflows totally.

[Guido Appenzeller]

And I mean, boy, I'm not sure I'm maybe the person coding least here, but I think the most interesting insight is it has changed a lot over the last even six months how I use these models. It used to be that you take your favorite chatgpt or something like that, and you give it a prompt and out comes a program, and you copy that into your editor and you see if it works.

[Matt Borenstein]

Right. That's sort of the Stack overflow replacement thing. Exactly. When you inevitably hit a problem, instead of going to Stack overflow, you go to ChatGPT and it actually gives you.

[Guido Appenzeller]

Code back, copy, paste, but from a different source. Right.

[Matt Borenstein]

And this was like six months ago. This was state of the art. This wasn't like that long ago. Maybe nine months.

[Guido Appenzeller]

Nine months ago? Yeah, nine months. But then. So then the next step was you started having integrate things that are integrated in your IDE, right. GitHub, copilot, then cursor. That basically allows you to use autocomplete, which is a big step forward. It's no longer like monolithic questions, but it's sort of in the flow. Then this split up into autocomplete at a line level, I can ask questions about paragraphs, or I can have a separate chat interface where I can have longer discussions. Then the IDE started to be able to use command line tools. So suddenly I can say, hey, can you set up my new Python project with UV or something like that? And it could. Could basically run commands to do all that work. And I think where we are today is when I want to write a new piece of software, or this is not production Code right? This is like. But I want to try something out. The first thing I do is I start writing a spec, right? I'm start basically a very high level. Here's what I'd like to do. And it's still fairly abstract and not very well thought through. And then I basically ask the model maybe something a sonnet 3.5 or 3.7 or Gemini. Here's what I'd like to do. Does this make sense? Please ask any questions that are unclear and then write me more details back. And then the model gets to work. And usually there's lots of questions. For me it's like, hey, I need an API key for that. You know, very simple things or more complex things like how do you want to manage state, should we put this in a database? Should we just dump it into a file? Or something like that. And so it's basically a back and forth discussion that helps me clarify my thinking. And the model is almost a sparing's partner to think through the process, which is really weird in a way. But it works, right? And so over time you basically get more detailed specs, not only when you have them. Then you ask the model model to start implementing. And all of that comes with a fair amount of context. Also together with the model I have my standard Python coding guidelines. This is how I like to do commenting. This is how I like to do more object oriented versus more procedural is how I like to structure my classes. I'm an object oriented guy.

[Matt Borenstein]

We're talking Java here or what?

[Guido Appenzeller]

No, no Python. Do you want to have type Python or untype Python? All these things, right? So it's a lot about context, it's a lot about explaining your general development methodology. It's a lot about a back and forth with a model now where you sort of together figure out something. That's how I'm. How are you coding?

[Yoko Lee]

I guess like compared to maybe six months ago. How I use coding agent nowadays is I give it more of the world knowledge. Before I was mostly relying on what's a foundational model's knowledge. And it's funny because when you ask the coding agent when do you think today is? It's always like 2023 and then all the specs that we'll give you are from like 2024 at best depending on when the knowledge cut off is. Nowadays I think it's very natural for me to reach out to like linear, here's the ticket and I just give my idea. Pull my idea into cursor, cursor agent will Take a first step at implementing it. So that's one kind of workflow change. The other one is more user prompted active queries. So before I may need to copy paste documentation into my little cursor window, now I just ask the cursor agent to like, hey, can you use Firecraw to go search for the most up to date, like clerk documentation? And it will actually fetch a page and it will read it.

[Matt Borenstein]

Cool. That works.

[Yoko Lee]

Yeah, it actually works. And then it will work. It's using MCP or it uses mcp, but it's like instrumentation detail. It could be a tool call or whatever. But there's more integration to the real world now.

[Matt Borenstein]

You guys sound much more planful than me. The scenario for me is like Saturday night I finally have an hour free and I have a weird idea for an app and I just dive right into it and ask Cursor to do everything. And I've always found it works really well for high complexity, high kind of annoyance factor. Things like front end. Like if anybody on earth can remember all of the CSS classes that people use now for margins and padding, it's like, it's, you know, I don't think that person exists.

[Yoko Lee]

AI sen center a div yet.

[Matt Borenstein]

Yes. Oh yeah.

[Yoko Lee]

We should do a benchmark on centering a div.

[Matt Borenstein]

Totally. Yeah. Tutorial on div center. I mean, it's one of these hard problems for no reason. Right. There's just like five different ways to center text and elements and I can never remember any of them. And AI models are really good at this. Right. And they now can do it when you start going to more niche libraries and function calls. That's where I always run into trouble. So I love this Fire crawl kind of idea because usually I'm hunting for docs and then putting them back in or something like that.

[Yoko Lee]

Yeah. Sometimes I also copy paste like a minified doc because they have the LLMs txt like the developer tool docs. I just give the URL add doc and then enter the URL and ask Cursor to implement that. And that works too.

[Matt Borenstein]

Has anything gone really wrong for you guys yet? Doing sort of AI assisted coding?

[Yoko Lee]

Not really wrong per se, but a lot of how we code is dependent on the agent behavior on how the client implemented the agent. One example is there's this very cool tool that actually generate like very pretty pages and send back like a react component, like an HTML page for the coding agent to refer to. So one time I asked Cursor agent to like reach out to this tool implement based on whatever it told you. Cursor agent reaction was very interesting. It looked at the code, it says, oh, this looks great, let me give you a new version. So it didn't adopt whatever that was returned.

[Guido Appenzeller]

Interesting.

[Yoko Lee]

Yeah. Which is like a very interesting like agent to agent communication. Chris or EJ is like, I don't.

[Derek]

Agree with this direction, but surely serious engineers wouldn't lower themselves to use AI generated code, right? Well, there's at least a strong argument that they should consider it depending on what they need to do and what a model can reasonably output.

[Matt Borenstein]

So you've done a bunch of work on mcp, Yoko. How do you think that plays into this?

[Yoko Lee]

I think MCP to its essence is just a way to provide context, the most relevant context to llc. So it helps that a long tail of MCP servers nowadays can be leveraged whatever client you are using. So that's what's empowering the kind of experience I was just describing. I can use linear MCP, I can use GitHub MCP to pull in the relevant context. And tool calling is like a technical detail how they implemented fetching the context. But the crux of the MCP is actually the context part. What is the most relevant thing I can provide to you as a model so you can help me better?

[Matt Borenstein]

And so do you think having these kinds of tools available in an IDE means AI coding is kind of more productive or a better fit for kind of senior developers? Because a knock against this for a long time has been that vibe coders, for lack of a better word, are kind of producing great demos and junior developers are kind of getting up to speed faster. But the people I've always affectionately called neckbeards, right, the people who own the cluster and stop you from breaking things things or like own the overall architecture are sort of skeptics. Do you think this is one way to get the neck beards engaged?

[Yoko Lee]

I think it depends on what the very senior engineers are optimizing for. There are very senior application engineers who are just very good at fleshing out ideas. So in this case it's a more evenly distributed skill set. You just need to put the stack together. But there are very senior engineers who are say, optimizing best in the world for optimizing for distributed systems that I think we're not quite there yet. Just because the coding agent first, it can't fetch any and all state of the distributed system. It's a lot of human intervention when it comes to how to solve certain problems, but I feel like we're on the way there. Given enough context window, enough tool calling capabilities to bring just the right knowledge into the model today, I think most IDEs have a limit on the number of tools it can handle. I remember it was like 40 or 50 or something. So it naturally limits what's the context and what's the tools that the coding agent can leverage.

[Guido Appenzeller]

I think there's sort of a pattern that the more sort of esoteric the problem is or the more novel the problems you're trying to solve, the more context you have to provide. Right. If I'm like, hey, write me a blog or what is it? Write me an online store, like the simplified version that's of a standard, I don't know, undergrad software development class problem. So the amount of samples on the Internet is more or less infinite. The models have seen this a gazillion times and incredibly good in regurgitating this code. If you have something for which there's very little training code, that typically all goes away. And it's all sort of. You have to specify exactly what you want. You need to provide the context, you need to provide the API specifications much, much harder.

[Matt Borenstein]

And it will very confidently give you a wrong answer too. I can't tell you the number of times I'm like, oh my God, this function exists. I had no idea. It's exactly what I needed. It's like, wait, no, it doesn't exist.

[Guido Appenzeller]

And once it does, it, it's very hard to get it off. If you're saying like, oh, the function doesn't exist, it hallucinates a new one.

[Matt Borenstein]

It's like, oh, I'm so sorry, here's another function that doesn't exist that might work. Yeah.

[Guido Appenzeller]

I think what models today are very bad at is telling you if they don't know something.

[Yoko Lee]

Yeah. Do you think RL would change that in the training process? If you, theoretically, you give it all the relevant environments in the world, it can do all the things it's needs to do to simulate a distributed system and debug it.

[Guido Appenzeller]

Look, I think in the extreme case, if you are the first person on the planet to write code that solves a specific problem, there's just zero training data out there. I think it'll always be very hard. I think the models are not really creative so far. They can do a little bit of transfer, but not much. So if you say there's a brand new chip which has a new architecture and you're the first one to write a driver for it, it's going to be a Fairly manual task. I think the good news is that is 0.01% of all software development. Right. For the, I don't know, you know, hundred thousand ERP system implementation or so. Right. That we have tons of training data. I think these tools can be very, very powerful.

[Derek]

As AI coding tools continue to improve, it's natural to question what it even means to learn computer science. Is prompt engineering a more important skill than a deep knowledge of how computer systems work? The short answer, no, it's not. Because to truly optimize, troubleshoot and scale out even the simplest application, you must be able to look deeper than a surface level abstraction.

[Matt Borenstein]

We haven't talked about vibe coding too much, but. Right, but there's this idea that people who aren't developers can now kind of write code, which is pretty cool, Right. And sort of feels like something that should happen. We're not like priests of the computer where we need to intervene between ordinary people and the processor. Right.

[Guido Appenzeller]

It should be that anytime we indoctrinated before.

[Matt Borenstein]

Yeah, exactly. There's no seminary of. Well, maybe, maybe there were seminaries of.

[Guido Appenzeller]

Computers, I don't know, CS departments.

[Matt Borenstein]

Yeah, exactly. But it kind of makes sense that people should be able to control computers in direct ways, not just in sort of pre baked programs that have been given to them. So this is, I think a super interesting and super exciting thing.

[Guido Appenzeller]

I think there's a question there. Is that true at all scales or is this a little bit like, look, everybody can build a shack, but you cannot build a skyscraper. Right.

[Matt Borenstein]

Well, so this is exactly why I bring it up. Right? The demos that everybody does their first time, they're trying a website generator or cursor or something like that probably are not doing that much for the rest of humanity this sort of first weekend project. But if you assume some portion of people who give that a try maybe start to climb the ladder and do increasingly sophisticated things and by the way, in a totally different way than from the three of us would probably do it. Having learned sort of programming the hard way, I just have a ton of optimism that that creates all sorts of kind of new things. You know, you have a new pool of people writing software in a new way who may look at the world in a completely new way. I just have a ton of optimism that gives you kind of new stuff, new applications, new programs and new ways of using computers and computing that we haven't had before.

[Yoko Lee]

This actually reminds me a lot of the 2000s, like when blog was the new word on the blog and everyone was like, I Need a new blog. And then we rushed to create our own blog and there comes WordPress. People are still using WordPress, by the way. I'm surprised by that. And this wave of vibe coding almost felt like everyone and my mom and my mom's neighbor are trying to use the models to create personal software. So where it came from, personal static content to personal CRM to manage your relationship or something, how deep do the software go? I don't know. I don't think it's very deep, but it doesn't matter as long as there's personal utility. I think Martin tweeted about this earlier. He was like, you should still learn to code. If you're operating on one abstraction, you need to learn the abstraction lower than where you're operating from, which is very fair. And I keep coming back to that because I wonder, what is the one level lower abstraction for live coders? Is that code? Is that the ide? Is that something else? But curious about you guys. Take.

[Guido Appenzeller]

I think this is a super good question. Let me try to rephrase the question a little bit. What is the thing that future people that want to do software development need to learn? Right. Is it one level deeper? Is it actually something that's sitting more to the side? I mean, especially some people say, look, there's no point in learning CS anymore. It's all about social, emotional learning and the kind of things. I'm not sure I agree with that. Right. But it's.

[Matt Borenstein]

I feel like that comes up every 20 years or so.

[Guido Appenzeller]

Definitely is a cycle there. Yeah. Honestly, I have absolutely no idea how the equivalent of computer science education will look like in five years. Right. When we're on the other side of this, you'll probably. I mean, historically, what happened when we did similar things, say with calculation, when we went from adding numbers manually to Excel. Right. It's not that the whole job category disappeared. Right. It's more that bookkeepers became accountants or something like that. Entering data and writing down numbers and adding them manually became less important and doing higher level, more abstract concepts became more important. So if I pattern match that one to one, the guess would be that explaining the problem statement, explain the algorithmic foundations. Explaining architecture and explaining data flow is getting more important. And the nitty gritty coding, what's the most clever way to unroll the for loop? That's a very specialized, more niche discipline.

[Matt Borenstein]

It does almost feel like we're waiting for something, doesn't it? If you think about a sort of classical computer science undergraduate education, you don't just Learn kind of the latest thing. At least in a lot of programs you learn. You may do a semester of sort of assembly, right?

[Yoko Lee]

You actually learn all the oldest things.

[Matt Borenstein]

Yeah, you start with the old or you know, we even had to take like a processors course, right? And I'm the world's worst computer engineer. But I got in there and I was like connecting gates and like that was fun. So you learn, you learn like how processors work, you learn assembly. We did a course on Lisp, which was cool. We did file systems and some bits of operating systems. And you learned Java. Like Java was state of the art at the time. That's why I mentioned it before. Not, not anymore, obviously. So it's tempting to say this is, this is like the next thing that is built on top of those things and that you would learn to code kind of only for historical reasons or for educational reasons. I just don't know yet if that's actually true. Like a lot of the kind of layers we've added on, on top over, over, over the course of decades are, are things that truly are a new programming interface. AI is not actually a programming interface, right. It's not actually a framework. It's. It's sort of a tool that uses things you already, helps you use things you already have. So I just, that just makes me wonder if we're waiting for kind of the, the next iteration of this thing. Like the thing that AI actually can change about the way computers are programmed. For instance, it could just be prompts that are somehow translated to code in a more direct way. And agents as we see them now are kind of a starting point there. So that, that's what I'm sort of curious about.

[Derek]

What about the future of programming languages? How does our relationship with formal languages like Python or Java evolve if we can generate code using natural language today?

[Guido Appenzeller]

I think AI is not just a higher level language abstraction or something like that, but could it become one over time?

[Matt Borenstein]

That's the question. Do you think?

[Guido Appenzeller]

Yes, I think it could. I mean, look, I think we really haven't figured that out yet. I mean, if I look at a classic, say compiler design or in programming languages, if I would have LLMs as a tool, I would probably think about very differently about how I would build a compiler. And I don't think we've seen that work its way through yet. I have no idea how it's going to look like. But if I can basically define certain things in human language in an efficient way and maybe in a sort of tight enough way that I Can use this directly as input for a compiler. That could change a lot of things.

[Yoko Lee]

Analogy here would be like because a lot of companies are building agent based systems and then when you take a look at that system, when you see what the agents are building, you're like, oh, this is what I learned in operating system class many years ago. These are processes. One process for another one and then hence the task to another one and then something else. Manage the resources of the system. I don't think we have the framework. This is why I think the CS education will not go away, because it gives you a way to compare the two things. Otherwise you wouldn't have known there's a thing called process in the first place. But at the same time, I don't think on the on top of the foundational model, we have invented the paradigm to make that work as if it's an operating system.

[Matt Borenstein]

Formal languages exist for a reason, right? I guess is the one thing I would say, whether that's a programming language or a specification language or something like that, there has to be some high bandwidth expressive way for a person to design software or anything but software in this case. So I just have a hard time seeing Python going away or programming languages going away entirely. To Yoko, your point about you have to understand at least one level of abstraction down. It is an interesting question if some will be more popular than others because they're kind of more AI native in a way. We're sort of seeing Python and JavaScript are kind of leading the pack right now, but it's not clear. Tooling, I think is another really interesting thing. Like we're seeing a bunch of new Python tooling come out right now, which is kind of cool because the Python ecosystem is kind of more active than ever. And you can imagine that sort of has an impact on how well does it work with kind of the AI add ons to the language too. So I just don't think we can toss these things out completely.

[Yoko Lee]

I think the reason behind you need to know a level like an abstraction level deeper is if and when you need to do optimization on the system you're writing, you just need to know how to optimize that. If you don't, then you really don't need to know. There's a lot of people who back in the days coding Java had never heard of JVM or know how it worked. Just like creating a calculator using Java, you don't need to know jvm. But if you want to optimize for runtime, threading you do need to know jvm. It's very similar with the Vibe coding use cases. If you're just building a marketing website, I don't think you need to know the next level of optimization. Like, unless you're serving something at scale, then you probably need to know what CDNs are, how to cache pages, things like that. But at the same time, if you are someone who aspire to serve something at scale and then want to flesh out the real service one day, it's really hard to get away without knowing the underlying knobs. Because the essence is there are certain things computers can do and these things are defined by formal languages. One language is buried under the other, and then to touch these knobs and to know what to even do, you need to know these languages. Yeah. So curious about your take too, Greedel.

[Guido Appenzeller]

I agree. I think formal languages won't go away because ultimately they seem complicated. But I think effectively formal language is often the simplest type representation you can find to specify intent. Doing that in a language like natural language is often very imprecise and you need a lot more words to get the. To get the same result. I think the interesting question at the moment is are there cases where AI has enough context from understanding humans and enough context from you inserting clever at symbols and pulling in additional pages that it can take for a certain subset of problems, natural language descriptions and translate it accurately? And I mean, it seems like there are areas where that's possible, right? That's what we're using every day when we use AI for coding. So can you hybridize this somehow that you actually create a language of that type? I don't know yet.

[Matt Borenstein]

I mean, your distinction is really interesting, right? Like this word complicated is sort of overloaded in one sense. It can mean a highly complex system that has a lot of pieces and you never quite know how it's going to behave. On the other hand, it may mean just kind of hard to use. Right. And I think people sometimes see programming languages complicated in the sense that they're hard to use or hard to learn. Right. You need to learn this kind of new language to speak in. They're actually very simple, right. You can draw a tree that sort of encapsulates the entire set of things that can be expressed in that language. So it's funny, we're switching to this thing called AI coding that's easier to use, but actually much more complicated. Under the hood, insert meme about giant green monster with a mask on, like here. So to your point, it's like how do you sort of handle that? And is it some hybrid solution or something else? I know the guys at Cursor have always talked about kind of formal specifications, which I think you alluded to also, Guido, as kind of like writing a spec in a really clear way is kind of the task that people will be faced with more and more over time.

[Guido Appenzeller]

It's almost like an annealing process between you and the AI to go from some loosely formed model that you have and a loosely formed model that the AI has to a tight spec that you can implement.

[Yoko Lee]

At the end of the day, this is so true. I talked to a classical Vibe coder recently and then because, like, my question was, do you really need the coding interface? Like, you know how you enter a prom it generation bunch of code? And this Vivekder's answer was so interesting. He said, I like that the AI is generating code and showing me. It's very empowering for me to see that I generated all this code, but when I want to go in and actually change something myself, I don't know where to start. So it does tell me that there is a gap between what the AI generated and where five coders operate. It does feel like there is a product somewhere between, like. Like we want to give them the power to actually change the underlying apps too.

[Matt Borenstein]

I mean, this is not restricted to people who are not experienced programmers, by the way. Like, if one of us tried to Vibe code an app after four to five turns, if you went in to try to edit the code, it would be very difficult. It's very opaque. What's going on?

[Yoko Lee]

I ran into this when I was trying out the Blender mcp. I've never used Blender before. Kind of like it's just really hard piece of software to get into. So I installed the MCP server on my Cursor IDE and then I was able to prompt like a mini statue of a 16Z infra just very easily. But when it comes to modifying this 3D representation, like, that's where things start to break apart. I don't even know where to start, why I need to modify.

[Matt Borenstein]

Yeah, like a flat surface.

[Yoko Lee]

Exactly.

[Matt Borenstein]

10,000 polygons, textures.

[Yoko Lee]

Yeah. But there's a lot of opportunities here, and I could kind of existing in the gaps of AI and Vibe coders and what the representation is today.

[Matt Borenstein]

What's really cool about this is you're sort of creating a new layer of context and a new layer of intent in software programming that didn't exist before. So, for instance, can AI help port Old code, right? This is one of these, like the banks have been trying to drop COBOL for a hundred years or something like that. And personally I think the answer is kind of no, right? Like it can definitely help, but it doesn't solve the hard problem. And I mean that in the following way, right? Like AI may be able to transpile COBOL to Java, but there's a huge amount of context in what went into creating that COBOL code that's been totally lost, right, over the course of decades. In many cases, something that started as an airline booking system became an airline booking plus HR plus, you know, fetch the coffee system. And many of the people who contributed to it and by the way, didn't write a lot of documentation or comments, may not be around anymore at the company or, you know, on this earth, right? This is a problem that AI, I think, can help and not solve. But what's actually even more interesting about this to me when we talk about specifications is like if they had been using AI at the time to create those systems, there would be this whole other record of what their intention was when they were creating the software that kind of comes for free, right? It's not something you have to go back and do. And I think that's something that's kind of cool now. Like if we see this kind of take off more and more, you have this kind of other set of metadata that can capture the software intent in a slightly different way.

[Guido Appenzeller]

It's almost like a high level language abstraction, isn't it?

[Matt Borenstein]

But it's. I think it's different, right? Like, because it doesn't, like you can't compile it down, you know what I mean?

[Guido Appenzeller]

Sort of you can.

[Matt Borenstein]

I agree, it's sort of.

[Guido Appenzeller]

But you're raising a very interesting point there. I recently talked to some large enterprises that are using AI to basically take legacy code bases, specifically mainframes or Cobol and PL1 is the other good one there, and move that to more modern languages. And it's super interesting. They have exactly the issue that you described, which is that if you just look at the old code base, you often have no idea what the intent was. And if you just try to translate that, you pick up many of the idiosyncrasies of that old programming language, right? I mean, Java has much more modern constructs that you didn't have in Fortran. Maybe you want to use some of cobol. Maybe you want to use some of those. So what I've heard from now, multiple organizations that they're saying the Most efficient way for them is to actually go first and try to create a spec. Use the AI to create a spec from that code. Right. And once they have the spec then to re implement the spec. And that gets them much better results, much more compact code, much more modern code than what they had originally.

[Matt Borenstein]

And that is sort of an AI assisted problem for sure. Both of those problems, I think.

[Guido Appenzeller]

Yes it is.

[Matt Borenstein]

That's very cool.

[Yoko Lee]

That's interesting because I was actually just thinking about it's actually much easier to rewrite modern software. Like modern meaning something in the backpack 10 years. It's like easier to implement from angular to react. Especially both frameworks are well understood by the agent. It's much harder if the state PHP.

[Guido Appenzeller]

To angular is a little bit harder.

[Yoko Lee]

I mean Laravel is working out pretty well. So that one's easier depends on what kind of framework you're using. It's much harder if the state one is spanning across many software systems because you just need to do some discovery or have an agent that can have access to this discovery. I can see that working out. And two, there's specificities on the hardware some of these things are running on. Like for example, for the runtime, maybe I give it enough memory for the stock container. I need to have specific configs to make this work. Sometimes, like to your point, all of that is lost until the day we can take a snapshot of the runtime. How is this run? What's the requirement of this? Because it's hard to migrate systems like that.

[Matt Borenstein]

I'm now getting like pre nightmares of like something goes down in prod and you're digging through the ChatGPT logs to like try to figure out what someone might have accidentally tried to do.

[Derek]

Even if we never fully lose our connection to the fundamentals of computer science and programming, some things are going to change as a result of AI generated code. And one of them might be how we balance certainty from or reliability with utility when it comes to deploying non deterministic systems.

[Matt Borenstein]

Guido, I have sort of an interesting question for you. If you think of AI as a primitive in an application, not just a tool to write code, it does seem like it's kind of pushing the frontier of the degree of uncertainty and non deterministic behavior we can have in software, right? Like if you think like really old days, probably predating a lot of us and our listeners, you know, you just write software for like a local machine and you could have a pretty good expectation of how it was going to execute. We had this new thing Called the network. Right. Which is very hard to predict how it's going to behave. But you can kind of express it in the same terms. It feels like a problem that you can wrap your arms around. It feels like AI is kind of an extension of that in a way where, like, you actually don't know what's going to happen when you add AI into your software or use it to write code or whatever. How do you think about that? Do you think that's a reasonable way to look at it? And are there any lessons from the networking world that would. Could help us figure out what's going to happen in AI?

[Guido Appenzeller]

Yeah, I mean, I want to say probably because I don't think we have the lessons fully digested yet. When we went to network systems, there were sort of new failure modes, like timeouts and then new remedies for this, like retries. And once you got to sort of distribute the database, you had to worry about atomicity and rollbacks. In a digital context, things got very complicated very quickly. And I think for some of these design, some of the design patterns, even today, we don't have very good software architectures yet. Right. They're still.

[Matt Borenstein]

And they may be kind of unsolvable, some of these problems. Right.

[Guido Appenzeller]

I mean, I think the fundamental problem is not solvable, but you can at least make it as easy as possible for a developer. Right. I mean, everything is just tools for the developer to cushion some of the blow. Models are funny because at temperature zero, a model is technically deterministic. Right. So it's not so much that different inputs, that the same input would result in different outputs. That's something we do by choice. I think the bigger problem is that an infinitesimally small change in the input can have an arbitrary large effect.

[Matt Borenstein]

So it's a chaotic system, you're saying?

[Guido Appenzeller]

Chaotic system. Exactly.

[Matt Borenstein]

The user could put anything into the text box and the system is chaotic enough that you get, you know, like it used to be, you just had to check for apostrophes and then you could execute a database statement. Like there's only a few things that could break a text box now. Like kind of anything could happen when, when someone enters text.

[Guido Appenzeller]

That's right. Ignore all previous instructions.

[Matt Borenstein]

But that's a really interesting thing. You're saying that it may be the case that we just need to expose the primitives and capabilities of the system in a way that developers can use, not necessarily tamp down all the failure modes. The equivalent of a timeout, for instance.

[Guido Appenzeller]

I think that's one part of it. But I think we also have to change our expectations. So I talked to one large bank and they implemented software to basically generate text. And one of the important things in financial institutions is never give investment advice. And so, so you're trying to have an LLM that is very helpful and never even implicitly gives investment advice. That's often unsolvable problem. Right? You can get better and better and better, but you can never completely rule it out. And you can add a second LM that tries to catch it, but it also will occasionally not catch something because it thinks it's helpful. And at the end of the day they basically made a decision to say we can't build a software system that never does this. We have to change our metrics, we basically have to go. And I think they ended up with something like it has to be whatever half the probability of a human, of a well trained human doing the same situation.

[Derek]

Zooming out a bit more, maybe you've heard the analogy that Internet protocol is the narrow waste of the Internet, often visualized like an hourglass with IP being the standard in the middle that allows interoperability between Internet infrastructure and applications. Does AI have or need its own narrow waste? Is the prompt up to the job?

[Yoko Lee]

If we were to zoom out a little bit, you were there for the whole Internet history and then you were a pioneer on a lot of the networking research. So how the Internet came to be is there is a narrow waste of the Internet somehow that happened. Do you think it would be a similar dynamics playing out in AI at all? Like is there analogy? Maybe the waste is never narrow for AI.

[Guido Appenzeller]

Like for waste the narrow waste? I think it's the prompt.

[Yoko Lee]

Oh, interesting. Why is that the case?

[Guido Appenzeller]

I mean, look, typically these big tech cycles are built on abstractions that allow you to encapsulate the complexity underneath in a very narrow API for say a database. It was SQL in the early database of the transaction databases. Where how does the database under the SQL query work? It's something with B trees, we learned that in grad school. But that doesn't really matter anymore. I just need to be able to specify the query. And I think that's the same thing that led to the rise of modern ML, right? You no longer need the overpaid staff or PhD that trains a model for you, but instead you can now express and steer the model with a prompt. And so a fairly say mediocre Python programmer can suddenly leverage a very powerful LLM just by prompting it.

[Yoko Lee]

Interesting. If you were to double click on the Prompts. Do you think it's like a natural language representation of what you want to do or is it because there's no standard there? Prompts can be anything and everything. It's partly a narrow way to it.

[Guido Appenzeller]

It's not a formal language.

[Yoko Lee]

Right? It's not a formal language.

[Matt Borenstein]

It's clearly not. Not like English either though, right? It makes me think latent. Yeah, I mean we're all learning kind of a new language in order to prompt these things. And actually it's a little different for each model. Sort of dialects, you know, we've got like a translation issue, all that kind of stuff.

[Guido Appenzeller]

I mean, will we ever have a formal prompting language? Maybe.

[Matt Borenstein]

I think there are some overpriced Stanford PhDs working on that problem. I'm hopeful to see what they come up with.

[Yoko Lee]

Our agent frameworks. Formal prompting language and I think a little bit. A little bit, yeah.

[Guido Appenzeller]

We're certainly starting to see prompts with structure where it's like, I don't know, user something, agent response or something like that. Or you know, think and think, beginning of the answer, end of the answer and you say okay Gudo. It's like two tags and a lot of text. That's not very formal, but I think the first starting points are there. You could see future models getting trained and fine tuned on a more structured approach presentation.

[Yoko Lee]

Oh yeah, we're already seeing this happening. Right? There's models. Every model has JSON mode now. And then how you define what you want out of the JSON mode is like you give it a type system. It's like you can prompt like I want you to generate three fruits but only return it to fruits colon like, you know, apples, like fruit, like types of fruits. And then you could define it in your code saying I only want your answer to be have the food key, I don't want anything else. I guess that's kind of formalization long term.

[Guido Appenzeller]

I actually wonder if the. For a reasoning model where a lot of the thinking sort of happens internally, if the model is generating the user facing machine facing output, it's going to be a different model from the model doing the reasoning, if that makes sense. Right. So I like a really chatty model or somebody else wants a more terse model or if we want to generate JSON output, we have yet another model. So you could see sort of the model output layer delaminating at some point from the reasoning layer.

[Derek]

Finally, a simple question whose ultimate answer will have enormous ramifications for the software development industry. Is there or should there be a natural bifurcation between Vibe coding and enterprise software development.

[Guido Appenzeller]

In the future, do we think there's going to be different Vibe coding models versus Enterprise coding models?

[Yoko Lee]

I actually don't think so. Well, I define Vibe coding as you kind of let the model. You have a spec. You let the model generate whatever it needs to with the implementation detail. You don't care about the implementation, but you do care about what comes out of that implementation is what you wanted.

[Guido Appenzeller]

So it's less formal, less constrained than classic coding. What is the difference between Vibe coding and classic coding?

[Yoko Lee]

I think for classic coding, you have to make a lot more choices in what you want to put in a code. So I want to use this SDK, not the other one. For Vive coding. You just don't care about the underlying technical details as long as. Yeah, as long as it gets things done, but you still care about the higher level needs. Otherwise, why are you writing this?

[Guido Appenzeller]

Got it.

[Yoko Lee]

So I can totally see enterprise users doing Vive coding. And that's a compliment.

[Derek]

And that's it for this episode. Thanks for listening. As always, if you enjoyed the episode or if you learned something, please do share the podcast among your friends, family and colleagues and rate the podcast on your platform of choice. And keep listening. We've got more great episodes coming up.