AI Engineering Revolution: Winners, Chaos & What’s Next | FirstMark

Summary

The MAD Podcast with Matt Turck

Episode: AI Engineering Revolution: Winners, Chaos & What’s Next | FirstMark
Date: July 3, 2025
Host: Matt Turck
Guest: David Walter (FirstMark Capital)

Episode Overview

In this episode, Matt Turck and his FirstMark colleague David Walter take a venture capital lens to one of the biggest stories of the year: the meteoric rise of AI-powered software engineering. They explore how generative AI has supercharged the productivity and capacity of engineering teams, creating runaway growth for startups in the space — but also a range of novel challenges, opportunities, and industry ripple effects. Drawing from historical analogies, in-the-trenches anecdotes, and recent survey data, they break down both the excitement and the chaos unleashed by AI coding tools, with lessons for founders, CTOs, and investors navigating the ever-evolving landscape.

---

Key Discussion Points & Insights

The Unprecedented Growth of AI Coding Startups

• Notable companies have experienced explosive ARR growth:
  • Cursor: $500M ARR in ~2 years, possibly the fastest B2B growth ever ([00:00], [07:31])
  • Lovable: $0 → $60M ARR in two quarters
  • GitHub Copilot: $400M ARR, 15M developers ([00:00], [08:01])
  • Windsurf and Replit: huge ARR leaps in short timeframes ([09:25])
• “There really is no plateau in sight for innovation. You might miss something spectacular and industry shifting.”
  — David Walter [00:11]

Why Has Generative AI Flourished in Coding?

• Four main reasons AI is especially well-suited for code ([04:12]):
  1. Massive, high-quality open source training data, especially from GitHub
  2. Code is structured and precise — ideal for machine understanding
  3. Coding relies on patterns AI can easily learn and reproduce
  4. Clear ROI: saves huge amounts of “grunt work” and time
• Adoption has been rapid because coding already had behaviors like “copy/paste from Stack Overflow” and code completion (IntelliSense, 1996) which set the stage ([06:28]).

Industry Impact — Productivity & Process Shifts

• Tangible improvements across engineering teams:
  • 30-50% faster throughput
  • 12% increase in PR merges
  • 17% more time on roadmap features vs. maintenance
  • 82% of engineers already use AI to write code ([11:03])
• “That is just, again, an adoption curve that is pretty much unprecedented.”
  — David Walter [11:41]
• The UI and “distribution model” for AI coding fits seamlessly into dev workflows (IDEs), driving easy uptake ([11:46]).

Challenges: Production Surges Lead to “Cleanup Crews”

• Historical analogy: like the Gutenberg press or the Ford assembly line, big tech leaps create downstream chaos, requiring new markets and industries (“cleanup crews”) for quality, safety, management, and governance ([13:31], [14:50]).
• In software, AI codegen has vastly accelerated code output (commits), but downstream DevOps (CICD, testing, QA, review) is straining and “kind of breaking” ([17:00], [19:10]).

Historical Parallels:

• Gutenberg Press → Book editors, publishers, libraries, regulation to handle misinformation & overload
• Ford Assembly Line → Inspectors, mechanics, highways—and new risks, regulations
  — “As production surges, you see all of these problems come in the wake of that production and then, and then new industries come around.”
  — David Walter [17:19]

New Roles and Organizational Changes

• Engineers in some orgs are shifting to “professional code reviewers” as AI generates more code ([20:09])
  • “People sitting in a toll booth letting cars pass or not pass, very much doing the same thing with code.” — David Walter [20:16]
• Hiring shifts: The highest unemployment rates among college majors now include CS grads, despite strong demand at a macro level ([29:15])
  • Rise of the “editor,” reviewer, and prompt engineer — less demand for traditional code monkeys
• The structure of teams is evolving:
  • Smaller teams, more reliant on reviewing and managing AI-written code ([31:12])
  • Product, engineering, and design (EPD) boundaries are blurring; if anyone can code, how do teams work? ([33:23])

What’s Breaking? Where Are the Bottlenecks?

• Security vulnerabilities are spiking ([24:02]):
  • Models trained on open source potentially propagate old vulnerabilities
  • CI/CD pipelines, testing, and QA are overwhelmed
  • New types of “flake”—i.e., tests that unpredictably pass/fail
• “A new focus on code review that is pretty unrelenting, that I'd say has gone from a step in a process to very much a job in and of itself.”
  — David Walter [25:54]

AI Reviewing AI & The Next Wave of Tools

• Can AI review AI-generated code? Agentic tools starting to emerge: semantic QA, code reviewers, auto-patching tools ([25:54], [27:12])
  • Companies are racing to solve newly emerging pain points as fast as they appear
• “All of the problems that I'm talking about are very much our opportunities as investors and founders.”
  — David Walter [26:00]

How CTOs & Leaders Must Adapt

• Major decisions loom on:
  • Talent: Editors/prompt engineers over just coders ([29:15])
  • Architecture: From design docs to machine-readable guardrails, “IaC-esque” constraints for AI ([31:12])
  • Team structure: Fewer, smaller, multidisciplinary teams
  • Governance: New need for code lineage (“provenance”), fine-grained reviews, and tight constraints ([35:01])
  • Security: Emergence of tools that catch bugs as code is written, not days later ([35:01])

Is the Market Tapped Out? Or Is There Still Opportunity?

• On competitive barriers:
  “It's always incredibly easy to say the alpha's left the room... At once there's a million things to go build. And on the other hand, there's also probably a million people trying to do it.”
  — David Walter [40:30]
• Developers are “picky” buyers, so there’s still plenty of room for new entrants with unique approaches
• Giant platforms don’t close the market, they open new waves of downstream need (e.g., AWS & the cloud)

The Power of Social Proof & Distribution

• Startups can ride the “halo effect” from being included in the hot new engineering stack (e.g. Supabase and Neon benefit from Cursor/Lovable’s growth) ([42:34])
• In this platform shift, brand new companies can be more credible or desirable than older, bigger incumbents if perceived as “AI native” ([44:23])
  • “You can be a one year old or two year old company and actually be a lot more credible than a 5, 7, 10 year old company...”
    — Matt Turk [44:23]
• Word-of-mouth in the B2B developer universe is more intense than in consumer tech

The Rise, Fall, and Rise of Developer Tools (from an investment perspective)

• “Sweet revenge” for developer tools: once seen as tough, “cheap” buyers, but now the hottest segment in generative AI ([46:15], [47:07])
  • “Their relative importance across organizations has seemed to just go up and to the right over the last decade or so.”
    — David Walter [47:25]
  • The “definition of a developer tool” is broadening—Stripe as a dev tool example

---

Notable Quotes & Memorable Moments

• On speed of innovation:
  “If you go offline and you aren't keeping up with the updates for a week at a time, you really might miss something spectacular and industry shifting.”
  — David Walter [11:05]

• On historical cycles:
  “With every surge in production, there's just a cleanup crew that naturally comes and a new market and industry that follows in its wake.”
  — David Walter [13:31]

• On engineering roles changing:
  “Many of [Malte Ubl’s] great engineers are actually becoming predominantly professional code reviewers... as AI code gen has just totally increased the rate at which they're producing and outputting code commits.”
  — David Walter [20:09]

• On hiring market shifts:
  “Computer science grads are actually among the top five or six majors graduating from college right now with the highest unemployment rate... There's a huge demand for software engineering as a concept, [but] the people who are trained in that practice are actually not in high demand.”
  — David Walter [29:15]

• On new skillsets:
  “We're seeing CTOs very much focused on hiring great editors and reviewers and prompt engineers who can almost shape and validate and curate that AI generated output.”
  — David Walter [30:07]

• On constraints and governance:
  “Do whatever you want with AI, but whatever you pull over the fence needs to be a few hundred lines.”
  — Matt Turk [34:40]

• On market opportunity:
  “It would be like saying, well, AWS, Azure and GCP came around and they ate up the whole cloud opportunity. So I guess there's no more money in cloud anymore. Of course that wasn’t the case.”
  — David Walter [41:42]

• On platform shift credibility:
  “You can be a one year old or two year old company and actually be a lot more credible than a 5, 7, 10 year old company... because you’re part of that platform shift and you’re ‘AI native’...”
  — Matt Turk [44:23]

---

Timestamps for Important Segments

• [00:00] – Opening statistics on explosive ARR growth (Cursor, Lovable, Copilot)
• [04:12] – Why coding is a uniquely good fit for generative AI
• [07:31] – How AI coding behaviors build on existing developer habits
• [11:03] – Measurable impacts of AI tools on engineering orgs
• [13:31] – Historical analogies: Gutenberg Press, Ford assembly line, and their “cleanup crew” industries
• [19:10] – Downstream strain: debugging, security, testing, quality review breakpoints
• [20:09] – The rise of professional code reviewers
• [22:18] – Survey data: positive and negative effects on engineering orgs
• [24:02] – Deep dive: what’s breaking (security, pipelines, QA)
• [27:12] – Emerging investment opportunities from new bottlenecks (auto-patching, agentic review tools)
• [29:15] – CTO decisions: hiring, architecture, team structure, and governance in the AI era
• [33:23] – The democratization of coding and its impact on enterprise team dynamics
• [35:01] – Governance: enforcing review constraints, code provenance, managing velocity/entropy
• [40:30] – Is the era of opportunity over? How to find “alpha” in a crowded field
• [44:23] – Social proof and AI-native credibility for startups
• [46:15] – The changing fortune of developer tools as a VC category

---

Conclusion

The engineering revolution brought on by generative AI is exhilarating, chaotic, and wide open — but also creating new, urgent bottlenecks throughout the software development pipeline. Waves of disruption echo historical tech leaps, spawning entirely new industries as the “cleanup crew” behind every productivity surge. As engineering roles, hiring, team structures, and best practices shift under our feet, the enduring opportunities for founders lie in innovative solutions to the chaos, tactically riding new distribution channels and platform effects. For VCs and founders alike, the only constant is rapid change and the pressing need to stay plugged into the fast-moving ground truth of engineering in the AI-native era.

Transcript

[David Walter]

Cursor announced they're at $500 million of ARR. Lovable has gone 0 to $60 million of ARR in the last two quarters. GitHub copilot $400 million of ARR. There really is no plateau in sight for innovation. You might miss something spectacular and industry shifting.

[Matt Turk]

Welcome to the Matt Podcast. I'm Matt Turk from firstmark. Today we're doing something a little different. My firstmark colleague David Walter and I sit down to unpack one of the key trends of the year, the rise of AI coding. We talk about key startups in the space, draw some lessons from history and think through second order effects on the rest of the ecosystem.

[David Walter]

With every surge in production, there's just a cleanup crew that naturally comes and a new market and industry that follows in its wake.

[Matt Turk]

If, thanks to AI, everyone can now be a developer, then what does that mean for software? The role of the CTO as well as the broad tech industry going forward?

[David Walter]

For ctos, this is going to be sort of a hallmark moment for them over the next five years or so where they have a lot of important decisions to make across talent, architecture, team structure, governance principles, security, and what opportunities.

[Matt Turk]

And challenges does that create for engineers, founders and investors.

[David Walter]

All of the problems that I'm talking about are very much our opportunities as investors and founders and we see a ton of them across all of these spaces. I'll call out a few.

[Matt Turk]

This is a VC lens on one of the biggest stories of the year and very relevant for anyone trying to make sense of the ever changing AI landscape. All right, David, excited to do this. Today we're going to talk about a really hot topic, the impact of AI on engineering. So it's a little bit of a different format. For this episode of the MAD podcast we have a presentation. We're going to go through it and talk about it together. So I'll let you drive and get started.

[David Walter]

Yeah, well thanks so much for having me on. So quick context. I was asked to pull this presentation together for the CTO summit that we threw last week. You know, I would say it is one of the most interesting times in engineering, in this broader ecosystem of technology. And so maybe I'll just kick off the presentation with talking about a quick summary of how we've arrived at this generative AI moment in our engineering ecosystem by just giving a quick highlight of, call it the six trends or so that have demarked the last two decades in the world of developers. So I'll start with pre cloud when we used to just deploy Code directly.

[Matt Turk]

Onto servers, aka prehistoric times.

[David Walter]

Correct. And then around call it 2010 along with cloud, DevOps finally got coined, which was really the merging of dev and operations into this whole subsequent tool set and set of workflows that emerged around it like Git, cicd, et cetera, that we all know today and that very much spurred this generation of SaaS explosion. All of the small mid large cap companies today that we think of as being venture backed very much belong in that category of company. And then two sort of things between then and now that I think are worth noting. One is the proliferation of APIs, which essentially gave all of those SaaS applications the ability to very quickly expand their capabilities by almost outsourcing things like payments and messaging and search to companies like Stripe and Algolia and Message Bird. And then two is abstraction. So you think about companies like Hashicorp and Snyk and DBT and Vercel very much catering to an audience of developers who are using code interfaces to all of a sudden do things that historically have been very much outside the purview of developer workflows. And so think about infrastructure provisioning with Terraform or think about dbt when all of a sudden we're now data engineering through a code interface. And over that time, so from call it 2006 all the way to now, estimated about 7x growth in the number of global software developers, which I think of as sort of an index to this tech ecosystem. And that's where we've arrived today. So generative AI, this massive shift in delivery model in the way that developers work and one that I, you know, in this presentation really talk about being probably the most impactful thing on this page.

[Matt Turk]

It's interesting to think about why coding and development has been such a major success in generative AI. When people talk about, okay, what generative AI applications have been successful so far, coding seems to be number one by far. There is something about code that lends itself particularly well to AI. First of all, because there's tons of training data out there. So GitHub has been a godsend for AI training because so much of GitHub is public. So publicly available repositories, we're talking about hundreds of millions of repositories on GitHub to the coding language by definition is highly structured and very precise. It's a grammar that you have to get right for the machine to work. And then three, it operates through patterns which lend themselves well to AI training. And then four, there's a very clear ROI to all the things. So people spend a lot of time doing grunt work around coding. So like anything that enables them to automate the work is incredibly, incredibly impactful.

[David Walter]

Yeah, I think you nailed it. The first point you made is really interesting, which is this corpus of open source code that all of these models could use as training data, um, which is very much true. And actually I think later in the presentation we'll talk a little bit about some of the bad or somewhat negative effects of that corpus of data that those models have been trained on and how that's created. Specifically security concerns. The empirical nature of code has certainly made it a great early use case for AI. And then the last thing I'd say is just developers have always been the type of Persona that loves to try new things. And so as a, as a buyer and a user, they always made a ton of sense for AI. And obviously, you know, across the broader world of consumers, it's been well documented that ChatGPT has made its way into, you know, millions of people's hands. So it's not to say that AI is not a mass market phenomenon already, but I think from an enterprise perspective they've just been a great buyer for tools like this and the results have been very much measurable.

[Matt Turk]

Yeah, and that's really interesting thoughts in that in some ways generative AI has accelerated a behavior that already existed, meaning that developers have been using Stack overflow for many years. So the idea of using somebody else's code or pre existing code copying and pasting is not a new behavior. Versus a lot of other things that people outside of coding need to do with generative AI, which is figure out how to use those things. In many ways it's leveraging the existing behavior. And also the concept of having code completion in the IDE is not new either. I believe IntelliSense was created in 96 or something like that by Microsoft, which was already a automatic code completion. So that, that may be another reason why adoption has been so dramatically fast.

[David Walter]

Yeah, absolutely. And so maybe to put to, to kind of exemplify a lot of what we've been talking about, we're about 24 months into this wave, give or take. It's changed a ton of behavior. It's taken our community by storm, but it's also just created a ton of really, really interesting special companies that have grown especially quickly. And so cursor maybe three weeks ago announced they're at $500 million var.

[Matt Turk]

Is that actually the fastest growing company of all time? Maybe in B2B?

[David Walter]

I believe so. Lovable which allows people to create prototypes and web apps through a series of prompts, has gone 0 to $60 million of ARR in the last two quarters. GitHub, Copilot, which I know you just had Thomas on the podcast not too long ago. $400 million of ARR, 15 million developers. Very much the steward of this category. First Mover.

[Matt Turk]

Yeah, exactly. The interesting thing that Thomas highlighted, among other many other things, and it's really an episode worth listening to, was how they actually were thinking about AI when GitHub was acquired in 2018. So it's not just that they released Copilot a full year before the whole chatgpt craze, but they had been thinking about it way ahead of time. I thought that was really interesting. And yeah, Copilot came out in 2020 and running on Codex, which was the first coding. Yes, coding model by OpenAI, which came out in 2021, I think, which was basically based on GPT3 at the time.

[David Walter]

And recently has had a large resurgence in a new form. A couple other examples. So V0 again, you just had Guillermo on the podcast.

[Matt Turk]

This is another great episode. Sorry, I'll stop shilling the podcast on the podcast. But that was phenomenal. He's such an incredible founder.

[David Walter]

Windsurf, which is rumored to have sold to OpenAI, hit $100 million of ARR quite recently. And then Replit, which went from 10 to $100 million of ARR in just the last six months.

[Matt Turk]

Yeah, after being at it for a solid 10 plus years. So I also had Amjad on the podcast. I promise I'll stop now. But that was another great episode. He's such an incredibly thoughtful guy. And yeah, I mean, the story of Replit was that it was, from what I understand as an outsider, not an investor, pretty flat for many, many years with a little bit of a kind of like hobbyist student kind of user base. And when they launched their agent a few months ago, it's been. Well, what you see on the screen here, there's just dramatic acceleration. Such a wonderful story.

[David Walter]

Yeah, it's fascinating. You know, historically they were known for being very much in the hobbyist, indie developer educational setting. And what's interesting is actually that is the setting where people are now growing up in an AI native way from an engineering standpoint. And so in many ways, like doubling down and betting on that distribution channel over the long run has actually probably created the moment that they're having. Yep, well said. So anyways, you know, point is just to say, like, this is a very special group of companies that is ramping quickly and has no sign of stopping. And there really is no plateau in sight for innovation. I mean, it's just, you know, for all of your listeners who spend their time on Twitter and TechCrunch and VentureBeat every day, it can feel quite overwhelming. Yeah.

[Matt Turk]

Which is something that we as VCs certainly do not do.

[David Walter]

Never. My argument in this presentation really is that behind all of the marketing gloss, there really is more substance than there ever has been before. And if you, I hate to say this, but if you go offline and you aren't keeping up with the updates for a week at a time, you really might miss something spectacular. And industry shifting. And so far we've seen really remarkable tangible results across our network of engineering organizations. So hopefully this can just put some numbers to at an organizational level, how code generation is really changing engineering process. We've seen a 30 to 50% faster throughput. We've seen 12% increase in PR merges. This is a really important compounding stat here on the bottom left, which is a 17% increase in the amount of time folks are spending on roadmap versus maintenance and keeping the lights on and support. And then in my mind, most impressively, 82% of people we've spoken to are already using AI to write code. And so that is just again, an adoption curve that is pretty much unprecedented. And I think we can talk more about why that is, but I think predominantly it lends itself again to the audience and the arena of distribution ides where people live today. These aren't net new interfaces, for the most part, very much embedded in people's style of working. And again, with very similar distribution mechanisms and keystrokes and delivery models. And so it's just been a fascinating couple years.

[Matt Turk]

On that note of people using AI, I guess there's another topic we could cover, but you know, we've got to pick. We could talk about this for hours. So we got to pick what we discussed. But then there's that whole discussion of autonomous agents versus more of a sort of coding co pilot and I think it's more shades of gray in between. Now. It used to be a little bit of a starker kind of distinction not that long ago. Now it looks like the co pilots are starting to be agentic in many ways. But still, it's like this really interesting question in the industry where for the cursors and the lovable on one hand you have companies like Devon trying to build fully autonomous agents that basically go away and come back with a fully baked product, which by all accounts doesn't seem to be working yet. But that seems to be pointing to a future that is pretty mind blowing.

[David Walter]

Yeah, I would say today what we're seeing is this spectrum of people and use cases that vary by experience and complexity. And for now, many of the agentic solutions that are truly end to end, you know, complete something, are very much being pointed at lower level tasks that I'd say are quote, unquote, more mindless, that have less dependency complexity, you know, that have less of a need for context and knowledge. And that's worked very well. So yeah, there really is a diverse array of ways that you can apply AI to this problem. And I think many of the best organizations and engineering leaders I talk to deploy all sorts of things across their stack, depending again on the use case and the people. You know, I would say despite all of these amazing numbers and stats, I want to pose sort of a historical analogy here to draw to this space basically to say that with production surges, it tends to be the case that actually a lot of problems emerge and in their wake, markets follow. Or maybe a punchier way to say that would be with every surge in production, there's just a cleanup crew that naturally comes and a new market and industry that follows in its wake. Yeah, so if you allow me, we're going to go back in time as far back as the 15th century and talk about a couple examples where we've seen this happen and how we might be able to draw some analogies to today.

[Matt Turk]

Let's do it.

[David Walter]

So starting with the Gutenberg Press, this really was the advent of our ability to print books at scale. And so in the 1440s in Europe, we went from about 40 to 3,6k pages being printed per day. I think over the next 60 years or so, there were 20 million books in circulation in Europe, up from 3,000.

[Matt Turk]

Big disruption. Right. It put monks out of business. The, you know, people that spend their entire lives are copying by hand.

[David Walter]

Yes. They turn to making chartreuse. So anyways, this sounds like a great thing, right? I would hope. We all love books, but actually, like many of the content issues that we see today, there were a lot of challenges that emerged, namely misinformation, mass reproduced quality issues, a bunch of informational overload. And so we saw a bunch of new industries emerge in the wake of the press, which were all the things that we think about when we think about books. The printing, publishing houses, editors, libraries. Now for consumers to deal with the abundance of options that they have almost serving as physical indexes of the many books that they could access. And then a bunch of regulation that came around around licensing and censorship and again all the things that we associate with content today. Another example, almost 500 years later was the Ford assembly line. So in the early 1900s, Henry Ford invented the first continuously moving assembly line and we dropped the time per car assembly wise by almost 90%. And so over the next 20 or so years, the output of Model Ts soared to about 10 million in the US and we went from a society that was mostly horse and carriage, railroad and trolley driven to all of a sudden being a car country. So we still have car challenges today, but if you can imagine, then they felt much more extreme. Things like infrastructure strain, safety issues, the need to build all of the roads and infrastructure to support this new economy, factory workers rights, environmental issues. And so again we saw these industries emerge. I'll highlight the ones that feel very much endemic to auto so quality inspectors, mechanics, dealerships, gas station attendants, and then on the regulation side, all the licensing, our driver's licenses, our license plates, traffic police, and again all of the build out over the following 50 years or so of the freeways, highways, roads that we all use and drive on today. And so, you know, there are a bunch of other examples of this in history. I sort of arbitrarily chose those two. But railroads, map making, the postal system, I think smartphones is probably the most recent example where all of a sudden we have tons of compute in our hand and the ability to take as many pictures as we'd like in a day. And in its wake you've seen a ton of industry emerge, mostly social media, you know, influencer marketing, et cetera. And then all of the regulation around privacy and biometrics, that still feels very top of mind today. And that has been evolving over the last decade or so. And so again, I would just posit, as production surges, you see all of these problems come in the wake of that production and then, and then new industries come around. And so we didn't bury the lead here. If you think about code, I would argue we're very much seeing a similar trend where on the canonical DevOps cycle diagram you're seeing codecommit just accelerate so much. And today at least many of the processes that are downstream of that, our CICD pipelines, our testing suites, our build infrastructure, the way that we think about observability and monitoring has very much stayed the same. But it's kind of breaking. And so I'll talk a Little bit more about that.

[Matt Turk]

Yeah. What happens next? Yeah. On yet another episode of the MAD podcast, just a couple of weeks ago, we had Brendan Humphries, CTO of Canva, who was talking about that. Exactly. Which is okay. It's great that you can create code. But we had one guy who submitted a. A PR that was 50,000 lines and they have a peer review culture and they basically had to make the point that it was not okay to just lob over the fence. 50,000 lines. And now, good luck. Somebody's got to review it. So, yeah, more code.

[David Walter]

Now what to put some numbers to how these pressure points are starting to begin to explode. We're just seeing a lot more time spent debugging. We're seeing a ton more security vulnerabilities, which we alluded to a little bit earlier. And we were talking about how these models have actually been built and on what data. We're seeing a ton of performance issues emerge as a result. You know, I know you did your podcast with Guillermo. I actually did an event with Malte Ubel, who's the CTO at Vercel. And.

[Matt Turk]

Yeah, and that event is on our guild. You know, since this is turning out to be a shilling episode, we can shill the first Mark Guild. So Guilds is an name for our private communities that we have at firstmark, where we have a bunch of people on a per job basis from the portfolio, but also from outside of the portfolio. And this specific event was from our CTO guild. And so you ran a sort of intimate fireside chat with the CTO of Vercel, just for context.

[David Walter]

So this was a great event for our CTO Guild. And Malte, among many interesting things, I think one of the most fascinating things that he said was that most of his great engineers who have been with the company for a while as they've dog fed, you know, V0, and they've used things like Cursor and Windsurf in house, is that many of his great engineers are actually becoming predominantly professional code reviewers. And this notion that as AI code gen has just totally increased the rate at which they're producing and outputting code commits, you need people who almost act as people sitting in a toll booth letting cars pass or not pass, very much doing the same thing with code. But people in those seats that have great taste and know what bad or good or great looks like. And so I thought that was sort of the tip of the iceberg, as you think about. You have these large engineering organizations that have all been trained somewhat similarly on how they should all think about working together and what sorts of jobs that certain people should be doing and not doing and how teams should work together. And we've seen this really, really fast shift where individual ICs who used to be, you know, green dots Everywhere on their GitHub repo are all of a sudden becoming code reviewers. And so, yeah, I thought that was a really interesting analogy and I'll expand more on it, but that's fascinating.

[Matt Turk]

Right, so you used to have front end engineers and back end engineers and full stack people. And yeah, that might be a future where people are none of that and just everybody's a code reviewer, which opens up this whole conversation about how do you become a good engineer in the future in this new age of AI and how people should be trained and all those things. But let's keep going.

[David Walter]

We did a study at Firstmark across, I think it was a little over 300 engineering organizations. And we asked them, just sentiment basically on CodeGen and how it's made them feel about certain metrics that they use to measure their engineering output.

[Matt Turk]

Yeah, so let's go through some of this, including for people that may be listening to this in a podcast only version. So what are some of the metrics and numbers we have here?

[David Walter]

Yeah, so I would say everything in purple, unsurprisingly, which is positive, is very much linked to speed and efficiency and velocity. And so most of these stats are measuring on some way or in some way or another time time spent to develop something, the number of times in a certain period that we commit, how often we're deploying. And so all of those things have been very much sped up in this age of AI. But what's interesting is many of those things don't take into account, like what happens afterwards. And so again, we could make as many commits as we want, but that's very much just the first step in the DevOps process as it exists today. In blue are most of the sort of negatively impacted areas across the survey that we did, which are mostly reliability and process oriented. And so like take mean time to restore, for example. If something goes down, how long does it take to restore it? Well, all of a sudden, if we don't have provenance over the piece of code that's relevant to something that happened, an incident, it's much harder for us to go back and say, oh, actually there was a bug over there. And so that's been what's generally interesting. I think this slide really just goes to say, again, everybody's getting sped up the amount of commits is skyrocketing. But all the stuff that we measure when we think about efficiency and all the downstream things that we want to happen post code commit seem to be faltering.

[Matt Turk]

Yeah, I love it. It re anchors the reality of, okay, it's cool to have AI, but now what does that mean? Right.

[David Walter]

Yeah. So I can be a little more specific here in terms of what's breaking. Yeah, let's do it. We can start with security. Put simply, we're just seeing more security vulnerabilities than ever before and, and new.

[Matt Turk]

Types of security vulnerability.

[David Walter]

Yes, I would say new types of security vulnerabilities, especially given, and this is sort of separate from the conversation about coding, but given deep fakes and, and all the stuff that's happening in the arena of email right now, certainly on that side. And then also in, in, in the world of cogent, our earlier conversation, many of these models have been trained on really large bodies of open source code. And many of those pieces of code just share vulnerabilities and bugs that you can't see coming in the same way because again, you're just not, you're not operating with the same level of decisiveness and meticulousness. And so as a result, you're seeing things like CICD pipelines and build systems completely breaking. We're seeing, I mean, folks have had brittle CICD pipelines for a while now. We've seen a lot of companies emerge over the last decade to sort of ameliorate that and change the way that we think about cicd, but the flake rate is just much higher. And that's true of testing too. So flakes basically is just your ability to either trust or not trust a pass fail result. And if you have a flake, it essentially means something passed and then it failed, but the inputs were the same. And so we're just seeing like flakiness generally across code skyrocketing. I would say build has come under a lot of duress. It's just much harder to cache code, which a lot of build tools try to do when codegen is relatively stochastic. And so again, like the notion that given the same set of inputs or similar inputs, you're just not going to get the same set of outputs. And then QA and code review are areas that we've talked about, but we've seen QA processes just completely overwhelmed. And on the code review side, we're just seeing a new focus on code review that is pretty unrelenting, that I'd say has gone from a step in a process to very much a job in and of itself.

[Matt Turk]

Yeah. And I'm sure we're going to talk about it, but obviously the question is to which extent can AI review AI?

[David Walter]

Unfortunately, for those people that are viewing the podcast, that are engineers or practitioners, all of the problems that I'm talking about are very much our opportunities as investors.

[Matt Turk]

And founders.

[David Walter]

And founders. And we see a ton of them across all of these spaces. So I won't go into every single one, but I'll call out a few on the security side. So there's a. There's a lot of companies, older companies, Fortune, a thousand companies that use EOL software. So maybe they're on like a very old version of CentOS or, you know, maybe they're using a old package, old version of an open source library. And the notion that they could just upgrade is actually quite. It's a difficult thing to do. And so we're seeing a handful of companies that have actually pioneered what's called auto patching. And so the ability just to, instead of like forcing folks to make a potentially breaking change, just patch a project and let things stay status quo and smooth, that's a capability that. You'd be shocked. There are thousands of engineers that have spent days, weeks, months trying to just keep something afloat because of an old dependency or an old package.

[Matt Turk]

Sounds brittle.

[David Walter]

Yeah. You know, across I'll say qa, we're seeing a lot of interesting agentic solutions that are reasoning based on essentially semantic requirements. Better to almost tell the solution what the solution needs to be than to try to do any sort of like diff analysis or anything else that again, given the stochasticism of AI, actually might take you down a dark place. And then in code review, we're just seeing a lot of tools that are getting really fast adoption that are doing essentially agentic first pass checks for code. So, yeah, the space is moving quickly and I would imagine if we did another check in in six months, we'd probably be talking about a whole new slew of problems, but also a bunch of solutions that have addressed many of the things that we've talked about today. It's moving fast.

[Matt Turk]

Yeah. It's fascinating to think about all of this as a system where everything is interdependent and this keeps on shifting.

[David Walter]

I'll use this a bit to shine some light on a handful of the companies across these different categories, we VCs.

[Matt Turk]

So we have to do a little landscape and put logos in categories.

[David Walter]

Correct.

[Matt Turk]

This is what we do best.

[David Walter]

We'll find a better name for this at some point. The AI landscaping, the mad land. You just throw this in the MAD landscape. Right. But yeah, it's been fascinating. And maybe I'll use this slide just to say this is one small sliver of the total companies in this market. And again, back to the point about us being on Twitter and reading blogs and news articles. There are probably a dozen companies that come out every day that sit in this space. And so it's made this job both exhilarating and fun to spend time in.

[Matt Turk]

This category, make things harder on the company side because there's so much noise, you need to just power your way through noise and for people to start noticing you, the bar is higher. So hype in the red hot market comes with pros and cons.

[David Walter]

Yeah. And then most interestingly, I would say, you know, we've talked mostly about engineers themselves, but I think for ctos, this is going to be sort of a hallmark moment for them over the next five years or so where they have a lot of important decisions to make across talent, architecture, team structure, governance principles, security. I wanted to take the opportunity, especially given we were talking to a room of CTOs, just to sort of talk through those things and see what's different. So I'll start with hiring and I'll share an anecdote that I had read recently, which is that computer science grads are actually among the top five or six majors graduating from college right now with the highest unemployment rate. Which is, I mean, it is shocking. And, you know, I think it speaks in many ways to the fact that despite that there's a huge demand for software engineering as a concept, the people who are trained in that practice are actually not in high demand. And so, you know, I very much think, whereas we used to focus on hiring these canonical 10x engineers and developers who can write code, we're seeing CTOs very much focused on hiring great editors and reviewers and prompt engineers who can almost shape and validate and curate that AI generated output.

[Matt Turk]

The CTO of Canva was saying that while this is not the only reason that paused on hiring for a bit, in part to figure out what to do, as we alluded to earlier. Then the question becomes, how do you become a great editor or QA person if you don't write the code and build fundamental knowledge and habits around the core? How do you build the taste that you need to be able to review those things? It's a complicated topic.

[David Walter]

The idea of prompting too, is just fascinating. So I was Having an interesting conversation with an engineer the other day who was talking about the ability to one shot Salesforce. And by one shot they mean, could I recreate Salesforce in just one sitting with one of these tools? I mean, obviously the answer is no. But if you ask why, it's mostly because actually if you're trying to recreate something through a series of prompts, you actually have to really understand what it is that you're creating. And most of us, when we do that, we're starting visually, we think we understand maybe the database logic sitting underneath an app. But Salesforce is a fantastic example of a very, very complex deep app with a ton of different use cases and integrations and reporting structures. And so it's not just the reviewing side, I think that will evolve, but it's also just how we think about prompting and that being a skill set. So that is fascinating. I think on the architecture side too, we're seeing a similarly sized shift where we used to sort of define these systems via design docs and then a bunch of human enforced conventions that might have sat in a confluence doc somewhere to follow. Whereas today we're seeing just this huge push to define systems in a very machine enforceable way where you're almost setting these IAC esque guardrails that AI can just conform to and see and know in a way that is much more stress free for a CTO who's putting, I mean, let's say like 10 Devin engineers to work. So that, that, that is also an equally large shift. We alluded to this too. I'd say the third thing is team structure. So we used to, to your point about front end engineers and back end engineers, we used to just organize teams in a very structured way. And one of the fascinating things I think that's come out of AI generally speaking, is we're just seeing companies and teams being able to do a lot more with a lot less. And I think that's especially true in the engineer context, which, you know, again, speaks to this college grad stat. But we're seeing smaller teams where some people just review and manage AI written code. And that's very much okay. And it spans from front end to back end to systems and everything in between.

[Matt Turk]

There is the whole question of what happens as you start getting product people to create code, I mean functioning applications. And how does that fit if everybody's a coder? This change of Persona within the enterprise of who actually produces code and the fact that it can in theory be everyone. What does that mean?

[David Walter]

Yeah, I mean Historically, EPD engineering product design has been a super complicated hierarchy system in companies where you have a ton of different handoffs between engineers and designers and product people and company to company. It always looked very different whether product people interface directly with engineers at all, whether they were really there to ideate and then hand off entirely. And so to your point, as the technical barrier to create something has dropped, it is straining and changing the way that those teams operate together quite a bit. And your story from Canva is exactly right. Like this notion that we would do like a peer review of that much code, there needs to be a change in the way that folks think about governance. And this culture of reviews in general is changing very quickly. And so, and by the way, to.

[Matt Turk]

Close the loop on that story, I think the punchline was that the CTO reinforced the fact that every full review needs to be a few hundred lines, not thousands, certainly not tens of thousands. But that was the outcome. So, like, do whatever you want with AI, but like whatever you pull over the fence needs to be a few hundred lines.

[David Walter]

Yeah. Enforcing constraints, yes. A big thing we've seen come out of this is this notion of provenance, which is in layman's terms, essentially the lineage of code. So who owns it, what are the dependencies, where did it come from? And increasingly again with just the amount of volume that's, that's that we're seeing across code in general and then the new sort of sources of code that aren't coming from like specific human identities in an organization. This notion of provenance has actually been pushed to its outer limit and remains from a governance standpoint and for a bunch of downstream processes, just super important. The last two things I'll talk about are velocity and security. So on velocity, a lot of engineering folks have historically talked about eliminating bottlenecks. And so, you know, I mean, it could be any number of things that cause a bottleneck in that entire DevOps flow. Whereas now very much there's a focus on containing this idea of entropy, and then managing from a governance standpoint, the coherence of AI systems to system architecture. That's been another really interesting theme where you have these broad based products that are being used by everyone, but everyone's stack looks very different. And how can you conform and give context to an AI product, especially in this world, for what it is that the constraints need to look like? And so we've seen from a velocity standpoint, these notions of entropy and coherence just much more important today than they were two years ago. And then We've talked about security, but I think we'll see a lot of things that come out of this space that look very much like products that are actually catching bugs and problems at the time of write and that will be a very interesting shift in the ASPM world. So yeah, maybe, maybe, maybe to bring it full circle again to this analogy about productivity surges and booms. We see a ton of challenges that come from those historically and we see new industries come in the wake of those challenges. And I think we're very much, for now seeing that in Cogen. And so, you know, I'm as excited as I've ever been to be an investor in this space. I think it's rare to see problems happening as fast as the companies that are solving those problems grow. Everything is just moving so fast. And so again, I think in six months time maybe we'll get back together and this will look either like something that's been completely solved or something where the problems have shifted in form factor completely. But yeah, it's been fascinating and it.

[Matt Turk]

Also feels like a lot of those companies that are doing incredibly well to some extent are also an experiment in the making. Meaning that they are, from what we hear, a lot of unsolved issues in these companies. One is retention. There's a lot of use cases around prototyping and creating new things, but time will tell. Whether that sticks as an industry, we don't know yet. And there's reportedly open questions around gross margins as well, which means that on a unit basis a lot of those companies operate as at a loss. The more they serve customers, the more they lose money. Which I think the industry obviously collectively hopes is just a moment in time that's related to a certain cost structure and then disappears at scale. But it's not for the faint hearted.

[David Walter]

Yeah, look like many spaces in AI at the application layer, many of these products started out with what I would say are very simple delivery models of technology that wasn't quite their own with really smart distribution strategies. And I mean, you know, in many ways I think the IDE companies are pushing the outer limits of what zero switching costs could really look like. But what's interesting is now they all have a war chest of, of money, of usage, of talent, and they are going after much harder problems that are very much differentiated and proprietary. And so, you know, it will be fascinating to see how the space changes over time, but I would say the parameters with which we've or within which we've operated thus far have been very much unique. Right where I mean, we're talking about forking VS code, which is.

[Matt Turk]

Which is what Cursor did.

[David Walter]

That is not something that I would have seen coming if you had asked me in 2018 to imagine what generative AI would look like. Right. But that has been thus far the most impressive business that's been built in this space and they've done a phenomenal job.

[Matt Turk]

So what do we think that means for founders in the space? Do we think there's more opportunity, less opportunity, it's more complicated or clearer now, in particular vis a vis what the large companies are doing? Because certainly Microsoft has been making big moves, but Google has a bunch of products in the space and there seems to be one more competitor somewhere every day. You were mentioning the pace of innovation. Think. A couple of weeks ago, Mistral came up with their own code product, which was a combination of several pieces that they had before. But there's so many companies in the space. So is that a good space if you're thinking of starting a company? Or has the alpha largely left the room?

[David Walter]

I think it's always incredibly easy to say the alphas left the room, especially if you're the person who wants to start a company and you don't know where to begin. Again, we talk about being overwhelmed by headlines. I couldn't imagine being a founder without an idea right now because at once there's a million things to go build. And on the other hand, there's also probably a million people trying to do it. Look, I think you'll get very different perspectives on this. Developers have always been a very opinionated, picky buyer, and that's created a lot of opportunity for a lot of different companies that start with very specific frameworks or ways of doing a certain task or a delivery model, and they get uptake if they are right, at least among a subset of people. And so I think it's rare. And GitHub is a great counterexample of this, which is. I mean, it's almost like a consumer top 1000 Alexa domain. It's probably much higher than a thousand. But in general, like you don't. It's rare that you see ubiquity is what I'm trying to say in the world of developers. So I still believe just on that alone, there's a ton of opportunity left to go build something really interesting. And then two, I would say, again, it's a. It's a market that's young, fast, but also huge. And to everything I've talked about in this, in this presentation, a lot of the early opportunity, you could say the alpha is gone, so to speak. But all of the companies that have absorbed that and captured that have been moving at such breakneck speed that there's just a ton of derivative stuff to go do now. And so it would be like saying, well, aws, Azure and GCP came around and they ate up the whole cloud opportunity. So I guess there's no more money in cloud anymore. Of course that wasn't the case. It just meant that there were going to be all these new things that we needed around the ability to be cloud based. And I mean, we could talk for hours about what those things were. But I think loosely that analogy holds where we have a new way of writing code and there are going to be a lot of products that exist to serve the new needs that come along with that.

[Matt Turk]

And certainly there's been a halo effect to those incredibly fast growing products that have served products that were part of their stack. So famously Supabase and Neon on the database side have had a massive sort of uptake based on the success of Cursor and Lovable. So there are strategies there for startups that are interesting. If you can get close to any of those products, there's some really sort of interesting derived velocity to gain.

[David Walter]

And this is the magic of like the Twitterverse and everything else. You see more reviews and love and hate for software tools in the B2B universe than you do sometimes for like mass market consumer phenomenons. And so yeah, to your point, the ability to become part of the de facto stack for building a company in this era, whether it's on the database side with a postgres database like Supabase or Neon, or, you know, whether it's with the actual tool like Cursor that you're using to then write and deploy that code. That is a huge opportunity right now. And I think with this surge in people that are deciding to build something all at once, there is a good social proof element to what people want to be doing and what's been working and what's not. And I think that's, you know, we talk a lot here at firstmark about what are the new ways that you can have advantages as a company, especially in a world where it's never been easier to build product. And you know, in many ways it feels like we are at risk of being like a copycat world where you see some success online and then you go copy it the next day. But it feels like marketing and distribution and the ability to communicate directly one to one with Your audience has never been more important. And a lot of the companies that we're talking about today do an exceptional job of that.

[Matt Turk]

And it's probably true of all big platform shifts, but in this one it's even more obvious than in prior ones. You can be a one year old or two year old company and actually be a lot more credible than a 5, 7, 10 year old company which may be 10x100x your size. But because you're part of that platform shift and you're like AI native, people take you more seriously than they do, which must be infuriating for the older companies that are just, you know, bigger and have products that work. But that tension between like, you know, young companies with great demos on Twitter versus slightly older companies, we're not talking about companies that have been around for 100 years, you know, is kind of amazing to watch.

[David Walter]

So everybody wants to buy things that were bought by the most discerning people. And the notion of like, who is the most discerning person just seems to ping pong around and change over time. So to your point, most people will care, at least in our world, what the smartest buyer at Cursor thought about a given tool than what the smartest person at the, you know, 80 billion software business that IPO ed in, in 2012 might think.

[Matt Turk]

Yep.

[David Walter]

And I mean, even in my 10 or so years of investing, it's been wild to watch the shift almost socially of like, who are those companies that everybody is looking at for guidance on the right way to do things or the right tool to buy for a certain space? And yeah, you're 100% right. It's a very powerful thing to say. Some of these companies that we talked about today and many others are your customers or are your partners or are people that are willing to even put their name next to yours on an infographic. And so yeah, it moves quickly and it changes often.

[Matt Turk]

And by the way, taking a step back from an investor perspective, it's fascinating that those most successful companies in the AI generative AI world would be developer tools because historically, at least for certain VCs, there was a little bit of a love hate relationship to dev tools. Sometimes it was hard categories, sometimes it was an unloved category. But there was a perception that developers are difficult people. They're very hard to reach, they're cheap, they don't want to pay money. Therefore it's hard to build big developer tool companies. And look, you could argue that between GitLab and Datadog and other companies, the proof was already in the pudding. But this is a sweet revenge for anyone that ever doubted developer tool as a category.

[David Walter]

Yes, I think that sentiment has been shared widely. What's interesting to think about too is just over time how the definition of developer tool has changed. And so, like take HashiCorp, for example. It's a tool that very much. Terraform is a tool that very much developers use. Super successful outcome, you know, recently sold IBM for over $5 billion. I think it was six, $7 billion. And what's been interesting to watch and why I use that example is just again, developers have been, while they have been cheap, so to speak, and picky and opinionated, their relative importance across organizations has seemed to just go up and to the right over the last decade or so. Their ability to make influential decisions on stack products, et cetera, really has seemed to change over time. And it's interesting to say that now, given now we're talking about maybe their skill set is less needed now more than ever. But I think developer tools in and of itself is sort of this loose category that but very much used to be tools that developers use. And maybe that was limited to DevOps and now it feels like it's tools that have either development implications or that have interfaces that developers use, whether or not they're for the development process. And that has yielded a much bigger set of companies. And so Stripe is a great example. Like Stripe is not a developer product in the typical sense. Historically, nobody has thought of payments as a developer problem. But the magic of what that company was was that they were able to cater it to developers and made it really easy to use and adop play around with in a sandbox. And I mean, we all know how that story played out. And so I think, yes, while they've been a hard group to sell into, I think if you can do it right and kind of capture that taste and that like, feeling of capturing the moment, it has yielded some of the larger outcomes that we've seen across our world.

[Matt Turk]

Wonderful. Well, that feels like a great place to live it. David, thanks so much for doing this. This is fun. And indeed, the question is, you know, in six months from now, when we do this again, as we should, should, will all of this be still true? Partly true. Will. I've completely changed in the context where stuff changes every week.

[David Walter]

Yeah. Well, thanks so much for having me. It was a treat.

[Matt Turk]

All right, great. Thanks a lot. Hi, it's Matt Turk again. Thanks for listening to this episode of the MAD podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already, or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.