Better Offline: "AI Isn't Making Engineers 10x As Productive" with Colton Voege

Podcast: Better Offline
Host: Ed Zitron (Cool Zone Media & iHeartPodcasts)
Guest: Colton Voege (Software Engineer)
Air Date: September 19, 2025

Episode Overview

In this episode, host Ed Zitron talks with software engineer and writer Colton Voege about the real-world impact of AI coding tools like large language models (LLMs) on software engineering productivity. The discussion centers around Voege’s essay, “No, AI Is Not Making Engineers 10x as Powerful,” addressing misconceptions about LLMs “replacing” engineers and exploring the actual tasks AI tools can (and can't) help with in day-to-day software development.

Key Discussion Points & Insights

1. What Does a Modern Software Engineer Actually Do?

(03:49 – 05:44)

• Colton Voege describes his work as a web application developer, building interactive applications in the browser (e.g., Amazon, Google Drive).
• Most software engineers today work in web app development, as do most who interact with software via web browsers.

Notable Insight:

“If not the majority of engineers work in web application development, then probably the plurality do right now.”
— Colton Voege (04:18)

2. AI Coding Tools: Where Are They Useful and Where Are They Not?

(06:00 – 07:55)

• AI is “shockingly good” at generating runnable code and especially useful for repetitive, ‘boilerplate’ code—low-intent, high-volume tasks.
• But producing code is just one part of engineering; much of the job is thoughtful planning about architecture, integration, and minimizing “tech debt” (writing code that makes future changes harder).

Notable Quote:

“AI is really good at dealing with things that are annoying...but generating code really isn’t the hard part of being a software engineer.”
— Colton Voege (06:00)

3. Tech Debt, Context, and the Limits of LLMs

(07:55 – 13:19)

• Tech debt is an unavoidable yet manageable part of software. Over-generating boilerplate via LLMs can actually increase tech debt.
• LLMs struggle with code context, such as reusing existing code properly or maintaining consistent style/architecture.

Notable Quote:

“It does coding, but it doesn’t do software engineering. And software engineering is kind of this broader practice of like everything that comes together around coding.”
— Colton Voege (21:35)

4. Programming Languages, Bad Code, and LLM Limitations

(15:48 – 22:42)

• JavaScript’s wild evolution and compatibility legacy make it tricky for LLMs, which are trained on the ‘Internet average’—including bad code.
• LLMs often default to patterns found online, including outdated or incorrect practices, because they statistically infer what “should come next.”

Notable Quotes:

“There are just vast swaths of terrible JavaScript [online].”
— Ed Zitron (20:41)

“They are statistical inference models…very good at generating what they think should come next based on probabilities.”
— Colton Voege (19:56)

5. LLMs in Actual Work: Benefits and Pitfalls

(22:47 – 26:08)

• Voege uses LLMs sometimes for one-off scripts or for working with tools he doesn’t want to learn deeply—but doesn’t trust or scale their use for critical core work.
• “Vibe coding”—quickly generating something without truly understanding it—only makes sense for disposable tasks.
• LLMs can make mistakes that are catastrophic at scale and introduce risks, like importing malicious packages (“slop squatting”).

Memorable Moment:

“You could destroy your entire company with a catastrophic security breach…”
— Colton Voege (28:10)

• LLM hallucinations and error-prone statistical shortcuts are always present risk factors, no matter how ‘good’ the models get.

6. The Productivity “Mirage” and Who’s Really Excited About AI Coding

(34:08 – 37:53)

• Much of the AI productivity hype comes from people who don’t code, including high-profile venture capitalists and tech pundits, or from seeing rare “quick win” moments and extrapolating broadly.
• Many managers, executives, and media don’t understand how complex software jobs really are.

Notable Quote:

“A lot of people don’t know what work is like...they think that things like coding is just...write 10,000 lines of code and then walk home, but now I can write 20 billion lines of code because that’s all my job is.”
— Ed Zitron (36:05)

7. Why LLMs Fail at Context, Integration, and “Real” Productivity

(37:53 – 41:44)

• LLMs can be great at generating toy examples or quick hacks, but quickly fall down when codebases become complex, or when unique styles or deep context are required.
• The analogy to writing: LLMs write high school essays well, but can’t produce coherent or convincing books—loss of thread, inability to build context.

Notable Quote:

“LLMs are really good at writing the classic school level five-paragraph essay. But everybody who actually writes anything at all knows that the five-paragraph essays you wrote in high school are terrible and nobody wants to read something like that.”
— Colton Voege (41:44)

8. LLMs Don’t Replace Human Roles—They’re Shallow Tools

(41:49 – 44:16)

• Recurrent cycles of “AI/automation will replace X role” (graphic designers, copywriters, etc.) always run into the messy reality that these jobs are about human judgment, integration with teams, and handling complex requirements, not just ‘outputting’ product.
• Factory/robotics analogy: human judgment and flexibility are core, not replaceable by automated output.

9. The Flawed Metrics of AI Code Contribution

(43:11 – 45:39)

• Big claims like “30% of Google’s code is written by AI” are misleading; it often means mundane code autocompletion or small snippets, not true creative work.
• Attributing code to AI is like saying “autocorrect wrote parts of your book.”
• Even when AI creates features, a human must spend expertise prompt-engineering, reviewing, and correcting—so the “work” isn’t all shifted to the machine.

Notable Quote:

“Did it write the code? Yes. But did it do the task? Not really. Because it needed somebody else to do some support work to make it even possible for it to do it.”
— Colton Voege (45:16)

Notable Quotes & Timestamps

• “AI is really good at dealing with things that are annoying...but generating code really isn’t the hard part of being a software engineer.” — Colton Voege (06:00)
• “It does coding, but it doesn’t do software engineering.” — Colton Voege (21:35)
• “You could destroy your entire company with a catastrophic security breach...” — Colton Voege (28:10)
• “A lot of people don’t know what work is like...they think that things like coding is just...write 10,000 lines of code...now I can write 20 billion lines of code because that’s all my job is.” — Ed Zitron (36:05)
• “LLMs are really good at writing the classic school level five-paragraph essay. But...the five-paragraph essays you wrote in high school are terrible...” — Colton Voege (41:44)
• “Did it write the code? Yes. But did it do the task? Not really. Because it needed somebody else to do some support work to make it even possible for it to do it.” — Colton Voege (45:16)

Segment Timestamps

• 03:49: Introduction to Colton Voege & his role
• 06:00: What AI coding tools actually do well and where they fail
• 07:55: Tech debt and context in engineering
• 19:56: How LLMs work, and what makes JavaScript tough for them
• 21:35: Coding vs. true software engineering
• 22:47: Real-life uses for LLMs, ‘vibe coding,’ and limits
• 26:10: Dangers—security risks and ‘slop squatting’
• 34:08: Who really claims AI is 10x productivity and why?
• 36:05: Misunderstanding of the software job by non-engineers
• 41:44: LLMs’ inability to write well or maintain context
• 43:11: The myth of massive AI codebase contributions

Tone & Takeaways

The conversation is frank, technical but accessible, and tinged with frustration at tech industry hype. Both Ed and Colton express that excitement about AI is detached from the nuanced, collaborative, and contextual realities of real software engineering. While LLMs are useful for certain repetitive tasks, they cannot replace thoughtful, context-aware engineering—and their risks (technical debt, security, consistency) are underappreciated outside the developer community.

Where to Learn More

• Colton Voege's blog: colton.dev
• Show notes & article: Linked in podcast description