Modern CTO Podcast — “Why You Can’t Go Off Vibes in Software Development”

Guest: Stephen Poletto (Field CTO at Span)
Host: Joel Beasley
Date: March 19, 2026

Overview

This episode tackles the critical pitfalls of making software development decisions “off vibes,” particularly in the rapidly evolving age of AI. Stephen Poletto, Field CTO at Span, joins Joel Beasley to untangle the hype, doomer narratives, and the emerging best practices in the tech industry. Together, they explore how data, discipline, and constraint-based programming are shaping the new reality for engineering teams, and what it means for technology leaders navigating AI’s impact on productivity and team structure.

Key Discussion Points & Insights

1. The Vibes Disconnect: Why Rationality Matters

• Dangers of “going off vibes”: The media and tech industry are incentivized to promote polarized narratives—either AI as an exponential productivity booster or as a catastrophic threat, leaving little room for practical, data-driven perspectives. (04:00)
• Stephen’s mission: Stephen aims to fill that gap with neutral, research-backed analysis.
  • “Let’s use data to inform what’s actually happening and what’s working and what’s not… be a reasonable non-polarized voice.” (01:44, Stephen Poletto)

2. Clickbait Culture & The Risk for Tech Leadership

• Both host and guest admit the pressures of creating sensational headlines, which can ultimately erode rational decision-making.
  • “If everyone’s incentivized for [hype], from me to the billion-dollar media company, what does that do long term to humanity?” (02:47, Joel)

3. Modern Organizational Shifts: AI-Driven Layoffs and Team Structures (05:24–10:00)

• Notable reference: Jack Dorsey at Block laying off half the staff, not due to distress but as an “offensive maneuver for AI efficiency.”
• Two emerging approaches:
  • Lean & Efficient: Achieve similar output with fewer people, focusing on gross margins.
  • Expand & Experiment: Use AI to pursue more product ideas once out-of-reach due to cost or staffing.
• “AI has the potential to compress the process … reduce coordination cost.” (08:59, Stephen)

4. Customizing AI Strategies to Business Context

• There’s no “one size fits all”—leaders must understand their unique stages, industries, and regulatory demands before applying outside advice. (13:10–14:58)
• “I always look: what are the needs of this team in this organization at its current point? ... And then I just look outside to see if there’s anybody... that has come to the same conclusion.” (13:10, Joel)

5. Constraints, Harness Engineering, and Quality in the AI Era

• From “vibe coding” to constraint-based approaches: Early AI adoption led to increased code volume and quality issues, including incidents at major players like AWS.
• Harness Engineering:
  • Delegating tasks to agents with carefully designed environmental constraints—via custom linters, pre-commit hooks, markdown guidelines, and limiting access to sensitive operations.
  • “...All of these practices of how do you create an environment of constraints, and then under those constraints the agents can thrive, that whole practice is… being referred to as harness engineering.” (20:38, Stephen)

6. The Evolving Software Development Loop (22:15–24:18)

• End goal: A future where expressing intent is all that’s needed for perfect execution, akin to a king delegating to officers.
• “Taste and judgment has not yet been expressed by these systems … the cost of shipping features is now so low, they’re shipping too many features … there’s this taste and judgment layer that sits on top.” (22:42–23:51, Stephen quoting the CEO of Linear)

7. Existential Questions Around AI and Human Value

• What remains for humans once AI surpasses us intellectually?
  • Stephen references “Homo Sapiens” and wonders what uniquely human value will look like when intellect is no longer exclusive. (26:28–27:25)
• The risks associated with AI in physical form (robotics) spark more concern than purely software-based automation. (27:25–28:40)

8. AI’s Evolution Is Obvious—But the Path Isn’t (30:14–32:38)

• Rapid AI evolution since GPT’s early days—from autocomplete tools to autonomous bug-fixing PRs without human intervention.
  • “...Now there is no human reviewing that bug... a customer wrote a bug report. Here’s a PR to fix it... That wasn’t happening two years ago…” (31:43, Stephen)

9. Span: Engineering Intelligence for the AI Era

What Span Offers (33:17–36:16)

• Integrates with development tools (source control, ticket systems, AI tools) to build a comprehensive work graph.
• Tracks bottlenecks, investment envelopes, productivity trends, token spend, and adoption of AI tools—not just how people “feel.”

Problems Span Solves

• Quantifying true AI impact vs. perceived productivity
• Identifying and addressing bottlenecks and skill gaps
• Combining tooling with consultative recommendations and benchmarking against anonymized peer data

10. Evidence & Studies: The Productivity Paradox

• Users perceive higher productivity with AI, but cycle times and other metrics sometimes degrade (Meter study, 36:24).
• Span’s advanced methodology:
  • Quantifies “dosage” of AI per pull request.
  • Tracks code review burden, rounds of feedback, post-merge defects.
  • “Code review burden is way higher... as a byproduct of AI adoption.” (40:55, Stephen)
• Emphasis on shifting code quality checks left—baking in constraints early to minimize later errors. (39:10–41:10)

11. Harness Engineering Is the New Guardrail Design

• Span facilitates knowledge sharing as constraint-based practices mature across its customer base.
• “We give a fancy label to it called harness engineering, which is really just guardrail design and safe engineering environments.” (43:00, Stephen)

12. Career Reflections: The Value of Varied Experience

• Stephen’s journey: From Apple and Dropbox, to startup Lattice, to Span—following curiosity and impact rather than preset plans.

• "I've never had a long term plan with my career... That just happened as a function of me chasing impact and intellectual stimulation." (48:06, Stephen)

• Observational insight from Joel: Many top leaders have similar stories—no rigid plan, just a commitment to curiosity, trying new things, and adding value wherever they land. (48:24–49:04)

Notable Quotes & Moments

• On the dangers of vibes:
  • “Last year it was vibe coding... now people are dealing with the hangover.” (16:09, Stephen)
• On AI as junior engineers:
  • “They’re kind of like children or junior engineers… sometimes they follow your instructions and sometimes they do what they want to do.” (19:14, Stephen)
• On concrete approaches:
  • “How do you create an environment of constraints... under those constraints, the agents can thrive.” (20:38, Stephen)
• On the rapid culture shift:
  • “Entering 2026, it feels like a new world order. ... How do we become an AI native software development loop?” (29:00, Stephen)
• On measurement and reality:
  • “Engineers self reported significant productivity gain... but [studies] saw a degradation in productivity. ... People feel more productive but they’re not actually more productive.” (36:24, Stephen)

Timestamps for Major Segments

• 01:44 — Stephen on “Don’t Go Off Vibes”
• 05:24 — AI’s impact on layoffs & org structure (Block/Jack Dorsey examples)
• 10:00–14:58 — Tailoring team structure & AI adoption by business context
• 16:09 — The shift from “vibe coding” to constraint-based AI programming
• 19:14 — Harness engineering & controlling LLMs
• 22:42–23:51 — The role of judgment and “taste” in product decisions
• 26:28–28:40 — Existential questions about AI, humanity & robot risks
• 31:43 — From autocomplete to autonomous pull requests
• 33:17 — What Span does, how it helps engineering leaders
• 36:24 — The Meter study: perceived vs actual AI productivity
• 40:55 — Code review burden and shifting quality left
• 43:00 — Harness engineering = guardrail design
• 48:06 — Stephen's career reflections
• 53:08 — How to get started with Span

Wrap-Up & How to Learn More

• Learn more: https://span.app
• Connect: LinkedIn for updates and upcoming reports
• “Hopefully we can add some value and color to this emerging conversation around AI’s role in the software world.” (53:08, Stephen)

Tone Note: The conversation is a mix of technical insight and candid, playful banter. Both speakers emphasize curiosity, humility, and data-driven leadership in a sector too often swayed by hype.

This summary captures the episode’s narrative arc, directly attributes speaker insights, and highlights essential landmarks for further listening or reference. Perfect for technology leaders, engineering managers, or anyone navigating the AI-software revolution.