Latent Space x Unsupervised Learning: Crossover Special

Date: March 29, 2025
Podcast: Latent Space: The AI Engineer Podcast
Guests: Swix and Alessio (Latent Space), Jordan, Jacob & others from Unsupervised Learning

Episode Overview

This special crossover brings together hosts and minds behind two of the most influential AI podcasts: Latent Space and Unsupervised Learning. The panel embarks on a candid, free-ranging discussion about the turbulent, ever-evolving world of AI engineering, focusing on major surprises of the past year, what’s currently under- or overhyped, the shifting product and infrastructure landscape, defensibility at the app layer, and the big outstanding questions for the industry. Both shows’ hosts reflect on the nuances of open source, model company strategies, emerging agents, infra challenges, and more—offering listeners a frank look at where software 3.0 is heading.

1. Key Discussion Points & Insights

Reflecting on the Biggest Surprises in AI (00:55–11:00)

• Reasoning Models Surpass Expectations

  • Swix notes a “whiplash” moment—pretraining scaling laws seemed exhausted, but reasoning models emerged powerfully, practically on cue with Ilya Sutskever’s "Scaling is Dead" talk.

    “It’s so over and then we’re so back, in such a short period.” — Swix (01:36)

• Open Source Progress and Stagnation

  • Panelists are divided: Open source models like DeepSeek advanced rapidly (sometimes to the surprise of the market), yet enterprise uptake lags, with adoption under 5% and declining.

    “Open source model usage in enterprises is at like 5% and going down.” — Swix (02:59)

  • Speed of open-source catching up isn’t systemic, rather “how fast DeepSeek caught up.” Other teams mostly distill from leaders, and DeepSeek’s future open-sourcing is uncertain.

    “There’s no team open source, there’s just different companies and they choose to open source or not.” — Swix (04:16)

• Market & Product Shocks

  • The market reacted dramatically to model releases (e.g., Nvidia dropping 15% after DeepSeek), despite technical breakthroughs being years in the making.
  • App layer dynamics shift: Models are commoditizing quickly, and application differentiation happens at the wrapper/product layer.

• Low-Code Builders Miss the AI Boat

  • Despite their reach, tools like Zapier and Retool missed capturing the AI builder market, which ended up seized by new entrants with fresh perspectives rather than extensions of existing platforms.

    “Not all of them missed it. Why?” — Swix (08:05)

• Apple’s Disappointing AI Play

  • Apple Intelligence launch underwhelmed, mishaps like the BBC misreporting eroded confidence despite Apple’s prime position in personal AI assistants.

Overhyped & Underhyped in AI Right Now (10:34–16:12)

• Overhyped: Agent Frameworks

  • The proliferation of agent frameworks like LangChain is considered overhyped, as foundational patterns are still too volatile for deep framework bets. It’s the “jQuery era of agents,” not React yet.

    “It feels like the jQuery era of agents and LLMs... They’re just building single-file big frameworks.” — Alessio (11:31)

• Underhyped:

  • Apple's Private Cloud Compute: Apple’s privacy-forward cloud compute is seen as a sleeper foundational tech.

  • Memory & Stateful AI: The lack of robust, persistent memory for agents is a bottleneck—true innovation awaits better abstractions for knowledge retention and learning.

    “If there was a better memory abstraction, then a lot of our agents would be smarter and could learn on the job…” — Swix (15:36)

  • Inference and Verticalization: Net-new “model builders” flood the scene, most lacking clear product differentiation and struggling to compete with generalist incumbents.

What Has Product Market Fit in AI, and How Is the Application Layer Evolving? (21:34–36:47)

• Model Companies Are Encroaching on Product Territory

  • Dynamics are changing as foundation model builders (Anthropic, OpenAI) move up the stack, sometimes alienating their biggest model clients (e.g., Cursor in coding tooling).

    “Model companies want to get into the product layer... that’s going to boil up on Cursor vs. Anthropic.” — Jacob (21:34)

  • Agent builders that handle the “schlep” (deep integration, enterprise sales, support) may retain their edge even if foundation models commoditize.

• Current & Emerging AI PMF (Product Market Fit)

  • Clear winners so far: Code copilots, customer support agents, deep research (long-form question answering/reporting).

  • Copilot-style tools unlock $100M+ markets, and OpenAI’s Deep Research product launch reportedly generated “billions in ARR”—a sign of immediate, tangible PMF.

    “For me, the Product Market Fit bar at the time was $100M. What use cases can reasonably fit $100 million?” — Swix (25:45)

  • Voice AI & Summarization are on deck as the next big things, with applications in scheduling, intake, and appointment handling where partial automation yields outsize value.

  • Second Wave Opportunities: Tools that drive revenue (vs. just cost savings) may prove more defensible longer-term, supplanting first-wave BPO-style automation.

Defensibility at the Application Layer & Infra Landscape (39:00–47:43)

• Defensibility: It's All About Network Effects, Brand, Velocity

  • Recognized that classic defensibility tropes (unique data, proprietary models) have largely been red herrings.

  • The real moats: Being the default, staying top-of-mind with buyers, moving fast, and compounding small product advantages.

    “It’s the thousand small things that make the user experience delightful and the speed at which they move...” — Jacob (41:48)

  • Example: Chai Research builds defensibility by being a marketplace with network effects, not by model IP alone.

• Infra: What’s Hot and What’s Not

  • Most promising: Model-adjacent infra (memory, code execution, search) and AI-powered new approaches to cybersecurity.
  • GPU clouds/data centers remain capital-intensive but necessary; infra that simply wraps models faces commodity pricing downward pressure.
  • App Layer Remains Hottest: "It’s not 50/50—applications are far and away more interesting right now." — Swix (46:24)

Unanswered Questions & The Future (50:01–60:58)

• Biggest Unanswered Questions:

  • Reinforcement Learning Beyond Code/Math? Can RL be extended from verifiable domains (like software) into fuzzier, judgment-heavy areas (law, marketing, sales)?

    “If not, we’ll be stuck with agents in verifiable domains and copilots everywhere else.” — Alessio (50:55)

  • Ratcheting Up Reliability: OpenAI’s “rule of nines”—each order of magnitude increase in reliability is exponentially more expensive. Will infra (hardware, e.g., Nvidia’s dominance) keep pace?

    “How are we going to scale this next part? Is Nvidia just going to continue to be dominant?” — Jordan (51:27)

  • Agent Authentication: How will products securely authenticate agents acting on users’ behalf? Will solutions involve crypto or something like biometric auth (Worldcoin, etc.)?

• Quickfire Round Highlights:

  • Dream podcast guests: John Carmack, Andrej Karpathy, the unfiltered OpenAI story.
  • Top news sources: The Latent Space Discord (curated by Swix), in-person conversations in SF, “private” and public discourse often intermingle in surprising ways.

2. Notable Quotes & Timestamps

• “It was so over and then we’re so back in such a short period.” — Swix, on the rapid pivot from scaling laws to reasoning models (01:36)
• “Open source model usage in [enterprises] is at like 5% and going down.” — Swix (02:59)
• “If there were an Illuminati, this would be what they planned…” — Swix, on the timing between pretraining’s ostensible decline and the rise of inference/advanced reasoning (01:50)
• “It’s actually kind of surprising that Nvidia fell like 15% in one day because of DeepSeek.” — Swix (05:41)
• “It takes a certain kind of AI engineering founder mindset to build from whole cloth rather than be tied to existing paradigms.” — Swix, why low-code incumbents missed the AI builder wave (08:23)
• “Right now it feels like... the jQuery era of agents and LLMs. Maybe we need React.” — Alessio (11:31)
• “If there was a better memory abstraction, then a lot of our agents would be smarter and could learn on the job.” — Swix (15:36)
• “I’m still surprised how many net new companies there are training models. I thought we were past that.” — Jacob (16:12)
• “You can make less enemies if you’re just a model builder, right?” — Alessio, on B2B vs. B2C model company dynamics (18:35)
• “For me, the PMF bar was $100 million. At the time, it was Copilot, Jasper… Cursor I think was on there as a coding agent. That list will just grow over time.” — Swix (25:45)
• “It’s the thousand small things that make the user experience, design, everything just delightful … and the speed at which they move…” — Jacob (41:48)
• “It’s not 50/50 [infra vs. apps]—it’s just very clearly the application layer has been way more interesting.” — Swix (46:24)
• “If RL doesn’t work for non-verifiable domains, then we’ll be stuck with agents in code/math, and only copilots for things like sales.” — Alessio (50:55)
• “Each order of magnitude increase in reliability is an order of magnitude increase in compute.” — Jordan, referencing OpenAI (51:27)
• “Sometimes you put a mic in front of someone and they yap for an hour; other times, you put them in front of a prestigious conference and they drop some alpha.” — Swix, on building community and extracting insights (60:15)

3. Timestamps for Major Segments

• 00:55 — Year-in-review surprises: reasoning models, open source, infra, and market dynamics
• 11:41 — Over/under-hyped trends: agent frameworks, Apple's cloud, memory, new model companies
• 21:34 — Model companies moving up the stack, application layer dynamics and shift to product
• 24:20 — Product market fit, investing at the edge vs. waiting for proven markets
• 31:05 — Customer support as AI’s "killer app", wave 2: growth and revenue, not just cost cutting
• 36:47 — Can current-gen models support entire new app verticals, or is progress bottlenecked?
• 39:00 — Defensibility: brand, network effects, velocity, user experience
• 45:09 — Infrastructure: where the value lies, hot and cold categories, capital efficiency
• 50:01 — The biggest unanswered questions: RL, hardware scaling, agent authentication
• 55:44 — Dream podcast guests, news sources, Discord community
• 59:40–61:07 — Plugs: Latent Space newsletter/YouTube/Discord, AI Engineer conference

4. Style & Tone

The episode is lively, highly technical but unvarnished—with the speakers regularly poking fun at themselves, each other, and the AI echo chamber. There's a spirit both of mutual respect and conversational informality (“Schlep is sticky,” “maybe we need React,” “I didn’t think we’d still have new model companies!”), making complex topics feel accessible and engaging for both technical insiders and curious onlookers.

5. Where to Go Next

• Latent Space newsletter, Discord, YouTube: https://latent.space
• Unsupervised Learning podcast / YouTube: ("Just a humble podcast"—subscribe for sharp AI analysis)

This special serves both as a deep pulse-check on AI’s current state and a forward-looking guidepost for anyone involved in building, investing, or simply understanding the next era of software powered by generative AI and LLMs.