Latent Space x Unsupervised Learning: Crossover Special
Date: March 29, 2025
Podcast: Latent Space: The AI Engineer Podcast
Guests: Swix and Alessio (Latent Space), Jordan, Jacob & others from Unsupervised Learning
Episode Overview
This special crossover brings together hosts and minds behind two of the most influential AI podcasts: Latent Space and Unsupervised Learning. The panel embarks on a candid, free-ranging discussion about the turbulent, ever-evolving world of AI engineering, focusing on major surprises of the past year, what’s currently under- or overhyped, the shifting product and infrastructure landscape, defensibility at the app layer, and the big outstanding questions for the industry. Both shows’ hosts reflect on the nuances of open source, model company strategies, emerging agents, infra challenges, and more—offering listeners a frank look at where software 3.0 is heading.
1. Key Discussion Points & Insights
Reflecting on the Biggest Surprises in AI (00:55–11:00)
-
Reasoning Models Surpass Expectations
- Swix notes a “whiplash” moment—pretraining scaling laws seemed exhausted, but reasoning models emerged powerfully, practically on cue with Ilya Sutskever’s "Scaling is Dead" talk.
“It’s so over and then we’re so back, in such a short period.” — Swix (01:36)
-
Open Source Progress and Stagnation
-
Panelists are divided: Open source models like DeepSeek advanced rapidly (sometimes to the surprise of the market), yet enterprise uptake lags, with adoption under 5% and declining.
“Open source model usage in enterprises is at like 5% and going down.” — Swix (02:59)
-
Speed of open-source catching up isn’t systemic, rather “how fast DeepSeek caught up.” Other teams mostly distill from leaders, and DeepSeek’s future open-sourcing is uncertain.
“There’s no team open source, there’s just different companies and they choose to open source or not.” — Swix (04:16)
-
Market & Product Shocks
- The market reacted dramatically to model releases (e.g., Nvidia dropping 15% after DeepSeek), despite technical breakthroughs being years in the making.
- App layer dynamics shift: Models are commoditizing quickly, and application differentiation happens at the wrapper/product layer.
-
Low-Code Builders Miss the AI Boat
- Despite their reach, tools like Zapier and Retool missed capturing the AI builder market, which ended up seized by new entrants with fresh perspectives rather than extensions of existing platforms.
“Not all of them missed it. Why?” — Swix (08:05)
-
Apple’s Disappointing AI Play
- Apple Intelligence launch underwhelmed, mishaps like the BBC misreporting eroded confidence despite Apple’s prime position in personal AI assistants.
Overhyped & Underhyped in AI Right Now (10:34–16:12)
What Has Product Market Fit in AI, and How Is the Application Layer Evolving? (21:34–36:47)
Defensibility at the Application Layer & Infra Landscape (39:00–47:43)
-
Defensibility: It's All About Network Effects, Brand, Velocity
-
Recognized that classic defensibility tropes (unique data, proprietary models) have largely been red herrings.
-
The real moats: Being the default, staying top-of-mind with buyers, moving fast, and compounding small product advantages.
“It’s the thousand small things that make the user experience delightful and the speed at which they move...” — Jacob (41:48)
-
Example: Chai Research builds defensibility by being a marketplace with network effects, not by model IP alone.
-
Infra: What’s Hot and What’s Not
- Most promising: Model-adjacent infra (memory, code execution, search) and AI-powered new approaches to cybersecurity.
- GPU clouds/data centers remain capital-intensive but necessary; infra that simply wraps models faces commodity pricing downward pressure.
- App Layer Remains Hottest: "It’s not 50/50—applications are far and away more interesting right now." — Swix (46:24)
Unanswered Questions & The Future (50:01–60:58)
2. Notable Quotes & Timestamps
- “It was so over and then we’re so back in such a short period.” — Swix, on the rapid pivot from scaling laws to reasoning models (01:36)
- “Open source model usage in [enterprises] is at like 5% and going down.” — Swix (02:59)
- “If there were an Illuminati, this would be what they planned…” — Swix, on the timing between pretraining’s ostensible decline and the rise of inference/advanced reasoning (01:50)
- “It’s actually kind of surprising that Nvidia fell like 15% in one day because of DeepSeek.” — Swix (05:41)
- “It takes a certain kind of AI engineering founder mindset to build from whole cloth rather than be tied to existing paradigms.” — Swix, why low-code incumbents missed the AI builder wave (08:23)
- “Right now it feels like... the jQuery era of agents and LLMs. Maybe we need React.” — Alessio (11:31)
- “If there was a better memory abstraction, then a lot of our agents would be smarter and could learn on the job.” — Swix (15:36)
- “I’m still surprised how many net new companies there are training models. I thought we were past that.” — Jacob (16:12)
- “You can make less enemies if you’re just a model builder, right?” — Alessio, on B2B vs. B2C model company dynamics (18:35)
- “For me, the PMF bar was $100 million. At the time, it was Copilot, Jasper… Cursor I think was on there as a coding agent. That list will just grow over time.” — Swix (25:45)
- “It’s the thousand small things that make the user experience, design, everything just delightful … and the speed at which they move…” — Jacob (41:48)
- “It’s not 50/50 [infra vs. apps]—it’s just very clearly the application layer has been way more interesting.” — Swix (46:24)
- “If RL doesn’t work for non-verifiable domains, then we’ll be stuck with agents in code/math, and only copilots for things like sales.” — Alessio (50:55)
- “Each order of magnitude increase in reliability is an order of magnitude increase in compute.” — Jordan, referencing OpenAI (51:27)
- “Sometimes you put a mic in front of someone and they yap for an hour; other times, you put them in front of a prestigious conference and they drop some alpha.” — Swix, on building community and extracting insights (60:15)
3. Timestamps for Major Segments
- 00:55 — Year-in-review surprises: reasoning models, open source, infra, and market dynamics
- 11:41 — Over/under-hyped trends: agent frameworks, Apple's cloud, memory, new model companies
- 21:34 — Model companies moving up the stack, application layer dynamics and shift to product
- 24:20 — Product market fit, investing at the edge vs. waiting for proven markets
- 31:05 — Customer support as AI’s "killer app", wave 2: growth and revenue, not just cost cutting
- 36:47 — Can current-gen models support entire new app verticals, or is progress bottlenecked?
- 39:00 — Defensibility: brand, network effects, velocity, user experience
- 45:09 — Infrastructure: where the value lies, hot and cold categories, capital efficiency
- 50:01 — The biggest unanswered questions: RL, hardware scaling, agent authentication
- 55:44 — Dream podcast guests, news sources, Discord community
- 59:40–61:07 — Plugs: Latent Space newsletter/YouTube/Discord, AI Engineer conference
4. Style & Tone
The episode is lively, highly technical but unvarnished—with the speakers regularly poking fun at themselves, each other, and the AI echo chamber. There's a spirit both of mutual respect and conversational informality (“Schlep is sticky,” “maybe we need React,” “I didn’t think we’d still have new model companies!”), making complex topics feel accessible and engaging for both technical insiders and curious onlookers.
5. Where to Go Next
- Latent Space newsletter, Discord, YouTube: https://latent.space
- Unsupervised Learning podcast / YouTube: ("Just a humble podcast"—subscribe for sharp AI analysis)
This special serves both as a deep pulse-check on AI’s current state and a forward-looking guidepost for anyone involved in building, investing, or simply understanding the next era of software powered by generative AI and LLMs.