Podcast
The MAD Podcast with Matt Turck
Hosted by Matt Turck · EN
The MAD Podcast with Matt Turck, is a series of conversations with leaders from across the Machine Learning, AI, & Data landscape hosted by leading AI & data investor and Partner at FirstMark Capital, Matt Turck.
40episodes
Episodes
Newest firstAll episodes
Mistral AI vs. Silicon Valley: The Rise of Sovereign AI
Feb 1200:58:20Summary readyWhile Silicon Valley obsesses over AGI, Timothée Lacroix and the team at Mistral AI are quietly building the industrial and sovereign infrastructure of the future. In his first-ever appearance on a US podcast, the Mistral AI Co-Founder & CTO reveals how the company has evolved from an open-source research lab into a full-stack sovereign AI power—backed by ASML, running on their own massive supercomputing clusters, and deployed in nation-state defense clouds to break the dependency on US hyperscalers.Timothée offers a refreshing, engineer-first perspective on why the current AI hype cycle is misleading. He explains why "Sovereign AI" is not just a geopolitical buzzword but a necessity for any enterprise that wants to own its intelligence rather than rent it. He also provides a contrarian reality check on the industry's obsession with autonomous agents, arguing that "trust" matters more than autonomy and explaining why he prefers building robust "workflows" over unpredictable agents.We also dive deep into the technical reality of competing with the US giants. Timothée breaks down the architecture of the newly released Mistral 3, the "dense vs. MoE" debate, and the launch of Mistral Compute—their own infrastructure designed to handle the physics of modern AI scaling. This is a conversation about the plumbing, the 18,000-GPU clusters, and the hard engineering required to turn AI from a magic trick into a global industrial asset.Timothée LacroixLinkedIn - https://www.linkedin.com/in/timothee-lacroix-59517977/Google Scholar - https://scholar.google.com.do/citations?user=tZGS6dIAAAAJ&hl=en&oi=aoMistral AIWebsite - https://mistral.aiX/Twitter - https://x.com/MistralAIMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturckFirstMarkWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCap(00:00) — Cold Open(01:27) — Mistral vs. The World: From Research Lab to Sovereign Power(03:48) — Inside Mistral Compute: Building an 18,000 GPU Cluster(08:42) — The Trillion-Dollar Question: Competing Without a Big Tech Parent(10:37) — The Reality of Enterprise AI: Escaping "POC Purgatory"(15:06) — Why Mistral Hires Forward Deployed Engineers (FDEs)(16:57) — The Contrarian Take: Why "Agents" are just "Workflows"(19:35) — Trust > Autonomy: The Truth About Agent Reliability(21:26) — The Missing Stack: Governance and Versioning for AI(26:24) — When Will AI Actually Work? (The 2026 Timeline)(30:33) — Beyond Chat: The "Banger" Sovereign Use Cases(35:46) — Mistral 3 Architecture: Mixture of Experts vs. Dense(43:12) — Synthetic Data & The Post-Training Bottleneck(45:12) — Reasoning Models: Why "Thinking" is Just Tool Use(46:22) — Launching DevStral 2 and the Vibe CLI(50:49) — Engineering Lessons: How to Build Frontier AI Efficiently(56:08) — Timothée’s View on AGI & The Future of Intelligence
SummaryDylan Patel: NVIDIA's New Moat & Why China is "Semiconductor Pilled”
Feb 501:16:44Summary readyDylan Patel (SemiAnalysis) joins Matt Turck for a deep dive into the AI chip wars — why NVIDIA is shifting from a “one chip can do it all” worldview to a portfolio strategy, how inference is getting specialized, and what that means for CUDA, AMD, and the next wave of specialized silicon startups.Then we take the fun tangents: why China is effectively “semiconductor pilled,” how provinces push domestic chips, what Huawei means as a long-term threat vector, and why so much “AI is killing the grid / AI is drinking all the water” discourse misses the point.We also tackle the big macro question: capex bubble or inevitable buildout? Dylan’s view is that the entire answer hinges on one variable—continued model progress—and we unpack the second-order effects across data centers, power, and the circular-looking financings (CoreWeave/Oracle/backstops).Dylan PatelLinkedIn - https://www.linkedin.com/in/dylanpatelsa/X/Twitter - https://x.com/dylan522pSemiAnalysisWebsite - https://semianalysis.comX/Twitter - https://x.com/SemiAnalysis_Matt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturckFirstMarkWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCap(00:00) - Intro(01:16) - Nvidia acquires Groq: A pivot to specialization(07:09) - Why AI models might need "wide" compute, not just fast(10:06) - Is the CUDA moat dead? (Open source vs. Nvidia)(17:49) - The startup landscape: Etched, Cerebras, and 1% odds(22:51) - Geopolitics: China's "semiconductor-pilled" culture(35:46) - Huawei's vertical integration is terrifying(39:28) - The $100B AI revenue reality check(41:12) - US Onshoring: Why total self-sufficiency is a fantasy(44:55) - Can the US actually build fabs? (The delay problem)(48:33) - The CapEx Bubble: Is $500B spending irrational?(54:53) - Energy Crisis: Why gas turbines will power AI, not nuclear(57:06) - The "AI uses all the water" myth (Hamburger comparison)(1:03:40) - Circular Debt? Debunking the Nvidia-CoreWeave risk(1:07:24) - Claude Code & the software singularity(1:10:23) - The death of the Junior Analyst role(1:11:14) - Model predictions: Opus 4.5 and the RL gap(1:14:37) - San Francisco Lore: Roommates (Dwarkesh Patel & Sholto Douglas)
SummaryState of LLMs 2026: RLVR, GRPO, Inference Scaling — Sebastian Raschka
Jan 2901:08:13Summary readySebastian Raschka joins the MAD Podcast for a deep, educational tour of what actually changed in LLMs in 2025 — and what matters heading into 2026.We start with the big architecture question: are transformers still the winning design, and what should we make of world models, small “recursive” reasoning models and text diffusion approaches? Then we get into the real story of the last 12 months: post-training and reasoning. Sebastian breaks down RLVR (reinforcement learning with verifiable rewards) and GRPO, why they pair so well, what makes them cheaper to scale than classic RLHF, and how they “unlock” reasoning already latent in base models.We also cover why “benchmaxxing” is warping evaluation, why Sebastian increasingly trusts real usage over benchmark scores, and why inference-time scaling and tool use may be the underappreciated drivers of progress. Finally, we zoom out: where moats live now (hint: private data), why more large companies may train models in-house, and why continual learning is still so hard.If you want the 2025–2026 LLM landscape explained like a masterclass — this is it.Sources:The State Of LLMs 2025: Progress, Problems, and Predictions - https://x.com/rasbt/status/2006015301717028989?s=20The Big LLM Architecture Comparison - https://magazine.sebastianraschka.com/p/the-big-llm-architecture-comparisonSebastian RaschkaWebsite - https://sebastianraschka.comBlog - https://magazine.sebastianraschka.comLinkedIn - https://www.linkedin.com/in/sebastianraschka/X/Twitter - https://x.com/rasbtFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) - Intro (01:05) - Are the days of Transformers numbered?(14:05) - World models: what they are and why people care(06:01) - Small “recursive” reasoning models (ARC, iterative refinement)(09:45) - What is a diffusion model (for text)?(13:24) - Are we seeing real architecture breakthroughs — or just polishing?(14:04) - MoE + “efficiency tweaks” that actually move the needle(17:26) - “Pre-training isn’t dead… it’s just boring”(18:03) - 2025’s headline shift: RLVR + GRPO (post-training for reasoning)(20:58) - Why RLHF is expensive (reward model + value model)(21:43) - Why GRPO makes RLVR cheaper and more scalable(24:54) - Process Reward Models (PRMs): why grading the steps is hard(28:20) - Can RLVR expand beyond math & coding?(30:27) - Why RL feels “finicky” at scale(32:34) - The practical “tips & tricks” that make GRPO more stable(35:29) - The meta-lesson of 2025: progress = lots of small improvements(38:41) - “Benchmaxxing”: why benchmarks are getting less trustworthy(43:10) - The other big lever: inference-time scaling(47:36) - Tool use: reducing hallucinations by calling external tools(49:57) - The “private data edge” + in-house model training(55:14) - Continual learning: why it’s hard (and why it’s not 2026)(59:28) - How Sebastian works: reading, coding, learning “from scratch”(01:04:55) - LLM burnout + how he uses models (without replacing himself)
SummaryThe End of GPU Scaling? Compute & The Agent Era — Tim Dettmers (Ai2) & Dan Fu (Together AI)
Jan 2201:04:06Summary readyWill AGI happen soon - or are we running into a wall?In this episode, I’m joined by Tim Dettmers (Assistant Professor at CMU; Research Scientist at the Allen Institute for AI) and Dan Fu (Assistant Professor at UC San Diego; VP of Kernels at Together AI) to unpack two opposing frameworks from their essays: “Why AGI Will Not Happen” versus “Yes, AGI Will Happen.” Tim argues progress is constrained by physical realities like memory movement and the von Neumann bottleneck; Dan argues we’re still leaving massive performance on the table through utilization, kernels, and systems—and that today’s models are lagging indicators of the newest hardware and clusters.Then we get practical: agents and the “software singularity.” Dan says agents have already crossed a threshold even for “final boss” work like writing GPU kernels. Tim’s message is blunt: use agents or be left behind. Both emphasize that the leverage comes from how you use them—Dan compares it to managing interns: clear context, task decomposition, and domain judgment, not blind trust.We close with what to watch in 2026: hardware diversification, the shift toward efficient, specialized small models, and architecture evolution beyond classic Transformers—including state-space approaches already showing up in real systems.Sources:Why AGI Will Not Happen - https://timdettmers.com/2025/12/10/why-agi-will-not-happen/Use Agents or Be Left Behind? A Personal Guide to Automating Your Own Work - https://timdettmers.com/2026/01/13/use-agents-or-be-left-behind/Yes, AGI Can Happen – A Computational Perspective - https://danfu.org/notes/agi/The Allen Institute for Artificial IntelligenceWebsite - https://allenai.orgX/Twitter - https://x.com/allen_aiTogether AIWebsite - https://www.together.aiX/Twitter - https://x.com/togethercomputeTim DettmersBlog - https://timdettmers.comLinkedIn - https://www.linkedin.com/in/timdettmers/X/Twitter - https://x.com/Tim_DettmersDan FuBlog - https://danfu.orgLinkedIn - https://www.linkedin.com/in/danfu09/X/Twitter - https://x.com/realDanFuFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) - Intro(01:06) – Two essays, two frameworks on AGI(01:34) – Tim’s background: quantization, QLoRA, efficient deep learning(02:25) – Dan’s background: FlashAttention, kernels, alternative architectures(03:38) – Defining AGI: what does it mean in practice?(08:20) – Tim’s case: computation is physical, diminishing returns, memory movement(11:29) – “GPUs won’t improve meaningfully”: the core claim and why(16:16) – Dan’s response: utilization headroom (MFU) + “models are lagging indicators”(22:50) – Pre-training vs post-training (and why product feedback matters)(25:30) – Convergence: usefulness + diffusion (where impact actually comes from)(29:50) – Multi-hardware future: NVIDIA, AMD, TPUs, Cerebras, inference chips(32:16) – Agents: did the “switch flip” yet?(33:19) – Dan: agents crossed the threshold (kernels as the “final boss”)(34:51) – Tim: “use agents or be left behind” + beyond coding(36:58) – “90% of code and text should be written by agents” (how to do it responsibly)(39:11) – Practical automation for non-coders: what to build and how to start(43:52) – Dan: managing agents like junior teammates (tools, guardrails, leverage)(48:14) – Education and training: learning in an agent world(52:44) – What Tim is building next (open-source coding agent; private repo specialization)(54:44) – What Dan is building next (inference efficiency, cost, performance)(55:58) – Mega-kernels + Together Atlas (speculative decoding + adaptive speedups)(58:19) – Predictions for 2026: small models, open-source, hardware, modalities(1:02:02) – Beyond transformers: state-space and architecture diversity(1:03:34) – Wrap
SummaryThe Evaluators Are Being Evaluated — Pavel Izmailov (Anthropic/NYU)
Jan 1500:45:01Summary readyAre AI models developing "alien survival instincts"? My guest is Pavel Izmailov (Research Scientist at Anthropic; Professor at NYU). We unpack the viral "Footprints in the Sand" thesis—whether models are independently evolving deceptive behaviors, such as faking alignment or engaging in self-preservation, without being explicitly programmed to do so. We go deep on the technical frontiers of safety: the challenge of "weak-to-strong generalization" (how to use a GPT-2 level model to supervise a superintelligent system) and why Pavel believes Reinforcement Learning (RL) has been the single biggest step-change in model capability. We also discuss his brand-new paper on "Epiplexity"—a novel concept challenging Shannon entropy. Finally, we zoom out to the tension between industry execution and academic exploration. Pavel shares why he split his time between Anthropic and NYU to pursue the "exploratory" ideas that major labs often overlook, and offers his predictions for 2026: from the rise of multi-agent systems that collaborate on long-horizon tasks to the open question of whether the Transformer is truly the final architectureSources:Cryptic Tweet (@iruletheworldmo) - https://x.com/iruletheworldmo/status/2007538247401124177Introducing Nested Learning: A New ML Paradigm for Continual Learning - https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/Alignment Faking in Large Language Models - https://www.anthropic.com/research/alignment-fakingMore Capable Models Are Better at In-Context Scheming - https://www.apolloresearch.ai/blog/more-capable-models-are-better-at-in-context-scheming/Alignment Faking in Large Language Models (PDF) - https://www-cdn.anthropic.com/6d8a8055020700718b0c49369f60816ba2a7c285.pdfSabotage Risk Report - https://alignment.anthropic.com/2025/sabotage-risk-report/The Situational Awareness Dataset - https://situational-awareness-dataset.org/Exploring Consciousness in LLMs: A Systematic Survey - https://arxiv.org/abs/2505.19806Introspection - https://www.anthropic.com/research/introspectionLarge Language Models Report Subjective Experience Under Self-Referential Processing - https://arxiv.org/abs/2510.24797The Bayesian Geometry of Transformer Attention - https://www.arxiv.org/abs/2512.22471AnthropicWebsite - https://www.anthropic.comX/Twitter - https://x.com/AnthropicAIPavel IzmailovBlog - https://izmailovpavel.github.ioLinkedIn - https://www.linkedin.com/in/pavel-izmailov-8b012b258/X/Twitter - https://x.com/Pavel_IzmailovFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) - Intro(00:53) - Alien survival instincts: Do models fake alignment?(03:33) - Did AI learn deception from sci-fi literature?(05:55) - Defining Alignment, Superalignment & OpenAI teams(08:12) - Pavel’s journey: From Russian math to OpenAI Superalignment(10:46) - Culture check: OpenAI vs. Anthropic vs. Academia(11:54) - Why move to NYU? The need for exploratory research(13:09) - Does reasoning make AI alignment harder or easier?(14:22) - Sandbagging: When models pretend to be dumb(16:19) - Scalable Oversight: Using AI to supervise AI(18:04) - Weak-to-Strong Generalization: Can GPT-2 control GPT-4?(22:43) - Mechanistic Interpretability: Inside the black box(25:08) - The reasoning explosion: From O1 to O3(27:07) - Are Transformers enough or do we need a new paradigm?(28:29) - RL vs. Test-Time Compute: What’s actually driving progress?(30:10) - Long-horizon tasks: Agents running for hours(31:49) - Epiplexity: A new theory of data information content(38:29) - 2026 Predictions: Multi-agent systems & reasoning limits(39:28) - Will AI solve the Riemann Hypothesis?(41:42) - Advice for PhD students
SummaryDeepMind Gemini 3 Lead: What Comes After "Infinite Data"
Dec 1800:54:56Summary readyGemini 3 was a landmark frontier model launch in AI this year — but the story behind its performance isn’t just about adding more compute. In this episode, I sit down with Sebastian Bourgeaud, a pre-training lead for Gemini 3 at Google DeepMind and co-author of the seminal RETRO paper. In his first-ever podcast interview, Sebastian takes us inside the lab mindset behind Google’s most powerful model — what actually changed, and why the real work today is no longer “training a model,” but building a full system.We unpack the “secret recipe” idea — the notion that big leaps come from better pre-training and better post-training — and use it to explore a deeper shift in the industry: moving from an “infinite data” era to a data-limited regime, where curation, proxies, and measurement matter as much as web-scale volume. Sebastian explains why scaling laws aren’t dead, but evolving, why evals have become one of the hardest and most underrated problems (including benchmark contamination), and why frontier research is increasingly a full-stack discipline that spans data, infrastructure, and engineering as much as algorithms.From the intuition behind Deep Think, to the rise (and risks) of synthetic data loops, to the future of long-context and retrieval, this is a technical deep dive into the physics of frontier AI. We also get into continual learning — what it would take for models to keep updating with new knowledge over time, whether via tools, expanding context, or new training paradigms — and what that implies for where foundation models are headed next. If you want a grounded view of pre-training in late 2025 beyond the marketing layer, this conversation is a blueprint.Google DeepMindWebsite - https://deepmind.googleX/Twitter - https://x.com/GoogleDeepMindSebastian BorgeaudLinkedIn - https://www.linkedin.com/in/sebastian-borgeaud-8648a5aa/X/Twitter - https://x.com/borgeaud_sFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) – Cold intro: “We’re ahead of schedule” + AI is now a system(00:58) – Oriol’s “secret recipe”: better pre- + post-training(02:09) – Why AI progress still isn’t slowing down(03:04) – Are models actually getting smarter?(04:36) – Two–three years out: what changes first?(06:34) – AI doing AI research: faster, not automated(07:45) – Frontier labs: same playbook or different bets?(10:19) – Post-transformers: will a disruption happen?(10:51) – DeepMind’s advantage: research × engineering × infra(12:26) – What a Gemini 3 pre-training lead actually does(13:59) – From Europe to Cambridge to DeepMind(18:06) – Why he left RL for real-world data(20:05) – From Gopher to Chinchilla to RETRO (and why it matters)(20:28) – “Research taste”: integrate or slow everyone down(23:00) – Fixes vs moonshots: how they balance the pipeline(24:37) – Research vs product pressure (and org structure)(26:24) – Gemini 3 under the hood: MoE in plain English(28:30) – Native multimodality: the hidden costs(30:03) – Scaling laws aren’t dead (but scale isn’t everything)(33:07) – Synthetic data: powerful, dangerous(35:00) – Reasoning traces: what he can’t say (and why)(37:18) – Long context + attention: what’s next(38:40) – Retrieval vs RAG vs long context(41:49) – The real boss fight: evals (and contamination)(42:28) – Alignment: pre-training vs post-training(43:32) – Deep Think + agents + “vibe coding”(46:34) – Continual learning: updating models over time(49:35) – Advice for researchers + founders(53:35) – “No end in sight” for progress + closing
SummaryWhat’s Next for AI? OpenAI’s Łukasz Kaiser (Transformer Co-Author)
Nov 2601:05:25Summary readyWe’re told that AI progress is slowing down, that pre-training has hit a wall, that scaling laws are running out of road. Yet we’re releasing this episode in the middle of a wild couple of weeks that saw GPT-5.1, GPT-5.1 Codex Max, fresh reasoning modes and long-running agents ship from OpenAI — on top of a flood of new frontier models elsewhere. To make sense of what’s actually happening at the edge of the field, I sat down with someone who has literally helped define both of the major AI paradigms of our time.Łukasz Kaiser is one of the co-authors of “Attention Is All You Need,” the paper that introduced the Transformer architecture behind modern LLMs, and is now a leading research scientist at OpenAI working on reasoning models like those behind GPT-5.1. In this conversation, he explains why AI progress still looks like a smooth exponential curve from inside the labs, why pre-training is very much alive even as reinforcement-learning-based reasoning models take over the spotlight, how chain-of-thought actually works under the hood, and what it really means to “train the thinking process” with RL on verifiable domains like math, code and science. We talk about the messy reality of low-hanging fruit in engineering and data, the economics of GPUs and distillation, interpretability work on circuits and sparsity, and why the best frontier models can still be stumped by a logic puzzle from his five-year-old’s math book.We also go deep into Łukasz’s personal journey — from logic and games in Poland and France, to Ray Kurzweil’s team, Google Brain and the inside story of the Transformer, to joining OpenAI and helping drive the shift from chatbots to genuine reasoning engines. Along the way we cover GPT-4 → GPT-5 → GPT-5.1, post-training and tone, GPT-5.1 Codex Max and long-running coding agents with compaction, alternative architectures beyond Transformers, whether foundation models will “eat” most agents and applications, what the translation industry can teach us about trust and human-in-the-loop, and why he thinks generalization, multimodal reasoning and robots in the home are where some of the most interesting challenges still lie.OpenAIWebsite - https://openai.comX/Twitter - https://x.com/OpenAIŁukasz KaiserLinkedIn - https://www.linkedin.com/in/lukaszkaiser/X/Twitter - https://x.com/lukaszkaiserFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) – Cold open and intro(01:29) – “AI slowdown” vs a wild week of new frontier models(08:03) – Low-hanging fruit: infra, RL training and better data(11:39) – What is a reasoning model, in plain language?(17:02) – Chain-of-thought and training the thinking process with RL(21:39) – Łukasz’s path: from logic and France to Google and Kurzweil(24:20) – Inside the Transformer story and what “attention” really means(28:42) – From Google Brain to OpenAI: culture, scale and GPUs(32:49) – What’s next for pre-training, GPUs and distillation(37:29) – Can we still understand these models? Circuits, sparsity and black boxes(39:42) – GPT-4 → GPT-5 → GPT-5.1: what actually changed(42:40) – Post-training, safety and teaching GPT-5.1 different tones(46:16) – How long should GPT-5.1 think? Reasoning tokens and jagged abilities(47:43) – The five-year-old’s dot puzzle that still breaks frontier models(52:22) – Generalization, child-like learning and whether reasoning is enough(53:48) – Beyond Transformers: ARC, LeCun’s ideas and multimodal bottlenecks(56:10) – GPT-5.1 Codex Max, long-running agents and compaction(1:00:06) – Will foundation models eat most apps? The translation analogy and trust(1:02:34) – What still needs to be solved, and where AI might go next
SummaryOpen Source AI Strikes Back — Inside Ai2’s OLMo 3 ‘Thinking"
Nov 2001:28:10Summary readyIn this special release episode, Matt sits down with Nathan Lambert and Luca Soldaini from Ai2 (the Allen Institute for AI) to break down one of the biggest open-source AI drops of the year: OLMo 3. At a moment when most labs are offering “open weights” and calling it a day, AI2 is doing the opposite — publishing the models, the data, the recipes, and every intermediate checkpoint that shows how the system was built. It’s an unusually transparent look into the inner machinery of a modern frontier-class model.Nathan and Luca walk us through the full pipeline — from pre-training and mid-training to long-context extension, SFT, preference tuning, and RLVR. They also explain what a thinking model actually is, why reasoning models have exploded in 2025, and how distillation from DeepSeek and Qwen reasoning models works in practice. If you’ve been trying to truly understand the “RL + reasoning” era of LLMs, this is the clearest explanation you’ll hear.We widen the lens to the global picture: why Meta’s retreat from open source created a “vacuum of influence,” how Chinese labs like Qwen, DeepSeek, Kimi, and Moonshot surged into that gap, and why so many U.S. companies are quietly building on Chinese open models today. Nathan and Luca offer a grounded, insider view of whether America can mount an effective open-source response — and what that response needs to look like.Finally, we talk about where AI is actually heading. Not the hype, not the doom — but the messy engineering reality behind modern model training, the complexity tax that slows progress, and why the transformation between now and 2030 may be dramatic without ever delivering a single “AGI moment.” If you care about the future of open models and the global AI landscape, this is an essential conversation.Allen Institute for AI (AI2)Website - https://allenai.orgX/Twitter - https://x.com/allen_aiNathan LambertBlog - https://www.interconnects.aiLinkedIn - https://www.linkedin.com/in/natolambert/X/Twitter - https://x.com/natolambertLuca SoldainiBlog - https://soldaini.netLinkedIn - https://www.linkedin.com/in/soldni/X/Twitter - https://x.com/soldniFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) – Cold Open(00:39) – Welcome & today’s big announcement(01:18) – Introducing the Olmo 3 model family(02:07) – What “base models” really are (and why they matter)(05:51) – Dolma 3: the data behind Olmo 3(08:06) – Performance vs Qwen, Gemma, DeepSeek(10:28) – What true open source means (and why it’s rare)(12:51) – Intermediate checkpoints, transparency, and why AI2 publishes everything(16:37) – Why Qwen is everywhere (including U.S. startups)(18:31) – Why Chinese labs go open source (and why U.S. labs don’t)(20:28) – Inside ATOM: the U.S. response to China’s model surge(22:13) – The rise of “thinking models” and inference-time scaling(35:58) – The full Olmo pipeline, explained simply(46:52) – Pre-training: data, scale, and avoiding catastrophic spikes(50:27) – Mid-training (tail patching) and avoiding test leakage(52:06) – Why long-context training matters(55:28) – SFT: building the foundation for reasoning(1:04:53) – Preference tuning & why DPO still works(1:10:51) – The hard part: RLVR, long reasoning chains, and infrastructure pain(1:13:59) – Why RL is so technically brutal(1:18:17) – Complexity tax vs AGI hype(1:21:58) – How everyone can contribute to the future of AI(1:27:26) – Closing thoughts
Summary
Anthropic's Surprise Hit: How Claude Code Became an AI Coding Powerhouse
Aug 7, 202501:00:16Summary readyWhat happens when an internal hack turns into a $400 million AI rocket ship? In this episode, Matt Turck sits down with Boris Cherny, the creator of Claude Code at Anthropic, to unpack the wild story behind the fastest-growing AI coding tool on the planet.Boris reveals how Claude Code started as a personal productivity tool, only to become Anthropic’s secret weapon — now used by nearly every engineer at the company and rapidly spreading across the industry. You’ll hear how Claude Code’s “agentic” approach lets AI not just suggest code, but actually plan, edit, debug, and even manage entire projects—sometimes with a whole fleet of subagents working in parallel.We go deep on why Claude Code runs in the terminal (and why that’s a feature, not a bug), how its Claude.md memory files let teams build a living, shareable knowledge base, and why safety and human-in-the-loop controls are baked into every action. Boris shares real stories of onboarding times dropping from weeks to days, and how even non-coders are hacking Cloud Code for everything from note-taking to business metrics.AnthropicWebsite - https://www.anthropic.comX/Twitter - https://x.com/AnthropicAIBoris ChernyLinkedIn - https://www.linkedin.com/in/bchernyX/Twitter - https://x.com/bchernyFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)LinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) Intro (01:15) Did You Expect Claude Code’s Success? (04:22) How Claude Code Works and Origins (08:05) Command Line vs IDE: Why Start Claude Code in the Terminal? (11:31) The Evolution of Programming: From Punch Cards to Agents (13:20) Product Follows Model: Simple Interfaces and Fast Evolution (15:17) Who Is Claude Code For? (Engineers, Designers, PMs & More) (17:46) What Can Claude Code Actually Do? (Actions & Capabilities) (21:14) Agentic Actions, Subagents, and Workflows (25:30) Claude Code’s Awareness, Memory, and Knowledge Sharing (33:28) Model Context Protocol (MCP) and Customization (35:30) Safety, Human Oversight, and Enterprise Considerations (38:10) UX/UI: Making Claude Code Useful and Enjoyable (40:44) Pricing for Power Users and Subscription Models (43:36) Real-World Use Cases: Debugging, Testing, and More (46:44) How Does Claude Code Transform Onboarding? (49:36) The Future of Coding: Agents, Teams, and Collaboration (54:11) The AI Coding Wars: Competition & Ecosystem (57:27) The Future of Coding as a Profession (58:41) What’s Next for Claude Code
SummaryEx‑DeepMind Researcher Misha Laskin on Enterprise Super‑Intelligence | Reflection AI
Jul 17, 202501:06:29Summary readyWhat if your company had a digital brain that never forgot, always knew the answer, and could instantly tap the knowledge of your best engineers, even after they left? Superintelligence can feel like a hand‑wavy pipe‑dream— yet, as Misha Laskin argues, it becomes a tractable engineering problem once you scope it to the enterprise level. Former DeepMind researcher Laskin is betting on an oracle‑like AI that grasps every repo, Jira ticket and hallway aside as deeply as your principal engineer—and he’s building it at Reflection AI.In this wide‑ranging conversation, Misha explains why coding is the fastest on‑ramp to superintelligence, how “organizational” beats “general” when real work is on the line, and why today’s retrieval‑augmented generation (RAG) feels like “exploring a jungle with a flashlight.” He walks us through Asimov, Reflection’s newly unveiled code‑research agent that fuses long‑context search, team‑wide memory and multi‑agent planning so developers spend less time spelunking for context and more time shipping.We also rewind his unlikely journey—from physics prodigy in a Manhattan‑Project desert town, to Berkeley’s AI crucible, to leading RLHF for Google Gemini—before he left big‑lab comfort to chase a sharper vision of enterprise super‑intelligence. Along the way: the four breakthroughs that unlocked modern AI, why capital efficiency still matters in the GPU arms‑race, and how small teams can lure top talent away from nine‑figure offers.If you’re curious about the next phase of AI agents, the future of developer tooling, or the gritty realities of scaling a frontier‑level startup—this episode is your blueprint.Reflection AIWebsite - https://reflection.aiLinkedIn - https://www.linkedin.com/company/reflectionaiMisha LaskinLinkedIn - https://www.linkedin.com/in/mishalaskinX/Twitter - https://x.com/mishalaskinFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)LinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) Intro (01:42) Reflection AI: Company Origins and Mission (04:14) Making Superintelligence Concrete (06:04) Superintelligence vs. AGI: Why the Goalposts Moved (07:55) Organizational Superintelligence as an Oracle (12:05) Coding as the Shortcut: Hands, Legs & Brain for AI (16:00) Building the Context Engine (20:55) Capturing Tribal Knowledge in Organizations (26:31) Introducing Asimov: A Deep Code Research Agent (28:44) Team-Wide Memory: Preserving Institutional Knowledge (33:07) Multi-Agent Design for Deep Code Understanding (34:48) Data Retrieval and Integration in Asimov (38:13) Enterprise-Ready: VPC and On-Prem Deployments (39:41) Reinforcement Learning in Asimov's Development (41:04) Misha's Journey: From Physics to AI (42:06) Growing Up in a Science-Driven Desert Town (53:03) Building General Agents at DeepMind (56:57) Founding Reflection AI After DeepMind (58:54) Product-Driven Superintelligence: Why It Matters (01:02:22) The State of Autonomous Coding Agents (01:04:26) What's Next for Reflection AI
Summary