Podcast
Inside the Black Box: Cracking AI and Deep Learning
Hosted by Arshavir Blackwell, PhD · EN
How do Large Language Models like ChatGPT work, anyway?
15episodes
Episodes
Newest firstAll episodes
Building a House Without Blueprints: When Interpretability Tools Work — and When They Don’t
Feb 2700:18:04Tap to summarizeThis episode of Inside the Black Box: Cracking AI and Deep Learning explores a new theoretical framework that unifies sparse autoencoders (SAEs), transcoders, and crosscoders — and what it tells us about when mechanistic interpretability actually works. We start by demystifying these tools and how they use sparse features to uncover internal concepts and computations in large language models, from DNA detectors to deception circuits in Claude 3 Sonnet. Then we introduce the linear representation hypothesis and the geometry of concepts as directions in activation space, along with the challenge of superposition when thousands of concepts must fit into limited dimensions. Finally, we dive into Tang et al.’s recovery theorems, the compressed sensing roots of their approach, and why these results matter for using SAEs as a reliable “microscope” on model internals, especially in the context of fine-tuning and LoRA experiments. Along the way, we confront the uncomfortable possibility that the linear picture may break down at frontier scales — and what that would mean for the future of interpretability as a safety strategy.
Transcribe →
I Told My LLM Not to Say "Empower"
Feb 1900:06:34Tap to summarizeIn this episode of Inside the Black Box: Cracking AI and Deep Learning, Arshavir Blackwell, PhD, takes engineers and researchers inside the practical mechanics of LoRA, low‑rank adaptation methods that make it possible to fine‑tune multi‑billion‑parameter language models on a single GPU.
Transcribe →Beyond the Surface of AI Intelligence
Feb 900:14:43Tap to summarizeThis episode dives into why judging AI by behavior alone falls short of proving true intelligence. We explore how insights from mechanistic interpretability and cognitive science reveal what’s really happening inside AI models. Join us as we challenge the limits of behavioral tests and rethink what intelligence means for future AI.
Transcribe →
Unlocking BERTs Hidden Grammar
Feb 300:09:06Tap to summarizeExplore how BERT’s attention heads reveal an emergent understanding of language structure without explicit supervision. Discover the role of attention as a form of memory and what it means for the future of AI language models.
Transcribe →
Cracking the Code of AI Interpretation
Jan 2800:10:11Tap to summarizeDive into how we naturally explain neural networks with folk interpretability and why these simple stories fall short. Discover the journey toward mechanistic understandability in AI and what that means for how we talk about and trust large language models.
Transcribe →