We talk with Mengye Ren, Assistant Professor at NYU's Center for Data Science, about what intelligence actually means once you step outside a benchmark, and why scaling a single centralized model isn't the whole story.

We get into why intelligence has to be defined in open environments, not closed ones, and what that means for how we measure progress. We push on the creativity question: today's models sample bottom-up from a softmax or a Gaussian, with no internal loop of consideration, and as Mengye puts it, we haven't understood creativity yet and we're already prepared to hand it over.

We also talk about what's missing for the next paradigm: continual learning, memory, embodied grounding, and smaller models that actually accumulate experience instead of re-deriving everything from scratch each call. Along the way, we get into JEPA and latent variables, biology as inspiration vs. blueprint, why frontier labs don't lean on explicit latents, the limits of synthetic data and world models, agent-to-agent communication, model uncertainty and forecasting, and whether ML education still matters when AI writes the experiments.

A grounded, contrarian conversation about where AI research should be looking next, beyond benchmarks, beyond scale.

Timeline

00:00 — Intro and welcome

01:24 — What is intelligence? Defining it relative to objectives and open environments

04:19 — Is intelligence really the path to human flourishing, or is it productivity?

04:57 — Safety, scalable oversight, and whether stronger models help or hurt

06:09 — What does "alignment" actually mean?

07:18 — Centralized vs. decentralized models: objectivity vs. personal meaning

08:50 — Hinton vs. LeCun: where Mengye stands on AI risk

10:29 — Bottom-up vs. top-down architectures and feedback loops

21:28 — Biology and AI: inspiration, not blueprint

24:14 — Biological plausibility, spiking nets, and where the analogy breaks

25:39 — JEPA, Mamba, and architectures beyond the transformer

27:31 — Language as a special modality: abstraction built for communication

29:04 — Are we too locked into the current paradigm? Risk of creativity collapse

30:09 — Synthetic data, simulation, and the brain's own generative models

31:43 — World models and physical AI: how babies actually learn 33:03 — The case for smaller, continually learning models

37:02 — The role of academic research in a frontier-lab world

39:47 — Why LLMs aren't funny: the creativity gap

40:35 — What research areas matter most: embodiment, continual learning, creativity

42:05 — Creativity is bounded by experience — and why bottom-up sampling isn't enough

45:35 — Agent-to-agent communication and the limits of sub-agents

46:39 — Model confidence, epistemic uncertainty, and forecasting

49:44 — Tokenization, static vs. dynamic worlds, and always-learning systems

52:20 — Latent variables, JEPA, and why frontier models skip them

53:40 — The future of ML education when AI writes the experiments

Music:

"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
"Palms Down" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
Changes: trimmed

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.