Podcast
GenAI Level UP
Hosted by GenAI Level UP · EN
[AI Generated Podcast]
Learn and Level up your Gen AI expertise from AI. Everyone can listen and learn AI any time, any where.
Whether you're just starting or looking to dive deep, this series covers everything from Level 1 to 10 – from foundational concepts like neural networks to advanced topics like multimodal models and ethical AI. Each level is packed with expert insights, actionable takeaways, and engaging discussions that make learning AI accessible and inspiring.
🔊 Stay tuned as we launch this transformative learning adventure – one podcast at a time. Let’s level up together! 💡✨
21episodes
Episodes
Newest firstAll episodes
The Great Undertraining: How a 70B Model Called Chinchilla Exposed the AI Industry's Billion-Dollar Mistake
Aug 3, 202500:13:37Tap to summarizeFor years, a simple mantra has cost the AI industry billions: bigger is always better. The race to scale models to hundreds of billions of parameters—from GPT-3 to Gopher—seemed like a straight line to superior intelligence. But this assumption contains a profound and expensive flaw.This episode reveals the non-obvious truth: many of the world's most powerful LLMs are profoundly undertrained, wasting staggering amounts of compute on a suboptimal architecture. We dissect the groundbreaking research that proves it, revealing a new, radically more efficient path forward.Enter Chinchilla, a model from DeepMind that isn't just an iteration; it's a paradigm shift. We unpack how this 70B parameter model, built for the exact same cost as the 280B parameter Gopher, consistently and decisively outperforms it. This isn't just theory; it's a new playbook for building smarter, more efficient, and more capable AI. Listen now to understand the future of LLM architecture before your competitors do.In This Episode, You Will Learn:[01:27] The 'Bigger is Better' Dogma: Unpacking the hidden, multi-million dollar flaw in the conventional wisdom of LLM scaling.[03:32] The Critical Question: For a fixed compute budget, what is the optimal, non-obvious balance between model size and training data?[04:28] The 1:1 Scaling Law: The counterintuitive DeepMind breakthrough proving that model size and data must be scaled in lockstep—a principle most teams have been missing.[06:07] The Sobering Reality: Why giants like GPT-3 and Gopher are now considered "considerably oversized" and undertrained for their compute budget.[07:12] The Chinchilla Blueprint: Designing a model with a smaller brain but a vastly larger library, and why this is the key to superior performance.[08:17] The Verdict is In: The hard data showing Chinchilla's uniform outperformance across MMLU, reading comprehension, and truthfulness benchmarks.[10:10] The Ultimate Win-Win: How a smaller, smarter model delivers not only better results but a massive reduction in downstream inference and fine-tuning costs.[11:16] Beyond Performance: The surprising evidence that optimally trained models can also exhibit significantly less gender bias.[13:02] The Next Great Bottleneck: A provocative look at the next frontier—what happens when we start running out of high-quality data to feed these new models?
Transcribe →
DeepSeek LLM: The Open Source AI Revolution
Jan 24, 202500:15:42Tap to summarizeDive into the groundbreaking world of DeepSeek LLM, an open-source language model that's challenging the dominance of closed-source AI. This episode unpacks the secrets behind DeepSeek's impressive capabilities, exploring its unique Mixture-of-Experts (MoE) architecture that optimizes performance and allows it to run efficiently on consumer-grade hardware. We'll delve into its multi-stage training process, from massive pre-training to supervised fine-tuning and reinforcement learning, revealing how DeepSeek learns through trial and error, even developing human-like self-verification and reflection. Discover how DeepSeek excels in diverse domains, from complex math and coding challenges to general reasoning tasks, often outperforming even established models. We'll also explore DeepSeek's specialized tools like DeepSeek Coder and DeepSeek Math, demonstrating its versatility, and look at how its knowledge distillation process allows smaller models to inherit its advanced reasoning abilities, making powerful AI more accessible to all. Join us as we explore the potential impact of DeepSeek, both for the scientific community and for everyday applications, and discuss the ethical considerations that come with these advanced AI tools.
Transcribe →
Titans: Learning to Memorize at Test Time
Jan 18, 202500:17:36Tap to summarizeAre current AI models hitting a memory wall? Join us as we delve into the fascinating research behind "Titans: Learning to Memorize at Test Time," an innovative approach to AI learning. The podcast covers key concepts from the paper, including: The challenges of long-term memory in AI, noting that models like Transformers are good at understanding immediate relationships but struggle with retaining information from the past. How the Titan model addresses these limitations by equipping AI with both short-term and long-term memory. The concept of "learning to memorize at test time", where the model figures out what is important to remember as it encounters new information. The use of a surprise-based approach, where the model prioritizes information that is most surprising or unexpected. The combination of surprise-based long-term memory with a more traditional short-term memory. The way long-term memory is stored, which is within the parameters of a deep neural network. The use of a technique similar to gradient descent with momentum for efficient memory formation. The model's built-in forgetting mechanism to manage memory capacity and prioritize important information. The use of attention to guide the search for relevant information in long-term memory. The ability of Titans to handle longer sequences of information by using long-term memory to free up short-term memory. The advantages of Titans in real-world applications such as language modeling, common sense reasoning, and the needle in a haystack problem. The three variants of the Titan architecture: Memory as a Context (MAC), Memory as a Gate (MAG), and Memory as a Layer (MAL). Each variant uses long-term memory differently.
Transcribe →
Memory Layers at Scale: Revolutionizing AI Efficiency and Factuality
Jan 3, 202500:19:10Tap to summarizeJoin us for an in-depth exploration of the groundbreaking research paper, "Memory Layers at Scale." Discover how trainable key-value lookup mechanisms are transforming the landscape of AI by making large-scale models more efficient, accurate, and capable of continuous learning. We'll unpack the innovations behind memory layers, including product-key lookup and parallel memory techniques, and discuss their implications for democratizing AI development. Learn how these advancements are paving the way for smarter, more adaptable AI systems while addressing challenges like computational efficiency, scalability, and ethical considerations. Whether you're an AI enthusiast, a researcher, or just curious about the future of intelligent systems, this episode offers insights into a paradigm shift in AI development.
Transcribe →
LOCOMO: Unlocking Long-Term Memory in Conversational AI
Dec 31, 202400:08:23Tap to summarizeHow well can AI remember and use information in long conversations? This episode explores the groundbreaking LOCOMO dataset, a unique resource designed to evaluate long-term conversational memory in Large Language Models (LLMs). We delve into the challenges of current AI in maintaining coherent, empathetic conversations over multiple sessions. Discover how the LOCOMO dataset, generated through a human-machine pipeline with unique personas, temporal event graphs, and multimodal dialogue capabilities, is pushing the boundaries of conversational AI. We discuss key findings from experiments using base models, long-context LLMs, and Retrieval Augmented Generation (RAG) techniques, revealing limitations and promising approaches for improving long-term memory. We'll also examine the ethical considerations of creating realistic conversational agents that can remember our past interactions. Learn about the importance of structured information like observations about speakers and retrieval based methods, in order to create truly conversational AI.
Transcribe →
Beyond the Short Chat: Exploring Long-Term Memory in AI
Dec 30, 202400:12:34Tap to summarizeReady for a deep dive into the fascinating world of large language models? In this episode, we push AI chatbots to their conversational limits—spanning hundreds of turns, multiple sessions, and even images—to find out how well they remember and understand context over time. We delve into a groundbreaking dataset called “Locomo” that evaluates an AI’s ability to recall events, summarize complex stories, and navigate tricky, adversarial questions. We also discuss how giving these models structured notes (or “observations”) can dramatically improve their performance—and why they still struggle with understanding time, cause and effect, and cleverly worded “gotcha” questions. Finally, we look ahead at emerging possibilities when AI gains access to richer, multimodal inputs like audio and video. Join us for a thought-provoking conversation on what it takes to give AI a more human-like sense of memory, context, and experience—and why it matters for the future of technology and society.
Transcribe →
Generative AI: Ethical Considerations, Future Trends, and a Path for Continued Learning - Level 10
Dec 20, 202400:19:30Tap to summarizeThis final episode wraps up our journey into the world of generative AI, providing a crucial overview of the ethical and societal considerations, and emerging trends shaping the future of this rapidly evolving field. We'll synthesize key concepts discussed throughout the series, and highlight resources for continued learning, providing a solid foundation for listeners to further their own exploration of generative AI. In this episode we will: Delve into the ethical implications of generative AI, including discussions on bias, fairness, privacy, intellectual property, and the potential for misuse. We will also cover the importance of responsible AI development and highlight the need for regulatory frameworks. Explore emerging trends in generative AI, such as advancements in model architectures, integration with other technologies, personalization, and sustainability efforts. We will discuss the potential societal impacts of generative AI, including effects on employment, and the importance of human-AI collaboration. Synthesize key learnings from previous episodes to give a comprehensive review of the field of generative AI, ranging from the fundamentals of deep learning, variational autoencoders, and GANs to more advanced topics like diffusion models, multimodal AI, and large language models. Offer a pathway for continued learning, including recommended readings, online courses, and practical exercises. We will highlight resources like the "Mapping the Ethics of Generative AI: A Comprehensive Scoping Review", and others that can support ongoing growth in this area. This episode serves as a springboard for your continued exploration of Generative AI, equipping you with the knowledge to engage thoughtfully with the ethical and societal implications while also helping you to keep up with the latest advancements. #genai #levelup #level10 #learn #generativeai #ai #aipapers #podcast #deeplearning #machinelearning #ethic
Transcribe →
Beyond GPT-4V and Sora: Multi-Modal Generative AI - Level 9
Dec 20, 202400:24:30Tap to summarizeThis podcast offers a comprehensive exploration of multi-modal generative AI. We examine the two dominant families of techniques, the multi-modal large language models (MLLM) and diffusion models, covering their probabilistic modeling procedures, multi-modal architecture designs, and advanced applications in image/video large language models, as well as text-to-image/video generation. We look at how these models are being used in text-to-image/video generation and then dive into the future directions of unified models, controllable generation, and lightweight multi-modal AI. Online Tutorials: "Multimodal Generative AI: Vision, Speech, and Assistants " by Coursera: Offered by Codio, this course covers AI applications in image-to-text, text-to-speech, and speech-to-text tasks, along with the Assistant API. It includes practical labs and exercises to enhance learning. “Technical Fundamentals of Generative AI” by Stanford Online: Developed by the Stanford Institute for Human-Centered Artificial Intelligence (HAI), this course explores the technical aspects of generative AI, including multimodal systems for creating images and videos. It also examines the broader implications of these technologies on society. #genai #levelup #level9 #learn #generativeai #ai #aipapers #podcast #deeplearning #machinelearning #multimodal
Transcribe →
From Noise to Creation: Diffusion Models - Level 8
Dec 19, 202400:14:17Tap to summarizeExplore the revolutionary world of diffusion models, a cutting-edge AI technology that learns to reverse the process of turning data into noise to generate new, high-quality content. We'll break down the science behind these models, including how they use stochastic differential equations (SDEs) to transform data and the role of the score function in guiding the reverse process. We'll discuss how methods like SMLD and DDPM fit into this framework, and examine the differences between VE and VP SDEs, and how they relate to different types of noise. We'll cover sampling methods like predictor-corrector (PC) samplers, and how they combine prediction and correction for better results. You'll also learn about the many applications of diffusion models, including image and music generation, protein design, text-to-image synthesis, controllable text generation and solving inverse problems. We'll touch on conditional generation using techniques like classifier guidance and classifier-free guidance, and how they allow for more control and adaptability. Finally, we'll explore how diffusion models are being used for black-box optimization, and why the quality of training data matters. Online Tutorials: "Understanding Diffusion Models: A Deep Dive into Generative AI" on Unite.AI: An in-depth article exploring the workings of diffusion models and their significance in generative AI. "Diffusion and Score-Based Generative Models" on MIT OpenCourseWare: A tutorial covering the theory, methods, and applications of diffusion and score-based generative models. Whether you're an AI enthusiast, researcher, or curious listener, this episode will ignite your imagination and inspire you to dream big. #genai #levelup #level8 #learn #generativeai #ai #aipapers #podcast #deeplearning #machinelearning #diffusionmodels #sde #diffusion
Transcribe →
Learning with BERT and GPT-3: Bridging the Human-AI Gap - Level 7
Dec 18, 202400:14:04Tap to summarizeJoin us on a fascinating journey into the world of natural language processing, where we explore groundbreaking advancements in AI learning. From BERT's innovative masking strategies to GPT-3's remarkable few-shot learning capabilities, we discuss how these models are transforming our understanding of language and intelligence. Dive into the ethical implications, exciting applications, and the evolving relationship between human creativity and machine intelligence. Whether you're an AI enthusiast or a curious learner, this episode will spark new ideas and redefine how you think about the future of technology. Online Tutorials: "Fine-Tuning BERT for Sentiment Analysis" on Towards Data Science: A step-by-step guide to fine-tuning BERT for sentiment classification tasks. #genai #levelup #level7 #learn #generativeai #ai #aipapers #podcast #deeplearning #machinelearning #bert #gpt #gpt3
Transcribe →