DeepSeek has taken the AI world by storm, sparking excitement, skepticism, and heated debates. Is this the next big leap in AI reasoning, or is it just another overhyped model? In this episode, we peel back the layers of DeepSeek-R1 and DeepSeek-V3, diving into the technology behind its Mixture of Experts (MoE), Multi-Head Latent Attention (MLA), Multi-Token Prediction (MTP), and Reinforcement Learning (GRPO) approaches. We also take a hard look at the training costs—is it really just $5.6M, or is the actual number closer to $80M-$100M?

Join us as we break down:

DeepSeek’s novel architecture & how it compares to OpenAI’s models
Why MoE and MLA matter for AI efficiency
How DeepSeek trained on 2,048 H800 GPUs in record time
The real cost of training—did DeepSeek underestimate their numbers?
What this means for the future of AI models

At the end of the episode, we answer the big question: DeepSeek – WOW or MEH?

Key Topics Discussed:

DeepSeek-R1 vs. OpenAI’s GPT models
Reinforcement Learning (GRPO) and why it’s a big deal
DeepSeek-V3’s 671B parameters and 37B active parameters
The economics of training large AI models—real vs. reported costs
The impact of MoE, MLA, and MTP on AI inference & efficiency

References & Further Reading:

DeepSeek-R1 Official Paper: https://arxiv.org/abs/2501.12948
Philschmid blog: https://www.philschmid.de/deepseek-r1
DeepSeek Cost Breakdown: Reddit Discussion
DeepSeek AI's Official Announcement: DeepSeek AI Homepage