DeepSeek has taken the AI world by storm, sparking excitement, skepticism, and heated debates. Is this the next big leap in AI reasoning, or is it just another overhyped model? In this episode, we peel back the layers of DeepSeek-R1 and DeepSeek-V3, diving into the technology behind its Mixture of Experts (MoE), Multi-Head Latent Attention (MLA), Multi-Token Prediction (MTP), and Reinforcement Learning (GRPO) approaches. We also take a hard look at the training costs—is it really just $5.6M, or is the actual number closer to $80M-$100M?

Join us as we break down:

• DeepSeek’s novel architecture & how it compares to OpenAI’s models
• Why MoE and MLA matter for AI efficiency
• How DeepSeek trained on 2,048 H800 GPUs in record time
• The real cost of training—did DeepSeek underestimate their numbers?
• What this means for the future of AI models

At the end of the episode, we answer the big question: DeepSeek – WOW or MEH?

Key Topics Discussed:

• DeepSeek-R1 vs. OpenAI’s GPT models
• Reinforcement Learning (GRPO) and why it’s a big deal
• DeepSeek-V3’s 671B parameters and 37B active parameters
• The economics of training large AI models—real vs. reported costs
• The impact of MoE, MLA, and MTP on AI inference & efficiency

References & Further Reading:

• DeepSeek-R1 Official Paper: https://arxiv.org/abs/2501.12948
• Philschmid blog: https://www.philschmid.de/deepseek-r1
• DeepSeek Cost Breakdown: Reddit Discussion
• DeepSeek AI's Official Announcement: DeepSeek AI Homepage