The Engineering Unlocks Behind DeepSeek | YC Decoded

TL;DR

DeepSeek R1 (reasoning model) and V3 (base model) achieve performance comparable to OpenAI o1 at a fraction of the cost via algorithmic innovations: FP8 training, mixture-of-experts (37B/671B active parameters), multi-head latent attention (MLA) compressing KV cache by 933x, multi-token prediction (MTP), and Group Relative Policy Optimization (GRPO) for pure reinforcement learning. The $5.

Key insights

• GPUs in AI labs typically achieve only ~35% model flops utilization (MFU); the rest of the time they wait for data movement.
• NVIDIA’s advantage is not just GPUs but an integrated system (InfiniBand, CUDA, developer experience) that makes a cluster behave like “one giant GPU.”
• DeepSeek’s FP8 accumulation fix periodically merges calculations into FP32 to prevent compounding numerical errors.