Building Dota Bots That Beat Pros - OpenAI's Greg Brockman, Szymon Sidor, and Sam Altman

TL;DR

OpenAI built a Dota 2 bot that beat professional players at The International using reinforcement learning with self-play, heavy engineering infrastructure (Docker, Kubernetes, gRPC), and minimal novel ML research.

Key insights

• The majority of work on the Dota bot was engineering (person‑months) vs. ML science (person‑weeks); many improvements came from scaling existing algorithms, not inventing new ones.
• Selecting a game for AI research depends heavily on non‑technical factors: Linux support, a game API (even if originally for mods), and a company like Valve that supports hackable games.
• Behavioral cloning learns imitation of observed actions, not intent – the bot would mimic movements (e.g., creep blocking) without understanding the underlying goal; fine‑tuning with RL was required t