State-Of-The-Art Prompting For AI Agents

TL;DR

The video reveals that state-of-the-art prompting for AI agents involves long, structured prompts (up to six pages) using markdown and XML tags, with a system/developer/user prompt hierarchy. Meta-prompting — where an LLM generates improved versions of its own prompt — is a powerful, widely-used technique. Evals, not prompts, are the true crown-jewel data asset.

Key insights

• The best prompts begin by explicitly setting the LLM's role (e.g., "You're a manager of a customer service agent") and breaking down the task into bullet points, then step-by-step plans.
• XML tag formatting within prompts improves LLM adherence because many models were post-trained on XML-like inputs during RLHF.
• Prompts are split into system prompt (company-wide API/behavior), developer prompt (customer-specific context), and user prompt (end-user input) to avoid becoming a consulting company.