What Happens When A $30B Founder Uses ChatGPT

TL;DR

The speaker, Darmesh (CTO of HubSpot), gained early access to the GPT API in 2020 and built a chat application two years before ChatGPT launched. He explains core technical concepts: the context window limits how much text an LLM can process at once, vector embeddings and RAG (retrieval augmented generation) allow feeding relevant documents into the context window, and tool calling lets LLMs prete

Key insights

• The context window is a sheet of paper analogy: both the user’s input and the model’s response must fit on that paper; frontier models have ~100,000–200,000 tokens (about one book), which is why large
• Vector embeddings turn documents into points in high-dimensional space (up to 3,072 dimensions with OpenAI’s latest algorithm), enabling semantic search that retrieves the most relevant documents base
• RAG fakes out the context window: a vector store holds all documents, and when a question is asked, the five most relevant documents are pulled and stuffed into the context window, allowing the LLM to