Context and Tokens
When you have a conversation with an AI coding agent, it might seem like the AI remembers everything you've discussed. But AI agents have limits on how much they can "hold in mind" at once. Understanding these limits helps you communicate more effectively.
What Are Tokens?
AI agents don't process text word by word. They break text into tokens — pieces that might be words, parts of words, or punctuation. The word "understanding" might become two or three tokens. Code often tokenizes differently than prose.
Here's a rough sense of scale:
- "Hello" = 1 token
- "Hello, how are you?" = approximately 5 tokens
- A typical paragraph = around 100 tokens
- A full page of text = 500-700 tokens
- A large code file = thousands of tokens
You don't need to count tokens precisely. Just know that everything you share — your questions, code snippets, conversation history — gets converted to tokens.
The Context Window
The context window is the total number of tokens an AI can consider at once. Think of it as a whiteboard: there's only so much space. When the whiteboard fills up, something has to be erased to make room for new information.
Context windows vary by AI model:
- Smaller models: around 4,000 tokens
- Medium models: around 16,000 tokens
- Larger models: 100,000+ tokens
This window includes everything: your current message, the conversation history, any code or files you've shared, and the AI's own responses.
Why This Matters
When conversations get long, the AI may "forget" earlier parts. That function you discussed twenty messages ago? It might no longer be in the context window. The AI isn't being forgetful on purpose — it literally can't see that information anymore.
This explains why:
- Long conversations sometimes lose coherence
- Sharing huge code files can crowd out other context
- Starting fresh conversations sometimes works better than continuing old ones
Working Within Limits
Effective AI collaboration means being mindful of context. Share relevant code, not entire projects. Summarize earlier decisions when returning to a topic. If the AI seems to have forgotten something important, remind it.
The context window is a constraint, but understanding it turns it into something you can work with rather than fight against.