4. Tokens and Context Windows, Explained

You'll leave with: a mental model of tokens and context windows, and a quick way to estimate how big your prompt is.

What is a token?

AI models don't read letters or words the way you do. They split text into small pieces called tokens. OpenAI's help center describes tokens as the building blocks of text its models process: they can be as short as one character or as long as a whole word, and spaces, punctuation and partial words all count.

Handy rules of thumb for English, from OpenAI's help center:

• 1 token ≈ 4 characters
• 1 token ≈ ¾ of a word, so 100 tokens ≈ 75 words
• 1-2 sentences ≈ 30 tokens
• 1 paragraph ≈ 100 tokens
• about 1,500 words ≈ 2,048 tokens

Quick maths you can do in your head

Count characters (or words) and divide: characters ÷ 4 ≈ tokens. A 2,000-character prompt is about 500 tokens. A 600-word article is about 800 tokens.

Input, output, and thinking tokens

Usage is tracked in a few categories. OpenAI's help center lists:

• Input tokens: everything you send.
• Output tokens: everything the model writes back.
• Cached tokens: reused tokens from earlier in the conversation, often billed at a reduced rate (this matters mostly if you use an API).
• Reasoning tokens: on some advanced models, extra “thinking steps” that happen before the final answer.

Both input and output count toward limits. OpenAI's developer docs also note that for some models, input and output tokens are priced differently.

What is a context window?

The context window is everything the model can look at when it answers: the text it can “see” right now. Anthropic's documentation describes it as the model's working memory, and it includes your instructions, the whole conversation so far, any documents you pasted, tool results, and the model's own answer being written.

Think of a desk. Your desk has a fixed size. Everything on it, your notes, the papers you pasted, the model's previous replies, takes space. Once the desk is full, something has to come off.

Sizes vary by model and keep growing. Anthropic's docs list some current models with context windows up to 1 million tokens, but that's not the same for every model or every plan. Check the provider's page.

Why bigger doesn't mean “paste everything”

Anthropic's docs warn that more context isn't automatically better: as token count grows, accuracy and recall can degrade, a phenomenon they call context rot. Their research article describes context as a finite resource with diminishing returns: the model has an “attention budget”, and every extra token uses some of it. So curating what's in context matters just as much as how much room you have.

Why AI “forgets” in a long chat

• The chat grew past the window, and older messages were dropped or summarized.
• Important details were buried among lots of other text.
• A new chat starts with an empty desk. The model doesn't remember your previous chat unless the app has a memory feature or you re-supply the info (Projects and skills are for this: Lesson 8).

Related: Prompt Library: each prompt shows a rough token estimate

Mark lesson as complete