Token
Definition
A token is the small chunk of text an AI language model reads and writes, usually a short word or a piece of a longer one. In English one token averages about three-quarters of a word, and context windows and API prices are counted in tokens.
By Ronan Liedmeier, fractional head of growth and course author · Published 17 October 2026 · Last updated 17 October 2026
Also called: tokens, tokenization
Tokens are the unit everything gets measured in: how much fits in the context window and what an API call costs.
How it works
Before the model sees your text, a tokenizer splits it into pieces from a fixed vocabulary. A common word like "the" is one token. Rare words, names and long numbers get split into several. OpenAI's rule of thumb for English is about four characters, or three-quarters of a word, per token, so 100 tokens come to roughly 75 words. Other languages can change that ratio.
Paid API use is billed on tokens in (your prompt, files and chat history) and tokens out (the reply). An agent that loops resends the whole history on every step, so input tokens pile up fast.
Example
A support team's triage flow classifies each new ticket and sends the company's full style guide along with every one. The classifier only needs two paragraphs of it. Cutting the rest shrinks the input on every call, and the tickets still land in the same queues.
Common mistake
Teams that never measure tokens per run don't see the waste until the invoice arrives: a large document resent on every loop, or a top model used for simple sorting. Get the output right first, then check cost per run and cut what the model doesn't need to see.
Learn it in AI Academy
- How AI Actually Works (The 20% You Need)Beginner · 12 minFree
- Cost & Latency AwarenessAdvanced · 13 min
Related terms
- Context window
- A context window is the maximum amount of text an AI model can take into account at once, measured in tokens. It holds your messages, any files you add, its instructions and its own replies, and whatever doesn't fit is left out.
- Prompt caching
- Prompt caching is an AI API feature that stores the processed start of a prompt you send repeatedly, so later requests beginning with exactly the same text are cheaper and faster. Only the repeated beginning is reused; the answer is still written fresh each time.
- API (application programming interface)
- An API (application programming interface) is a set of requests a software service agrees to accept from other programs, a bit like a menu. A program sends a request such as "create a contact" and the service sends back data or a confirmation.
- Latency (AI response time)
- Latency is the time an AI system takes to respond, from the moment a request is sent until the answer, or its first words, arrive. In agent workflows it adds up across every model call and tool call in a run.