Tokens Explained: The Currency That Determines What AI Can and Cannot Remember

A token is roughly equivalent to three or four characters of text, or approximately three quarters of a word. The word 'understanding' might be represented as two tokens: 'under' and 'standing'. The word 'AI' is a single token. A space, a comma, or a hyphen is typically a separate token.

The reason AI systems work with tokens rather than words or characters is technical: the tokenisation approach allows the model to handle different languages, code, and specialised terminology more efficiently than word-by-word processing would allow.

For practical purposes, you can think of tokens as roughly equivalent to words in terms of measuring length. A page of business text contains approximately 250-300 tokens.

The context window is the maximum number of tokens an AI model can process in a single interaction. It determines how much text the AI can see and consider at once.

If you submit a document to an AI for analysis, the document must fit within the context window. If you are having a long conversation with an AI, the earlier parts of the conversation that no longer fit in the context window are effectively forgotten, because the model can only work with what is currently in its context.

Context windows vary significantly between AI systems. Earlier GPT-3 models had context windows of around 4,000 tokens (roughly 3,000 words). GPT-4 Turbo has a context window of 128,000 tokens. Claude 3.5 Sonnet supports 200,000 tokens. This difference is significant for business tasks that involve long documents: a 200,000 token context window can accommodate a 150-page report or a substantial set of contracts for simultaneous analysis.

For executives choosing between AI tools for specific business tasks, context window size is a practical criterion.

For analysing short documents, answering questions, or drafting communications, context window size is rarely limiting. Standard interactions fit well within even modest context windows.

For analysing long contracts, board papers, regulatory filings, or extensive research documents, context window size matters a great deal. A model with a 4,000 token context window cannot analyse a 50-page document in a single interaction. A model with a 200,000 token context window can analyse multiple documents simultaneously and maintain coherent analysis across all of them.

For agentic workflows where the AI is maintaining context across a long sequence of steps or across a large knowledge base, context window size can be the determining factor in whether the workflow is technically feasible.

AI API pricing is typically measured in cost per thousand tokens (for input and output separately). Understanding tokens therefore helps understand AI cost models.

For organisations deploying AI at scale, token economics matter. Processing a 100-page document costs more in API terms than processing a 10-page document, because the longer document consumes more tokens. Longer responses cost more than shorter responses. Understanding token consumption is therefore part of designing economically sustainable AI workflows at scale.