Context Window
Natural Language ProcessingContext Window — the maximum text length a language model can process at once. Measured in tokens, the window covers both input and output: An 8K context window means a maximum of 8,000 tokens for prompt and response combined. The limitation arises from the quadratic complexity of the attention mechanism in Transformers — doubling the context quadruples the attention cost. Development has been rapid: from 2K (early GPT models) to 8K (GPT-4) and 1M tokens (for example current Claude and Gemini models), and as many as 10M tokens for Llama 4 Scout, according to Meta. Practically relevant: the more the window holds, the more likely the model is to overlook details, long before it is full (Anthropic calls this “context rot”).
Example
A user feeds a 100-page document (approx. 75K tokens) into a model with an 8K context window — that doesn't work. With a 128K model, the document fits and there are still 53K tokens left for analysis.
Also known as
Context Length