Context Window
Natural Language ProcessingContext Window — the maximum text length a language model can process at once. Measured in tokens, the window covers both input and output: An 8K context window means a maximum of 8,000 tokens for prompt and response combined. The limitation arises from the quadratic complexity of the attention mechanism in Transformers — doubling the context quadruples the attention cost. Development has been rapid: from 2K (early GPT models) to 8K (GPT-4), 200K (Claude), and 1M tokens (Gemini). Practically relevant: with long conversations or extensive documents, you quickly hit limits.
Example
A user feeds a 100-page document (approx. 75K tokens) into a model with an 8K context window — that doesn't work. With a 128K model, the document fits and there are still 53K tokens left for analysis.
Also known as
Kontextfenster
Used in
Sources
- Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. NeurIPS 2017
- Anthropic. (2023). Introducing 100K Context Windows
- Jurafsky, D., & Martin, J.H. (2024). Speech and Language Processing (3rd ed. draft)