Transformers and LLMs

Context Window, explained

A context window is the maximum amount of input and generated tokens a model can consider at one time.

Back to Glossary

Part of the Semantic Notion technical glossary.

Short Definition

A context window is the maximum amount of input and generated tokens a model can consider at one time.

Intuition

It is the model's working room. If information falls outside the window, the model cannot directly attend to it.

Technical Definition

Context window length is the supported sequence length for a model's token processing, constrained by architecture, memory, and serving implementation.

Example

A 128K-token context window can hold much more source material than an 8K-token window, but it may cost more to use.

Common Misunderstandings

A larger context window does not guarantee better use of all context.

Long context still needs retrieval, ranking, and evaluation discipline.

Same Category

More from Transformers and LLMs

AttentionAttention is a neural network mechanism that lets a model decide which parts of an input are most relevant when producing a representation or output.Cross-AttentionCross-attention lets one sequence attend to information from a different sequence or modality.Hybrid AttentionHybrid attention combines different attention patterns, such as local and global attention, inside one model.KV CacheA KV cache stores attention keys and values from previous tokens so autoregressive generation does not recompute them every step.Multimodal ModelA multimodal model can process or generate more than one kind of data, such as text, images, audio, or video.Positional EncodingPositional encoding gives a transformer information about token order and position.

Essays that use this concept

Retrieval and RAGWhat Is RAG in AI? Retrieval-Augmented Generation Explained

Start here

Need the broader concept map?

Return to the glossary index to move from this definition into adjacent AI, ML, mathematics, and computer science terms.

Browse Glossary