Context window

Every request to a model is assembled into a single sequence of tokens. If a long chat or a large document would exceed the window, the application must truncate it, summarise earlier turns or retrieve only the relevant parts. Context windows have grown substantially over time, making it possible to process whole reports or codebases in a single request.

Why it matters: the window determines whether a tool can analyse a long contract in one pass, remember earlier parts of a conversation or work across a large codebase. Larger contexts also cost more, because every token in the window is billed as input on each request.

Example: if you paste a 300-page report into an assistant whose window is too small, it may read only part of it or ask you to split it, while a RAG system would retrieve just the relevant sections.

Limits: a large window does not guarantee the model uses all of it well. The "Lost in the Middle" study (Liu et al., 2023) found that models can miss information placed in the middle of long inputs, so state key instructions and facts clearly.