What Is the Context Window?

Also called context length

Short Answer

The context window is the maximum amount of text, counted in tokens, that a language model can consider at once: your instructions, the conversation so far, any documents you add and its own answer. Anything outside the window is invisible to the model. Sizes range from thousands to over a million tokens.

  • Taught in lesson 8

Why It Matters

The context window decides how much information an AI system can use for one answer. Designing assistants, RAG systems and agents is largely about choosing what goes into this limited space, so understanding it is central to building AI that is accurate and affordable.

How It Works

A bigger window lets you give the model more material, but it is not free: cost rises with every token, answers get slower, and research has shown models use information in the middle of a long context less reliably than information at the start or end.

That is why engineers do not just paste everything in. They select the most relevant pieces, with retrieval-augmented generation, and manage what stays in the window, a practice now called context engineering.

Everything counts against the window: the system prompt, retrieved documents, previous messages, tool results and the answer itself. When a conversation grows too long, applications must drop, summarise or retrieve selectively, otherwise old information falls out. A large window helps, but filling it costs money and time, and models do not use every part of a long input equally well, so careful selection usually beats sending everything.

Context Window in Practice

Everyday Example

In a long chat, an assistant may forget a detail you gave at the start once the conversation grows beyond its context window.

Industry Example: Pharma

A full regulatory dossier can exceed any context window, so a submission assistant retrieves only the sections relevant to each question instead of reading everything.

Where You Meet It

  • Model specifications, listed in tokens.
  • Long documents that do not fit in one request.
  • Chat assistants that forget early parts of a long conversation.
  • RAG systems that must choose which passages to include.

Key Ideas Around It

Token limit
The maximum number of tokens in one request and answer.
Truncation
Cutting text that does not fit.
Summarisation
Shortening history so key facts stay in the window.
Context engineering
Choosing what goes into the window for each request.

A Common Mix-Up

A context window is not memory. The model does not remember past conversations between requests; applications have to send the relevant history again each time.

Check YourselfWhy does an assistant sometimes forget what you said at the start of a long chat?

Because the earliest messages no longer fit in the context window, or were summarised away.

Learn It on ai2hired

S0 · #8Tokens, Context Windows and Model Families

Coming in Season 0, Start Here. Every ai2hired lesson is free on YouTube.

Questions People Ask

Is a bigger context window always better?

No. It costs more, runs slower and attention to details can drop. Picking the right content usually beats adding more content.

Does the context window include the answer?

Yes. Input and output share the same window, so a very long prompt leaves less room for the answer.

Learn AI From Zero to Hired

Free lessons on YouTube, each one built from what real AI job ads ask for.