Context window

A context window is the finite working space for one AI request. The task, system instructions, conversation history, uploaded files, tool descriptions, tool results, and generated answer all use that space. Google says some Gemini models accept 1 million or more tokens, enough for about 50,000 lines of code or eight average English novels. That is a capacity number, not a promise that the model will find every fact. Research has found that long-context models often recover facts better near the beginning or end than in the middle. For client work, the useful question is simple: can the model find the current rule, approved price, and client constraint when the request runs? A full archive is not automatically a good working set.

What uses the window?

Almost everything the model receives and produces uses part of the window. Google’s token guide describes the context window as the combined input and output limit. Its usage records count the prompt, system instructions, conversation history, cached material, tool use, and generated answer. Images, audio, and video also use tokens after processing.

Think of the window as the packet for one job. It may contain standing instructions, the client’s brief, three approved examples, an email thread, notes returned by a search tool, and earlier turns in the chat. The answer needs room in that same packet.

This is why a chat can reach its limit before the visible conversation looks enormous. The interface may carry instructions, files, and tool definitions that sit beside the messages. When the packet is full, the application must reject new material, shorten the history, summarize it, or remove older parts. The product decides which behavior applies.

A token is smaller than a sentence and is not the same as a word. You do not need to count tokens by hand. You do need to remember that each extra policy, transcript, and old draft competes for finite room.

Why can a fact fit and still be missed?

A fact can fit inside the window and still be hard for the model to recover. That is the key limit of capacity alone.

In 2024, Nelson Liu and co-authors published Lost in the Middle in the Transactions of the Association for Computational Linguistics. They tested models on questions using several documents and on tasks that required finding facts. Performance was often highest when the relevant fact appeared near the beginning or end. It fell when the same fact moved into the middle.

Google Research reported a similar U-shaped bias in 2024. The researchers found that models paid more attention to the beginning and end of long input even when those parts were not the relevant ones.

Here is what that looks like in a service firm. A proposal thread starts with the approved price. It then collects six drafts, two meeting transcripts, copied research, and a late scope change. The price still fits. The model uses a number from draft four because that number is harder to separate from the pile.

A reviewer sees a confident total and assumes the model had all the information. It did. The working packet was arranged badly.

Is a bigger window better?

A bigger window helps when a job genuinely needs more source material. It can let a model compare a long contract, trace project decisions, or review many call transcripts without splitting them first. Google’s long-context guide says a 1-million-token window can hold about eight average English novels or more than 200 average podcast transcripts.

More room also brings three costs.

First, the firm pays to process more input. Prices vary by model and change over time, but sending the same 300-page manual on every request costs more than sending the three pages that govern the task.

Second, longer input takes longer to start answering. Google’s guide says longer queries generally increase time to the first token.

Third, the model may become less reliable at finding facts when the request asks for more at once. Google says a model may reach about 99% on a single fact-retrieval query while performance varies widely when several facts must be found in the same context. Its advice is to omit tokens that are not needed.

The right size is enough room for the working set. A client archive is storage. The current brief, approved rules, and relevant examples are context. Moving the entire archive onto the working desk does not make the job easier to check.

Context window, context, and memory

These terms name different parts of the system.

Context is the information available while the model handles a request. The context window is the maximum amount of that information it can hold at once. Memory is information saved outside the current request and brought back later.

A client preference stored in a CRM becomes context when the application retrieves it and places it in the current request. Once it enters, it uses the context window like every other instruction or document.

This distinction explains why an AI can know something in one conversation and miss it in another. The information may exist in storage without being retrieved. It may have been dropped from an old thread. It may still be present but buried in a long packet. Those are different failures and need different fixes.

Context engineering manages the selection. It decides what belongs in the packet, what gets fetched later, and what is removed when it becomes stale. The context window only tells you how much room exists.

How should you manage the window in client work?

Keep a clean working packet for each repeated job. Four habits cover most situations.

Start a fresh thread when the job changes. A proposal, delivery review, and renewal decision should not share one endless conversation. Old turns create plausible material that belongs to another decision.

Put current rules before reference material. Give the model the approved pricing rule, scope boundary, and output format before transcripts and examples. Put the request after the source material. Google’s guide recommends placing the query at the end of a long prompt.

Replace stale material. Do not append a corrected policy below the old one and expect the model to choose. Remove the old version. The same rule applies to superseded briefs and rejected drafts.

Test fact recovery. Before a workflow touches client work, ask questions whose answers sit at the beginning, middle, and end of the packet. A model accepting a file proves only that the file fit. The real test is whether it recovers the facts the work depends on.

This is why documenting the process first matters. A workflow cannot select the right context if nobody has written down which rules and records govern the job. The model limit is real. The operating failure starts earlier when the firm has no source of truth to put inside it.

Where does this sit on the Delivery Model Ladder?

A context window matters at every stage, but who prepares it changes. At Stage 1, Enhanced, on the Delivery Model Ladder, a person assembles the working packet and asks the model to handle more of the task. A larger window makes that easier. It does not change delivery by itself.

Stage 2 starts when an AI workflow selects the client record, current policy, and relevant examples on its own. Context stops being whatever someone happened to paste. It becomes part of process design.

That handoff is part of the Production Gap. A demo works because its creator knows which five documents to upload. Production works only when the system can choose the right five on the hundredth run, after two policies change and another quarter of client email arrives.

The limit is straightforward. A larger window cannot choose an authoritative source, fix a missing rule, or prove an answer is correct. Put the current working set in the window and test whether the model can recover what the task needs.

Quick answers

Does AI remember everything in a long chat? No. Earlier messages use the same finite window as the current request. The product may drop or summarize older material, and information that remains can still be harder to recover in the middle.

What happens when the context window is full? The application must refuse more input, remove older material, shorten it, or start a new request.

Does a larger context window make AI more accurate? It gives the model access to more information. Accuracy still depends on whether the right information was supplied, whether stale material conflicts with it, and whether the model can recover the facts the task needs.

How much context should I provide? The smallest complete working set: current instructions, current facts, and enough examples to show the standard. Leave the archive in storage until the task needs something from it.