Context

Context is the working set an AI can use for one request or workflow step. It can hold the task, instructions, earlier messages, approved client records, examples, tool descriptions, and tool results. Anthropic defines it as the tokens included while a model produces an answer. OpenAI also separates information held by the surrounding application from information the model can see. A CRM record does not become context until the workflow sends it in an instruction, input, retrieval result, or tool result. That boundary matters in client work. A renewal workflow may have the right file in storage and still send an expired rate. Good context is the smallest complete set of current, approved information needed for this step.

What counts as context?

Anything the model can use now is context. That includes system instructions, the immediate request, earlier messages that remain in the conversation, examples, retrieved passages, tool definitions, and tool results.

A renewal proposal makes the boundary clear. The firm may store the signed agreement in Drive, current rates in a pricing system, account history in the CRM, and an approved proposal as an example. Those records are available to the application. They help the model only after the workflow selects them and places the useful parts in the request.

OpenAI’s Agents SDK documentation draws the same line between local application context and model-visible context. A customer object, logger, or permission record can stay in the application. The model sees only what the application sends. That can be a small field, a retrieved passage, or the result of a tool call.

People often build this packet by hand during a demo. Someone opens the current file, pastes the relevant paragraphs, removes the old note, and explains which price applies. The answer works because a person selected the context. Production requires the workflow to repeat that selection without relying on one person’s memory.

Prompt, context, and context window

A prompt asks for the job. Context is the prompt plus the other material the model can use. A context window is the capacity for that material and the model’s answer.

Term	Plain meaning	Renewal example
Prompt	The request and task-specific instructions	Draft the renewal with the approved scope and current rate
Context	Everything available to the model for this step	Prompt, scope, rate, account notes, example, and tool results
Context window	The input and output capacity for this request	The packet must fit beside the answer the model will produce

The large language model supplies learned patterns and generates the answer. Context supplies the information for this client and this task. A stronger model may reason better over the packet. It still cannot read a record the application never sends.

Google’s token documentation says the context window covers input and output tokens. System instructions, prompt text, history, cached content, tools, and generated output all use capacity. A larger window gives the workflow more room. It does not decide which version of a scope or policy is current.

Context, memory, and retrieval

Memory is information saved outside the current request for later use. Retrieval-augmented generation searches outside sources and brings selected evidence into the request. Context is what the model can use after those steps.

Suppose a client approved short weekly reports. Saving that preference creates memory. Searching the client record during the next reporting run is retrieval. Putting the approved preference beside the current metrics makes it context for that run.

These parts fail in different ways. Memory can preserve an outdated preference. Retrieval can return the wrong client’s record. Context can contain both the old and current instruction without showing which one is authoritative.

Context engineering is the work around the packet. It decides what enters, what stays out, what gets fetched later, what remains available from an earlier step, and what must be removed. Context is the information. Context engineering is how the system selects and maintains it.

Why can more context hurt the answer?

More input gives stale, repeated, or weak material more chances to compete with the fact that matters.

Chroma’s Context Rot report tested 18 models in July 2025 while keeping the task simple and increasing input length. Performance became less reliable in different ways across the models. The order, similarity, and structure of the information also changed the result.

OpenAI’s current model guidance reports a related lesson from an internal coding-agent sample. Leaner system prompts improved evaluation scores by about 10 to 15 percent while cutting total tokens by 41 to 66 percent and cost by 33 to 67 percent. OpenAI calls those ranges directional and asks teams to test representative work. They are not a service-firm benchmark.

Long context does not fail in one fixed way. Google Research’s 2024 positional-attention study found that tested models paid more attention to the beginning and end of long input even when relevance was elsewhere. Its calibration method improved tested retrieval-augmented generation by up to 15 percentage points. A separate 2025 Google study did not find the same lost-in-the-middle effect for simple fact retrieval with Gemini 2.5 Flash, including facts near the input limit.

The useful rule is to test the actual job. Do not assume a document helps because it fits. Do not assume every long packet fails. Use the smallest complete set, keep the important source easy to identify, and add cases where plausible distractors sit beside the needed fact.

What makes context safe for client work?

Useful context needs five properties: it is relevant, current, approved, scoped to the right client, and traceable to its source. Each missing property creates a different risk.

Failure	What the model sees	Operating control
Stale fact	Last quarter’s rate beside the current rate	One source of truth and an expiry rule
Scope leak	Another client’s note in the retrieved set	Client and matter filters before retrieval
Unapproved rule	A draft policy that looks final	Approval state stored with the document
Hidden conflict	Two sources disagree without a warning	Route the conflict to a named owner
No trace	A claim with no visible origin	Record the source used for the output

Document the process first. A workflow cannot select the current exception rule if that rule lives only in a manager’s head. Writing it down makes selection possible. Naming its owner and approval state makes selection safer.

The source of truth stays outside the model. Context carries a working copy into the step. When a price, scope, or policy changes, the workflow should fetch the current version instead of keeping yesterday’s copy in a long prompt.

How should a service firm test context?

Start with one repeated deliverable and five real cases. A renewal workflow might use an ordinary renewal, missing account notes, a recent scope change, two rate sheets that disagree, and a record from the wrong client with a similar name.

Score whether the workflow selected the right source, kept client scope intact, asked when facts were missing, exposed conflicts, and recorded what it used. These checks belong in an eval. Run the same cases when the prompt, source map, retrieval rule, tool, model, or workflow changes.

Do not optimize token count before the cases pass. Remove repeated instructions, stale examples, and irrelevant tool descriptions one group at a time. Rerun the same eval after each change. The right packet is the smallest one that still handles the ordinary case and the expensive exceptions.

Privacy needs its own check. Model-visible context may contain client information. Retention, training, and access rules depend on the provider, product, account settings, and contract. Verify those terms for the exact system before sending client data. A glossary definition cannot make that decision for the firm.

Where does context sit on the Delivery Model Ladder?

Context exists at every stage. It does its main delivery work at Stage 2, Augmented, on the Delivery Model Ladder.

At Stage 1, Enhanced, a person gathers records, pastes useful parts into a chat, checks the answer, and moves it into the next system. The AI improves one person’s work. The context still lives in that person’s judgment.

At Stage 2, a shared workflow retrieves current records, applies client scope, supplies written rules, sends conflicts to a named person, and records the source used. Context becomes part of the delivery system.

At Stage 3, AI-native, shared context is maintained across delivery instead of rebuilt for each task. That is part of what makes a service business AI-native. It also gives an AI employee ongoing access without unlimited authority.

Context can make current information available. It cannot make an unapproved source authoritative, recover knowledge that was never recorded, or prove the final output is correct. Before this step ships, list what it may trust, where each item comes from, when it expires, and what happens when sources disagree.