On this page
Put Coworker to work on your stack.
Connect Salesforce, Slack, Jira and run your first agent in minutes.
Book a demoEnterprise AI
Context Engineering: Deciding What an AI System Actually Sees
Coworker AI explains context engineering: the discipline of deciding what goes into a model's window, why less beats more, and how to measure it.
Context engineering is the practice of deciding what an AI system sees on a given request: which instructions, which retrieved material, which tools, which memory, in which order, and how much of each.
It is distinct from prompt engineering, which is about how you phrase the instruction. Prompt engineering matters, and it is largely a solved craft. Context engineering is the harder problem, because the window is a budget and almost every decision about how to spend it involves a trade-off that is not obvious.
Why it became the harder problem
When windows were small, the constraint was capacity: fit the task in. Now that windows are large, the constraint is attention: the model can technically receive vast input and does not use all of it well.
Two things follow, and they surprise people.
Adding context often makes output worse. Not slower or more expensive, though also those. Worse. Irrelevant material competes with relevant material, and the more plausible the irrelevant material, the more expensive it is to ignore. This is context rot, and it is the single most common self-inflicted failure in retrieval systems.
The stated window is not the usable window. Models attend unevenly across long inputs, with material at the start and end used more reliably than material in the middle. Research on how models use long contexts documented this, and independent testing found accuracy falling with input length even when the relevant fact was unambiguous.
So the discipline is not "how much can I fit". It is "what is the smallest set of things that makes this answer correct".
The four inputs you are budgeting
Every request is assembled from four sources, and they compete for the same space.
| Input | What it is | Failure mode when overused |
|---|---|---|
| Instructions | System prompt, task framing | Bloated preambles nobody maintains |
| Retrieval | Passages pulled for this query | Near-misses crowding out the answer |
| Tools | Definitions of what the model can call | Fixed cost on every request, used or not |
| Memory | Durable facts from previous sessions | Stale facts asserted confidently |
Most teams instrument none of these separately, which is why they cannot tell which one is eating the budget. Splitting token counts by category is usually the first measurement that changes behaviour.
Tools are the invisible line item
Worth isolating because it is consistently missed. Every tool exposed to a model occupies window space on every request, whether or not it is called. Connect fifty tools through MCP and a meaningful slice of the window is spent before the user has typed anything.
Scoping the tool set to the task is simultaneously a cost control, a quality control, and a security control. Few decisions do all three.
The principles that hold up
Retrieve less, and more precisely
The single highest-value change for most systems. Cutting retrieval from twenty passages to five frequently improves answers while reducing cost and latency at once.
The instinct to retrieve generously comes from a reasonable fear of missing something. The way out is precision, not volume.
Filter before you rank
Similarity search over a whole corpus ranks everything against the query and returns the closest matches, which is exactly how near-misses arrive. Constraining first, by entity, owner, time period or permission, and only then ranking within that set, produces a far cleaner input.
This is the practical case for structure. A knowledge graph lets you express "facts about this account, valid now, that this person can see" as a constraint rather than a hope. You stop asking the model to ignore the wrong things by never including them.
Distinguish durable from live
Some facts change constantly and must be fetched at request time. Some are stable and should have been remembered. Re-fetching a stable fact every session is paying full latency and tokens to rebuild something that did not change, and rebuilding it slightly differently each time, so answers are not repeatable.
Sorting your context into live and durable is the decision that agent memory exists to serve.
Position matters, so use it
Given that models attend more reliably to the start and end of long inputs, put the instruction and the most critical material at the edges. This is a workaround rather than a principle, and it works.
Compact rather than accumulate
Long conversations should be summarized progressively. The thing to check is what compaction discards, because a summary that drops a decision made forty turns ago has traded one failure for another.
Split the task
A request needing 100,000 tokens is often several requests needing 10,000 each. Decomposition sidesteps degradation entirely and is usually cheaper. Anthropic's guidance on context engineering for agents makes a similar argument for treating the window as a scarce resource to curate rather than a container to fill.
Coworker
Put Coworker to work on your actual stack
Connect Salesforce, Slack, Jira and run your first agent in minutes.
How to measure it
You cannot engineer what you are not measuring, and most teams measure none of this.
Token counts by category per request. Instructions, retrieval, tools, memory, conversation. The split is almost always surprising.
The distribution, not the mean. The p99 request is where unbounded retrieval and runaway loops appear. Averages hide exactly the requests that cause the bill.
Answer quality against input size. The direct test for context rot: ask the same question in a short focused prompt and in your full production context. If the short version wins, you have your answer.
Retrieval precision, not just recall. How many retrieved passages were actually used. Optimizing recall alone is what produces the problem.
All of this lives in LLM observability, and without it context engineering is guesswork with confident language attached.
Context engineering versus prompt engineering
| Prompt engineering | Context engineering | |
|---|---|---|
| Question | How do I phrase this? | What should be here at all? |
| Unit | The instruction | The whole assembled request |
| Failure | Model misunderstands the task | Model has the wrong information, or too much |
| Fix | Rewrite the wording | Change what gets retrieved, remembered or exposed |
| Maturity | Largely settled craft | Actively unsolved |
Prompt engineering has not stopped mattering. It has stopped being where the gains are, because a well-phrased instruction sitting on top of badly chosen context still produces a bad answer.
Where teams get it wrong
Treating the window as free. It is billed on every request and it degrades as it fills. Both costs are continuous.
Optimizing recall. Retrieval evaluation that rewards finding the relevant passage, with no penalty for the eleven irrelevant ones that came with it, drives exactly the wrong behaviour.
Connecting tools because you can. Each one is a fixed tax on every request.
Never forgetting. Memory that only accumulates gets slower, more expensive and self-contradictory. Decay, succession and retirement are features.
Blaming the model. When answers degrade, the instinct is to swap models. The cause is usually what was assembled into the request, and swapping is an expensive detour that leaves the actual problem in place.
Where Coworker AI fits
Most context problems are connection problems wearing a disguise. The window is empty of your company's information until something fills it, and filling it well means supplying the answer rather than the material an answer might be derived from.
Coworker AI connects to 50+ tools, maintains organizational memory across them as a structured graph, and supplies scoped, permission-inherited context to agents. It exposes the same through MCP, which keeps the tool-definition overhead flat instead of growing with every system you connect.
Pro is $29.99 per user per month, Max is $149.99, and Enterprise pricing is on request. Book a demo to see it on your own stack.
Frequently asked questions
What is context engineering?
The practice of deciding what information reaches a model on each request and in what form: instructions, retrieved passages, tool definitions and memory. It became the central discipline once context windows grew large enough that the limiting factor stopped being capacity and became how reliably the model uses what it is given.
How is context engineering different from prompt engineering?
Prompt engineering is about phrasing the instruction. Context engineering is about what belongs in the request at all. A well-phrased prompt sitting on badly chosen context still produces a bad answer, which is why the harder work moved.
Does more context produce better answers?
Usually not. Beyond the material genuinely needed, additional context competes for attention with the relevant material and accuracy falls. Cutting retrieval often improves answers while also reducing cost and latency.
What should I measure?
Token counts split by category per request, the distribution rather than the mean, answer quality against input size, and retrieval precision rather than recall alone. Without that instrumentation, context engineering is guesswork.
Why do tool definitions matter for context engineering?
Because every tool exposed to a model occupies window space on every request, whether it is used or not. Connecting many tools imposes a fixed cost on every interaction, which is simultaneously a spend problem, a quality problem and a security surface.
What is the fastest way to improve context quality?
Retrieve fewer, better-scoped passages, and filter by entity, time and permission before ranking by relevance. Most systems retrieve too generously out of a fear of missing something, and that fear is what causes the degradation.
Related reading
Ready to get started?
Put Coworker to work inside your actual stack
Connect Salesforce, Slack, Jira, whatever you use, and run your first agent in minutes.