Context engineering for AI agents is the problem hiding behind a specific, recognisable failure: an agent that books three meetings and updates a CRM record does not read the same context twice. On step one it reads a calendar. On step nine it reads its own earlier tool output, a summary of that tool output, a system prompt, and whatever got carried forward from steps two through eight, some of it useful and some of it dead weight nobody removed. By step twenty, most of what is sitting in that window was written by the agent itself, and nobody engineered any of it on purpose. That accumulation, not the first prompt, is where agent behaviour actually goes wrong.
The term picked up momentum through 2025 and into 2026 as teams building genuinely long-running agents kept hitting the same wall: a bigger context window did not fix flaky multi-step behaviour, and better prompts did not either. Anthropic's engineering team put a name to what was actually breaking, and it was neither of those things.
What context engineering for AI agents actually means
Anthropic defines context engineering as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts." That last clause is the important one for agents specifically. A chatbot's context is mostly the conversation. An agent's context is the conversation plus every tool definition, every tool result, every intermediate plan, and whatever it wrote to its own scratch memory an hour ago.
An agent, in the same document's own framing, is just "LLMs autonomously using tools in a loop." Every pass through that loop is a fresh inference call, and every fresh inference call re-reads the same accumulating pile. Context engineering is the job of deciding, on each pass, what stays in the pile, what gets compressed, and what gets thrown away entirely because it already did its job.
This is a genuinely different discipline from prompt engineering. A prompt engineer worries about the wording of one instruction. Someone doing context engineering for AI agents worries about the shape of the entire working memory a multi-step task accumulates, hour after hour, tool call after tool call.
Context rot: the failure mode a bigger window does not fix
The most counterintuitive part of this problem is that it is not caused by running out of space. Chroma's research team tested 18 frontier models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, on how performance changes as input length grows, and found something models do not advertise: accuracy degrades well before the window is full, and the degradation is uneven across models and task types. A model with a 200,000-token window can show meaningful drop-off at 50,000 tokens, long before anyone would call the window "full."
That gap between technical capacity and reliable capacity is what the industry now calls context rot, and it explains a specific symptom agent builders keep reporting: an agent that performs well for the first several steps of a task and then starts making subtly worse decisions, not because it hit a hard limit, but because its attention is spread across accumulated tool output, most of which stopped being relevant several steps ago.
The architectural reason is not mysterious. Transformer attention requires comparing every token against every other token, an n-squared relationship that gets strained as sequences grow, and models see comparatively less training data at the longest sequence lengths, so the parameters specialised for very long-range dependencies are thinner than the ones handling short ones. None of this shows up as an error message. It shows up as an agent that quietly gets worse the longer it runs, which is exactly the shape of bug that is hardest to catch in a demo and easiest to hit in production.
"The set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts."
Anthropic, Effective context engineering for AI agents
Three techniques that actually hold up on long-running tasks
Compaction, structured note-taking, and sub-agent delegation each address a different point in an agent's lifecycle, and most production agents end up using more than one.
Compaction summarises an agent's conversation and tool-call history at a natural checkpoint, then reinitialises its working context from that summary rather than the full transcript. The discipline in doing this well is deciding what to keep: architectural decisions and unresolved issues need to survive compaction, while the raw tool output that led to them usually does not. Compact too aggressively and the agent forgets why it made a choice; compact too conservatively and the context bloats right back up within a few more steps.
Structured note-taking has an agent maintain an external file, something like a running NOTES.md, that persists independently of its context window. This is the same instinct a person has when a task runs longer than they can hold in their head: write it down somewhere durable rather than trust short-term memory. The agent can re-read that file at any point without needing the entire history that produced it still sitting in its window.
Sub-agent architectures hand a narrow, well-defined piece of work to a specialised agent operating in its own context window, and only the condensed result, typically in the range of one to two thousand tokens, returns to the orchestrating agent. This is the closest analogue to delegating a research task to a colleague and asking for a summary rather than their entire browser history. It keeps the parent agent's window small, but it does not remove the need for the sub-agent's own output to be checked for freshness and for who is entitled to see it, since a sub-agent's retrieval carries the same risks as any other.
| Technique | What it targets | Where it commonly goes wrong |
|---|---|---|
| Compaction | Long conversation and tool-call history | Discarding an unresolved decision along with the noise |
| Structured note-taking | Facts that must survive across many steps | A note file nobody re-reads before acting on stale content |
| Sub-agent delegation | Narrow, self-contained pieces of a larger task | Treating the sub-agent's summary as pre-vetted when it still needs entitlement and freshness checks |
| Condensed working memory | A live system of record too large to re-read raw every time | Serving a condensation that was never refreshed against the source |
Where memory and entitlement fit into the same problem
Most discussions of context engineering stop at the conversation window, but a production agent's context also includes whatever a memory layer hands it, and that introduces two questions a transcript-only view of the problem does not: is this fact still current, and is the asker even allowed to see it.
Condensed working memory, a summarised and continuously refreshed representation of a source system rather than the raw source itself, solves the first half by carrying a freshness signal on every object it serves, so an agent is not reasoning from a fact that quietly went stale three weeks ago. The second half, entitlement, has to be enforced at the same point the memory is scored and ranked, not as a filter applied to the final answer, because a synthesis step that ever receives an object it should not have already leaked something by the time anyone notices. That is the same principle behind permission aware rag: access control is not a separate system bolted onto retrieval, it is one more factor inside the same scoring function.
This is also where MCP as a protocol earns its keep for context engineering specifically. An agent that can call out to a live system of record over MCP, rather than relying on whatever got stuffed into its context at task start, has an option a static prompt never had: pull the current answer instead of trusting an ageing one. The agent memory layer built around that pattern is doing context engineering by design, not as an afterthought bolted onto a chatbot.
Why this keeps mattering more, not less, through 2026
The gap between how many organisations use AI at all and how many have agents running reliably in production has stayed wide even as adoption headlines got louder. McKinsey's State of AI survey, fielded in mid-2026 across nearly 1,700 organisations, found that while 88% now use AI somewhere in the business, fewer than one in ten have deployed AI agents at scale in any single business function. The gap between "we tried an agent" and "we trust an agent" is, more often than the marketing suggests, a context problem: an agent that behaves well in a demo with a short, clean context and badly on a real multi-hour task with an accumulated, messy one.
Practical work on mitigating this is still active research rather than a solved problem. A 2026 paper on long-horizon search agents frames the same failure this article has described in more formal terms, treating context rot as a diagnosable condition with specific interventions rather than an unavoidable cost of long context, which is a useful sign that the field is moving from noticing the problem to building repeatable fixes for it.
Getting started without over-engineering it
A team building its first genuinely long-running agent does not need every technique in this article on day one. A reasonable order is: measure first, so you know whether degradation is actually happening on your specific tasks rather than assuming it is; add structured note-taking for anything that must survive more than a handful of steps, since it is the cheapest of the three techniques to implement; move to compaction once conversations regularly run long enough that early tool output has stopped being relevant; and reach for sub-agent delegation only once a single task genuinely branches into independent pieces of work, since splitting a task that does not actually branch just adds coordination overhead without shrinking anyone's context.
The teams that get this right tend to treat context as a resource with a lifecycle, written, used, condensed, and eventually discarded on purpose, rather than as a transcript that only ever grows. That shift in framing, more than any single technique, is what separates an agent that degrades quietly over a long task from one that does not.
Ready to see condensed, entitlement-aware memory behind an agent that runs on real systems of record? Start with the quickstart or see how Contextely compares to a self-hosted memory layer.