A support team wires up an AI assistant to answer questions using their help centre and internal wiki. The first version works well: short answers, accurate, fast. Someone decides more context would make it smarter, so they start pasting entire wiki pages, full ticket histories, and old Slack threads into every prompt. The answers get slower, vaguer, and occasionally wrong in ways the short version never was. That pattern, more input producing worse output, is the clearest sign of a context engineering problem, and it is far more common than most teams realise.
Context engineering is the practice of deciding deliberately what a language model sees before it answers a question, rather than assuming more information is automatically better. It sits upstream of prompt engineering, which is about phrasing an instruction well. You can write a perfect instruction and still get a poor result if the context sitting around it is bloated, irrelevant, or stale.
Why does prompt stuffing make answers worse?
Prompt stuffing means pasting large blocks of raw text, entire documents, full conversation logs, whole database exports, into a prompt on the assumption that giving a model everything is safer than giving it a curated subset. It feels intuitively correct: surely more information helps. In practice it tends to hurt in three specific ways.
First, models do not weigh every token in a long context equally. Attention spreads unevenly across a long input, and the exact fact you need can sit in a part of the context the model effectively skims. Second, irrelevant text is not neutral. It competes with the relevant text for the model's limited attention and can actively pull an answer in the wrong direction, especially when the irrelevant material contains superficially similar patterns. Third, cost and latency scale with every token sent, so stuffing a prompt with material the model will not use productively is paying twice: once in money, once in answer quality.
The MDPI systematic review of retrieval augmented generation in enterprise settings treats this as one of the reasons RAG has matured past naive "search and paste" implementations: production systems now invest specifically in choosing and structuring what reaches the model, rather than maximising volume.
What context engineering actually looks like in practice
Good context engineering usually has three ingredients: selection, condensation, and structure.
Selection means retrieving only what is relevant to the specific question being asked, rather than everything that could conceivably be related. This is where permission aware rag also lives, since who is allowed to see a piece of information is one more filter applied at the same selection step, not a separate concern bolted on afterwards.
Condensation means summarising a source down to the facts that matter, rather than handing over the source verbatim. A 40-page contract does not need to enter the prompt in full if the question is about a renewal date and a termination clause. Condensed working memory, a summarised and continuously refreshed representation of a source system, is one answer to this: instead of re-reading the whole contract every time, a system keeps a short, current summary on hand and only goes back to the original when that summary goes stale.
Structure means presenting information in a form the model can use efficiently: labelled sections, clear attribution of which fact came from which source, and a consistent format across similar queries, rather than a wall of undifferentiated text.
| Approach | What reaches the model | Typical failure |
|---|---|---|
| Prompt stuffing | Full documents, entire chat histories, raw exports | Diluted attention, higher cost, occasional wrong answers buried in noise |
| Naive RAG | Top-N retrieved chunks, unfiltered by role or freshness | Irrelevant chunks crowd out the right one, stale facts served as current |
| Deliberate context engineering | Selected, condensed, structured facts relevant to the specific asker and question | Requires upfront design work and a source of truth for what is current |
Table 1: three common ways context reaches a model, and where each tends to break down.
A worked example
Consider a 30-person marketing agency using an AI assistant to answer client questions about campaign performance. Early on, someone connects the assistant directly to the analytics export and the full campaign brief for every client, on the logic that the assistant should "have everything." A staff member asks a simple question: what's the click-through rate on the Meridian campaign this month. The assistant, faced with a prompt containing five other clients' briefs and eleven months of historical data, occasionally answers with a number from the wrong campaign or the wrong month, because the right figure is sitting in the middle of a very long context alongside numbers that look almost the same.
The fix was not to make the model smarter. It was to change what reached it: retrieve only the Meridian campaign's current month data, condensed to the handful of metrics that matter, before the question ever became a prompt. The assistant's underlying capability did not change. What changed was the discipline applied before that capability was invoked.
Context engineering versus prompt engineering
The two disciplines are often confused because both aim at getting a better answer out of the same model. Prompt engineering optimises the instruction: word choice, examples, output format requests. Context engineering optimises the material the instruction is answered against. A well-engineered prompt sitting on top of poorly engineered context still tends to underperform, because no amount of clever phrasing recovers a fact that was diluted or simply never selected in the first place. Treat them as separate jobs, even if the same person handles both.
Common pitfalls
- Assuming a bigger context window removes the need for selection. A model that can technically accept 200,000 tokens still attends to them unevenly, and every irrelevant token is a small tax on the answer's accuracy.
- Confusing "more retrieved chunks" with "better retrieval." Retrieving 40 loosely relevant chunks instead of 5 tightly relevant ones usually makes an answer worse, not more thorough.
- Serving condensed memory without a freshness check. A summary is only useful if it reflects current reality. A condensed fact that never gets re-checked against its source becomes a confident, wrong answer with no warning attached.
- Building context selection and access control as two separate systems. They are the same problem from two directions: what should this asker see, and what is actually relevant to this question. Solving them separately tends to produce a system that is either too permissive or too restrictive in ways nobody notices until it matters.
"Granting LLMs unchecked autonomy to take action can lead to unintended consequences, jeopardizing reliability, privacy, and trust."
OWASP Top 10 for Large Language Model Applications, on excessive agency
That warning is usually read as a security point, and it is, but it applies just as well to context volume. A model given unchecked access to everything a company knows, with no deliberate selection of what it should actually see for a given question, is exposed to the same category of risk even before anyone asks a malicious question.
Where this connects to the MCP ecosystem
The Model Context Protocol exists partly to standardise how an AI system pulls context from external sources on demand, rather than having every piece of context pre-loaded into a prompt at the start. An MCP server can expose a narrow, well-defined slice of a system of record, and a client calls it only when that specific slice is needed. That pattern, fetch what's needed when it's needed, is context engineering expressed as infrastructure rather than as a one-off decision inside a single prompt.
Contextely is one example of a tool built around this pattern: it acts as condensed working memory over a company's systems of record, refreshed when it goes stale, with entitlement checked at retrieval rather than after the fact. It is not the only way to do context engineering well, and for a single-user assistant remembering personal preferences from conversation, a tool like Mem0 solves a genuinely different version of this problem. The discipline matters more than any specific product: decide what the model needs to see, and build the machinery to give it exactly that, no more and no less.
Getting started
Start by auditing one existing AI feature you already have. Look at an actual prompt sent to the model, not the intended design. Count how much of that prompt the model actually needed to answer the specific question asked. If it's under half, you have a stuffing problem worth fixing, and the fix is rarely "use a smarter model." It is almost always upstream: better selection, real condensation, and a freshness check on anything you're treating as a fact rather than a live lookup.
If access control is part of that audit too, the deeper mechanics of checking who is entitled to see what at retrieval time are covered in our permission aware rag guide. Contextely's own docs walk through how condensation and staleness handling work together in practice, and the pricing page shows what a free tier looks like if you want to test the pattern against your own systems before committing to it.