Context engineering is the practice of deciding everything a language model sees when it answers: not only the question you type, but the instructions around it, the documents and tool results inserted into it, the saved memory about the user, and the conversation so far. All of it shares one context window, and the model answers from that text and nothing else. The term spread in 2025, as more AI products became agents that assemble most of their context automatically.
Prompt engineering is mostly about how to word a request. Context engineering is about what the model should have in front of it on each call, and in what order.
From a prompt to a context
In a chat app you write most of the context yourself: the app adds a system prompt and the history, and you add the rest. In an application the balance flips. A user types one sentence, and the code around the model adds instructions, a user profile, three help articles found by search, the list of available tools and the output of the last tool call. The user's sentence may be a small fraction of what the model reads.
When such a system answers badly, the fix is rarely in the wording. The usual cause is that the model had the wrong material: a missing fact, an outdated tool result, an irrelevant document that looked relevant to the search.
What goes into the context window
A typical call in an AI application contains some or all of these, in roughly this order:
- System instructions: the role, rules and output format, usually fixed for the whole app. See system prompts.
- Tool definitions: names, descriptions and parameters of the tools the model may call.
- Examples: a few sample inputs and outputs that show the expected behaviour.
- Memory: facts saved from earlier sessions, such as the user's plan, language or preferences.
- Retrieved documents: passages found by searching a knowledge base for this question (retrieval-augmented generation, or RAG).
- Conversation history: earlier turns, either verbatim or summarized.
- Tool results: output from searches, code runs or API calls made during this task, as in a ReAct loop.
- The current message: what the user just asked.
The block below is one assembled context for a support assistant. Switch the parts off one at a time. Without the context part the model cannot know the customer's plan; without the input part it has no product facts, and its constraints tell it to say so rather than guess.
Hi Dana, offline mode is part of the Pro plan, and your account is currently on the Free plan, so it is not available to you at the moment. With Pro, notes you create during the flight sync automatically when your phone reconnects. One limit to know about: attachments larger than 20 MB are not available offline.
Notice that the correct answer depends on joining two sources: the memory (Free plan) and a document (offline mode is Pro only). Neither alone is enough, which is typical. Much of context engineering is making sure the pieces that need each other arrive together.
Four ways a context goes wrong
- Missing information. The model fills gaps with plausible guesses, which is where many hallucinations come from. Add the fact, or tell the model what to do when a fact is missing.
- Too much material. Every irrelevant paragraph costs tokens and competes for attention. Liu et al. 2023, "Lost in the Middle: How Language Models Use Long Contexts", found that the models they tested used information at the beginning or end of a long input more reliably than information in the middle. Newer models handle long inputs better, but sending the few passages that answer the question is still cheaper, and easier for the model to use, than pasting the whole manual.
- Stale information. A tool result from ten steps ago can describe a file or a balance that has since changed. If the model sees both versions, it may use the old one.
- Conflicts. Two documents disagree, or memory says one thing and the user another. Tell the model which source wins, for example "the user's latest message overrides saved memory".
Ordering the context
Order changes both results and cost.
- Stable parts first. System instructions, tool definitions and fixed reference material rarely change between calls. Several API providers offer prompt caching, which reuses the processing of an identical beginning of the input, so an unchanging prefix makes repeated calls cheaper and faster.
- Long material before the question. For a long document or a large set of passages, put the material first and the question and final instructions after it. Anthropic's prompting guide, for one, recommends this order for long inputs, with the question right before the model starts writing.
- Label every piece. Wrap each source in tags such as
<document>,<memory>or<tool_result>, with a source name. Labels let the model tell data from instructions and let you ask it to cite where an answer came from. Delimiters and XML tags covers the formats.
Trimming a long context
Every turn of a chat resends the whole history, so long sessions grow until something has to go. Depending on the app, it may summarize or drop older messages, or ask you to start a new chat. You get better results by trimming deliberately.
- Keep the last few turns word for word and replace older ones with a summary.
- Once a tool result has been used, replace it with a one-line note of what it showed.
- Keep exact identifiers in every summary: file names, function names, IDs, error messages and numbers. A summary that says "the config file" where the history said
settings.prod.yamlloses the one detail that mattered. - In a chat app, when the conversation drifts, start a new chat with a handoff note. The prompt below writes one.
Goal: replace the signup form's hand-written checks with one shared validation schema used by both the browser and the server.
Decisions
- One schema file,
signupSchema.ts, imported bySignupForm.tsxandapi/signup.ts, so the two sets of rules cannot drift apart. - Error messages stay word for word, because the help center quotes them.
Current state
- Browser validation uses the schema and
SignupForm.test.tsxpasses. - The server still calls the old
validateSignup()inapi/signup.ts.
Exact details: passwords need at least 8 characters and one number. Email error text: "Please enter a valid email address."
Next step: replace validateSignup() with the schema and run the API tests.
Open question: should an already registered email return 409 or 400?
Memory across sessions
Memory is context that outlives a conversation: facts written to storage at the end of one session and loaded into the next. Chat apps offer versions of this, such as saved memories or project instructions that are added to every chat in a project. In your own application, memory is a table or a notes file that your code reads and inserts. Two rules keep it useful: store facts that stay true (plan, language, preferred stack), not transcripts; and load only what is relevant to the current task, because memory competes for the same space as everything else.
Assembling context in code
In an application, context engineering is ordinary code. This sketch with the Anthropic Python SDK puts the fixed rules and memory in the system prompt, keeps only recent history, and places labelled documents before the question.
import anthropic
client = anthropic.Anthropic()
MODEL = "your-model-id" # e.g. from your provider's model list
def build_context(question, docs, history, memory, max_messages=6):
documents = "\n".join(
f'<document source="{d["source"]}">\n{d["text"]}\n</document>' for d in docs
)
system = (
"You are the support assistant for Acme Notes. Answer only from the documents. "
"If they do not cover the question, say so.\n"
f"<memory>\n{memory}\n</memory>"
)
# history holds complete user/assistant pairs, so an even slice starts with a user turn
recent = history[-max_messages:]
user = f"<documents>\n{documents}\n</documents>\n\n{question}"
return system, recent + [{"role": "user", "content": user}]
# question, docs, history and memory come from your application
system, messages = build_context(question, docs, history, memory)
response = client.messages.create(model=MODEL, max_tokens=1024, system=system, messages=messages)
print(response.content[0].text)
Each decision in that function (which documents, how many messages, where memory goes) is a context engineering choice, and each one is worth testing on real questions the same way you would test a change to the wording.
Frequently Asked Questions
What is context engineering?
Context engineering is the work of choosing, ordering and trimming everything a language model receives on a call: the system instructions, examples, retrieved documents, tool definitions and results, saved memory, conversation history and the user's message. The model answers only from that text, so what is in it, and what is left out, decides the quality of the answer.
What is the difference between context engineering and prompt engineering?
Prompt engineering is mostly about wording the instructions. Context engineering covers the whole input, much of which is assembled by code rather than typed by a person: which documents to retrieve, which tool results to keep, how much history to include and in what order. In a chat you write most of the context yourself; in an app or an agent, most of it is chosen by the system around the model.
Is more context always better?
No. Irrelevant or outdated material competes with the parts that matter, costs tokens, and can contradict the current state. Research on long inputs has found that models can miss information placed in the middle of a long context. Include what the task needs, label it, and remove what it no longer needs.
Why does a long chat get worse over time?
The whole conversation is resent on every turn, so old mistakes, abandoned ideas and superseded code stay in the context and keep influencing replies. Once the chat outgrows the context window, the app has to drop or summarize older messages. Starting a new chat with a short summary of the decisions and the current state often works better than continuing.
What is RAG in context engineering?
RAG (retrieval-augmented generation) means searching your own documents for passages relevant to the question and inserting them into the context before the model answers. It is one of the main context engineering tools: the model gets current, specific facts it could not know from training, and you can tell it to answer only from those passages.
