Context Engineering: Why More Context Can Make an AI Agent Worse

Context is the agent’s working state

Prompt engineering usually focuses on the instructions given to a model: its job, constraints, format, and examples. Context engineering is broader. It is the work of selecting and maintaining everything the model can use at a particular turn, including those instructions, the user request, retrieved material, tool outputs, task state, and relevant prior decisions.^[1]

This distinction is especially important for agents. A one-shot answer may need a well-written prompt and a document. A multi-step system has to decide what information should still be visible after ten actions, what should be retrieved only when needed, and what should be summarized for the next session. The agent’s context is not a transcript; it is an actively managed workspace.^[1][2]

Why more context is not automatically better

A large context window makes it possible to supply more material. It does not make every added token useful. Old plans can conflict with newer ones; duplicate instructions can create ambiguity; raw tool output can hide the one fact that matters; and irrelevant background can distract from a narrow decision. Research on long-context models has also shown that placement and retrieval within long inputs can affect performance, rather than every part of a prompt being used equally well.^[1][3]

• Useful now: the current failure, the few files involved, the test command, and the repository’s relevant convention.
• Useful later: the complete ticket archive, an old migration discussion, and raw logs from unrelated services.
• Potentially harmful now: superseded instructions, duplicate retrieved passages, large unfiltered command output, or stale conclusions presented as facts.

The point is not that long context is bad. It can be decisive for document analysis, codebases, and multi-step work. The engineering task is to preserve the information that carries forward while removing the information that merely consumes attention. Good context design starts from the next decision, not from the amount of data available.^[1]

Retrieval, compaction, and notes are different tools

A robust agent rarely keeps its entire past in the active prompt. Instead it uses a few complementary mechanisms. Retrieval brings in an external fact when a specific question arises. Compaction turns a long interaction into a shorter state summary. Durable notes capture decisions, constraints, and handoff information that must survive beyond the current context window.^[1][4]

1. Retrieve just in timeBring in a document, code file, or record only when the next action needs it. Keep the query and source visible so a reviewer can trace the claim back to evidence.
2. Compact without laundering uncertaintySummaries should retain unresolved questions, the source of key facts, and the limits of what was checked. A tidy summary that turns a guess into a conclusion is worse than raw history.
3. Write a durable handoffSave the goal, current state, verified results, failed attempts, and next safe action outside the model’s immediate context. This helps the next run start from evidence instead of reconstructing the past.

Memory is related but not identical. A memory system decides what can persist and later be retrieved across tasks or sessions. Active context is what the model sees right now. Mixing these ideas leads to an easy mistake: saving everything forever and then injecting all of it into every turn. Persistence should be selective, and retrieval should be scoped.^[4][1]

Build the minimum viable context for the next decision

Imagine an agent asked to diagnose a failing test. It does not need every issue in the tracker or every file in the repository. Its first working context might include the exact failure, the relevant test and implementation files, a short project convention, the approved commands, and the rule that it must not change unrelated behavior. After it runs a test, the result becomes part of the next context. If it discovers a dependency contract, it can retrieve that contract then—not pre-load every dependency.

Goal: What outcome is being pursued right now?
Facts: Which observations are verified and still current?
Constraints: What may not change? What needs approval?
Capabilities: Which few tools or sources are relevant?
State: What has already been tried, and what happened?
Exit: What evidence will let the agent finish or hand off?

This is not a universal template. A research agent, coding agent, and support agent will carry different information. The reusable principle is to make context a designed input rather than an accidental accumulation of chat history and API responses.

Four context failures worth recognizing

1. Stale authorityAn old decision remains in the prompt after conditions changed. Fix it with timestamps, source attribution, and explicit replacement rules.
2. Context dumpThe agent receives every possible file, message, or tool result. Fix it with scoped retrieval and summaries that are reviewed for uncertainty.
3. Missing operational stateThe agent knows the goal but not what was already attempted or which action is safe next. Fix it with structured task state and handoff notes.
4. Unclear tool affordancesThe agent has a tool but cannot tell when to use it, what it returns, or whether it changes the world. Fix it with precise contracts and small, relevant tool sets.

Context engineering is not a magical prompt trick. It is ordinary systems work applied to an unusual interface: selecting state, setting boundaries, preserving provenance, and measuring whether a better working set produces a better outcome. That makes it one of the highest-leverage places to improve an agent before changing the model itself.^[1]