Beyond the Prompt: The Agent System Stack Explained

An agent is not one thing

The word “agent” is often used as if it describes one capability: a particularly smart model that can take actions. That hides the engineering choices that determine whether the system is useful. A model can reason over a request, but it cannot inspect a codebase, search a knowledge base, create a calendar event, or verify a result unless another system gives it those capabilities and controls how they are used.^[1][6]

A useful mental model is to separate the agent into layers. Each layer answers a different question: What should the model do? What does it know right now? What can it touch? What persists? What keeps the work safe and observable? When does it continue? And how do we know the outcome is correct?

A prompt tells the agent what to do. Context gives it a working world.

Prompt engineering is the craft of writing and organizing instructions for a model. It still matters. A clear instruction can tell an agent its role, constraints, available choices, and desired output. But an agent operating across many steps needs more than one instruction. It needs a useful working state at each turn.^[2]

Context engineering is the broader discipline of choosing what enters that working state: system instructions, user requirements, recent messages, tool definitions, retrieved documents, project notes, intermediate results, and constraints. The hard problem is not collecting the maximum possible information. It is selecting the smallest high-signal set that helps the model make the next good decision.^[2]

• Prompt: “Investigate the failing test and make the smallest safe fix.”
• Context: the failing test output, relevant files, repository conventions, prior decisions, available commands, and the current task status.
• Poor context: a dump of every file, every old conversation, and dozens of tools the agent does not need.

More context is not automatically better. Long contexts can include stale decisions, duplicate instructions, irrelevant history, and distracting tool output. A capable agent often needs to retrieve information just in time, keep durable notes outside its active context, and discard details that no longer help the current decision.^[2]

Tools, memory, and MCP solve different problems

These terms are often mentioned together because they all extend a model beyond a single response. They should not be treated as synonyms. A tool lets an agent read or act in another system. Memory preserves information across turns or sessions. Model Context Protocol, or MCP, is a standard way for external servers to expose capabilities such as tools, resources, and prompts to an AI application.^[8][2]

The practical consequence is that each layer needs its own design questions. For tools: What actions are allowed, and how are arguments validated? For memory: What deserves to persist, who can retrieve it, and how is stale information corrected? For MCP: Which external capabilities should be exposed, and what permissions should the host enforce? Giving an agent more of any of these can increase capability, but it can also increase cost, confusion, and risk.

The harness turns model capability into behavior

A harness is the system that lets a model operate as an agent in a real environment. It typically constructs context, exposes tools, executes approved actions, stores state, applies permissions, records traces, and returns observations to the model. In coding work, it may also provide a workspace, test commands, version control, and a sandbox.^[3][7]

This is why two products using the same underlying model can feel radically different. One may supply a focused workspace, clear tool contracts, a small permission set, useful logs, and independent verification. Another may expose too many vague tools, carry forward noisy context, and accept an agent’s final sentence as proof that the work is complete.^[3][4]

1. Tool contractThe harness defines what a tool does, what inputs are valid, what errors look like, and whether the action is read-only or changes external state.
2. EnvironmentThe harness gives the agent a controlled place to work: a browser, sandbox, codebase, database, or another bounded surface.
3. State and observabilityIt records task progress, tool results, and failures so the next session—or a human—can understand what happened.
4. Permission and recoveryIt limits what the agent may do, rejects invalid actions, and provides a path to retry, stop, roll back, or escalate.

A loop is the agent’s work cycle—not permission to run forever

A tool-using agent usually works in a loop: it receives a goal and context, chooses an action, observes the result, and decides what to do next. The emerging phrase “loop engineering” is used for designing the conditions around that repeated work: triggers, goals, state, verification, budgets, stopping rules, and human handoffs.^[9][2]

The label is new enough that definitions still vary. Some people use “loop engineering” to mean the runtime loop inside one agent. Others mean a scheduled system that repeatedly discovers work, delegates it, verifies the result, and records state. The shared insight is more durable than the jargon: reliable autonomy needs clear goals, feedback, limits, and evidence of completion.^[9][3]

• Goal: What outcome is the agent allowed to pursue?
• Observation: What evidence does it use to judge the last action?
• Adjustment: What can it retry, change, or ask for?
• Boundary: What may it never do without approval?
• Stop rule: When does it finish, pause, or hand off?

Evals turn “it worked” into evidence

An agent can produce a convincing final message even when it has not completed the underlying task. A research agent may cite weak sources. A coding agent may say a bug is fixed without running the right tests. A browser agent may report that a form was submitted when the page state never changed. Evaluations, or evals, are the tests that distinguish the transcript from the outcome.^[5]

For agent systems, an eval has a task, one or more trials, and a grader. The grader checks something meaningful: a database record exists, a test suite passes, a file changed correctly, an answer is grounded in retrieved sources, or a required safety boundary was respected. Because model outputs vary, one impressive attempt is not a reliability claim. Repeated trials and trace review matter.^[5]

1. Define the outcomeWrite down what must be true in the world when the task is genuinely complete—not what the agent should say at the end.
2. Create a deterministic check where possibleUse a test, schema check, expected record, policy assertion, or another objective verifier before relying on model judgment.
3. Run more than onceTreat each attempt as a trial. Record success, latency, cost, incorrect actions, retries, and human intervention.
4. Read the tracesA pass rate alone cannot tell you whether the agent succeeded for the right reason, exploited a loophole, or nearly caused a failure.

When an agent fails, ask which layer failed

The stack is not academic vocabulary. It is a debugging tool. Instead of saying “the AI got confused,” identify the failed layer. That turns a vague complaint into a change you can test.

1. Wrong answer from the right informationInspect the instructions, examples, and model choice before assuming a retrieval or tool problem.
2. Right information never reached the modelInspect context selection, retrieval, memory scope, and whether tool output was too large or poorly described.
3. Agent used the wrong tool or wrong argumentInspect the tool contract, descriptions, argument validation, permission boundaries, and action previews.
4. Agent lost progress or behaved differently after a restartInspect task state, durable notes, handoff artifacts, and context compaction.
5. Agent claimed success without completing workInspect the eval. The system needs an outcome grader, not another instruction to “double-check.”

Start with a smaller system than you think

You do not need a multi-agent swarm to apply this stack. Start with one agent, a narrow goal, a small tool set, an explicit stop condition, and one independently verifiable outcome. Add memory only when tasks truly span sessions. Add delegation only when parallel work or specialization creates a clear advantage. Add a loop only when the agent needs to adapt based on what it observes.^[6][1]

• Can I explain the task and finish condition in one paragraph?
• What is the minimum information the agent needs for its next decision?
• Which tools are necessary, and which actions require approval?
• What state must survive a restart—and what should expire?
• What will stop the loop before it wastes time, money, or trust?
• What independent check proves the task actually succeeded?

Those questions matter more than whether a product calls itself agentic. A dependable AI system is not the one with the most jargon around it. It is the one whose capabilities, boundaries, and evidence of success are clear.