Seven layers between the model and the result: reading the guide against its own evidence
A guide that splits a production agent into seven layers, whose strongest claim is that the model was never the problem. This reading reorders the argument by evidence strength — practitioner notes, one public incident, sponsored copy — adds the two layers it skips (cost and observability), and ends with eight checks.
Abstract. Most production agent failures do not happen in the model layer. Split the distance from model to result into seven layers — model, instructions and context, tools, memory, loop, evals and guardrails, human signature — and the most common reflex (swap the model, rewrite the prompt) acts on the one layer least likely to be broken. This piece reorders the guide’s argument by evidence strength, adds the two things it leaves out (cost and observability), and ends with eight checks.
1. The claim, and its shape#
The skeleton is a hiring analogy: hand a brilliant new hire an office with no onboarding, no logins, no notebook, no process and no review, and they fail too — because nobody built the job around them.
The shape matters: it is a job description. Six of the seven layers are the things a company gives a real employee: an onboarding doc, system access, a notebook, a work loop, a review process, and final signing authority. So the order cannot be shuffled — it is the order of hiring someone and putting them to work.
The two outside quotes are labelled as such: Anthropic in June 2025 (“the last mile often becomes most of the journey”) and in September 2025 defining agents as “LLMs autonomously using tools in a loop”. The second draws a boundary: something that answers one question is not an agent.
2. Seven layers, and how each one looks when it breaks#
| Layer | What it does | How it looks broken | Minimum viable move |
|---|---|---|---|
| 1 Model | reads, plans, chooses the next step | swapping the model and rewriting the prompt: the common mistake | score with the strongest model first, then downgrade by evals; pin exact versions, never aliases |
| 2 Instructions and context | everything the model sees before acting | sounds competent, knows nothing about your process | rewrite an SOP the team actually follows into the system prompt; stable rules resident, task facts per run |
| 3 Tools | reading data, taking action | all talk, or it acts and oversteps | mark every action read or write; prefer existing MCP servers; scoped credentials; strict inputs (ids, absolute paths, fixed dates) |
| 4 Memory | what survives outside the context window | recall degrades as the chat grows; last week’s decisions vanish | MEMORY.md plus write access to that file only; compact before long sessions; delete stale notes weekly |
| 5 Loop and orchestration | act, look, decide, stop | never finishes, runs away, nobody knows which step broke | exit conditions in the instructions; max_turns in code; tracing on; recurring work on cron |
| 6 Evals and guardrails | tell you if it is good, stop it when it is not | scores that do not match, or one prompt away from deleting prod | 20 real tasks plus an LLM judge; staging database, test accounts, read-only keys; any workflow reading private data and untrusted input gets a human gate before sending |
| 7 Human signature | the final yes | a client email nobody approved went out | three risk tiers; approval cards for the high tier; log who approved |
The layer most often skipped is the third one’s input discipline. The quoted Anthropic test is unusually hard: “if a human engineer can’t definitively say which tool should be used in a given situation, an AI agent can’t be expected to do better.” And their admission that while building a coding agent they “spent more time optimizing tools than the overall prompt” is the part worth stealing.
3. Read the evidence in layers#
This guide is sponsored (by Viktor, with a “how Viktor handles this layer” section at the end of each one). That does not undermine the frame, but it should decide how much weight each sentence carries. Sorted by origin, three kinds:
| Kind | Examples | How much to trust it |
|---|---|---|
| Practitioner notes from the labs | “the last mile often becomes most of the journey”; rewritten tool descriptions cut completion time 40%; subagents beat a single agent by 90.2% on their research eval at about 15x the tokens | direction is credible; the numbers are their own evals, so retest on your task |
| A public incident | July 2025: a Replit agent deleted a customer’s production database during a declared code freeze | the hardest item in the piece: the fix was dev/prod separation, one-click restore and a planning mode, not a better prompt |
| Vendor copy | 70,000+ teams, 9,800+ tools, 12 hours a week saved, 12 workflows in 15 days | unfalsifiable; background only — though admitting misses (a spend mismatch, a wrong formula) is a point in its favour |
The rule is simple: borrow the points, not their strength — layer seven stands on the Replit incident, which is independent, while “Saves 12 hours a week” stands on a vendor page, so it stands on nothing.
4. Three mechanisms worth pulling out#
First, evals before downgrades. Score 20 real tasks on the strongest model, then swap one layer to a cheaper model and rerun; keep the swap only if the pass rate holds, and never swap two at once. It turns “which model” from a taste question into a measurable engineering one.
Second, a tool description is an interface. Name, when to use it, when not to, strict inputs — those four make a contract with the agent. The Anthropic test is a good acceptance criterion: if a human engineer cannot tell which tool fits, the interface is not designed yet.
Third, a permission boundary is not a prompt. This is the hardest conclusion in the piece and the easiest to skip: “do not delete production” is an instruction; a read-only credential, a staging environment and a deny rule are boundaries. Simon Willison’s lethal trifecta works directly as a checklist — private data, untrusted content and an outbound channel, all three at once, is an exfiltration risk.
5. Autonomy is granted per action#
Layer seven gets three tiers: low risk runs alone, medium runs and leaves a trail, high waits for a signature. The operational sentence is the demotion rule: an action moves down a tier only after dozens of approvals with no edits.
That turns autonomy from a property of the agent into a property of the action, which dissolves the usual dilemma: the question is now whether this action should be opened up.
6. Its boundaries: the two things it skips#
For a guide, it leaves little out — but two gaps matter:
- Cost and latency. Exactly one number touches cost (subagents at about 15x tokens), and in production those two decide whether something ships. The 20-row eval measures accuracy, not dollars per task or p95 latency.
- Observability. Layer five gets one line about turning tracing on. In practice you need to answer “which layer’s failure rate rose this week”, which means recording metrics per layer rather than one success rate.
The multi-agent material, meanwhile, rests on one number (+90.2%) and one warning (an early version spawned 50 subagents for simple queries). That is enough to support “maximise a single agent first”; it is not enough to design an organisation.
7. Eight checks#
- How many real tasks are in your eval set? Are there 20, ugly ones included?
- Is every model version pinned, or do you use an alias like
latest? - Was the system prompt rewritten from an SOP the team actually follows?
- Is every tool marked read or write, and do write tools have their own scoped credentials?
- Does the agent’s write access to
MEMORY.mdcover that file only? - Are exit conditions in the instructions, and is there a
max_turnsin code? - Which workflow has all three: private data, untrusted input, outbound sending?
- How long is the high-risk list, and does every item have an approval card and a record of who approved?
If three or more answers are missing, the model is usually not the problem.
8. Open questions#
- At what scale does a 20-row eval stop working? When the task distribution has dozens of shapes, can “swap one layer at a time” survive, or does it need a layered eval?
- The demotion rule needs dozens of unchanged approvals, and that count depends on the quality of the approval record. If approval is a habitual click, the rule decays into “enough time passed”.
- If performance comes mostly from tool descriptions and context quality, is the reusable asset the seven-layer frame, or those dozens of SOPs?
References#
- Original: https://x.com/undefinedki/status/2116154822583799887 (How to Build a Production AI Agent: The 7 Layers Between the Model and the Result)
- Anthropic, on building effective agents and context engineering, 2025
- OpenAI, A practical guide to building agents, January 2026
- Simon Willison, the lethal trifecta, June 2025
- Reporting on the Replit production database deletion, July 2025