How one prompt grew into a company brain: Recall in four generations
A retrieval skill, four generations deep, and the two hard decisions behind it: store the source upstream and treat everything else as a rebuildable projection, and let the server own recall while the calling model owns precision. With four sets of measurements and four steps you can copy.
In one line: it turns “remembering” into a retrieval contract — the source is stored upstream and never edited, the index and sandbox are rebuildable projections, and the server owns recall while precision and citation go back to the frontier model that asked.
Source material: the author’s long post, plus the open skill repository (the engine is not open source; the skills are, MIT, 13 of them). Numbers below are labelled: measured by the author, or counted by me in the repository.
1. Four generations, each pushed by the last one’s failure#
| Stage | Shape | The failure that forced the next | Numbers |
|---|---|---|---|
| April | one SKILL.md telling the agent where transcripts live (one JSONL per session under ~/.claude/projects) and how to grep them | the agent improvised every time: slow and uneven | one file |
| May–June | three scripts: find_sessions.py, extract_user_prompts.py, dump_turn_context.py | every search read the whole disk, and Codex was unreadable | 5,000+ files, ~5 GB by July |
| July | one recall.py: SQLite full-text index plus an entity table (PR numbers, file paths, job IDs), reading both harnesses | single machine, single person | on a frozen set of 57 questions, top-5 went 23% → 63%; search 15.7s → 0.16s |
| August on | a service: collector → ingest → storage → projections → sandbox, with four MCP tools | other machines, other people | below |
After the scripts existed, agents still grepped by hand about nine times for every single time they used a script. The tool was not adopted because it was slower than paging through files.
2. Why it is called recall, not answer#
The name is the design. The server’s job is not to miss anything; precision belongs to the caller. In the author’s own instruction to the agents building it: focus hard on recall, and let the serverless execution and the grepping handle precision.
That shows up in the interface: every tool returns pointers plus a receipt — the session id, the matched passage, and where in the original it sits. The agent has to open that session before it cites. This single constraint blocks the usual rot of retrieval systems: feed the model a summary, and the model treats the summary as fact.
3. The pipeline: source upstream, projections downstream#
Six parts, each answering a specific problem:
| Part | Why it exists | What it costs |
|---|---|---|
| collector (every machine, every account) | sessions are scattered across machines | something to install and keep credentialed everywhere |
| ingest: redact, write each session once | everything downstream derives from what ingest writes | idempotency is yours to build; a bad write is un-washable |
| object store (manifest + JSONL) | chosen over a page model and over managed memory APIs because those store a summary, and they wanted the source | more bytes, slower scans, patched downstream |
| Turbopuffer projection (BM25 + vectors, filtered by person/source/time, computing its own embeddings) | words and meaning must be queried together | an $800 embedding month pushed it there; a full rebuild takes hours |
| Parquet, one set per source × month | broad questions should not open sessions | another pipeline to maintain |
| Postgres catalog (people/sources/grants/receipts) | checked before every tool call, to decide what may be seen | one more service that must never be wrong |
| sandbox (Archil mount, read-only, no network) | send the code to the data instead of gigabytes to the agent | no network; you get DuckDB, ripgrep, Python |
The point of the table is the direction: everything that can change lives downstream. The index can be swapped, Parquet can be rebuilt, the sandbox can be recreated — because the layer above is never rewritten.
4. Four tools for four kinds of question#
| Question | Tool | What actually happened |
|---|---|---|
| What did I work on this week | scope + exec | 51 sessions across 8 sandboxes, about 1.1 GB read in a minute, nothing left storage, a few KB returned |
| How has tool use changed since June | scan | no session opened; DuckDB over Parquet returned a small table — Codex went from a small share in June to most by mid-September, then back to about a third in October |
| How was that benchmark built | search + exec | words, meaning and exact identifiers in one query, reranked, ~2 s to pointers; the top three were opened to confirm, each with a receipt |
| I need to take over recall | search × N | one search per milestone, 2–4 s each; when a hit lands near the moment instead of on it, narrow the window and search again. The handoff doc cites every claim and becomes the next agent’s system prompt |
That last one is the most reusable trick in the piece: replace “hand over by making someone reread the sessions” with “produce a document with receipts”. The author argues it is better than resuming a session, because the document carries context from outside it — decisions, abandoned options, where to start.
5. Removing the agent in the middle#
The first service had a recall agent inside: your agent asked it, it searched, it answered. They took it out. The reasoning is blunt: the caller is already a frontier model, good at pulling evidence, running its own searches and summarising; a second agent in the middle repeats that work and conveniently hides the evidence.
The decision is counter-intuitive because it deletes the layer that looks smarter. But the cost is real: if the retrieval layer answers on its own, the caller loses the chance to check.
6. Running it like a product#
Agents wrote most of the code; the author spent his time on architecture. Once it worked end to end they started climbing: pick a metric, set a target, let agents work toward it.
| Metric | Before | After a week | The bar today |
|---|---|---|---|
| search p95 | 20 s | 1.4 s | under 2 s |
| right session in top-20 | 54% | 88% | — |
| right session in top-5 | 63% (local index era) | — | at least 90% |
The system then became their instrument for observing themselves: by late October, 53% of agent sessions read memory while 26% called a skill; the author’s own share of Claude messages fell from roughly 4% in the summer to under 1%.
7. Prices and traps#
| Trap | What it looked like | The way around |
|---|---|---|
| embedding bill | $800 in one month | move search into a managed index; treat it as a rebuildable projection |
| a tool nobody uses | about 1:9 script vs hand-grep | measure adoption before capability; slow is the same as useless |
| summaries replacing sources | managed memory APIs, page models | keep only the source as fact, the summary as a projection |
| scattered permissions | each tool filtering on its own | one catalog, resolved before every call |
| only the skill is open | the repo holds SKILL.md and plugin manifests, not the engine | you build the backend yourself; the AgentRun DSL repo is separate |
Worth being precise about that last row: parcha-skills holds 13 skills and 18,466 lines of Python, but the recall engine is not in it — recall/README.md says the implementation, installers and deployment configuration are maintained separately. This is a readable design plus a usable skill interface, not a “clone and run” project.
8. Four steps to copy#
- Store the source cleanly first. Object storage plus a manifest, written once, never edited. Without that layer, every index change is a migration.
- Build a 57-question frozen set. They measured 23% → 63% with 57 questions. Your set is probably that size too; do not wait for an eval platform.
- Measure the tool-vs-manual ratio once. They moved when they saw 9:1. Without that number you cannot tell whether the new tool is work or decoration.
- Start tools with
scopeandexec, notsearch. Those only need a range, a complete list, and the ability to run code next to the data; they do not require good ranking quality on day one.
9. Open questions#
- A read-only, network-less sandbox is safe, but it also means the agent cannot install a package or patch a script. When a session needs a new dependency to be parsed, does that boundary get a controlled gap?
- Source upstream, projections downstream, and a rebuild that costs hours. At a hundred people and an order of magnitude more sessions, is that window still acceptable, or must the projections become incremental?
- Writing handoffs as documents with receipts turns tacit knowledge into citable objects. If everything is done that way, does the documentation become a second log nobody reads?
References#
- Original: https://x.com/miguelriosen/status/2118075831780340092
- Skill repository: https://github.com/Parcha-ai/parcha-skills
- gbrain (the project they modelled the first index on): https://github.com/garrytan/gbrain
- AgentRun: https://github.com/Parcha-ai/agentrun