Retrieval as Navigation: DecisionRAG Asks a Tree Instead of Scoring a Corpus

The bottleneck in retrieval is not the size of the candidate pool. It is what you let the model see, and in which order. DecisionRAG replaces scoring with asking, and returns a 65.4 top-1 hit over 2.68M passages in 1.7 seconds. It also loses three cells, plainly.

English voiceover, 3 minutes 12 seconds. Same content as the Chinese cut.

The paper is Retriever-Free Retrieval: Navigating Documents with Decision Models, from Cornell and ASAPP, arXiv 2610.35868, 21 pages (abstract , PDF ). Every number below traces back to a table or an appendix. Anything I recomputed is marked as such.

1. The wrong row, ranked first#

The paper opens with one query. Who is the lead in Christopher Nolan’s film from the year 2000. BM25 and dense retrieval both push Gladiator to the top: it is also a 2000 film, the words match, the vectors match. Memento is the page that answers the question, and it sits far down the list.

The retriever is not lazy. Its interface has one shape, compare the query with one candidate at a time. That interface looked fine until the generators got two generations better while the retrievers kept doing term counts and cosine.

The paper’s framing is a capability asymmetry. What the retriever fails to find, even the strongest generator cannot recover. The framing is standard, and the repair it picks is the part worth reading.

2. Where the research stood#

Three families of repair, each with a visible ceiling.

The third family is the neighbour. It already has a hierarchy and already decides at each level; it just writes those decisions out token by token. Hand the same tree to GPT-5 and make it name the branches to keep: 28.2 seconds per query on ConditionalQA, 250.8 seconds on Natural Questions (Table 6).

The repair replaces “generate a list of IDs” with “a batch of yes/no questions in one forward pass”. Decision models (Jev 1.13, OpenAI’s gpt-6-luna-decisions) do exactly that: given a context and a set of questions, return a distribution over each question’s options, generating no tokens.

3. The design: three constraints#

Figure 1 What gets removed is the premise that every passage must face the query alone, not the passages themselves.

Strip the machinery and three constraints remain.

One forward pass per level. Every candidate title of a level goes into one shared context, and each candidate gets a yes/no question: is this branch worth walking. A level that does not fit one request is split: the 1,000 cluster descriptions on Natural Questions go out as eight parallel requests, the ~17,500 page titles in batches of 20k tokens. Pruning is irreversible. A branch dropped at one level never returns for this query.

Scores form in company. A yes/no probability is normalised over yes and no only, not across candidates, so a score from one request is not comparable with a score from another. The paper’s fix is to judge candidates together, so each probability forms with a better candidate standing next to it. That is not calibration. It is relativisation.

No index, no task training. No first-stage search at query time, no vector index loaded. Budgets and routing rules were chosen on design sets (MultiDoc2Dial training samples, 300 ConditionalQA training questions, 50 synthetic MTRAG conversations plus 131 MTRAG-UN turns). No weights move.

The two judgment modes map to two needs. Inner levels want recall, so they use parallel yes/no questions, whose count is limited only by context length and whose probabilities do not sum to one. The last level wants rank one, so it uses a multiple-choice question where twenty candidates compete and the probabilities do sum to one. Jev caps options at 255, so choice is applied only to the top 20 after the binary pass, and to the top 5 on Natural Questions.

4. What a query looks like#

Natural Questions is a three-level tree.

Two patches carry the cost of this design.

• A passage score is multiplied by its page’s probability. The binary scores of two levels are not on one ruler; the multiplication is how the gap is papered over.
• Page-title scores are pooled across requests. The 1,000 clusters go out as eight requests and the 17,500 titles as a batch of requests, and then the scores are mixed and ranked together. By the paper’s own §3.2 argument, scores from separate requests are not comparable. Only MTRAG re-scores its top 50 in one request. This is the least clean part of the design, and the paper neither hides it nor prices it.

Budgets differ per dataset because hierarchies differ: MultiDoc2Dial is 4 help centres → 413 pages → 4,110 passages, ConditionalQA 652 pages → 58,483 paragraphs, MTRAG 172 products → 8,578 pages → 72,442 passages, Natural Questions 1,000 clusters → 108,626 pages → 2,682,017 paragraphs.

5. Nine metrics, six wins#

Figure 2 Two of the six wins are 1.7 and 2.5 points wide, so read the intervals before the headline.

The three losses, stated.

• ConditionalQA top-1 loses 3.7 points to GPT-5 reranking. Adding dense candidates closes it (73.4).
• Natural Questions top-10 recall loses 2.3 points to an 8B reranker, while nDCG@10 leads by 1.4. It orders the head better and the tail worse.
• MultiDoc2Dial R@1 is a tie with GPT-5 reranking, 47.4 against 47.5. The difference is R@10: 83.6 against 78.6.

One sensitivity check is easy to miss. MultiDoc2Dial contains the same text under different IDs; de-duplicating by text lifts the paper’s R@1 from 47.4 to 55.8 (57.0 with dense candidates) and the dense-plus-8B-reranker baseline from 33.6 to 40.3. Both rise, the gap goes from +13.8 to +15.5, and the ordering holds (Appendix E.1).

Training an 8B judge on in-domain data reaches 60.7 R@1 on MultiDoc2Dial, 5.7 above the best published trained retriever; training on MultiDoc2Dial alone reaches 63.1 (Appendix Table 17). The framework also improves from gradients.

6. What the shared context is worth#

Figure 3 47.8 judged alone, 54.3 at fifty per request, 60.6 as one choice question over all 200.

Three ablations, all recomputable.

First, the shared context is relativisation, not calibration. A candidate that looks relevant on its own drops when a better one is present. It answers “which of these is worth walking”, not “is this probability right”.

Second, the gain saturates around 50 candidates per request. That is good news and bad news at once: the 17,500 page titles can be split without loss; and the argument that scores must be globally comparable weakens, because once a level is split you are ranking scores from different contexts, which is the thing §3.2 says cannot be done.

Third, choice reranking is a cheap closing step: one request of 0.12 to 0.15 seconds, 4 to 33% more cost, and 5.3 points of top-1 on ConditionalQA. On MTRAG it only promotes the highest-probability passage to rank one, because a long reference reply is grounded in several passages.

7. Fewer candidates, better first place#

Figure 4 On Natural Questions: gold present 98.4% against 94.0%, gold ranked first 59.4% against 66.4%.

The same Jev model, doing the same job two ways.

Fewer candidates, better first place, and the gains survive the intervals: +7.0 [3.4, 10.8] and +7.7 [2.6, 13.3]. On MultiDoc2Dial it is only +1.3, and on MTRAG +0.4, so this is not a universal law.

The paper’s explanation is that the tree hands the model whole pages in reading order, while a reranker hands it isolated passages. That runs alongside the DAPR line on document-aware passage retrieval. The real value of the numbers is that they split “retrieval quality” into two things: candidate coverage and ranking ability. The hierarchy gives up coverage and buys a better ranking condition.

It also shows where the remaining error lives. On ConditionalQA the three-page configuration misses the gold page only 8.5% of the time; ranking a different page’s paragraph first accounts for 9.6%, and ranking another paragraph of the gold page first accounts for 13.7%. On MultiDoc2Dial, given the gold page, the passage stage still only reaches 62.7 R@1. Navigation answers “did we find the page”. The rest is passage ranking, which is exactly the shared-context judgment.

8. The cost, and four soft spots#

Figure 5 At 15% kept, R@10 is 86.2 against 84.5 for flat scoring, at 2.3 cents against 12.1.

The paper claims 34 to 145 times lower latency and 21 to 25 times lower cost. I recomputed both from Tables 5 and 6 and they hold: 28.2/0.82 = 34.4, 250.8/1.73 = 145, 6.1/0.29 = 21.0, 57.2/2.33 = 24.6.

Discount both multiples anyway. The baselines are timed as a single local GPU reranking 100 candidates; the paper is timing an API with up to eight concurrent requests per query, and Appendix F says outright that these are not controlled speedups. On that same GPU, dense encoding plus search takes 0.05 to 0.08 seconds, so almost all of the 7.5 seconds is the reranker waiting on the GPU. Serve the reranker properly and the multiple shrinks.

Cost is not logarithmic either. Latency grows with depth (T(N) ≈ t_req ⌈log_b N⌉; with b=100, 10⁹ passages need five decisions, about a second), but each level judges k·b candidates, and cost follows that product. Natural Questions already costs 2.33 cents per query, against 1.40 for GPT-5 reranking.

1. The hierarchy is synthetic. On Natural Questions the 1,000 clusters are k-means over e5-base-v2 embeddings of title plus first sentence, and each cluster description is one sentence Qwen3-8B writes from the page titles. Wikipedia titles carry signal, so this level works. Swaps contracts, logs, or code in and the top level collapses. The limitations section admits this and never measures it on a corpus whose titles say nothing.
2. Scores from different requests are pooled. See §4. It is the least clean part of the design.
3. “No task training” has an asterisk. No weight moves, but budgets, routing, and question wording were all chosen per dataset on design sets. The wording matters: “should the agent’s next reply be based on passage n” and “does paragraph n contain the information needed” are different questions.
4. Read the evaluation setup. MultiDoc2Dial’s official test split is unusable, so the paper reports 4,201 validation turns. Natural Questions is a fixed 500-question sample of a 3,610-question test split. MTRAG’s labels were expanded by GPT-5, which judged 87.9% of the original gold passages as needed, pooling the top-5 of 18 systems (Appendix A.3).

One more, not the paper’s fault: the headline result depends on Jev 1.13, a commercial decision model. The open-weight judges it trains point the right way, but a general 8B judge reaches only 40.4 R@1, below Jev’s 47.4, and the 1.7B model reaches 20.5, which is below BM25’s 20.6. “Works without training” is true of one closed model.

9. How to use it, and two open questions#

Three judgments for anyone trying this on their own corpus.

• Spend the budget at the top, not at the bottom. Raising the top level from 1% to 15% takes R@10 from 63.7 to 86.2. Raising the page budget from 3 to 10 buys +2.6 R@10 at +83% cost and +28% latency.
• Mix it with dense retrieval. Adding the dense top-20 improves R@10 on all four benchmarks (83.6→84.7, 71.9→76.3, 86.7→88.9, 34.2→36.6), which says the two recover different evidence. Retriever-free is a design stance, not a deployment recommendation.
• Measure your titles first. Sample a hundred page titles and ask whether a human could judge relevance from the title alone. If not, the first level does not exist yet, and the work is in the titles or the structure.

Two questions I cannot answer yet.

1. Can the descriptions come from the corpus instead of a frontier model, via link graphs or tables of contents? Even the cluster summaries here are model-written, and that dependency is never measured on its own.
2. Can decision-model scores be calibrated across levels and across requests? Right now two patches carry it, multiplying a passage score by its page probability and re-scoring the top 50 in one request, and neither patch’s cost is quantified.

References#

• Wan, Jünger, Du, Liu, Krishnan, Weinberger. Retriever-Free Retrieval: Navigating Documents with Decision Models. arXiv 2610.35868, 2026. Abstract / PDF .
• Related: Jev 1.13 (typesafe/jev-1.13 ), DAPR (Wang et al., ACL 2024), RAPTOR (Sarthi et al., ICLR 2024), LATTICE (Gupta et al., TMLR 2026), PageIndex (VectifyAI ), RankGPT (Sun et al., EMNLP 2023), Qwen3-Embedding and Qwen3-Reranker (Zhang et al., 2025).
• On this blog: the decision gate JevAct adds to an agent . Same family of decision model, placed before execution rather than before retrieval.

Two cuts of the explainer, one per language, walk through the same nine sections.