No Vector Store: a 25.87 MiB SQLite Index Reaches 86.62 Recall@10

The semantic call can wait until after the question arrives. Keep words and counts in the index, and MuSiQue R@10 moves from 54.41 to 86.62 on a 2.36-second build with no model calls.

The paper is SAG2: Embedding-Free Multi-Hop Retrieval with BM25 and JEV, by Yuchao Wu at Zleap AI, arXiv 2610.22588, 15 pages (abstract , PDF ). Code lives at Zleap-AI/SAG-Benchmark . It continues the author’s own SAG, which kept event and entity records in MySQL and vectors in Elasticsearch. This one drops both. Every number below traces to a table number in the paper, and anything I divided out myself is marked “my arithmetic”.

1. Where Erik Hort was born#

Appendix A works one question through the system. The question asks where Erik Hort was born. One passage is about Hort himself, matches the question on words, and lands inside BM25’s top 100. A second passage is about Montebello, New York, and only becomes useful once you know Hort was born there. It shares almost no words with the question, so the top 100 does not contain it.

The loop runs like this. Seeds can only come from highly scored passages, and the Hort passage is one of them. The word “montebello” appears in the seed body, gets weighted by four factors, then runs as a query on its own. The Montebello passage comes back through that term. The final ten contain both passages, and recall on this question goes from 50% to 100%.

That question is not a special case. On 389 of the 1,000 MuSiQue questions the final output holds at least one passage that was outside the initial BM25 top 100, 433 passages in total. BM25 scores 54.41 R@10, and the loop scores 86.62.

None of those 32 points comes from a vector operation.

2. Where the field had stopped#

The old problem in multi-hop retrieval is finding the middle passage. Three families each stall in a different place.

The second family leads this paper’s table. On MuSiQue R@10, NV-Embed-v2 reaches 77.27, SAG reaches 83.37, SAG2 reaches 86.62. Structure buys more than a bigger encoder, which is why the line was worth continuing.

SAG pays at preparation time. It runs a large model over the corpus to extract events and entities, encodes every passage separately, and keeps the two halves in two databases. Appendix D gives the cost of the earlier system on MuSiQue, 4.57 hours and 28.26 million tokens. That figure and SAG2’s 2.36 seconds are not a controlled comparison, on different hardware and different models, and the paper says so.

SAG2 replaces that stage. It keeps the principle of computing associations at query time, per question, but the index holds only source text and term counts.

3. One file, three scoring rounds#

Indexing is local text processing. Adjacent paragraphs merge up to roughly 2,200 characters, an oversized paragraph stays whole, and the text is tokenized into SQLite FTS5 with title, heading and body fields weighted 2, 2, 1. MuSiQue’s 11,656 passages take 2.36 seconds, the file is 25.87 MiB, and model calls are zero, cost zero.

Query time is three scoring rounds and two expansions.

Round one takes up to 80 distinct question tokens in an OR expression, pulls the top 100, and has JEV judge each one with an empty context. The ten highest scored passages that have not been expanded become seeds. Twenty expansion terms are drawn from their bodies, each term runs its own query, and the per-term lists interleave until 100 new candidates exist. The pool is the union of five protected initial passages, the accumulated seeds and the new candidates, deduplicated to 115, and all of them are scored again.

Round two repeats the motion with a tighter door, only 20 new candidates, the remaining slots filled from previously scored ones in descending score. The pool is 125. Two rounds is the default, so three scoring rounds, then no model at all. A local reorder passes the top ten through TF-IDF cosine overlap and a determinantal point process, keeps the first slot and the set, and leaves the rest in score order.

That 340 is arithmetic on the configuration, and section 3.5 gives the same decomposition (my arithmetic). It counts passages judged, not HTTP requests and not tokens. A long passage or a ten-passage shared context makes each judgement more expensive, and the paper puts that sentence right next to the bound.

4. The judge is one yes-or-no question#

The part worth stealing is the prompt. JEV’s Noul interface takes natural-language context and a batch of questions, and returns a yes-probability per question without generating reasoning text. SAG2 asks it one thing, whether the candidate itself supplies a necessary fact in a fully supported answer chain.

Four clauses tighten the question.

• A necessary intermediate identification fact counts, so the birthplace does score yes.
• Context may connect facts but may not be attributed to the candidate.
• Topical association or optional background does not count.
• No outside knowledge.

The second clause stops the judge from being fooled by the context it was handed. The third stops a passage being rewarded for adjacency. Together they make the score stateful, because the same passage can score differently in round three than in round one.

Expansion terms are not just frequent words. Each term’s priority multiplies four factors and takes the maximum over seeds.

a_r(t) = max over seeds d [ (1 + log tf(t,d)) · I(t) · (1 + g_q(t,d)) · w_r(d) ]
I(t)   = log(1 + (N - df(t) + 0.5) / (df(t) + 0.5))
w_r(d) = 0.5 + 0.5 · s_r(d) / max_s          (range 0.5 to 1)

The four factors are frequency, rarity, connection to the original question, and the seed’s own evidence score. The weight floor of 0.5 means a low-scoring seed is downweighted, not discarded. The g_q factor is the fine detail. A term can appear in several sentences of a seed, and the score takes the sentence that shares the most rare tokens with the question, which is what lifts a word like “montebello” out of a sentence that happens to matter.

5. Recall on three benchmarks#

The development subsets hold 1,000 questions each, and each corpus pools the supporting and distractor passages shipped with its subset. SAG2 has the best R@5 and R@10 among the nine methods, with SAG and HippoRAG 2 behind it.

On MuSiQue, SAG wins at R@1 and R@2, 36.82 against 34.30 and 63.62 against 62.18, so SAG2’s advantage only appears once the cut-off widens to 5 and 10. On HotpotQA, SAG and NV-Embed-v2 both sit at 97.70 R@10, and SAG2’s 99.00 is 1.30 points more on a corpus that was already easy.

Split by annotated hops, the pattern is clearer. More hops means more value from the second expansion, 2.32 points on two-hop questions, 3.27 on three-hop, 6.17 on four-hop (subtracting the table gives 6.18, and the paper prints 6.17 on a rounding step).

Four-hop AE@10 is 28.31. Full-evidence coverage counts the whole chain, which is stricter than recall, and on more than seven of ten four-hop questions ten passages still miss part of the evidence. A lexical entry closes one gap in the chain and leaves the rest.

6. Half a million enterprise documents#

The second experiment changes the setting. EnterpriseRAG-Bench is a synthetic collection of company communications, 511,962 documents of messages, email, tickets and meeting transcripts. Chunks run to about 1,000 characters with a 1,200 ceiling, every parameter stays the same, and results deduplicate by source document before the top ten.

The vector column is the surprise. 46.0 sits below BM25’s 68.4 and below it in every single category, reaching 24.8 on semantic rewrites. Those figures come from the benchmark’s own published tables and I did not re-run them. One reading is that internal text is full of product names, ticket numbers and people, and an embedding smooths exactly those strings away while a lexical index matches them as they are.

SAG2 reaches 81.10 on the same corpus. Project related at 83.58 and completeness at 76.24 are where expansion earns its keep, both being cases where the answer is spread across documents. Semantic rewrites reach only 58.40, because a lexical entry cannot serve a phrasing with no shared word.

The Bash Agent figure of 55.8 does not compare directly, since its output set has no fixed top-ten cut-off. The paper notes this and does not claim equal compute budgets across the three.

7. The gain is not a bigger pool#

The easy misreading is that this is a larger candidate pool with extra steps. Three controls on the same 1,000 MuSiQue questions take that apart.

The first two rows carry 13.52 points for the judge alone. Expansion adds 15.43. The second expansion adds 3.26, where returns fall off, and the paper calls the one-round configuration the cheaper operating point at 36.29% fewer judgements (my arithmetic).

The budget control scores a one-shot pool of roughly 331 candidates per question, as many judgements as SAG2 spends in three rounds, and reaches 69.82, 16.80 points short, with a 0.64% difference in judgement count (my arithmetic). On the 984 questions that can be matched exactly the gap is 16.62, so the conclusion holds. Spending the budget on capacity does nothing. Spending it in two rounds does.

The context control isolates the other half. Keep the expansion policy, remove the shared evidence, and R@10 falls to 76.25 while full-evidence coverage falls from 69.50 to 52.20, with judgement counts almost unchanged. Those ten points were bought by letting a later round see what an earlier round read.

The paper also runs three alternatives. RM3 alone lands below BM25, 47.83 against 54.41. GAR-BM25 with final-round JEV reaches 73.11 on 100,000 judgements, 69.82% fewer than SAG2 and 13.51 points lower. Those two numbers set the real distance to existing feedback methods, and they show that paying for the middle rounds returns something.

8. The cost#

The last two rows are the real barrier. The index is local and the judgement is not, so questions and candidate text leave the network. An internal corpus under a data-residency rule stops this architecture at the design stage. The author lists that risk in the limitations and names a locally deployable judge as future work, with no timeline.

The second cost sits on the index side. New evidence has to match an expansion term literally, and the terms are drawn from the body of the previous round’s evidence. Synonyms, aliases, abbreviations and phrasings with no shared word never enter. The Montebello example works because the word happens to appear in the seed. The 58.40 on semantic rewrites is what that gap costs.

The third is comparability. SAG’s row uses BGE-Large with Qwen3.6-Flash while SAG2 uses JEV, so two whole systems with their own judges are being compared, and the 3.25-point gap cannot be booked to the architecture alone. The paper says so.

9. What you can check, and two open questions#

To try it, build an FTS5 index over a corpus you own, take the top hundred, and ask a cheap model one yes-or-no question per passage, whether it is a necessary step in an answer chain. Any model that returns a yes-probability will do, no JEV needed. If that rerank lifts recall more than ten points over BM25, the expansion stage is worth writing.

The first open question is where expansion stops paying. One round is worth 15.43 points, two rounds 3.26, and the curve is flat there. Whether a third round turns negative is untested. The 340 ceiling assumes two rounds, and a corpus with longer passages or bigger pools would hit a token limit before it hits that ceiling.

The second is scale. MuSiQue has 11,656 passages, and BM25’s top hundred already holds most of the relevant evidence. On a private corpus of five million passages the starting hundred is worse, the seeds get worse with it, and the whole loop assumes the seeds are good. The enterprise run reaches 512,000 documents but measures document-level recall, and never reports how much seed quality fell. That number decides whether one parameter set works on a real corpus.

References#

• Wu, Y. SAG2: Embedding-Free Multi-Hop Retrieval with BM25 and JEV, arXiv:2610.22588, October 2026. https://arxiv.org/abs/2610.22588
• Wu, Y. et al. SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges, arXiv:2606.15971, 2026. https://arxiv.org/abs/2606.15971
• Sun, Y. et al. EnterpriseRAG-Bench: A RAG Benchmark for Company Internal Knowledge, arXiv:2605.05253, 2026. https://arxiv.org/abs/2605.05253
• Code and evaluation scripts. https://github.com/Zleap-AI/SAG-Benchmark
• TypeSafe, JEV Noul interface documentation. https://typesafe.ai