Three papers on arXiv this month attack the same weak point in retrieval-augmented generation from three directions. One regulates how sub-queries evolve during multi-step retrieval. One fixes contrastive decoding so it stops overwriting answers the passage actually supports. One measures whether the retrieval tricks you already trust still help once you stack them.

Read together, they describe something most RAG teams learn the hard way. The components are usually fine. The composition is where accuracy goes to die.

RAG breaks in two places

There's the retrieval side, where you decide what to fetch. And there's the generation side, where you decide whether the model actually uses what you fetched. Tooling treats these as separate problems. Teams staff them as separate problems. The failures leak across the boundary anyway.

The scale involved matters here. The LASAGNA evaluation spans corpora from 5K to 200K documents, and that range is not a detail. At 5K documents you can rerank every candidate and still stay inside a sane latency budget. At 200K, reranking becomes a cost problem, and the design that won on the small corpus stops winning.

Three papers, one thread

Paper	Failure it targets	Mechanism	Headline result
MAPS	Sub-queries drifting along one retrieval direction	Parallel branch generation plus a reset reasoning state	Beats representative RAG methods across six QA benchmarks and multiple backbones
LASAGNA	Isolated component gains not surviving composition	450 controlled configurations, 8 benchmarks, 5K to 200K docs	+47% predicted from isolated gains, +17% delivered in composition
Accept-or-Repair	Contrastive decoding overwriting passage-supported answers	Passage check first, up to two contrastive repairs only on failure	+12.5 points on counterfactual passages, +2.3 on ordinary ones

Different problems, same shape. Each one is a case where a technique that looks correct in isolation does damage once it's wired into a live pipeline.

Search momentum: the sub-query that keeps pointing the same way

Interleaved RAG systems generate a sub-query, retrieve, reason, generate the next sub-query. IRCoT and its descendants all work this way. The trouble is that each new sub-query is conditioned on the entire reasoning trajectory, including every sub-query already issued.

MAPS names the accumulated effect of that conditioning search momentum. The finding is blunt: query generation tends to keep evolving along trajectories already established. By the fourth step, your sub-queries are paraphrases of the first one. You paid four retrieval calls for one piece of evidence, and the passages you never asked about stay invisible.

The fix has two parts. First, keep the retrieved knowledge and a refined reasoning state, but drop the historical reasoning trace before generating the next sub-query. That one change removes the conditioning that produces drift. Second, replace the sequential loop with parallel branches: identify complementary retrieval directions at each iteration, issue one sub-query per direction, retrieve them concurrently, then consolidate the evidence into the state.

Parallel retrieval at depth three means three concurrent calls instead of three serial ones. Wall-clock latency stays roughly flat while coverage goes up.

Quick take: Query drift in agentic RAG comes from conditioning every new sub-query on the whole reasoning trace, so the highest-leverage fix is discarding that trace, not writing better prompts.

The composition tax: 47% promised, 17% delivered

LASAGNA is an open-source framework for controlled RAG evaluation, and its authors pre-declared three hypotheses before running anything. Then they ran 450 configurations across eight benchmarks.

The first finding is the one that should worry you. Three retrieval techniques, each strong on its own, predict a combined improvement of +47%. Used together, they deliver +17%. The missing 30 points don't show up in any single-component ablation, which is exactly how teams end up shipping a pipeline that underperforms its own benchmark table.

Key numbers450 RAG configurations tested across 8 benchmarks and corpora from 5K to 200K documents +47% predicted from isolated retrieval gains, +17% delivered when composed +12.5 points recovered on counterfactual passages by Accept-or-Repair +2.3 points recovered on ordinary passages, where nothing was supposed to be broken

The second finding cuts deeper. Retrieval quality and answer faithfulness don't stack the same way, and optimizing one can hurt the other. A reranker that lifts recall is doing its job. Whether the model then quotes the passage instead of its own weights is a separate question with a separate answer.

The third finding kills the small-corpus shortcut. The best composition changes as the corpus grows. Whatever won on your 8K-document prototype is a hypothesis about your 200K-document production index, not a result.

Accept or repair: contrastive decoding's collateral damage

Contrastive decoding targets a real problem. Models answer from parametric knowledge even when the provided passage contradicts it. The technique contrasts token predictions under the full prompt against a modified prompt with the passage stripped out, then amplifies the difference so the model can't lean on what it memorized.

It works, and it's indiscriminate. Because the contrast runs at every token, it also perturbs answers the passage genuinely supports. You trade knowledge override for a new failure where correct, grounded answers get rewritten for no reason.

Accept-or-Repair treats that as a routing problem. Sample an answer. Check whether the passage supports it. If it does, keep it and do nothing else. If it doesn't, try up to two replacements with contrastive decoding and return the first one that passes the check. Replacements that merely repeat the no-passage answer are rejected when the passage favors something else.

	Contrastive decoding at every token	Accept-or-Repair
Training required	None	None
When it acts	Every token of every answer	Only when the sampled answer fails the passage check
Effect on passage-supported answers	Can alter them	Leaves them untouched
Extra compute	A second forward pass per token	Up to two extra generations, only on failures
Result	Baseline	+12.5 over sampling on counterfactuals, +2.3 on ordinary passages

The detail I keep coming back to: AoR beats eight contrastive and prompting baselines, and it still wins when each baseline gets the same rejection rule bolted on. More telling, applying only the rejection rule (replace just the unsupported initial answers, no repair at all) improves the average score of every decoder tested. The passage check is doing most of the work. The repair pass is cleanup.

What I hit when I wired these together

I've been running an IRCoT-style loop on an internal corpus for a few months, and the drift description matches what I saw before I had a name for it. By step three my sub-queries were rephrasings of step one. I blamed the generator and spent a week on prompt variants. The fix that actually moved numbers was truncating the reasoning history passed into the query generator.

Then I did the thing LASAGNA warns about. I added a query rewriter, a reranker, and context compression in one sprint, measured the whole stack against the baseline, and got a gain smaller than the reranker alone had produced in its own ablation. I assumed one of the components was broken. The honest explanation is that they were competing for the same improvement, and the second and third were mostly redundant.

On the decoding side, I'd been running contrastive decoding unconditionally and quietly accepting that some grounded answers came back worded differently than the passage supported. Adding a support check before generation, and only invoking the contrast when it failed, cut those rewrites and cost almost nothing on the passages that were already fine.

My team's takeaway was unglamorous. Most of the accuracy was sitting in verification, not in generation tricks.

Common Pitfalls

Tune retrieval and faithfulness against the same metric

Recall@k and answer faithfulness respond to different knobs. A configuration that maximizes retrieval hit rate can increase the rate at which the model quotes irrelevant-but-similar passages. Keep both metrics in the loop or you'll optimize your way into a worse product.

Stack techniques without re-running the whole stack

A +20% technique and a +15% technique do not make +35%. LASAGNA's three-technique result predicted +47% and delivered +17%. Any component you add needs a full-pipeline ablation, not a component-level one.

Validate at 5K documents and ship at 200K

The winning configuration changes with corpus size. Build a smaller index for fast iteration if you want, but re-run the final comparison at production scale before you commit. This is the cheapest experiment in the entire pipeline and the one most often skipped.

Apply contrastive decoding at every token

Blanket contrast fixes knowledge override and breaks answers that were already correct. Gate it: check the sampled answer against the passage, and only spend contrastive decoding on the failures.

Let one reasoning trace steer every subsequent sub-query

Concatenating the full history into the query generator feels like giving the model context. In practice it produces momentum, and momentum produces four paraphrases of the same query. Reset the state, keep the evidence, and branch across directions instead of walking deeper into one.

A pipeline shape that survives all three

The three findings compose into a workflow: check first, repair second, re-retrieve last.

The gate is cheap. A support check costs one pass, and AoR's results suggest it lifts every decoder it touches. The repair is bounded at two attempts, so worst-case latency is predictable. Re-retrieval is the expensive branch and it only fires when both the original answer and the repairs fail, which in a working system should be rare.

One Thing to Remember

Across all three papers, the highest-return component is verification. Checking that an answer is supported by the retrieved passage, and checking that a sub-query is actually asking something new, beat every generation-side trick in these results. If you have one engineering week to spend on RAG quality, spend it building a support checker before you spend it on a better decoder.

The Bottom Line

If your interleaved retrieval loop produces sub-queries that look like paraphrases after step two, reset the reasoning state before you generate the next query. MAPS gets its gains from discarding history, which costs nothing at inference time, and parallel branching keeps wall-clock latency flat while widening coverage.

If your evaluation corpus is a few thousand documents and production is six figures, treat your ablation table as a hypothesis. LASAGNA shows the best composition flips with corpus size, so re-run the winning configuration at production scale. That single experiment is worth more than another month of component tuning.

If you're using contrastive decoding to stop the model from overriding passages with memorized facts, wrap it in a passage check. The unconditional version rewrites correct grounded answers. The gated version gains 12.5 points on counterfactual passages and 2.3 on ordinary ones, and the check alone improves every decoder it's attached to.

Sources

• MAPS: Macro-Adaptive Parallel Reasoning for Regulating Search Momentum in Retrieval-Augmented Generation
• Accept or Repair: Preserving Passage-Supported Answers in Contrastive Decoding
• Retrieval Augmented Generation Compositionality
• MAPS code and resources: github.com/NEUIR/MAPS
• Accept-or-Repair code (announced, releasing): github.com/js-lee-AI/AoR