Instead of always handing an AI the same top five search results, can the ranker learn how many to send?
How should a reranker adjust both document order and retrieval count dynamically?
This explores how a reranker can stop relying on a fixed top-k list and instead decide, for each query, both which documents come first and how many to hand to the generator.
This explores how a reranker can stop relying on a fixed top-k list and instead decide, for each query, both which documents come first and how many to hand to the generator. The corpus has one direct answer: treat the reranker as a decision-maker that gets graded on what the generator does with its choices. DynamicRAG trains the reranker as a reinforcement learning agent. The reward is the quality of the LLM's final output, so the reranker learns to adjust order and count together, per query, with no hand-set cutoff Can document count be learned instead of fixed in RAG?. Training has two phases. First it imitates good rankings (behavior cloning), then it refines through generator feedback. Simple queries end up with fewer documents, and complex ones get more.
Order and count are worth learning together because both come down to where attention goes. Discounted cumulative gain, the classic retrieval metric, builds this in. It discounts relevance by position because readers examine top results more carefully than lower ones How can evaluation metrics reflect graded relevance and user attention?. A generator reading a context window behaves similarly, so a document's usefulness depends on where it sits and on how much company it has. A fixed k is a blunt guess about that trade-off. It is one case of a broader failure the corpus names: fixed rules, such as retrieving at fixed intervals, waste context because they ignore what the query needs Where do retrieval systems fail and why?.
The corpus also shows other ways to make ranking responsive without RL. TempRALM adds a time-relevance term to the semantic similarity score and gets up to 74% improvement on documents with multiple time-stamped versions. It needs no retraining or index changes Can retrieval systems ground answers in the right time?. A related finding comes from recommendation, where LLMs ranking items ignore the order of a user's history by default. Recency-focused prompts bring that sensitivity back Why do language models ignore temporal order in ranking?. Together they suggest that "dynamic" can be cheap, since adding a signal or a prompt can be enough. Only the learned approach also decides how many documents to return.
Two lines of work hint at problems the corpus doesn't test on DynamicRAG itself. Process supervision, meaning feedback on intermediate retrieval steps, substantially beats final-answer-only rewards in agentic RAG Does supervising retrieval steps outperform final answer rewards?. That raises the question of whether a reranker's reward should score its selection step directly rather than only the generator's final answer. And ranking systems trained on their own past outputs can amplify their own decisions unless position bias is modeled explicitly Why do ranking systems need to model selection bias explicitly?. A reranker that learns from feedback on the documents it chose is exposed to that risk. These are open questions the neighboring notes point to. They are not results the corpus reports for dynamic reranking.
Sources 7 notes
DynamicRAG trains a reranker as an RL agent using LLM output quality as reward, learning to adjust both document ordering and count for each query. Two-phase training with behavior cloning followed by RL with generator feedback enables the agent to calibrate document selection to query complexity.
Jarvelin and Kekalainen's DCG and nDCG metrics handle graded relevance by accumulating relevance scores with a position discount factor that devalues late-retrieved documents. This binds evaluation to observed user behavior: users examine top results more carefully than lower-ranked ones, making ranking position matter.
RAG systems fail at three structural levels: adaptive triggering (fixed intervals waste context), semantic-task mismatch (embeddings measure association, not relevance), and mathematical limits (embedding dimension constrains representable document sets). These require fundamentally different retrieval approaches, not tuning.
TempRALM adds a temporal term to retrieval scoring alongside semantic similarity, achieving up to 74% improvement over baseline systems when documents have multiple time-stamped versions. The approach requires no model retraining or index changes.
LLMs can extract preferences from interaction histories but disregard temporal order by default. Recency-focused prompts and in-context examples activate latent order-sensitivity, improving ranking without retraining.
Show all 7 sources
Fine-grained feedback on intermediate retrieval steps significantly boosts agentic RAG performance compared to final-answer-only rewards. DPO trained with both positive and negative step feedback outperforms PPO and single-direction training by directly contrasting good and bad retrieval chains.
YouTube's multi-objective ranker uses MMoE for conflicting objectives and a shallow position tower to remove selection bias from training data. Without both mechanisms, models converge on degenerate equilibria that amplify their own past decisions.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- On the Theoretical Limitations of Embedding-Based Retrieval
- RAG-Gym: Systematic Optimization of Language Agents for Retrieval-Augmented Generation
- Chain-of-Retrieval Augmented Generation
- You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures
- Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
- Large Language Models are Zero-Shot Rankers for Recommender Systems
- Retrieval-augmented reasoning with lean language models
- A New Role for Relevance: Guiding Corpus Interaction in Agentic Search