Does limiting reasoning per turn improve multi-turn search quality?
When language models engage in iterative search cycles, does capping reasoning at each turn—rather than just total compute—help preserve context for subsequent retrievals and improve overall search effectiveness?
The overthinking cluster established that extended reasoning within a single query degrades accuracy beyond a critical token threshold. ASearcher extends this to multi-turn search: each turn's reasoning must also be capped, but for a different reason. In multi-turn search, the problem is not just variance inflation within one response — it is that excessive reasoning in one turn consumes context that subsequent retrieval rounds need.
The mechanism: in an iterative search cycle (query → retrieve → reason → refine query → retrieve again), each reasoning step takes up context. If turn N uses its full reasoning budget, turn N+1 has less context available to incorporate new retrieved evidence. The search agent effectively degrades its own ability to update on new information by overthinking in early turns.
This is a distinct failure mode from single-turn overthinking. Single-turn overthinking produces high variance output from one extended reasoning chain. Multi-turn overthinking produces a degraded retrieval loop where later turns are operating with less fresh evidence than they need. The fix is different: not just total compute capping, but per-turn reasoning budgets that preserve context headroom for subsequent iterations.
Since Do iterative refinement methods suffer from overthinking?, this finding places multi-turn search squarely in the same family of problems. The timescale is the retrieval cycle rather than the self-revision step, but the mechanism — sequential iteration that amplifies rather than corrects — is identical. The practical implication: DR agent design must set per-turn reasoning limits, not just overall query time limits.
Inquiring lines that read this note 63
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What capability tradeoffs emerge when scaling model reasoning abilities? How should inference compute be adaptively allocated based on prompt difficulty?- How should we allocate compute between reasoning and retrieval iterations?
- How does per-token adaptive compute improve efficiency in recurrent reasoning?
- How much does inference budget improve self-generated search performance?
- Does parallel retrieval outperform sequential search chains at test time?
- What makes proactive tool retrieval better than single-round semantic matching?
- How does hierarchical query planning versus flat prompting affect multi-source retrieval?
- Can long-context readers handle compositional tasks or just semantic search?
- Does filtering passages before generation improve large model answer quality?
- Could real-time search systems avoid era sensitivity in legal reasoning?
- Can parallel retrieval chains avoid the context consumption problem?
- Why does single-round retrieval fail on multi-step tasks across different domains?
- What limits exist on retrieval budget during inference?
- What computational cost does trajectory-bursty inference impose on per-query context requirements?
- Can adaptive per-step decisions outperform uniform retrieval policies across different reasoning tasks?
- How should retrieval systems handle multi-hop reasoning and iterative information needs?
- How does query decomposition reduce retrieval costs at inference?
- How can per-step decisions about knowledge retrieval improve reasoning over uniform policies?
- How does search budget affect answer quality at test time?
- How do search tasks differ from derivation tasks in reasoning efficiency?
- How should iterative research tasks limit context per reasoning turn?
- How does query planning as a separate step improve multi-hop retrieval coherence?
- How does overthinking in early turns degrade later retrieval rounds?
- Why do long-horizon reasoning tasks need per-turn step limits rather than just compute budgets?
- Does unrestricted reasoning per search step degrade iterative quality over time?
- What is the optimal balance between search rounds and reasoning depth per round?
- How does reflection-based query refinement differ from single-pass retrieval strategies?
- Why do per-turn thinking budgets matter alongside iterative retrieval depth?
- Do expansion-reflection loops and chain-of-retrieval approaches solve the same problem?
- What happens to iterative search quality when reasoning depth is unconstrained?
- What distinguishes iterative query refinement from pure self-revision loops?
- Why do per-turn reasoning caps improve iterative search quality?
- Does full conversation history improve or degrade multi-turn retrieval accuracy?
- How does multi-turn dialogue improve user satisfaction in search interactions?
- Can multi-turn rewards fix models that lose track midway?
- Can multi-turn aware rewards improve alignment beyond single-turn helpfulness?
- Why does multi-turn RL generate orders of magnitude more tokens than single-turn?
- What makes session-aware multi-turn tracking necessary for asynchronous training?
- Can multi-turn reinforcement learning improve tool use in language models?
- Why do single-turn RL methods fail to generalize to multi-turn tasks?
- Why does extended reasoning fail for search and knowledge retrieval tasks?
- Why do reasoning models wander instead of searching systematically?
- Can reasoning in free text then formatting separately recover performance?
- What quality filters distinguish useful reasoning enrichment from shallow repetition?
- What makes multi-turn critique trajectories more effective than single-turn reasoning chains?
- Does the parallel versus sequential trade-off appear in retrieval-augmented generation systems?
- How much does retrieval budget improve when triggered by dual signals instead of fixed intervals?
- What distinguishes systematic search from wandering exploration in reasoning?
- How does structured environment state compare to transcript replay for multi-turn reasoning?
- How do turn-level retrieval failures differ from dialogue-level accumulation failures?
- What causes multi-turn dialogue quality to degrade over time?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does more thinking time always improve reasoning accuracy?
Explores whether extending a model's thinking tokens linearly improves performance, or if there's a point beyond which additional reasoning becomes counterproductive.
extends: the overthinking threshold applies within each search turn, not just in single-turn reasoning
-
Do iterative refinement methods suffer from overthinking?
Iterative refinement approaches like Self-Refine structurally resemble token-level overthinking in o1-like models. Does revision across multiple inference calls reproduce the same accuracy degradation seen within single inferences?
grounds: ASearcher is the retrieval-domain instance of this synthesis insight; multi-turn search is the operational context
-
Does extended thinking actually improve reasoning or just increase variance?
When models think longer, do they reason better, or do they simply sample from a wider distribution of outputs that happens to cover correct answers more often? This matters because it determines whether test-time compute is genuinely scaling reasoning capability.
extends: per-turn variance inflation compounds across retrieval iterations, not just within one response
-
Why does vanilla RAG produce shallow and redundant results?
Standard RAG systems get stuck in a single semantic neighborhood because their initial query determines what documents are discoverable. The question asks whether fixed retrieval strategies fundamentally limit knowledge depth compared to iterative exploration.
design constraint complement: OmniThink solves retrieval scope via reflection-expansion; this note solves per-turn depth via reasoning budgets; complete iterative retrieval design requires both
-
Can retrieval be extended into multi-step chains like reasoning?
Standard RAG retrieves once, but multi-hop tasks need intermediate steps. Can we train models to plan retrieval sequences the way chain-of-thought trains reasoning, and scale retrieval at test time?
CoRAG's tree search offers a structural alternative: instead of sequential deepening that consumes context across turns, branch retrieval chains in parallel and aggregate; best-of-N sampling over retrieval chains avoids the per-turn context pressure
-
Can reinforcement learning scale beyond single-turn language tasks?
Most RL for LLMs targets simple single-turn problems. This research asks whether RL can handle multi-turn interactive environments with sparse rewards and rich environmental feedback, like real software engineering tasks.
validates: SWE-RL shows RL can learn per-turn discipline through training rather than inference-time limiting; the SWE domain's rich intermediate feedback (compiler traces, test logs) enables RL to discover the same per-turn budgeting that ASearcher imposes architecturally
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- RAG-R1 : Incentivize the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism
- SSRL: Self-Search Reinforcement Learning
- ComoRAG: A Cognitive-Inspired Memory-Organized RAG for Stateful Long Narrative Reasoning
- Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses
- The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning
- Test-time Prompt Intervention
- Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models
- Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL
Original note title
long-horizon research tasks require limiting reasoning steps per turn not just total compute because unrestricted thinking degrades iterative search quality