Line of inquiry
Inquiring lines›How can we train models to be prop…›What training approaches and contr…›this line of inquiry
How does reinforcement learning discover and optimize reasoning strategies versus supervised training?
A broader line of inquiry — a family of 70 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 70
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- When does RL discover genuinely novel reasoning strategies versus timing optimization?
- Does reinforcement learning preserve reasoning quality better than supervised fine-tuning?
- How does RL compress reasoning path diversity during training?
- Why does prolonged RL discover strategies absent from any base model sample?
- Does reinforcement learning learn optimal per-turn reasoning discipline?
- Can single-problem fine-tuning match full RL pipeline reasoning gains?
- Can reinforcement learning fix the reasoning gaps that supervised fine-tuning misses?
- Why does RL behavior differ between standard reasoning tasks and complex planning domains?
- Can out-of-distribution tests expose memorization in reinforcement learning fine-tuned models?
- Does format-based pretraining determine how models respond to reinforcement learning?
- Why does alternating RL training stabilize learning better than simultaneous updates?
- How does reinforcement learning on outcomes reinforce template-matching rather than computation?
- How do sparse parameter updates enable when-not-how training to work?
- Why does outcome-based RL specifically lose diversity during training?
- Can reinforcement learning teach AI when to ask clarifying questions?
- Can multi-turn reinforcement learning improve tool use in language models?
- Can explicitly optimizing for semantic diversity during RL training improve both quality and variation?
- Can in-context reinforcement learning match human sample efficiency on real problems?
- Should larger compute budgets allocate more resources to reinforcement learning?
- Does RL scaling deliver much worse returns than inference scaling?
- Does the reinforcement learning improvement rate depend on model initialization?
- What role does reinforcement learning play in optimizing inference compute?
- How do Q-value models improve action selection compared to value models?
- Why does supervised fine-tuning on diverse demonstrations expand exploration diversity compared to RL?
- Why do high entropy tokens carry most of the learning signal in RL?
- Why does standard RL cause traces to collapse into redundant reasoning paths?
- Why does exploration diversity behave differently under reinforcement learning versus supervised fine-tuning?
- Can in-context learning replicate the timing effects that RL teaches models?
- Can RL directly optimize attention distributions instead of text generation?
- Why do models follow a two-phase pattern of procedural then strategic learning?
- What role does natural language play in breaking reinforcement learning performance plateaus?
- Does representational distance account hold for on-policy reinforcement learning training?
- Can smaller models achieve domain expertise through focused RL training?
- How does imitation pretraining followed by RL exploration compare to either method alone?
- How does post-training shift models from passive prediction to on-policy action?
- Can LLM-synthesized behavioral heuristics compete with learned policy improvements?
- Do policy gradient methods carry less training signal than other RL algorithms?
- Why does multi-turn RL generate orders of magnitude more tokens than single-turn?
- How does LLM simulation of APIs avoid instability without sacrificing training signal?
- What makes supervised fine-tuning worsen RL exploration later?
- Can decomposing consistency into multiple metrics improve reinforcement learning for dialogue?
- What breaks when you apply reinforcement learning after supervised fine-tuning?
- Can offline reinforcement learning improve dialogue policy baseline performance?
- Can RL-trained policies outperform text-space optimizers for evolving skill repositories?
- Does sparsity in RL arise from training on policy-distribution data?
- Does task ordering affect multi-task reinforcement learning outcomes?
- Can pretraining alone achieve chess performance without reinforcement learning?
- How do thought actions represent policy improvement steps in practice?
- Why does reinforcement learning training degrade model calibration?
- How does credit assignment work across many sequential decision steps in language models?
- Why do single-turn RL methods fail to generalize to multi-turn tasks?
- What distinguishes high-signal prompts from low-signal ones in RL training?
- How should humans specify deterministic abstractions of RL problems?
- How does trajectory filtering handle noise when language models use code execution tools?
- How does reinforcement learning differ from chain-of-thought distillation?
- How do cascaded probabilistic models compare to reinforcement learning for per-query system design?
- Can models learn both what and how to study through reinforcement learning?
- Can curriculum degradation of document quality accelerate policy learning?
- How does non-reasoning SFT prevent overfitting before RL training begins?
- Can reinforcement learning agents optimize better on automatically scored tasks?
- Should user simulators be trained via RL like agents or decomposed into trackable state components?
- How do residual connections and layer norm stabilize training in deep RL?
- What makes software engineering environments better suited for RL than other interactive domains?
- How does active selection of training content differ from random reinforcement sampling?
- Why do medium-difficulty problems produce more stable learning gains?
- Can meta-reinforcement learning explain why this bias pattern emerges rationally?
- How does scaffolding unstable mechanics improve reinforcement learning for search?
- Does replay fidelity hold when policies explore branches history never visited?
- How are exploration policies refined against the simulator in practice?
- How does behavior cloning reduce complexity before RL training in rerankers?