Line of inquiry
Inquiring lines›How do training data and architect…›What training signals and data sel…›this line of inquiry
How do different post-training structures affect learning outcomes?
A broader line of inquiry — a family of 100 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 100
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- What makes advantage shaping more stable than reward shaping for tool training?
- Why does alternating RL training stabilize learning better than simultaneous updates?
- How does post-training shift models from passive prediction to on-policy action?
- Can distillation and reward optimization happen in a single training loop?
- Does partial trace guidance work better than curriculum learning for hard problems?
- Does the productive difficulty band ever stabilize during training?
- Can feedback loop frequency harm performance on finite task sets?
- Can continuous spectrum training outperform sequential SFT-then-RL stages?
- How do complete multi-turn trajectories differ from isolated task examples?
- How does decomposing training telemetry by reward components provide dense feedback?
- Why do dense rewards plus hard constraints outperform single fixed rewards?
- How does absolute-advantage weighting concentrate training on boundary cases?
- How does model scale affect anticipatory behavior in structured training?
- How should multi-objective post-training balance competing behavioral goals?
- How do different training objectives shift whether models over-predict or under-predict?
- How do evaluative versus directive signals differ in next-state training?
- What distinguishes high-signal prompts from low-signal ones in RL training?
- How does process supervision relate to execution-signaled feedback approaches?
- Why do queries with low cross-rollout variance produce degenerate gradients?
- Can trajectory quality filtering improve model training in noisy environments?
- Can distillation methods extract directional guidance that scalar RL cannot access?
- When does outcome reward signal become informative during model training?
- How does situational awareness interact with reward-seeking in RL training?
- What makes a sub-goal verifiable enough to provide dense feedback signals?
- Why does outcome-only reinforcement learning need more than double the tokens to train agents?
- How do tree rollouts convert outcome rewards into step-wise process supervision?
- How does early branch divergence differ from late branch divergence in supervision signals?
- What distinguishes surface mechanisms from the training regimes that produce them?
- Do different function-calling subtasks have different entropy profiles during training?
- How does temporal anchoring maintain learning signals when preference gaps collapse?
- Why does imitation learning alone plateau without outcome-based refinement?
- What makes reasoning tokens identifiable within rollout groups for better rewards?
- Does incidental optimization pressure on CoT produce goal suppression without deliberate strategy?
- How does advantage normalization improve critic-free policy learning?
- Does reverse-curriculum learning approximate process supervision using only outcome signals?
- Do fed-back concepts or the auxiliary objective alone drive the performance gain?
- Can curriculum degradation of document quality accelerate policy learning?
- Which recipe choices determine the asymptotic ceiling in RL training?
- How does tree-search topology convert outcome rewards into intermediate supervision?
- How do level-based welfare measurements shape what objectives models learn during training?
- How do you extract reward signals when all rollouts fail?
- Can granular sub-task training for function calling improve both open and proprietary models?
- How does process-focused feedback compare to outcome-focused feedback in skill training?
- What deployment modes work best for trajectory-aware reward signals?
- Why do zero-advantage rollouts destabilize training beyond just wasting compute?
- Can parallel trajectories reveal better decision branches without human labeling?
- How does sliding the start state backward create informative learning signals?
- What causes on-policy distillation to become unstable at scale despite dense rewards?
- How do weights, selection, and prompts create different geometric landscapes of accessible behaviors?
- Why does externalizing bookkeeping raise effective feedback compute?
- Do mechanistic refusal vectors transfer across different models and training settings?
- Why do certain tokens at certain difficulties drive most of RLVR's learning signal?
- What makes a task at the edge of competence optimal for RL?
- Can constant penalties replace teacher-provided advantages in token supervision?
- What tokens do RL-trained summarizers learn to keep for ranking?
- Can curvature measurements predict task difficulty without behavioral labels?
- Can explicit goal state scaffolding at inference time transfer to autonomous tracking through training?
- How do you prevent stale reward signals when skills evolve during deployment?
- Can gradient approximation at equilibrium replace backpropagation through time in practice?
- Why do harness validators shape what models learn to emit?
- Can tree-GRPO work with extremely noisy or sparse outcome reward signals?
- How do training objectives shape what a world model actually learns?
- How should guidance levels adapt as the model's capability boundary shifts?
- How does branching depth in tree rollouts determine process supervision granularity?
- Why do medium-difficulty problems produce more stable learning gains?
- How does task-oriented fine-tuning compare to preference tuning methods?
- How does active selection of training content differ from random reinforcement sampling?
- Does negative reinforcement alone achieve what full RL training accomplishes?
- What failure modes do imitation and outcome methods each address?
- How do weight visualizations reveal temporal structure in cyclic training?
- Why does curriculum learning with tight budgets beat fixed-budget approaches?
- How does scaffolding unstable mechanics improve reinforcement learning for search?
- Does weight decay directly cause contractive behavior near training examples?
- Why do next-turn reward objectives fail to encourage multi-turn goal progress?
- Can dynamic variance weighting replace fixed objective combination weights?
- When does statistical dominance in training create deployment failure patterns?
- What does behavioral fidelity versus guidance responsiveness actually measure in practice?
- What distinguishes learnable perturbations from bifurcation-triggered motive shifts?
- How does pretrained knowledge constrain what adaptation strategies can achieve?
- What makes content informative and not-yet-mastered for reinforcement during pretraining?
- How does off-policy data reuse inside trust regions affect convergence guarantees?
- Why does group-relative normalization make uniform episode rewards work across rollouts?
- How does a challenger's escalating difficulty function as curriculum?
- How do developmental curriculums emerge from learning progress signals?
- What makes exploration a verifiable and measurable training objective?
- Can backward transfer measurements reliably predict optimal multi-task training order?
- Can group-relative normalization be modified to resist shortcut trajectories?
- What makes session-aware multi-turn tracking necessary for asynchronous training?
- Why do six different RLVR algorithms converge on similar performance levels?
- How does trajectory burstiness compare to other structural properties that shape emergent capabilities?
- Why does retrieval chain training unlock scaling laws in QA?
- What role does KL penalty strength play in format selection?
- Can unified policies handle negative feedback and critique transformation simultaneously?
- What makes Effective Rank Acceleration a stable training signal for dual-channel incentives?
- How does DVAO balance reward components differently than VPO spreads them?
- How should training distribution distance be defined when the policy evolves?
- How does behavior cloning reduce complexity before RL training in rerankers?
- What features does a sample reinforce when it moves bands?
- How does KL penalty strength affect the degree of format collapse during RL?
- Can utility control modify LLM values more effectively than output filtering?