What training signals and data selection strategies optimize learning?

27 specific questions

• Why does majority voting reward work better than other test-time aggregation methods?
• How does training-time voting differ from inference-time majority voting over samples?
• How does majority voting fail when reasoning samples lack genuine diversity?
• Does majority voting reliably signal correctness without risking reward hacking?
• When does multi-agent voting help versus hurt performance on tasks?
• Can test-time voting improve reasoning beyond the base model's original capabilities?
• Does majority voting prevent confident but incorrect answers from being reinforced?

42 specific questions

• How does the ratio of synthetic to real training data affect model collapse?
• How do quality and diversity in synthetic data interact with accumulation schedules?
• Is model collapse a property of data replacement or synthetic data itself?
• At what point does output quality outweigh diversity value in synthetic data tasks?
• Why does synthetic-only data degrade performance even when matched in size?
• Can synthetic data preserve the diversity needed for transcendence to work?
• Can synthetic data diversity preserve the transcendence effect or does it collapse?

30 specific questions

• Does iterative DPO preserve the same misalignment mechanisms as online reinforcement learning?
• Does iterative DPO generalize like online reinforcement learning?
• Does iterative DPO generalize identically to online RL on the same benchmark tasks?
• How does iterative DPO compare to full RLVR for studying emergent misalignment?
• Can iterative DPO serve as a tractable proxy for studying on-policy misalignment?
• What theoretical argument connects iterative DPO dynamics to online RL learning?
• What misaligned behaviors does iterative DPO produce compared to online RL?

49 specific questions

• Can neural networks represent symbolic structures without explicit mechanisms?
• How do neural networks decompose tasks into modular subnetworks that transfer?
• Does the linear representation hypothesis reflect networks or reflect our analysis tools?
• What are fractured entangled representations in neural networks?
• Does recursive generalization explain why neurons develop single interpretable functions?
• Why does knowledge storage separate from reasoning circuits in neural networks?
• How do neural networks decompose complex tasks into modular subnetworks?

100 specific questions

• What makes advantage shaping more stable than reward shaping for tool training?
• Why does alternating RL training stabilize learning better than simultaneous updates?
• How does post-training shift models from passive prediction to on-policy action?
• Can distillation and reward optimization happen in a single training loop?
• Does partial trace guidance work better than curriculum learning for hard problems?
• Does the productive difficulty band ever stabilize during training?
• Can feedback loop frequency harm performance on finite task sets?

47 specific questions

• Does scaling data automatically produce compositional reasoning or just better feature encoding?
• Where do neural networks still fail at compositional generalization despite scaling?
• Does scaling model size solve compositional generalization problems?
• Can recursion alone drive generalization better than model scale?
• Can scaling alone create compositional generalization without explicit binding mechanisms?
• Why do long-context language models struggle with compositional reasoning tasks?
• How does scaling and training data enable compositional behavior without symbolic mechanisms?

44 specific questions

• Why does training data format shape reasoning strategy more than domain content?
• How does training format shape reasoning strategy more than content?
• Does training data format shape model reasoning more than domain content?
• How does training data structure shape reasoning strategy more than domain content?
• How much does training data format influence reasoning strategy versus domain content?
• How much does training data presentation format shape reasoning ability?
• Does training data format shape reasoning strategy more than domain content?

113 specific questions

• Can models learn better from critiquing errors than imitating correct responses?
• Does teacher-style refinement of training data transfer equally to all student model distributions?
• Does selecting examples from multiple complexity levels outperform selecting only high-quality examples?
• Why do weaker teacher models sometimes produce better training signals than stronger ones?
• How should training incorporate external critique versus encouraging self-correction?
• What makes some training data teach brittle answers versus robust reasoning?
• Why do explicit quality criteria outperform learning quality from examples alone?

63 specific questions

• Does optimizing directly for semantic diversity improve both reasoning quality and exploration?
• Can explicitly optimizing for semantic diversity during RL training improve both quality and variation?
• Does semantic diversity in output space compete with reward-component diversity?
• How does diversity collapse during iterative self-improvement cycles?
• When does RLHF reduce diversity and when does it preserve semantic variation?
• How does cohort diversity prevent label-free reasoning from collapsing into homogenized answers?
• Can diversity-aware RL objectives prevent format convergence?

63 specific questions

• Do long-term memory modules outperform consolidation into fast weights?
• Does moving memory outside model weights avoid the limitations of in-weight retention?
• Why does fine-tuning for continuous space cause catastrophic forgetting?
• Can data pruning strategies exploit the finite nature of memorization capacity?
• Why does fine-tuning models for continuous reasoning cause catastrophic forgetting?
• Can in-weight memorization scale beyond model parameter count limits?
• Why does specializing to one task make future task learning harder?

86 specific questions

• Do task-relevant parameter changes naturally concentrate in sparse regions?
• Can models reach high interpretability without sacrificing capability through sparsity training?
• Does sparse parameter updating improve test-time training's computational cost?
• How do sparse weight patterns affect model interpretability?
• Can sparse approximations reveal interpretable structure hidden in existing dense models?
• Should production deployments scale budgets with sequence length for sparse models?
• Could activation sparsity signal task difficulty and guide routing decisions?

60 specific questions

• Why does self-correction during generation produce reliable labels without exemplars?
• How does error distribution during training affect a model's ability to self-correct?
• How does error avalanching compound failures in self-training iterations?
• Can models learn to generate their own training examples effectively?
• Can self-consistency checks fully prevent error avalanching in self-training loops?
• Why does self-consistency fail as a proxy reward for correctness?
• What role does the self-consistency threshold play in preventing error reinforcement?