Line of inquiry
Inquiring lines›How do language models develop gen…›How do inference-time strategies a…›this line of inquiry
How do pretraining data composition and task diversity shape reasoning?
A broader line of inquiry — a family of 77 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 77
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Does task diversity in pretraining data transfer reasoning better than larger models?
- Why do reasoning gains resist clear attribution to specific training changes?
- Can smaller amounts of diverse reasoning demonstrations replace exhaustive factual training data?
- How much does pre-training frequency predict reasoning task performance?
- How do single training examples activate reasoning capabilities in language models?
- Can models maintain reasoning-output coupling while improving domain accuracy?
- Can small demonstration sets unlock general reasoning without large question data?
- How does question difficulty and breadth affect what models learn to reason?
- Does token-level reasoning during pretraining improve general reasoning without task-specific supervision?
- Does model scaling improve knowledge storage faster than reasoning ability?
- Can models learn to select exemplars based on reasoning skills rather than complexity?
- Can mathematical reasoning improvements transfer across problem subdomains?
- Why does reasoning training improve math but hurt knowledge tasks?
- How can one training example improve reasoning across thousands of unseen problems?
- Can reasoning learned from language modeling actually transfer to knowledge-intensive domains?
- Why does explicit theory injection work better than example-based learning for reasoning tasks?
- Can models recover knowledge with completely unrelated retraining tasks?
- Can training improve reasoning coherence without improving actual correctness?
- Why do reasoning tasks improve more than retrieval from lookup memory?
- What kinds of reasoning tasks reveal the ceiling of text-only training?
- Why do single examples trigger large reasoning improvements in models?
- How does evaluation setting affect measured reasoning capabilities in language models?
- Can articulating latent reasoning processes improve transfer across domains?
- Why does eliminating proxy-model filtering improve reasoning emergence in pretraining?
- What makes procedural knowledge in documents generalize better than facts?
- Why does distillation transfer reasoning patterns with few examples?
- Can curriculum learning by reward variance improve reasoning scalability?
- Can reasoning catalyst data serve as a stable foundation for test-time training?
- What makes reasoning-specific post-training different from standard parameter scaling?
- Can a single model trained on two tasks predict untrained decision tasks?
- Why does reconstructing hidden thought processes during training improve cross-domain transfer?
- Why does general reasoning not transfer to knowledge-intensive medical domains?
- Can training on diverse related tasks be more efficient than task-specific training?
- How do procedural versus factual knowledge differ in pretraining versus fine-tuning?
- Can models trained on longer contexts develop better fundamental reasoning abilities?
- Why do knowledge and reasoning train in different network layers?
- What makes knowledge-rich specialized domains structurally different from general reasoning tasks?
- Does specialized training in one domain create capability cliffs elsewhere?
- How does cross-domain reasoning transfer differ from domain-specific knowledge transfer?
- Does reasoning quality matter equally across different types of tasks and use cases?
- What makes some contexts learnable as rules versus requiring model retraining?
- Can episodic and semantic memory improve long-horizon task reasoning?
- Does fine-tuning on NLI tasks reduce or amplify frequency bias?
- Why does imitation learning create a ceiling for reasoning capability?
- Why does semantic similarity retrieval enable skill transfer to novel situations?
- Do text-space skills transfer learning across different frontier models?
- How does RPT compare to learning when versus how to deploy reasoning?
- Do knowledge access methods like search improve reasoning or just coverage?
- How do task frequency and complexity interact with model capacity during training?
- How much does pretraining contribute to ToM performance versus task-specific training?
- Why does target probability matter more than task logical complexity?
- Can contrastive learning teach models to switch between logical and emotional reasoning?
- How many document exposures does procedural knowledge versus factual information require?
- Why are receiver attention heads narrower in reasoning models than base models?
- How does a single training example trigger phase transitions in reasoning output?
- Which medical tasks benefit most from domain-adaptive pretraining?
- Can a single correct example seed exponential improvement in mathematical reasoning?
- Why does reasoning transfer across different numbers but factual recall does not?
- How does the functional separation of knowledge and reasoning affect adaptation methods?
- Does task superposition explain how models learn from multiple in-context trajectories?
- Which domains need knowledge injection versus reasoning-focused training?
- Can attribute decomposition improve other interactive reasoning tasks beyond clinical questioning?
- How do training-time and inference-time knowledge injection techniques compare?
- Can training-time debate work on tasks beyond mathematics and verifiable answers?
- Can extracted skills transfer effectively across different domains and model architectures?
- How do task difficulty and skill type interact in model performance?
- Why do medical and math domains need different types of model improvements?
- Why do medical and mathematical tasks require fundamentally different model capabilities?
- Why do recursive belief models require different training than logical derivation?
- What techniques work best for injecting domain knowledge at training time?
- How does concept vocabulary size affect the efficiency gains from joint training?
- Can reasoning skills trained on law improve performance in STEM?
- Why does NLI fine-tuning amplify frequency bias instead of teaching inference?
- How does business logic specification replace annotated training datasets?
- Does balancing four pedagogical capabilities improve tutoring or create performance tradeoffs?
- What specific tasks should evaluate whether models understand pedagogical sequencing?
- How much of Occamy's result comes from training versus the base model?