SYNTHESIS NOTE
Do base models already contain hidden reasoning ability?
Explores whether reasoning capability emerges during pre-training as a latent feature rather than being created by post-training methods like reinforcement learning or fine-tuning.
Three convergent findings build a strong case that reasoning capability is primarily a pre-training phenomenon:
Finding 1 (Base Models paper): Base models already spontaneously demonstrate strong reasoning capabilities and "aha moment" self-reflection patterns when sampled sufficiently. Reasoning traces generated by RL-fine-tuned models are already present in base model outputs — they just appear with lower frequency. RL biases generation toward high-reward patterns; it doesn't create new patterns.
Finding 2 (Steering): A hybrid model using base model weights + thinking model steering vectors recovers 91% of the performance gap to thinking models while steering only 12% of tokens. The reasoning mechanisms (backtracking, uncertainty estimation, subgoal-setting) already exist as directions in the base model's activation space.
Finding 3 (CFT/RLVR): Critique Fine-Tuning on a single problem can unlock reasoning potential at RLVR-level effectiveness. By exposing the model to diverse critiques of varied incorrect solutions to one problem, CFT activates reasoning patterns already latent in the base model without requiring hundreds of GPU hours of RL training.
Finding 4 (CoT-Decoding): Pre-trained LLMs inherently contain CoT reasoning paths that can be elicited simply by altering the decoding procedure. Rather than greedy decoding, inspecting top-k alternative tokens reveals that CoT paths are frequently present in the model's probability distribution. A confidence metric differentiates CoT from non-CoT paths — the model shows increased confidence in its final answer when a CoT reasoning path is present. This is entirely unsupervised, requiring no prompting, tuning, or training modifications — purely a decoding change. CoT-decoding adds a fourth mechanism to the latent capability evidence: RL steering, CFT, RLVR, and now decoding all unlock reasoning already present.
Finding 5 (SAE Reasoning Steering): Sparse Autoencoders decompose model activations into interpretable features, revealing latent features causally associated with reasoning behavior. Steering a single identified reasoning feature at the first generation step matches or exceeds CoT performance across six model families up to 70B parameters — without any explicit CoT prompting. The reasoning mode triggers early in generation and is robust enough to override prompt-level \no_think instructions. This is the most direct mechanistic evidence yet: the capability is not just present (as CoT-decoding shows) but causally controllable through a single latent dimension. See Can we trigger reasoning without explicit chain-of-thought prompts?. Together with CoT-decoding (Finding 4), this establishes five independent elicitation mechanisms: RL steering, CFT, RLVR, decoding, and SAE feature steering — all converging on the same latent capability.
The synthesis: post-training methods are selectors, not creators. They select which of the base model's latent capabilities to express reliably in context. The implication is that the main bottleneck for reasoning is not capability acquisition (which happens during pre-training on the world's text) but capability elicitation.
RLVR evidence deepens this: Two additional findings from the RLVR literature reinforce the latent-capability thesis. First, 1-shot RLVR achieves a 37-point jump on MATH500 (36%→73.6%) from a single training example. After the model perfectly memorizes its one example, test accuracy continues improving for 1,400 more steps — post-saturation generalization. The data is exhausted, but activation continues. See Can a single training example unlock mathematical reasoning?. Second, spurious rewards — random, incorrect, or format-only — improve Qwen2.5-Math nearly as much as correct rewards (~21-25% improvement). But the same spurious rewards fail completely for Llama3.1 and OLMo2. The differentiating variable is not reward quality but pretraining: Qwen's code-reasoning pretraining creates latent capability that any optimization pressure can activate. See Why do random rewards improve reasoning for some models but not others?. Together with the pass@k finding that RLVR narrows capability scope rather than expanding it, the evidence converges: RLVR is a catalyst that triggers a phase transition from broad pretraining distribution to reliable sampling of correct answers.
This partially contradicts Can simple rewards alone teach complex domain reasoning? — that note documents genuine capability emergence in domain-specialized contexts (medical, mathematical). The reconciliation: emergence may reflect reliable expression of latent capability, not creation from scratch. The distinction matters for research direction: if capability already exists, the investment in RL may be better directed toward elicitation methods.
The implication for Can prompt optimization teach models knowledge they lack?: the same principle extends to reasoning capability, not just knowledge.
Inquiring lines that read this note 397
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Should agent evaluation measure execution quality beyond task success?- What cognitive capabilities do agents need to internalize social feedback?
- Can trajectory-level visibility separate refusals from real skill gaps?
- What makes conceptual inquiry the fastest high-scoring AI interaction pattern?
- How do different LLM integration paradigms affect inheritance of pretraining biases?
- Can frozen world models from training cutoff remain adequate for real-world reasoning?
- How does instrumental reasoning reproduce pre-Enlightenment knowledge structures?
- What would an AI trained for emancipatory reasoning look like?
- Can AI answers decouple from the reasoning processes that produced them?
- What counts as a prior versus a skill in intelligence testing?
- Can machine learning systems explain their own reasoning to human overseers?
- Can better attention mechanisms close the gap between human and AI frame-activation?
- Why is metacognition neglected as a foundational AI research area?
- Can metacognitive categories be learned instead of fixed by human designers?
- Can AI-generated moves teach humans to think differently than memorization?
- What other hidden biases might aggregate metrics fail to distinguish from reasoning?
- How can benchmark accuracy scores mask the absence of interpretable reasoning structure?
- When does knowledge activation fail across different model architectures?
- Can latent reasoning architectures work as retrofits to existing models?
- Can models learn when to invoke search during reasoning tasks?
- What makes reasoning capability a pre-training rather than post-training phenomenon?
- Can activation steering directly steer models toward concise reasoning without prompting?
- Can targeted activation steering surface latent reasoning in base models?
- Why does inference-time thinking hurt proactive critical thinking in vanilla models?
- Can RL training teach models when to activate reasoning versus when to skip it?
- Can activation-space steering vectors replicate thinking model performance without retraining?
- Why does latent reasoning override no-think instructions in models?
- What other triggers can activate the latent reasoning capability?
- Does RL training actually restore the critical thinking that reasoning models lose?
- What makes thought identifiability provable without auxiliary training data?
- Do base models contain latent reasoning that minimal training can unlock?
- How does policy initialization with sub-policies enable emergent thinking?
- Can activation steering vectors compress reasoning without retraining models?
- Can pretraining signals unlock latent reasoning that post-training merely activates?
- Do base models truly possess latent reasoning capability?
- Does latent reasoning capability exist in base models before any training?
- What distinguishes reasoning activation mechanisms across different training methods?
- Can you steer reasoning by directly manipulating SAE features?
- Does RL training activate latent meta-learning capacity or create it from scratch?
- Can models reason at inference without specialized internal training?
- How much training data is truly necessary to unlock latent model reasoning?
- Can activation steering compress reasoning without retraining models?
- Does the base model already contain latent reasoning capability?
- Can distillation from stronger models create genuinely new reasoning abilities?
- Can models possess latent reasoning capability that training signals fail to unlock?
- How does the prefrontal cortex inspire artificial reasoning architectures?
- Why does pre-training provide the raw material for emergent thinking?
- Can we predict when a model will develop thinking behaviors?
- What mechanisms activate latent reasoning capabilities already present in base models?
- Can structured workflows unlock latent reasoning abilities that raw models don't show?
- Do base models already contain latent behavioral principles waiting to be amplified?
- Can minimal training signals unlock latent reasoning capability in base models?
- Can minimal training signals unlock reasoning already latent in pretrained representations?
- What latent reasoning capability do base models already possess before training?
- Can we steer model reasoning by manipulating single features?
- How do two-phase training dynamics explain reasoning emergence?
- What makes some reasoning strategies genuinely novel versus latent?
- Does base model geometry predict which associations persist through intervention?
- Can steering a single latent feature replicate chain-of-thought performance?
- What distinguishes genuine reasoning activation from memorization-assisted answer recall?
- Can marginal hints integrate better into reasoning than comprehensive explanations?
- Do explicit reasoning chains improve or harm performance on complex judgment tasks?
- Can latent reasoning in continuous space scale beyond supervised reasoning tasks?
- Why might latent reasoning capture types of thinking that verbalized CoT cannot?
- Why do models learn reasoning form instead of actual abstract inference?
- Do depth thresholds correspond to transitions between procedural and strategic learning?
- Are difficult tasks more monitorable because reasoning externalization becomes necessary?
- Does thought consolidation address the confirmatory reflection problem in reasoning models?
- Can scaffolding frameworks isolate inductive reasoning from deductive confounds?
- Can latent space represent reasoning dimensions that text cannot?
- Can continuous latent reasoning match discrete chain-of-thought without training modifications?
- Can reasoning evaluation metrics reward actual reasoning instead of theater?
- How do timing and search internalization interact during reasoning post-training?
- Can reasoning happen in latent space without chain of thought?
- Why does latent-level prediction beat token-level prediction for reasoning?
- Can latent reasoning scale test-time compute without verbal tokens?
- Can latent reasoning scale test-time compute without verbalized tokens or special training?
- Does the latent-explicit gap widen beyond 3B parameters on reasoning tasks?
- Can latent reasoning stay readable without explicit token-by-token decoding?
- How do depth and length constraints emerge in latent reasoning without explicit training?
- Why do richer mental representations sometimes fail to predict better outcomes?
- How do pretraining biases interact differently with prompts across model tiers?
- Can prompting inject new knowledge into already-trained AI models?
- Can users inject entirely new knowledge into models through prompting alone?
- Why does prompting discover capabilities that need reward-driven refinement?
- How does in-context learning trigger phase transitions in model behavior?
- When does knowledge distillation produce student models superior to teachers?
- How does subliminal learning differ from statistical model collapse?
- Why does adversarial training force deeper reasoning than surface imitation?
- Does importance sampling actually recover capabilities lost to hard sample training?
- What emergent behaviors do models develop when trained on underspecified pedagogical tasks?
- What makes some training data teach brittle answers versus robust reasoning?
- Do correlated human errors prevent models from transcending their training sources?
- Does the heuristic dominance ratio vary predictably across model architectures?
- How do foundation models develop task-specific heuristics instead of world models?
- Why do foundation models develop task-specific heuristics instead of causal understanding?
- Does the pretrained prior actually constrain what internalized search can discover?
- What is the relationship between reasoning depth and verbalization requirements?
- Can step-level deliberation flags guide other reasoning systems?
- Does explicit reasoning help or hurt tasks requiring continuous nuanced judgment?
- Can extended reasoning training capture individual strategic thinking styles?
- Do chain-of-thought explanations reveal genuine reasoning or trigger latent features?
- Are chain-of-thought traces anthropomorphizing how AI models really reason?
- Can a single model implement fast thinking, slow thinking, and tool use?
- What makes token-level reasoning during pretraining different from test-time chain-of-thought?
- Does reasoning require verbalization to be trainable and controllable?
- Does adding reasoning to models degrade other capabilities like rule inference?
- Does reasoning training create blind spots in premise detection?
- Do spurious rewards activate reasoning without teaching new skills?
- Why do spurious reward signals improve reasoning for some pretrained models?
- Can intrinsic confidence signals improve both calibration and reasoning performance?
- Does RL training redirect self-doubt into productive gap analysis?
- How can we turn reasoning model failures into useful training signals?
- Can RLVR expand a model's reasoning capabilities beyond its training ceiling?
- Why do current RLVR methods fail to expand reasoning capability beyond base model boundaries?
- How does non-reasoning SFT prevent overfitting before RL training begins?
- Can reinforcement learning add missing domain knowledge to fine-tuned reasoning models?
- Does reinforcement learning learn optimal per-turn reasoning discipline?
- How does RL refine reasoning paths without simply adding model capability?
- Why does combining reasoning distillation with RLVR outperform either training stage alone?
- How does the pretrained prior set a capability ceiling for reward model exploration?
- What limits RL's ability to scale for reasoning at training time?
- Can extended RL training unlock genuinely new reasoning strategies models cannot discover otherwise?
- Does reinforcement learning preserve reasoning quality better than supervised fine-tuning?
- Does RL teach models when to use reasoning or how to reason?
- How does the pretrained prior constrain the ceiling for empathy RL improvements?
- Can one training example activate mathematical reasoning in RL-trained models?
- Can reinforcement learning fix the reasoning gaps that supervised fine-tuning misses?
- Can one training example activate mathematical reasoning without reinforcement learning?
- Does reinforcement learning teach models how to reason or when to reason?
- Can reinforcement learning close the gap between LLM reasoning and action?
- What capacity threshold determines whether RL teaches activation versus shortcut learning?
- Why does extended reasoning training improve exploration without adding new capabilities?
- Can base models spontaneously produce reasoning traces without any RL training?
- Why does the pretrained prior determine the exploration ceiling?
- Can approximate or noisy reference answers work for RL-based reasoning training?
- Can RL create new reasoning primitives that pretraining never established?
- How do extrapolative and contextual generalization measure RL reasoning gains?
- When does reinforcement learning actually produce true reasoning gains in models?
- Does targeting the edge of competence during RL pretraining unlock true reasoning gains?
- Why do reasoning gains from RL require models trained with headroom and edge-of-competence data?
- Can reinforcement learning resolve the gap between knowing facts and knowing how to act?
- Can penalizing reasoning transitions fix underthinking without fine-tuning models?
- How does optimizing for accuracy during training degrade downstream reasoning quality?
- Why does reasoning fine-tuning reduce model abstention capacity by 24 percent?
- Does domain training degrade reasoning ability even when benchmark scores rise?
- Why does fine-tuning degrade reasoning quality even as accuracy improves?
- Why does domain accuracy improve while reasoning quality degrades after supervised fine-tuning?
- Does supervised fine-tuning improve accuracy while damaging the quality of reasoning?
- Does fine-tuning models for specific tasks destroy their ability to reason?
- Does reasoning fine-tuning actually reduce a model's ability to abstain?
- Do reasoning models trade instruction following for deliberative capability?
- How does behavioral fine-tuning differ from factual knowledge encoding in models?
- How does fine-tuning on natural language inference affect fallacy susceptibility?
- Do models trained for reasoning lose their ability to decline questions?
- What happens when reasoning fine-tuning eliminates model refusal mechanisms entirely?
- How does scaling reasoning capability actually reduce instruction-following ability?
- Why does supervised fine-tuning degrade reasoning quality despite raising accuracy?
- How do reasoning training methods sacrifice some thinking skills while improving others?
- Does reasoning fine-tuning actually damage a model's ability to abstain?
- Why do SFT models memorize patterns instead of learning generalizable reasoning?
- Why do instruction following and reasoning capability trade off in training?
- Does reasoning fine-tuning actually harm a model's ability to abstain?
- Can reasoning fine-tuning improve both capability and instruction compliance together?
- Why does reasoning fine-tuning reduce a model's ability to abstain?
- Does penalizing thought transitions improve reasoning without model retraining?
- Does reasoning training actively undermine the abstention capacity safety training created?
- Does supervised fine-tuning improve reasoning or just response formatting?
- Can training models on backward reasoning improve their forward planning ability?
- How does backward reasoning during training improve forward reasoning capability?
- Can conditioning generation on difficulty probes reduce overthinking on simple tasks?
- Why do reasoning-optimized models show no resistance advantage on agreement tasks?
- Why does reasoning fine-tuning reduce models' ability to abstain?
- How does preference learning differ from supervised finetuning for reasoning?
- Does fine-tuning improve domain accuracy at the cost of reasoning quality?
- Why do more capable reasoning models become harder to control by instruction?
- Does looped pretraining build reasoning more efficiently than supervised fine-tuning?
- Why does fine-tuning function as character training rather than capability training?
- Can AI systems execute strategies without conscious intention behind them?
- Can AI models learn tacit procedural knowledge that exists only in laboratory practice?
- Why does explicit theory injection work better than example-based learning for reasoning tasks?
- Can models learn to select exemplars based on reasoning skills rather than complexity?
- How much does pre-training frequency predict reasoning task performance?
- Does model scaling improve knowledge storage faster than reasoning ability?
- Can a single model trained on two tasks predict untrained decision tasks?
- Why does imitation learning create a ceiling for reasoning capability?
- What makes reasoning-specific post-training different from standard parameter scaling?
- Does specialized training in one domain create capability cliffs elsewhere?
- How does the functional separation of knowledge and reasoning affect adaptation methods?
- Why do recursive belief models require different training than logical derivation?
- Can models trained on longer contexts develop better fundamental reasoning abilities?
- Can training improve reasoning coherence without improving actual correctness?
- How does a single training example trigger phase transitions in reasoning output?
- Can contrastive learning teach models to switch between logical and emotional reasoning?
- Can reasoning catalyst data serve as a stable foundation for test-time training?
- How can one training example improve reasoning across thousands of unseen problems?
- Why are receiver attention heads narrower in reasoning models than base models?
- Why does reasoning training improve math but hurt knowledge tasks?
- How do single training examples activate reasoning capabilities in language models?
- Why does eliminating proxy-model filtering improve reasoning emergence in pretraining?
- Why do reasoning tasks improve more than retrieval from lookup memory?
- Can models maintain reasoning-output coupling while improving domain accuracy?
- Can smaller amounts of diverse reasoning demonstrations replace exhaustive factual training data?
- Does token-level reasoning during pretraining improve general reasoning without task-specific supervision?
- Why do knowledge and reasoning train in different network layers?
- Can models recover knowledge with completely unrelated retraining tasks?
- What kinds of reasoning tasks reveal the ceiling of text-only training?
- Can articulating latent reasoning processes improve transfer across domains?
- How do task frequency and complexity interact with model capacity during training?
- Does task diversity in pretraining data transfer reasoning better than larger models?
- Can small demonstration sets unlock general reasoning without large question data?
- How does question difficulty and breadth affect what models learn to reason?
- Which domains need knowledge injection versus reasoning-focused training?
- How do procedural versus factual knowledge differ in pretraining versus fine-tuning?
- Can curriculum learning by reward variance improve reasoning scalability?
- Why do reasoning gains resist clear attribution to specific training changes?
- How much of Occamy's result comes from training versus the base model?
- Why does reconstructing hidden thought processes during training improve cross-domain transfer?
- Can a single SAE feature control reasoning behavior across model families?
- How does factoring perception from reasoning improve sparse-label learning?
- Does representational density emerge from training data exposure during pretraining?
- How can interpretability methods account for shifting representational density across task conditions?
- Does latent density emerge during pretraining from training data familiarity?
- What makes bilevel metacognition architectural rather than emergent in current systems?
- What structural differences emerge between early generic skills and later meta-strategy skills?
- Why does decomposition ability transfer across domains but solving ability does not?
- How does early commitment in reasoning differ from early exploitation in planning?
- How does the generation-verification gap limit AI self-improvement capabilities?
- Does verification-conditioned use concentrate AI in tasks where juniors already have expertise?
- What makes active reasoning through dialogue harder than passive reasoning?
- Do different game types reveal different strategic reasoning capabilities in LLMs?
- How much do mechanistic interpretability findings reflect true reasoning architecture?
- How does LatentQA differ from predefined concept steering like representation engineering?
- Can knowledge encoded in model representations fail to influence generation?
- What happens to representational structure during model pretraining phases?
- Can probing reveal whether encoded facts actually influence model outputs?
- Where do humans and language models actually diverge in reasoning ability?
- Why do open-source models trained on proprietary outputs still fail at reasoning?
- Why do models automatically adjust reasoning length to problem difficulty?
- Does reasoning structure match explicit versus implicit task demands?
- Do base models and reasoning models fail in opposite directions on uncertainty?
- What explains the gap between perplexity performance and actual reasoning capability?
- Why do difficult problems force models to develop reasoning strategies?
- Can external classifiers reliably decide when a model should reason?
- Why does naive randomness fail to improve stochastic latent reasoning models?
- How do reasoning-related features behave when trained on near-impossible problems?
- What does pass@k reveal about base model reasoning capacity?
- Is reasoning failure caused by task complexity or training distribution gaps?
- How does contrapositive augmentation change the tractability of reasoning tasks?
- Why does structured stochasticity help reasoning more than naive randomness?
- How does learnability at the observer's current state prevent novelty from breaking model reasoning?
- Can activation patching reveal which reasoning steps actually matter?
- Can interventions on model components prove mechanism without explaining encoding?
- Can models be trained to hide causal influences in their explanations?
- How much does training data format shape what reasoning strategy emerges?
- Why does training format shape reasoning strategy more than domain?
- Why does training data format shape reasoning strategy more than domain content?
- Does training data format shape model reasoning more than domain content?
- What makes training data quality more important than quantity for reasoning?
- How does training data distribution create asymmetric competence across relation types?
- How does training format shape reasoning strategy more than content?
- How much does training data presentation format shape reasoning ability?
- How does training data format shape which reasoning patterns emerge in models?
- Why does training data format shape reasoning strategy more than content?
- Can training format itself shape what reasoning strategy a model learns?
- Does training data format shape reasoning strategy more than domain content?
- What pretraining formats encode latent reasoning strategies that RLVR can surface?
- How much does training data format influence reasoning strategy versus domain content?
- How does training data structure shape reasoning strategy more than domain content?
- How do critique models prevent policy entropy collapse during reasoning training?
- Does policy entropy collapse limit how many iterations of reasoning training work?
- How does policy entropy during training affect search discipline during inference?
- How does entropy-adaptive sharpening affect exploration in reasoning tasks?
- Why do models show performative reasoning on easy tasks but genuine reasoning on hard ones?
- Can models hide their reasoning in continuous space rather than natural language?
- Can models distinguish between activated knowledge and genuine reasoning?
- Can models learn when to think versus answer directly?
- What distinguishes coherent reasoning from inaccurate but plausible predictions?
- What metric distinguishes deep reasoning from superficial information propagation?
- Can thought quality alone be trusted to guide model training?
- Do reasoning models fail to report processes that actually influence their answers?
- Do models genuinely reason harder on difficult tasks or just appear to?
- Can activation probes detect reasoning that models omit from text?
- Can interpretability tools distinguish genuine reasoning from fabricated reasoning inside models?
- Does constraining AI access during early task phases preserve skill formation?
- Can personalized AI learning systems actually widen rather than narrow educational gaps?
- What counts as knowledge versus skilled performance in AI-mediated learning?
- Can energy minimization replace reasoning-specific reinforcement learning for system 2 thinking?
- How much reasoning depth do we actually need for most real-world tasks?
- Does more inference compute help reasoning models match specialized domain performance?
- Does inference-time compute improve pretraining data efficiency in practice?
- What inference-time scaling benefits emerge from reasoning before each prediction?
- Can reasoning models outperform non-reasoning models with more inference compute?
- Does more inference compute help close gaps between different training regimes?
- Can we detect and measure circuit formation before generalization emerges?
- How do knowledge and reasoning circuits interfere in the same neural network?
- How does mechanistic interpretability complement learning mechanics in explaining deep learning?
- How does representational density emerge from training data familiarity?
- Why does context information fail to override prior training associations?
- Does foundational model training or user priors more strongly shape final outputs?
- Why does monological training prevent models from overriding statistical priors?
- Does finetuning facts into weights overwrite existing model capabilities?
- Do emergent abilities result from genuine new capabilities or implicit in-context learning?
- What skills can large models identify and organize about their own abilities?
- Why does the gap between theoretical expressiveness and learned capability matter?
- Can models develop situational awareness without explicit training for it?
- What training interventions could close the perception-action gap?
- What role does pretraining play in distinguishing system capability from deployed behavior?
- Can models be trained to verify feasibility before proposing plans?
- How do capabilities-focused models exploit evaluation gaps?
- Do models use different reasoning standards for simulations versus real-world scenarios?
- Why do multi-year trials create inherent limits that model intelligence cannot overcome?
- Does synthetic fine-tuning create evaluation awareness similar to natural model reasoning?
- Will future training data teach models to connect awareness with gaming?
- Could future AI capability jumps change what counts as learnable?
- Why do human-curated thought examples fail to improve model thinking?
- Does reasoning trace style explain why RL post-training improves model reasoning?
- How does post-training on traces improve performance without semantic reasoning?
- Can models maintain auditable reasoning while achieving high accuracy?
- What distinguishes genuine capability gains from coherent but invalid reasoning traces?
- Why does a relativistic critic outperform absolute scoring in adversarial reasoning training?
- Can adversarial critics force genuine reasoning the same way critique fine-tuning does?
- How well do metagaming latents transfer across different evaluation tasks?
- Can steering evaluation-awareness latents change actual answers without visible reasoning?
- Do personality traits occupy specific mechanistic locations in pretrained models?
- Does pre-training encode personality patterns that fine-tuning later activates?
- Can fitted strategy models distinguish genuine mental-state representation from learned game policies?
- Why does reasoning effort fail to improve theory of mind performance?
- Does formal reasoning training actively degrade social reasoning ability?
- How do emotional and social simulations enable better hypothetical reasoning?
- Why does additional reasoning effort not improve theory of mind performance?
- Why does reasoning volume fail to improve theory of mind performance?
- Does reasoning effort correlate with social reasoning accuracy?
- Can structured theory of mind benchmarks measure genuine mental state reasoning?
- When does self-reflection actually help reasoning models improve?
- Can training on reasoning traces teach actual self-correction or only confident first answers?
- How does model capability relate to personality conditioning flexibility?
- What role does inductive bias play versus model capacity in practice?
- What separates knowledge from reasoning in neural network layers?
- What role does curriculum design play in reasoning emergence?
- How does o1-style reasoning relate to learned search processes versus memorized solutions?
- Can random rewards improve reasoning models if pretraining is suitable?
- Does process supervision recover reasoning accuracy better than outcome rewards in latent space?
- Why do binary reward tasks train better reasoning than judgment-based ones?
- How does pretrained knowledge constrain what adaptation strategies can achieve?
- What makes content informative and not-yet-mastered for reinforcement during pretraining?
- Why do pretrained model priors reduce the usefulness of retrieved experience?
- Why does a replay mechanism prevent reasoner skills from over-specializing?
- Why do reasoning-optimized models show no sycophancy resistance advantage?
- Can reasoning training fix sycophancy if it is not a reasoning failure?
- What role does self-learning play in improving agent reasoning without annotation?
- Does the 78-demonstration principle apply to other AI capabilities beyond agency?
- Does reasoning ability help agents learn from feedback faster?
- How much does extended thinking actually improve model reasoning ability?
- Can we improve reasoning by amplifying information at mutual information peaks?
- Why do different model training approaches produce different overthinking thresholds?
- Why do AI benchmarks measure accuracy instead of reasoning quality?
- Could deliberate training toward benchmarks overstate real gains in AI capability?
- How do induction heads learn to overwrite computational representations?
- What data properties enable transformers to learn sequential decision-making in context?
- Is premature decision-making a form of underthinking in transformer models?
- How sensitive is analogical reasoning emergence to training data and scale?
- Can goal information injected at inference time replace goal-conditioned training?
- Do base models or search agents win at open-ended prediction tasks?
- What role does verifier design play in reasoning capability gains?
- What makes the verifier the load-bearing component of reasoning training?
- Which specific capabilities must AI develop beyond current language model abilities?
- Can models generate intelligence or only reflect human discourse?
- How does AI system design amplify model capabilities beyond the weights?
- Does test-time scaffolding close capability gaps without retraining models?
Related concepts in this collection 16
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Concept map
Do base models already contain hidden reasoning …
Can simple rewards alone teach complex domain reas…
Does RL teach reasoning or just when to use it?
Can prompt optimization teach models knowledge the…
Can non-reasoning models catch up with more comput…
Can a single training example unlock mathematical …
Why do random rewards improve reasoning for some m…
Does RLVR actually expand what models can reason a…
Does procedural knowledge drive reasoning more tha…
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
your link
semantically near
linked from elsewhere
-
Can simple rewards alone teach complex domain reasoning?
Does reinforcement learning on difficult problems with basic accuracy rewards produce sophisticated reasoning strategies without explicit chain-of-thought training? This challenges assumptions about what domain AI models need to learn effectively.
partially contradicted: "emergence" may be reliable expression of latent capability, not creation
-
Does RL teach reasoning or just when to use it?
Does reinforcement learning in thinking models actually create new reasoning abilities, or does it simply teach existing capabilities when to activate? This matters for understanding where reasoning truly emerges.
mechanism: if base models have capability, RL teaches timing of deployment
-
Can prompt optimization teach models knowledge they lack?
Explores whether sophisticated prompting techniques can inject new domain knowledge into language models, or if they're limited to activating existing training knowledge.
extends to reasoning capability not just knowledge
-
Can non-reasoning models catch up with more compute?
Explores whether inference-time compute budget can close the performance gap between standard models and those trained for reasoning, and what training mechanisms might enable this.
qualified: targeted activation methods can close most of the gap
-
Can a single training example unlock mathematical reasoning?
Explores whether one example is enough to dramatically improve math problem-solving in language models, and whether learning continues after perfect memorization.
strongest evidence: one example activates 37-point gain with continued generalization
-
Why do random rewards improve reasoning for some models but not others?
When RLVR training uses meaningless reward signals, some models gain reasoning improvements while others don't. What determines which models can benefit from optimization pressure without meaningful feedback?
pretraining determines activation potential; reward signal is the catalyst, not the teacher
-
Does RLVR actually expand what models can reason about?
Explores whether reinforcement learning from verifiable rewards teaches models genuinely new reasoning skills or simply makes existing capabilities more reliable. Pass@k analysis suggests the latter.
pass@k confirms RLVR selects from existing capability, does not create new
-
Does procedural knowledge drive reasoning more than factual retrieval?
Explores whether models learn reasoning through general procedures across diverse documents rather than memorizing specific facts. This matters for understanding what pretraining data actually teaches models to reason.
identifies what the latent capability consists of: procedural knowledge synthesized from diverse pretraining documents that demonstrates how to reason, not what to recall; this is what minimal training signals activate
-
Can models learn when to think versus respond quickly?
Explores whether a single language model can adaptively choose between extended reasoning and direct responses based on task difficulty. This matters because it could make inference more efficient by allocating compute only when needed.
concrete implementation of the latent-capability thesis: Thinkless trains only a routing token via DeGRPO, not reasoning capability; the design premise is that capability is already present and what's needed is adaptive activation
-
Can models learn to internalize search algorithms through training?
Can chain-of-thought reasoning be taught as an explicit search process that models learn to implement internally? This matters because it could unlock algorithmic optimization rather than just output optimization.
extends beyond activation: Meta-CoT claims linearized search traces can teach genuinely new search capability, not just unlock existing patterns — testing the boundary of the latent-capability thesis
-
Does reinforcement learning on theory of mind collapse with model scale?
When RL improves social reasoning, does the quality of reasoning depend on model size? The question matters because accuracy alone may hide whether models are actually thinking or just pattern-matching.
the scale-dependent finding adds a social-reasoning dimension: 7B models have latent ToM capability that RL can activate, but smaller models lack sufficient latent capacity for social reasoning, suggesting a domain-specific threshold below which the latent-capability thesis does not hold
-
Does reinforcement learning update only a small fraction of parameters?
Investigating whether RL algorithms consistently modify only 5–30% of model parameters across different LLMs and RL methods, and what structural properties those sparse updates possess.
parametric signature of latent capability: RL touches only 5-30% of parameters because the rest already encode adequate reasoning; the sparsity is intrinsic and consistent across 7 algorithms and 10 models, confirming capability preexists in the weights
-
Can next-token prediction become a reasoning task with RL?
Does reinforcement learning applied to next-token prediction during pretraining encourage genuine reasoning rather than surface memorization? This matters because it could unlock reasoning capability without requiring labeled data or human feedback.
strengthens the foundation: RPT may create stronger latent capabilities than standard pretraining by embedding RL reasoning patterns during pretraining itself, making the subsequent minimal-signal activation even more effective
-
Can models improve themselves on tasks without verifiable answers?
Most self-improvement methods require verifiable correctness signals like math or code. Can models improve on open-ended instruction tasks where right answers aren't automatically checkable? And what minimal training is needed to unlock this?
extends the minimal-signal thesis to general instruction tasks: 1000 demonstrations of reasoning enrichment are sufficient to enable iterative self-improvement, consistent with the latent capability thesis — the catalyst teaches articulation of reasoning, not reasoning itself
-
Can careful selection of 78 demos outperform massive training datasets?
Does strategic curation of high-quality demonstrations unlock agentic capability more efficiently than scaling training data? LIMI achieved 73.5% on AgencyBench with 78 samples versus 10K+ samples for competing models, suggesting data quality may matter more than quantity.
extends the latent-capability thesis from reasoning to autonomous agency: 78 curated trajectories outperform 10K+ samples, suggesting agentic behavior is also a latent capability that minimal signals can activate
-
Can we trigger reasoning without explicit chain-of-thought prompts?
This research asks whether models possess latent reasoning capabilities that can be activated through direct feature steering, independent of chain-of-thought instructions. Understanding this matters for making reasoning more efficient and controllable.
most direct mechanistic evidence: single latent feature causally controls reasoning activation across 6 model families up to 70B
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Eliciting Reasoning in Language Models with Cognitive Tools
- Base Models Know How to Reason, Thinking Models Learn When
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models
- Large Language Models Think Too Fast To Explore Effectively
- Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
Original note title
base models already possess latent reasoning capability that minimal training signals can unlock