Models don't 'grow into' analogical reasoning with scale — they borrow it from patterns in their training data.
How sensitive is analogical reasoning emergence to training data and scale?
This explores whether the analogical reasoning that shows up in transformers is a robust emergent capability or something tightly bound to what the model saw in training and how big it is.
This reads the question as asking whether analogical reasoning is a stable skill models 'grow into' with more data and parameters, or a fragile artifact of training coverage. The corpus leans hard toward the second answer — analogical and reasoning-like behavior is far more sensitive to *what's in the training distribution* than to raw scale. Mechanistically, transformers do have a real analogy circuit: they align relational structure geometrically in embedding space and then apply a learned functor to map one domain onto another How do transformers perform analogical reasoning across domains?. But that circuit is assembled from patterns seen in training, not conjured from size.
The sharpest evidence that data proximity dominates comes from work showing reasoning failures track *instance novelty*, not task complexity. Models don't hit a wall at some difficulty threshold — they succeed on any reasoning chain that resembles training instances and fail on unfamiliar ones, because they've fit instance-level patterns rather than a general algorithm Do language models fail at reasoning due to complexity or novelty?. Chain-of-thought reinforces the point: it degrades predictably once you shift task, length, or format away from the training distribution, producing fluent-but-invalid reasoning Does chain-of-thought reasoning actually generalize beyond training data?. That's the signature of imitation — reproducing familiar reasoning *schemata* — rather than genuine abstract inference Does chain-of-thought reasoning reveal genuine inference or pattern matching? Why does chain-of-thought reasoning fail in predictable ways?. Even the length of a reasoning trace turns out to encode 'how close is this to my training data' rather than 'how hard is this problem' Does longer reasoning actually mean harder problems?.
Here's the twist that reframes the scale question: scale may not be the lever people think it is. Base models already carry latent reasoning capability, and five independent methods — RL steering, critique fine-tuning, decoding tweaks, SAE feature steering, RLVR — all just *elicit* what's already there. The bottleneck is elicitation, not capability acquisition Do base models already contain hidden reasoning ability?. A 3B model with the right curriculum and multi-domain RL matches much larger systems on verifiable tasks, so reasoning competence is a property of the post-training pipeline, not parameter count Can small models match frontier reasoning without massive scale?.
So the honest answer is that 'emergence' is the wrong frame. Analogical reasoning doesn't switch on at some scale — it's latent structure that becomes usable when training data and post-training coverage align relational patterns the model can reuse. What you'd notice you didn't know to ask: the same data can produce very different reasoning behavior depending on the verifier, optimizer, scaffold, and budget wrapped around it — the reusable unit isn't a dataset but the whole feedback interface What is the actual reusable unit of reasoning data?. And you can push reasoning *earlier* by treating chain-of-thought as an exploratory action rewarded during pretraining itself, which suggests where in the pipeline analogical structure gets planted is a design choice, not a fixed function of scale Can chain-of-thought reasoning be learned during pretraining itself?.
Sources 10 notes
Mechanistic analysis reveals transformers perform analogical reasoning via two stages: geometric alignment of relational structure in embedding space, followed by learned functor application. This signature appears in both synthetic tasks and pretrained LLMs.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
DataAlchemy experiments show CoT fails systematically under distributional shifts in task, length, and format. Models produce fluent but logically inconsistent reasoning — imitating reasoning form without valid underlying logic.
CoT works by constraining models to reproduce familiar reasoning patterns from training, not by enabling novel symbolic reasoning. Performance degrades predictably under distribution shifts—the signature of imitation rather than capability emergence.
CoT guides models to pattern-match reasoning structure rather than perform genuine inference. This explains distribution-bounded failures, why structural coherence matters more than content correctness, and why performance optimizes against interpretability.
Show all 10 sources
Controlled A* maze experiments show trace length correlates with difficulty only in-distribution but decouples entirely out-of-distribution. Trace length primarily reflects recall of training schemas, not adaptive computation.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
A 3B model trained with curriculum SFT and multi-domain RL reaches 94.3 AIME26 and 80.2 LiveCodeBench scores matching much larger systems. The result is bounded to verifiable tasks with checkable ground truth, where RL can provide clean reward signals.
The reusable unit in post-training is a feedback interface entangled with six factors: verifier, base model, lineage, optimizer, scaffold, and budget. Changing any one alters the same data's effect, making attribution tractable only when these are jointly released.
RLP treats CoT as exploratory action during pretraining, using log-likelihood improvement as verifier-free reward. Applied to Qwen3-1.7B and Nemotron-Nano-12B, the method improves math and science benchmarks substantially, suggesting reasoning can be planted earlier in training.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- CoT is Not True Reasoning, It Is Just a Tight Constraint to Imitate: A Theory Perspective
- Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning
- Hierarchical Reasoning Model
- Break the Chain: Large Language Models Can be Shortcut Reasoners
- Eliciting Reasoning in Language Models with Cognitive Tools
- A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap