Is AI reasoning thrown off by problems that are new, or only by ones too far beyond what it can learn?
How does learnability at the observer's current state prevent novelty from breaking model reasoning?
This explores whether reasoning breaks on new problems because they are new, or because they fall outside what the model can currently learn from, and how matching new material to the model's present ability keeps it from breaking.
This explores whether reasoning breaks on new problems because they are new, or because they fall outside what the model can currently learn from. The corpus points to the second. Novelty alone isn't the culprit. Novelty beyond the model's current reach is.
First, what 'breaking' looks like. Do language models fail at reasoning due to complexity or novelty? finds that reasoning models don't fail at a complexity threshold. They fail when a specific instance is unfamiliar. A long chain of reasoning succeeds if the model has seen similar instances, because it has fit instance-level patterns rather than general algorithms. The observer-relative part comes from What can a bounded observer actually learn from data?. It measures how much structure a computationally limited learner can extract from data, separating learnable regularity from what merely looks like noise to that learner. The same data can be rich structure to one observer and static to another. So 'how novel is this?' has no fixed answer until you say who is looking and when.
Training results show what happens on each side of that line. How does model ability change what samples teach? shows that what a sample teaches depends on its difficulty relative to the model's current ability. The productive band of medium-difficulty problems drifts as the model improves, so a static difficulty label goes stale within steps. Step outside the band on the hard side and it's worse than wasted. Do overly hard RLVR samples actually harm model capabilities? finds that nearly impossible problems teach degenerate shortcuts, like repeating answers and skipping computation. Group-relative scoring treats rare lucky successes as high-advantage, and those shortcuts then contaminate capabilities the model already had. That is the sense in which novelty breaks reasoning. Fall out of the band on the other side and the failure is quieter. Why do language models collapse into generic templates? shows that when every attempt on a prompt scores alike, the task signal fades and regularization pulls the policy toward generic, input-agnostic templates. Filtering to high-variance prompts recovers performance.
So what keeps a model in the band? Several notes point to the same habit: stay near what the model can already do. Does staying close to the base model preserve learning ability? finds that models kept close to their base distribution stay able to learn the next task, while parameter-only RL drifts further and stalls when the domain changes. Do base models already contain hidden reasoning ability? explains why nearby steps are cheap. Five different methods all elicit reasoning already latent in the base model, so post-training selects more than it creates. Can a model's own consensus replace ground truth labels? offers a label-free way to find the edge. It distills only on rollouts where the model disagrees with itself, using its own consensus as the teacher. Inconsistency marks ground that is learnable but not yet learned.
Two further points. Breakage is easy to miss. Can models be smart without organized internal structure? shows that models can reach perfect accuracy with fractured internal organization that only fails under perturbation or distribution shift, so a standard benchmark won't tell you whether novelty was absorbed cleanly. And when novelty is truly out of reach, the alternative to learning it is offloading it. Can a stronger model lift a weaker one at test time without retraining? shows a stronger model's harness nearly doubling a weaker model's Theory-of-Mind performance by moving unstable reasoning into deterministic code and task-specific routing. The corpus has no study that measures learnability step by step and tracks breakage directly. This account is assembled from separate findings that agree.
Sources 10 notes
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
Epiplexity formalizes the structural information a computationally bounded observer can extract from data, separating learnable regularity from time-bounded entropy. This task-free measure correlates with out-of-distribution generalization and explains why some datasets enable broader transfer than others.
A sample's learning value depends on the interaction between its difficulty and the model's current ability, not difficulty alone. The productive band of medium-difficulty problems drifts during training, making static difficulty estimates obsolete within steps.
Training on nearly-impossible problems causes models to learn degenerate shortcuts rather than genuine reasoning, and these shortcuts contaminate pre-existing capabilities. Group-relative normalization treats rare accidental successes as high-advantage trajectories, reinforcing answer repetition and computation-skipping instead of sound reasoning patterns.
When within-prompt reward variance is low, task gradients weaken and regularization dominates, pushing policies toward generic outputs. SNR-Aware Filtering—selecting high-variance prompts before updates—recovers performance across tasks and scales.
Show all 10 sources
FST-trained models stay up to 70% closer to their base distribution than parameter-only RL, and this reduced drift preserves the model's ability to learn subsequent tasks effectively. Parameter-only approaches stall when task domains change, while low KL drift enables sustained adaptation.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Unsupervised on-policy self-distillation using the model's own majority-vote consensus matched or surpassed supervised methods on five benchmarks. The key mechanism distills only on self-inconsistent rollouts, using agreement as the teaching signal rather than external labels.
Models trained with SGD can contain all the linearly decodable features needed for a task while maintaining fundamentally broken internal organization. This makes them vulnerable to perturbation and distribution shift invisible to standard evaluation metrics.
A stronger model built inference-time harnesses that nearly doubled weaker model performance on Theory-of-Mind benchmarks without retraining, primarily by moving unstable reasoning into deterministic code and task-specific routing rather than encouraging extended reasoning.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
- Mechanistically Interpreting the Role of Sample Difficulty in RLVR for LLMs
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
- Large Language Model Reasoning Failures
- The Invisible Leash: Why RLVR May Not Escape Its Origin
- Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?