Does AI reasoning get stuck in a narrow lane equally in medicine, math proofs, and everyday tasks — or unevenly?
Does epistemic narrowness appear equally across professions, proofs, and other reasoning tasks?
This explores whether AI reasoning stays confined to a narrow band of what it has seen or can check, and whether that confinement looks the same in professional fields like medicine, in formal proofs and math, and in other reasoning tasks.
This explores whether AI reasoning stays confined to a narrow band of what it has seen or can check, and whether that looks the same across professions, proofs and other tasks. The corpus has no note that measures this side by side under that name. The closest evidence points to no: the narrowness is uneven, and it follows how checkable a domain is and how much it leans on facts rather than procedures.
Math and code are where reasoning looks broadest, and that's partly because they're easy to grade. A 3B model trained with a well-designed post-training pipeline matches much larger systems on AIME and LiveCodeBench, but the result is explicitly frontier-reasoning-is-a-property-of-the-post-training-pipeline-not-the-parameter|bounded to tasks with checkable ground truth, where reinforcement learning gets clean reward signals. Professions rarely offer that. Other work finds that what limits reasoning beyond math and code is scaling-reasoning-beyond-math-and-code-is-gated-by-question-diversity-not-method|question diversity, not the training method. When models are trained on millions of varied, hard questions, reasoning transfers into economics and the social sciences. Part of the narrowness in professional domains therefore comes from what the models were trained on, and it can shrink.
Medicine shows a different kind of narrowness. One study locates knowledge-resides-in-lower-network-layers-and-reasoning-in-higher-layers-this-fu|knowledge in lower network layers and reasoning in higher ones. That would explain why reasoning training helps math but can hurt knowledge-heavy fields like medicine. Training that widens one domain can narrow another. A companion finding fits this. Reasoning draws on procedural-knowledge-in-pretraining-documents-drives-reasoning-generalization-un|broad, transferable procedural knowledge, while factual recall depends on narrow, document-specific memorization. This is my inference rather than something the papers test directly. Proofs are mostly procedure, so they may generalize well. Professions depend heavily on specific facts, so they would stay narrow.
The narrowness also sits below the level of a whole domain. Reasoning failures track lrm-reasoning-breakdown-is-driven-by-instance-level-unfamiliarity-not-task-level|instance-level unfamiliarity, not task complexity. A model can look general on proofs while only covering problems close to ones it has seen. Some apparent success is also hollow. Twelve of fourteen models get worse when constraints are removed, because they conservative-bias-hides-behind-apparent-reasoning-success-most-models-perform-wo|default to the harder, more conservative option instead of reasoning about the constraints. Narrowness can hide behind correct answers in any domain.
One thing does appear roughly equally. Accuracy reasoning-performance-degrades-with-input-length-even-far-below-context-window-l|drops sharply with input length even far below the context limit, and the drop doesn't depend on the task. So the corpus suggests three patterns. Some limits are uniform, such as input length. Some are graded by domain, such as verifiability and knowledge versus procedure. Some are hidden inside apparently competent performance. What the collection lacks is a direct comparison of specific professions, such as law, clinical practice or accounting, against proofs.
Sources 7 notes
A 3B model trained with curriculum SFT and multi-domain RL reaches 94.3 AIME26 and 80.2 LiveCodeBench scores matching much larger systems. The result is bounded to verifiable tasks with checkable ground truth, where RL can provide clean reward signals.
Reasoning scales in open-ended domains when trained on diverse, difficult questions rather than better algorithms. NaturalReasoning's 2.8M curated questions show distillation and self-training transfer reasoning capability across STEM, economics, and social sciences.
Two-phase inference model shows knowledge retrieval operates in lower network layers while reasoning adjustment happens in higher layers. This separation explains why reasoning training improves math but can degrade knowledge-intensive domains like medicine.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
Show all 7 sources
Twelve of fourteen models perform worse when constraints are removed, dropping up to 38.5 percentage points. Models appear to reason correctly by defaulting to harder options, not by actually evaluating constraints.
FLenQA shows reasoning accuracy drops from 92% to 68% at just 3000 tokens of padding, far below context window capacity. The degradation is task-agnostic, uncorrelated with language modeling performance, and persists even with chain-of-thought prompting.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
- On the Reasoning Capacity of AI Models and How to Quantify It
- A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap
- The Invisible Leash: Why RLVR May Not Escape Its Origin
- Eliciting Reasoning in Language Models with Cognitive Tools