Line of inquiry
Inquiring lines›How do language models develop gen…›How do inference-time strategies a…›this line of inquiry
What makes reasoning models struggle with ill-posed and unfamiliar problems?
A broader line of inquiry — a family of 100 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 100
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Why do models overthink underspecified problems instead of rejecting them?
- Why do reasoning models wander instead of searching systematically?
- Is reasoning failure caused by task complexity or training distribution gaps?
- Do reasoning models overthink ill-posed questions instead of recognizing incompleteness?
- Why do reasoning models fail on structurally unfamiliar instances?
- What mechanisms cause reasoning models to wander rather than focus?
- Why do reasoning models fail at learning hidden rules from sparse exceptions?
- Do reasoning models switch approaches when encountering local difficulty?
- How does instance novelty rather than chain length explain reasoning failure?
- Can reasoning models succeed at logic but fail at execution?
- Can explicit optimal algorithms prevent reasoning model collapse at high complexity?
- Why do smaller models lose reasoning faithfulness more than larger models?
- Why do language models overthink simple questions when given extra time?
- Where do humans and language models actually diverge in reasoning ability?
- Why do non-reasoning models work better under extreme decomposition than reasoning models?
- Can models overthink and underthink at the same time?
- How do single wrong steps corrupt entire reasoning chains?
- Are reasoning models more vulnerable to persuasion than standard models?
- Why might rationales that predict common text patterns fail on hard novel reasoning?
- Why do models overthink easy problems and underthink difficult ones?
- Do base models and reasoning models fail in opposite directions on uncertainty?
- Does reasoning structure match explicit versus implicit task demands?
- Why do difficult problems force models to develop reasoning strategies?
- When is numeric computation the real bottleneck versus reasoning depth?
- Why do reasoning models fail when input length increases even below context limits?
- Can weak models reason better when freed from cognitive load by structure?
- Can reasoning models reject ill-posed questions or do they overthink?
- Why do language models struggle with backward reasoning compared to forward?
- Why do reasoning models confidently generate wrong answers instead of abstaining?
- Why do models automatically adjust reasoning length to problem difficulty?
- Why do language models struggle with formal logical reasoning and joins?
- Why do simple math problems get worse with longer reasoning chains?
- Why do structured reasoning representations sometimes reduce rather than improve error detection?
- Is the reasoning cliff actually a tool-use problem?
- How do reasoning-related features behave when trained on near-impossible problems?
- How does learnability at the observer's current state prevent novelty from breaking model reasoning?
- Can symbolic solvers rescue language models from logical reasoning failures?
- What explains the gap between perplexity performance and actual reasoning capability?
- What makes deductive reasoning so brittle in language models overall?
- Why does step-by-step reasoning fail when tool outputs get very large?
- Why do open-source models trained on proprietary outputs still fail at reasoning?
- Why do reasoning-optimized models still fall for logical fallacies in conversation?
- Can outcome-focused objectives explain failures in reasoning evaluation?
- Does premature confidence signal flawed reasoning in language models?
- Why do reasoning model failures stem from execution rather than reasoning?
- Can external classifiers reliably decide when a model should reason?
- Why does removing semantic content collapse reasoning in language models?
- Why does convergence stability sometimes mislead about reasoning correctness?
- Why does reasoning performance degrade as input length increases?
- What causes reasoning quality to degrade during long research tasks?
- Why do some reasoning models fail to detect redundancy in concurrent coordination?
- Can models distinguish between logical impossibility and their own execution limits?
- What makes a background condition relevant to a specific reasoning task?
- Do reasoning-enabled and prompt-hardened conditions show the same architectural penalty?
- Why does second-hop reasoning fail when composed with out-of-distribution triples?
- Do tool-use capabilities actually help models recover from errors in reasoning?
- Why does the Chinese Room argument miss the deeper abstraction problem?
- What limits external scaling when a model lacks reasoning foundation?
- How does the frame problem differ between symbolic and statistical reasoning systems?
- What design changes could make constraint inference more reliable without explicit cuing?
- What failure modes emerge when scheme classification feeds downstream reasoning pipelines?
- Why do reasoning models verbalize reasoning shortcuts less than necessary?
- Why does output alignment fail to catch internally incoherent reasoning?
- Why do epistemic failure modes cluster around world model limitations?
- How do failed branches remain in context and contaminate subsequent reasoning?
- What causes snowball errors to accumulate across reasoning steps in language models?
- Can problem structure and representation format be mismatched intentionally?
- Do tool-enabled reasoning models close the gap on constraint satisfaction?
- What role do cyclic fixed points play in stable reasoning?
- Can minimal adversarial triggers disrupt reasoning across multiple unrelated queries?
- Why do models detect false assumptions but still fail to correct them appropriately?
- Do reasoning failures stem from strategy or from calculation breakdown?
- How can minimal pairs expose reasoning failures that single-instance accuracy metrics miss?
- What implicit premises do language models skip even with correct surface reasoning?
- What triggers overthinking versus underthinking in reasoning models?
- Do sparse arithmetic circuits explain all language model reasoning abilities?
- Why do familiar patterns that support correct answers sometimes drive errors?
- What makes constraint satisfaction problems epistemically cleaner than other reasoning tasks?
- How do dependency errors propagate through incorrectly formalized definitions?
- What does pass@k reveal about base model reasoning capacity?
- How does reasoning instability prevent models from modeling individuals?
- Why does naive randomness fail to improve stochastic latent reasoning models?
- How do semantic failure modes map to attentional and intentional layers?
- How does contrapositive augmentation change the tractability of reasoning tasks?
- Why do harder puzzles cause all models to collapse despite larger token budgets?
- Why do invalid reasoning prompts work as well as valid ones?
- Can machine learning encode pragmatic reasoning about when rules should bend?
- Why do smaller LLMs fail at zero-shot argument scheme classification?
- Why do models commit to answers early on easy versus hard tasks?
- Which constraint types do reasoning models handle best?
- What makes Compound-QA expose weaknesses in monologue reasoning?
- Why does single-shot learning fail in REVTHINK's multi-source reasoning tasks?
- What makes financial reasoning particularly vulnerable to general PRM failures?
- Why does structured stochasticity help reasoning more than naive randomness?
- Why do even deterministic models show abrupt shifts across time?
- Why do unresolved items cluster in structured patterns rather than randomly?
- What tree depth is achievable before GPU memory becomes the bottleneck?
- What mechanism explains why context management prevents overflow failures most?
- How much does schema bloat actually degrade reasoning in large language models?
- What mechanism causes confident false answers under high cognitive load?