Theme of inquiry
Why do model reasoning limitations persist despite increased capability?
A question within its area, explored through 8 lines of inquiry below — each a family of specific questions the research asks.
39 specific questions
- Why do correct reasoning traces tend to be shorter than incorrect ones?
- Do correct reasoning traces tend to be shorter than incorrect ones?
- Why are incorrect reasoning traces longer than correct ones?
- Why do correct reasoning traces in language models tend to be shorter?
- Why are correct reasoning traces consistently shorter than incorrect ones?
- Why do correct reasoning traces appear shorter than incorrect ones?
- Why are shorter reasoning traces more reliable than longer correct ones?
70 specific questions
- Can thinking token density explain reasoning performance beyond total length?
- Does thinking-token overuse actually degrade reasoning accuracy in practice?
- How do thinking tokens exhibit diminishing returns beyond a critical threshold?
- How does reasoning accuracy degrade when token budgets exceed critical thresholds?
- What happens to reasoning accuracy when models use more thinking tokens?
- Do tokens beyond a critical threshold actually improve reasoning quality?
- How much of a model's reasoning tokens are unnecessary for reaching the final answer?
50 specific questions
- Does longer reasoning always improve model accuracy on complex tasks?
- How does extended thinking affect variance in reasoning model outputs?
- How much does extended thinking actually improve model reasoning ability?
- Why does increased reasoning effort fail to improve accuracy consistently?
- When does extended thinking hurt performance on easier problems?
- Can extended thinking genuinely improve reasoning or just increase variance?
- Why does extended thinking increase output variance without improving reasoning quality?
146 specific questions
- Do reasoning models switch approaches when encountering local difficulty?
- How do reasoning improvements suppress a model's ability to abstain?
- Does reasoning fine-tuning actually harm a model's ability to abstain?
- Does reasoning fine-tuning actually reduce a model's ability to abstain?
- Why do models overthink underspecified problems instead of rejecting them?
- Does reasoning fine-tuning actually damage a model's ability to abstain?
- Why do reasoning models fail at learning hidden rules from sparse exceptions?
42 specific questions
- Why does additional reasoning effort not improve theory of mind performance?
- Can language models solve theory of mind benchmarks without genuine mental state reasoning?
- Why do reasoning models perform poorly at theory of mind tasks?
- Why does reasoning volume fail to improve theory of mind performance?
- Why do LLMs excel at reasoning tasks but show weaker theory of mind capabilities?
- Does scaling language models actually improve theory of mind reasoning abilities?
- Can language models develop genuine theory of mind or only surface strategies?
101 specific questions
- Does chain-of-thought reasoning cause model behavior or merely reflect it?
- Do chain-of-thought explanations reveal genuine reasoning or trigger latent features?
- Does chain of thought reasoning faithfully reflect what a model actually believes?
- Why does reasoning in chain of thought not match causal influence?
- Why do models rarely admit to their actual reasoning in chain-of-thought traces?
- Does chain-of-thought text causally drive reasoning or merely reflect it?
- Why do we measure reasoning quality by reading visible chains?
43 specific questions
- Do CoT-based monitoring systems degrade when models optimize against them?
- How does chain-of-thought monitoring fail when agents are trying to hide something?
- Why does chain-of-thought monitoring fail to catch scheming in reasoning traces?
- Can models deliberately obfuscate reasoning to defeat chain-of-thought safety monitors?
- Can chain of thought monitoring reliably catch model misbehavior?
- Does chain-of-thought monitoring fundamentally degrade under optimization pressure?
- Does optimizing against CoT monitors inevitably produce obfuscated reasoning?
80 specific questions
- Can reasoning traces prove models are actually reasoning versus mimicking?
- Why do reasoning traces fail to accurately reflect model decision-making?
- Can reasoning traces serve purposes beyond producing the final answer itself?
- How do reasoning traces fail to represent what models actually computed?
- Why do corrupted traces maintain performance as well as correct traces?
- What makes a reasoning trace causally sufficient versus merely stylistically plausible?
- Why do reasoning traces persuade users without improving their accuracy?