Theme of inquiry
Why does model confidence diverge from actual reasoning quality?
A question within its area, explored through 7 lines of inquiry below — each a family of specific questions the research asks.
39 specific questions
- What separates behavioral self-awareness from genuine introspective access in models?
- Does behavioral self-awareness depend on genuine introspection or statistical pattern matching?
- What separates behavioral self-awareness from genuine introspective capability?
- Could models use introspective awareness to detect and conceal their own misalignment?
- Why should we distrust model introspection as a transparency tool?
- What distinguishes performative self-reports from genuine introspective access in models?
- How misaligned are verbal reports from internal model computation?
28 specific questions
- Does training for persuasiveness harm a model's factual accuracy?
- Can post-training methods that increase persuasiveness also decrease factual accuracy?
- Why does AI persuasiveness increase while factual accuracy systematically decreases?
- Can post-training techniques create persuasive advantage where none existed?
- What mitigation frameworks exist for managing AI persuasion capabilities?
- Why do persuasive AI techniques also reduce factual accuracy?
- Can models become more convincing without becoming more correct?
54 specific questions
- How does self-revision in reasoning chains amplify confidence in wrong answers?
- Why does model self-revision increase confidence while degrading accuracy?
- Why does single-model self-revision amplify confidence in incorrect answers?
- How does self-revision on wrong answers increase model confidence further?
- Why do reasoning models struggle with self-evaluation and revision?
- Why do reasoning models amplify confidence in incorrect answers during self-revision?
- Why does self-reflection during training fail to improve model self-correction?
30 specific questions
- Do users track model confidence instead of actual accuracy?
- How should designers measure and explain semantic uncertainty to users?
- Why is confidence a dangerous proxy for accuracy in human-AI interaction?
- What happens when confident language masks uncertainty in AI outputs?
- Can organized response format trick users into overestimating AI reliability?
- How do confident system outputs weaken user skepticism about their reliability?
- Why do users trust overconfident AI outputs across different languages?
57 specific questions
- Does layer-wise prediction stabilization provide a stronger trace quality signal than confidence alone?
- Can uncertainty estimates based on model self-assessment reliably signal errors?
- Can intrinsic confidence signals improve both calibration and reasoning performance?
- Can confidence levels reliably detect when a model is overthinking?
- Why does convergence stability sometimes mislead about reasoning correctness?
- What role does confidence play in balancing overthinking versus underthinking?
- Does model confidence actually correlate with robustness against prompt variations?
28 specific questions
- Why does majority voting reward work better than other test-time aggregation methods?
- How does training-time voting differ from inference-time majority voting over samples?
- How does majority voting fail when reasoning samples lack genuine diversity?
- Does majority voting reliably signal correctness without risking reward hacking?
- Can test-time voting improve reasoning beyond the base model's original capabilities?
- When does multi-agent voting help versus hurt performance on tasks?
- How does training-time consensus differ from inference-time majority voting over samples?
44 specific questions
- Can models identify information gaps without just guessing or refusing to answer?
- How do reasoning improvements suppress a model's ability to abstain?
- Can models learn to stop thinking when a question lacks necessary information?
- Does reasoning fine-tuning actually harm a model's ability to abstain?
- Why does reasoning fine-tuning reduce models' ability to abstain?
- Why do reasoning models confidently generate wrong answers instead of abstaining?
- Does training for better reasoning reduce an AI system's ability to abstain?