Theme of inquiry
How can systems understand user intent despite frequency bias?
A question within its area, explored through 5 lines of inquiry below — each a family of specific questions the research asks.
102 specific questions
- Does moving memory outside model weights avoid the limitations of in-weight retention?
- Can precomputed inferences be stored in memory modules between model interactions?
- Does including full context always degrade memory retrieval quality in practice?
- Do long-term memory modules outperform consolidation into fast weights?
- What persistent memory architectures best support storing precomputed inferences across sessions?
- How do adaptive memory modules compare to feedback-based working memory for long context?
- Do less technical users perceive memory inference differently than expert users do?
58 specific questions
- How do mechanistic features compare to natural language for interpretability?
- Can mechanistic interpretability findings guide practical interventions in model design?
- Why do feature visualizations alone fail to establish mechanistic claims?
- What makes representation engineering better than mechanistic interpretability for detecting hidden objectives?
- Can mechanistic interpretability explain explanation-execution disconnection?
- Does highlighting input features reduce human over-reliance on machine outputs?
- How much do mechanistic interpretability findings reflect true reasoning architecture?
45 specific questions
- Why do knowledge and reasoning train in different network layers?
- Why do higher network layers capture procedural knowledge but lower layers store facts?
- How do retrieval heads interact with layer-level separation of knowledge and reasoning?
- Why does knowledge storage separate from reasoning circuits in neural networks?
- How do procedural versus factual knowledge differ in pretraining versus fine-tuning?
- What separates knowledge from reasoning in neural network layers?
- Does model scaling improve knowledge storage faster than reasoning ability?
58 specific questions
- Why does attention quality degrade as context length increases?
- What makes sparse attention more reliable for long-context retrieval?
- Why do longer sequences tolerate higher sparsity than shorter ones?
- Can retrofitted sparse attention ever match jointly-trained sparse attention?
- Which attention heads are essential for maintaining factuality in sparse models?
- Does attention bias explain grounding failure in language models?
- Does attention linearity alone explain the efficiency gains over standard transformers?
40 specific questions
- Does transformer attention architecture systematically bias models toward sycophancy?
- Why do transformer attention patterns show positional and sequential bias across tasks?
- Why does transformer attention architecture reinforce sycophancy and agreement?
- How does transformer attention structurally bias models toward prominent and repeated content?
- Does transformer attention architecture inherently bias models toward sycophancy?
- What role does attention structure play in creating position bias?
- How does transformer attention bias toward repeated and context-prominent content?