Theme of inquiry
What mechanisms enable neural systems to overcome inherent biases?
A question within its area, explored through 5 lines of inquiry below — each a family of specific questions the research asks.
73 specific questions
- Do long-term memory modules outperform consolidation into fast weights?
- Does moving memory outside model weights avoid the limitations of in-weight retention?
- Can precomputed inferences be stored in memory modules between model interactions?
- Why does specializing to one task make future task learning harder?
- Can data pruning strategies exploit the finite nature of memorization capacity?
- Why does fine-tuning for continuous space cause catastrophic forgetting?
- Why do accumulated memory systems sometimes hurt continual learning?
28 specific questions
- How do retrieval heads achieve sparse attention naturally in transformers?
- How do attention heads separate text retrieval from internal thought representation?
- How do attention patterns and circuits function as algorithmic representations?
- Does transformer attention architecture fundamentally prevent topic-aware memory?
- What computation remains in the attention heads that programs cannot capture?
- Why are receiver attention heads narrower in reasoning models than base models?
- Does attention linearity alone explain the efficiency gains over standard transformers?
19 specific questions
- How can frame sampling and ranking improve temporal understanding in long-video retrieval?
- Can parsing screens into structured elements before acting improve vision models?
- Can temporal ranking improve retrieval without modifying the underlying video model?
- Why do image captions create different friction than pure video data?
- Can text-based and vision-based screen understanding achieve similar performance?
- How can affordance become a primary retrieval signal instead of a filter?
- How does annotation-based pretraining compare to self-supervised video masking for screen understanding?
32 specific questions
- Does transformer attention architecture systematically bias models toward sycophancy?
- How does transformer attention structurally bias models toward prominent and repeated content?
- Why does transformer attention architecture reinforce sycophancy and agreement?
- Why do transformer attention patterns show positional and sequential bias across tasks?
- How does transformer attention bias toward repeated and context-prominent content?
- What role does attention structure play in creating position bias?
- Does transformer attention architecture inherently bias models toward sycophancy?
28 specific questions
- Can fixing hallucination address AI's structural epistemic problem?
- Does inevitable LLM hallucination make detection metric validity critical?
- How do external safeguards like retrieval augmentation prevent hallucination?
- Why do language models hallucinate even with perfect training?
- Can architectural changes reduce hallucination without external retrieval or verification?
- Why do hallucination rates differ between vendor AI products and student-used models?
- Why does model confidence fail to detect hallucinations on rare entity pairs?