Theme of inquiry
How do training methods and scaling affect model behavior?
A question within its area, explored through 3 lines of inquiry below — each a family of specific questions the research asks.
72 specific questions
- Does fine-tuning actually change model capabilities or only output distribution?
- How do finetuning and pretraining improvements differ in their effects on model capabilities?
- Does specialized training in one domain create capability cliffs elsewhere?
- How do retrieval and fine-tuning trade off flexibility against training cost?
- How much can externalized skills improve models before hitting diminishing returns?
- Why do proprietary models improve with training while open-source models decline?
- What performance trade-offs emerge when composing multiple independently trained model capabilities?
64 specific questions
- Does harness optimization generalize across different benchmarks and agent architectures?
- How do evolved harness edits generalize across different benchmark domains?
- Can mid-tier models benefit more from harness improvements than frontier models?
- Why do mid-tier models benefit most from memorized harness fixes?
- Does harness scaling represent a fundamentally different path than model scaling?
- Why do evolved harnesses often fail to generalize beyond their training tasks?
- Can harness updates benefit agents equally across all model sizes?
30 specific questions
- Why must world models be nested rather than flat and uniform?
- How do world models decompose between representation of facts versus generative mechanisms?
- Can a world model have rich representations without adequate data coverage?
- What distinguishes task-specific heuristics from genuine world models?
- Can world models simulate actionable possibilities instead of just predicting next states?
- Why do foundation models develop heuristics instead of world models?
- Why does integrating world models with decision-making systems matter?