Line of inquiry
Inquiring lines›How do training data and architect…›What does iterative improvement ac…›this line of inquiry
Can harness optimization rival model scaling in effectiveness?
A broader line of inquiry — a family of 81 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 81
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Does harness optimization generalize across different benchmarks and agent architectures?
- Does harness scaling represent a fundamentally different path than model scaling?
- Do harness-driven accuracy gains appear on other benchmarks?
- Can mid-tier models benefit more from harness improvements than frontier models?
- How do agentic systems hide harness failures from benchmarks?
- How do evolved harness edits generalize across different benchmark domains?
- Can better agent harnesses solve problems that model scale alone cannot?
- Can harness updates benefit agents equally across all model sizes?
- Can weaker models benefit equally from harness updates as stronger ones?
- How much does harness design contribute to reported model capability scores?
- Why do mid-tier models benefit most from memorized harness fixes?
- How do prompt optimization and code harnesses compare for capability transfer?
- Can mid-tier models benefit more from self-generated harness updates than others?
- Can a harness design be optimized independently of the model inside it?
- Why do evolved harnesses often fail to generalize beyond their training tasks?
- What components of agent scaffolding most impact domain-specific output quality?
- Can individual harness components improve accuracy on specific benchmarks?
- Do evolved harness edits capture reusable strategies or task-specific memorization?
- How much realized agent capability comes from the harness versus the model?
- Can harnesses that rewrite themselves through reviewed commits achieve state-of-the-art performance?
- Why do research agents optimize harness mechanisms over autonomous weight scaling?
- Can harness edits overfit to training tasks while appearing to improve performance?
- Why do useful harness updates often disappear during model evolution?
- How do model tier and harness quality interact in agent self-improvement?
- How do different harness designs produce different agent behaviors from the same model?
- Would frontier closed models show the same harness indifference?
- How does harness structure affect planner token efficiency compared to model size?
- Do harness improvements eventually internalize into core model behavior over time?
- Can harness edits distill reusable strategies or mostly memorize task-specific fixes?
- How much does harness design separate from model intelligence affect benchmark scores?
- Can weaker models match stronger ones by reorganizing harness-side components?
- What persistent failures remain unsolved despite harness evolution efforts?
- Do evolved harness edits learn reusable strategies or just memorize task-specific fixes?
- How does externalizing reasoning into harness artifacts improve agent reliability?
- What cognitive burdens should move from model parameters into harness infrastructure?
- How do scaffolding improvements change agent capability across different task types?
- Can harness evolution be redirected toward distilling transferable procedures instead?
- Why do evolved harness edits mostly memorize rather than generalize?
- Do models co-adapt their harnesses to specific executor strengths?
- What happens when different harnesses project the same model?
- How much token efficiency can harness-level intervention achieve versus training approaches?
- Why do mid-tier models benefit more from memorized harness shortcuts?
- Why does the harness layer accumulate distributed behaviors over time?
- Why do vendor harness accuracy claims lack rerun verification?
- What causes weak models to fail at activating harness artifacts?
- How much does scaffold choice interact with model selection in agents?
- Can runtime behavior mapping help localize harness deficiencies?
- Can harness edits trained on one batch transfer to new tasks?
- Can model training address failures that really originate in harness gaps?
- What makes behavior localization the bottleneck in agent harness evolution?
- What role does effective feedback compute play in agent harness scaling?
- What makes harnesses more tangled than other types of agent code?
- Do gains from harness-based agents transfer across different search benchmarks?
- How does editing the harness layer differ from updating model weights?
- Which domains see models exceed human harness design quality?
- Does test-time scaffolding close capability gaps without retraining models?
- What distinguishes the fast scaffold learning loop from parametric model weight updates?
- What makes a harness low-friction for model strategy?
- What makes a harness a first-class object rather than invisible scaffolding?
- How much of AI improvement comes from tools versus model capability?
- Why does harness benefit capacity peak at mid-tier models, not frontier scale?
- How does AI system design amplify model capabilities beyond the weights?
- What task set size resolves a real 5-point harness difference reliably?
- How should harness scaffolding be treated as a first-class object?
- Does harness benefit depend on which model tier you use?
- Do shell-state handling and answer aggregation fixes transfer across different models unchanged?
- How much does rerunning the same harness change SWE-bench task outcomes?
- What makes skills worth externalizing into a persistent harness?
- Which foundation model tiers most benefit from harness updates?
- How can harnesses externalize bookkeeping so models focus on semantic judgment?
- How should we allocate model budget between evolvers and harness users?
- How does cost stability in harness swaps compare to accuracy stability?
- What happens when you project the same model onto different harnesses?
- Do cached input prices change which harness costs less?
- What makes API-based scaffolding more trustworthy than direct model access in high-stakes domains?
- Can scaffold-only modifications achieve lasting gains without updating the foundation model weights?
- What explains the threefold cost differences between harnesses?
- How much does executor choice change a harness's actual performance?
- Are SchemeArena's scenario factors fully crossed to separate bundled changes?
- What should an external contract for model improvement actually contain?
- How much external enforcement does each model need before utility drops?