Can a cheaper AI catch up to a smarter one just by getting better tools and organization around it?
Can weaker models match stronger ones by reorganizing harness-side components?
This explores whether a weaker model can close the gap with a stronger one not by changing its weights, but by changing the scaffolding around it: how the harness is organized, what code handles the work, and what checks sit around the model.
This explores whether a weaker model can close the gap with a stronger one by changing the scaffolding around it (the harness) instead of its weights. The corpus says yes, sometimes. The best evidence is narrow but concrete, and it comes with conditions.
The most direct case is code localization. When a harness repository was reorganized around runtime behavior, with an explicit map from behaviors to the code that implements them, weaker planners matched stronger models on code localization across every precision and recall metric. Win rates rose 10–19 points while planner token use fell 8–13% (Can explicit behavior maps help weaker planners compete with stronger models?). The weaker model didn't get smarter. The map did some of the searching for it.
A related result points the same way. A stronger model built inference-time harnesses that nearly doubled a weaker model's score on Theory-of-Mind benchmarks. The gains came mostly from moving unstable reasoning into deterministic code and routing each task type to the right handler, not from asking the model to think longer (Can a stronger model lift a weaker one at test time without retraining?). Harness design can also carry over across models. One execution-system runbook lifted several fixed models on Terminal-Bench 2.1, and it transferred to newer models unchanged (Can execution harnesses lift model performance without retuning weights?). So the pattern is to take fragile work off the model and give it to structure.
The corpus also shows where this stops working. A committee of weak-model calls matches a strong model only when there is an external soundness signal, such as tests, proofs, or type checks. Sampling more produces more correct candidates but can't pick the right one (When can weak models match strong model performance?). Reorganizing components only helps when something in the harness can tell good output from bad.
The weakest models may not benefit at all. Producing useful harness edits looks about equally hard for every model tier, but benefiting from them peaks in mid-tier models. Weak models often fail to invoke the harness, and strong models struggle to follow its instructions faithfully (Do stronger models always evolve harnesses better?). Improvements to individual harness modules can also be learned separately from any one benchmark. Modules evolved on data disjoint from the benchmarks gave gains on unseen tasks and domains (Can harness modules improve separately from benchmark data?).
Putting these together, a weaker model can match a stronger one when the harness supplies what the weaker model lacks: a behavior map, deterministic code for brittle reasoning, or a verifier. Reorganizing alone won't do it. The corpus has no evidence that harness changes close every gap, and the gains it documents are task-specific, in localization, Theory-of-Mind, and verifiable coding. The weakest tier may also be unable to use the harness in the first place.
Sources 6 notes
A behavior-to-code mapping representation improved win rates by 10–19 points while reducing planner tokens by 8–13%. Weaker planners using this mapping matched stronger models' code localization across all precision and recall metrics.
A stronger model built inference-time harnesses that nearly doubled weaker model performance on Theory-of-Mind benchmarks without retraining, primarily by moving unstable reasoning into deterministic code and task-specific routing rather than encouraging extended reasoning.
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
Sampling alone amplifies coverage but cannot select correct solutions. Reliable performance matching requires external soundness signals—tests, proofs, or type checks—that convert latent correct proposals into actual selections.
Model capability to produce useful harness edits stays constant across tiers, but capacity to actually benefit from those edits follows an inverted U-shape, peaking in mid-tier models. Weak models fail to invoke harnesses; strong models struggle with faithful instruction-following.
Show all 6 sources
ModularRSI evolves harness modules independently using contrastive trajectories on benchmark-disjoint data, showing consistent gains across unseen tasks and domains. The approach isolates mechanism-level improvements from task-specific adaptation by aggregating evidence across tasks before updating components.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- Rethinking the Evaluation of Harness Evolution for Agents
- Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
- StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
- HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?
- Scaling Laws for Agent Harnesses via Effective Feedback Compute
- DarwinX: Evolving Agent Harnesses Through Natural Selection
- RRSI: Regularized Recursive Self-Improvement of Agent Harnesses