Can agent harnesses be automatically optimized across many environments?

SoL-Pi treats the agent harness, not the model or the serving stack, as the object of automated search. The authors scale "auto-research loops across increasingly numerous and diverse environments for harness rollouts" and report that four mechanisms "survive selection": action execution, context compaction, observation handling, and delegated reading. On the 51-task EdgeBench evaluation the assembled stack reaches performance comparable to the Pi harness on both GPT-5.6 Sol and Opus 5, while recorded token traffic falls by 44.7–49.0% and API cost by about one third. The best individual candidates are reported to improve model performance by 5.3–12.8% and token efficiency by 9.8–18.2%.

The paper's reasoning is that long unattended coding-agent trajectories make token use "a first-order systems concern," and that existing efficiency work (faster kernels, quantization, cheaper models) leaves the harness untouched. Harness-level optimization is framed as orthogonal to those, since it needs no additional model training. The scale argument is the distinctive part: harness artifacts found in one setting tend not to generalize, and the authors take their cross-model consistency as "preliminary evidence" that exposing the harness to many tasks and updating it from the resulting trajectories mitigates this. They call the long-term direction "pretraining the harness" and state it as a hypothesis, not a result.

Against the neighbors, this is a search-discovered counterpart to hand-designed harness efficiency. Can explicit behavior maps help weaker planners compete with stronger models? also cuts planner tokens through harness structure, but by a human-specified representation, where here the mechanisms are selected by the loop. It is consistent with Does raw token spending actually predict agent performance?: if raw tokens explain little of success, then removing nearly half of them at comparable performance is what that account would allow. It also gives Where does agent reliability actually come from? a cost-side datum, since the gains sit in the harness layer. Two model families are a thin test of the claim in Do stronger models always evolve harnesses better?, which is about who writes and who uses updates, and this excerpt does not vary either role.

The excerpt does not establish how selection works, how many environments or loop iterations were run, or what each of the four mechanisms does beyond its name. It gives no ablation showing which mechanism carries the savings, and it does not define "comparable" statistically. The dollar savings are described as "estimated." Generalization is claimed only "within the evaluated setting," so transfer to other benchmarks, harnesses or weaker models is untested here. The defensible reading is that a scaled harness search found a large token saving without a visible performance loss on one benchmark, not that harnesses can now be reliably pretrained.