Does breaking AI work into small, checkable steps erase the gap between weak and strong models — while messy, open tasks make it worse?

This explores whether tight, checkable tasks let weaker AI setups catch up, while open-ended work makes the differences bigger. No single paper in the corpus tests this directly. Several findings point the same way, though, with one important exception.

On the closing side, bounding a task can be enough to make raw model strength matter less. MAKER breaks million-step tasks into tiny subtasks and votes at each step. When the breakdown goes far enough, small non-reasoning models can finish with zero errors, which reverses the usual habit of sending a bigger model at a harder problem Can extreme task decomposition enable reliable execution at million-step scale?. The same thing happens with the scaffolding around a model. Improving the execution harness (how the model runs commands, manages context and reads results) lifts frozen models on a terminal benchmark, and the same playbook carries over to newer models without changes Can execution harnesses lift model performance without retuning weights?. Automated harness tuning across many environments roughly halves token use at the same performance Can agent harnesses be automatically optimized across many environments?. When the task has a clear finish line, much of the gap turns out to be about engineering, not intelligence.

On the widening side, the problem starts with measurement. Automated benchmarks favour tasks that are precisely specified and easy to grade automatically. As a result they both overstate and understate what frontier systems can do. Long, messy, real-world evaluations tell a different story Do automated benchmarks hide what frontier AI systems can really do?. Training shows the same split. Training on structured domains like math makes a model's outputs less varied, while creative domains make them more varied. Training on structured work first protects open-ended skills from being flattened, worth a 6.2% gain Does training order reshape how models handle different task types?. Post-training also charges a hidden fee. It makes models sharper on easy cases but removes rare solutions they could otherwise reach, so base models eventually find more solutions when given many attempts Do base models find more solutions than post-trained ones?. Even instruction tuning may teach models what an answer looks like more than what the task means Does instruction tuning teach task understanding or output format?. In short, optimizing for bounded success can quietly cost open-ended ability.

Open tasks also expose a specification gap. Richard Socher argues that reward hacking continues because AI optimizes what was said, not what was meant Why do AIs keep gaming rewards instead of serving intent?. The looser the task, the bigger the space between the instruction and the intent. A related finding: models stuck on a numerical score can get past it once they receive written critiques explaining why they failed Can natural language feedback overcome numerical reward plateaus?. A single score works for a bounded task. Open progress needs richer feedback.

The exception is that some gaps don't close even when the task is bounded. Non-reasoning models never match reasoning models, however much compute they get at answer time, because training is what makes extra thinking useful Can non-reasoning models catch up with more compute?. So the more accurate version of the question's idea is this: bounding a task lets scaffolding and task breakdown stand in for model strength, but it can't replace capabilities that were never trained in. The surprise is that the effort to close gaps on bounded tasks may itself be widening the open-ended ones.