Splitting an AI's plan from its actual execution: does watching both steps make oversight harder for humans, or does it actually help?

This explores whether splitting an AI's work into a visible plan and a separate execution step adds to the burden on the people reviewing it. The short answer: the corpus has no study that measures reviewer cognitive load directly. What it does have points the other way. When the split is built around how humans actually work, it seems to make oversight easier.

The closest evidence is Cocoa, a system that lets researchers assign each plan step to themselves or to an agent, and revise the plan partway through execution Can users steer AI agents better by revising plans mid-execution?. You might expect an extra layer of plan management to feel like more work. In a lab study (n=16), users found it easier to steer than a plain chat interface, ease of use stayed the same, and field users especially valued being able to delegate work explicitly. The sample is small, but the result matters: the split gave people a natural place to step in, rather than a stream of output to untangle after the fact.

There's also a reason from the machine side. Models that keep a separate 'decomposer' (which breaks the problem into steps) apart from a 'solver' (which carries them out) are more accurate, because the two jobs stop interfering with each other Does separating planning from execution improve reasoning accuracy?. The surprising part is that decomposition skill transfers across domains and solving skill doesn't. For a reviewer, that suggests the plan is the most portable and checkable part of the work. You can judge whether a plan makes sense without being an expert in every step.

A broader worry sits underneath the question. AI can separate a polished final product from the reasoning that produced it Does AI separate intellectual form from the thinking behind it?. A single block of finished output leaves a reviewer to reconstruct intent that was never shown. Making the plan explicit is one way to bring that reasoning back into view. Whether it lightens or adds to the load probably depends on design. Research on AI monitoring compares oversight strategies at equal review cost and equal false-alarm workload Does added monitoring improve protection at acceptable cost?, which reflects the view that reviewer attention is the scarce resource. That paper reports no results yet, though. Peer review offers a similar lesson: optional, gated AI feedback got 27% of reviewers to improve their reviews without swamping them Can LLM feedback help peer reviewers improve their own reviews?.

The takeaway you might not have expected: the real question isn't whether to separate planning from execution. It's whether the human gets to act at the plan stage, or only gets to inspect after everything has run. The evidence so far favors acting at the plan stage. A proper cognitive-load measurement is still a gap in this collection.