As AI makes execution cheap, is the real skill shifting to judging what's worth doing and what counts as success?

This explores why better thinking tools seem to raise the value of human judgment instead of replacing it, and whether that effect really grows as the tools get more capable. The clearest framework in the collection comes from Agrawal, Gans and Goldfarb How do cognitive tools reshape the three types of worker skill?. They split worker skill into three parts. The first is implementation, which means actually carrying out the task. The second is opportunity judgment: spotting what's worth doing. The third is payoff judgment: knowing which outcome is better. Cognitive tools make implementation cheap, so the scarce inputs become knowing where to aim and recognizing a good result. One correction to the question's premise: the corpus doesn't show that tools complement *all* judgment more as they become sophisticated. Only payoff judgment complements tools reliably. The same framework explains why computers widened inequality while AI often narrows skill gaps inside firms. It's less a story about tool sophistication and more about which slice of skill the tool takes over.

There's a catch that keeps this from being a simple story of judgment winning. The same authors show that managers rationally hand decisions to machines even when their own judgment would be better Do managers delegate to machines even when human judgment is better?. Judgment takes deliberate effort, and attention is limited. So as tools get better at more things, judgment becomes more valuable per use, but people also spend less of it overall. Judgment ends up rationed toward the decisions where it matters most.

Other work suggests why judgment can't simply be handed over. AI can produce polished intellectual products without the reasoning and values that would normally stand behind them Does AI separate intellectual form from the thinking behind it?. A sophisticated tool makes it harder to tell good work from good-looking work, so someone still has to judge what the output is actually worth. Meanwhile, users tend to read fluent AI output as careful reasoning and to accept answers that confirm what they already believe Why do people trust AI outputs they shouldn't?. So the judgment a person brings is often doing the work they assume the tool is doing.

The most surprising parallel comes from inside AI systems themselves. When reasoning steps are packaged as separate "cognitive tools" that a model calls, GPT-4.1's math scores jump without any retraining Can modular cognitive tools unlock reasoning without training?. The tools don't add new ability. They bring out skill the model already had. Multi-agent brainstorming shows the same pattern: diverse teams of agents only beat a solo agent when the members have real domain expertise. Without it, they do worse Does cognitive diversity alone improve multi-agent ideation quality?. The common thread, for people and for models, is that tools multiply what's already there and don't supply it. That is likely the strongest reason tools complement judgment rather than replacing it.

One caveat: the collection has no study that tracks this complementarity as tools become more capable over time. The "more as they become sophisticated" part is a reasonable extrapolation from these mechanisms, not something the corpus shows directly.