Can separating generator and evaluator agents reduce self-praise bias?

Anthropic's Prithvi Rajasekaran, writing on the company's engineering blog, reports that separating the agent that produces work from the agent that judges it is "a strong lever" against a specific failure: agents asked to evaluate their own output "tend to respond by confidently praising the work—even when, to a human observer, the quality is obviously mediocre." The post states this bias "is particularly pronounced for subjective tasks like design," where there is no binary check equivalent to a verifiable software test, but adds that "even on tasks that do have verifiable outcomes, agents still sometimes exhibit poor judgment that impedes their performance."

The fix is not full elimination of leniency but a change in what is tractable to tune. Rajasekaran writes that "the separation doesn't immediately eliminate that leniency on its own; the evaluator is still an LLM that is inclined to be generous towards LLM-generated outputs." What changes is the engineering target: "tuning a standalone evaluator to be skeptical turns out to be far more tractable than making a generator critical of its own work, and once that external feedback exists, the generator has something concrete to iterate against." The design borrows its structure from Generative Adversarial Networks, pairing a generator with an evaluator in an iterative loop, with the evaluator's critique feeding back into the next generation attempt.

This frames the self-evaluation problem as a choice between two different places to put judgment: a second AI agent, or a human. Can orchestration layers make coding agents more auditable? keeps judgment and takeover points with a person and explicitly avoids adding another autonomous agent; Rajasekaran's harness goes the other way, delegating the critique itself to a dedicated AI evaluator and treating that delegation, not human oversight, as the lever against self-praise. Can language models build and maintain their own agent harnesses? argues more generally that a harness's design is separable from the model running inside it; this source is a concrete instance of that separation, in which the harness-design problem resolves into giving one agent a distinct role as skeptical judge, as built out further in Can subjective design judgment be turned into consistent grading criteria?.

The excerpt gives no measurement of how much the generator-evaluator split improves outcomes relative to self-critique, no sample size, and no comparison condition beyond Rajasekaran's own account of an earlier harness hitting "ceilings." It is a single engineer's design report on Anthropic's own blog, not an evaluated study, and the claim that a standalone evaluator is "far more tractable" to tune is asserted from experience rather than demonstrated with a controlled comparison. What follows at the strength of the evidence given is narrower: within this one build, splitting generation from evaluation was the design choice that let iteration continue past the point where asking a single agent to judge itself stalled.