When one AI builds something and another judges it, does that setup actually learn the way GAN image generators do?
How does the generator-evaluator split function like a generative adversarial network?
This looks at what happens when you give one AI agent the job of making something and a separate one the job of judging it, and asks how far that setup really borrows from GANs, the classic design where a generator network tries to fool a discriminator network.
This looks at the 'one model makes, another model judges' setup used in agent systems, and asks whether it actually works like a generative adversarial network (GAN). In a GAN, a generator and a discriminator are trained against each other. The generator gets better because the discriminator keeps getting harder to fool. The corpus suggests the agent version keeps the structure of a GAN but changes how it gets better. No weights are updated. The generator improves because the evaluator's written critique feeds back into the next attempt, over and over, while the system is running.
The clearest example is Anthropic's planner-generator-evaluator harness, which is openly modeled on GANs Can subjective design judgment be turned into consistent grading criteria?. The evaluator does more than say 'real or fake.' It turns a fuzzy question like 'is this design beautiful?' into specific criteria it can grade the same way every time. It uses example grades to keep its scores from drifting, and it penalizes generic output. That penalty does what the discriminator does in a GAN: it pushes the generator away from safe, average output. The harness can loop for up to four hours on a single build, using a live browser. The reason for splitting the roles at all is practical. A model asked to grade its own work tends to praise itself, especially on subjective tasks. It turned out to be much easier to tune a separate, skeptical judge than to make the generator critical of itself Can separating generator and evaluator agents reduce self-praise bias?. The 'adversary' is there mainly to stop the system from flattering itself.
The idea of pairing a generator with a judge shows up elsewhere in the corpus without the GAN label. The Consensus Game treats decoding as a game between a generator and a discriminator that have to agree on an answer. Finding the point where both are satisfied let a 7B model match a 540B one with no fine-tuning Can generative and discriminative models reach agreement?. That is the cooperative version of a GAN: the two sides aim for agreement instead of trying to beat each other. The evaluator side is also changing. Process reward models that reason step by step before giving a verdict beat models that just output a score, using a small fraction of the labeled data Can generative reasoning beat discriminative models with less training data?. So the best judges are turning into generators themselves, which blurs the line a GAN depends on.
There is also a pull in the other direction. Post-Completion Learning trains a model to grade its own output in the unused space after its answer ends, which puts the evaluator back inside the generator at zero inference cost Can models learn to evaluate their own work during training?. Whether that brings back the self-praise problem is an open question the corpus doesn't settle.
The less obvious lesson is about what happens when the evaluator's verdicts start feeding a system's memory and not just its next draft. Training on unfiltered AI output wears away rare and unusual patterns, generation after generation Does training on AI-generated content permanently degrade model quality?. A gate that checks generated answers before they are added to the knowledge base lets a RAG system grow without polluting itself Can RAG systems safely learn from their own generated answers?. In both cases the evaluator does more than make one output better. It decides what the whole system is allowed to learn from, which is a bigger job than a GAN discriminator was ever given.
Sources 7 notes
Anthropic's three-agent system rephrases subjective questions like 'is this beautiful?' into concrete criterion-based assessments that a model can grade consistently. Running up to four hours per build with live browser interaction, the evaluator reduced score drift through few-shot calibration and penalized generic outputs.
Anthropic's testing found that asking a single agent to judge its own work produces inflated self-praise, especially on subjective tasks. Splitting evaluation into a dedicated skeptical agent proved far more tractable to tune than making a generator self-critical, allowing iteration to continue past where self-evaluation stalled.
The Consensus Game frames decoding as a signaling game where generator and discriminator must agree on answers. Equilibrium-Ranking finds their joint policy, enabling 7B models to match 540B model performance without fine-tuning.
GenPRM and ThinkPRM reframe process supervision as generative tasks with CoT reasoning before judgment, achieving superior performance on far fewer labels. A 1.5B GenPRM beats GPT-4o; ThinkPRM uses only 1% of PRM800K labels to surpass full-dataset discriminative verifiers.
Post-Completion Learning exploits unused sequence space after model output to train self-assessment capabilities during training while maintaining zero inference cost. The model learns to compute its own reward functions, internalizing evaluation rather than relying on external reward models.
Show all 7 sources
Models trained on mixtures of real and AI-generated data progressively lose rare events and unusual patterns across VAEs, GMMs, and LLMs. Each generation compounds the loss, making genuine human data increasingly valuable.
Systems can add generated answers to their retrieval corpus when outputs pass entailment verification, source attribution checks, and novelty detection. This prevents hallucinations from polluting future retrievals while allowing genuine knowledge accumulation.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Harness design for long-running application development
- LLM Evaluators Recognize and Favor Their Own Generations
- Transcendence: Generative Models Can Outperform The Experts That Train Them
- Post-Completion Learning for Language Models
- GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
- UR2: Unify RAG and Reasoning through Reinforcement Learning
- CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning
- The Consensus Game: Language Model Generation via Equilibrium Search