When an AI agent's attempt passes the check, is that run automatically good enough to train the next model on?

This explores what turns a recorded agent run into a trustworthy training example once a check says it worked, and whether passing that check is enough. The corpus suggests it usually isn't: validation is a gate, and what gets through the gate still needs sorting. The most direct evidence is in GRPO-RoC, where a trajectory that reaches the right answer can still be a poor teacher. If it got there through a messy string of failed code calls, the model learns to tolerate those errors. So the pipeline treats good and bad runs asymmetrically. Successful trajectories are filtered again for cleanliness, while diverse failures are kept as negative signal. That let a 14B model reach frontier-level math in 510 RL steps (Why do correct code trajectories teach models to tolerate errors?).

A second question is what 'valid' should mean. A terminal score says an answer came out right, not that the agent got there the intended way. BenchShield addresses this by recording infrastructure evidence and letting operators claim that a run followed the intended evaluation path, not just that it scored well (Can infrastructure evidence replace terminal scores in benchmark validation?). For training data, this suggests the validity stamp should describe the process as well as the result. The validator can also change form. Where running code is impractical, structured reasoning templates reached 93% accuracy at checking whether two patches are equivalent, which the note argues is reliable enough to act as an RL reward for some task types (Can structured reasoning replace code execution for RL rewards?).

After validation, a trajectory can also be cut into pieces. One outcome check on a long run is a thin signal, but the run's own structure can supply denser ones. Tree-GRPO, Supervised RL and ToolPO each get step-level credit from tree topology, agreement with expert actions, or where the tool calls sit, with no separately annotated process rewards (Can trajectory structure replace hand-annotated process rewards?). Confidence gives another cheap signal. Runs where the model commits early and then rationalizes tend to be flawed, and rewarding gradual confidence growth improved Countdown accuracy by 42 points without process labels (Can confidence trajectories reveal when reasoning goes wrong?). A correct final answer with a suspicious confidence curve is therefore a weaker example than its label suggests.

The less comfortable part is how much of what a validated example teaches is form. Binary correctness rewards encourage confident guessing and degrade calibration unless a Brier-score term is added (Does binary reward training hurt model calibration?). Logically invalid chain-of-thought examples performed nearly as well as valid ones (Does logical validity actually drive chain-of-thought gains?), and instruction tuning on deliberately wrong instructions scored 43% against a 42.6% random baseline (Does instruction tuning teach task understanding or output format?). RL also tends to lock onto one dominant pretraining format within the first epoch (Does RL training collapse format diversity in pretrained models?). If a validated trajectory mostly transfers the shape of a good run, then how it is filtered matters as much as whether it passed.

A validated trajectory becomes a good example after a further selection step. Successes are filtered for clean process, failures are kept as contrast, and the validator checks the path as well as the score. Structure or confidence signals then add step-level credit, and calibration is guarded against the overconfidence that binary rewards reward. The corpus has little on validating the strategy behind a run, as opposed to its execution, and outcome checks alone see that part poorly.