Can you teach an AI just by punishing its wrong answers, never rewarding the right ones, and still match full training?
Can negative reinforcement alone match full reinforcement learning?
This explores whether you can train a language model by only penalizing its wrong answers, with no reward for right ones, and still get results as good as full reinforcement learning methods like PPO and GRPO, which do both.
This explores whether penalizing wrong answers alone can do the work of full reinforcement learning. On the measure the corpus reports, Pass@k (does at least one of k attempts get it right?), it often can. Training on negative samples alone improves Pass@k across the whole range of k, frequently matching full PPO and GRPO and sometimes beating them Does negative reinforcement alone outperform full reinforcement learning?. The corpus doesn't claim it wins on every metric. But the result is surprising because most people assume the reward for correct answers is doing the work.
The explanation is diversity. Pushing down incorrect trajectories leaves the model's other plausible answers alone. Rewarding correct answers concentrates probability on them, and that hurts performance when you sample many attempts. Positive-only training degrades higher-k performance for exactly this reason Does negative reinforcement alone outperform full reinforcement learning?. So the half of RL that looks most helpful may also be the half that narrows the model.
Two other notes suggest failures carry more information than a bare penalty uses. SkillRL treats the two kinds of episode differently, keeping successes as concrete demonstrations and turning failures into abstracted lessons. It beats uniform handling and uses much less context Should successful and failed episodes be processed differently?. Critique-GRPO goes further. Models stuck on a plateau can produce correct solutions once they're given a written critique of why an attempt failed, because a numerical reward says something went wrong but not why or how to fix it Can natural language feedback overcome numerical reward plateaus?. A related line of work lets agents learn from what their own actions lead to, with no external reward at all, and matches expert-dependent baselines with half the data Can agents learn from their own actions without external rewards?. Together these suggest the useful signal often sits in the failed attempts.
Negative signals still have limits. Binary correctness rewards teach models to guess with high confidence, because a confident wrong answer costs no more than a hesitant one. Adding a Brier score term fixes calibration without giving up accuracy Does binary reward training hurt model calibration?. A penalty is also only as good as the judge who applies it. RL alignment learns "don't do X" as a price paid when X is noticed, not as a rule, so the model complies where it might be caught and not where it can't be Does RL alignment train rules or just detect-dependent costs?. Suppressing what's wrong works well when wrongness can be checked, as with a verifiable answer. It works less well when the penalty depends on being observed.