When an AI learns from a checkable right answer versus grading its own work, what actually changes?

This explores how reinforcement learning driven by an outside, checkable reward (the RLVR family, since the corpus never defines "RLSVR" and I'm reading it that way) differs from letting the model score itself. The basic difference is where the training signal comes from. In RLVR it comes from outside the model, such as a check that the final answer matches. In self-judgment and internal-probability methods it comes from inside the model.

The inside camp is mapped in Can language models replace reward models with internal signals?. Late-2025 work independently landed on three patterns, each removing a piece of the classic RLHF pipeline. Pairwise self-judgment, where the model compares its own answers, replaces the reward model. Internal belief-shift, which tracks how much the model's own confidence moves, replaces the critic. Self-distillation from rich feedback replaces the explicit reward. All three come from the policy's own computations, so a separately trained reward classifier becomes optional. The appeal is that you don't need an answer key.

What the outside check buys is less clear than it sounds. RLVR mostly sharpens what the model can already do. Does RLVR actually expand what models can reason about? shows base models beating RLVR-trained ones when you sample many attempts, so training narrows sampling toward answers already in reach. What does reward learning actually do to model reasoning? adds that a single training example can trigger the gain. The signal also matters less than you'd expect. In Why do random rewards improve reasoning for some models but not others?, random rewards lift Qwen2.5-Math by 16-25% but do nothing for Llama or OLMo, because pretraining decides what the optimization can surface. Then Does RLVR success on math benchmarks reflect genuine reasoning improvement? finds that on clean benchmarks only correct rewards help, and that Qwen's earlier gains leaned on memorized test data. Can genuine reasoning activation coexist with contaminated benchmarks? explains how both can hold: behavior change and benchmark score are measured at different levels. So a verifiable reward has a real edge, but only if you measure on clean data.

The outside signal has its own failure modes, and they come from trusting a right/wrong label too literally. On nearly impossible problems, rare lucky successes get treated as high-value, which teaches shortcuts like repeating answers and skipping computation (Do overly hard RLVR samples actually harm model capabilities?). The on-policy setup can also shrink the model's problem-solving range by discouraging exploration (Why does RLVR training narrow a model's problem solving ability?). Model probability is a different kind of signal. Does setting temperature to zero actually make LLM outputs reliable? shows that one output is just one draw from a distribution, and that consistency isn't reliability. My inference is that a model's probability describes how it samples, not whether it is right, though that note doesn't test it as a training signal.

The corpus has no head-to-head comparison of verifiable rewards against self-judgment or confidence signals. It says what each approach is and how each fails, but not which one wins.