After training, is an AI's reasoning newly built — or just dug out from skills it already had?

This explores how much of an LLM's behavior is actually created by post-training reward signals, as opposed to already sitting in the pretrained model and simply being brought out by reward training. The corpus leans toward a surprising answer: for reasoning ability, rewards mostly select from what is already there. For temperament and default dispositions, they seem to leave a larger mark.

The strongest evidence comes from studies of reinforcement learning with verifiable rewards (RLVR). As What does reward learning actually do to model reasoning? summarizes, RLVR makes models more likely to land on correct answers they could already reach, without expanding what they can solve at all. A single training example can be enough to trigger the effect. Even spurious rewards, which don't track correctness, work almost as well as correct ones, provided the model had the right pretraining. That points to the reward acting as a switch rather than a teacher. The physical footprint matches. Does reinforcement learning update only a small fraction of parameters? finds that RL changes only 5–30% of a model's parameters, and that roughly the same subnetwork gets updated across different random seeds. That suggests the reward is steering an existing structure, not building a new one.

How the reward is shaped also matters, in ways that complicate any simple 'rewards make the model' story. Does negative reinforcement alone outperform full reinforcement learning? shows that training only on penalties for wrong answers matches or beats full RL. Rewarding correct answers, on the other hand, can narrow the model's range of answers by piling probability onto a few paths. So much of what post-training does may be pruning bad behavior rather than adding good behavior. The source of the signal is also shifting. Can language models replace reward models with internal signals? describes methods where the model's own judgments replace a separate reward model, and Can reference examples make LLM judges reliable enough for self-improvement? shows an LLM judge anchored to reference answers can match a trained reward model. Increasingly, the 'post-training reward' is partly the pretrained model grading itself, which makes the two harder to separate.

Personality and disposition tell a different story. Do LLMs show reproducible psychological profiles when given standardized tests? finds that nine different LLMs each have stable, distinctive psychological profiles, yet all converge on a shared prosocial, stability-seeking pattern. That shared layer looks like a fingerprint of alignment training, sitting on top of model-specific traits that likely come from earlier in training. Fine-tuning also leaves traces the model can report on. Can language models describe their own learned behaviors? shows models fine-tuned to behave a certain way can describe that behavior without ever being trained to do so. But Can language models actually introspect about their own states? warns that most of what models say about themselves echoes human writing in their training data, not genuine insight into their own workings. So asking a model where its behavior comes from won't settle the question.

The corpus has no study that cleanly measures what share of behavior comes from rewards, so take the overall picture as directional. Post-training rewards seem to decide which of the model's existing abilities show up and what its default 'personality' looks like. The abilities themselves were mostly set during pretraining. If you want one place to start, What does reward learning actually do to model reasoning? is the clearest window into why a reward can matter enormously for behavior while adding almost nothing to capability.