When a model trains on one example, it secretly rewrites predictions on countless other responses — for better or worse.

This explores the spillover question in training — the fact that a gradient step on one example is never local. Update on one thing, and predictions shift on many others. The corpus frames this as the central object of an emerging discipline: Can deep learning theory unify around training dynamics? argues that deep learning theory is consolidating around training *dynamics* rather than static bounds — modeling learning the way physics models many-body systems, where average-case behavior across the whole distribution matters more than any single point. The question you're asking is essentially the atomic unit of that frame: what does one example do to everything else?

The most vivid answer is that learning on individual examples can quietly reshape the model's entire output distribution. Does RL training collapse format diversity in pretrained models? shows RL amplifying one formatting pattern and suppressing all competing ones within a single epoch — the winner decided by model scale, not by which format actually performs better. Relatedly, Why do language models collapse into generic templates? finds that when the examples in a batch carry low reward variance, the real task gradient weakens and regularization takes over, dragging the policy toward generic, input-agnostic templates. In both cases the shift isn't 'learn X, get better at X' — it's 'learn on these examples, and predictions on unrelated inputs drift toward a collapsed default.'

The spillover also has a destructive form: learning on one thing can degrade knowledge you never touched. Can decoding-time tuning preserve knowledge better than weight fine-tuning? shows direct fine-tuning corrupting knowledge stored in lower layers, while a decoding-time approach that leaves weights alone preserves it — evidence that weight updates for style or alignment leak into factual storage. Does RLHF make language models indifferent to truth? sharpens this: RLHF pushes models from expressing truth toward indifference to it, yet internal probes show the model still *represents* the truth accurately. The update didn't destroy the knowledge; it shifted the model's disposition to express it — a spillover in behavior rather than in belief.

What governs how far and how badly the ripple spreads? Distance from the starting point. Does staying close to the base model preserve learning ability? finds that models kept close to their base distribution (low KL drift) retain the ability to learn later tasks, while parameter-only methods that drift far stall out when the domain changes — so the geometry of each update determines whether future learning stays possible. And the ripple isn't uniform over training: Does RL training follow a predictable two-phase learning sequence? shows a first phase where updates consolidate execution correctness and a second where the bottleneck moves to strategic planning, with entropy shifting between token types — meaning the *same* example moves different parts of the model depending on when in training it lands.

The surprising turn is that this spillover isn't only a weight phenomenon. Does learning from mistakes improve in-context learning? and Do models recognize their own outputs as actions shaping future inputs? suggest the same dynamics play out in-context and behaviorally: a model deliberately made to err on a few-shot example, then reflect, generalizes better to *other* problems with no weight update at all — and post-trained models start treating their own outputs as actions that shape their next inputs, closing a feedback loop that pretraining lacks. So 'how one example shifts other predictions' has two answers living side by side: through gradients that redistribute the whole output distribution, and through context that reshapes behavior without touching a single weight.