Should past update success guide future model changes?

The paper's central claim is that "Past success in one source context does not authorize modifying every future parent." In an autonomous post-training loop, where a system proposes an update, trains a candidate, evaluates it and uses the feedback to pick the next proposal, the accumulated history of updates and outcomes "is not self-executing." An update's effect "depends on its parent model, data mixture, training stage, and evaluation contract," so evidence that an update helped under one context "may therefore be misleading under the current context." The authors name this problem conditional experience transfer.

The paper gives two costs of getting it wrong. Authorizing a context-incompatible update "consumes scarce training budget," and if the trained child is then promoted, it "changes the parent checkpoint and the relevance of later evidence." A bad reuse decision therefore does more than waste one run; it shifts the ground under the evidence that remains in memory. The proposed method, Boundary-Calibrated Intervention Transfer (BCIT), gates reuse before weight-changing training. It binds an observed effect to its source context, checks applicability conditions, vetoes candidates with named hard conflicts, and runs a bounded training trial on the current parent when the case is unresolved. Fully trained candidates still face a shared adoption rule that promotes or rolls back, and "only observed events extend memory." Reuse is checked at three points: before training, in a trial, and at adoption.

Two nearby notes treat the same loop from other sides. Why do LLM agents ignore condensed experience summaries? finds agents that fail to act on experience; this paper addresses the opposite failure, experience treated as context-free permission, and together they bracket the problem between ignoring history and following it unconditionally. Does constraining edits make skill learning more stable? reaches a similar conclusion for skill text, that control makes self-improvement stable, and BCIT's veto and rollback are comparable gates applied to weight updates. The substrate matters here. How does the substrate change which behaviors an optimizer can reach? separates weights from text, and Can frozen models learn better by extracting context into skills? operates on a frozen model. BCIT works on the weight side, where a promoted child rewrites the parent that later evidence is measured against. That reading of the contrast is mine; the excerpt does not draw it.

The excerpt is thin on evidence. It says only that "across the evaluated evidence chain, update effects vary across contexts," that under matched information BCIT "authorizes fewer harmful candidates while retaining beneficial ones," and that under matched compute it "attains a higher cross-task mean than all evaluated shared-model alternatives." It does not name the tasks, models or baselines, define "harmful" or "beneficial," give effect sizes, say how applicability conditions and hard conflicts are specified, or say how large the bounded trial is. The scope is one evaluated chain of updates, so generality is untested here. What the passages support is narrower than a general rule: a log of past updates is a set of hypotheses to check against the current parent, not a lookup table, and post-training compute is better allocated conditionally on evidence than unconditionally.