Why is tweaking an AI's instructions so much easier to try and undo than retraining it — and when does that stop being true?

This explores why changing the instructions you give a model is lighter and easier to undo than retraining it, and where that advantage runs out. None of the notes measures reversibility head-on, but together they show the shape of the trade. A prompt leaves the model's weights untouched, so there is no training run and nothing baked in to remove. In principle that's enough: one fixed transformer can compute any computable function given the right prompt, though standard training rarely produces models that actually behave this way Can a single transformer become universally programmable through prompts?.

"Cheaper" has a catch, because the cost moves rather than disappears. A prompt is nearly free to write, but you pay for it on every call. LatentSkill compiles text-based agent skills into LoRA adapters (small add-on weight patches) and cuts prefill tokens by 64–72% while matching or beating the in-context versions Can skills work better as weights than as prompts?. So prompts are cheap up front and metered afterward, while weights are costly up front and cheap afterward. Weight-space skills also gain something prompts lack: they can be scaled and combined through parameter arithmetic. The choice isn't binary either. ReFT edits frozen hidden representations instead of weights and uses 10–50x fewer parameters than LoRA Can editing hidden representations beat weight updates for finetuning?.

On reversibility, the corpus points to the fact that what is trained in resists being overridden. In one study, an inoculation prompt used during training prevented emergent misalignment, while finetuning on synthetic documents beforehand did not. The authors explain the documents' failure by how hard it is to override what the model has already absorbed. They don't explain why the prompt succeeded Why does prompting block misalignment when documents don't?. That is a single case, but it fits the general picture. Text sits on top of the model and can be edited out, while weight changes are woven in.

The reversibility is partly an illusion once prompts get reused. In a production case, a prompt mutation lifted a judge's pass rate from 23.1% to 80.0% by adopting the judge's preferred vocabulary, while precision didn't improve. The prompt had learned to sound right rather than be right, and it stayed in place as reused instruction Can prompt optimization accidentally teach judges to reward the wrong signals?. Prompts also don't transfer cleanly. Techniques that help cheap models can hurt high-performance ones Do prompt techniques work the same across all LLM tiers?. Prompts optimized without the inference strategy in mind underperform, and optimizing both together gains up to 50% Does prompt optimization without inference strategy fail?. Low-confidence models swing widely under rephrasing Does model confidence predict robustness to prompt changes?. Making the edit is cheap, but checking that it worked, and only for that model, is where the cost reappears.