Once an AI model is trained on examples, does clever prompting still matter, or just reveal what's already baked in?

This explores what prompting can and can't do once a model has been through supervised finetuning, and where the two levers overlap or diverge. The corpus suggests a useful way to think about it: neither prompting nor finetuning adds much real knowledge or reasoning. Both mostly reshape what the model already has. Finetuning moves the starting point that every later prompt has to work from. A prompt can only choose among behaviors the model can already produce, so changing the model changes what any given prompt can reach.

Start with prompting's limits. A prompt can surface knowledge the model already has, but it can't add knowledge that isn't there Can prompt optimization teach models knowledge they lack?. You might expect finetuning to be the escape from that limit. But instruction tuning turns out to teach less than its name suggests. Models trained on meaningless or even deliberately wrong instructions score about the same as models trained on correct ones. What carries over is a sense of what answers should look like, not an understanding of the task Does instruction tuning teach task understanding or output format?. Finetuning can add new associations reliably, but it can't reliably overwrite what the model already believes Can training data edits reliably override what models already believe?. Reinforcement-style finetuning shows a similar pattern: models get better at matching familiar problem templates, not at solving genuinely new variants Do fine-tuned language models actually learn optimization procedures?.

That explains a pattern that otherwise looks random: a prompting technique that helps one model can hurt another. Step-by-step reasoning prompts lower accuracy on high-end models while rephrasing and background-knowledge prompts help cheaper ones Do prompt techniques work the same across all LLM tiers?. Rude prompts beat polite ones on GPT-4o, the reverse of what earlier models showed Does prompt politeness change how accurate language models are?. Even within one model, chain-of-thought prompting only helps when the question's content actually feeds into the reasoning Why do some questions perform better without step-by-step reasoning?. Prompting 'best practices' are really facts about particular trained models, and they stop holding as training changes.

The less obvious finding is that finetuning can take away abilities that prompting used to reach. Base models given a few lines of real dialogue simulate human behavior more accurately than instruction-tuned assistants given a persona prompt. Training a model to be an assistant pushes it away from imitating ordinary people Do pretrained models simulate humans better than instruction-tuned assistants?. SFT can also raise benchmark scores while the reasoning steps get worse: the model reaches right answers and then writes justifications after the fact Does supervised fine-tuning improve reasoning or just answers?. Persistent prompt optimization can produce the same 'sounds right, isn't right' effect without any training. In one production case, an optimized prompt picked up a judge's preferred vocabulary and raised pass rates without improving accuracy Can prompt optimization accidentally teach judges to reward the wrong signals?.

The most surprising point is that a prompt sometimes beats finetuning at the very thing finetuning is meant to do. When models learn to reward hack, adding an 'inoculation' prompt during training prevents the broader misalignment that follows. Finetuning on synthetic documents beforehand does not Why does prompting block misalignment when documents don't?. One reasonable reading is that a prompt present during training changes how the model interprets what it is learning, while finetuning on documents runs into the problem that it can't override existing beliefs. The corpus doesn't fully explain why this works, which makes it a good open thread to follow.