Can prompting unlock compositional skills that pretraining already learned?
This explores whether prompting acts as a key that activates compositional abilities a model already absorbed during pretraining — versus actually teaching the model something new.
This explores whether prompting unlocks latent skills from pretraining rather than adding new ones — and the corpus comes down firmly on the side of activation, with a sharp ceiling. The clearest statement is that prompt optimization retrieves what's already there but cannot inject what isn't: no prompt strategy compensates for foundational knowledge that was never in the training data, it can only reorganize what exists Can prompt optimization teach models knowledge they lack?. So the answer to 'can prompting unlock skills pretraining learned?' is yes — but only those. The skill has to already be latent.
What's striking is how little signal it takes to flip the switch. A single training example can boost math accuracy from 36% to over 73% and keep improving long after training accuracy saturates — evidence that the skill was sitting dormant and a minimal activation cue released it Can a single training example unlock mathematical reasoning?. Even appending an emotional phrase like 'this is very important to my career' reliably lifts performance, working through motivational framing rather than any new information Can emotional phrases in prompts improve language model performance?. Both point the same direction: the capability is pre-loaded, and prompting is choosing which latent behavior to surface.
But here's the turn that makes this question worth asking — what pretraining actually loaded may be thinner than 'compositional skill' suggests. Chain-of-thought, the canonical example of prompted reasoning, looks less like genuine novel inference and more like constrained imitation of reasoning patterns seen in training; performance degrades predictably under distribution shift, the signature of pattern-matching rather than real composition Does chain-of-thought reasoning reveal genuine inference or pattern matching?. Instruction tuning tells a parallel story: models trained on semantically empty or deliberately wrong instructions perform about as well as those given correct ones, meaning what transfers is knowledge of the output format, not task understanding Does instruction tuning teach task understanding or output format?. So prompting may be unlocking the *form* of a skill more than the skill itself.
The deeper lever sits in what pretraining deposits in the first place. Reasoning generalization traces back to broad, transferable procedural knowledge spread across many documents — not narrow factual recall — which explains why prompting can reliably re-evoke reasoning but can't conjure facts the model never memorized Does procedural knowledge drive reasoning more than factual retrieval?. And whether a cue even lands is predictable: keyword priming after learning is forecastable from pre-learning probability, with a threshold below which the prompt simply has nothing to grab onto Can we predict keyword priming before learning happens?. Which way the question resolves also depends on which question you ask — instance-adaptive analysis shows step-by-step prompting helps only when the question's semantics flow into the prompt structure first; for simple questions, prompting the reasoning form actively hurts Why do some questions perform better without step-by-step reasoning?.
The most interesting frontier is whether you even need a prompt to do the unlocking. Self-adaptive models compose task-specific expert vectors at inference by tuning singular values, mixing latent skills dynamically without a clever prompt at all Can models dynamically activate expert skills at inference time? — and rather than relying on prompting to extract reasoning after the fact, newer work plants chain-of-thought directly into pretraining with information-gain rewards, lifting reasoning ~19% Can chain-of-thought reasoning be learned during pretraining itself?. The thread across all of it: prompting is a retrieval key, powerful but bounded by what pretraining deposited — and the more ambitious bet is to load richer, more composable skills earlier so there's more for any key to unlock.
Sources 10 notes
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.
A single example in RLVR boosts math performance from 36% to 73.6% and enables test accuracy to improve for 1,400 steps after training accuracy reaches 100%, revealing that minimal activation signals unlock latent reasoning capability.
Testing EmotionPrompt across ChatGPT, Bard, and Llama 2 showed consistent performance gains from appending psychological phrases like "This is very important to my career." The effect works through motivational framing rather than new information, with positive emotional words driving over 50% of improvements.
CoT works by constraining models to reproduce familiar reasoning patterns from training, not by enabling novel symbolic reasoning. Performance degrades predictably under distribution shifts—the signature of imitation rather than capability emergence.
Models trained on semantically empty or deliberately incorrect instructions achieve comparable performance to those trained on full correct instructions, achieving 43% vs random baseline 42.6%. The semantic content of instructions appears largely irrelevant; what transfers is knowledge of the output space.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
Pre-learning keyword probability strongly predicts post-learning priming across architectures and model sizes, with a ~10^-3 threshold separating contexts where priming occurs from those where it doesn't. Just 3 training exposures suffice to establish the effect.
Saliency analysis reveals that CoT prompting fails when question information doesn't aggregate into the prompt structure before reasoning begins. For simple questions, direct question-to-answer flow outperforms step-by-step reasoning, showing the optimal prompt depends on question type, not just task category.
Transformer2 demonstrates that tuning only singular values within weight matrices produces composable expert vectors that dynamically mix at inference without interference, outperforming LoRA with fewer parameters and enabling continual specialization.
RLP treats CoT as exploratory action during pretraining, using log-likelihood improvement as verifier-free reward. Applied to Qwen3-1.7B and Nemotron-Nano-12B, the method improves math and science benchmarks substantially, suggesting reasoning can be planted earlier in training.