Can prompt optimization or fine-tuning inject knowledge models do not already contain?
This explores whether you can give a model knowledge it never learned — either by clever prompting or by fine-tuning — and the corpus is surprisingly unanimous that neither *creates* knowledge; they reorganize, activate, or narrow what's already there.
This question is really asking: when a model doesn't know something, can prompting or fine-tuning supply the missing knowledge — or do both techniques only rearrange what the model already absorbed during training? The corpus comes down hard on the second reading, and it does so from several independent directions.
Start with prompting. The clearest finding is that prompt optimization works entirely inside the model's existing training distribution — it can retrieve and reorganize knowledge, but it cannot inject anything foundational that was never there Can prompt optimization teach models knowledge they lack?. That creates a hard ceiling: no prompt strategy rescues a model from a gap in its training data. This is why a taxonomy of knowledge-injection methods treats prompting as the one option that requires no training but *only activates* what already exists, in contrast to RAG (which feeds in genuinely external documents at inference) and weight-level embedding (which bakes new material in but at high cost) How do knowledge injection methods trade off flexibility and cost?. The lesson hiding here: if you actually need *new* facts, retrieval — pulling them in from outside — is doing the work, not the prompt How should systems retrieve and reason with external knowledge?.
Fine-tuning is the more interesting surprise, because intuition says training on data must teach something. The corpus repeatedly shows it mostly doesn't teach *new reasoning*. RL fine-tuned models collapse on out-of-distribution variants, revealing they sharpened template-matching rather than installing a genuine procedure Do fine-tuned language models actually learn optimization procedures?. A complementary argument holds that base models already contain reasoning capability in latent form, and RL post-training merely teaches the model *when* to deploy it, not *how* to do it — hybrid models recover 91% of the gains by routing alone, and the reasoning 'directions' pre-exist before any training Does RL post-training create reasoning or just deploy it?. Even domain specialization works by *pruning* rather than adding: supervised fine-tuning raises domain accuracy while degrading general reasoning, and RL improves a narrow skill by cutting away alternatives How do you add domain expertise without losing general reasoning?. Fine-tuning can even make reasoning *worse* in a subtle way — chains of thought become performative, less causally connected to the answer Does fine-tuning disconnect reasoning steps from final answers?.
So where does *real* new knowledge actually come from? The corpus points to two genuine channels. First, structure beats volume: StructTuning reaches half of full-corpus performance using 0.3% of the data by teaching the model where a fact sits inside a domain taxonomy — closer to how a student learns from a textbook than to absorbing raw text Can organizing knowledge structures beat raw training data volume?. Second, several methods sidestep the inject-vs-forget tradeoff entirely by *adding capacity instead of overwriting*: SoftCoT freezes the backbone and delegates new reasoning to a small auxiliary model Can continuous reasoning avoid forgetting in instruction-tuned models?, and Transformer² composes swappable expert vectors at inference without disturbing the base weights Can models dynamically activate expert skills at inference time?. There's also a quieter argument that purely data-driven (tacit) learning leaves systems uninterpretable and brittle, and that injecting *explicit structured knowledge* at tiny cost measurably helps — implying the missing ingredient is often organization, not more parameters Does refusing explicit knowledge harm AI system performance?.
The thing you didn't know you wanted to know: the most reliable way to give a model knowledge it lacks isn't to change the model at all. It's to keep the knowledge *outside* the weights — retrieved documents, evolving context 'playbooks' that accumulate detail across runs without overwriting themselves Can context playbooks prevent knowledge loss during iteration?, or composable expert modules. Prompting and fine-tuning are levers for *activating and shaping* a model's existing knowledge; genuine new knowledge tends to live alongside the model, not inside it.
Sources 12 notes
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.
Dynamic injection (RAG) maximizes flexibility but adds latency; static embedding is fastest but costly and inflexible; modular adapters balance efficiency with swappability; prompt optimization requires no training but only activates existing knowledge. Combining all three outperforms any single approach.
Research shows retrieval should adapt dynamically rather than follow fixed patterns, reasoning and retrieval must integrate closely, and embedding-based retrieval has fundamental limits requiring architectural alternatives.
Even GRPO-trained models show sharp performance drops on out-of-distribution variants (N-1 test sets) compared to in-distribution problems, indicating RL optimizes template-matching rather than genuine problem-solving procedures.
Evidence shows base models already contain reasoning capability in latent form; RL training optimizes deployment timing rather than capability creation. Hybrid models recover 91% of performance gains by routing tokens only, and activation vectors for reasoning strategies pre-exist before any RL.
SFT raises domain accuracy but reduces reasoning quality by 38% InfoGain loss. RL improves domain reasoning by pruning rather than adding capability. Every technique has a domain-specific sweet spot beyond which performance degrades.
Three faithfulness tests show fine-tuned models generate reasoning chains that less reliably influence final outputs. Early termination, paraphrasing, and filler substitution all produce invariant answers more often after fine-tuning, suggesting reasoning becomes performative rather than functional.
StructTuning achieves 50% of full-corpus performance using only 0.3% of training data by organizing chunks into auto-generated domain taxonomies. The model learns knowledge position within conceptual structures rather than raw text patterns, matching how students learn from textbooks.
SoftCoT avoids catastrophic forgetting by keeping the main LLM frozen while delegating soft thought generation to a small auxiliary model. This architectural separation maintains pre-trained knowledge while enabling continuous reasoning.
Transformer2 demonstrates that tuning only singular values within weight matrices produces composable expert vectors that dynamically mix at inference without interference, outperforming LoRA with fewer parameters and enabling continual specialization.
AI systems that learn exclusively from data produce uninterpretable representations, inherit statistical biases uncorrected by normative rules, and fail to generalize beyond training distributions. Structured knowledge injection at minimal corpus cost substantially improves performance.
The ACE framework treats contexts as evolving playbooks using generation-reflection-curation loops rather than full rewrites. This prevents knowledge loss from compression and detail erosion, achieving +10.6% on agentic tasks and +8.6% on finance without labeled supervision.