Teaching an AI new facts takes different routes, and each one trades upfront training cost against how easily you can change them later.

This explores the main ways to give a language model knowledge it doesn't already have, and how each one trades training cost against flexibility. The corpus offers a useful starting map: four routes, each suited to a different deployment constraint How do knowledge injection methods trade off flexibility and cost?. Dynamic injection (RAG) needs no retraining and updates the moment you change the documents, but every query pays for the lookup in latency. Static embedding (fine-tuning the knowledge into the weights) gives the fastest answers but costs the most up front, and the knowledge is hard to revise afterward. Modular adapters sit in between: you train small add-on modules that can be swapped in and out. Prompt optimization costs nothing to train. Notably, the same note reports that combining routes beats any single one, so the real design question is how to mix them.

The cheap end has a hard limit. Prompting can only bring forward knowledge the model already absorbed in pretraining. It reorganizes what is there and cannot add anything new Can prompt optimization teach models knowledge they lack?. If the domain knowledge was never in the training data, no clever prompt will supply it. That is why prompting counts as a separate route rather than a cheaper version of the others.

The tradeoff isn't fixed, either, and two findings suggest that how you train matters as much as how much. StructTuning reaches about half of full knowledge-injection performance with just 0.3% of the training data. It does this by sorting text chunks into an automatically generated domain taxonomy, so the model learns where each fact sits in the field the way a student learns from a textbook's chapter structure Can organizing knowledge structures beat raw training data volume?. A related note argues that small amounts of explicit, structured knowledge fix problems that learning purely from data leaves behind, such as poor robustness and representations nobody can interpret Does refusing explicit knowledge harm AI system performance?. On the training-signal side, RLAG uses reinforcement learning that rewards both correct answers and sound explanations. It embeds domain knowledge more coherently than standard supervised fine-tuning Can reinforcement learning embed domain knowledge more effectively than supervised fine-tuning?. A broader overview warns that every adaptation method has a domain-specific sweet spot and hidden costs. Visible accuracy gains can come with quieter losses in reasoning faithfulness and format flexibility How do domain training techniques actually reshape model behavior?.

The less obvious part of the map is a set of routes that avoid touching the main model at all. MeMo trains a separate memory model to hold the new knowledge Can a separate memory model inject knowledge without touching the LLM?. Unlike RAG, its inference cost doesn't grow with the size of the corpus, and it works with frozen proprietary models you can't fine-tune. The price is up-front training and a fixed capacity limit. In agent research, the same idea shows up as learning without weight updates. VOYAGER stores skills as executable code in a searchable library and builds complex skills out of simpler ones, which avoids catastrophic forgetting Can agents learn new skills without forgetting old ones?. AgentFly improves an agent's behavior entirely through episodic memory modules and reaches 87.88% on GAIA with the model's parameters untouched Can agents learn continuously from experience without updating weights?.

So the choice is less a two-way split between flexible and expensive than a question of where the knowledge lives: in the prompt, in an external store, in a plug-in module, in a companion model, or in the weights. Each location fixes when you pay (at training time or on every query) and what you can still change later. The corpus has less on direct, head-to-head cost comparisons across all these routes. The four-way taxonomy note is the closest thing to one, so start there before the specialized papers.