Why does teaching an AI how to do something often beat handing it more facts about the subject?
What makes procedural knowledge better than factual knowledge for authoring tasks?
This explores why giving an AI the how of a task (steps, rules of thumb, design principles) tends to improve what it produces more than giving it more facts about the subject, especially for open-ended making tasks like writing, designing and coding.
This explores why giving an AI the how of a task (steps, rules of thumb, design principles) tends to improve what it produces more than giving it more facts about the subject. The corpus points to one main reason: facts and procedures behave differently. Facts are narrow and have to be stored, while procedures are broad and transfer. An analysis of 5 million pretraining documents found that reasoning draws on how-to material spread across many diverse sources, while factual recall depends on memorizing specific facts from specific documents (Does procedural knowledge drive reasoning more than factual retrieval?). When researchers looked at what agent skills actually do, about 66% of successful cases worked by anchoring the agent's actions, and under 5% worked by supplying a missing fact (Do skills teach procedures or inject missing facts?).
Authoring tasks sharpen this because there is rarely a fact to look up. The quality of a slide deck, a design or a program comes from the order of steps and the judgment applied at each one. That judgment is often the tacit knowledge specialists carry in their heads. In one industrial case, writing a specialist's rules and design principles into an agent's scaffolding raised output quality by 206%, and non-experts using it were rated at expert level. The gain came from externalizing that expertise, not from a bigger model (Can codified expertise let non-experts match specialist output?). Distilling tutorial videos, code repositories and articles into a reusable skill library lifted artifact quality by about 12 points across seven authoring domains (Can tutorial videos teach software agents reusable skills?).
Procedures also have practical advantages over facts. They can be mined from source code with no prior agent experience and checked by rebuilding the code (Can source code replace experience as skill raw material?). They compound: routines extracted from past tasks and stacked hierarchically gained more as the new tasks looked less like the old ones (Can agents learn reusable sub-task routines from past experience?). A skill document can even be trained like model weights, with edits accepted only if they improve results on held-out tests. That adds no inference cost and carries over to other models (Can skill documents be optimized like neural network weights?). Putting the procedure in explicit code also lets each step see only the context it needs (Can algorithms control LLM reasoning better than LLMs alone?).
The corpus also draws limits, so procedure is not simply better than facts. Prompting can only reorganize what the model already knows, and no prompt can supply knowledge missing from training (Can prompt optimization teach models knowledge they lack?). Knowledge and reasoning seem to sit in different layers of the network, so training for reasoning can help math and hurt medicine (Why does reasoning training help math but hurt medical tasks?). Procedures are also fragile in their own ways. Skills fail when retrieved wrongly, invoked out of context or followed too rigidly (Do skills teach procedures or inject missing facts?). Playbooks erode when rewritten wholesale, which is why incremental updates matter (Can context playbooks prevent knowledge loss during iteration?). One gap remains: no note here runs a controlled facts-versus-procedure comparison on authoring specifically. The case is assembled from studies of skills, scaffolding and pretraining data that point the same way.
Sources 11 notes
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
Analysis of 8,135 trials shows procedural anchoring accounts for 65.7% of skill cases versus 4.5% for knowledge injection. Skills fail when retrieved incorrectly, invoked out of context, or followed too rigidly.
An industrial case study embedding domain rules and design principles into an LLM agent's scaffolding achieved 206% output-quality improvement and expert-level ratings from non-experts, bypassing the need for specialist oversight. The capability gain came from externalizing tacit expertise into structured harness components, not from model scale.
Resource2Skill distills tutorial videos, code repositories, and articles into a Skill Wiki that improved artifact quality by +11.9 percentage points across seven authoring domains compared to agents without skill access, outperforming two baseline approaches in most test cases.
Code2Skill extracts over 1 million auditable skills from GitHub repositories through source-blind reconstruction verification, improving performance from 42.90 to 47.90 macro-average across eight benchmarks, outperforming trajectory and document-derived alternatives.
Show all 11 sources
Agent Workflow Memory induces sub-task routines at finer granularity than full tasks, abstracts example-specific values, and compounds them hierarchically. This produces 24.6% relative gain on Mind2Web and 51.1% on WebArena, with larger gains as train-test gaps widen.
SkillOpt treats skill documents as trainable external state of frozen agents, using a text-space optimizer with held-out validation gating to accept only edits that improve performance. Across 52 benchmark cells and seven models, the approach matches or exceeds baselines while adding zero inference cost and enabling transfer across models.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.
Two-phase inference model shows knowledge retrieval operates in lower network layers while reasoning adjustment happens in higher layers. This separation explains why reasoning training improves math but can degrade knowledge-intensive domains like medicine.
The ACE framework treats contexts as evolving playbooks using generation-reflection-curation loops rather than full rewrites. This prevents knowledge loss from compression and detail erosion, achieving +10.6% on agentic tasks and +8.6% on finance without labeled supervision.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Demystifying Agent Skills: Why They Work-Until They Don't
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
- Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose
- ColluSkill: Adversarial Cross-Skill Composition for Evading Agent Skill Scanners
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- Grounded Skill Synthesis from Code at Scale for Agentic Intelligence
- Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviors