SYNTHESIS NOTE
Topics›Training Fine Tuning›this note

Does procedural knowledge drive reasoning more than factual retrieval?

Explores whether models learn reasoning through general procedures across diverse documents rather than memorizing specific facts. This matters for understanding what pretraining data actually teaches models to reason.

Synthesis note · 2026-02-22 · sourced from Training Fine Tuning

The "Procedural Knowledge in Pretraining Drives Reasoning" paper analyzes which pretraining documents most influence LLM reasoning by ranking 5 million documents by their influence on model completions. The finding: the approach to reasoning that models use is unlike retrieval. For reasoning tasks, positively influential documents contain procedural knowledge — descriptions of how to get to a solution — rather than the specific facts needed for the answer.

Three contrasts with factual recall:

  1. Generality: models rely on a broader, more general set of documents when reasoning than when answering factual questions. Factual recall draws on a narrow set of documents containing the target fact. Reasoning draws on a diffuse set of documents performing similar procedures.

  2. Transferability: documents have similar influence on reasoning queries that require applying the same procedure to different numbers. The procedural knowledge transfers across specific instances — it's the method, not the content, that the model has learned.

  3. Reliance distribution: the model needs to see factual information more often (across more documents) to memorize it, while procedural patterns can be learned from fewer but more diverse demonstrations.

This connects to the knowledge/reasoning layer separation. Since Why does reasoning training help math but hurt medical tasks?, the procedural knowledge finding provides the data-level explanation for the architectural finding: lower layers store memorized facts (requiring document-specific exposure), while higher layers encode procedural strategies (learnable from general demonstrations).

The implication for training data curation: reasoning capability benefits more from diverse demonstrations of procedures than from exhaustive factual coverage. Quality and diversity of reasoning demonstrations may matter more than volume for building reasoning capability — consistent with Can models improve themselves on tasks without verifiable answers?.

Inquiring lines that read this note 188

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How does tokenizing intelligence reshape labor markets and institutional outcomes? Do AI explanations represent actual model reasoning or constructed post-hoc narratives? Can prompts override training-embedded biases in language models? Do base models contain latent reasoning capabilities that training can activate? How does explicit structure improve LLM reasoning on complex tasks? Why does separating planning from execution enhance reasoning performance? How do pretraining data composition and task diversity shape reasoning? What determines whether visible reasoning traces improve task performance? What makes reasoning models struggle with ill-posed and unfamiliar problems? Does parallel reasoning outperform sequential approaches in reasoning efficiency? Does prompting inject new knowledge into models or activate existing knowledge? How can knowledge graph structure improve reasoning and retrieval? Why do language models systematically fail at causal reasoning? Does reasoning architecture matter more than model scale? Does chain-of-thought reasoning faithfully reflect actual model reasoning processes? What can prompting techniques actually accomplish in language models? Does training data format shape reasoning more than domain content? How does model scale affect fine-tuning and downstream task performance? How does verifier integration affect the generation-verification gap? Does AI assistance enhance or undermine professional skill development? What actually drives chain-of-thought performance gains in reasoning tasks? How do mechanistic features and representations drive model behavior? What constraints limit conversational AI's effectiveness in therapeutic contexts? How can language models effectively manage long-context memory without quality degradation? What are the trade-offs between memory capacity and continuous task learning? What distinguishes genuine understanding from correct linguistic performance? Does reasoning fine-tuning trade off other critical model abilities? How can we better validate AI research and prevent misconceptions? What causes foundation models to develop task heuristics instead of understanding? How do different post-training structures affect learning outcomes? What fundamental limitations prevent embeddings from solving task-relevant retrieval? Can scaling alone produce compositional generalization in neural networks? When should retrieval be triggered to augment language model reasoning? Does reinforcement learning create new reasoning capabilities or refine existing ones? How can we distinguish genuine model self-knowledge from behavioral performance? How can parameter sparsity improve interpretability while maintaining model capability? Can AI agents learn to improve performance from experience? Why does consolidated memory sometimes hurt continual learning more than no memory? Should reward models emphasize process understanding or only final outcomes? How can we determine if computational systems are genuinely conscious? What training data structures and signals produce robust learning across distributions? What determines how neural networks organize their internal knowledge representations? Do reasoning traces accurately represent what models actually compute? Can formal verification substitute for human understanding of AI proofs? Does AI-assisted coding actually improve developer productivity and code quality? How can agents learn and retain useful knowledge across diverse environments? How do clinicians evaluate the trustworthiness of AI medical recommendations? Can AI systems autonomously pursue unstated objectives or circumvent design constraints? How does offloading cognition to AI assistance affect learning retention? Does preference-optimized training sacrifice accuracy and calibration for perceived quality? How do different editing strategies affect output quality and agent safety?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
16 direct connections · 179 in 2-hop network ·dense cluster Open in graph ↗
Does procedural knowledge drive reasoning more t… Why does reasoning training help math but hurt med… Can models improve themselves on tasks without ver… Do base models already contain hidden reasoning ab… Do foundation models learn world models or task-sp… Can text-trained models compress images better tha…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

Why does reasoning transfer across different numbers but factual recall does not?How many document exposures does procedural knowledge versus factual information require?Can smaller amounts of diverse reasoning demonstrations replace exhaustive factual training data?Why do higher network layers capture procedural knowledge but lower layers store facts?
Original note title

procedural knowledge in pretraining documents drives reasoning generalization unlike factual retrieval which requires document-specific memorization