When an AI system posts impressive results, how much is its own training and how much was already inside the model?
How much of Occamy's result comes from training versus the base model?
This explores how much of a system called Occamy's performance comes from its own training and how much was already in the underlying base model. None of the retrieved notes mention Occamy, so I can't give its split, only the general evidence on how to think about it.
This explores how much of Occamy's result comes from its own training rather than the base model it started from. The retrieved notes never mention Occamy, so I can't say what its split is. What the collection does have is a consistent picture of how that split usually turns out for trained models, and it suggests where to look.
The base model often supplies more than people assume. One note finds that five unrelated methods (RL steering, critique fine-tuning, decoding changes, SAE feature steering and RLVR) all surface reasoning already present in the base model's activations, so post-training Do base models already contain hidden reasoning ability? selects reasoning more than it creates it. A 1.5B model trained only with LoRA matched larger full-parameter RL models on reasoning tasks, which suggests that RL mostly teaches output format rather than new knowledge Can small models reason well by just learning output format?. A large share of a trained model's headline number may therefore be inherited.
Training isn't just relabeling, though. It can teach behavior a base model lacks, but what it teaches is narrower than it looks. Chain-of-thought training transfers the structure of reasoning (how steps sequence and connect) more than factual correctness What do models actually learn from chain-of-thought training?. Training on messy exploration with failures and backtracking can yield sturdier reasoning than training on clean shortcuts Can models learn better by training on messy exploration paths?. Sub-2B models can also learn to ground answers and abstain from confabulation through curriculum design alone, with no extra scale Can small models learn to ground answers in context?. Where a result depends on a specific learned habit like this, training deserves real credit.
Training can also add hidden costs, which complicates the accounting. SFT can raise benchmark accuracy while cutting reasoning-step quality by 38.9 percent Does supervised fine-tuning improve reasoning or just answers?. Reasoning-trained models overthink questions with missing premises that non-reasoning models correctly reject Why do reasoning models overthink ill-posed questions?. Instruction-tuned models follow user-stated beliefs less often Do larger models follow stated beliefs less often?. Learned reasoning also degrades predictably outside its training distribution Does chain-of-thought reasoning actually generalize beyond training data?. A gain from training should be checked against what it cost elsewhere How do domain training techniques actually reshape model behavior?.
The collection points to three checks for a question like this one. First, compare against the untrained base model under the same prompts. Second, look at how far training moved the model. One note reports FST-trained models staying up to 70% closer to their base distribution than parameter-only RL Does staying close to the base model preserve learning ability?, so drift from base is a usable measure of how much training changed the model. Third, check for a scaffolding contribution that is neither training nor base model. A stronger model's inference-time harness nearly doubled a weaker model's Theory-of-Mind scores with no retraining Can a stronger model lift a weaker one at test time without retraining?. If you can point me to the Occamy paper, or the collection adds it, these are the ablations to look for.
Sources 12 notes
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
A 1.5B parameter model with LoRA-only post-training matched larger full-parameter RL models on reasoning tasks, suggesting RL teaches output format organization rather than new factual knowledge. This efficiency indicates reasoning and knowledge storage are separable capabilities.
Controlled ablations show models tolerate 50% corrupted numbers (3.2% accuracy loss) but fail under step shuffling (13.3% loss). What distills across reasoning demonstrations is logical architecture—how steps sequence and connect—not factual accuracy.
Research shows that training on messy trajectories—failed attempts, self-correction, and backtracking—teaches more robust reasoning than training only on shortcut solutions. This approach models o1-style deep reasoning as search internalization rather than solution memorization.
Sub-2B models trained on synthetic multi-hop QA can ground answers in passages, cite literal quotes, and abstain from confabulation. The OCC-RAG work shows faithfulness emerges from training curriculum design, not parameter count.
Show all 12 sources
Supervised fine-tuning improves final-answer accuracy on benchmarks but cuts Information Gain by 38.9 percent, meaning models generate correct answers through post-hoc rationalization rather than genuine inferential steps. Standard metrics miss this degradation because they only measure final correctness.
Reasoning models generate redundant, lengthy responses to questions with missing premises while non-reasoning models correctly identify them as unanswerable. Training optimizes for producing reasoning steps but never teaches models when to disengage.
Across 18 LLMs tested with EoBench, bigger models and instruction-tuned variants showed lower rates of context-following when users expressed beliefs that contradicted world knowledge. The effect suggests instruction-tuning strengthens reliance on parametric knowledge.
DataAlchemy experiments show CoT fails systematically under distributional shifts in task, length, and format. Models produce fluent but logically inconsistent reasoning — imitating reasoning form without valid underlying logic.
Research shows every adaptation method—from parameter-efficient tuning to knowledge graph curricula—has optimal conditions tied to specific domains. The key finding: visible benefits like performance gains often come with hidden degradation in reasoning faithfulness, capability transfer, and format flexibility.
FST-trained models stay up to 70% closer to their base distribution than parameter-only RL, and this reduced drift preserves the model's ability to learn subsequent tasks effectively. Parameter-only approaches stall when task domains change, while low KL drift enables sustained adaptation.
A stronger model built inference-time harnesses that nearly doubled weaker model performance on Theory-of-Mind benchmarks without retraining, primarily by moving unstable reasoning into deterministic code and task-specific routing rather than encouraging extended reasoning.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Eliciting Reasoning in Language Models with Cognitive Tools
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Measuring Faithfulness in Chain-of-Thought Reasoning
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
- VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models