You can improve an AI without retraining it, using notes, skills and code. What's different about that?
What makes frozen model reasoning different from weight-based parameter updates?
This explores what changes when a model improves through things wrapped around fixed weights (memory, skills, code, decoding rules, representation edits) instead of through retraining the weights themselves.
This explores what changes when a model improves through things wrapped around fixed weights (memory, skills, code, decoding rules) instead of through retraining the weights. The corpus suggests the biggest difference isn't how smart the result is. It's where the improvement lives, and that decides whether you can read it, move it, and undo it.
Improvements around a frozen model live in text, code, and state you can open up. Skills extracted from context lifted GPT-4.1 from 11.1% to 16.5% on CL-bench and carried over to other model backbones Can frozen models learn better by extracting context into skills?. A harness runbook tuned on one model transferred unchanged to newer ones Can execution harnesses lift model performance without retuning weights?. Memory that records when a lesson applies beat generic reflection by 23 points and carried into new environments Can frozen language models continually improve through memory structure alone?. A weight update is tangled up with one model's internals, while these are more like a recipe that any cook can pick up. Layering state outside the model also lets you tell harness failures from model failures Can external state caches let models solve harder problems?, which is hard to do once the weights have moved.
The line also blurs from the weight side. Evidence that RL post-training teaches when to reason rather than how Does RL post-training create reasoning or just deploy it? fits with RL touching only 5–30% of parameters, in the same places across random seeds Does reinforcement learning update only a small fraction of parameters?. If much of what a weight update does is deployment, meaning deciding when to use abilities already there, then frozen methods are doing the same job in the open. A thought-switching penalty at decoding time stops reasoning models from abandoning promising paths, with no fine-tuning Why do reasoning models abandon promising solution paths?. A stronger model nearly doubled a weaker one on Theory-of-Mind tasks by moving unstable reasoning into deterministic code Can a stronger model lift a weaker one at test time without retraining?. That last move matters because extended thinking often produces more text rather than more computation. Reasoning variants show no consistent edge on numerical optimization Do reasoning models actually beat standard models on optimization?, and handing the procedure to code sidesteps the problem.
Frozen versus weights isn't a binary switch either. Representation finetuning keeps the weights fixed but learns small interventions on hidden representations, with 10–50x better parameter efficiency than LoRA Can editing hidden representations beat weight updates for finetuning?. The real question is what layer you edit: the weights, the internal activations, the context, or the code around the model.
Each route has its own costs. Weight updates can carry side effects that headline scores miss. Binary-reward RL degrades calibration by rewarding confident guessing Does binary reward training hurt model calibration?, and identical accuracy can hide badly organized internals Can models be smart without organized internal structure?. Frozen approaches are capped by what the model already holds, and gains like 11.1% to 16.5% are real but modest. The corpus has no head-to-head test of the same task solved both ways, so these are parallel lines of evidence rather than a verdict.
Sources 12 notes
Extracting natural-language rules from context into reusable skills improves frozen model reasoning without weight updates. On CL-bench, this lifts GPT-4.1 from 11.1% to 16.5%, with skills transferable across model backbones.
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
Agents using causal-form memory (preserving applicability conditions) outperform generic reflection by 23 points on repeated trials and gain 4-17 points transferring to new environments, showing memory shape matters more than parameter updates.
Prime Agent organizes persistent state in four levels (weights, context, persistent REPL plus subagents, disk-backed history) to let models read and write addressable state beyond their instruction stream. The approach isolates harness failures from model failures and reported gains on ARC-AGI-3, though specific components remain unablated.
Evidence shows base models already contain reasoning capability in latent form; RL training optimizes deployment timing rather than capability creation. Hybrid models recover 91% of performance gains by routing tokens only, and activation vectors for reasoning strategies pre-exist before any RL.
Show all 12 sources
Across seven RL algorithms and ten LLM families, RL induces intrinsic parameter sparsity of 5–30% without explicit regularization. Critically, these sparse updates are nearly full-rank and nearly identical across random seeds, indicating structural rather than arbitrary parameter selection.
Reasoning LLMs exhibit two reinforcing failures: wandering (invalid exploration) and underthinking (premature path-switching). Decoding-level interventions like thought-switching penalties improve accuracy without fine-tuning, suggesting viable solutions exist but are abandoned prematurely.
A stronger model built inference-time harnesses that nearly doubled weaker model performance on Theory-of-Mind benchmarks without retraining, primarily by moving unstable reasoning into deterministic code and task-specific routing rather than encouraging extended reasoning.
Reasoning variants with extended CoT show no consistent advantage over standard models on constraint-bound numerical tasks like optimal power flow. Extended thinking produces more text, not more iterative computation, suggesting the bottleneck is numeric procedure rather than reasoning steps.
ReFT learns task-specific interventions on frozen model representations rather than updating weights, with LoReFT (low-rank linear subspace variant) dramatically outperforming LoRA across reasoning, instruction-following, and NLU benchmarks while using far fewer parameters.
Binary correctness rewards incentivize high-confidence guessing because they don't penalize confident wrong answers. Adding the Brier score as a second reward term mathematically guarantees joint optimization of accuracy and calibration without trade-off.
Models trained with SGD can contain all the linearly decodable features needed for a task while maintaining fundamentally broken internal organization. This makes them vulnerable to perturbation and distribution shift invisible to standard evaluation metrics.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Useful Memories Become Faulty When Continuously Updated by LLMs
- A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
- Large Language Model Reasoning Failures
- The Invisible Leash: Why RLVR May Not Escape Its Origin
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- On the Reasoning Capacity of AI Models and How to Quantify It