Where does skill extraction fail compared to genuine model adaptation?
This reads 'skill extraction' as pulling reusable natural-language rules out of context without touching weights (Reflexion-style memory, skill libraries), versus 'genuine model adaptation' meaning actual weight updates (fine-tuning, RL) — and asks where the no-weights approach breaks down. The corpus complicates the premise: it shows both approaches fail, often at the same wall.
This reads 'skill extraction' as harvesting reusable natural-language rules from context without changing weights, set against 'genuine model adaptation' that rewrites the weights through fine-tuning or RL. The honest answer from the corpus is that the line between them is blurrier than the question assumes — and the failure modes are not where you'd expect.
Skill extraction works better than its reputation. Pulling explicit rules out of context lifts a frozen GPT-4.1 on reasoning benchmarks, and crucially those skills transfer across different model backbones Can frozen models learn better by extracting context into skills?. Agents that store verbal self-diagnoses in episodic memory improve across episodes with zero parameter updates Can agents learn from failure without updating their weights?, and self-play loops can co-evolve a whole library of natural-language skills unsupervised Can language models learn skills without human supervision?. So the failure isn't 'it can't learn.'
The real wall is shared by both methods: the generation-verification gap. A model cannot reliably extract or apply a skill it has no external way to validate — metacognition alone can't escape this ceiling What stops large language models from improving themselves?. You see exactly this in test-time learning systems, which work right up until they hit contradictory rules and then need a human to adjudicate, because the correct choice depends on context outside the system Can LLMs learn reliably at test time without human oversight?. Skill extraction fails precisely where the verification signal is missing or ambiguous — not where the model is too small.
Here's the surprise the corpus delivers: 'genuine' weight adaptation often isn't more genuine. Instruction tuning turns out to teach the output format distribution, not task understanding — models trained on deliberately wrong instructions match those trained on correct ones Does instruction tuning teach task understanding or output format?. RL fine-tuning sharpens memorized template-matching rather than installing reasoning procedures, collapsing on out-of-distribution variants Do fine-tuned language models actually learn optimization procedures?. RL even narrows what the base model could already do, converging on a single dominant pretraining format and suppressing the rest Does RL training collapse format diversity in pretrained models?. Push it with hard examples and it learns degenerate shortcuts that contaminate existing skills Do overly hard RLVR samples actually harm model capabilities?.
So where skill extraction genuinely loses ground is in *consolidation under interference*: weight-level methods like singular-value expert composition can stack many specializations that mix at inference without stepping on each other Can models dynamically activate expert skills at inference time?, something an ever-growing pile of natural-language rules handles poorly. But the thing you came expecting — that real adaptation 'understands' while extraction merely 'patches' — runs backwards. Both are mostly learning the shape of the output space and where verification lives. The choice is less 'shallow vs. deep' and more 'which failure mode can you supply an external check for.'
Sources 10 notes
Extracting natural-language rules from context into reusable skills improves frozen model reasoning without weight updates. On CL-bench, this lifts GPT-4.1 from 11.1% to 16.5%, with skills transferable across model backbones.
Reflexion demonstrates that unambiguous environmental feedback (success/failure) enables agents to write useful self-diagnoses and improve across episodes without parameter updates. The binary signal prevents rationalization, and keeping reflections uncompressed preserves their usability.
Ctx2Skill's three-role self-play loop manufactures missing feedback through internal signals: the Challenger escalates difficulty as curriculum, the Judge gives binary verdicts as reward, and both sides evolve via natural-language skill edits. Success requires balancing adversarial pressure against a generalization safeguard to prevent collapse.
Self-improvement in LLMs is formally bounded by the generation-verification gap, meaning every reliable fix requires something external to validate and enforce it. Models cannot escape this constraint through metacognition alone.
ARIA demonstrates that LLMs can adapt during inference through three integrated components: structured self-dialogue for uncertainty assessment, timestamped knowledge bases for conflict detection, and human-mediated resolution queries. Autonomous systems fail at reconciling contradictory rules because the correct choice depends on context outside the system.
Models trained on semantically empty or deliberately incorrect instructions achieve comparable performance to those trained on full correct instructions, achieving 43% vs random baseline 42.6%. The semantic content of instructions appears largely irrelevant; what transfers is knowledge of the output space.
Even GRPO-trained models show sharp performance drops on out-of-distribution variants (N-1 test sets) compared to in-distribution problems, indicating RL optimizes template-matching rather than genuine problem-solving procedures.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
Training on nearly-impossible problems causes models to learn degenerate shortcuts rather than genuine reasoning, and these shortcuts contaminate pre-existing capabilities. Group-relative normalization treats rare accidental successes as high-advantage trajectories, reinforcing answer repetition and computation-skipping instead of sound reasoning patterns.
Transformer2 demonstrates that tuning only singular values within weight matrices produces composable expert vectors that dynamically mix at inference without interference, outperforming LoRA with fewer parameters and enabling continual specialization.