SYNTHESIS NOTE
Can models dynamically activate expert skills at inference time?
Can language models efficiently discover and compose task-specific capabilities on the fly without modifying base weights? This explores whether test-time adaptation through expert vector composition outperforms fixed fine-tuning approaches.
Transformer2 introduces Singular Value Fine-tuning (SVF): instead of modifying full weight matrices or even low-rank adaptations, SVF extracts and tunes only the singular values within a model's weight matrices. This produces compact expert vectors that are inherently composable — they can be dynamically mixed at inference without interference.
The inference mechanism has two passes:
- First pass (dispatch): The model executes on the input and observes its own test-time behavior, gathering information about what skills the current problem requires.
- Second pass (adaptation): The framework combines available expert vectors based on the first-pass analysis, providing a targeted modification to the base weights specifically tailored to the task.
Three adaptation strategies provide monotonic performance benefits with increasing access to test-time conditions, enabling deployment-scenario-appropriate tradeoffs.
The key properties that make this work:
- Compositionality: SVF expert vectors combine naturally because they operate on orthogonal singular value dimensions. LoRA adapters, by contrast, modify rank-k subspaces that may interfere when composed.
- Efficiency: SVF trains far fewer parameters than LoRA while outperforming it. Expert vectors are compact enough to store many specializations.
- Continual learning: New expert modules can be developed offline and added without catastrophic forgetting, because the base model weights are never modified — only the singular value modulation changes.
The neuroscience parallel is deliberate: the brain activates specific regions depending on the task and dynamically reconfigures its functional networks in response to changing demands. Transformer2 operationalizes this for LLMs.
The deeper principle: the requisite capabilities for many downstream tasks already exist within pretrained models. The bottleneck is not knowledge but activation — knowing when to deploy which capability. This aligns with Does RL teach reasoning or just when to use it?, extending it to the architecture level: self-adaptation is about routing to existing capabilities, not creating new ones.
Inquiring lines that read this note 70
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do neural networks separate factual knowledge from reasoning abilities? Which computational strategies best support reasoning in language models?- Could superposed decoding algorithms maintain multi-task representation during generation?
- Why do singular value experts compose better than low-rank adapter subspaces?
- Can text-space optimization and audit governance coexist in a single skill lifecycle?
- Can a trained decoder replace both search and parameter updates?
- Does task superposition explain how models learn from multiple in-context trajectories?
- Why does full multi-task fine-tuning perform worse than sequential training?
- How do ensemble methods apply within a single model?
- Can a single model trained on two tasks predict untrained decision tasks?
- What performance trade-offs emerge when composing multiple independently trained model capabilities?
- Does training on granular tasks beat training on the full function calling problem?
- Can extracted skills transfer effectively across different domains and model architectures?
- How do transformers stitch together learned behaviors when adapting to new tasks?
- Can training on diverse related tasks be more efficient than task-specific training?
- Can prompting inject new knowledge into already-trained AI models?
- Can prompting unlock compositional skills that pretraining already learned?
- Can prompt optimization inject new knowledge into language models?
- Can prompt optimization or fine-tuning inject knowledge models do not already contain?
- What knowledge can prompt optimization actually activate in trained models?
- Do text-space skills transfer learning across different frontier models?
- What techniques work best for injecting domain knowledge at training time?
- How do training-time and inference-time knowledge injection techniques compare?
- What training cost tradeoffs exist between fine-tuning and other knowledge injection methods?
- Can language models learn to form ad-hoc conventions through training?
- What architectural changes would let language models develop genuine functional competence?
- Can dynamic instance-specific prompt selection solve the generalization problem across tasks?
- What deployment tradeoffs emerge between single-pass and multi-pass inference adaptation?
- When should full-parameter post-training be used instead of LoRA adaptation?
- Does sparse parameter updating improve test-time training's computational cost?
- How should skill libraries coordinate with gradient-based weight optimization?
- Where does skill extraction fail compared to genuine model adaptation?
- Why does the gap between theoretical expressiveness and learned capability matter?
- Can expert vectors learned offline transfer across multiple model architectures?
- How does pretrained knowledge constrain what adaptation strategies can achieve?
- What happens to base model capabilities when you apply finetuning?
- Can models adapt and combine search strategies beyond their training algorithm?
- How much does pretraining quality affect the modularity of fine-tuned models?
- Can specialized components replace single fully-trained models in deployment?
- Why does adaptation concentrate in low-dimensional subspaces of weights or representations?
- Can smaller models achieve domain expertise through focused RL training?
- How do sparse parameter updates enable when-not-how training to work?
- How much task-similar finetuning data does test-time training actually need?
- How do task frequency and complexity interact with model capacity during training?
- Can learned priors effectively select and weight ensemble members by inference budget?
- Can single-hop knowledge automatically compose into multi-hop capability?
- Why does scaling data and model size improve compositional generalization?
- Why does specializing to one task make future task learning harder?
- Can memory-based adaptation and gradient fine-tuning operate on complementary timescales?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Concept map
Can models dynamically activate expert skills at…
Does RL teach reasoning or just when to use it?
How do knowledge injection methods trade off flexi…
Can isolating task-specific parameters prevent mul…
Can decoding-time tuning preserve knowledge better…
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
your link
semantically near
linked from elsewhere
-
Does RL teach reasoning or just when to use it?
Does reinforcement learning in thinking models actually create new reasoning abilities, or does it simply teach existing capabilities when to activate? This matters for understanding where reasoning truly emerges.
Transformer2 operationalizes this at the architecture level via test-time expert composition
-
How do knowledge injection methods trade off flexibility and cost?
When and how should domain knowledge enter an AI system? This explores the speed, training cost, and adaptability trade-offs across four injection paradigms, and when each approach suits different deployment constraints.
SVF occupies a new position: lightweight training, dynamic inference, composable
-
Can isolating task-specific parameters prevent multi-task fine-tuning interference?
Explores whether identifying and protecting task-specific parameter regions can prevent the performance degradation that occurs when fine-tuning models on multiple tasks simultaneously. This matters because it could enable safe multi-task adaptation without sacrificing individual task performance.
SVF achieves similar goals through singular value decomposition rather than region identification
-
Can decoding-time tuning preserve knowledge better than weight fine-tuning?
Explores whether applying alignment signals at inference time rather than modifying model weights can better preserve the factual knowledge learned during pretraining while still achieving alignment goals.
complementary inference-time adaptation: proxy-tuning uses a single expert distributional shift, SVF composes multiple expert vectors; both avoid base weight modification but SVF provides finer-grained multi-skill composition via orthogonal singular value dimensions
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Transformer2: Self-adaptive LLMs
- Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence
- Not All Parameters Are Created Equal: Smart Isolation Boosts Fine-Tuning Performance
- An Emulator for Fine-Tuning Large Language Models using Small Language Models
- QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration
- Mixture-of-Experts Meets Instruction Tuning: A Winning Combination for Large Language Models
- Exploring Format Consistency for Instruction Tuning
- Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains
Original note title
self-adaptive LLMs compose expert vectors at inference via two-pass singular value fine-tuning