How much task-similar finetuning data does test-time training actually need?
This explores the data-efficiency question behind adapting a model to a task at or near inference time — and the corpus's surprising answer is that 'how much' matters far less than 'which,' because a small slice of well-matched data routinely beats the full pile.
This reads the question as: when you fine-tune a model on data resembling the target task, how little can you get away with? The collection's most direct answer is startling — the LESS method Can we train better models on less data? selects roughly 5% of instruction data by gradient similarity to the target capability, and training on that sliver consistently *beats* training on the whole dataset. The reason reframes the whole 'how much' instinct: full datasets contain examples that actively hinder the task by nudging the model's reasoning strategy in the wrong direction. So the answer isn't 'more is safer' — past a small, well-matched core, extra data is often a tax, not a bonus.
That principle shows up from several angles. Counterintuitively, the *content* of task-similar data may matter less than its shape: models trained on semantically empty or even deliberately wrong instructions hit nearly the same performance as those trained on correct ones Does instruction tuning teach task understanding or output format?. What transfers is knowledge of the output space — the format — not task understanding. If a few examples are mostly teaching the model 'what answers look like here,' you don't need many. And more isn't automatically better-quality data: teacher-refined examples that exceed a student's learning frontier degrade it Does teacher-refined data always improve student model performance?, and over-hard training samples push models into degenerate shortcuts that contaminate skills they already had Do overly hard RLVR samples actually harm model capabilities?. Volume and difficulty can both backfire.
The genuinely lateral move in the corpus is to ask whether you need task-specific weight updates at all. Self-adaptive models tune only the singular values of weight matrices to build composable 'expert vectors' that mix at inference time Can models dynamically activate expert skills at inference time?, and proxy-tuning shifts behavior at decoding time while leaving base weights untouched — preserving knowledge that direct fine-tuning corrupts in the lower layers Can decoding-time tuning preserve knowledge better than weight fine-tuning?. These sidestep the data-quantity question by adapting cheaply or reversibly rather than retraining.
It's worth naming where this sits in the larger map: researchers split test-time methods into *internal* (training the model to reason on its own) and *external* (search and verification layered on at inference) How do internal and external test-time scaling compare?, and these complement rather than compete. The data-hungry part is the internal, capability-building side — which is exactly where the selection findings bite hardest. If a tiny, gradient-matched set builds the capability, external inference-time techniques can extract the rest of the performance without more training data at all.
The thing you may not have known to ask: the honest answer to 'how much' is 'less than you think, and the wrong extra data costs you.' The frontier question has quietly shifted from *quantity* to *selection* — finding the few percent that actually match the task, and knowing when to skip weight updates entirely.
Sources 7 notes
LESS uses low-rank gradient features to select instruction data most similar to target capabilities, and training on the selected 5% consistently outperforms full dataset training. The improvement occurs because mixed datasets contain examples that actively hinder specific skills by shifting reasoning strategy away from task requirements.
Models trained on semantically empty or deliberately incorrect instructions achieve comparable performance to those trained on full correct instructions, achieving 43% vs random baseline 42.6%. The semantic content of instructions appears largely irrelevant; what transfers is knowledge of the output space.
Teacher-refined data degrades performance when it exceeds the student's learning frontier, even if objectively higher quality. Students should filter refinements using their own statistical profile to retain only compatible improvements.
Training on nearly-impossible problems causes models to learn degenerate shortcuts rather than genuine reasoning, and these shortcuts contaminate pre-existing capabilities. Group-relative normalization treats rare accidental successes as high-advantage trajectories, reinforcing answer repetition and computation-skipping instead of sound reasoning patterns.
Transformer2 demonstrates that tuning only singular values within weight matrices produces composable expert vectors that dynamically mix at inference without interference, outperforming LoRA with fewer parameters and enabling continual specialization.
Proxy-tuning closes 88-91% of the alignment gap while surpassing direct fine-tuning on knowledge tasks by leaving base model weights untouched. Direct fine-tuning corrupts knowledge storage in lower layers, whereas proxy-tuning applies distributional shifts that primarily affect reasoning and style.
Research shows test-time scaling methods split into internal (training models for autonomous reasoning) and external (inference-time search and verification). They complement rather than compete; internal builds capability while external extracts performance from existing capability.