Can LoRA match full fine-tuning performance in practice?

Thinking Machines Lab's research team, led by John Schulman, reports that across a set of post-training experiments, low-rank adaptation (LoRA) — a parameter-efficient fine-tuning method that replaces a weight matrix W with W′=W+γBA using two much smaller matrices — "learns with the same sample efficiency as FullFT and achieves the same ultimate performance" for supervised fine-tuning on small-to-medium instruction-tuning and reasoning datasets. The team calls this the "low-regret regime" and reports it "covers most post-training scenarios."

The post ties this parity to specific conditions rather than treating LoRA as uniformly equivalent to full fine-tuning (FullFT). LoRA must be applied to all weight matrices, "especially MLP and MoE layers" — attention-only LoRA underperforms even when parameter counts are matched by raising rank (a rank-256 attention-only adapter underperforms a rank-128 MLP-only adapter with "approximately the same number of parameters"). Capacity must also exceed dataset size: once datasets "exceed LoRA capacity," training efficiency degrades, scaling with the ratio of model capacity to dataset size rather than hitting a hard floor. Optimization dynamics differ too — the team finds the optimal learning rate for FullFT is "lower by a factor of 10" than for high-rank LoRA, and that LoRA is less tolerant of large batch sizes, a property the post attributes to the "product-of-matrices parametrization" itself rather than something a higher rank fixes.

The finding complicates the parameter-scaling picture drawn by How should finetuning scale with model and data size?, which found parameter-efficient tuning scaling largely ineffective; this post narrows that claim by specifying conditions — full weight-matrix coverage, sufficient rank relative to dataset size — under which a PEFT method does match full fine-tuning, rather than contradicting the earlier result outright. It sits alongside Does dataset size matter more than dataset fit? and Can we finetune huge language models on a single GPU? as a further LoRA-family data point, here addressing performance parity rather than memory footprint or dataset selection.

The post does not establish where the low-regret regime's boundary falls in absolute dataset-size terms, nor does it test architectures or domains beyond Llama 3 and Qwen3 (including one mixture-of-experts model) on the Tulu3 and OpenThoughts3 datasets. It is Thinking Machines Lab's own research blog post, not a peer-reviewed study, so the claims should be read as findings from this specific experimental setup rather than a general law about LoRA. The practical implication the team draws — that LoRA can substitute for full fine-tuning across "many applications" of post-training — follows only as strongly as that experimental scope supports.