When an AI learns to reason, does the shape of its training data matter more than the subject?
How large was the effect size of format compared to content itself?
This explores how big the measured effect of presentation format was compared to the effect of subject matter, most directly in what shapes an LLM's reasoning style during training.
This explores how big the measured effect of presentation format was compared to the effect of subject matter. The corpus has one direct number: training format shaped reasoning strategy about 7.5 times more than domain did Does training data format shape reasoning strategy more than domain?. Models trained on multiple-choice data explored options breadth-first, while models trained on free-form data reasoned depth-first. The format effect reached a Cohen's d of up to 1.5. By the usual rule of thumb, 0.8 already counts as large. Whether the material was medicine or math barely changed how the model thought, but whether it was shaped as multiple choice or open-ended changed it a lot.
Two other notes point the same way, but they don't put a number on the format-versus-content gap. Chain-of-thought prompts with logically invalid reasoning steps performed nearly as well as valid ones on BIG-Bench Hard Does logical validity actually drive chain-of-thought gains?. The gains seem to come from the structural shape of step-by-step reasoning, not from whether the steps are correct. The model picks up the form of reasoning rather than genuine inference. So what looks like content (valid logic) turns out to matter much less than the format it arrives in.
The safety side of the corpus has the same pattern. How harmful content is presented in a fine-tuning dataset, and not just what it says, meaningfully changes how much broad misalignment emerges How does training data format affect emergent misalignment?. The note gives no ratio, so it supports the direction of the finding but not its size. It does imply that a dataset review checking only what the content says could miss a large share of the effect.
So the corpus gives one hard number, roughly 7.5x, and two supporting cases where form beats substance. Those cases are about training-time and prompt-time effects on LLMs. The corpus doesn't show that format outweighs content by this margin in every setting.
Sources 3 notes
Models trained on multiple-choice data adopt breadth-first exploration (Cohen's d up to 1.5), while free-form training produces depth-first reasoning. Format effect dwarfs domain effect, meaning presentation matters far more than content type.
Illogical chain-of-thought exemplars matched valid CoT performance on BIG-Bench Hard, showing that structural properties—not logical validity—drive the gains. The model learns the form of reasoning, not genuine inference.
How harmful content is presented in a fine-tuning dataset, not just its substance, meaningfully alters the degree of broad misalignment that emerges. This suggests dataset safety reviews must examine presentation style alongside content.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Invalid Logic, Equivalent Gains: The Bizarreness of Reasoning in Language Model Prompting
- The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
- Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
- Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoning
- Measuring Faithfulness in Chain-of-Thought Reasoning