If many training examples share the same template, are they really one example copied, or fresh examples from one blueprint?
Do identical task structures mean repeated instances or new synthetic samples with same design?
This explores whether a shared task design (same structure, format or skeleton) is best treated as one example repeated many times or as fresh synthetic examples built from the same blueprint, and how each behaves in training.
This explores whether a shared task design means the same example repeated or new synthetic examples built from the same blueprint. No note in the corpus tests repeated copies against fresh samples head-on, so what follows is stitched together from neighboring work. The corpus leans toward the blueprint reading, with a warning attached.
The blueprint reading is well supported. TarGEN builds synthetic data from "instance seeds", which are small atomic task elements, instead of full input-output examples. It seeds the inputs, then constrains how labels get generated, and it gained 1-3 points on SuperGLUE tasks. It also works for domains with no prior examples at all Can synthetic data replace seed examples in task generation?. So a task design can exist and produce new samples without any single example to copy.
Sharing a design doesn't make the samples interchangeable, though. Two notes show that the variation inside the design decides whether the data is any good. ToolFlow's synthetic tool-calling data looked unrealistic when tools were sampled at random, because unrelated tools can't plausibly work together. Sampling from relevance graphs and planning the dialogue first fixed it Why does random tool sampling produce unrealistic synthetic training data?. Synthetic dialogues showed the same pattern. Realism needed subtopic, persona and context variation stacked together, and that stack reached 90.48% of in-domain dialogue performance Can synthetic dialogues become realistic through layered diversity?. The design fixes the skeleton, and the diversity has to be added on purpose.
It also helps to know what a model picks up from a repeated structure. Models tuned on semantically empty or deliberately wrong instructions did about as well as those tuned on correct ones. What seems to transfer is the shape of the output space, not understanding of the task Does instruction tuning teach task understanding or output format?. That means fresh samples with the same design may mostly teach the format. Reinforcement learning shows the risk of leaning on sameness. When rewards barely vary within a prompt, policies collapse into generic, input-agnostic templates Why do language models collapse into generic templates?. RL also tends to lock onto a single dominant pretraining format within the first epoch Does RL training collapse format diversity in pretrained models?.
Two more notes suggest that "identical" is slippery. A set-level reward found that diverse, mediocre reasoning traces beat redundant strong ones, because an aggregator needs varied raw material Can diverse mediocre traces outperform redundant expert traces?. Another argues that the reusable unit of reasoning data isn't the example. It's the feedback interface around it, meaning the verifier, base model, optimizer and budget, and changing any one of these changes what the same data does What is the actual reusable unit of reasoning data?. So an identical design can behave differently depending on what surrounds it. What the corpus can't say is whether exact repeats or fresh samples generalize better when the design is held fixed.
Sources 8 notes
TarGEN generates synthetic data using atomic task elements (instance seeds) instead of full input-output examples, achieving 1-3 point improvements on SuperGLUE tasks. The approach works by constraining label generation after seeding inputs, enabling data creation for domains with no prior examples.
Random tool sampling fails because unrelated tools cannot credibly compose, and Q&A framing ignores multi-turn dialogue coherence. ToolFlow shows that sampling tools from relevance graphs and generating with dialogue plans closes this gap.
Research shows that realistic synthetic dialogues require three multiplicative layers: subtopic specificity, Big Five persona variation, and 11 contextual characteristics via Chain of Thought reasoning. This structured approach captures 90.48% of in-domain dialogue performance.
Models trained on semantically empty or deliberately incorrect instructions achieve comparable performance to those trained on full correct instructions, achieving 43% vs random baseline 42.6%. The semantic content of instructions appears largely irrelevant; what transfers is knowledge of the output space.
When within-prompt reward variance is low, task gradients weaken and regularization dominates, pushing policies toward generic outputs. SNR-Aware Filtering—selecting high-variance prompts before updates—recovers performance across tasks and scales.
Show all 8 sources
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
SPIRAL shifts RL reward from individual traces to sampled sets, optimizing for complementarity rather than per-trace accuracy. Diverse mediocre traces outperform redundant strong ones because aggregators need raw material to arbitrate, not confirmation.
The reusable unit in post-training is a feedback interface entangled with six factors: verifier, base model, lineage, optimizer, scaffold, and budget. Changing any one alters the same data's effect, making attribution tractable only when these are jointly released.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
- Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
- Mechanistically Interpreting the Role of Sample Difficulty in RLVR for LLMs
- A Survey on Post-training of Large Language Models
- ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis
- A Little Human Data Goes A Long Way
- The Art of Scaling Reinforcement Learning Compute for LLMs
- Orchestrating Synthetic Data with Reasoning