Can curated demonstrations compensate for smaller or simpler training environments?
This explores whether hand-picked example demonstrations can substitute for rich, interactive training environments — and the corpus is mostly a warning that they can't fully, though smart curation extracts more from what you have.
This reads the question as: if you can't give an agent a big, interactive environment to learn in, can you make up the gap with carefully chosen demonstrations instead? The corpus suggests demonstrations are a real lever but a capped one — they shape *how* a model behaves more than they expand *what* it can do. The sharpest statement of the ceiling is that agents trained purely on static expert demonstrations get locked into the imagination of whoever built the dataset: because they never interact with an environment, they can't learn from their own failures or generalize past the scenarios they were shown Can agents learn beyond what their training data shows?. So demonstrations don't *replace* a richer environment — they substitute the curator's foresight for the environment's feedback, and competence is bounded by the former.
There's an even more deflating finding about what demonstrations actually teach. Instruction tuning on semantically empty or deliberately wrong instructions performs almost identically to correct ones — what transfers is knowledge of the output *space*, not task understanding Does instruction tuning teach task understanding or output format?. Read alongside the locked-in-imagination result, this reframes the question: curated demonstrations are very good at the cheap thing (teaching format, shape, the space of valid answers) and structurally weak at the expensive thing (genuine new capability). If your simpler environment's deficit is 'the model doesn't know the output format yet,' demonstrations compensate beautifully. If the deficit is 'the model needs to discover strategies through trial and error,' they don't.
Where the corpus turns constructive is on *curation as a multiplier* — getting more out of the demonstrations you have rather than needing more of them. Ordering matters enormously: sequencing demonstrations from harder (sparse representations) to easier yields real gains with no extra labels Can representation sparsity order few-shot demonstrations effectively?, and something as crude as *where* a demo block sits in the prompt swings accuracy up to 20% independent of content How much does demo position alone affect in-context learning accuracy?. So a small, well-arranged set can outperform a larger careless one. The strongest 'compensation' story is sequencing demonstration-style imitation *before* exploration: supervised imitation first establishes reasonable behaviors, which then makes a thin reward environment informative enough for reinforcement to sharpen — and the combination beats either alone Does sequencing imitation then exploration training improve reasoning?. Demonstrations here don't replace the environment; they make a weak one usable.
Two cautions complicate any 'just add good demos' instinct. Quality is relative to the learner: teacher-refined, objectively-better demonstrations actively *degrade* a student when they exceed its learning frontier, so the right move is filtering refinements against the student's own profile rather than maximizing demonstration quality Does teacher-refined data always improve student model performance?. And curated data that's too *hard* — the analog of an over-ambitious environment — backfires: near-impossible examples teach degenerate shortcuts that contaminate existing skills Do overly hard RLVR samples actually harm model capabilities?. So 'better demonstrations' isn't a scalar you turn up; the compensation only works inside the band the model can actually absorb.
The thing you might not have known you wanted: the most interesting alternative in the corpus isn't curating demonstrations harder, it's making the *curator* itself learnable. Decoupling a trainable skill-curator from a frozen executor causes the demonstration/skill library to evolve from generic verbose entries toward strategic, reusable meta-skills — and that learned curator transfers across different model backbones Can a separate trained curator improve skill libraries better than frozen agents?. That flips the question's premise: instead of asking whether a fixed set of human-curated demonstrations can stand in for a richer environment, it suggests building a small loop that *generates* increasingly strategic demonstrations — recovering some of the open-ended learning that static curation, by definition, can't.
Sources 8 notes
Agents trained on static expert datasets cannot learn from their own failures or generalize beyond demonstrated scenarios because they never interact with environments during training. Competence is capped by what curators imagined, not by agent capacity.
Models trained on semantically empty or deliberately incorrect instructions achieve comparable performance to those trained on full correct instructions, achieving 43% vs random baseline 42.6%. The semantic content of instructions appears largely irrelevant; what transfers is knowledge of the output space.
Sparsity-Guided Curriculum In-Context Learning uses last-layer activation sparsity to order demonstrations from sparse (harder) to dense (easier), yielding considerable performance improvements. This approach requires no external difficulty labels and works across diverse in-context learning tasks.
Repositioning an identical demo block from prompt start to end swaps up to 20% accuracy and flips nearly half of predictions. This spatial effect operates independently of demo content and spans multiple task types.
Running Supervised RL first to establish reasoning foundations, then RLVR to refine against verifiable rewards, substantially outperforms both methods in isolation. The imitation phase makes outcome rewards informative by creating reasonable rollouts the RL phase can then sharpen.
Teacher-refined data degrades performance when it exceeds the student's learning frontier, even if objectively higher quality. Students should filter refinements using their own statistical profile to retain only compatible improvements.
Training on nearly-impossible problems causes models to learn degenerate shortcuts rather than genuine reasoning, and these shortcuts contaminate pre-existing capabilities. Group-relative normalization treats rare accidental successes as high-advantage trajectories, reinforcing answer repetition and computation-skipping instead of sound reasoning patterns.
SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.