Why do successful and failed trajectories need different memory processing?
This explores why agents that learn from their own experience seem to get more out of treating wins and losses asymmetrically — storing them in different forms rather than dumping both into memory the same way.
This explores why agents that learn from their own experience seem to get more out of treating wins and losses asymmetrically — storing successes in one form, failures in another, rather than logging both the same way. The corpus converges on a surprisingly consistent answer: a success and a failure carry different *kinds* of information, so compressing them identically throws away what each is actually good for.
The clearest statement is SkillRL's, which keeps successful episodes as concrete demonstrations — replay this, it worked — while distilling failures into abstracted lessons rather than verbatim transcripts Should successful and failed episodes be processed differently?. The logic is that a success is valuable as a *template* you can imitate, whereas a failure is valuable only as a *warning* — and a warning doesn't need the full play-by-play, just the principle that prevents the repeat. ReasoningBank reaches the same place from the memory side: storing strategy-level hints extracted from both successes *and* failures beats success-only memory and beats hoarding raw trajectories, because the useful residue of a failure is a strategy, not a recording Can agents learn better from their failures than successes?.
There's a second, sharper reason failures can't be stored like successes: failures contaminate. The self-conditioning work shows that when a model's own prior errors sit in its context, performance degrades non-linearly — the model starts imitating its mistakes, and scaling the model doesn't fix it Do models fail worse when their own errors fill the context?. So a failure left in raw, demonstration-shaped form is actively dangerous; it has to be transformed into something that teaches without modeling the bad behavior. Reflexion's trick is exactly this transformation — it converts a binary success/failure signal into a verbal self-diagnosis stored as episodic memory, and crucially keeps those reflections uncompressed so they stay usable, while the *trajectory* itself isn't what's replayed Can agents learn from failure without updating their weights?.
The asymmetry also shows up on the training side, not just memory. GRPO-RoC filters *positive* trajectories hard for quality — keep only the clean ones — while *preserving diverse failures* as negative signal, letting a 14B model reach frontier math performance Why do correct code trajectories teach models to tolerate errors?. If you filtered both symmetrically you'd lose the very diversity that makes failures informative. ReasonFlux-PRM makes the complementary point for reasoning traces: a failed step inside a thinking trace is often *informative exploration*, not an error to be penalized, so process-reward models that treat all deviation as wrong degrade badly Why do standard process reward models fail on thinking traces?.
What ties it together — and what you might not have expected — is that this mirrors how human experts actually reason: you remember a few successful moves vividly and concretely, but your failures collapse into general principles ("don't open with that"), not frame-by-frame memories. The machine result is that uniform consolidation isn't just inefficient, it's a category error — and the deeper hint from the memory-as-substrate work is that adaptive intelligence may *be* the structured, selective reuse of past inference, where what you choose to keep concrete versus abstract is the whole game Can cognition work by reusing memory instead of recomputing?.
Sources 7 notes
SkillRL demonstrates that treating successful episodes as concrete demonstrations and failures as abstracted lessons achieves state-of-the-art performance on complex tasks while using substantially less context than uniform approaches. The asymmetry mirrors human expert reasoning and avoids the degradation seen in uniform consolidation methods.
ReasoningBank shows that storing strategy-level reasoning hints from both self-judged successes and failures outperforms success-only memory and raw trajectory storage. Coupled with test-time scaling, memory and compute compound rather than substitute, creating a novel scaling law where accuracy improves through cumulative interaction history.
Error accumulation in context causes non-linear performance degradation in long-horizon tasks. Model scaling does not fix this; only test-time compute through thinking models reduces the effect by preventing error-contaminated context from biasing reasoning.
Reflexion demonstrates that unambiguous environmental feedback (success/failure) enables agents to write useful self-diagnoses and improve across episodes without parameter updates. The binary signal prevents rationalization, and keeping reflections uncompressed preserves their usability.
GRPO-RoC filters positive trajectories for quality while preserving diverse failures as negative signal, allowing a 14B model to reach frontier math performance in 510 RL steps, surpassing much larger models with cleaner reasoning.
Standard PRMs degrade on trajectory format because thinking traces include branching, backtracking, and weaker coherence than polished responses. ReasonFlux-PRM addresses this by supervising both trajectories and responses, treating failed steps as informative exploration rather than errors.
Memory-Amortized Inference proposes intelligence arises from structured reuse of prior inference paths over topological memory, inverting RL's reward-forward logic into cause-backward reconstruction. This duality explains energy efficiency and suggests memory trajectories form the substrate of adaptive thought.