When an AI team learns from blaming one member for a mistake, does that lesson still help on a different task?
Can gradients extracted with agent-level supervision transfer across different benchmarks?
This explores whether the correction signals you get from blaming a specific agent for a failure (often written as text critiques, sometimes called 'textual gradients') keep working when you move them to a different benchmark than the one they came from.
This explores whether correction signals that blame a specific agent for a failure, often written as text critiques rather than numbers, still work on a different benchmark than the one they came from. The corpus has no note that runs that test directly, so it can't give a yes or no. What it does have is evidence on which kinds of learned signals travel between benchmarks and which stay tied to where they were learned.
The strongest evidence for transfer comes from system-level improvements rather than model-level ones. StateM's runbook was tuned around frozen weights, yet it carried over to newer models without modification and lifted several models on Terminal-Bench 2.1 Can execution harnesses lift model performance without retuning weights?. AIDE2's gains held on four held-out benchmarks, including weather forecasting, which sits outside the distribution it was selected on Do AIDE2's improvements transfer to unseen tasks?. A language world model trained on next-state prediction transferred across seven domains Can language models learn to simulate agent environments?. In each case, what was learned was about how the system operates, not about one task's answers, and that seems to be what lets it move.
Text-form feedback also has a practical advantage. The survey framing calls it the fast non-parametric loop: updates to prompts, memory and tools are cheap and reversible compared with weight updates Do self-improving agents really split into two distinct loops?. If a critique from benchmark A hurts on benchmark B, you can remove it. The direct evidence on portability is thin, though. Reflexion shows a plain success or failure signal is enough for an agent to write useful self-diagnoses, with gains across episodes Can agents learn from failure without updating their weights?. AgentFly reports 87.88% on GAIA validation from memory operations alone Can agents learn continuously from experience without updating weights?. Neither note says those learned lessons carry over to a new benchmark.
The main risk is that a signal distilled from one benchmark inherits that benchmark's boundaries. Agents trained on static expert demonstrations are capped by what the curators imagined, not by what the agent could do Can agents learn beyond what their training data shows?. A critique like 'this agent should re-check file paths' may only make sense in one environment. My inference, not a finding in the notes, is that signals derived from trajectory structure are the more portable candidates. Tree topology, tool-call positions and unread-but-retrieved distractors don't depend on task content, and they can replace hand-annotated process rewards Can trajectory structure replace hand-annotated process rewards? Can search agent behavior yield reliable process rewards for reasoning?. Another hint is that long-horizon success was predicted by persistence in the feedback loop, not by the quality of the first attempt What predicts success in ultra-long-horizon agent tasks?. That suggests the loop may transfer better than any specific correction it produces.
The clean test would be to extract the signals on benchmark A, apply them on benchmark B, and compare against signals extracted fresh on B. Nothing here reports that comparison for agent-level blame.
Sources 10 notes
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
The paper reports that AIDE2's improvements transfer to four held-out benchmarks spanning machine learning, algorithm engineering, and physics-based weather forecasting—the last being outside the selection task distribution. This demonstrates transferable gains beyond overfitting to the selection set.
Qwen-AgentWorld demonstrates that native language world models trained via next-state prediction on 10M+ trajectories outperform real-environment training on three benchmarks and transfer across seven domains, positioning next-state prediction as a foundation objective for agents.
A survey framework organizes self-improving agents into two update mechanisms: slow parametric loops updating foundation model weights, and fast non-parametric loops updating prompts, memory, and tools. Recent progress concentrates in the fast loop because scaffold updates are cheaper and reversible than weight updates.
Reflexion demonstrates that unambiguous environmental feedback (success/failure) enables agents to write useful self-diagnoses and improve across episodes without parameter updates. The binary signal prevents rationalization, and keeping reflections uncompressed preserves their usability.
Show all 10 sources
AgentFly formalizes agent learning as a Memory-augmented MDP with three memory modules (case, subtask, tool) that enable credit assignment and policy improvement entirely through memory operations. The approach achieved 87.88% on GAIA validation without modifying LLM parameters.
Agents trained on static expert datasets cannot learn from their own failures or generalize beyond demonstrated scenarios because they never interact with environments during training. Competence is capped by what curators imagined, not by agent capacity.
Tree-GRPO, Supervised RL, and ToolPO each convert sparse outcome rewards into dense step signals by exploiting different structural features—tree topology, expert-aligned actions, and tool-call positions—eliminating the need for annotated process reward models.
LongTraceRL mines entity-level reasoning signals from what search agents read but don't cite—the hardest distractors—and applies rubric rewards only to correct answers, structurally blocking reward fabrication while capturing intermediate reasoning quality.
Across 17 frontier models on 36 expert-curated optimization tasks, repeated benchmark-edit-incorporate cycles within a wall-clock budget proved the dominant success predictor. Most models terminated early or burned budget unproductively; Claude Opus 4.6 stood out as persistent.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Agent Learning via Early Experience
- AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs
- Useful Memories Become Faulty When Continuously Updated by LLMs
- StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
- LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards
- Dream-RSI: Recursive Self-Improvement through Evolving Worlds
- Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
- Reasoning Language Models: A Blueprint