Do agents with the same performance gain follow the same learning pathway?

PAST-Bench asks whether retained experience actually improves a personal agent over time, a question the paper says "has not been systematically tested." Its answer has two parts. Improvement is "real but uneven across capabilities," and "agents with the same headline gain can differ markedly in whether that gain is supported by evidence of the intended pathway." The conclusion compresses this to one line: "similar persistence gaps can hide different persistence paths." A later-task gain, taken alone, does not show that the agent saved something, retrieved it, and applied it.

The design follows from that worry. Each agent runs ordered sequences of fresh-session tasks under matched conditions that turn retained experience on and off, across 26 scenarios and 204 episodes covering memory, procedural reuse, information gathering, and update. The paper calls this a "performance-attribution benchmark": it pairs persistence-on and persistence-off runs within task families and reports mechanism evidence alongside task scores. The stated aim is to separate three contributions to later-task performance, namely the base model, the runtime, and the retained experience. A score that rises when persistence is on could come from any of the three, and only the pairing plus the mechanism check can tell them apart.

This adds a layer to Does a single benchmark score actually predict agent readiness?. That note argues that one score hides separate capabilities. PAST-Bench makes a narrower claim inside a single axis: two agents can tie on the same gain and still differ in whether it came through the intended pathway. The paper also treats "self-evolution" as capability-specific, so improvement on memory need not predict improvement on update. It is a close cousin of Why do LLM agents ignore condensed experience summaries?, which asks the same suspicious question, whether agents really use the experience they hold, and answers it with causal perturbations of that experience. PAST-Bench asks it at the level of the whole save, retrieve, update loop. In the terms of Do self-improving agents really split into two distinct loops?, the paper works in the fast scaffold layer, which it calls a "more immediate and operational layer" than modifying parameters, learning algorithms, or agent architectures.

The excerpt is silent on how mechanism evidence is scored, what Overall Δ and Mech measure beyond their names, and which seven models and four frameworks produced which gaps, so the size and frequency of the mismatch cannot be judged from it. The one intervention result it reports is modest and carries a caveat. Under MiniMax-M2.7, Hermes+ raises mean Overall Δ from +0.13 to +0.15 and Mech from 0.64 to 0.73, with its clearest gain on Update, but the paper says the +0.02 Overall difference is "smaller than run-to-run variation" and treats Hermes+ as "a diagnostic scaffold rather than a universal improvement." The excerpt does not say whether the Mech change clears that variation. What holds at this strength is a measurement point: a claim that a personal agent improves from experience needs the on-versus-off comparison and evidence of the pathway, and the headline gain alone falls short of both.