INQUIRING LINE

An AI that keeps a fresh, shared skill library gets more capable with every run — not just every model upgrade.

Why do persistent, resynchronized artifacts compound harness capability gains?

This explores why keeping durable, regularly-updated working artifacts — playbooks, skill libraries, shared documents that get re-synced across an agent's runs — makes an agent's tooling (its 'harness') get better over time rather than plateauing.


This explores why keeping durable, regularly-updated working artifacts makes an agent's harness compound its capability instead of stalling — and the corpus suggests the answer is about where knowledge lives and how it's maintained, not raw model strength. The core mechanism is externalization: when hard-won know-how is written into a persistent store the agent consults, each run can build on the last without overwriting weights. VOYAGER shows this most cleanly — executable skills kept in an embedding-indexed library, with complex skills composed from simpler ones, let an agent learn continuously while dodging the catastrophic forgetting that weight-update methods suffer Can agents learn new skills without forgetting old ones?. The artifact is the memory; the model stays frozen and free.

But 'persistent' alone isn't enough — the resynchronization matters, because artifacts rot. The ACE framework's key move is treating context as an evolving playbook updated through generation-reflection-curation loops rather than full rewrites, precisely to prevent brevity bias and 'context collapse,' where compression quietly erases the details that made the artifact useful Can context playbooks prevent knowledge loss during iteration?. Why that discipline is non-negotiable becomes vivid in the failure case: frontier models silently corrupt about 25% of document content across long delegated relay chains, with errors compounding round after round and never plateauing Do frontier LLMs silently corrupt documents in long workflows?. Left unmanaged, artifacts compound decay just as readily as capability. Structured, incremental resync is what flips the sign.

The compounding is strongest when curation is treated as its own job. SkillOS decouples a trainable curator from a frozen executor, and the repository drifts away from generic verbose additions toward actionable execution logic and reusable cross-task meta-strategies — and that trained curator generalizes across different executor backbones Can a separate trained curator improve skill libraries better than frozen agents?. This is the deep reason artifacts beat raw scale: the capacity to *produce* useful harness edits is roughly flat across model tiers, while the capacity to *benefit* from them peaks in the middle — weak models never invoke the harness, very strong ones over-follow it Do stronger models always evolve harnesses better?. Capability gains come from the artifact ecosystem, not from a bigger brain reading it.

Structure in the artifact is what lets gains stack instead of blur. MetaGPT finds that standardized engineering documents outperform conversational coordination, with agents pulling from a shared environment rather than passing noisy messages Does structured artifact sharing outperform conversational coordination? — the same logic scales down to a single agent's harness. And when the persistent layer holds governance rather than skills, the effect repeats: an agent that recorded 889 governance events over 96 days followed rules encoded in the memory it actually consulted during decisions, which beat external policy precisely because it was resident in the loop Can governance rules embedded in runtime memory actually protect autonomous agents?.

The quietly surprising takeaway: the bottleneck on compounding isn't storage or retrieval fidelity — it's how work gets broken into artifact-sized pieces. Decomposition granularity gates everything downstream, with standard LLM decomposition hitting only 34% step-level recall and correcting the step count recovering 75% of the gains What blocks skill retrieval in task decomposition?. A related move, reconstructing memory by actively traversing and pruning a graph rather than fetching fixed chunks, buys up to 23% on reasoning tasks Can agents reconstruct memory on demand instead of retrieving it?. Persistent resynchronized artifacts compound because they turn one-shot cleverness into a maintained, composable asset — but only when something disciplined keeps re-cutting them to the right size and pruning the rot.


Sources 9 notes

Can agents learn new skills without forgetting old ones?

VOYAGER demonstrates that storing executable skills in an embedding-indexed library and composing complex skills from simpler ones allows agents to learn continuously while avoiding the forgetting that occurs with weight-update-based methods. Environmental feedback refines skills while an automatic curriculum drives continual exploration.

Can context playbooks prevent knowledge loss during iteration?

The ACE framework treats contexts as evolving playbooks using generation-reflection-curation loops rather than full rewrites. This prevents knowledge loss from compression and detail erosion, achieving +10.6% on agentic tasks and +8.6% on finance without labeled supervision.

Do frontier LLMs silently corrupt documents in long workflows?

Testing 19 models across 52 domains shows even advanced systems degrade documents by ~25% over extended relay tasks, with errors compounding silently without plateauing through 50 round-trips.

Can a separate trained curator improve skill libraries better than frozen agents?

SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.

Do stronger models always evolve harnesses better?

Model capability to produce useful harness edits stays constant across tiers, but capacity to actually benefit from those edits follows an inverted U-shape, peaking in mid-tier models. Weak models fail to invoke harnesses; strong models struggle with faithful instruction-following.

Show all 9 sources
Does structured artifact sharing outperform conversational coordination?

MetaGPT demonstrates that agents producing standardized engineering documents achieve superior coordination compared to conversational exchange. Active information pulling from shared environments eliminates noise and mirrors efficient human workplace infrastructure.

Can governance rules embedded in runtime memory actually protect autonomous agents?

A persistent agent recorded 889 governance events across 96 active days, with safeguards encoded directly into the memory layer the agent consulted during operation. Runtime-resident governance proved more effective than external policies because the agent actually accessed it during decision-making.

What blocks skill retrieval in task decomposition?

Standard LLM decomposition reaches only 34% step-level recall, gating retrieval success. Correcting step count recovers 75% of gains in iterative methods, shifting the bottleneck to representation-level reranking rather than vocabulary alignment.

Can agents reconstruct memory on demand instead of retrieving it?

MRAgent achieves up to 23% gains on reasoning tasks by reconstructing memory through active graph traversal that prunes paths based on accumulated evidence, while reducing token and runtime cost compared to fixed-retrieval pipelines.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.

Research prompt for your LLMexpand ↓

Copy into ChatGPT or Claude to take this line of inquiry further — it asks the model to find newer work and re-test which earlier constraints still hold.

You are a research analyst. Still-open question: why do persistent, resynchronized working artifacts make an agent's harness COMPOUND capability instead of stalling — and does that still hold?

What a curated library found — and when (dated claims, not current truth; findings span ~2023–2026):
- Externalization beats weight updates: know-how written to a persistent, indexed store lets each run build on the last while the model stays frozen, dodging catastrophic forgetting (~2023).
- But artifacts rot: treating context as an evolving playbook via generation-reflection-curation loops (not full rewrites) prevents brevity bias and 'context collapse' (~2025).
- Failure case: frontier models silently corrupt ~25% of document content over long delegated relay chains, errors compounding round after round with no plateau (~2026).
- Curation as its own job: a trainable curator decoupled from a frozen executor drifts repos toward actionable, reusable meta-strategies and generalizes across executor backbones (~2026).
- Tension: capacity to PRODUCE useful harness edits is roughly flat across model tiers, while capacity to BENEFIT peaks in the middle — weak models never invoke, strong ones over-follow (~2026). Graph memory reconstruction buys up to 23% on reasoning.

Anchor papers (verify; mind their dates): MetaGPT (2023), Agentic Context Engineering/ACE (2025), LLMs Corrupt Your Documents When You Delegate (2026), SkillOS (2026), Harness Updating Is Not Harness Benefit (2026).

Your task: (1) Re-test each constraint: for every finding, judge whether newer models, training, tooling, orchestration (memory, caching, multi-agent), or evaluation has RELAXED or OVERTURNED it; separate the durable question from the perishable limit, cite what resolved it, and name where a constraint still holds. (2) Surface the strongest CONTRADICTING or SUPERSEDING work from the last ~6 months — especially any disputing the flat-produce/peaked-benefit split or the 25% corruption rate. (3) Propose 2 research questions assuming the regime has moved.

Cite arXiv IDs; flag anything you cannot ground in a real paper.