Can better tools fix LLM document editing errors?
Does giving LLMs agentic tool access—like diffing, re-reading, or structured editors—improve their reliability on long-horizon document workflows? Understanding whether the problem is tool limitations or decision-making quality matters for reliability engineering.
A natural intuition for fixing LLM document corruption: give the model better tools. Let it diff its own output, re-read the file, call a structured editor instead of regenerating prose. The DELEGATE-52 evaluation tests this directly and finds that agentic tool access does not improve performance on the benchmark.
The finding rules out a class of proposed fixes. Tool wrappers, ReAct loops, and structured editing affordances are not addressing the failure mechanism — they are downstream of it. The degradation comes from the model's own decisions about what to change and how, not from limitations of the editing interface. A model that decides to flip a numeric value will flip it through any tool you give it.
This also disambiguates two senses of "agent." The first sense — LLM-plus-tools, where capability is gated on tool affordances — predicts that tool access should improve document workflows. The second sense — LLM-as-decider, where the model's judgment about what to edit is the bottleneck — predicts that tool access should be roughly orthogonal. The DELEGATE-52 result favors the second.
The implication for workflow design: reliability gains on long delegated work probably come from changing what the model decides (better prompting, verification loops, decomposition into smaller reversible steps) rather than from upgrading what it can act through. Tool engineering helps when capability is interface-limited; it does not help when capability is judgment-limited.
Inquiring lines that read this note 19
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Why do multi-turn conversations degrade AI intent and coherence?- What repair strategies work best at each level of Clark's ladder?
- How do insert-expansions and third position repair together cover full repair lifecycle?
- How do insert-expansions differ from third position repair in timing?
- Do architectural changes or training fixes better prevent agreement failures?
- What breaks when multiple agents share and revise the same artifacts?
- How should organizations redesign workflows if LLMs cannot solve optimization directly?
- Why do frontier models corrupt more documents than weaker models during workflows?
- What causes silent document corruption in long LLM workflows?
- How does model tier affect whether errors delete or corrupt document content?
- Can tool use or self-conditioning fix degradation in extended LLM workflows?
- Can delegation prevent silent corruption in long delegated workflows?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do frontier LLMs silently corrupt documents in long workflows?
Explores whether advanced language models introduce undetectable errors when delegated multi-step tasks, and whether degradation continues accumulating beyond initial rounds of processing.
same paper, the parent finding
-
Why do language models fail to act on their own reasoning?
LLMs produce correct explanations far more often than they produce correct actions. What causes this knowing-doing gap, and can training methods close it?
adjacent: knowledge does not transfer to action
-
Where does agent reliability actually come from?
Exploring whether LLM agent performance depends on larger models or on thoughtful system design choices like memory, skills, and protocols that shift cognitive work outside the model.
partial counterpoint: harnessing helps elsewhere
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- LLMs Corrupt Your Documents When You Delegate
- Towards a Science of Scaling Agent Systems
- Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words
- Solving a Million-Step LLM Task with Zero Errors
- Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
- Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose
- interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
- Why Do Multi-agent LLM Systems Fail?
Original note title
agentic tool use does not improve llm document-editing reliability — tools are not a fix for long-horizon delegation drift