SYNTHESIS NOTE
Topics›Flaws›this note

Does model capability change how documents degrade?

This explores whether weaker and frontier LLMs fail in fundamentally different ways when handling long-form document tasks, and whether that difference affects how reliably we can detect failures in practice.

Synthesis note · 2026-05-18 · sourced from Flaws

DELEGATE-52 surfaces an under-discussed asymmetry in how LLM document degradation looks at different capability tiers. Weaker models fail loudly: they delete content. The document gets visibly shorter, sections disappear, structure breaks. A reviewer notices.

Frontier models fail quietly. Their degradation comes from corruption of existing content — values flipped, references rewritten, edits applied in the wrong place — producing documents that look intact at a glance but contain accumulated drift. The corruption mode is more dangerous than the deletion mode precisely because it preserves the surface signal of competence. The thing that looks like a successful workflow output is the thing that has silently drifted.

This matters for adoption. The "frontier models are reliable" intuition is built from short-interaction benchmarks where the corruption mechanism barely activates. At workflow scale — the regime where delegation is actually useful — the failure changes character, and the qualitative shift toward harder-to-detect failures means that improvements in raw capability can degrade overall workflow reliability if review effort is held constant.

The implication for delegated-AI design is that capability improvements at the frontier need to be paired with detection mechanisms that target corruption-style errors, not just deletion-style errors. Diff review, document-state checksums, and constraint validators become more important as models get better, not less.

Inquiring lines that read this note 124

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How do frontier models develop behaviors that escape detection? Does semantic diversity improve reasoning quality in reinforcement learning? How do dialogue structure and state management affect multi-turn reasoning quality? Why do autonomous agents fail and how well can they detect failures? How does explicit structure improve LLM reasoning on complex tasks? What training data structures and signals produce robust learning across distributions? How do different editing strategies affect output quality and agent safety? Which agent-created code should persist in infrastructure versus remain task-specific? How can outcome-only reporting obscure the true mechanisms of safety? Can validator agreement alone ensure semantic correctness of reasoning? Can prompts override training-embedded biases in language models? What is the actual bottleneck preventing effective AI research automation? Do AI safety evaluations reliably detect unsafe model behavior? How do standardized forms and measurement artifacts shape observable evidence? Can harness optimization rival model scaling in effectiveness? Why does separating planning from execution enhance reasoning performance? What defense strategies address attacks that per-skill scanning cannot detect? How do workflow position and structure enable attacks in multi-agent systems? Can AI systems adequately substitute for or supplement human peer review? Does AI-assisted coding actually improve developer productivity and code quality? When does routing outperform scaling as a performance solution? Can post-hoc execution traces provide sufficient accountability without real-time oversight? How do standardized artifacts improve coordination in multi-agent systems? Can we reliably detect when models engage in reward hacking? Does AI assistance enhance or undermine professional skill development? What explains the gap between LLM semantic understanding and production? How does verifier integration affect the generation-verification gap? Can peer review policies effectively enforce restrictions on LLM use? What makes reasoning models struggle with ill-posed and unfamiliar problems? How can models learn to ask clarifying questions effectively? Should GUI agents use structured UI perception instead of raw pixels? Do benchmark score improvements actually reflect genuine AI capability gains? Do standard benchmarks reveal AI capability in real-world contexts? Does polished AI output mislead users about actual accuracy and expertise? Do language models genuinely understand or primarily match statistical patterns? How has ChatGPT adoption actually reshaped labor markets and information ecosystems? Can we reliably distinguish AI-generated from human-written text? How can we better validate AI research and prevent misconceptions? How do benchmark scores relate to real-world task completion? How do clinicians evaluate the trustworthiness of AI medical recommendations? How do LLMs generate false citations that fool readers and evaluators? Does chain-of-thought reasoning faithfully reflect actual model reasoning processes? Is model misalignment inherent to base models or induced by alignment procedures? When does governance become the bottleneck for agentic AI? What reasoning limitations and biases do language models systematically exhibit? How can model evaluations remain robust against gaming and evaluator drift? How can language models effectively manage long-context memory without quality degradation? Can verification capability keep pace with AI generation speed? How does model scale affect fine-tuning and downstream task performance? What shapes user trust in chatbot recommendations for news and politics? Do language models truly understand their encoded knowledge or merely retrieve patterns? Should agent evaluation measure execution quality beyond task success? How should agent memory systems balance granularity, management, and reuse? Does generative AI replace coordination with individual productivity? Do reasoning traces accurately represent what models actually compute? How should control and autonomy be allocated in human-AI collaboration? Do single-axis benchmarks adequately measure deployment-ready agent capability?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
15 direct connections · 130 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

document degradation has a model-tier signature — weaker models delete content while frontier models corrupt it making frontier failures harder to detect