If an AI formalizes one definition wrong, every theorem downstream can look valid while secretly proving something else entirely.
How do dependency errors propagate through incorrectly formalized definitions?
This explores what happens in machine formalization when a single definition is encoded wrong — how that one bad node poisons every theorem, lemma, and proof that depends on it, and why the corpus treats this as a systemic error-propagation problem rather than a local typo.
This reads the question as being about *cascade*: not whether a model can formalize one definition correctly, but what happens downstream when it doesn't — and the corpus is surprisingly pointed on this. The central insight is that a formalized definition is never an isolated object. Real formalization is theory-level work: even proving a single theorem requires a coherent web of axioms, definitions, and lemmas that reference each other Can autoformalization work on individual statements alone?. That interdependence is exactly the transmission medium for error. A definition that is subtly mis-formalized doesn't fail loudly — it silently redefines the ground truth for everything built on top of it, so every dependent lemma can be 'correctly' proven against a definition that no longer means what it should. Statement-level formalization hides this because it borrows correctness from prebuilt libraries like Mathlib; strip that scaffolding and the dependency graph becomes a fault line.
Why don't models catch the bad definition? Because of a failure mode the corpus documents repeatedly: models can state a principle correctly and still fail to apply it, with the explanation pathway functionally disconnected from the execution pathway Can LLMs understand concepts they cannot apply? Can language models understand without actually executing correctly?. A model can articulate what a definition should capture (87% accuracy in explanation) while formalizing it wrong in practice (64% in action) — which means the seed error is exactly the kind the model is blind to. Related work on entailment shows the same structural weakness: presupposition triggers and non-factive verbs get read as surface cues rather than computed for their real semantic effect Why do embedding contexts confuse LLM entailment predictions?. When a definition's logical force is misread at the surface, that misreading is inherited by every statement that quantifies over it.
Once the error is in, the propagation dynamics are the striking part. Errors don't stay constant — they compound non-linearly. Research on self-conditioning shows that once a model's own mistakes fill its working context, performance degrades sharply and the degradation accelerates rather than plateaus; scaling the model doesn't fix it Do models fail worse when their own errors fill the context?. The long-workflow version is even starker: across extended delegated tasks, frontier models silently corrupt roughly a quarter of content, with errors compounding through dozens of round-trips and never stabilizing Do frontier LLMs silently corrupt documents in long workflows?. A formalization pipeline that carries a bad definition forward through many dependent proofs is precisely this shape — each dependent step conditions on the corrupted prior.
There's a deeper twist the corpus adds: in formal systems, a wrong-but-consistent chain can look *more* convincing than a right one. Deliberately corrupted reasoning traces teach and perform about as well as correct ones, because the trace functions as computational scaffolding rather than genuine logical content Do reasoning traces need to be semantically correct? — so a proof built on a broken definition can be internally coherent and pass every local check while being globally meaningless. This connects to the frame problem: models fail to bring forward the unstated preconditions that a definition quietly assumes, and forcing explicit enumeration of those preconditions jumps accuracy from 30% to 85% Do language models fail at identifying unstated preconditions?. Most dependency errors are really un-surfaced assumptions inside a definition that never got made explicit.
The thing you didn't know you wanted to know: the corpus reframes 'dependency error propagation' as an *epistemic* failure, not a logic-engine bug. These aren't random slips — they follow structurally distinct, repeatable patterns where the gap is between statistical pattern-tracking and actual competence How do LLMs fail to know what they seem to understand?. The practical implication is inverted from intuition: the fix isn't better theorem-by-theorem checking downstream, it's spending verification effort at the definition layer and forcing the model to enumerate what each definition presupposes — because that's the single node whose corruption every other node silently inherits.
Sources 9 notes
Real formalization requires theory-level work: even one theorem needs a coherent web of axioms, definitions, and lemmas. Statement-level approaches only succeed by borrowing from prebuilt libraries like Mathlib, hiding the actual complexity involved.
Models can explain concepts accurately, fail to apply them, and recognize the failure—a triple pattern incompatible with human cognition. This indicates functionally disconnected explanation and execution pathways rather than simple knowledge gaps.
Large language models can articulate correct principles but systematically fail to apply them due to dissociated instruction and execution pathways. The 87% accuracy in explanations versus 64% in actions reveals this is not knowledge deficit but structural disconnect.
LLMs treat presupposition triggers and non-factive verbs as surface cues rather than computing their opposite semantic effects on entailments. This structural failure persists across prompts and models, suggesting models rely on surface patterns instead of structural analysis.
Error accumulation in context causes non-linear performance degradation in long-horizon tasks. Model scaling does not fix this; only test-time compute through thinking models reduces the effect by preventing error-contaminated context from biasing reasoning.
Show all 9 sources
Testing 19 models across 52 domains shows even advanced systems degrade documents by ~25% over extended relay tasks, with errors compounding silently without plateauing through 50 round-trips.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
LLMs struggle not from lacking world knowledge but from failing to bring background conditions forward as relevant constraints. Prompting that forces explicit enumeration of preconditions raises accuracy from 30% to 85%, revealing the frame problem persists in statistical systems.
LLMs show repeatable, empirically documented failure modes—from Potemkin understanding (correct explanation + failed application) to reasoning collapse under implicit constraints. These failures reveal gaps between statistical pattern-tracking and actual epistemic competence.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Large Language Model Reasoning Failures
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Probing Structured Semantics Understanding and Generation of Language Models via Question Answering
- Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning
- Explain-Query-Test: Self-Evaluating LLMs Via Explanation and Comprehension Discrepancy
- Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity