Can AI convert a math theorem into formal logic without quietly changing what it actually means?

This explores whether LLMs can translate theorems faithfully into formal logic, and the collection's answer is largely no — not because the models produce broken syntax, but because they lose meaning while looking correct. The most direct evidence: LLMs generate well-formed logical expressions that are semantically wrong, with errors clustering exactly where fidelity matters most — scope ambiguity, quantifier precision, and how finely a predicate is drawn Can large language models translate natural language to logic faithfully?. A theorem is a machine of quantifiers and scopes; get those subtly wrong and you've written a different theorem that happens to read the same.

Why does this happen? A cluster of notes suggests the problem is that LLMs reason through meaning-by-association rather than symbol manipulation. When you decouple semantic content from the logical structure of a task, performance collapses even when the correct rules are sitting right there in context Do large language models reason symbolically or semantically?. Faithful formalization is precisely the operation of stripping semantics down to symbols — the thing these models are worst at. Worse, they lean on statistical mass from pretraining: given equivalent phrasings, they prefer the textually frequent one regardless of meaning Do language models really understand meaning or just surface frequency?, and their judgments track whether a proposition was attested in training rather than whether it actually follows Do LLMs predict entailment based on what they memorized?. Translation faithfulness requires the opposite reflex: honor the source even when it's unusual.

Here's the thing you might not expect — the task is bigger than the sentence. Faithful autoformalization can't work statement-by-statement at all: even a single theorem needs a coherent web of axioms, definitions, and lemmas, and the statement-level successes you see mostly borrow that scaffolding from prebuilt libraries like Mathlib, hiding the real work Can autoformalization work on individual statements alone?. So "translate this theorem" is quietly asking the model to reconstruct an entire theory's meaning, which raises the fidelity bar rather than lowering it.

There's also a subtler failure the corpus names: a model can explain a concept correctly, then fail to apply it, and even recognize its own failure — a pattern of "potemkin" understanding where explanation and execution run on disconnected pathways Can LLMs understand concepts they cannot apply?. A model that can define a quantifier fluently but misplaces it in formalization is exhibiting exactly this. One hopeful wrinkle from the same failure literature: the errors are asymmetric — models understand formal language better than they generate it Can large language models translate natural language to logic faithfully?, which points toward verification-in-the-loop (check the candidate formalization) rather than trusting one-shot generation.

And the ceiling is partly formal, not just engineering. Any computable LLM must produce wrong outputs on infinitely many inputs, and internal self-correction can't erase that — external safeguards aren't optional Can any computable LLM truly avoid hallucinating?. For theorem translation that's oddly good news: formal logic comes with a proof checker, an external oracle that can catch the semantic slips generation can't avoid. Faithfulness, on this reading, won't come from a better translator alone — it comes from pairing the translator with something that can verify the translation preserved the theorem.