Machine-checked proofs are now nearly free to run — so why does someone still need to double-check they're proving the right thing?

This explores whether machine-checkable proofs can close the gap between *checking* (is this proof valid?) and *auditing* (does this proof establish what we actually care about?). The corpus says no. Formal systems don't remove the gap. They move it, and they make the remaining human work denser. The clearest evidence is a 2026 OpenAI corpus in which proof checking became essentially free, but for every formal statement that needed a human expert to confirm it meant what it was supposed to mean, there were 379 machine-checked proofs Does free proof checking actually reduce verification burden?. The proof kernel tells you the logic holds. It can't tell you whether you formalized the right claim. So abundant verification turns into scarce adjudication: checking scales, and expert judgment becomes the bottleneck.

The gap gets wider when you look at what formalizing really involves. One theorem needs a whole supporting web of axioms, definitions, and lemmas. Approaches that formalize one statement at a time only look like they work because they quietly lean on prebuilt libraries like Mathlib Can autoformalization work on individual statements alone?. Each of those borrowed definitions is another place where the formal version could drift from the intended meaning, and nobody audits most of them. The same issue shows up outside mathematics. Systems like interwhen can now generate provably correct Lean and z3 checkers directly from prose policy documents Can we automatically generate formal verifiers from policy text?. But the LLM does the translating, so the hard question becomes whether the translation was faithful. The checker is only as trustworthy as the step that wrote it.

Mathematicians have reached the same conclusion from a different direction. The Leiden Declaration puts responsibility for correctness on human authors alone. Its reason is that a proof does two jobs: it establishes certainty, and it conveys understanding. Formal verification can deliver the first but not the second Can AI-generated proofs ever replace human mathematical understanding?. Read that way, auditing isn't leftover checking that we haven't automated yet. It's a different activity: deciding what a result means and whether it matters.

The agent-safety literature points to a practical middle ground. It doesn't try to close the gap. It designs around it. One approach orders cheap, unarguable mechanical checks before the contestable judgments, and plants known-bad cases as alarms Can deterministic checks protect LLM judges from failure?. Another checks intermediate steps of a long reasoning trace as it is generated, not only the final answer, which raised task success from 32% to 87% Where do reasoning agents actually fail during long traces?. These methods have limits too. Checks that look at one action at a time can't express rules about sequences of actions, so some violations only appear across a history of individually acceptable steps Can stateless checks ever catch sequence-level constraint violations?. Cryptographic commitments can prove that a record wasn't tampered with without revealing what it says. That separates proof from disclosure, but someone still has to read the content to audit it Can commitments protect sensitive agent data while enabling verification?.

The takeaway: each time checking gets cheaper, the bottleneck moves to the question of whether we asked the right question. The Darwin Gödel Machine shows how far that pressure can go. It gave up on formal proofs entirely and validated its self-improvements with benchmarks Can AI systems improve themselves through trial and error?. That trades certainty about the wrong thing for evidence about the right thing. It's a revealing choice, though it doesn't settle the matter.