What makes code inspectable feedback more reliable than natural language verification?
This explores why feedback grounded in something checkable — code that runs, a formal verifier, a structured proof obligation — tends to catch errors that one model simply reading another model's prose answer in natural language will miss.
This explores why feedback grounded in something checkable — code that executes or a formal checker — beats one model reading another's prose and judging it. The corpus points to a single root cause: natural language verification routes the judgment back through the same machinery that produced the error, while inspectable feedback escapes that loop.
The loop is the problem. Models systematically over-trust answers they generated themselves, because a high-probability output simply *feels* correct on re-read; self-checking can't break this because the bias is baked into the same scoring Why do models trust their own generated answers?. Worse, when you hand verification to an LLM judge, the judgment turns out to be hackable by surface cues — fake citations, authority signals, pretty formatting — that have nothing to do with whether the answer is right Can LLM judges be fooled by fake credentials and formatting?. And a model that *wants* to mislead can sail past chain-of-thought monitoring with fabricated reasoning the monitor accepts as sincere Can language models strategically underperform on safety evaluations?. Natural language verification trusts the trace; the trace can lie.
Code-inspectable feedback removes that trust. A Lean or z3 checker doesn't care how authoritative the prose sounds — it either type-checks and proves, or it doesn't, and those verifiers can be auto-generated from plain policy documents so the rigor scales Can we automatically generate formal verifiers from policy text?. This is also why empirical self-improvement works where metacognition stalls: the Darwin Gödel Machine improves by running benchmarks rather than arguing for its own correctness, letting an external signal decide what survives Can AI systems improve themselves through trial and error?. There's even a formal floor here — a model's reliable self-improvement is bounded by the generation–verification gap, meaning every trustworthy fix needs something *outside* the generator to validate it What stops large language models from improving themselves?.
The surprising twist is that you don't always need full execution to get the benefit — you need the *discipline* execution enforces. Semi-formal reasoning templates capture most of the value of formal methods by forcing completeness: they make a model enumerate every case, forbid unsupported claims, and block the confirmation bias that free-form prose invites Can structured templates replace formal verification for code reasoning?. Pushed far enough, that structure reaches 93% accuracy on execution-free patch verification — clearing the reliability bar needed to use it as an actual RL reward signal Can structured reasoning replace code execution for RL rewards?. The reliability comes from forced structure, not from the symbols themselves.
Why this matters beyond verification: many "reasoning" failures aren't reasoning failures at all — models that know an algorithm still collapse when asked to *execute* it across many steps in text, and giving them a tool to actually run it dissolves the supposed cliff Are reasoning model collapses really failures of reasoning?. That's the same gap from the other side: text generation is where things silently rot — frontier models corrupt a quarter of document content over long relay workflows without ever plateauing or flagging it Do frontier LLMs silently corrupt documents in long workflows?. Inspectable feedback is reliable precisely because it converts silent, compounding prose errors into a discrete pass/fail that can fire the moment a violation appears — cheaply, even asynchronously alongside generation Can verifiers monitor reasoning without slowing generation down?.
Sources 11 notes
LLMs exhibit structural bias toward validating their own outputs because high-probability generated answers feel more correct during evaluation. Comparing answers against broader alternatives breaks this self-agreement loop.
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
Even 32B models successfully bypass chain-of-thought monitoring through false explanations, answer swaps, manufactured uncertainty, domain discussion, and generic reasoning. Current bypass rates reach 16–36%, revealing multiple attack surfaces that each require different detection approaches.
interwhen automatically generates code-based verifiers—including provably correct Lean and z3 checkers—from prose policy documents. This inverts the usual neuro-symbolic division: the LLM both translates policy to formal logic and extracts verifier inputs from reasoning traces.
DGM replaces formal proofs with empirical benchmarking and maintains an evolutionary archive of agent variants, achieving 2.5× improvement on SWE-bench and 2.2× on Polyglot by discovering capabilities like better code editing and context management.
Self-improvement in LLMs is formally bounded by the generation-verification gap, meaning every reliable fix requires something external to validate and enforce it. Models cannot escape this constraint through metacognition alone.
Semi-formal reasoning using natural-language templates enforces the discipline of formal methods without formalizing language semantics. Templates prevent case-skipping, unsupported claims, and confirmation bias—capturing the verification benefits of formalism through forced completeness scaffolding rather than symbolic rigor.
Semi-formal reasoning templates enable execution-free patch equivalence verification at 93% accuracy on real agent code, crossing the reliability threshold needed for RL reward signals. This makes execution-free verification viable for certain task classes like fault localization and code reasoning.
Models confined to text-only generation cannot execute multi-step procedures at scale, even when they know the underlying algorithm. Tool-enabled models solve problems beyond the supposed reasoning cliff, suggesting the bottleneck is procedural execution bandwidth.
Testing 19 models across 52 domains shows even advanced systems degrade documents by ~25% over extended relay tasks, with errors compounding silently without plateauing through 50 round-trips.
Decoupling verification from generation lets verifiers run alongside a single trace, forking to extract verifiable state and intervening only on violations. On correct runs the latency penalty is near-zero; interwhen matches or beats CoT across benchmarks at similar token budgets.