Give an AI reviewer more time to check line by line, and it can catch real errors top conferences let through.

This explores whether giving an AI reviewer more thinking time and more checking steps lets it find real errors in papers that expert human reviewers let through. The corpus has one direct answer, and it is a qualified yes. An agentic reviewer called PAT spends extra inference compute walking through proofs and experiments line by line. It found 34% more mathematical errors than a model asked to review in a single pass, and it found critical flaws in papers already accepted at STOC (a top theoretical computer science conference) and ICML (a major machine learning conference) Can inference scaling help reviewers catch errors humans miss?. Human reviewers don't miss these errors because they can't understand them. They miss them because line-by-line checking is tedious, and few people have time to rework every step of a 40-page appendix. Extra compute is good at exactly that kind of patient work.

The word doing the work there is *agentic*. More raw thinking time is not the main source of the gain. The gain comes from a structure that gathers evidence and checks it. An agent-based evaluator that actively collects evidence was about 100 times more stable in its verdicts than a plain LLM judge Can agents evaluate AI outputs more reliably than language models?. A paper-writing system improves reliability by separating the model's judgment from steps that can be run and checked mechanically Can separating judgment from verification improve research paper reliability?. The same pattern shows up in reasoning research. Models that 'collapse' on long problems often know the right method but can't carry out hundreds of steps as plain text, and tools let them push past that limit Are reasoning model collapses really failures of reasoning?. Compute also only pays off when the model was trained to use it well: non-reasoning models don't catch up to reasoning models however many tokens they get Can non-reasoning models catch up with more compute?. Search steps follow a similar curve with diminishing returns Do search steps follow the same scaling rules as reasoning tokens?.

Here is the twist. Catching flaws humans miss is not the same as being a good reviewer. AI reviewers show a 'hivemind' effect: they agree with each other far more than human reviewers do, so their blind spots are shared rather than offset. Simply rewording a paper, with no change to the science, raised AI scores by almost half a point Can AI systems safely replace human peer reviewers?. Reasoning models also tend to fail on unfamiliar *instances* rather than on hard ones Do language models fail at reasoning due to complexity or novelty?. That is a worry for review, because the papers that matter most are the novel ones. So machines and humans fail in different places. Machines are strong at exhaustive checking and weak at judging novelty. Humans are the reverse, and that same gap let an AI-generated paper pass a workshop's blind review before its own authors later found a citation error in it Can AI-generated papers pass peer review undetected?.

That points to the most promising setup: machine checking that supports human reviewers rather than replacing them. In a randomized trial at ICLR 2025, optional AI feedback on reviews led 27% of reviewers to revise, and independent raters judged the revised reviews more informative Can LLM feedback help peer reviewers improve their own reviews?. Proposals to repair conference review already treat it as a shared-accountability problem between authors, reviewers and venues Can two-stage review and badges fix AI conference peer review?. An inference-scaled checker fits naturally as a fourth party that does the tedious verification. The surprise is which flaws inference scaling catches. They aren't subtle insights beyond human reach. They are mechanical errors that humans could find but rarely have time to look for.