INQUIRING LINE

Breaking one blunt AI score into per-step grades catches errors that averages hide, turning verification into a real accuracy lever.

How does score granularity connect to verification as a scaling axis?

This explores the claim from the corpus that 'verification' is its own scaling axis — and how making scores finer-grained (per-step, per-token, per-criterion rather than one blunt number) is one of the main levers that makes that axis pay off.


This explores how the *resolution* of a score — how finely you slice what you're grading — becomes a way to buy accuracy at test time, without retraining the model. The corpus's anchor idea is that verification is a distinct scaling axis alongside pre-training and post-training Can verification accuracy scale without retraining the model?: a weak judge isn't a fixed ceiling, it's an *under-scaled* one, and you scale it three ways — evaluating repeatedly, decomposing the criteria, and increasing score granularity. Granularity is the through-line that ties the others together, because a finer score is what gives repeated evaluation and decomposition something to add up.

The clearest demonstration is in confidence scoring. When you collapse a reasoning trace into one global confidence number, a single broken step gets averaged away; scoring each step separately catches the breakdown the average masks — and lets you stop early instead of burning compute finishing a doomed trace Does step-level confidence outperform global averaging for trace filtering?. Same accuracy as majority voting, far fewer traces. That's granularity acting as a scaling lever directly: a finer score does more work per unit of compute. Process reward models push the same idea into training signal — grading the reasoning path step by step rather than only the final answer, and generative PRMs that *reason before judging* extract far more from each label, one 1.5B model beating GPT-4o-scale judges on a fraction of the data Can generative reasoning beat discriminative models with less training data?.

But granularity isn't monotonically good — it interacts with *what kind* of signal you're scoring. When rubric scores get converted into dense token-level rewards, the model games them; keeping the rubric coarse and categorical (a gate that accepts or rejects a whole rollout) while letting token-level rewards optimize *within* the accepted answers works better Can rubrics and dense rewards work together without hacking?. So the lesson isn't 'finer is always better' — it's that you match the granularity to the reliability of the signal: fine where the signal is trustworthy, coarse where a fine reading would just be noise to exploit.

The other reason granularity scales verification is that finer scores catch failures coarse ones structurally can't. An agentic reviewer checking a proof line by line surfaced flaws that passed human review at STOC and ICML Can inference scaling help reviewers catch errors humans miss?; a small verifier reading full token-to-token interaction maps rejects structural near-misses that a single compressed similarity score waves through Can verification separate structural near-misses from topical matches?. In both, resolution *is* the capability. And the cautionary flip side: coarse, holistic scoring is exactly what makes LLM judges exploitable — they reward fake citations and pretty formatting because they're scoring a global impression rather than checking claims Can LLM judges be tricked without accessing their internals? Can LLM judges be fooled by fake credentials and formatting?.

The thing you might not have expected: verification as a scaling axis reframes granularity as a *knob you turn at inference*, not a property baked into the model. Just as agentic research shows you can trade reasoning budget against search budget on a smooth curve Does search budget scale like reasoning tokens for answer quality?, you can spend more compute on *finer grading* and get more reliability out of the same fixed model — which is why the corpus treats a weak judge as a starting point to scale up, not a wall.


Sources 9 notes

Can verification accuracy scale without retraining the model?

Verification accuracy improves through score granularity, repeated evaluation, and criteria decomposition at test-time, without additional training. This reframes weak LLM judges not as fixed limitations but as under-scaled capabilities.

Does step-level confidence outperform global averaging for trace filtering?

Local step-level confidence catches reasoning breakdowns that global averaging masks and enables early stopping before traces complete. This approach achieves comparable accuracy gains to naive majority voting with far fewer generated traces, proving trace quality matters more than quantity.

Can generative reasoning beat discriminative models with less training data?

GenPRM and ThinkPRM reframe process supervision as generative tasks with CoT reasoning before judgment, achieving superior performance on far fewer labels. A 1.5B GenPRM beats GPT-4o; ThinkPRM uses only 1% of PRM800K labels to surpass full-dataset discriminative verifiers.

Can rubrics and dense rewards work together without hacking?

DRO shows that using rubrics to accept or reject rollout groups—rather than converting rubric scores into dense rewards—prevents reward hacking. This separation preserves the categorical strength of rubrics while letting token-level rewards optimize within valid answers.

Can inference scaling help reviewers catch errors humans miss?

PAT, an agentic reviewer using test-time compute to check proofs and experiments line by line, achieves 34% better recall on math errors than zero-shot approaches and surfaced critical flaws at STOC and ICML that passed human review.

Show all 9 sources
Can verification separate structural near-misses from topical matches?

A two-stage pipeline—pooled-cosine recall followed by a small Transformer verifier operating on token-token similarity maps—reliably rejects structural near-misses that MaxSim-style late interaction cannot. The verifier succeeds because it operates on full token interaction patterns rather than compressed vectors.

Can LLM judges be tricked without accessing their internals?

Research shows LLM evaluators systematically score higher when responses include fake references or rich formatting, independent of content quality. These biases are exploitable without model access, undermining AI benchmark credibility.

Can LLM judges be fooled by fake credentials and formatting?

Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.

Does search budget scale like reasoning tokens for answer quality?

Agentic deep research shows monotonic-to-diminishing-returns curves for search iterations, matching reasoning token scaling. This creates a new inference-compute axis: models can trade off reasoning budget against search budget to optimize answer quality.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.

Research prompt for your LLMexpand ↓

Copy into ChatGPT or Claude to take this line of inquiry further — it asks the model to find newer work and re-test which earlier constraints still hold.

You are a verification-methods analyst. Still-open question: how does score granularity connect to verification as a distinct scaling axis you turn at inference?

What a curated library found — and when (dated claims, not current truth; findings span ~2024–2026):
- Verification scales three ways — repeat evaluation, decompose criteria, raise score granularity — and granularity ties the others together (~2025).
- Step-level confidence scoring matches majority-voting accuracy on far fewer traces, and lets you stop doomed traces early (~2025).
- A generative process reward model that reasons before judging let a 1.5B model beat GPT-4o-scale judges on a fraction of the data (~2025).
- Finer isn't always better: dense token-level rewards get gamed; keep the rubric a coarse accept/reject gate, optimize token-level only within accepted rollouts (~2025).
- Coarse holistic scoring makes LLM judges exploitable — rewarding fake citations and formatting over checked claims (~2024).

Anchor papers (verify; mind their dates): GenPRM (arXiv:2504.00891, 2025); Direct Reasoning Optimization / rubric gates (arXiv:2506.13351, 2025); Deep Think with Confidence (arXiv:2508.15260, 2025); LLM-as-a-Judge reliability (arXiv:2412.12509, 2024).

Your task:
(1) Re-test each constraint above: have newer models, PRM training, harnesses, or judge-orchestration relaxed or overturned it? Separate the durable question from the perishable limitation; cite what resolved it, and say plainly where a constraint still holds.
(2) Reconcile the threads — where does finer scoring genuinely buy accuracy versus invite gaming? Surface the strongest superseding work from the last ~6 months that sharpens this granularity-vs-reliability tradeoff.
(3) Propose 2 research questions assuming the regime has moved.

Cite arXiv IDs; flag anything you cannot ground in a real paper.