If you rewrite a paper's wording again and again, does an AI reviewer keep raising its score, or stop caring?
How do LLM reviewer scores respond when rewriting is applied recursively or jointly?
This explores what happens to an AI reviewer's score when a paper's wording is rewritten more than once, either in repeated passes that build on each other or as several rhetorical changes applied at the same time.
This explores what happens to an AI reviewer's score when a paper's wording is rewritten more than once, either in repeated passes that build on each other or as several rhetorical changes applied at the same time. The corpus doesn't answer that directly. It has evidence on single rewrites, but nothing that tests stacking or looping them.
A single rewrite already moves scores. Rewriting a manuscript's rhetoric while keeping the science identical shifts LLM reviewer scores measurably. Evidence framing and novelty stance produce the biggest swings, and the size of the effect varies by reviewer model and by the paper's original quality tier How much does rhetorical style shift AI review scores?. So the same rewrite doesn't do the same thing to every reviewer or every paper, and that unevenness is a reason not to assume repeated or combined rewrites add up smoothly. A similar pattern shows up in recommender prompting, where tricks that help cheap models can hurt top-tier ones Do prompt techniques work the same across all LLM tiers?.
Nearby work on LLM judges suggests why stacking is worth worrying about. Fake references and rich formatting raise scores regardless of content quality, and they need no access to the model's internals Can LLM judges be fooled by fake credentials and formatting? Can LLM judges be tricked without accessing their internals?. These cues don't depend on meaning, and independent cues are the kind you could combine. Whether layered cues add up, saturate, or cancel is untested in this collection. Treat it as an open question, not a finding.
Two other notes bear on how you'd measure it. Reading the expected value over scoring-token logits gives continuous scores instead of discrete ones with many ties Can reading logit distributions break ties in LLM judging?. Small cumulative drift across rewrite rounds would show up there, where coarse scores would hide it. A structured novelty pipeline that extracts claims, retrieves related work, and then compares Can structured pipelines make LLM novelty assessment reliable? reasons from claims rather than prose. That makes it a plausible contrast case for rhetoric-driven score inflation, though the note doesn't test it against rewrites.
Sources 6 notes
Rewriting manuscripts' rhetoric while preserving scientific content moves LLM reviewer scores measurably. Evidence framing and novelty stance produce the largest contrasts; effects vary by reviewer model and interact with the paper's original quality tier.
A 23-prompt benchmark across 12 LLMs shows rephrasing and background-knowledge prompts boost cheap models, while step-by-step reasoning reduces accuracy in high-performance models. Task structure, not generic best practices, determines which prompts help.
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
Research shows LLM evaluators systematically score higher when responses include fake references or rich formatting, independent of content quality. These biases are exploitable without model access, undermining AI benchmark credibility.
Computing the expectation over scoring-token logit distributions yields continuous verifier scores instead of discrete tokens, substantially reducing ties and improving discrimination between solutions without additional training or models.
Show all 6 sources
A three-stage pipeline (extract claims, retrieve related work, compare) reached 86.5% reasoning alignment and 75.3% conclusion agreement with human reviewers on 182 ICLR submissions, outperforming holistic LLM baselines.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge
- When Reject Turns into Accept: Quantifying the Vulnerability of LLM-Based Scientific Reviewers to Indirect Prompt Injection
- Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge
- Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solution
- Humans or LLMs as the Judge? A Study on Judgement Biases
- Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback
- ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
- The Thin Line Between Comprehension and Persuasion in LLMs