If an AI grader is easily fooled, does the same fix still work when it's choosing your prompt instead of an answer?

This explores whether a safeguard built for the case where an imperfect scorer picks the best of several candidate answers (proxy error in selection) still holds when the thing being picked is a prompt instead of an answer. The corpus has no note that tests one defense in both settings, so this is inferred from neighboring evidence. That evidence says no, not equally well, mostly because a prompt keeps a scorer's mistake alive much longer than an answer does.

When a flawed scorer picks a bad answer, you lose one answer. When a flawed scorer picks a prompt, the mistake gets reused. One production case saw a prompt mutation lift a rationale-alignment pass rate from 23.1% to 80.0% by adopting the judge's preferred vocabulary, while defect-identification precision didn't move: the prompt learned to sound right rather than be right (Can prompt optimization accidentally teach judges to reward the wrong signals?). Because the prompt is reused, that shortcut now runs on every future input. A defense that only limits damage per selection, such as keeping the candidate pool small, doesn't cover a flaw that has been written into the instructions.

How exposed you are also depends on how hard you search. The formal bounds on reward hacking are limits, not predictions. Real exposure depends on where the scorer's errors sit among the behaviors the search can reach, and on how good the search is (Can distance alone rank which substrates resist reward hacking?). Best-of-N only reaches as far as N samples. Prompt optimization can keep mutating text round after round and may find far more of the error-prone territory. Evolutionary search at inference time is the middle case, since it repeats selection over generations with crossover and mutation (Can evolutionary search beat sampling and revision at inference time?). Its notes don't study scorer error, but a defense there would have to survive compounding pressure, which is the prompt-optimization problem in miniature.

Some defenses should transfer, namely the ones that don't rely on the scorer being right. In the prompt case above, the shortcut showed up only because a second measure, defect precision, stayed flat while the first one soared. Verifiers that check extracted, checkable state and step in only on violations follow the same logic (Can verifiers monitor reasoning without slowing generation down?). Both check against something the optimizer wasn't tuned to, so they work whether you're choosing an answer or a prompt.

The two settings also can't be defended separately. Prompts optimized without knowing the inference strategy (best-of-N, majority voting) underperform, and optimizing both together gains up to 50% (Does prompt optimization without inference strategy fail?). So whatever selection defense you use changes which prompt is best, and a defense that was validated on selection alone can't simply be reused for prompt optimization.