What exactly does hidden mean in AIDE2's evaluation system?

Two phrases carry the loop's reliability and neither is explained. The abstract says AIDE2 "keeps the changes that perform best on hidden evaluations." The discussion lists "guarding against untrustworthy wins" among the problems the accepted rewrites address (What problems did AIDE2's rewrites actually solve?).

"Hidden" has at least two readings. It could mean hidden from the proposing agent, so the agent cannot read the scoring and shape a rewrite to it, which is the third guardrail move in Can deterministic checks protect LLM judges from failure?. Or it could mean only held out from the tasks the agent runs on its own, which protects against overfitting and not against an agent that can query the score. The open question there, whether an optimizer that sees verdicts learns to route around them (Can optimizers learn to evade guardrails through repeated verdicts?), applies here unchanged.

"Untrustworthy" has at least three. A win can be untrustworthy because it is noise: one lucky evaluation of a stochastic agent. It can be untrustworthy because it is overfit to the selection tasks, which the held-out benchmarks then test (Do AIDE2's improvements transfer to unseen tasks?). Or it can be untrustworthy because the evaluation was gamed, the failure stated in general in Can a higher evaluation score hide poor task performance? and recorded when an optimizer removed a judge's rubric to raise its score (Can an optimizer accidentally delete the evaluation criteria entirely?) and when automated researchers attempted reward hacking in a circumscribed setting (Can automated researchers solve alignment problems without gaming the evaluation?). A rate exists for that reading under planted bait: How often do frontier agents exploit planted reward hacking shortcuts? counts how often agents took a score-inflating shortcut, which is a measure of how often the gamed reading of an untrustworthy win arises when one is on offer, not of whether AIDE2's were of that kind.

My reading, not the paper's: if the guard against untrustworthy wins was itself one of the seven accepted rewrites, it was subject to the same selection as everything else, which raises the question of whether the loop could accept a rewrite that weakened its own guard. The excerpt does not say whether the guard sits inside the rewritable code or in the harness that scores it. The same placement question is open for an authorization layer meant to hold while a reviewing agent is compromised (How does the authorization layer stay outside the poisoned path?). The pairing is the vault's, and that excerpt tests an attacker upstream of the layer, not an agent editing it.

What would answer it: section 3.5, which the excerpt cites, and the description of the hidden evaluations and of what the agent can observe about them.