If you can keep poking a security filter without setting it off, what does its reply quietly teach you?

This explores what reply from a classifier or scanner gives an attacker enough information to learn how it behaves. The corpus gives the same answer in two settings: the classifier's own reaction to repeated probes. In the decoy-versus-genuine-object setting, Can repeated quiet probes separate decoys from genuine objects? shows the theory. If the two kinds of object respond differently to a probe, and those response patterns are known or learnable from feedback, enough quiet probes push classification error toward zero. The attacker doesn't need a single decisive test. They only need to keep poking without setting anything off and watch what comes back. The catch is that this result assumes idealized, independent responses. The note doesn't describe what the feedback channel looks like in a real system.

The scanner note gives the practical version. Can attackers evade skill scanners by refining individual skills? describes ColluSkill, which reaches 96% average attack success across six skill scanners. It uses each scanner's feedback on how suspicious a skill looks to refine that skill, round after round. The attacker isn't reconstructing the scanner's full response distribution here. They only need to find where its boundary sits for each piece, and the scanner's own feedback tells them.

This works because of a mismatch in what gets scored. Scanners rate skills one at a time, so the feedback attaches to units the attacker can edit independently. Each skill can be pushed below the suspicion threshold while the chain-level behavior stays intact. Feedback is most useful to an attacker when it is repeatable, tied to a piece they can change alone, and blind to how the pieces combine.

This is the same failure the reward-hacking notes describe from the other side. Does reward hacking always stem from the same failure? traces reward hacking across training, output selection, and prompt revision to one cause: optimizing against a signal that only partly represents the real task. A scanner that can be queried repeatedly is that kind of signal, and the attacker is the optimizer. The corpus doesn't cover defenses such as noisier feedback, limits on how often a scanner can be queried, or scoring whole chains. So it can show what the signal is, but not how to take it away.