If you keep quietly poking at a system, how many tries does it take to reliably tell fakes from the real thing?
How many probes does an attacker need to reach near-zero classification error?
This explores the decoy-detection setting, where an attacker sends repeated probes to tell fake objects from genuine ones, and asks how many probes it takes before the attacker's mistakes effectively vanish.
This explores the decoy-detection setting, where an attacker sends repeated probes to tell fake objects from genuine ones, and asks how many probes it takes before their mistakes effectively vanish. The corpus doesn't give a single number. It gives the shape of the answer: in the idealized case, enough quiet probes always work, and what changes is how the defender's countermeasures put a clock on the attacker.
The cleanest result is that with enough non-triggering probes, the attacker's classification error can be driven to zero. Can repeated quiet probes separate decoys from genuine objects? states this for independent responses, provided decoys and genuine objects respond differently. The attacker must also know those response patterns or be able to learn them from feedback. The note doesn't state a count. The usual statistical intuition applies, though it comes from general hypothesis testing rather than from the note: the more alike the two response patterns are, the more probes it takes to separate them. If they differ at all, patience wins.
The catch is the word 'quiet'. Those probes must not set off any alarm. When probing itself can trigger containment, the question changes from how many probes are enough to whether the attacker can collect them before being removed. How fast must a coalition gather observations before containment? frames this as a finite-sample race. A coalition of attackers has to accumulate enough observations before it gets shut out, so the defender's leverage is the time window, not the statistics. A decoy that triggers containment early makes each probe expensive and cuts off the path to zero error.
A related pattern shows up in a different attack setting. Can attackers evade skill scanners by refining individual skills? shows attackers using a defender's own feedback loop, refining malicious skills against six scanners until they reached 96% average attack success. That note gives no probe count either, but it makes the same point: every response the defender gives is information the attacker can use, and a defense that answers consistently teaches its own detector's blind spots.
The corpus has the limiting result, that error goes to zero when the distributions differ, and it has the defender's counter, which is to cap how long probing can go on. It has no formula for a specific number of probes, and the idealized independence assumption may not hold in practice.
Sources 3 notes
In idealized settings with independent responses, enough quiet probes let a classifier separate decoys from genuine objects with vanishing error if their response distributions differ and are known or learnable from feedback.
When probing triggers containment, the attacker's challenge shifts to sample accumulation against time. A finite-sample bound establishes how many observations a coalition needs before removal, pricing the separation in temporal constraints.
ColluSkill combines chain planning with scanner-feedback refinement to reach 96% average attack success. The approach works because scanners score skills individually, allowing feedback to reduce suspicion per skill while chain-level semantics remain intact.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- When Agents Talk: Honeytokens under Shared Memory
- Corrupt Plans, Clean Traces: Evading Chain-of-Thought Monitoring with Plan Injection
- ColluSkill: Adversarial Cross-Skill Composition for Evading Agent Skill Scanners
- Reinforcement Learning with Rubric Anchors
- Mechanisms of Introspective Awareness
- Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Tasks