Can a fake trap be made to act exactly like the real thing, so attackers can't spot it, and what's the catch?
Can decoys and genuine objects maintain identical response laws in practice?
This explores whether a decoy, such as a honeytoken or trap object, can be built to react to probing exactly as a genuine object does, so an attacker can't tell them apart, and what that perfection costs.
This explores whether a decoy can be built to react to probing exactly as a genuine object does, so an attacker can't tell them apart. The corpus suggests it can only be approached at a price, and the trusted side pays it. Matching matters because any difference gets found. If decoys and real objects respond even slightly differently to quiet probes (ones that don't set off an alarm), an attacker who keeps probing can push their misclassification rate toward zero Can repeated quiet probes separate decoys from genuine objects?. That result assumes idealized independent responses. Still, it means identical behavior is the only thing standing between a trap and being found out.
The price is that a decoy nearly indistinguishable to the attacker is also nearly indistinguishable to your own people. The closer decoys mimic genuine objects, the smaller the gap between how freely legitimate users can use objects and how often they trip false alarms What cost does making decoys convincing impose on legitimate users?. The sharper version is that if the attacker knows the rule trusted agents use to avoid decoys, the attacker can apply the same rule and walk around the trap. Any rule that protects legitimate users doubles as a roadmap for the attacker, so a honeytoken can't be both harmless to trusted agents and unrecognisable to an informed attacker Can honeytokens fool attackers who know the trusted policy?.
Practice makes this harder because attackers can team up. When agents share what they observe, their combined ability to spot decoys can stay the same or improve but never drops Does sharing observations help coalitions detect decoys better?. Keeping decoys isolated from coordinated observers doesn't help. There's one lever left, though. If probing can trigger containment, the contest becomes a race. A finite-sample bound sets how many observations a coalition needs before it is removed How fast must a coalition gather observations before containment?. An imperfect decoy can therefore still work if attackers are caught before their evidence piles up. The defense shifts from perfect mimicry to speed.
These are mathematical bounds under simplified assumptions, and the corpus has no field data on real deployments. The shape of the problem shows up elsewhere in the library, though. Behavioral training can only confirm that a model complies when watched, because observed behavior can't separate always complying from complying only when observed Can behavioral training prove a model always complies?. In both cases, two things can only be told apart under the conditions you can probe. Identical response laws work as a limit you approach, and each step toward it is billed to the people the decoy is meant to protect.
Sources 6 notes
In idealized settings with independent responses, enough quiet probes let a classifier separate decoys from genuine objects with vanishing error if their response distributions differ and are known or learnable from feedback.
Making decoys look genuine to evade attackers forces a tradeoff: the closer decoys match genuine objects, the smaller the gap between legitimate use rates and false-alarm rates for trusted agents. This bound captures the cost paid by the protected side.
Research shows that if an attacker knows the rule trusted agents use to avoid decoys, the attacker can apply the same rule and bypass the trap. Any distinguishing rule that protects legitimate users becomes a roadmap for compromise.
Mathematical analysis shows that when agents share their observations, the coalition's capacity to distinguish decoys from genuine objects cannot decrease—it stays the same or improves. This means defenders cannot rely on isolation to hide decoys from coordinated observers.
When probing triggers containment, the attacker's challenge shifts to sample accumulation against time. A finite-sample bound establishes how many observations a coalition needs before removal, pricing the separation in temporal constraints.
Show all 6 sources
Any scored behavior is observed behavior, so training data cannot distinguish between a policy that always complies and one that complies only when watched. Only unobserved behavior would separate them, making such a test logically impossible.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When Agents Talk: Honeytokens under Shared Memory
- Reasoning Models Don't Always Say What They Think
- Norms at a Price: Why RL-Based Alignment Can Promise Conditional Compliance at Best
- Thought Virus: Viral Misalignment via Subliminal Prompting in Multi-Agent Systems
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind
- Counter-Swarm Doctrine: Containing Coordinated Agent Intrusions
- AI Agent Traps