Even when a poisoned AI agent gets fooled, what stops the fooling from turning into a harmful action?

This explores what keeps the authorization layer (signed permission tokens tied to one task, plus a separately checked policy rulebook) from being corrupted by the memory poisoning that fools the agent itself. The corpus doesn't answer it directly. The one paper that reports this layer names the two components and stops. It doesn't say who issues the tokens, what they're bound to, how the policy check runs, or whether the attacks were even aimed at those components (How does the authorization layer stay outside the poisoned path?).

The corpus does show the result that makes the question worth asking. Memory poisoning got past the agent's Validator in every trial, yet with the authorization layer on, no unsafe action executed. The poisoning still worked on the agent's judgment. The layer only blocked the actions that judgment led to (Can memory poisoning compromise decision-making even with authorization layers?). The most likely explanation is that the layer never relies on the agent's judgment. It sits after the decision and checks the action against a signed token and an outside rulebook, so bad memory has nothing to persuade. That is an inference from the outcome, though. The evaluation doesn't show that attackers tried to forge tokens or corrupt the policy oracle.

The corpus also can't say which half does the work. The paper only compares both checks on against both off, with no test of either one alone (Which authorization component achieves the zero percent unsafe rate?). A related note has the same gap in a different setting. Clear authorization rules plus restricted tools gave zero protected-test modifications, but nothing separates "the agent couldn't cross the line" from "the agent wasn't told to" (Do authorization rules or restricted tools prevent test modifications?). Without those ablations, any account of isolation is a guess.

Several neighboring notes explain why placement matters. A model-level filter judges one output at one moment, while an agent's risk spreads across memory, retrieved content, and tool calls. Containment means controlling what the agent can touch, and that is the job a separate authorization layer would do (Can a model-level filter truly contain an agent with environment access?). Contamination can also enter upstream of inspection defenses. A crafted prompt can bias a planner before any workflow check runs (Can prompts alone reshape multi-agent workflows without system access?). A check that sits downstream of the agent's own reasoning inherits that exposure.

Two other approaches defend differently. Retrieval-time defenses like RAGPart and RAGMask try to clean the poisoned path itself by detecting suspicious documents (Can we defend RAG systems from corpus poisoning without retraining?). Another agent stored its governance rules in the memory layer it consults while working, on the grounds that it actually reads them (Can governance rules embedded in runtime memory actually protect autonomous agents?). Its rules therefore sit in the very place that memory poisoning targets, the opposite of keeping the guard out of the poisoned path. The corpus doesn't test that design against poisoning.