If the AI watching for bad behavior learned from the same data as the AI it watches, will it miss the same things?
Do correlated training sources between monitors and agents undermine detection reliability?
This explores whether a monitor (an AI that watches an agent for bad behavior) can be trusted when it learned from the same data as the agent it watches, since they might share the same blind spots.
This explores whether a monitor that learned from the same data as the agent it watches can be trusted, since the two might share blind spots. The corpus has no paper that tests this head-on, so what follows is inference from neighboring findings. They point toward yes, it probably weakens detection, and they also suggest what to do about it.
The core worry is that training data sets a ceiling. One note argues that agents trained on static expert demonstrations are capped by what the curators imagined, not by the agent's capacity (Can agents learn beyond what their training data shows?). If a monitor and an agent grew up on overlapping data, the monitor's idea of what counts as suspicious was drawn from the same imagination. A failure the agent never saw is one the monitor never learned to flag. A second note makes the same point from the detection side. Entity co-occurrence statistics from pretraining data caught hallucination risk even when the model was highly confident (Can pretraining data statistics detect hallucinations better than model confidence?). A model's own confidence is least trustworthy exactly where its training data is thin, and a monitor with the same gaps would be confidently wrong in the same places.
There is also a hint that a model is on especially familiar ground when reading its own kind of output. Post-trained models show 3-4x lower output entropy on-policy and behave as if they recognize their outputs as their own actions (Do models recognize their own outputs as actions shaping future inputs?). That note doesn't study monitoring. My hypothesis is that a monitor from the same lineage would find the agent's outputs unremarkable, because they look like what it would have written itself.
The corpus also suggests the fix is to make the monitor depend less on its own training-shaped judgment. An agentic evaluator that collected external evidence showed 0.27% judge shift, against 31% for a plain LLM-as-a-Judge (Can agents evaluate AI outputs more reliably than language models?). That result says nothing about shared training, but it shows that grounding a verdict in gathered evidence beats leaning on the judge's priors. The same note warns that errors shared through a common memory module cascaded, so shared context can undo the gain. Separately, pooling observations from several observers can only maintain or improve a coalition's ability to spot decoys (Does sharing observations help coalitions detect decoys better?). The word 'weakly' matters here. Observers that see identical things add nothing, so the benefit of a second monitor depends on it seeing something different.
One more caution comes from the multi-agent security literature. Putting two agents in a room doesn't by itself create a security effect, because a failure that interaction leaves unchanged is just a single-agent problem repackaged (Does a multi-agent setting automatically signal a security effect?). Correlated blind spots between monitor and agent would be exactly that case, with the second check leaving the failure unchanged. Whether correlation actually hurts detection in practice is an open empirical question here. The corpus doesn't contain a monitor-versus-agent experiment that varies how much their training overlaps.
Sources 6 notes
Agents trained on static expert datasets cannot learn from their own failures or generalize beyond demonstrated scenarios because they never interact with environments during training. Competence is capped by what curators imagined, not by agent capacity.
QuCo-RAG uses entity co-occurrence patterns from training data to trigger retrieval, successfully flagging hallucination risk even when models are highly confident. This data-side approach catches the root cause (unseen combinations) rather than the symptom (low confidence).
Post-trained language models exhibit a measurable shift where they recognize their outputs become their own future inputs, closing an action-perception loop absent in pretraining. Evidence includes 3-4x lower output entropy on-policy and behavioral signatures of trajectory recognition.
Eight-module agentic evaluation achieved 0.27% judge shift versus 31% for LLM-as-a-Judge on complex tasks. However, the memory module cascaded errors, revealing that agentic systems need error isolation mechanisms to maintain gains.
Mathematical analysis shows that when agents share their observations, the coalition's capacity to distinguish decoys from genuine objects cannot decrease—it stays the same or improves. This means defenders cannot rely on isolation to hide decoys from coordinated observers.
Show all 6 sources
Interaction between agents can leave failures unchanged, amplify them, create them through composition, or define new properties. Only amplification, composition, and emergent properties qualify as genuinely multi-agent effects; unchanged failures reflect single-agent problems repackaged.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Agent Learning via Early Experience
- Emergent Collusion in Long-Horizon LLM Agent Interaction
- Counter-Swarm Doctrine: Containing Coordinated Agent Intrusions
- Can AI Agents Agree?
- From Simulation to Enaction: Post-trained Language Models Recognize and React to their own Generations
- Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
- When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs
- Agent-as-a-Judge: Evaluate Agents with Agents