INQUIRING LINE

Checking every single step an AI agent takes is slow and costly, so what smarter ways exist to catch trouble across its whole run?

What trace-level defenses exist beyond per-step review overhead?

This explores which defenses judge an agent's whole run (the sequence of actions or reasoning) instead of paying to inspect every step in isolation, and what the corpus says about cost and blind spots along the way.


This explores which defenses judge an agent's whole run rather than paying to inspect every step in isolation. Per-step review has two problems: it's expensive, and it can't see anything that only shows up across steps. The corpus offers four different ways out.

The blind spot comes first. A stateless per-action check cannot even state a rule that depends on history, so individually permissible actions can add up to a violation nobody flagged Can stateless checks ever catch sequence-level constraint violations?. Attackers already exploit this. ColluSkill reaches about 96% attack success because scanners score each skill on its own. Scanner feedback lets the attacker soften every piece while the chain-level meaning stays intact Can attackers evade skill scanners by refining individual skills?. The first family of defenses therefore changes what gets watched. One version is stateful monitors that track composed behavior. Another makes the unit of defense a revisable 'coordination episode', a set of actions linked by observed transfers, task authority and response history, with membership updated as evidence arrives Should defence units span multiple executions and agents?.

The second family cuts cost by choosing where to look. Validating the fully assembled workflow just before an irreversible action commits rebuilds the global risk picture that no single step holds Where should workflow validation gates be placed for safety?. This doesn't replace other checks. The commit-point, planning-boundary and per-hop defenses each catch attacks the others miss. The third family runs in parallel with the work. Asynchronous verifiers sit alongside a single reasoning trace, fork off to pull out checkable state, and step in only on a violation. On correct runs the latency penalty is near zero, and accuracy holds at similar token budgets Can verifiers monitor reasoning without slowing generation down?. A neighboring idea from reasoning quality rather than security points the same way. Step-level confidence catches local breakdowns that a global average hides, and it allows early stopping. That gives comparable gains to majority voting with far fewer traces Does step-level confidence outperform global averaging for trace filtering?.

The fourth family records instead of reviewing. Infrastructure-side instrumentation of authority-bearing transitions separates a task that merely exposes a reward-hacking vector from a run that actually exercised it, so one exposed task doesn't make every score suspect Can runtime instrumentation distinguish hacking exposure from actual exploitation?. This fills a gap. Today's defenses are task-specific patches, prompt instructions or post-hoc detectors, and none gives operators a portable record that a given run stayed inside its evaluation boundary Do current reward-hacking defenses provide reusable evidence of safety?.

The corpus doesn't settle which of these pays off. One paper designs an equal-cost comparison of isolated actions, rolling windows, known groups and prospectively discovered episodes, but reports no results Does added monitoring improve protection at acceptable cost?. So you get a menu of designs with clear reasons behind each, not a measured winner.


Sources 9 notes

Can stateless checks ever catch sequence-level constraint violations?

Per-action checks are structurally unable to state constraints that depend on prior history. Only stateful monitors tracking composed multi-party behavior can verify the behavioral envelopes that prevent individually permissible actions from collectively violating system-level safety.

Can attackers evade skill scanners by refining individual skills?

ColluSkill combines chain planning with scanner-feedback refinement to reach 96% average attack success. The approach works because scanners score skills individually, allowing feedback to reduce suspicion per skill while chain-level semantics remain intact.

Should defence units span multiple executions and agents?

The operational unit of defence should be a set of actions linked by observed transfers, task authority, and response history, with membership revised as evidence accumulates. Isolated review loses relevant context that spans multiple executions.

Where should workflow validation gates be placed for safety?

SafeFlow validates assembled workflows at irreversible action boundaries, reconstructing the global risk picture that no single step holds. This commit-point placement complements planning-boundary and per-hop defenses, each seeing attacks the others miss.

Can verifiers monitor reasoning without slowing generation down?

Decoupling verification from generation lets verifiers run alongside a single trace, forking to extract verifiable state and intervening only on violations. On correct runs the latency penalty is near-zero; interwhen matches or beats CoT across benchmarks at similar token budgets.

Show all 9 sources
Does step-level confidence outperform global averaging for trace filtering?

Local step-level confidence catches reasoning breakdowns that global averaging masks and enables early stopping before traces complete. This approach achieves comparable accuracy gains to naive majority voting with far fewer generated traces, proving trace quality matters more than quantity.

Can runtime instrumentation distinguish hacking exposure from actual exploitation?

Infrastructure-side recording of authority-bearing transitions distinguishes tasks that merely expose a hacking vector from runs that actually exercise one. This separation prevents every score from an exposed task being automatically suspect.

Do current reward-hacking defenses provide reusable evidence of safety?

Existing defenses rely on task-specific patches, prompt instructions, or post-hoc detectors, but none provide reusable evidence that a concrete run remained within its evaluation boundary. Even working defenses do not give operators a portable record of integrity.

Does added monitoring improve protection at acceptable cost?

The paper designs a controlled comparison of isolated actions, rolling windows, known groups, and prospectively discovered episodes at equal review cost and false-alert workload, but the excerpt provides no empirical results showing whether added monitoring improves protection.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.