SYNTHESIS NOTE
Topics›Agent Harness›this note

Should safety harnesses be customized for each deployment?

Can a single safety harness design work across different models and domains, or does each deployment need its own tuned version? Understanding this matters for scaling AI safety practices efficiently.

Synthesis note · 2026-09-25 · sourced from Agent Harness

The paper's claim is that a safety harness for an LLM agent is not a portable artifact. Its abstract says existing harness designs "are typically built once by experts and applied across heterogeneous models and domains," while "the effective defense is inherently deployment-dependent." It names two failure directions: "A harness strict enough for one model over-blocks another, and a policy general enough to transfer across domains can miss the safety relations of the application." The discussion turns this into a prescription: "A harness should not be copied unchanged across deployments."

The reasoning is a division of labor between two sources of information. The domain "supplies the tool semantics, trusted boundaries, and threat contract that determine the relation to protect and the state it requires." The victim model and runtime "determine its realization: semantic or deterministic, cached or recomputed, and enforced before or after execution." The abstract adds that models differ in "how much external enforcement they need before utility starts to drop," which is why the same check can be right for one model and over-blocking for another. From this the authors conclude that EvoSafeHarness "optimizes a deployment instead of selecting one portable guardrail," and it is presented as a framework that synthesizes a harness "for a frozen model in a target domain." The introduction frames the stakes: the "unit of safety has expanded from a single utterance to an entire action trajectory," and harmful instructions can arrive through indirect prompt injection or direct harmful requests.

This sits close to two notes about what a check must be able to express. The point that a harness needs to know "what history they must remember" is the trajectory thesis in Can step-by-step approval miss harmful behavior patterns?, and it echoes the demand for state in Can stateless checks ever catch sequence-level constraint violations?. Those notes say what a check has to carry; this paper adds that which check, run at which point, is right depends on the deployment. Why does prompt hardening work for single agents but not multi-agent systems? is a consistent case from another paper, where one defense behaved differently across system shapes, though that variable is system architecture rather than model or domain. Which security protections actually slow down agent exploits? treats protections as a variable to control in evaluation; here the harness itself becomes the thing searched over per deployment.

The excerpt reports no results. It does not show that synthesized harnesses outperform expert-built or generic ones, how the optimization runs, what "utility" is measured against, or how much of the gain comes from tuning to the model versus the domain. The claims about over-blocking and missed safety relations are stated as premises, not demonstrated. The safe reading is a design argument: when a harness result is reported, the model and domain it was tuned on are part of the result, and transfer to another pair should be treated as untested.

Inquiring lines that read this note 12

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How does harness optimization generalize across different model architectures and domains? Can single-point security defenses protect multi-agent systems from multi-step attacks?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 117 in 2-hop network ·dense cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

a safety harness should be optimized for each deployment rather than copied — the domain fixes what to protect while the model and runtime fix how