Why should a safety promise hold for every possible case, not just on average or for the ones tested?

This explores why a safety guarantee should hold for every allowed case (a uniform bound), not just on average or for the cases someone tested. The corpus's answer is that safety is a worst-case property, and a uniform bound is the only kind of guarantee that speaks about the worst case.

One note spells out the distinction. What scope should a validator's statistical guarantee actually state? separates four guarantees that sound alike but aren't interchangeable: pointwise (this one task), domain-conditional (tasks of this kind), average-case (typical tasks) and uniform (any admissible task). Only the uniform one protects against the worst admissible task. An average-case number can look excellent while a small corner of inputs fails badly, and that corner is what a safety boundary exists to cover. The same note adds a catch: uniform is not the default. A validator that reports good performance without stating its scope is unscoped, not safely uniform. So a uniform bound is only the right choice if the certificate also says what 'every' ranges over.

Several other notes show how averages and spot checks mislead. Do AI guardrails refuse differently based on who is asking? found GPT-3.5 refusing at different rates for younger, female and Asian-American personas, and sycophantically declining political positions the user would disagree with. A guardrail that looks fine averaged over users can be uneven for particular ones. Do authorization rules or restricted tools prevent test modifications? shows a 0% unsafe action rate sitting next to a 100% judgment bypass rate. A clean measured rate doesn't tell you whether the system respected the line or was only prevented from crossing it. Does setting temperature to zero actually make LLM outputs reliable? makes the same point for a single output: temperature zero repeats one draw from the distribution, and consistency is not reliability.

A uniform bound also has to cover whole trajectories, not just individual steps. Can individual components pass safety checks if the system still fails? shows workflows failing while every step passes its local check. Can stateless checks ever catch sequence-level constraint violations? explains why: a per-action check cannot even state a rule that depends on history. Where should workflow validation gates be placed for safety? responds by validating at the commit point, where the global picture can be assembled. Behind this is Does a benign goal actually prevent harmful AI behavior?, which finds that harm comes from optimization structure, not typical intent. Typical-case reassurance can't stand in for a bound.

The corpus has limits here. It doesn't show that uniform bounds are cheap or achievable for LLM agents, and the direct argument comes from one note about validators. The wider claim is that weaker scopes don't answer the safety question. Even a uniform bound depends on who defines 'admissible': Who enforces invariants when agents cross organizational boundaries? points out that no one is named as owner of the invariants when agents cross organizational lines.