Did a new tool or method make AI failures worse, or were they always this common? What proof settles it?

This explores how to tell whether something really amplifies failures, or whether failures were already happening at that rate. The corpus has no single paper on this exact question. What it has is several studies that show, from different angles, what a trustworthy baseline has to contain.

The first requirement is a fair control with matched costs. One paper designs a four-way comparison of isolated actions, rolling windows, known groups, and prospectively discovered episodes, all held to the same review cost and false-alert workload (Does added monitoring improve protection at acceptable cost?). That equal-budget design is what would let you say monitoring changed outcomes rather than just consumed more effort. But the excerpt reports no results, so it gives you the shape of a baseline and nothing yet about whether amplification occurred.

The clearest case of amplification with a baseline is in RLVR training. Training on nearly impossible problems doesn't just fail to help. Rare accidental successes get treated as high-advantage trajectories, so answer repetition and computation-skipping get reinforced, and those shortcuts then contaminate capabilities the model already had (Do overly hard RLVR samples actually harm model capabilities?). The evidence for amplification is a before-and-after comparison. A behavior that used to be rare becomes common, and it spreads into tasks the model previously handled. Without that pre-training reference point, you couldn't distinguish it from a model that was always sloppy.

A flat failure rate can also hide a change in what the failures look like. Weaker models degrade documents by visibly deleting content, while frontier models corrupt them silently in ways that preserve surface integrity (Does model capability change how documents degrade?). Two systems could show similar error counts while one is far more dangerous at workflow scale. So a useful baseline records how detectable each failure is as well as how often it occurs. Something similar happens inside pipelines. An agentic evaluator cut judge shift from 31% to 0.27%, yet its memory module cascaded errors, and that only became visible by examining components separately (Can agents evaluate AI outputs more reliably than language models?).

Two more notes cover the risk of mistaking luck or coverage for a real effect. Zero temperature and a fixed seed give you the same output every time, but that output is still one draw from the model's distribution. Only repeated runs, 100 in the study, show whether a result is reliable or just consistent (Does setting temperature to zero actually make LLM outputs reliable?). Likewise, sampling many weak-model attempts amplifies coverage, meaning more correct answers appear somewhere in the pool. But it doesn't pick the right one unless an external check like tests, proofs, or type checks does the selecting (When can weak models match strong model performance?). The baseline there is the gap between having a correct answer in the pool and having one selected. Together the notes give four things a baseline needs: a matched-cost control, a pre-intervention capability check, failure-type tracking, and repeated runs. The corpus doesn't yet have a study that applies all four at once.