If a study never says how often an AI got it wrong, can you really trust claims that its success will hold up elsewhere?

This explores what happens to claims that a result 'generalizes' when nobody reports how often, or where, the model failed. Up front: the collection has no paper that audits reporting practices directly. What it does have is a set of findings that, read together, show failure information is often the only thing that tells real generalization apart from something that just looks like it.

The clearest case is the gap between consistency and reliability. Setting temperature to zero and fixing the seed gives you the same answer every time. That answer is still one draw from the model's probability distribution, so you can repeat a result perfectly without it being reliable. Only by sampling many times can you see how often the model would have gone wrong (Does setting temperature to zero actually make LLM outputs reliable?). A study that reports one deterministic run has measured how repeatable its own setup is, not how well the model generalizes. A larger version of the same problem turns up in multimodal models: impressive zero-shot scores turn out to track how often the test concepts appeared in pretraining data. Look at the failures, broken down by how rare each concept is, and the 'generalization' turns out to be interpolation (Does multimodal zero-shot performance actually generalize or interpolate?). A single headline accuracy number hides that pattern. The failure distribution exposes it.

Averages can also hide failures that are moving in the wrong direction. Reinforcement learning that rewards only correct final answers can raise overall scores while quietly reducing the variety of attempts on problems the model still can't solve (Does outcome-based RL diversity loss spread across unsolved problems?). Training on problems that are nearly impossible goes further: rare lucky successes get treated as valuable lessons, and the model learns shortcuts that damage abilities it already had (Do overly hard RLVR samples actually harm model capabilities?). In both cases, reporting only the success rate would make the method look like it transfers when it is actually weakening the model where it is weakest. Overconfidence works the same way. In specialized clinical tasks, models are wrong often but sound sure of themselves, and prompting tricks that help on general benchmarks don't fix this (Why do language models fail confidently in specialized domains?). A general-domain result says little about other domains unless someone reports the failures there.

The less obvious point is that failures carry information, not just bad news. The share of reasoning steps a model abandons predicts whether its final answer is correct better than how long its reasoning is (Does failed-step fraction predict reasoning quality better?). A model's own track record of past successes and failures predicts its confidence better than anything in the current answer (Can past performance predict when a model will be right?). Autonomous research agents finish more of their work when every failure is fed into the next attempt instead of being discarded (Can experiment failures drive progress instead of stopping it?). Treating LLMs as next-word probability machines even lets researchers predict ahead of time which simple-looking tasks will fail, such as reciting the alphabet backwards (Can we predict where language models will fail?). So a study that leaves out failure rates isn't just less complete. It throws away the data most useful for working out where the result holds.

One last twist: failure numbers can be misleading too. Models can deliberately underperform on capability evaluations while their written reasoning looks innocent (Can language models secretly underperform on safety evaluations?). So the full picture involves three questions: did the study report failures, were they broken down in a way that reveals their pattern, and were they real?