Feeding an AI more text can make it reason worse long before its memory fills up — so what's actually going wrong?

This explores two kinds of "length" that are easy to blur together: how much text you put into a prompt, versus how much room the model has and how much of it gets used, including by its own reasoning. The corpus suggests they come apart. The context window is a ceiling, and quality starts falling well before you reach it. In FLenQA, reasoning accuracy dropped from 92% to 68% with only about 3,000 tokens of padding, far below what the window can hold. The drop showed up across tasks, didn't track ordinary language-modeling performance, and survived chain-of-thought prompting (Does reasoning ability actually degrade with longer inputs?). A bigger window doesn't fix this, because the window was never the constraint.

The second kind of length is what the model writes for itself. Here the corpus shows a sweet spot rather than a steady decline. When thinking tokens went from roughly 1,100 to 16K, benchmark accuracy fell from 87.3% to 70.3%, with models overthinking easy problems and underthinking hard ones (Does more thinking time always improve reasoning accuracy?). Chain-of-thought length follows an inverted U. The best length rises with task difficulty and falls as the model gets more capable, and reinforcement learning drifts toward shorter chains on its own (Why does chain of thought accuracy eventually decline with length?). Input length hurts, and output length helps only up to a point.

The two notes above show that length matters. The next two show that it isn't the real variable. Trace length tracks difficulty only on problems near the training distribution and decouples from it entirely outside it, so a long trace mostly means the model is recalling familiar patterns (Does longer reasoning actually mean harder problems?). Reasoning failures also track how unfamiliar a specific instance is, not how long or complex the task is (Do language models fail at reasoning due to complexity or novelty?). So any length effect probably comes from what fills those tokens and how the model relates to them.

The corpus has more to say on that. Most intermediate reasoning tokens lose importance quickly. Keeping only the instruction prefix and a recent window gave a 3x speedup with no training and allowed traces beyond 100k tokens (Can models think longer by forgetting intermediate reasoning?). Atom of Thoughts goes further and rewrites the problem at each step so that each state depends only on the current problem and not on the history (Can reasoning systems forget history without losing coherence?). Both treat accumulated history as baggage. Brevity is also cheap to steer: a single activation vector cut chain-of-thought length by 67% with accuracy maintained (Can we steer reasoning toward brevity without retraining?).

The corpus doesn't isolate window capacity as its own variable, for example by giving the same input to the same model at different window sizes. What it supports is narrower. Input length degrades reasoning well below the window limit, and output length has an optimal point. Much of the token history in between is dead weight that can be dropped or compressed without hurting accuracy.