Can models think longer by forgetting intermediate reasoning?

The paper's central claim is that the usual way of scaling test-time compute, letting the model reason longer while keeping "the entire reasoning trace in memory via full attention," pays for tokens that no longer matter. The authors report that "most intermediate reasoning tokens lose importance as the model continues reasoning," and ask whether retaining them "is worth the cost." Their answer, Prefix Sliding, drops every token that is neither in the prefix nor in the window of the last few thousand tokens. The abstract says this makes existing models 3x faster without training while maintaining performance, and that reinforcement learning with Prefix Sliding can scale reasoning traces beyond a hundred thousand tokens.

The mechanism rests on a division of labor between two kept regions. The prefix "has key instructions and tools available to the model," so the task specification is never evicted, while the recent window holds "the current reasoning the model is working on." Everything between is treated as spent. Because both regions are fixed in size, "the total memory requirement" is capped "regardless of how long the model reasons." The introduction frames the cost side plainly: with full attention, the cost of each new token grows linearly with the tokens already generated. It also lists reliability problems of long contexts, namely distraction by old irrelevant tokens, context poisoning, repetitive loops and lost knowledge. The discussion adds that the method is more efficient than full attention even at horizons of only thousands of tokens, and that ablations show it beating both summarizing intermediate tokens and a vanilla sliding window.

Against the neighbors, this reads as an efficiency counterpart to the accuracy findings. Does more thinking time always improve reasoning accuracy? shows that more thinking tokens can hurt, and Does reasoning ability actually degrade with longer inputs? shows that long inputs hurt reasoning. Prefix Sliding shares their suspicion that a longer trace is not automatically a better one, but it responds by pruning the trace, not by capping the budget. Its nearest architectural contrast is Can neural memory modules scale language models beyond attention limits?, which keeps old information in compressed long-term memory. Prefix Sliding keeps none of it. It also sits alongside Can diffusion models commit to answers before full decoding?, where the saving also comes from noticing that part of the computation is already unnecessary.

The excerpt does not establish how importance was measured, since that analysis is deferred to the paper's section 2. It gives no model names, benchmarks (beyond LiveCodeBench in the limitation), window sizes or accuracy figures behind the 3x and hundred-thousand-token claims, and it does not say whether the accuracy-side problems above are what pruning relieves. The authors themselves flag that "sometimes this is not the case": on LiveCodeBench a larger window is needed to match full attention, so discarded tokens can carry information that later steps need. The defensible reading is that a fixed window suffices for much reasoning but not all, and that window size is a task-dependent setting, not a constant.