SYNTHESIS NOTE
Topics›Reinforcement Learning›this note

Can reasoning systems forget history without losing coherence?

Does treating each reasoning step as independent—rather than accumulating historical context—actually preserve problem-solving quality while reducing computational waste? This explores whether Markov-style memoryless reasoning can scale effectively.

Synthesis note · 2026-05-18 · sourced from Reinforcement Learning

Existing test-time scaling methods all carry history along. Chain-based methods preserve the entire reasoning trace to generate each next step. Tree-based methods track ancestor and sibling relationships across branches. Graph-based methods compound this with arbitrary node dependencies. As reasoning scales, the accumulated historical dependencies waste compute and — worse — interfere with the model's ability to reason effectively on the current state.

Atom of Thoughts (2502.12018) makes a different bet: each reasoning state should be a simplified problem equivalent to the original, with partial reasoning steps either transformed into known conditions or excluded as incorrect explorations. The state transition mechanism has two phases. First, decompose the current question into a dependency-based directed acyclic graph (DAG) capturing structural information. Second, contract the subquestions into a new independent question. Iterate the decomposition-contraction until reaching directly-solvable atomic questions.

The Markov property is the load-bearing claim. Each transition depends only on the current state — never on the path that produced it. This is not a heuristic; it is a structural property guaranteed by answer-equivalence preservation through contraction. If the contracted question yields the same answer as the original, no historical context is required to continue.

The cognitive science motivation is direct. Humans solve complex problems by identifying and resolving self-evident subquestions, then reformulating a simplified problem state — not by maintaining detailed reasoning processes for resolved components. The reformulation IS the memory management.

Two architectural advantages emerge. AoT eliminates the need for maintaining and computing historical information when scaling test-time compute, and atomic questions can be seamlessly integrated into existing TTS frameworks as a plug-in enhancement. Since Can recursive subtask trees overcome context window limits?, AoT is the language-level version of the same insight — TIMRUN prunes KV cache to free positional embeddings; AoT contracts subproblems to free conceptual context. Both reject the assumption that more history equals better reasoning.

Inquiring lines that read this note 121

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Do AI explanations represent actual model reasoning or constructed post-hoc narratives? How does offloading cognition to AI assistance affect learning retention? What makes context and memory handling unique in persistent AI systems? What makes reasoning models struggle with ill-posed and unfamiliar problems? What can prompting techniques actually accomplish in language models? How can language models effectively manage long-context memory without quality degradation? How can we determine if computational systems are genuinely conscious? How does silent agreement emerge and undermine deliberation in multi-agent systems? How does scaling test-time compute compare to increasing model size? Does parallel reasoning outperform sequential approaches in reasoning efficiency? How should test-time compute be allocated adaptively across task difficulties? What determines whether visible reasoning traces improve task performance? Why does separating planning from execution enhance reasoning performance? Does prompting inject new knowledge into models or activate existing knowledge? Why does consolidated memory sometimes hurt continual learning more than no memory? How do transformer circuits and attention mechanisms enable or limit reasoning? Can prompts override training-embedded biases in language models? How does explicit structure improve LLM reasoning on complex tasks? Do reasoning traces accurately represent what models actually compute? When do additional thinking tokens stop improving reasoning performance? How should agent memory systems balance granularity, management, and reuse? Does reasoning architecture matter more than model scale? How should context management strategies balance efficiency with agent reliability? How do pretraining data composition and task diversity shape reasoning? How can we better validate AI research and prevent misconceptions? How should agents balance abstract planning with concrete environmental grounding? How do diffusion language models compare to autoregressive architectures? Does architectural recurrence enable computational capabilities that fixed-depth models cannot achieve? What actually drives chain-of-thought performance gains in reasoning tasks? What are the trade-offs between memory capacity and continuous task learning? What prevents or enables error accumulation in self-training systems? How do dialogue structure and state management affect multi-turn reasoning quality? How do mechanistic features and representations drive model behavior? How do standardized artifacts improve coordination in multi-agent systems? Should agent evaluation measure execution quality beyond task success?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 138 in 2-hop network ·dense cluster Open in graph ↗
Can reasoning systems forget history without los… Can recursive subtask trees overcome context windo… Why does parallel reasoning outperform single chai… Do iterative refinement methods suffer from overth…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

How do KV cache pruning and subproblem contraction both free reasoning capacity?Why does parallel sampling become more efficient when reasoning branches are memoryless?Can bounded workspaces prevent overthinking better than summarization alone?What makes answer equivalence sufficient to discard a reasoning path?
Original note title

markov-style memoryless reasoning replaces accumulated-history test-time scaling with iterative decompose-then-contract