Can neural networks grasp abstract relationships like analogies purely through learned patterns, or do they need hidden logical machinery?
Can transformers abstract relational structure without explicit symbolic machinery?
This explores whether transformers can build and manipulate abstract relations (analogies, compositions, rules) purely through learned continuous representations — or whether they secretly need, and lack, the symbol-shuffling machinery of classical logic.
This explores whether transformers can genuinely abstract relational structure without explicit symbolic machinery — and the corpus gives a split verdict: yes, they build relational structure, but no, they don't manipulate it the way a symbolic system would. On the optimistic side, mechanistic work shows transformers really do handle analogy as geometry: they align relational structure in embedding space and then apply a learned transformation, a two-stage signature visible in both toy tasks and full pretrained models How do transformers perform analogical reasoning across domains?. A more philosophical framing pushes this further — LLMs operationalize Saussure's *langue*, learning meaning as a fully relational web compressed from text with no external referents at all Can language models learn meaning without engaging the world?. By that reading, relational abstraction without symbols isn't a limitation; it's the whole mechanism.
But the moment you ask for *systematic* relational reasoning, the floor gives way. When the same computation is dressed in unfamiliar semantics, transformers reduce compositional reasoning to memorized subgraph matching rather than applying rules, and errors compound across steps on novel combinations Do transformers actually learn systematic compositional reasoning?. Decouple meaning from logical form entirely and performance collapses even when the correct rules sit right there in context — the models lean on parametric commonsense and token associations, not formal manipulation Do large language models reason symbolically or semantically?. So the relational structure is real but tethered to training-distribution semantics; it doesn't lift free into abstract symbols the way "without symbolic machinery" would ideally imply.
There's a deeper architectural reason for this ceiling. A fixed-depth feedforward transformer has no native recurrent state to track evolving relations, so it must smear intermediate state across layers until depth runs out — which is exactly why chain-of-thought exists, as a costly external patch that writes relational state back into the token stream Why do transformers need explicit chain-of-thought reasoning?. This is a genuine complexity bound: standard transformers sit under the AC0/TC0 ceiling, and models that add recurrence — a hierarchical planner-plus-executor, or a looped shared-parameter block — solve Sudoku, mazes, and unseen compositions that flat transformers can't touch Can recurrent hierarchies achieve reasoning that transformers cannot? Can looped transformers generalize to unseen knowledge combinations?. The relational abstraction that eludes a vanilla transformer often appears once you give it iterated depth to *reprocess* structure.
The surprising twist is that transformers are already doing more symbolic-flavored work internally than their outputs reveal. Neural nets spontaneously carve compositional tasks into isolated modular subnetworks, and pretraining makes that modularity sharper and more reliable — structure emerges without anyone hand-coding it Do neural networks naturally learn modular compositional structure?. Models trained to hide reasoning compute the right answer in the first few layers, then actively overwrite it to emit format-compliant filler Do transformers hide reasoning before producing filler tokens?, and when reasoning chains are pruned by importance, it's the *symbolic computation* tokens the model fights hardest to keep Which tokens in reasoning chains actually matter most?. So the honest answer is: transformers abstract relational structure through continuous geometry rather than explicit symbols, and that carries them impressively far — but the systematic, out-of-distribution relational reasoning we associate with real symbol manipulation seems to demand something the flat architecture lacks, which recurrence and depth partially supply.
Sources 10 notes
Mechanistic analysis reveals transformers perform analogical reasoning via two stages: geometric alignment of relational structure in embedding space, followed by learned functor application. This signature appears in both synthetic tasks and pretrained LLMs.
Research shows LLMs learn culturally situated discourse patterns by compressing relational structure from text, demonstrating that fluent language generation requires no external referents or embodied grounding.
Research shows transformers succeed on in-distribution tasks by memorizing computation subgraphs from training data, not by learning systematic rules. They fail drastically on novel compositions, with errors compounding across reasoning steps.
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
Feedforward transformers lack native recurrent state-tracking and must push evolving state deeper into layers, eventually exhausting depth. Explicit chain-of-thought externalizes this state into tokens as a costly patch for a structural deficiency.
Show all 10 sources
The Hierarchical Reasoning Model couples slow abstract planning with fast detailed computation across two timescales, achieving near-perfect performance on Sudoku and mazes where chain-of-thought methods fail completely. With only 27M parameters and 1,000 samples, HRM escapes the AC0/TC0 complexity ceiling that constrains fixed-depth transformers.
Recurrent-depth transformers with shared parameters across iterations enable systematic generalization and depth extrapolation that vanilla transformers cannot achieve. This emerges through a sharp three-phase process: memorization, in-distribution, then out-of-distribution generalization.
Pruning experiments reveal that neural networks implement compositional subroutines in isolated subnetworks, with ablations affecting only their corresponding function. Pretraining substantially increases the consistency and reliability of this modular structure across architectures and domains.
Logit lens analysis shows models trained with hidden CoT tokens compute correct answers in layers 1-3, then actively suppress these representations in final layers to produce format-compliant filler output. The reasoning is fully recoverable from lower-ranked token predictions.
Greedy likelihood-preserving pruning reveals six functional token categories; symbolic computation tokens are preferentially preserved while grammar and meta-discourse are pruned first. Student models trained on these pruned chains outperform those trained on frontier-model compression.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Faith and Fate: Limits of Transformers on Compositionality
- Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
- How do Transformers Learn Implicit Reasoning?
- Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
- Scaling can lead to compositional generalization
- A Mechanistic Analysis of Looped Reasoning Language Models
- Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens
- Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiability