How do transformers perform analogical reasoning across domains?

Most work on transformer reasoning frames it as compositional — complex conclusions built by chaining simpler local steps. Analogy is a qualitatively different faculty: rather than chaining steps, it identifies shared relational structure across distinct domains and transfers it, producing the cognitive "leap" that Structure-Mapping Theory describes as preserving higher-order relations over surface features. "Emergent Analogical Reasoning in Transformers" formalizes this using functors from category theory — analogy as the inference of structure-preserving correspondences between entities across categories — and builds synthetic tasks to watch it emerge under controlled conditions.

The mechanistic finding is a clean two-part decomposition: (1) geometric alignment of relational structure in the embedding space, and (2) application of a functor within the transformer that carries that structure from source category to target. Emergence of this ability is highly sensitive to data characteristics, optimizer choice, and model scale — but the same trends and mechanisms appear in pretrained LLMs, so this is not just a synthetic-task artifact.

This matters because it separates two capacities the vault has mostly discussed together. Since Do transformers actually learn systematic compositional reasoning?, compositional reasoning is essentially interpolation over seen computation graphs — which means it does not explain cross-domain transfer. The functor mechanism is the missing account of transfer: relational structure abstracted in the geometry and re-applied elsewhere, which is closer to the "leap" than to subgraph lookup. It pairs naturally with How do transformers learn to reason across multiple steps? — both locate reasoning in emergent embedding-space geometry rather than explicit symbolic machinery.