To trust an explanation of how an AI works, you need both a theory of what a part does and proof it matters.

This explores how attention-circuit research pairs two kinds of evidence: what a component appears to encode or do (representational), and what happens to the model when you intervene on it (causal). The corpus's answer is that the strongest findings do both, in order. Can LLM understanding rely on just representation or causation alone? states the rule. Representational analysis alone finds correlates without proving they matter. Causal analysis alone shows an effect without explaining the function. Real mechanistic understanding comes from locating a candidate first and then testing it.

Retrieval heads show this most cleanly. Fewer than 5% of attention heads do the work of pulling facts out of long context, and they show up across model families and even in short-context models (What mechanism enables models to retrieve from long context?). That is the locating step. The causal step is pruning them: the model starts to hallucinate even though the right information is sitting in its context. The heads are what let the model use the information, not just a place where it passes through.

Two other notes run both halves of the argument in one study. Program synthesis writes executable code that matches 99% of an attention head's behavior, which is a precise, testable description rather than a vague natural-language label (Can executable programs replace transformer attention heads?). Swapping 30–40% of heads for their synthesized programs still preserves question-answering ability. That swap is the causal test that the program is the mechanism and not just a lookalike. Massive activations follow a similar path. A handful of input-agnostic values, up to 100,000× larger than the rest, act as hidden bias terms that concentrate attention on particular tokens, and they are described as indispensable, which is a causal claim (Do hidden massive activations act as attention bias terms?).

The causal side can also point to a shared root cause. Cancelling attention noise with differential attention reduces hallucination, makes in-context learning robust to example order, and shrinks activation outliers. These look like separate failures but seem to share one cause: over-attention to irrelevant context (Can fixing attention noise solve multiple LLM failures at once?). The outlier result sits right next to the massive-activation finding, though the notes don't draw that link themselves. In the same way, soft attention systematically over-weights repeated, prominent content, and regenerating the context to strip out irrelevant material breaks the pattern (Does transformer attention architecture inherently favor repeated content?).

The contrast case is the geometry work. Reasoning tasks carve paths with 2–3× higher mean curvature than lexical tasks (Does transformer reasoning leave a geometric signature in representation space?). As summarized, that is a representational signature: a measurement, with no intervention showing the geometry does anything. It is a candidate still waiting for its causal half. The pattern to take from all this is that a circuit finding backed by only one of the two is a hypothesis, and one backed by both is an explanation.