An AI framework catches thought distortions better on average — but 'better overall' can hide categories it still consistently misses.

This explores whether DoT — a structured three-stage prompting method for spotting cognitive distortions — actually delivers reliable detection across every distortion type, or whether the headline number hides limits. The corpus gives one direct data point and a lot of reason for caution. DoT does measurably help: separating subjectivity assessment, contrastive reasoning, and schema analysis yields a 10%+ improvement over zero-shot ChatGPT, and expert clinicians rated the explanations as genuinely useful for case formulation Can structured prompting improve cognitive distortion detection?. So the honest answer to 'all types, reliably?' is: the evidence shows aggregate improvement, not per-type reliability — a 10% lift on average is fully compatible with the framework being strong on some distortion categories and weak on others.

Where it gets interesting is *why* the structure helps, and here the wider corpus undercuts the comforting story. A run of findings shows that chain-of-thought style prompting often works by imitating the *form* of reasoning rather than performing it. Logically invalid reasoning chains score nearly as well as valid ones Does logical validity actually drive chain-of-thought gains?, deliberately corrupted traces teach as well as correct ones Do reasoning traces need to be semantically correct?, and the intermediate tokens turn out to be stylistic mimicry rather than causally necessary steps Do reasoning traces actually cause correct answers?. If a three-stage scaffold improves distortion detection partly because the staged *shape* nudges the model into a better answer pattern — not because it's truly diagnosing the psychological structure — then 'reliable across all types' is exactly the claim you'd expect to break.

It breaks specifically at the edges. CoT-style methods are distribution-bounded: they degrade predictably when the input drifts from what the model saw in training, producing fluent but logically inconsistent output Does chain-of-thought reasoning actually generalize beyond training data?, and the underlying mechanism is constrained pattern-matching, not abstract inference Why does chain-of-thought reasoning fail in predictable ways?. Cognitive distortions are not evenly distributed in this sense — common ones like catastrophizing or overgeneralization are heavily represented in clinical text, while subtler or rarer types are exactly the long-tail, out-of-distribution cases where structural mimicry fails. The framework most likely looks most reliable on the distortions that look most like its training examples.

There's also a deeper modeling limit worth knowing about. A framework built on staged, schema-driven analysis is implicitly betting that distortions can be decomposed cleanly — but human reasoning blends causal, associative, analogical, and emotion-driven moves, and approaches that capture one mode tend to miss the others Can causal models alone capture how humans actually reason?. Many cognitive distortions are precisely emotion-driven belief shifts, the category these structured methods represent least well. So the thing you didn't know you wanted to know: DoT's real value may be less 'comprehensive detector' and more 'reliable assistant on the common, well-represented distortions, plus clinically legible explanations' — and the right way to read its 10% headline is as an average that almost certainly masks a steep reliability gradient from common to rare types.