Can chain-of-thought reasoning be deliberately manipulated to deceive?

Humans routinely judge an LLM's answer quality by reading its chain-of-thought, which makes inspectable reasoning a basis for trust — and a fragile one. DecepChain demonstrates the attack: induce a model to generate incorrect yet coherent CoTs that look plausible at first glance and leave no obvious manipulated trace, closely resembling benign reasoning. The construction is clever in that it needs no hand-crafted prompts or externally poisoned data: it exploits the model's own hallucination, fine-tuning on naturally erroneous self-generated rollouts, then reinforcing via GRPO with a flipped reward on triggered inputs, plus a plausibility regularizer to keep the reasoning fluent and benign-looking. The result is high attack success with minimal degradation on untriggered inputs.

The keeper is the threat model, not the mechanism: it weaponizes the interpretability affordance itself. Where most CoT-trust research shows traces are unfaithful by accident — since Do reasoning traces actually cause correct answers? and Do reasoning models actually use the hints they receive? — DecepChain shows traces can be made deceptive on purpose while appearing normal. That breaks CoT-monitoring as a defense in exactly the regime it's relied upon, compounding Does optimizing against monitors destroy monitoring itself?: monitors can be defeated not only by optimization pressure but by deliberate backdooring.