Do people prefer the reasoning formats that help them verify?

The paper's central finding is a mismatch between what people like and what helps them check a model. In a controlled human study of six reasoning formats, participants "prefer planning- and decomposition-based representations," yet "simpler chain-of-thought traces better support verification, trust, and interpretability." The discussion states the same split as three measures that "do not always align": user preference, verification performance, and trust calibration. The authors conclude that the assumption "that more visible or structured reasoning uniformly improves human evaluation" does not hold.

The paper also reports that the preferred formats carry calibration risks. Participants raised "more false alarms on correct traces" and showed "high trust despite low willingness to verify." So the favored formats did not simply feel good and do no harm. They pushed judgments in two wrong directions at once, flagging errors that were not there and trusting without checking. The paper's own reading is that "perceived usefulness does not guarantee effective verification."

The framing behind this is the paper's most transferable move. Reasoning representations such as chain-of-thought, planning, decomposition, and template-based reasoning were introduced to improve model performance, then repurposed as explanations for judging whether a response is correct. That shift "creates an evaluation mismatch": user-facing explanations are "evaluated primarily with model-centric criteria." The study treats representations "as human-facing interfaces rather than proxies for model reasoning ability," and measures structural understanding, error detection and localization, and trust calibration.

This sits directly against the model-side literature in the vault. Faithfulness is the usual yardstick, as in Do language models actually use their reasoning steps?, and Can we measure reasoning quality beyond output plausibility? proposes better model-side properties in place of plausibility. This paper adds that even a faithful or high-fidelity trace still has to be assessed as an interface, because the format a person likes is not the format that lets them catch errors. It parallels Can LLM explanations actually help humans predict model behavior?, where explanations that feel convincing fail to help people predict behavior. Here the gap is between preferred format and verification support. It also gives a measured counterpart to the worry in Do reasoning traces actually cause correct answers? that traces generate confidence they have not earned, although the excerpt does not attribute the high trust to anthropomorphism.

The excerpt leaves a lot open. It does not give the sample size, the participants' expertise, the task domains, the effect sizes, or a list of which formats made up the six. Beyond chain-of-thought, planning, and decomposition, it does not say which formats did best or worst on any measure. It does not say how "simpler" was defined or whether the ordering holds as task complexity rises, even though tasks varied in complexity. What it supports is narrower than a rule about chain-of-thought. Preference ratings should not stand in for verification performance when choosing how to show reasoning to users, and any reasoning display meant for oversight needs to be tested against the judgments it is supposed to support.