Do reasoning traces help or hurt people solving problems?

A preregistered between-subjects study (N = 559) had participants solve ten LSAT-style reasoning problems under an Answer-only baseline, a Full-trace condition, or a Summary-trace condition. "Contrary to H4, participants who were exposed to Full-trace performed significantly worse and overestimated their performance to a greater degree than those who were not exposed to any trace." Summary-trace "preserved task performance at the no-trace baseline" and "led to lower overestimation levels, but no condition produced well-calibrated self-evaluation." Across all three conditions, "participants substantially overestimated their performance, and no trace format supported calibrated self-evaluation," though overestimation was largest under Full-trace.

The authors attribute the Full-trace harm to offloading rather than engagement: "qualitative patterns indicate that this is mediated by offloading and guided reasoning rather than by genuine critical thinking, as evidenced by the model traces. Traces acted as substitutes rather than supports for user reasoning." The paper frames this against "trace exposure reliably shifted the surface of the interaction — trust and hedonic appeal — but did not shift effort, usability, performance, or calibration in the predicted directions," concluding that "users engaged with the trace superficially, as a fluent artifact rather than as an object of evaluation." Trace exposure "inflated bias and left sensitivity unchanged, indicating that reasoning content does not feed the monitoring loop."

This extends the library's existing split between what a trace does for a model and what it does for a person. Do chain-of-thought traces actually help users understand model reasoning? shows R1 traces that maximize model accuracy are rated least interpretable by humans; this study adds that exposing a verbose trace to users doing their own reasoning task can actively lower their performance, not merely fail to raise their interpretability ratings. Do people prefer the reasoning formats that help them verify? documents a mismatch between preferred format and verification support in evaluating a model's output; this study documents a parallel mismatch in a different task — solving a problem oneself with trace exposure — where the full trace raises trust and hedonic appeal while performance and calibration move the opposite direction.

The excerpt ties the Full-trace harm to "an open-weight reasoning model exposing verbose intermediate output" (gpt-oss-20b), while Summary-trace and Answer-only both used GPT-5; the study does not report a same-model comparison across all three trace lengths, so model identity and trace verbosity are not cleanly separated as causes of the Full-trace performance drop. The excerpt also does not report effect sizes, demographic or expertise breakdowns, or whether the pattern would hold on tasks besides LSAT-style logical reasoning. What it does establish is that exposing a trace, in either tested format, raised trust and hedonic appeal without correspondingly raising objective performance or self-evaluation accuracy — a dissociation that any interface marketing traces as transparency should reckon with.