INQUIRING LINE

Full AI reasoning traces can feel more convincing to read, even when they quietly make you worse at the task.

Do full traces and summary traces feel equally pleasant to readers?

This explores whether people reading an AI's reasoning prefer the full step-by-step trace or a condensed summary, and whether that preference tracks what actually helps them.


This explores whether readers prefer seeing an AI's full step-by-step reasoning or a short summary, and whether the version that feels better is also the one that helps. The corpus doesn't measure pleasantness directly; no study here asks people how much they enjoyed reading a trace. What it does measure is close by and more surprising: how each format affects people's accuracy, their confidence, and how much they trust the AI. Across those studies, how a format feels and what it does to the reader often point in opposite directions.

Start with what full traces do. In a 559-person study, people shown full reasoning traces did worse on the task and overestimated their own abilities more than people shown nothing. People shown summaries matched the baseline accuracy Do reasoning traces help or hurt people solving problems?. A separate study found that the traces most useful for the model's own accuracy were the ones people rated least interpretable. The self-revisions and recursive back-and-forth that help the model are hard for a person to follow. Worse, those traces made people more willing to accept wrong answers Do chain-of-thought traces actually help users understand model reasoning?. So full traces are often the less pleasant read, and readers still lean on them more than they should.

Summaries are easier on the reader, and that is their own risk. Studies of a computer-use agent found detailed traces overwhelming, while summaries looked deceptively reasonable. People trusted a clean-looking account of a process even when the process was flawed, and letting readers click between the summary and the detail reduced this overreliance without removing it How detailed should agent action traces be for human oversight?. This fits a broader finding: without any evidence signals, readers can't tell fluent fabrication from truth. Showing them which claims had been verified restored that ability Can readers tell truth from fabrication without evidence signals?. Fluency feels good, but it tells the reader nothing about whether the content is right.

The deeper reason neither format is a reliable window into the model is that traces aren't faithful records of how the model reached its answer. Most erroneous steps in a trace don't affect the final answer, and the logic a trace presents doesn't match the model's actual internal dependencies Do reasoning traces actually show how models think?. Length is also misleading. Correct traces tend to be shorter than incorrect ones, because long traces pile up self-revisions that add errors Why do correct reasoning traces contain fewer tokens?. If you want a better summary, the sentences that matter most are the planning and backtracking moments that steer the rest of the reasoning Which sentences actually steer a reasoning trace?. A summary built around those turning points could be both easier to read and more honest.

The takeaway you might not have expected: no trace format in the 559-person study helped people judge their own performance more accurately Do reasoning traces help or hurt people solving problems?. Full traces are tiring to read, and summaries read smoothly, sometimes too smoothly. The question to design for is not which one readers like more but which one keeps them appropriately skeptical.


Sources 7 notes

Do reasoning traces help or hurt people solving problems?

A 559-person study found full traces worsened task performance and inflated overestimation of abilities, while summary traces matched baseline accuracy. Crucially, no trace format improved self-evaluation calibration across all conditions.

Do chain-of-thought traces actually help users understand model reasoning?

A 100-participant study found that reasoning traces most useful for model accuracy are rated least interpretable by humans, and actually increase user acceptance of incorrect answers. The properties that make traces good training signals (recursive structure, self-revision) make them cognitively opaque.

How detailed should agent action traces be for human oversight?

Three studies of a computer-use agent found detailed traces overwhelming and summaries deceptively reasonable-looking. Linked navigation between levels reduced but did not eliminate overreliance on plausible processes.

Can readers tell truth from fabrication without evidence signals?

In an 81-person study, participants given no provenance cues showed no significant truth discernment (p = .43), falling for fluent hallucinations as readily as ground truth. An idealized Provenance Density interface showing verified claims restored a +4.15 point gap (p < .001).

Do reasoning traces actually show how models think?

ReasoningFlow found that most erroneous steps in traces don't influence final answers, and critically, the discourse structure traces present linguistically does not match their actual internal causal pathways. This gap suggests traces are narrative surface rather than verified computation logs.

Show all 7 sources
Why do correct reasoning traces contain fewer tokens?

Across QwQ, DeepSeek-R1, and LIMO, correct solutions average fewer tokens than incorrect ones. Longer traces correlate with more self-revisions, which introduce and compound errors rather than improve reasoning quality.

Which sentences actually steer a reasoning trace?

Counterfactual resampling, attention analysis, and causal suppression all identify planning and backtracking sentences as thought anchors—sparse critical points that guide subsequent reasoning. These are functional pivots, not noise.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.