If you can see which words in a draft came from AI, does that change how much you trust and double-check the final text?
Why does visual differentiation between user and AI text affect writer trust?
This explores whether visually marking which words came from the AI and which came from the writer, through highlighting, color-coding or provenance traces, changes how much writers trust and check AI-drafted text, and how that carries over to the trust readers place in the writer.
This explores whether showing writers which parts of a draft came from the AI changes how they trust and check that text, and what happens to reader trust downstream. One caveat first: the collection has no study that directly tests highlighting or color-coding AI text inside an editor. What it does have is a set of findings that, read together, explain why that kind of visual boundary would matter.
Start with what happens when nothing marks the boundary. AI output arrives fluent, confident and well formatted, and that polish works like a halo: people take it as accurate and skip verification. Chat interfaces make this worse by offering no accuracy cues and nudging users on to the next question Does confident AI tone make users skip fact-checking?. The effect isn't limited to one culture or language. Users everywhere follow confidence signals rather than correctness Do users worldwide trust confident AI outputs even when wrong?. Even LLM judges fall for formatting and fake authority cues Can LLM judges be fooled by fake credentials and formatting?, which suggests that surface presentation shapes trust well beyond human psychology. In writing tools, the result is measurable: writers edited AI-generated paragraphs only 23% of the time, and their edits left the text about 96% the same Do writers actually edit AI-generated text before publishing?. Once AI text blends into the writer's own, it gets treated as already reviewed.
That matters because the unchecked text isn't neutral. A large study found AI assistance shifted how readers perceived the writer on all 29 measured traits, making writers seem more extreme, more confident, more agreeable and more privileged Does AI writing assistance change how readers perceive the writer?. Writers still preferred the AI rewrites most of the time, while objecting to the distortions those same rewrites introduced Can user preference guide AI writing tool alignment?. In other words, writers can't easily see what the AI changed about how they come across. Visual differentiation is one of the few interface moves that could make that drift visible at the moment a writer could still fix it.
The closest direct evidence comes from collaborative editors. Paired writers strongly preferred editors that showed when, where and how a collaborator had used AI. They said this helped them understand each other's thinking and verify AI-generated passages, though some found full visibility intrusive and felt self-conscious Do writers want to see each other's AI prompts in shared editors?. That tension ties into ownership: writers feel more ownership of AI text the more they actively shaped it, while personalizing the model did nothing for ownership Does user control over AI text shape feelings of ownership?. A visible boundary between 'mine' and 'the AI's' could work in two directions. It could prompt writers to engage with and reshape the AI's text, which builds ownership. It could also make them feel exposed.
The less obvious part is that the trust stakes run in both directions. Readers trust writers less once they learn AI was involved, and the drop is steepest in personal, interpersonal writing How does revealing AI authorship change reader trust?. With no clear signal, readers guess instead, and suspicion lands on human writers whose text carries none of the real markers of AI writing Do unfounded AI accusations harm human writers instead?. Detection doesn't clearly settle it either. AI fiction can be spotted through structural narrative choices Can AI stories be detected without analyzing writing style?, but whether heavy rewriting also fools detectors remains untested Do rewrites that hide authorship also fool AI detectors?. So the question of where the AI's text ends and the writer's begins doesn't go away when an editor hides it. It shows up later, when readers decide how far to trust the writer.
Sources 12 notes
Samsonov argues that AI's confident tone, correct grammar, and polished formatting create a halo effect that makes users trust accuracy without verification. The chat interface compounds this by offering no accuracy warnings and actively prompting users toward new questions instead of source-checking.
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
Writers edited AI-generated paragraphs only 23% of the time, with edits averaging 96% similarity to the original. This means AI's opinionated and distorted voice propagates with minimal human filtering before publication.
A study of 2,939 writers and 11,091 readers found AI assistance shifted every tested dimension—29 total—toward extremism, confidence, quality, agreeableness, and perceived privilege. Distortions were statistically significant and directional, not random noise.
Show all 12 sources
Writers prefer AI rewrites 63% of the time but object to systematic persona distortions those same rewrites introduce. Mitigation studies show polish and distortion are entangled at the model level—preference optimization produces both simultaneously.
Sixteen paired writers showed strong preference for higher levels of prompt visibility in shared editors, valuing awareness of when, how, and where AI was used. Benefits included understanding collaborators' thinking and verifying AI-generated text, though some found full sharing intrusive and self-conscious.
Study 1 found that greater user control over generated text raised sense of ownership, while personalizing the AI model had no impact on the AI Ghostwriter Effect.
A study of 261 readers found that disclosing AI authorship consistently lowered perceived trustworthiness, caring, and likability, with the steepest drops in interpersonal writing like personal interaction. Readers saw AI as incapable of genuine empathy, viewing its use as a violation of social expectations.
Accused comments lack features that distinguish AI text from human writing, suggesting accusations function as gatekeeping rather than detection. This inverts the AI-as-perpetrator framing, placing harm at the receiving side through reader skepticism.
StoryScope achieved 93.2% accuracy separating AI from human fiction using only discourse-level features like character agency and chronological structure, retaining 97% of performance while eliminating stylistic cues. These structural choices resist humanization because they require rewrites, not surface edits.
The paper asserts that rewritten messages evade AI-text detectors but provides no detector experiments, only attribution results showing stylistic convergence. The double erasure claim needs direct empirical testing.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Understanding Reader Perception Shifts upon Disclosure of AI Authorship
- Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing
- What Influences Readers' and Writers' Perceived Necessity of AI Disclosure?
- Measuring and Mitigating Persona Distortions from AI Writing Assistance
- The AI Ghostwriter Effect: When Users Do Not Perceive Ownership of AI-Generated Text But Self-Declare as Authors
- "It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models
- The Assistant Erased You: Measuring Loss of Authorship Signals in AI-Mediated Communication
- AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances