Do writers actually edit AI-generated text before publishing?
This research tests whether the "human-in-the-loop" safeguard against AI text quality issues actually works in practice. It examines how often writers revise AI-generated paragraphs and how substantially they change them.
A common reassurance about AI writing assistance is that humans remain in the loop — they will edit, correct, override. The persona-distortion study tested this assumption directly. Writers were given AI-generated paragraphs and asked to edit them until the text reflected their opinions to their satisfaction. The result: writers edited the AI-generated paragraphs only 23 percent of the time, and most edits were minor — median Levenshtein ratio of 0.96, meaning the edited text was 96 percent identical to the AI's original.
This finding has two implications. First, the standard "human-in-the-loop" defense against AI text quality concerns is empirically wrong at population scale. Editing is rare and shallow when it does occur. The AI's text is reaching its audience in nearly the form the model produced it. Second, this means the persona distortions documented in the same study — opinionated, confident, demographically privileged, emotionally compressed — propagate with minimal human modulation. The distortion is not filtered by the writer's revision; it is embraced or ignored.
This forecloses one common mitigation strategy: relying on the writer to detect and remove distortions before publication. The writer who would have caught and corrected the distortion is the same writer who, the study shows, mostly does not edit and mostly prefers the AI version even after being given the chance to edit it. The distortion arrives at the audience because the writer does not interrupt it. Any intervention that hopes to reduce AI's influence on public discourse cannot rely on the writer-as-gatekeeper assumption — that role, in practice, is not being performed.
Inquiring lines that read this note 191
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can we reliably distinguish AI-generated from human-written text?- Can AI text detectors reliably identify AI-generated websites?
- Why can't algorithms distinguish between human and AI generated content quality?
- Can readers detect when text was written or heavily influenced by AI?
- What properties of natural text does artificial text actually eliminate?
- Is statistical analysis the only reliable way to detect modern AI writing?
- Can marking AI provenance solve the grounding problem for generated text?
- Does AI writing style remain distinct when content is masked or paraphrased?
- Can token-level watermarks detect synthetic content better than stylometry alone?
- How do we discount AI-generated text when we lack cultural literacy for it?
- Can readers tell which parts of a document were AI-generated versus human-written?
- Can AI detectors reliably distinguish human from machine-generated text?
- Can a classifier distinguish machine-written text from poor human writing?
- Does the same rewriting that erases authorship also narrow measurable AI text markers?
- Can detection systems identify AI text rewritten to match human author style?
- Would detectors trained on unaltered AI text catch heavily rewritten versions?
- Can AI-rewritten text still be detected as machine-modified?
- Can readers actually distinguish AI text from human writing?
- Can readers distinguish machine-generated text from human-written comments?
- Can language models detect AI-generated text in blind evaluation tasks?
- Are AI systems trained to devalue content labeled as machine-generated?
- Can AI detectors confuse distinctive writing style for machine authorship?
- Does the semantic weight of AI-written content matter more than sentence count?
- Can AI text detection improve enough to help evaluators make better decisions?
- Do surface phrases reliably identify unedited machine-generated scholarship?
- How accurate are automated AI text detectors compared to human judgment?
- Can readers reliably distinguish AI-written abstracts from human-written ones?
- What prose features distinguish automatically generated text from human writing?
- How often do AI book summaries fabricate details when spot-checks are random?
- What makes AI-generated punditry different from human expert commentary online?
- Do AI-generated posts crowd out human voices without any coordination or intent?
- Do AI-generated summaries reduce user engagement with original content sites?
- What percentage of workplace communication now contains AI-generated content?
- Do AI-generated articles rank worse in Google Search than human-written ones?
- Do AI-generated articles receive less search traffic than human writing?
- How often does LinkedIn wrongly flag legitimate posts as AI-generated?
- Does detecting AI authorship actually improve social media feed quality?
- How much of LinkedIn's feed is genuinely AI-generated versus human-written content?
- Do AI-generated posts get more engagement than human-written ones?
- Do mixed human-AI posts rank differently than fully generated content?
- Does length of post explain differences in AI rates across formats?
- Can volume-based moderation catch AI-generated top-level posts effectively?
- Do audiences penalize AI-written posts through visible callouts at scale?
- Does the 'feel of AI' in unedited posts trigger audience backlash and detection?
- How do subreddit and author identity affect machine-generated text reception?
- Why is machine-generated text concentrated in certain Reddit communities?
- Does algorithmic adjustment of AI content exposure hold as supply grows beyond twelve months?
- How does lower organic traffic affect the economics of human-written web content?
- Can search performance data distinguish AI-generated content from human-written articles?
- What changes when published text was never written for its readers?
- How does the author-function itself change when AI replaces human authorship?
- Why does production time matter to the meaning of generated text?
- Does AI make writers appear more politically extreme to readers?
- Why does AI writing seem more competent and informative than human writing?
- Does AI writing make authors appear more privileged or educated?
- How does AI assistance affect perceived emotional tone in writing?
- Which reader-rated attributes converge most strongly when writers use AI?
- How does perceived writer confidence shift with AI-assisted composition?
- Does homogenization at the text level cause homogenization of perceived authors?
- What structural difference exists between AI posts and human conversational writing?
- Does AI writing erase markers of non-native English speaker identity?
- Can demographic distortion in AI writing affect who appears credible in public discourse?
- When do readers defer to AI text without genuine processing?
- What makes readers treat AI-generated text as authoritative?
- What specific distortions does AI writing assistance introduce into text?
- How do writer preferences for AI output affect their willingness to edit it?
- Does transparency about AI use change how audiences trust the writing?
- What interventions beyond writer revision could reduce AI distortion in published content?
- What textual properties make AI writing feel polished and confident?
- Do AI writing models systematically change the tone or confidence of personal opinions?
- Do writers recognize when AI text misrepresents their actual stance?
- Does AI-assisted writing change how readers perceive the author's demographics or background?
- How do demographic and emotional compression relate to writing quality?
- Why might writers trust AI renderings of their views over their own words?
- Why do users prefer AI-polished versions of their own writing over originals?
- What would it take for readers to inspect rather than assume authorship?
- How do writers decide when to delegate work to AI versus doing it themselves?
- Why does AI-generated content feel flat compared to human commentary?
- Why do AI outputs lack the stable content of written sentences?
- What textual properties cause writers to prefer AI-rewritten versions of their text?
- How do AI rewrites systematically shift how writers appear across demographic dimensions?
- Why do human stories land in statistically rarer regions than AI narratives?
- How do changes in human and AI writing distributions shift rarity measures over time?
- Can task framing influence whether writers experience genuine authorship during co-writing?
- What design changes could reduce unhelpful AI reliance in collaborative writing tools?
- Why do AI-inserted text and code suggestions survive at different rates?
- How do authors decide which story components must stay under human control?
- Why does AI ideation benefit individual writers but harm the collective pool?
- Does knowing an AI wrote something make people scrutinize it more critically?
- Does non-directive framing help writers maintain ownership of ideas from AI suggestions?
- How often should proactive writing assistants interrupt without disrupting cognitive flow?
- What timing strategies work best for delivering AI writing suggestions?
- Do writers prefer configuring AI partners ahead of time or in the moment?
- What makes writers feel self-conscious about their prompts in collaborative spaces?
- How do writers verify and revise AI-generated text before sharing it?
- What prose features actually separate AI text from human writing?
- Is user preference a reliable target for training AI writing assistants?
- How much do writers actually edit AI text before publishing it?
- How much do humans edit AI-generated text before publishing?
- Do writers edit AI assistance enough to fool content filters?
- Can explanation-based AI safeguards work in real-time writing interfaces?
- Do human readers still recognize authors after heavy AI rewriting?
- Does AI writing assistance make different authors sound more alike?
- Does asking AI to preserve voice recover lost authorship signals?
- Why does topic structure protect authorship signals from AI erasure?
- Does AI assistance distort how readers perceive writer identity and demographics?
- Can disclosure of AI involvement change how evaluators score writing quality?
- Does AI-assisted writing dilute the conversational value of social media?
- Does AI writing assistance distort a writer's authentic voice and persona?
- Can proofreading tools preserve writer voice better than full rewriting features?
- Did authors using AI write about different topics than others?
- How does disclosure of AI involvement change across private versus public writing contexts?
- Does engagement with machine text reflect quality or just stylistic acceptance?
- Does writer credibility suffer when readers suspect AI involvement?
- What prose features actually distinguish AI-generated text from human writing?
- Why do writers hesitate to disclose when they used AI tools?
- How do writers' perceptions of productivity compare to their actual output quality?
- Why does the disclosure penalty still hold even when readers have high AI literacy?
- Can writers build AI literacy in readers through interface design choices?
- Why do people rate AI-written text as better than human writing?
- Do measurable differences exist between AI text and human writing?
- Do writers who own AI output rely more heavily on its suggestions?
- What explains writers' concern that AI disclosure reduces their competence perception?
- Do writers benefit when they make their AI prompting activity visible to collaborators?
- What makes readers suspect AI involvement in academic writing they evaluate?
- How much does knowing about AI use actually change how readers judge text?
- Can readers detect AI involvement in writing when not explicitly told?
- Does directly copying AI text into writing change disclosure expectations?
- Does polished text presentation hide process-level authenticity from readers?
- When AI becomes invisible in writing tools, do writers stop disclosing it?
- Does ownership of AI text lead users to rely more on suggestions?
- How do AI tools change the relationship between writing effort and ability signaling?
- How much of AI-assisted comments remain the writer's own words?
- Why do people withhold AI credit even when using personalized text generation?
- What tools or practices help people disclose AI use in their writing?
- How do attribution norms for human ghostwriters compare to AI usage patterns?
- How does reliance on AI change when writers own the final product?
- Does AI assistance distort how readers perceive a writer's voice?
- Why does polished prose stop signaling merit once writing becomes easier?
- Does AI-written text score higher because of presentation alone or judgment shift?
- Do human reviewers detect rhetorical polish as a sign of AI authorship?
- Does polished AI output mislead readers when experts are not directly supervising the writing?
- What evidence exists about writing skill distribution across populations?
- Does AI-generated writing feel polished while remaining harder to understand?
- Can we verify fabricated text without redesigning the generation process?
- How do citation errors in AI-generated papers differ from human hallucinations?
- How often do journal editors catch obvious textual problems before publication?
- How often do fabricated sources in AI output escape citation checking?
- What distinguishes confabulation from honest errors in AI-generated academic content?
- What happens when AI generates content faster than humans can verify it?
- Can human researchers verify automated research methods before they become uninterpretable?
- Can validation work teach freelancers as much as producing original work?
- What detection mechanisms work best for corruption-style document errors?
- What distinguishes human language production from machine text generation fundamentally?
- How do closed-loop automated venues differ from human-in-the-loop review taxonomies?
- Should rhetorical polish in AI reviews be separated from actual technical accuracy?
- Can human reviewers detect when papers have been rewritten by AI?
- Do AI reviews depend more on writing style than scientific merit?
- How often do researchers violate rules about AI use in review?
- Can polished AI text fool both reviewers and detection methods?
- How often do researchers suspect peer reviews are written by AI?
- Does AI content in reviews correlate with differences in paper quality control?
- Can human reviewers reliably detect AI-written peer review text by sight?
- Can machine review catch flaws in AI-generated work that humans miss?
- How often do AI systems produce papers with undetected factual errors?
- Which feedback loops in AI-mediated review remain unmeasured or rarely observed directly?
- Can humans reliably detect whether research text was written by AI?
- How long do the protective effects of an AI literacy warning last?
- Can AI involvement in news discussion reduce perceived quality without reducing use?
- How do managers and individual contributors differ in their exposure to low-quality AI work?
- What does editing time reveal about worker judgment and accountability?
- Why do admissions offices penalize AI use when essays improve in quality?
- Do professional writing services already disadvantage applicants without access to editing help?
- Are rushed deadline submissions more likely to use AI assistance?
- How much do edited AI responses versus raw outputs affect clinician ratings?
- Why did primary care physicians review only 73% of AI-generated transcripts?
- Do expert physicians also prefer AI-written medical text when it is unlabeled?
- What observable quality dimensions distinguish slop from other forms of poor writing?
- What metrics best measure originality as statistical rarity in narrative choices?
- Does editing AI drafts build skill or replace skill-building practice?
- How do free-text edits and bounded action selection differ in risk?
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Measuring and Mitigating Persona Distortions from AI Writing Assistance
- Understanding Reader Perception Shifts upon Disclosure of AI Authorship
- Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing
- Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
- What Influences Readers' and Writers' Perceived Necessity of AI Disclosure?
- "It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models
- The AI Ghostwriter Effect: When Users Do Not Perceive Ownership of AI-Generated Text But Self-Declare as Authors
- Is it Cake or is it AI? A Systematic Review of Human Uncertainty in Distinguishing Generative Artificial Intelligence Content
Original note title
The 23 percent edit rate of AI writing assistance establishes that distortions reach audiences in nearly unedited form