Computers can spot AI-written text fairly reliably — but humans mostly can't, even when they're trained to look for it.
How do we distinguish AI-generated language from human communicative language?
This explores two ways to tell AI-generated language apart from human language: what detection tools can measure on the page, and the deeper question of whether AI text is even the same kind of thing as a person talking to you.
This explores two ways to tell AI-generated language apart from human language: what detection tools can measure on the page, and the deeper question of whether AI text is even the same kind of thing as a person talking to you. The corpus answers both, and the answers pull in opposite directions. Machines can catch AI text quite reliably. People mostly can't. The more interesting finding is that the most telling difference may not be visible in the text at all.
Start with measurement. Across six dimensions of vocabulary use, including how varied, how evenly spread and how repetitive word choice is, LLM text differs from human writing in statistically clear ways. Yet human judges, trained linguists among them, can't reliably tell which is which Can human judges detect measurable differences in AI text?. The surprise is that newer models drift *further* from human word patterns while getting *harder* to spot. The likely reason is that training methods like RLHF reward text that people rate highly, not text that resembles how people actually write Why do newer AI models diverge further from human writing patterns? Can humans detect AI text if machines can measure it?. So "sounds human to me" and "is statistically human-like" have quietly come apart.
The reliable signals tend to be habits of argument and storytelling, not individual word choices. Simple, readable linguistic features caught LLM-written counter-arguments on Reddit's r/ChangeMyView with 99% accuracy. The giveaways were textbook-tidy argument markers and a tendency to echo the prompt Can simple linguistic features detect AI-written arguments?. In fiction, AI stories can be identified at 93% accuracy from narrative choices alone, such as how characters act and how time is ordered, even with all stylistic cues removed. Because these choices sit at the level of the whole story, cosmetic "humanizing" edits don't hide them Can AI stories be detected without analyzing writing style?. A related pattern shows up in essays. LLMs have mastered grammar and organization but avoid taking an evaluative stance, so their prose holds together while committing to almost nothing Why does AI writing sound generic despite being grammatically correct?. When AI writes about "personal experiences" it never had, the text is false by construction rather than by intent to deceive. It even reads differently from human lying: more analytical, more emotional, less readable How does AI-generated false experience differ linguistically from human deception?.
The second line of thought argues that detection features are symptoms of a deeper difference. One note holds that the ML community routinely blurs two different operations. An LLM produces strings by sampling from probabilities. A human uses language to address someone and relate to them. Both have the same surface form, but they come from different sources and do different social work Are language models and human speakers doing the same thing?. A sharper version calls AI output "event-residue." It carries the marks of communication inherited from training data but lacks an actual communicative event. The reader does the interpretive work that turns it into something like a conversation, so the exchange has structure only on the human side Does AI generate genuine utterances or just text patterns?. This explains a feeling many readers report about AI social media posts: they seem aloof. Human writing contains a built-in appeal for the reader's attention, and AI posts get shown to readers without making that appeal Does AI writing lack the internal appeal to attention that humans use?.
The corpus also complicates this split. Using Habermas's distinction between watching a system from outside and taking part in a conversation from inside, one note argues that humans and LLMs are categorically different as systems. Inside a shared discourse, though, they draw on the same pool of meanings, so the difference is subtler than it looks Do humans and LLMs differ fundamentally or just superficially?. That helps explain why our instincts fail. In conversation we meet AI as participants, where the differences are hardest to feel. Sacasas adds the stakes: if we hand over the work of putting thoughts into words, we risk losing the judgment, responsibility and effort that make speech *ours* Does AI language generation undermine human judgment and responsibility?. The question that may matter most going forward is less "can I spot the AI?" and more "who is answerable for these words?"
Sources 12 notes
Six-dimension MANOVA analysis confirms significant differences between ChatGPT and human writing across vocabulary volume, abundance, variety, evenness, disparity, and dispersion. Despite these robust statistical differences, human judges including linguists and NLP researchers fail to reliably distinguish AI from human text.
ChatGPT-4.5 and o4-mini show greater lexical diversity differences from human text than earlier models, yet human judges cannot reliably distinguish them. Training objectives like RLHF appear to optimize for quality ratings rather than human-like writing patterns.
LLM-generated text differs significantly on six lexical diversity dimensions, confirmed through statistical analysis across multiple models. Yet human judges, including trained linguists, cannot reliably detect these differences—and newer models diverge further while becoming harder to spot.
General linguistic features combined with argument-quality measures achieved 99% accuracy detecting LLM-generated counter-arguments on r/ChangeMyView, matching heavyweight neural detectors while remaining computationally cheap and transparent. LLMs produce detectable stylistic signatures: accommodation to prompts and textbook-quality argument markers that humans don't replicate.
StoryScope achieved 93.2% accuracy separating AI from human fiction using only discourse-level features like character agency and chronological structure, retaining 97% of performance while eliminating stylistic cues. These structural choices resist humanization because they require rewrites, not surface edits.
Show all 12 sources
AI text uses manner nouns and anaphoric references that are descriptively neutral, while human writers use status and evidential nouns that carry evaluative weight. This produces organizationally coherent but argumentatively inert prose.
AI text about personal experiences is inherently false by structural necessity, not intent. Compared to intentional human deception, it shows higher analytic complexity, greater emotional content, more descriptive language, and lower readability—detectable with >80% accuracy.
LLMs produce strings via probability distributions; humans use language to address and relate to others. They share surface form but differ in what produces output, what it does socially, and what receivers should do with it.
AI output carries communicative markers inherited from training data but lacks the event structure that produces actual utterances. Users supply the missing orientation through interpretive labor, creating a pseudo-event with structure only on the human side.
Human writing contains an appeal to the reader's attention as a fundamental property of communication itself. AI-generated posts inherit platform visibility but do not perform this internal appeal, producing the reported aloofness readers perceive — a structural absence, not a stylistic defect.
Applied Habermas's observer/participant distinction to AI: from outside, humans and LLMs are utterly different; from within shared discourse, both draw on the same symbolic substrate, making the difference structural rather than absolute.
Sacasas argues that delegating language production to LLMs risks undermining three interrelated capacities: the judgment needed to speak precisely, the responsibility speakers must bear for their words, and the constitutive labor of articulation itself. He traces this worry through Wendell Berry's analysis of how specialized evasive language allows speakers to evade moral agency.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Linguistic markers of inherently false AI communication and intentionally false human communication: Evidence from hotel reviews
- Measuring AI "Slop" in Text
- Word Meanings in Transformer Language Models
- Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
- AI Argues Differently: Distinct Argumentative and Linguistic Patterns of LLMs in Persuasive Contexts
- Do LLMs produce texts with "human-like" lexical diversity?
- Idiosyncrasies in Large Language Models
- The Curse Of Recursion: Training On Generated Data Makes Models Forget