Can simple linguistic features detect AI-written arguments?

A combination of general-purpose linguistic features (lexical richness, syntactic complexity, type-token ratios) and argument-quality features (logical soundness, justification, engagement strategy) detects LLM-generated counter-arguments on r/ChangeMyView with nearly 99% accuracy. The features are interpretable — they name what they detect — and the detector is computationally cheap. External benchmark tests show this lightweight method performs comparably to heavyweight neural detectors in generalized detection scenarios.

The methodological point matters more than the accuracy number. Detection research has trended toward black-box classifiers — fine-tuned transformers that produce a yes/no without an explanation. The CMV result is the inverse: pick the right interpretable features and you get equivalent performance for a fraction of the compute, with the audit trail built in. The features are what does the work; the classifier is a wrapper.

The detection holds for one specific context — persuasive counter-arguments on CMV — and the authors are careful to flag the open questions: how does prompt design affect detectability, how does task type interact with the feature signature, how do these features behave under adversarial paraphrase. The 99% number is a ceiling for a specific genre, not a universal claim about LLM detection.

The forensic implication is the durable part. As long as LLM production mechanisms differ structurally from human production — stylistic mirroring of prompts, higher emotional positivity, textbook-quality argument markers — interpretable feature-based detection will find a target. Robust evasion would require LLMs to produce text whose features are human-like, not merely text whose content is convincing. That is a much harder optimization problem than current LLM training optimizes for.