Could the story of how text was typed, with its pauses, rewrites and rhythm, reveal AI help without scanning the finished page?
Can process-level signals like timing reveal LLM assistance without classifying finished text?
This explores whether we can tell someone used an AI writing assistant by watching how the text got made (pauses, typing rhythm, revisions) instead of running a detector on the finished document. The short answer is that this collection doesn't directly test it, but several notes explain why the idea makes sense.
This explores whether we can tell someone used an AI writing assistant by watching how the text got made (pauses, typing rhythm, revisions) instead of running a detector on the finished document. First, a caveat: none of the retrieved notes study keystroke logs, edit histories or timing-based detection directly. What the collection offers instead is a set of arguments for why the *process* might hold signals that the *product* hides.
The strongest argument is conceptual. One note argues that AI text generation is sequential but not temporal Does AI text generation unfold through temporal reflection?. Tokens come out in order, but no time for reflection passes between them. Human writing works differently: time spent thinking changes what comes next. If that's right, the meaningful difference between human and AI writing lives in duration. People hesitate, backtrack and rewrite, and those traces exist in the writing process even when the finished prose looks the same. Timing-based detection would be measuring exactly the thing the note says AI lacks. Pasted-in AI text has no reflective rhythm behind it.
The case against relying only on finished text comes from a different direction. Pattern-level detection is easy. Even GPT-2 can identify authors by style with 95% accuracy Can language models truly understand literary style?. But matching surface patterns isn't understanding, and surface patterns are what a person can edit, paraphrase or prompt away. AI judges themselves get fooled by surface features like fake citations and polished formatting Can LLM judges be fooled by fake credentials and formatting?. The trend also gets worse as models improve: weaker models visibly delete content, while frontier models corrupt it quietly and keep the surface looking intact Does model capability change how documents degrade?. The better the model, the less the finished text gives away.
There is one finished-text route the collection does support. Language models fail in predictable places, especially on tasks where the correct answer is statistically unlikely Can we predict where language models will fail?. That suggests detection might work better by checking for characteristic *errors* than by scoring style. Like timing, that approach looks at how the text was produced, not just how it reads.
The thing you might not have expected: the argument for process-based detection is really an argument about what writing *is*. If human writing means something partly because of the thinking time inside it, then a detector that only sees the final text is missing the most human part. Whether keystroke timing holds up in practice (against people who retype AI drafts, or who use AI for some passages and not others) is an open empirical question this collection doesn't yet answer.
Sources 5 notes
Token ordering in LLMs follows probabilistic selection without intervening reflection or revision. Human discourse gains meaning from temporal structure—time spent thinking changes what comes next—but AI text production lacks this duration-in-reflection despite appearing sequentially composed.
GPT-2 achieves 95% accuracy identifying authorship through style patterns alone, but lacks the evaluative framework to explain why those stylistic choices carry meaning. Detection without interpretation remains cataloguing, not criticism.
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
DELEGATE-52 shows weaker LLMs degrade documents through visible deletion, while frontier models degrade through subtle corruption that preserves surface integrity. This shift makes frontier failures harder to detect and potentially more dangerous at workflow scale.
By framing LLMs as autoregressive probability machines, researchers predicted tasks with low-probability target responses would be systematically harder, even when logically simple. Experiments confirmed predictions like backwards alphabet and letter counting.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Large Language Model Reasoning Failures
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- LLMs Get Lost In Multi-Turn Conversation
- Humans or LLMs as the Judge? A Study on Judgement Biases
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- Large Language Diffusion Models
- Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
- The Homogenizing Effect of Large Language Models on Human Expression and Thought