INQUIRING LINE

Does AI writing drift toward blander, more generic word choices the same way no matter what kind of document it's writing?

Do vocabulary drift patterns appear across different document types and domains?

This explores whether the way language models shift word choice (drifting away from the original or expected vocabulary) shows up in a consistent pattern across different kinds of documents and subject areas, or whether it stays local to one setting.


This explores whether vocabulary drift (models gradually moving word choice away from the source or the expert register) is one recurring pattern or a set of separate quirks tied to particular domains. The short answer is that the corpus has no study that tracks drift across document types side by side. What it does have are several findings from different settings that point to the same cause: models lean toward whatever words were most common in their training data, and that pull shows up wherever they write.

The clearest explanation is a simple fact about language. General words like "animal" appear more often than specific ones like "beagle," and models favor frequent words, so every time a model picks the more common paraphrase it also moves toward vaguer meaning Does word frequency correlate with semantic abstraction?. This doesn't depend on the domain. Any expert text, whether legal, medical or technical, loses its precise terms in the same direction. The same general-to-specific ladder is visible inside the models themselves: the main directions in embedding space split broad categories first and finer ones later, matching the WordNet hierarchy Do embedding eigenvectors organize taxonomy from coarse to fine?. If the structure is built in that deeply, it's reasonable to expect the drift to carry across document types, though that cross-domain claim is still an inference, not something these papers measured.

The pattern looks different in each domain. In business strategy advice, six LLMs recommended the same side of every strategic tension across 15,000 simulations. Changing the industry context shifted results only 11%, while simply reordering the options shifted them 19%, which suggests the models were recombining trendy vocabulary rather than reasoning about the context Do LLMs consistently favor the same strategic choices regardless of context?. In law, the drift runs along time: models handle older Supreme Court cases worse because recent cases dominate the training data, so older legal language gets shallower representations Why do language models struggle with historical legal cases?. That's vocabulary drift as a historical gap rather than a word-choice habit. Retrieval researchers face the same problem when documents exist in several time-stamped versions Can retrieval systems ground answers in the right time?.

The less obvious finding is that stronger models may make drift harder to see. When LLMs repeatedly edit documents, weaker models visibly delete content, while frontier models corrupt it quietly and keep the surface looking intact Does model capability change how documents degrade?. Put that next to the frequency finding and you get a worrying combination: a capable model can swap precise terms for common ones in a document that still reads fluently. There's also an early-warning signal. Whether a keyword will "prime" a model after training can be predicted from how likely the model already found that word beforehand, with a threshold around 10^-3 Can we predict keyword priming before learning happens?. Prior word probability seems to govern what sticks, which is the same frequency logic again.

To sum up: the corpus supports a shared mechanism (the pull of frequency toward general, recent, trend-coded words) that shows up in strategy, law and document editing. It doesn't yet contain a direct comparison of how strongly drift appears across genres like fiction versus code versus scientific abstracts. That comparison is the open question.


Sources 7 notes

Does word frequency correlate with semantic abstraction?

WordNet analysis shows hypernyms (general concepts) occur more frequently than hyponyms (specific ones). Combined with LLMs' frequency bias, this means preferring common paraphrases systematically drifts toward abstraction, erasing expert-level specificity.

Do embedding eigenvectors organize taxonomy from coarse to fine?

Leading eigenvectors of embedding Gram matrices separate broad taxonomic branches first, then progressively finer sub-branches—a coarse-to-fine spectral order that tracks the WordNet hypernym tree level by level, confirming predictions from co-occurrence statistics.

Do LLMs consistently favor the same strategic choices regardless of context?

Across 15,000 simulations, six LLMs recommended the same strategic choice in every tension tested. Industry context shifted bias only 11%, while option order—a framing artifact—shifted results 19%, revealing that models recombine trend-coded vocabulary rather than analyze context.

Why do language models struggle with historical legal cases?

Supreme Court overruling benchmark (236 pairs) reveals era sensitivity: models perform worse on historical cases than modern ones. Root cause is training corpus over-representation of recent cases, creating shallower representations of older precedent.

Can retrieval systems ground answers in the right time?

TempRALM adds a temporal term to retrieval scoring alongside semantic similarity, achieving up to 74% improvement over baseline systems when documents have multiple time-stamped versions. The approach requires no model retraining or index changes.

Show all 7 sources
Does model capability change how documents degrade?

DELEGATE-52 shows weaker LLMs degrade documents through visible deletion, while frontier models degrade through subtle corruption that preserves surface integrity. This shift makes frontier failures harder to detect and potentially more dangerous at workflow scale.

Can we predict keyword priming before learning happens?

Pre-learning keyword probability strongly predicts post-learning priming across architectures and model sizes, with a ~10^-3 threshold separating contexts where priming occurs from those where it doesn't. Just 3 training exposures suffice to establish the effect.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.