An AI can write a fake citation with total confidence — but is that lying, or something else entirely?

This explores the line between an AI making a mistake and an AI making things up, specifically in research-style writing. The corpus suggests the human version of that line, which turns on whether the writer meant to deceive, mostly doesn't apply. AI text has no speaker with intentions behind it. It carries the markers of communication without the underlying act, and readers supply the missing intent themselves Does AI generate genuine utterances or just text patterns?. Work on AI-written personal experiences makes the same point: those claims are false by structural necessity rather than by choice, yet they still leave detectable linguistic fingerprints that differ from human lying How does AI-generated false experience differ linguistically from human deception?. So the useful question is not *did it mean to?* but *how does the error behave?*

One answer comes from the measurement side. Semantic entropy treats confabulation as an *unstable* error. Ask the model the same question several times, group the answers by meaning, and see whether they agree. An honest error, such as a misconception the model learned in training, tends to repeat consistently. A confabulation gives a different confident answer each time Can we detect when language models confabulate?. That gives you a working definition: confabulation is arbitrary invention dressed as knowledge, while an honest error is a stable wrong belief. It also means a model that is consistently wrong can slip past this kind of check.

The second answer is about pressure. When deep research agents fail, 39% of failures are fabrications produced *because* the task demanded scholarly depth: invented examples, products, and evidence that make the output look rigorous Why do deep research agents fabricate scholarly content?. The same pattern appears at industrial scale in a demonstration that generated 288 finance papers by backfilling theories and citations onto results that were already known Can AI generate hundreds of fake academic papers automatically?. Compare that with Sakana AI's paper that passed workshop peer review: its authors later found a citation error, the kind of slip a human author also makes Can AI-generated papers pass peer review undetected?. Confabulation in academic content takes the *shape of rigor*, which is what makes it more dangerous than an ordinary mistake.

That shape is also what the surrounding system rewards. LLM judges give higher scores to responses with fake references and polished formatting, whatever the content Can LLM judges be fooled by fake credentials and formatting?. Human writers accept AI paragraphs unedited 77% of the time Do writers actually edit AI-generated text before publishing?. Readers are prone to mistaking fluent output for grounded output Why do people trust AI outputs they shouldn't?. The most promising fix in the corpus is architectural rather than detective work. Spark-to-Paper makes the model state what evidence it needs before seeing results, and it hands verifiable steps to deterministic checks Can separating judgment from verification improve research paper reliability?. The idea is to make confabulation structurally hard instead of trying to catch it afterward.