Do LLMs overgeneralize when summarizing scientific research?

When LLMs summarize science, they tend to drop the qualifiers that bound a study's conclusions — turning "in this sample, under these conditions" into a universal claim. Comparing 4,900 LLM-generated summaries to their source texts across ten models, most overgeneralized: DeepSeek, ChatGPT-4o, and LLaMA 3.3 70B did so in 26-73% of cases, and in a head-to-head LLM summaries were nearly five times more likely than human-authored ones to contain broad generalizations (odds ratio 4.85).

Two findings sharpen it into a design warning. First, prompting for accuracy backfires: asking explicitly for a summary "faithful to the original text" produced roughly twice the overgeneralization of a plain summarization request — the accuracy instruction made things worse, extending the pattern that adding accuracy-intended instructions can be counterproductive. Second, newer models were worse than earlier ones, so this is not a defect that scale and iteration are erasing.

The consequence for science communication is direct: LLMs systematically inflate the scope of findings, and the obvious mitigation (tell it to be accurate) is unreliable. This is the summarization-side complement to Can models express uncertainty instead of just answering? — overgeneralization is dropped epistemic qualification, the inverse of faithful uncertainty — and it grounds Does polished AI output trick audiences into trusting it? in a measured science-communication harm.