Does learning from AI summaries produce shallower knowledge than web search?

Seven experiments across two preregistered designs, a lab replication, and a downstream-adoption test (combined n = 10,426, including a primary sample of 1,104 after exclusions) find that participants who learned about a topic via ChatGPT summaries, rather than Google web links, developed shallower knowledge and produced lower-quality advice for others. In the first experiment, ChatGPT users spent less time on the search task (MGPT = 585.41s vs. MGoogle = 742.81s, F(1,1102) = 44.61, P < 0.001) despite submitting a similar number of queries (2.06 vs. 2.15, P = 0.401) — the authors read this as "less engagement with the search results," not less interactivity. ChatGPT users also reported learning fewer new things (MGPT = 3.43 vs. MGoogle = 3.86, P < 0.001), felt less "personal ownership over the knowledge they gained" (P = 0.009), and rated the information as less comprehensive (P < 0.001). A fourth experiment showed the downstream cost: independent recipients, blind to which platform the advice-writer had used, found ChatGPT-sourced advice "sparser and more generic" and were less willing to adopt it.

The paper's mechanism is effort substitution, not raw information quality: "this lower effort in assembling knowledge from LLM syntheses...risks suppressing the depth of knowledge that users gain, which subsequently affects the nature of the advice they form." It draws on "search-as-learning" research, which holds that web search builds knowledge structures through "posing queries, gathering and interpreting information from different websites, and then assembling this knowledge into a cohesive whole" — a recursive "sensemaking" process. LLMs perform that sensemaking for the user, so "this critical ingredient in learning is often diminished." The authors frame this as a trade of efficiency for depth, and report the effect held even when LLM summaries were augmented with real-time web links and when the search engine itself was held constant (Google's own "AI Overview" vs. its standard results).

This supplies the experimental, causal counterpart to what Do large language models narrow human expression and thought? argues from training statistics without new measurement: where that paper infers narrowing from how models are trained and shared, this one randomly assigns people to a platform and measures a narrowing outcome — self-reported depth, then independently judged advice quality — directly. It also gives a concrete, measured instance of what Why do people trust AI outputs they shouldn't? describes abstractly as Trap 2 — users substituting the fluency of a synthesized answer for the grounded effort of building their own understanding; here that substitution is operationalized as time-on-task and shows up downstream in advice quality.

The depth-of-learning measure is self-report (how much participants felt they learned, felt ownership of, judged comprehensive), not an independent knowledge test; only the downstream advice-sparseness judgment was made by blind, independent raters. The flagship experiment used a single, low-stakes topic (how to plant a vegetable garden) and two platforms, though the paper states the pattern replicated across other topics and tools in supplementary experiments not excerpted here. The design establishes a causal effect of summary-format learning on self-reported depth and on the objective quality of a downstream writing task, within this learn-then-advise structure — it does not establish that LLM-mediated learning produces shallower knowledge in higher-stakes settings, with repeated use, or absent an advice-writing task.