Does ChatGPT harm informal learning compared to Google Search?

In a between-subjects, 8-day longitudinal field experiment, participants recruited from Prolific were randomly assigned to pursue an informal learning task — "nutrition and meal planning" — using either ChatGPT or Google Search (with AI Overviews suppressed by appending "-ai" to every query). Using a daily diary protocol alongside pre- and post-learning knowledge tests, the researchers found that "on average, participants in the ChatGPT group had worse learning outcomes than those using Google, especially for higher-order critical learning." The ChatGPT group also "experienced diminished agency in their information-seeking processes, as they offloaded much of the information selection to AI, and consequently experienced greater meta-cognitive load arising from this reduced sense of control."

The paper traces the gap to two "sources of distortion in information access." First, a bias in the technology itself: ChatGPT "exhibits biases in output generation—towards generating solution-oriented artifacts over principled knowledge," with "little transparency or controllability over these biases." Second, a behavioral shift: the "conversational and socially-oriented interaction paradigm of current GenAI tools may inadvertently reduce exploration of the broader knowledge space," compared with a results page that affords "differentiating and assessing information." Mapped onto Ellis's information-seeking stage model, the authors describe ChatGPT as acting as a "filter" that absorbs the selecting, evaluating, and verifying steps a search engine leaves to the user — producing passive reception and, quoting Walker and Vorvoreanu, an "illusion of knowledge."

The result parallels Does ChatGPT help students code better but remember less?, where ChatGPT raised an artifact score while lowering recall and felt ownership — both studies find that the chat interface's ease buys worse retention of the understanding a later test actually rewards. It also complements Can metacognitive feedback stop students from offloading to AI?: this paper documents the cost of unconstrained chat-based offloading in situ, where the fraction-arithmetic study shows a design intervention at the "handover" moment — short of removing the AI — can recover some of that ground. Where this paper adds something neither of those covers is a second, independent distortion: a bias in what ChatGPT chooses to generate, not only in how much a learner hands over.

The excerpt gives no participant count, no detail on how the knowledge test was scored, and no quantified effect size for the learning-outcome gap, so the result's magnitude and statistical reliability cannot be judged from this text alone. The design also confounds the tool (ChatGPT vs. Google) with the interface paradigm (chat vs. results list), so it cannot separate "GenAI synthesis" from "conversational UI" as the active cause. At the strength this evidence allows: for informal, self-directed learning, a general-purpose chat tool is not a drop-in substitute for a search engine, and the paper's own proposed remedy — giving users visibility and control over which information-seeking stages get offloaded to AI — is a design direction it raises, not one it tests.