Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning

Generative AI (GenAI) tools offer increasing opportunities for augmenting human cognitive tasks. Among these tasks, information seeking is being rapidly reshaped by GenAI tools, with potentially profound implications for learning and knowledge acquisition. To investigate these implications, we conducted a between-subjects field experiment in which participants pursued informal learning by seeking information through either ChatGPT or Google Search over a span of 8 days. Using a daily diary protocol, we gathered insitu data on their information-seeking processes. Our findings show that participants in the ChatGPT group experienced diminished agency in their information-seeking processes, as they offloaded much of the information selection to AI, and consequently experienced greater meta-cognitive load arising from this reduced sense of control. We further highlight two sources of distortion in information access when using chat-based GenAI tools: biases in ChatGPT outputs, particularly towards providing solution-oriented artifacts over principled knowledge; and systematic shifts in users’ information-seeking behaviors, whereby the conversational and socially-oriented interaction paradigm of current GenAI tools may inadvertently reduce exploration of the broader knowledge space. As a result, on average, participants in the ChatGPT group had worse learning outcomes than those using Google, especially for higher-order critical learning. Our work suggests inherent tensions between offloading information seeking to AI and meaningful learning, and provides broader implications for understanding AI’s risks to human cognition.

Introduction. In recent years, advances in Generative AI (GenAI) have shifted its role from primarily automating routine tasks to increasingly augmenting human capabilities (Wu et al. 2025). This shift is evident in its widespread adoption across domains of knowledge work (e.g., software engineering) (Suri et al. 2024), creative practices (Suri et al. 2024), and formal education (Nugroho et al. 2025). While GenAI tools can substantially improve productivity and efficiency, these gains are often realized through cognitive offloading (Shen and Tamkin 2026)—the practice of shifting effortful cognitive tasks to external aids. Among the various cognitive tasks, information seeking is one that has undergone a profound shift with GenAIpowered technologies. As users increasingly move away from the previously dominant paradigm of retrieval-based search to “ask ChatGPT,” they are offloading much of the mental labor involved in information seeking to chatbots and answer engines (Shah and Bender 2022). In particular, GenAI tools offer advanced information synthesis capabilities (such as summary generation), combined with natural language understanding capabilities, to help process flexible and highly specified queries. While these affordances can support rapid information look-up, they also offload critical cognitive processes, such as selecting, evaluating, and verifying information, to AI systems (Hirvonen et al. 2024). Information seeking is foundational to many downstream cognitive tasks such as learning, decision making, problem solving, and creativity (Shah and Bender 2022; Spatharioti et al. 2025). Consequently, shifts in how users seek information through GenAI tools may influence subsequent cognitive outcomes. In particular, emerging evidence in education suggests that unconstrained use of popular GenAI tools such as ChatGPT can impair learning. For example, in classroom settings, heavier use of ChatGPT has been associated with poorer academic performance (Bastani et al. 2024), as GenAI tools can automate or accelerate answer-finding and task completion at the expense of deep conceptual understanding and genuine learning (Qu et al. 2025). Unlike prior work focused on formal educational settings, we examine GenAI’s impact on informal learning, through which individuals acquire knowledge and skills in everyday contexts (Qian, Fu, and Zhou 2025). Whether to develop an artistic skill, learn a new language, or acquire practical life skills, individuals often rely on informal, self-directed pursuits mediated by information systems. While these learning pathways were once defined by search engines—studied under the paradigm of “searching as learning” (Ghosh, Rath, and Shah 2018)—and social media platforms (Sengupta 2020), they are increasingly being reshaped by GenAI tools (Terzimehi ́c, B ̈uhler, and Kasneci 2025; Lira et al. 2025). Crucially, informal contexts lack the institutional guardrails and instructional oversight found in formal settings, making users more likely to offload cognitive efforts that may be essential for building durable expertise. To understand the mechanisms through which GenAI tools (and their unique affordances) may impact knowledge acquisition, we investigate the shifts in people’s information-seeking behaviors during informal learning in naturalistic settings. We conducted a between-subjects, longitudinal field experiment (8 days) in which participants were randomly assigned to perform an informal learning task using either ChatGPT, a widely used GenAI tool (OpenAI 2022), or the conventional Google Search. In addition to measuring participants’ knowledge gain, we utilized a daily diary protocol to gather in-situ data about their informationseeking processes, and contrasted those for the two groups (ChatGPT vs. Google) through the lens of an informationseeking stage model (Ellis, Cox, and Hall 1993). Our work seeks to answer the following research questions:

RQ1: How does the use of GenAI tools impact people’s informal learning outcomes? RQ2: How does the use of GenAI tools impact people’s information-seeking behaviors for learning?

Our research demonstrates the risks of offloading information-seeking tasks to GenAI tools. With GenAI acting as a “filter” of knowledge spaces and providing fewer affordances for differentiating and assessing information, participants had diminished agency and control in their information-seeking processes, and experienced greater meta-cognitive load as a result of this diminished sense of control. In addition, we highlight two sources of distortion in information access. First, distortion may arise from biases within the technology itself: ChatGPT currently exhibits biases in output generation—towards generating solution-oriented artifacts over principled knowledge, while providing little transparency or controllability over these biases.

Related work. Recent advances in GenAI introduce expanded forms of cognitive offloading. Unlike earlier digital tools that primarily support low-level, narrowly scoped cognitive tasks, GenAI tools increasingly support intermediate cognitive tasks such as synthesis, comprehension, and reasoning, and even automate complex cognitive workflows such as planning, decision making, and creativity. In other words, GenAI tools are increasingly integrated into human thinking processes, and in some cases, may even be replacing human thinking altogether (Lee et al. 2025). These shifts have spurred great public and academic interest in examining GenAI’s implications for human cognition, especially in knowledge work (Budzy ́n et al. 2025) and educational settings (Walker and Vorvoreanu 2025). A series of studies has examined how the use of popular GenAI tools impacts specific cognitive tasks such as student learning (Yang, Hsu, and Wu 2025), decision making (Spatharioti et al. 2025), creativity (Kumar et al. 2025), and critical thinking (Lee et al. 2025). A few common observations have emerged from this literature. First, the use of GenAI tools involves cognitive offloading and may therefore lead to reductions in overall cognitive effort and engagement, including meta-cognitive activities (Lee et al. 2025). This is further evidenced by a study comparing EEG-based brain activity among participants using ChatGPT, a search engine, and no external tools (Kosmyna et al. 2025). Second, this bypassing of cognitive effort impairs knowledge and skill acquisition (Bastani et al. 2024; Walker and Vorvoreanu 2025; Qu et al. 2025), contributes to deskilling (Macnamara et al. 2024), and creates psychological harms such as loss of agency and ownership of one’s work (Kosmyna et al. 2025). Even if GenAI tools are found to enable performance boosts in some contexts, studies commonly show that knowledge workers and students do worse, compared to those not using AI from the beginning, when they lose access to these tools (Kumar et al. 2025). While situated in this broad research examining GenAI’s impact on human cognition, particularly learning and knowledge acquisition, we identify a gap in the literature where the specific link between the increasing use of GenAI tools for information seeking and learning outcomes is under-investigated. As we review below, information seeking is a foundational component of learning, especially in everyday informal learning contexts where structured instruction and curricula are absent.

Method. We conducted a between-subjects, IRB-approved, longitudinal field experiment with a daily diary study protocol. Participants were recruited from Prolific (Section “Participants”), a crowdsourcing platform widely used for human-subjects experiments. Over a period of 8 days (excluding weekends), participants were asked to use the assigned technology to seek information to learn about “nutrition and meal planning” (i.e., to perform an informal learning task)—a popular topic for informal learning (Terzimehi ́c, B ̈uhler, and Kasneci 2025). They were also encouraged to use (but were not restricted to) the technology for their other usual day-to-day information needs. They were randomly placed into one of two conditions, representing the assigned technology to use:

• ChatGPT: a popular GenAI-powered application, with information support (“ask anything”) as one of its primary functions. • Google: the long-standing popular search engine, which currently controls over 90% of the global search engine market (Global Stats 2025). Google Search recently introduced “AI Overviews” (Google 2025), a feature that uses GenAI to provide a snapshot of key information related to users’ queries. To minimize infusion of direct GenAI outputs, we asked participants to append “-ai” (verified via screenshot review) to every search query to temporarily remove the panel (Rasool 2024).

By comparing ChatGPT with Google Search without AI overviews, we captured two distinct ends of the informationseeking spectrum, i.e., GenAI-driven vs. conventional approaches. The field experiment included three components:

• Onboarding; Pre-learning knowledge test: Participants consented to the study and completed (1) a demographics survey, (2) a pre-learning knowledge test to capture their baseline knowledge about the learning topic, and (3) example tasks to prepare for the daily diary study (see below). Refer to Appendix for the onboarding survey. • Diary study: Following onboarding, each day for eight days, participants were asked to complete two tasks as part of the diary study: (1) Informal learning task: Every morning, we provided a guiding prompt, asking participants to learn about a subtopic related to the informal learning topic (see below). They were required to use the assigned technology for at least 10 minutes and could use it however they preferred. While completing the learning task, they were asked to upload screenshots for any number of interactions with the technology and comment on them; (2) Diary entry task: After completing the learning task, participants were required to submit a diary entry reflecting on their information-seeking activities for the day. They were asked to reflect on (a) how they sought information to complete the learning task and (b) at least one additional instance of information seeking.1 • Post-learning knowledge test: Post experiment, participants completed a knowledge test designed to assess their learning outcomes.

Informal Learning Task. We selected an informal learning task centered on acquiring a practical life skill (Qian, Fu, and Zhou 2025)—specifically, to learn about “nutrition and meal planning.” This topic fits the criteria of informal learning as it is self-directed, potentially personally relevant, requires active information seeking, and allows people to construct knowledge in informal contexts. Selecting this topic also enabled us to create a knowledge test and measure participants’ learning outcomes, suitable for answering RQ1. To facilitate learning over the eight days, we created broad daily guiding prompts, outlined in the Appendix (Section “Guiding Prompts”), in reference to beginner-friendly, highly-rated, and heavily registered relevant courses (Felix Harder 2025a,b). For example, over the course of the eight days, participants were progressively asked to learn about macronutrients, how to plan meals tailored to a specific fitness goal, and strategies for eating healthy when traveling.

Discussion. Through a between-subjects, longitudinal field experiment using a diary study protocol, we found evidence that using ChatGPT for informal learning may lead to worse learning outcomes than using conventional search engines, especially for higher-order critical learning (Lee et al. 2015). Participants’ diary entries revealed that the worse learning outcomes in the ChatGPT group can be attributed to their reduced exploration of the knowledge space and lack of agency in information selection, combined with limited and biased information provided by ChatGPT, leading to an overall more unsatisfactory learning experience from a diminished sense of control. Reflecting on these observations, below we discuss two implications of cognitive offloading to AI for information seeking, learning, and broader contexts.

Fundamental Tensions between Cognitive Offloading and Learning Compared to conventional search engines, which arguably allow users to offload the cognitive tasks of information retrieval and storage, GenAI tools’ synthesis capabilities can offload much of the extended processes of collecting, selecting and assessing information to AI, resulting in diminished human agency and control over one’s own informationseeking process. While potentially bringing value via, e.g., improving the efficiency and potentially the quantity of information access, this shift of agency may present fundamental tensions for learning. As our qualitative results suggest, learners may lose the means and motivation to thoroughly explore the knowledge space, miss opportunities to engage in information filtering and assessment, which are key to critical learning and developing durable knowledge, and become passive receivers of information, forming only surface-level understanding with an “illusion of knowledge” (Walker and Vorvoreanu 2025). Since agency and control play critical roles in learning, we also observe participants experiencing heightened meta-cognitive load and frustration, and overall unsatisfactory user experience with GenAI tools for learning purposes. Our results echo, and provide explanations for, recent literature in the education field which finds that the “default” designs of GenAI tools, which are oriented towards cognitive offloading for productivity gain, combined with students’ unconstrained use without guardrails, are often detrimental to learning (Bastani et al. 2024; Qu et al. 2025). Notably, research efforts have emerged to develop dedicated, structured GenAI tools for education, which often focus on cognitive scaffolding instead of offloading, such as providing feedback, prompting reflection, and engaging in Socratic questioning to facilitate structured exploration of the knowledge space (Tabarsi et al. 2025). However, the development and adoption barriers for these structured tools remain steep, especially as unstructured, “general-purpose” GenAI tools such as ChatGPT are easily accessible. Outside formal education settings, people are even more likely to continue using these general-purpose GenAI tools for knowledge and skill acquisition, in both planned and unplanned fashions. To preserve and even enhance these opportunities for informal learning we suggest two areas to develop new sets of affordances in GenAI tools in addition to the scaffolding approaches explored in educational technology research. One area is to provide affordances for users to understand and control which processes or subprocesses— e.g., which of Ellis’ model’s information-seeking stages— are offloaded to AI. Such modular support would allow GenAI tools to be flexibly configured to meet different goals (e.g., learning versus productivity) and the demands of diverse cognitive tasks.

Conclusion. People seek information not only to retrieve facts, but to acquire knowledge and develop expertise. GenAI-powered applications are increasingly mediating access to information, and, for many, replacing conventional search engines. This paradigm change involves offloading of the informationseeking process to AI. To understand how GenAI tools may reshape information seeking and impact learning, we conducted a between-subjects, longitudinal field experiment comparing participants using ChatGPT versus Google Search to pursue informal learning. We found that, by offloading much of the information selection, the ChatGPT group had diminished agency in their information seeking and took a more passive role in learning. This presents fundamental tensions with learning, which also manifested as heightened meta-cognitive demands and unsatisfactory user experience for the ChatGPT group in attempting to regain control in their learning processes. The diminished agency led to, on average, worse learning outcomes in the ChatGPT group, accompanied by two sources of distortion that systematically shifted their information access. First, ChatGPT exhibits a bias toward generating solution-oriented artifacts over principled knowledge about ‘how’ and ‘why.’ Second, the socially-oriented interaction paradigm of current GenAI tools may have unintended consequences in shifting people’s information-seeking behaviors. Popular design choices such as the “chat” interface and personalization may inadvertently reduce exploration of the knowledge space.