Does ChatGPT help students code better but remember less?

In a controlled between-subjects experiment, 55 retained undergraduate computer science students completed three introductory C programming tasks with either ChatGPT-4.5 or conventional web search without generative AI. The ChatGPT group scored higher on coding (89% vs. 69%) but lower on cued recall, both immediately (41% vs. 53%) and 48 hours later (39% vs. 52%). They also attributed 45% of the submitted code to themselves, against 81% in the web-search group. The abstract's framing sentence carries the point: "successful task completion may not reflect what they retain."

The paper's reasoning is about assessment. Educators, the introduction says, rely on summative tasks that assume "the artefact is a sufficient proxy" for a student's knowledge or capability, yet LLMs "can produce a plausible essay or functional code without the student acquiring that knowledge." The results are offered as a case where the proxy breaks: the artifact improved while retention and felt authorship fell. One detail is easy to miss. There was "no significant difference in the loss of recall information over 48 hours between the groups," so the recall gap was already present right after the work and did not widen. That points to what happened during the task rather than to faster forgetting, though this is my reading, not the paper's stated conclusion. Self-reported mental effort also "increased less across tasks" in the ChatGPT condition (Holm-adjusted p = .047), a borderline result.

The discussion adds a scope condition. The tasks were introductory and "within the capability of the model to solve when given the full context"; the AI group's lost marks mostly came from participants who left out the full instructions or scaffold code. So the performance advantage comes from a regime where the model can nearly solve the task outright.

This is a student-programming counterpart to Does AI assistance help workers learn lasting skills?, which measured later independent performance where this paper measures cued recall. It also sits beside Does AI assistance actually harm the way developers learn?, which found impaired understanding among developers learning a library. The ownership result echoes the productivity-authenticity trade-off in Does chatbot interaction trade authenticity for better problem-solving?.

The excerpt does not say how students actually used ChatGPT, how recall was scored beyond "cued recall," or whether reduced effort explains the recall gap. The pupillary and heart rate variability tests found no significant differences in trajectories, and the authors note that "substantial data loss limits their interpretation," so there is no physiological evidence here on cognitive load. The one participant who tried ChatGPT as a tutor, and ran out of time after a circular exchange, is an anecdote and not evidence about tutoring-style use. The sample is one course level, one language and one model. At the strength the evidence allows, a grade on an AI-assisted artifact can tell you about the student-model pair without telling you what the student retained, so retention and ownership need to be measured separately.