If an AI tutor is taken away, does the dip in test scores fade with time, or is it permanent?
Does the crutch effect from AI tutoring fade over time?
This explores whether the performance drop students show after leaning on an AI tutor is temporary, like a withdrawal effect that wears off once the AI is gone, or whether it lasts.
This explores whether the 'crutch effect' (students doing worse once their AI help is taken away) is a short-term dip that wears off or a lasting cost. The short answer: no study in this collection follows students long enough after the AI is removed to say whether the gap closes. The evidence it does have points to a different way of seeing the question. The crutch may not be a dependence you recover from. It may be learning that never happened.
The clearest case is a 1,000-student trial with GPT-4 tutoring. Unrestricted access raised practice scores by 48%, but exam scores fell 17% once the AI was unavailable Does unguarded AI tutoring help or harm student learning?. A similar split shows up when two classroom trials are put side by side. Plain ChatGPT improved Turkish students' homework but lowered their test scores, while a tutor-style AI in Taipei raised exam results Does AI help or harm learning based on how it's designed?. The explanation offered in both cases is effort substitution: when the AI does the thinking, the student doesn't. If that's right, there is nothing to 'fade.' The exam gap reflects practice the student skipped, and closing it would take that practice, not just time away from the AI.
Other studies in the collection support this reading. Workers who used generative AI did much better on tasks with it, then showed no improvement when they did similar tasks alone afterward Does AI assistance help workers learn lasting skills?. Learners without AI ran into more errors and fixed them themselves, and they kept more of the skill. AI-assisted learners handed debugging to the AI and skipped the struggle that builds skill Does AI assistance remove a core learning channel through error work?. Part of the reason is that AI doesn't save time so much as move it from doing the task to prompting and reading outputs Does AI really save time, or just change how we spend it?.
The only study here that runs over a longer period suggests the cost may build up rather than wear off. In a four-month EEG study, brain connectivity dropped the more people relied on an LLM. LLM users showed the weakest engagement and had trouble recalling work they had just done Does AI assistance weaken our brain's ability to think independently?. The authors call this 'cognitive debt,' and debt grows unless it is paid down. A review of 14 studies describes a related drift, in which students who over-rely on AI dialogue systems increasingly swap careful reasoning for quick shortcuts Does relying on AI dialogue systems harm student critical thinking?.
The more useful finding is that the crutch effect can be prevented without giving up the AI. Hint-only guardrails in the GPT-4 trial kept most of the practice gains and removed the exam loss entirely Does unguarded AI tutoring help or harm student learning?. In a 704-person experiment, telling learners what offloading costs them halved their requests for answers and raised their unaided test scores by 51%. Rewarding effort did nothing Can metacognitive feedback stop students from offloading to AI?. So the open question of whether the effect fades matters less than this: the design of the tool, and whether learners understand what they lose by offloading, decide whether the crutch forms at all.
Sources 8 notes
A 1,000-student trial found that unrestricted GPT-4 access raised practice scores 48% but cut later exam scores 17% when AI was unavailable. Hint-based guardrails recovered most gains while eliminating the exam loss, suggesting design choices—not AI itself—drive harm.
Two RCTs found that plain ChatGPT access lowered Turkish high school math test scores despite better homework performance, while an AI tutor in Taipei raised final exam scores by 0.15 SD. The mechanism is effort substitution: giving answers short-circuits the mental work required for learning.
Wu et al. found that workers using generative AI performed substantially better on content tasks, but when performing similar tasks independently afterward, their performance showed no improvement. The capability did not transfer across contexts.
Research shows learners without AI encountered more errors and resolved them independently, resulting in higher skill retention. AI-assisted learners delegated debugging to AI, bypassing the cognitive work that produces learning—even those who debugged most with AI scored lowest on skill assessments.
Research shows AI doesn't reduce total task time; it reallocates it away from active work toward composing prompts and understanding outputs. This shift changes the cognitive demands and learning outcomes, making time-on-task a poor productivity metric.
Show all 8 sources
A four-month EEG study of 54 participants found that brain connectivity systematically scaled down with AI reliance—LLM users showed weakest neural engagement, poorest memory retention, and impaired ability to recall their own recent work.
A systematic review of 14 studies found that students who over-rely on AI systems—enabled by hallucinations, bias, and privacy failures—increasingly substitute fast heuristic judgments for effortful reasoning, undermining decision-making and analytical ability.
In a 704-person preregistered experiment, feedback that highlighted offloading costs reduced answer requests to an LLM by half and raised unaided test scores by 51%. An effort-based reward showed no measurable effect on either outcome.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- How AI Impacts Skill Formation
- Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality
- Generative AI without guardrails can harm learning: Evidence from high school mathematics
- Toward Measuring AI's Effects on Skill Formation: The Stock-Formation Gap
- Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performance
- Designing Against Deskilling: Metacognitive Feedback Reduces Cognitive Offloading to LLM Assistants
- AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
- AI Assistance Reduces Persistence and Hurts Independent Performance