Students using an AI tutor see their homework scores rise — but does that hide a loss they can't feel until the AI is gone?

This explores whether students notice, in the moment, that AI help is eroding what they can do on their own. One caveat up front: none of these studies directly asks students to rate their own learning. What the corpus does show is that the signals students normally use to judge their progress point the wrong way. In a 1,000-student trial, unrestricted GPT-4 access raised practice scores by 48% but cut later exam scores by 17% once the AI was gone Does unguarded AI tutoring help or harm student learning?. A Turkish high-school math trial found the same pattern: homework got better and test scores got worse Does AI help or harm learning based on how it's designed?. From the student's side, that looks like success. Their homework is going well, and the harm only shows up on an exam they take without the AI.

The problem runs deeper than misleading scores. Being busy with the AI can feel like learning. In a study of people learning to code, the learners who spent the most time debugging with AI scored lowest on later skill tests. Learners without AI ran into more errors, fixed them on their own, and kept more of the skill Does AI assistance remove a core learning channel through error work?. The struggle that feels unproductive is the part that builds skill. A four-month EEG study adds a harder result: heavy LLM users showed weaker brain connectivity and had trouble recalling work they had just produced Does AI assistance weaken our brain's ability to think independently?. If you can't remember your own essay, you're poorly placed to notice what you didn't learn writing it. Researchers describe the broader pattern as a slide toward fast mental shortcuts in place of effortful reasoning Does relying on AI dialogue systems harm student critical thinking?. A UK survey also linked frequent AI use to lower critical-thinking scores, and the link was stronger in younger users Does frequent AI tool use reduce our critical thinking?.

Outside observers can't see the loss easily either. Usage logs record what someone produced with AI help, not what they can now do alone. Platforms see skills being used but not skills being built Can we measure whether AI erodes independent skill?. So the student, the teacher looking at homework and the company reading dashboards all see the same flattering signal.

The hopeful finding is that pointing out the cost works. In a preregistered experiment with 704 people, feedback that showed the cost of offloading cut answer requests to the LLM by half and raised scores on tests taken without AI by 51%. A reward for effort had no measurable effect Can metacognitive feedback stop students from offloading to AI?. That suggests people aren't careless. They just can't see the trade-off until someone shows it to them. Design can also do this work for them: hint-only guardrails kept most of the practice gains without the exam loss Does unguarded AI tutoring help or harm student learning?, and a tutor built to guide rather than hand out answers raised exam scores in Taipei Does AI help or harm learning based on how it's designed?.

The surprising takeaway is that the answer isn't to ask students to be more self-aware. Most of the signals available to them reward the very behavior that hurts them. What helps is making the hidden cost visible, or building tutors that won't simply hand over the answer.