AI tutoring can boost practice scores while leaving students no better — or even worse — once the AI is gone. Why?

This explores why AI help can raise students' scores while they use it, yet leave them no better (or worse) once they have to work on their own. The clearest evidence in the collection is sharper than 'no transfer.' In a 1,000-student trial, unrestricted GPT-4 access raised practice scores by 48% but cut later exam scores by 17% when the AI was taken away Does unguarded AI tutoring help or harm student learning?. Students didn't just fail to gain. They lost ground compared with students who never had the tool. The surprising part is that a tutor limited to giving hints kept most of the practice gains and removed the exam loss. That points to how the tool is designed as the cause, rather than AI itself.

Why would a crutch feel like learning? One answer in the corpus has to do with self-perception. When AI produces smooth, polished output, people read that ease as a sign of their own competence, even though they didn't produce the work Does processing ease mislead users about their own competence?. Students walk away believing they understand material they only watched being solved. The same gap appears outside the classroom. In one survey, 90% of workers said they felt confident with AI, but only a quarter said it works on the first try, and half spent more time with AI than doing the task by hand. The overconfidence was strongest among younger workers Why do workers feel confident with AI but get poor results?.

A second answer is about what gets skipped. In a study of students working with chatbots, the groups solved problems better and produced more knowledge-heavy dialogue than peer groups. But they talked less overall and voiced far fewer opinions of their own Does chatbot interaction trade authenticity for better problem-solving?. If learning comes from wrestling with an idea in your own words, the chatbot may be quietly doing that wrestling for them. A related note finds that even correct AI suggestions can break a person's concentration and force them to rebuild their train of thought Does AI assistance always help reasoning or does it carry hidden costs?. A suggestion can help with one step and still harm the reasoning as a whole.

The workplace evidence reframes the question. Consultants using GPT-4 improved a lot on their assigned tasks, and the weakest performers gained the most Does AI help lower-skilled workers catch up to top performers?. That looks like the skill gap closing. In light of the tutoring trial, though, it may be a gap in output that closes only while the tool is present. Even that boost is smaller than it could be. People working with an LLM captured only about half of the model's accuracy gains, which means they did worse than the better of the two working alone Why does assisted accuracy capture only half the LLM gain?.

One limit: none of these studies tests 'transfer' in the strict sense of applying a skill to a new kind of problem. What the corpus shows is closely related: performance with the tool doesn't predict performance without it. The tool that would show the difference, hint-only tutoring that makes students do the final step, is one of the few interventions here with clear evidence behind it.