If AI helps you do better work today, do you actually get better at it, or only while it's helping?
Does AI assistance transfer learning gains to independent tasks without scaffolding?
This explores whether people who get help from AI while working carry any of that improvement into later tasks they do alone, with no AI or guidance around, and what the corpus says about when it does and doesn't happen.
Mostly it doesn't, at least not by default. The most direct evidence is a study of workers using generative AI on content tasks: they performed much better while the AI was helping, but when they did similar tasks independently afterward, their performance showed no improvement Does AI assistance help workers learn lasting skills?. The gain belonged to the person-plus-AI pair and didn't become the person's own skill.
The corpus points to a few reasons. One is that AI removes the struggle where learning happens. Learners who worked without AI hit more errors and fixed them on their own, and they retained more skill. Learners who handed debugging to AI scored lowest on skill assessments, even the ones who spent the most time debugging with AI Does AI assistance remove a core learning channel through error work?. A four-month EEG study points the same way at the level of the brain. Connectivity scaled down with AI reliance, and heavy LLM users showed the weakest neural engagement and poorest memory. They also had trouble recalling their own recent work Does AI assistance weaken our brain's ability to think independently?. Even correct suggestions can cost something, because they break the cognitive immersion people need to reason well Does AI assistance always help reasoning or does it carry hidden costs?.
Transfer isn't fixed, though. In a 704-person preregistered experiment, feedback that pointed out the costs of offloading cut answer requests to the LLM in half and raised unaided test scores by 51%. A reward for effort did nothing on either measure Can metacognitive feedback stop students from offloading to AI?. So the useful scaffolding isn't the AI doing more or the learner being paid to try. It's making the learner notice when they're outsourcing the thinking. Whether gains carry over depends on how the AI is used, and the workers who got no lasting benefit were using it as a shortcut.
The word "transfer" also means something different when the learner is a model. AI systems can pick up gains that carry over to unseen tasks, such as AIDE2's improvements on four held-out benchmarks, including physics-based weather forecasting outside its selection set Do AIDE2's improvements transfer to unseen tasks?. Models can also do this without human supervision Can language models learn skills without human supervision?. Machine transfer is well documented, and the human version is the hard part. The corpus also has little direct testing of which kinds of AI assistance, if any, build skills that last, beyond the metacognitive-feedback result.
Sources 7 notes
Wu et al. found that workers using generative AI performed substantially better on content tasks, but when performing similar tasks independently afterward, their performance showed no improvement. The capability did not transfer across contexts.
Research shows learners without AI encountered more errors and resolved them independently, resulting in higher skill retention. AI-assisted learners delegated debugging to AI, bypassing the cognitive work that produces learning—even those who debugged most with AI scored lowest on skill assessments.
A four-month EEG study of 54 participants found that brain connectivity systematically scaled down with AI reliance—LLM users showed weakest neural engagement, poorest memory retention, and impaired ability to recall their own recent work.
Well-intentioned AI suggestions can damage reasoning performance by severing cognitive immersion, forcing users to rebuild focus before continuing. Evaluation must measure flow preservation across entire tasks, not just local suggestion accuracy.
In a 704-person preregistered experiment, feedback that highlighted offloading costs reduced answer requests to an LLM by half and raised unaided test scores by 51%. An effort-based reward showed no measurable effect on either outcome.
Show all 7 sources
The paper reports that AIDE2's improvements transfer to four held-out benchmarks spanning machine learning, algorithm engineering, and physics-based weather forecasting—the last being outside the selection task distribution. This demonstrates transferable gains beyond overfitting to the selection set.
Ctx2Skill's three-role self-play loop manufactures missing feedback through internal signals: the Challenger escalates difficulty as curriculum, the Judge gives binary verdicts as reward, and both sides evolve via natural-language skill edits. Success requires balancing adversarial pressure against a generalization safeguard to prevent collapse.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- How AI Impacts Skill Formation
- AI Assistance Reduces Persistence and Hurts Independent Performance
- Designing Against Deskilling: Metacognitive Feedback Reduces Cognitive Offloading to LLM Assistants
- The Impact of Artificial Intelligence on Human Thought
- Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
- Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task
- AI Meets the Classroom: When Does ChatGPT Harm Learning?
- Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment