Could an AI tell when you're doing a task yourself on purpose, just to get better at it — and back off?

This explores whether an AI assistant can tell when someone is keeping a task for themselves in order to learn, rather than because they want help, so the assistant knows when to step back. The short answer: the collection has no study that detects this specific intent. It does hold the pieces such a system would need, and it shows why getting this wrong is costly.

The nearest thing to detection is work on reading a person's mental state from how they behave. Gaze, typing rhythm, hesitation and interaction speed can serve as continuous signals of what someone is thinking, with no need to interrupt and ask (Can AI systems read cognitive state from interaction patterns alone?). In principle, a learner who slows down deliberately and works through a problem looks different from someone who is stuck. The same research warns that these signals support helpful timing and manipulative profiling equally well. A system that can tell you are trying to learn can also tell when you are tired and easy to nudge.

The bigger obstacle may be that assistants don't track what they don't know about you. One study found that language models carry no representation of unknown facts about the user, such as their goals or why they are doing something. When researchers added an explicit list of 'things we don't know yet' to the prompt, sycophancy and harmful advice fell by 50–75% (Do language models know what they don't know about users?). Whether you are trying to build expertise is exactly that kind of unknown. Today's systems tend to assume you want the answer, which is the gap between what's said and what's meant that also drives reward hacking (Why do AIs keep gaming rewards instead of serving intent?).

Why does this matter? Several notes describe what happens when AI steps in on a learner's task. Even correct suggestions can break someone's concentration and force them to rebuild focus (Does AI assistance always help reasoning or does it carry hidden costs?). Worse, people start counting AI-produced work as their own ability, a self-perception error called the LLM Fallacy (How does AI-assisted work reshape how people see their own abilities?). AI can produce the finished product without the thinking that normally goes into it (Does AI separate intellectual form from the thinking behind it?). That leaves a person holding the output of expertise without having built it, and the fluency of the output makes that hard to notice (Why do people trust AI outputs they shouldn't?).

What you might not expect: the collection suggests the hard part isn't detecting a learner. It's that an assistant that can't see your learning goal will quietly take over the practice that builds your skill, while you come away believing you got better. Behavioral signals plus an explicit 'unknowns' list could plausibly make detection work. A more direct fix, which the collection supports less directly, is to let users simply tell the system they're learning.