Even a correct AI hint can cost you: interrupting your concentration mid-task may leave you worse off than no help.
What hidden costs does decision support feedback impose on learner focus and flow?
This explores what people lose when AI feedback or suggestions arrive mid-task, even correct ones, and what the corpus says about feedback that helps without breaking concentration.
This explores what people lose when AI feedback or suggestions arrive mid-task, even correct ones, and what the corpus says about feedback that helps without breaking concentration. Only one note tackles flow head-on, so the rest of this answer connects it to neighbouring findings.
The direct answer is that being right doesn't make a suggestion free. Well-meaning AI interventions in reasoning can hurt performance because they sever cognitive immersion, and the user has to rebuild focus before continuing (Does AI assistance always help reasoning or does it carry hidden costs?). The cost is invisible if you only grade each suggestion for accuracy. The note argues that evaluation has to measure flow across the whole task. A run of accurate hints can still leave someone worse off than no hints.
A second hidden cost is that the damage often shows up later. In a 704-person experiment, feedback that pointed out the cost of offloading answers to an LLM cut answer requests roughly in half and raised unaided test scores by 51%. A reward for making effort did nothing measurable (Can metacognitive feedback stop students from offloading to AI?). Ready help looks harmless while the learner is using it and only shows its price when it's taken away. The same experiment suggests what works better: feedback about the learner's own process, not more help on the task.
The corpus also shows that feedback can be correct and still not fit the learner. That work is on training models, so treat it as an analogy for people. Teacher-refined data made student models worse when it went beyond what the student could absorb, even though it was objectively better (Does teacher-refined data always improve student model performance?). Feedback also carries two things, a verdict on how well you did and a direction for what to change, and a bare score drops the second (Can scalar rewards capture all the information in agent feedback?). The tension is that the directional part is the most useful and probably the most likely to hijack your train of thought. The corpus doesn't test that.
Two more notes suggest ways around the problem, though neither is tested on human learners. Models need explicit training to ignore conversational distractors, because they are taught what to do but not what to ignore (Why do language models engage with conversational distractors?). An unrequested hint is a distraction of exactly that kind. Another line of work trains models to ask for feedback when they need it, instead of having it pushed on them (Can LLMs learn to ask for feedback during problem solving?). Pull-based feedback that the learner controls may protect focus better than push-based feedback, but the corpus doesn't test that with people yet.
Sources 6 notes
Well-intentioned AI suggestions can damage reasoning performance by severing cognitive immersion, forcing users to rebuild focus before continuing. Evaluation must measure flow preservation across entire tasks, not just local suggestion accuracy.
In a 704-person preregistered experiment, feedback that highlighted offloading costs reduced answer requests to an LLM by half and raised unaided test scores by 51%. An effort-based reward showed no measurable effect on either outcome.
Teacher-refined data degrades performance when it exceeds the student's learning frontier, even if objectively higher quality. Students should filter refinements using their own statistical profile to retain only compatible improvements.
Natural feedback carries two orthogonal types of information: evaluative (how well an action performed) and directive (how it should change). Scalar rewards capture evaluation but discard directional specifics that token-level distillation can recover, making the two complementary rather than redundant.
Fine-tuning on just 1,080 synthetic dialogues with distractor turns significantly improves topic resilience, revealing that the gap is not model capacity but absent training signal. Models learn to follow what-to-do instructions but not what-to-ignore instructions.
Show all 6 sources
Research shows that reformulating static tasks as pedagogical dialogues—where a teacher has privileged information and the student must learn to extract it—trains models to actively engage conversation as a problem-solving tool, not just imitate dialogue patterns.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- DiscussLLM: Teaching Large Language Models When to Speak
- CollabLLM: From Passive Responders to Active Collaborators
- Teaching a Large Language Model Tutor to Withhold the Answer: A Supervisor Architecture and an Evidence-Driven Method for Tuning Socratic Behavior
- Learning to Learn from Language Feedback with Social Meta-Learning
- Designing Against Deskilling: Metacognitive Feedback Reduces Cognitive Offloading to LLM Assistants
- CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues
- Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning