Does unguarded AI tutoring help or harm student learning?

In a randomized field trial with nearly a thousand high-school math students in Turkey, Bastani and colleagues tested two GPT-4-based tutors against a no-AI control during in-class practice sessions, then measured a later exam taken without any AI access. "GPT Base," a plain ChatGPT-style interface, raised practice scores 48 percent over control, and "GPT Tutor," built with teacher-designed guardrails instructing the model to give hints rather than answers, raised them 127 percent. On the subsequent unassisted exam, GPT Base students scored 17 percent worse than control — "unfettered access to GPT-4 can harm educational outcomes" — while the GPT Tutor group's exam deficit was "essentially eradicated," though not turned positive.

The mechanism the authors give is behavioral: analysis of student interactions shows GPT Base used "as a 'crutch'" — students "asking for and copying solutions" — while GPT Tutor prompted "more substantive" use, students "asking for help or independently attempting answers." Students also showed no awareness of the cost: the paper reports "evidence that students do not perceive any reduction in their learning or subsequent performance as a consequence of copying solutions," so the harm was invisible to the people absorbing it. The authors frame this as a prompt-design problem, not an inherent property of the model — the same GPT-4 base, reconfigured to withhold direct answers, keeps most of the practice gain and removes the exam loss.

This is the Turkey trial that Does AI help or harm learning based on how it's designed? cites from the outside, against a separate Taipei tutor trial; this note supplies the original mechanism — guarded versus unguarded prompting within the same study, same model, same students — that Mollick's cross-trial contrast only gestures at. It gives a concrete, measured instance of the pattern in Can we measure whether AI erodes independent skill?: practice-session output looked better under both AI arms, but independent exam performance — the "stock" of skill — moved in the opposite direction under the unguarded arm. The crutch behavior and the students' blindness to their own loss echo Does ChatGPT help writers think less deeply about revision?, though that study measured disengagement from the writing process directly rather than inferring it from later test scores.

The trial covers one high school, one subject (math practice problems), and one short-horizon outcome (the next exam); it does not show whether the exam deficit persists, compounds, or fades over a semester, nor whether the guardrail design generalizes outside a teacher-curated hint format. The comparison arms differ only in prompt design, not in model capability, so the finding is specifically about deployment choices, not about generative AI's effect on learning in general — the paper's own conclusion is that "decision-makers must be cautious about design choices," not that AI tutoring is inherently harmful.