Can an AI stand-in for a student predict how a lesson will land, or must it also react to teaching?

This explores whether a simulated student can forecast how a teaching intervention would work, and whether it needs both fidelity (acting like a specific real student) and responsiveness (changing when taught). The corpus suggests it does, because each property alone fails in the opposite direction. Can student simulators match both behavior and learn from teaching? tested this in chess, writing and math. State-tracking models reproduce an individual student's behavior well but ignore tutor corrections. Prompted role-play follows guidance fluently but doesn't capture how competent that particular student is. Only a two-stage recipe got both: train on pooled students first, then specialize per student.

An intervention's outcome is the gap between what the student does before and after being taught. Fidelity gets the starting point right and responsiveness gets the change right. This is my reading of the finding rather than something the paper states. A high-fidelity, unresponsive simulator would predict that every intervention does nothing. A responsive, low-fidelity one would predict that every intervention works, because the fake student picks up a hint instantly and never struggles the way a real one does. Neither gives you a usable forecast.

The corpus explains why each half is hard. Fidelity is hard because assistant-style models are trained to be helpful and competent, which is the opposite of a struggling learner. Do pretrained models simulate humans better than instruction-tuned assistants? finds that base models conditioned on real dialog samples simulate humans better than instruction-tuned assistants given a persona, because assistant training actively degrades human simulation. Can imitating ChatGPT fool evaluators into thinking models improved? shows a related trap: a model can copy the surface style of a stronger one and fool evaluators without gaining any of its capability. A simulated student that sounds like a confused learner may have no matching gaps in what it can do. Fidelity also has to last through a long tutoring session, and Can training user simulators reduce persona drift in dialogue? treats persona drift over many turns as a problem in its own right.

Responsiveness has to be selective, because real students can't absorb everything. Does teacher-refined data always improve student model performance? shows that a model taught with refinements beyond its learning frontier gets worse, even when the refinements are better in absolute terms. A believable simulated student should therefore take in only the guidance it is ready for. That is probably why per-student specialization matters in the two-stage recipe.

No note in the corpus directly tests whether a simulator with both properties predicts real classroom outcomes. The closest evidence comes from persona simulation in other fields, and it points to a limited kind of reliability. Can behavior-based personas predict A/B test outcomes? predicts A/B test direction well for large effects and poorly for near-zero ones. Can AI personas reliably replicate human experiment results? reports the same pattern, with replication tracking how strong the original evidence was. Simulated students with both properties would likely be good for screening out weak interventions and poor for judging subtle ones. Because real students' responses to teaching are often subtle, live testing would probably still be needed.