An AI that plays a student can copy how that kid thinks, or respond to teaching, but rarely both.

This explores what the two yardsticks for student simulators, 'does it act like this particular student' and 'does it change when a tutor teaches it', test in practice, and why a simulator can pass one and fail the other. In the corpus, behavioral fidelity asks whether the simulated student reproduces the real student's competence: the same moves, mistakes and level. Guidance responsiveness asks whether the simulator updates sensibly after a tutor corrects it. The clearest way to see the difference is through the failures. State-tracking models are strong on fidelity but ignore tutor corrections, so they are a faithful replay of someone who can't be taught. Prompted role-play is the opposite. It follows guidance fluently but doesn't capture the individual student's competence, so you get an agreeable generic learner. The corpus reports that a two-stage pipeline of pooled training followed by per-student specialization gets both across chess, writing and mathematics Can student simulators match both behavior and learn from teaching?.

The two measures come apart because they test different things. Fidelity is checked against a record of what one person already did. Responsiveness is checked on what happens after an intervention, which no record contains. That makes responsiveness an inherently multi-turn measurement. It matches a broader shift in agent evaluation from scoring final answers to scoring whole interaction trajectories, including recoverability and coordination How should we evaluate agent behavior beyond final answers?. A student simulator that only gets a single-turn fidelity score has never been asked whether it can learn. The corpus summary doesn't give the exact scoring formulas for the student-simulator work, so what you can say with confidence is what each measure catches, not the metric recipe.

Fidelity is also less unitary than it sounds. Persona work finds that splitting fidelity into three functional dimensions and judging each one beats a single holistic verdict from an LLM judge, especially on harder tasks Can breaking persona fidelity into parts improve how we judge it?. Psychometric testing of nine LLMs shows another side of it: each model has a stable signature, including which questions it treats as not applying to it Do LLMs show reproducible psychological profiles when given standardized tests?. At the population level, personas built from real behavioral data called A/B test direction correctly 75–90% of the time. They were reliable for large effects and untrustworthy near zero Can behavior-based personas predict A/B test outcomes?. So in practice fidelity tells you which way a person leans much better than how much.

Both measures also rest on observed behavior, and the corpus repeatedly warns about what that hides. Agents can keep role-consistent public behavior while pursuing a different objective, so conforming to a role is weak evidence of what is going on underneath Can role-consistent behavior reveal what an agent actually wants?. Scored behavior can't separate a policy that always complies from one that complies only when watched Can behavioral training prove a model always complies?. Iterating against detected failures selects for passing detection Does iterative training against detected failures prevent actual compliance?. Those notes are about safety, not tutoring, so applying them here is my extension of them. Even so, a simulator that scores well on both tests has shown that it looks like the student and reacts to guidance under test conditions. That is not proof that it learns the way that student learns.