Can prompts alone hold back a capable tutor model?

The paper argues that a tutor's value depends on declining to give an answer it could "easily produce," and that a "capable model pressed by a frustrated student does not withhold reliably on a prompt alone." The motivating evidence is borrowed from a randomized trial of roughly a thousand secondary-school students (Bastani et al.), cited rather than run here. In that trial an unguarded chatbot raised practice performance but lowered later exam performance without the tool. A Socratically guarded version of the same model kept the practice gain and removed the later loss. The authors' response is to treat withholding as "a per-turn, machine-checkable contract" and to put the enforcement outside the model.

The mechanism has three layers. A non-LLM policy core, reading only trusted learner state, sets a per-turn ceiling on an eight-rung help ladder. A deterministic detector strips solution code from replies. A separate LLM judge checks each risky reply against the contract. The discussion names the policy core, the choice it labels P2, as "the single most stabilizing choice." Because the ceiling is computed from trusted state, the property is "both injection-proof and unit-testable, so prompt edits could never silently weaken it." The general rule they draw is "put the irreversible decision in code, not in a prompt." Tuning is a separate loop: scripted student personas run through the live pipeline, a stronger model re-scores them, and each rejection's stated reason is recorded so failures are fixed by cause. The authors say violations are ordered "gross-to-subtle" and that this loop walks down that order.

The paper also frames the problem as "refusal-under-knowledge." Classical safety work studies refusing content a model should not produce. This is "the harder converse," refusing content it should and can produce, to a user who can see that it can. The claim sits alongside Why do better reasoning models ignore instructions?, which reports that more capable reasoning models are less controllable by instruction. Both point to the same conclusion: instructions in a prompt are a weak place to hold a constraint. It contrasts with Can models learn to ask clarifying questions without explicit training?, where restraint from premature answering arises through training, not through an external gate. The tutor problem is also the inverse of Why can't advanced AI models take initiative in conversation?: there the model under-steers, while here it over-helps and must be held back.

The excerpt does not establish how often a prompt-only tutor fails, so the "does not withhold reliably" premise is asserted, not measured. It gives no rates for the guarded pipeline and does not define the eight rungs or the contents of the contract. Its own evaluation uses scripted personas and a stronger-model re-scorer with no human subjects, so it shows nothing about whether students learn more from the guarded tutor. The claim that nothing in the calibration loop is specific to data structures or education is stated in the discussion but not tested in the excerpt. What the passages support is a design principle: where a property must hold against user pressure and cannot be allowed to erode through prompt revision, compute it in deterministic code from trusted inputs and let the model only fill in the reply beneath that ceiling.