Can an AI really tell what to teach first, or is it just writing lessons that look right?
What specific tasks should evaluate whether models understand pedagogical sequencing?
This explores how you would test whether a model really grasps the order in which ideas should be taught (what must come before what, for which learner), rather than just producing lesson-shaped text.
This explores how you would test whether a model really grasps the order in which ideas should be taught, rather than just producing lesson-shaped text. The corpus has no benchmark built for this, so what follows is assembled from neighboring work on curricula, teacher-student fit and how evaluations get fooled. The most useful thing the corpus offers is a list of ways a sequencing test can be passed without any understanding.
Models are good at imitating form. Instruction tuning mostly teaches the shape of the output space, and models trained on deliberately wrong instructions score close to models trained on correct ones Does instruction tuning teach task understanding or output format?. Models trained on deliberately corrupted reasoning traces do about as well as those trained on correct traces Do reasoning traces need to be semantically correct?. Chain-of-thought looks like constrained imitation that degrades predictably when problems drift from familiar patterns Does chain-of-thought reasoning reveal genuine inference or pattern matching?, and up to 67% of reasoning errors trace to local memorization of preceding tokens Where do memorization errors arise in chain-of-thought reasoning?. So asking for a syllabus on 'intro to statistics' mostly tests recall of thousands of existing syllabi. Better tasks use invented subject matter, such as a made-up prerequisite graph of unfamiliar concepts, where no canonical order exists to memorize.
A first task is to separate 'what depends on what' from 'what goes first.' Models handle causal relations better than temporal ones, because causal connectives are explicit in text while temporal order is usually implicit Why do LLMs handle causal reasoning better than temporal reasoning?. Prerequisites are causal-flavored (B can't be understood without A), while a syllabus is a temporal linearization. A model could therefore label every dependency correctly and still scramble the sequence. Test both, then compare: mark the prerequisite pairs, and separately order a shuffled set of concepts. A second task is sequencing for a specific learner. Teacher-refined data hurts when it exceeds the student's learning frontier, even when it is objectively better Does teacher-refined data always improve student model performance?, which is a sequencing failure in miniature. Give the model a learner's demonstrated knowledge plus candidate explanations, and check whether it rejects the excellent-but-too-advanced one. Include cases where the textbook order is wrong for this learner. Those are low-probability answers, which is exactly where models are predicted to stumble even on logically simple tasks Can we predict where language models will fail?.
A third task is interventional: does the order actually help? Ordering effects are measurable in training. Imitation first and RLVR second beats either alone, because imitation makes the later rewards informative Does sequencing imitation then exploration training improve reasoning?. Training structured tasks before creative ones gained 6.2% over joint training by protecting open-ended ability from entropy collapse Does training order reshape how models handle different task types?. So hold the content fixed and vary the order, ask the model which order will work better for a given learner, then run it and score the prediction. Judge the teacher by what the student achieves, not by how impressive the teacher's own answers are. Walmart's cross-encoder students, for example, ended up beating their LLM teachers Can smaller models outperform their LLM teachers with enough data?.
A fourth task is to check whether the model's sense of difficulty matches the learner's. Sparsity-guided curriculum in-context learning orders demonstrations using the model's own last-layer activation sparsity as a difficulty signal, with no labels needed Can representation sparsity order few-shot demonstrations effectively?. Compare three rankings: that internal signal, the model's stated difficulty ranking, and where human learners actually struggle. Wherever they disagree, the model's sequencing will drift. Finally, score at more than one level. Understanding comes in tiers (concepts, facts about the world, compact principles), and the higher tiers coexist with cruder heuristics rather than replacing them Do language models understand in fundamentally different ways?. A single pass/fail number hides that patchwork. Ask the model for an order and for why the order is forced, then test that reason on a case where it shouldn't hold.
Sources 12 notes
Models trained on semantically empty or deliberately incorrect instructions achieve comparable performance to those trained on full correct instructions, achieving 43% vs random baseline 42.6%. The semantic content of instructions appears largely irrelevant; what transfers is knowledge of the output space.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
CoT works by constraining models to reproduce familiar reasoning patterns from training, not by enabling novel symbolic reasoning. Performance degrades predictably under distribution shifts—the signature of imitation rather than capability emergence.
STIM framework identifies local, mid-range, and long-range memorization sources in CoT reasoning. Local memorization—based on preceding tokens—accounts for up to 67% of reasoning errors, especially as complexity increases and distributional shift occurs.
ChatGPT excels at causal relations but struggles with temporal ordering because causal connectives are explicit and frequent in training data, while temporal order is often implicit and must be inferred contextually.
Show all 12 sources
Teacher-refined data degrades performance when it exceeds the student's learning frontier, even if objectively higher quality. Students should filter refinements using their own statistical profile to retain only compatible improvements.
By framing LLMs as autoregressive probability machines, researchers predicted tasks with low-probability target responses would be systematically harder, even when logically simple. Experiments confirmed predictions like backwards alphabet and letter counting.
Running Supervised RL first to establish reasoning foundations, then RLVR to refine against verifiable rewards, substantially outperforms both methods in isolation. The imitation phase makes outcome rewards informative by creating reasonable rollouts the RL phase can then sharpen.
Omni-Thinker shows structured domains decrease output entropy while creative domains increase it. BWT-guided scheduling—training structured tasks first—yields 6.2% gains over joint training by preventing entropy collapse from damaging open-ended capabilities.
Walmart's student cross-encoders outperformed their LLM teachers when trained on sufficiently large augmented datasets of teacher-labeled queries. The student's broader input distribution exposure, smoothed by teacher predictions, enabled better generalization than the teacher achieved.
Sparsity-Guided Curriculum In-Context Learning uses last-layer activation sparsity to order demonstrations from sparse (harder) to dense (easier), yielding considerable performance improvements. This approach requires no external difficulty labels and works across diverse in-context learning tasks.
Mechanistic interpretability reveals conceptual understanding (features as directions), state-of-world understanding (factual connections), and principled understanding (compact circuits). Crucially, higher tiers coexist with lower-tier heuristics rather than replacing them, creating a patchwork of capabilities.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
- Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexity
- Long-context LLMs Struggle with Long In-context Learning
- Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs
- Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
- Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning