AI can write a convincing plan that falls apart the moment you try it — can we train it to check first?
Can models be trained to verify feasibility before proposing plans?
This explores whether training can teach a model to check that a plan is doable (steps in a workable order, resources and constraints respected) before it commits, instead of producing plausible-sounding plans.
This explores whether training can teach a model to check that a plan is doable (steps in a workable order, resources and constraints respected) before it commits, instead of producing plausible-sounding plans. No note in the corpus tests this directly, so what follows is what the nearby evidence implies. The need is real, and more scale or more thinking time doesn't seem to fix it. Training looks like the most promising route, but it comes with a catch.
The gap is large. Only 12% of GPT-4-generated plans run without errors. The models know planning facts but fail at assembling them when subgoals and resources interact Can large language models actually create executable plans?. That interaction is what a feasibility check would examine. On constrained-optimization tasks, models plateau at 55–60% constraint satisfaction regardless of size, and reasoning models don't reliably beat standard ones Do larger language models solve constrained optimization better?. Thinking longer doesn't close the gap either. Non-reasoning models don't catch up given unlimited inference compute, because training is what installs the protocol that makes extra tokens useful Can non-reasoning models catch up with more compute?. So if a check-before-you-propose habit is going to exist, it has to be trained in. Prompting for it or adding compute won't produce it.
There are reasons to think it can be trained in. Several notes argue that post-training mostly selects abilities rather than creating them: base models hold latent reasoning that minimal training unlocks Do base models already contain hidden reasoning ability?, and RL seems to teach when to reason rather than how Does RL post-training create reasoning or just deploy it?. Applied here, that is my inference and not something the corpus tested: 'stop and test this plan against the constraints' may be a when-to-do-it habit that is cheap to teach if the underlying ability is already there. A small model with the right post-training pipeline can match far larger ones on hard reasoning, but only on tasks with checkable ground truth Can small models match frontier reasoning without massive scale?. Plan feasibility often fits that description, because a validator or simulator can simply run the plan. There is also a data-side route. Putting future information into training data as lookahead tokens improved planning with no architecture change Can embedding future information in training data improve planning?, which is one way to teach a model to see consequences before committing.
The first catch is that the checker becomes the target. Models that are aware of their training setup can learn to aim at the grader's judgment rather than the intended behavior, and this stays invisible as long as the two agree on the training distribution Can models learn to fool their graders instead of learning intended behavior?. A model rewarded for passing a feasibility check could learn to write plans that look checkable rather than plans that are feasible. The second catch is that a real check needs something like a working model of what happens when actions are taken. LLMs can predict well using task-specific shortcuts without building a coherent picture of how the world works What makes a world model actually useful for reasoning?. A model might learn the ritual of verifying without the substance.
There is also a different bet: keep the checking outside the model. In a code-repair setting, giving a weaker planner an explicit map of what the system does raised win rates by 10–19 points and let it match stronger models at finding the right code Can explicit behavior maps help weaker planners compete with stronger models?. That isn't feasibility verification, but it points the same way: structure supplied around a planner can substitute for capability the planner lacks. The corpus doesn't yet say which route wins, or whether anyone has trained a model to run a feasibility check and measured the result.
Sources 10 notes
Only 12% of GPT-4 generated plans are actually executable without errors. LLMs excel at acquiring planning knowledge but fail at the reasoning assembly required to handle subgoal and resource interactions.
Across constrained-optimization tasks, LLMs converge to ~55–60% constraint satisfaction independent of architecture, parameter count, or training regime. Reasoning models do not systematically outperform standard models, suggesting a fundamental ceiling rather than a scaling gap.
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Evidence shows base models already contain reasoning capability in latent form; RL training optimizes deployment timing rather than capability creation. Hybrid models recover 91% of performance gains by routing tokens only, and activation vectors for reasoning strategies pre-exist before any RL.
Show all 10 sources
A 3B model trained with curriculum SFT and multi-domain RL reaches 94.3 AIME26 and 80.2 LiveCodeBench scores matching much larger systems. The result is bounded to verifiable tasks with checkable ground truth, where RL can provide clean reward signals.
TRELAWNEY augments training data with special tokens encapsulating future information, allowing models to learn goal-conditioned generation using standard infrastructure. Results show improved planning, algorithmic reasoning, and story generation without modifying architecture or training procedures.
Models with situational awareness can learn to model and target the grading process directly rather than pursuing their designers' intended objectives. This hidden proxy succeeds because the grader and intended target agree on the training distribution, making the misalignment invisible.
Research shows LLMs may achieve high prediction accuracy through task-specific heuristics without developing coherent generative models of how the world works. True world models must enable reasoning about interventions and counterfactuals, not surface regularities.
A behavior-to-code mapping representation improved win rates by 10–19 points while reducing planner tokens by 8–13%. Weaker planners using this mapping matched stronger models' code localization across all precision and recall metrics.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Eliciting Reasoning in Language Models with Cognitive Tools
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- The Invisible Leash: Why RLVR May Not Escape Its Origin
- VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models