Can an AI sense how hard a question is before answering — and think longer only when the problem actually demands it?

This explores how a system can estimate difficulty up front to allocate compute dynamically — spending less on easy prompts and more on hard ones. The corpus is unusually clear that the *payoff* here is real and large: inference effectiveness varies dramatically by prompt, and reallocating the same total budget adaptively beats both fixed budgets and simply scaling up the model Can we allocate inference compute based on prompt difficulty?. The same logic shows up at the level of individual reasoning traces: there's an optimal thinking length that shifts with difficulty, so a uniform token budget either over-thinks easy problems or under-thinks hard ones Does thinking time need to scale with problem difficulty?. And the harder the prompt, the more compute can substitute for raw model size — meaning the difficulty estimate is exactly what tells you *when* extra inference is worth it Can inference compute replace scaling up model size?, How should we spend compute at inference time?.

But here's the twist the corpus surfaces: the most tempting difficulty signal is a trap. You might assume that when a model produces a long chain of thought, it's working hard on a hard problem — so trace length could serve as a live difficulty gauge. It doesn't. Controlled maze experiments show trace length tracks difficulty only for problems the model has seen before; out of distribution the correlation collapses entirely, because trace length mostly reflects how close the problem is to training schemas the model can recall, not how much genuine computation it needs Does longer reasoning actually mean harder problems?. So a system that reads 'long output = hard, spend more' will systematically misallocate on exactly the novel problems where good allocation matters most.

The corpus points to a more reliable family of signals: internal *confidence*, measured locally rather than globally. Step-level confidence catches the moment a reasoning trace breaks down — a signal that global averaging smears away — and it lets a system stop early once a trace is clearly good or clearly failing, achieving the accuracy of majority voting with far fewer traces generated Does step-level confidence outperform global averaging for trace filtering?. That reframes 'estimate difficulty' as something you can do *while* solving, not only before: you keep generating until confidence resolves, so effort is allocated by the problem's actual resistance rather than by a guess made in advance.

There's also a structural route. Instead of one model both judging difficulty and solving, you can separate a decomposer that breaks a problem into steps from a solver that executes them — and the decomposition ability generalizes across domains even when solving doesn't Does separating planning from execution improve reasoning accuracy?. A decomposer that turns a problem into a step count is, in effect, a difficulty estimator whose output directly sizes the compute budget. The reward-model literature makes the complementary move: let the *evaluator* reason before it scores, spending variable test-time compute on how hard a judgment is Can reward models benefit from reasoning before scoring?.

Two cautions worth carrying away. First, dynamic allocation has a ceiling set by training, not inference: a non-reasoning model can't be rescued by pouring compute onto a hard prompt, because it never learned a protocol that makes extra tokens productive — difficulty-aware budgeting only helps a model that can actually use the budget Can non-reasoning models catch up with more compute?. Second, some 'hard' problems are hard for architectural reasons that more compute won't touch: autoregressive generation can't retract a committed token, so constraint-satisfaction problems stay out of reach no matter the budget Why does autoregressive generation fail at constraint satisfaction?. The most useful difficulty estimator, then, distinguishes not just easy-vs-hard but *worth-more-compute* vs *needs-a-different-mechanism*.