Does an AI's simple gut-check about its own confidence beat a fancy trained system for deciding when to look something up?

This explores two rival ways of deciding *when* an AI should reach for external documents: a simple threshold on the model's own confidence versus a trained policy that learns the retrieve-or-not decision. The corpus stakes out a genuinely surprising position — the cheap option often wins. Calibrated token-probability uncertainty (essentially: "how sure is the model about the next words?") consistently beats multi-call adaptive retrieval systems on single-hop questions and matches them on multi-hop ones, while burning a fraction of the model and retriever calls Can simple uncertainty estimates beat complex adaptive retrieval?. The headline isn't just efficiency — it's that the model's *self-knowledge* turns out to be a more reliable trigger than external heuristics built to second-guess it.

But "confidence threshold" and "learned policy" aren't a clean binary, and the interesting tension is that confidence can live on either side of the line. The DeepRAG work reframes retrieval as a Markov Decision Process, where the model *learns* at each reasoning step whether to pull external knowledge or trust what's already in its weights — and reports a ~22% accuracy jump, much of it from *not* retrieving when retrieval would only inject noise When should language models retrieve external knowledge versus use internal knowledge?. So the learned policy's advantage isn't only knowing when to fetch; it's knowing when *not* to. That reframes the comparison: a static threshold treats every step the same, while a learned policy can become situation-aware.

Where it gets richer is that confidence itself can be *trained into shape* rather than just read off the model. One thread uses answer-span confidence as a reward signal, which both sharpens reasoning and reverses the calibration damage that RLHF tends to cause Can model confidence work as a reward signal for reasoning?. That matters directly here: a confidence threshold is only trustworthy if the confidence is *calibrated*, and calibration is itself something you can optimize. So the two camps quietly converge — a learned policy can be the thing that makes a confidence threshold worth trusting in the first place.

A second axis the corpus surfaces is *what* the learned signal supervises. Rewarding the final answer alone is weaker than supervising the intermediate retrieval steps — process-level feedback that contrasts good and bad retrieval chains substantially outperforms outcome-only training Does supervising retrieval steps outperform final answer rewards?. And learned routing doesn't stop at retrieve-or-not: StructRAG trains a router to pick *which kind* of knowledge structure (tables, graphs, chunks) fits the query, grounded in cognitive-fit theory Can routing queries to task-matched structures improve RAG reasoning?. Once you can learn a policy, the decision space expands well past a single threshold.

The sharpest caution comes from the failure-mode literature: triggering is named explicitly as one of three *structural* breakdowns in RAG — fixed-interval retrieval wastes context, and the fixes are architectural, not parameter-tuning Where do retrieval systems fail and why?. Read across the corpus, the lesson lands somewhere unexpected: don't ask "threshold or policy?" Ask whether your confidence signal is calibrated enough to threshold on — and if you're going to learn a policy, learn it over *when not to retrieve* and *what structure to retrieve*, not just a fancier yes/no. The cheapest reliable trigger may be a well-calibrated model honestly reporting its own doubt.