Agentic Abstention: Do Agents Know When to Stop Instead of Act?

LLM agents are expected to act over multiple turns, using search, browsing interfaces, and terminal tools to complete user goals. Yet not every goal is well specified or achievable in the available environment. In such cases, a reliable agent should recognize that further interaction is unlikely to help and abstain from additional tool calls. We define Agentic Abstention, the problem of deciding when an agent should stop acting under uncertainty. Unlike standard LLM abstention, which is usually evaluated as a single-turn answer-or-abstain decision, agentic abstention is a sequential decision problem: an agent can answer, abstain, or gather more information at each turn, and the need to abstain may only become clear after interacting with the environment. We study this problem across web shopping, terminal environments, and question answering, evaluating 13 LLM-as-agent systems and 2 agent scaffolds on more than 28,000 tasks. Our results show that the main challenge is not only whether agents can abstain, but also when they abstain. Some agents never abstain when they should, while others do so only after many unnecessary interactions.

Introduction. Agents are designed to follow user instructions and act in an environment to complete tasks such as browsing the web [1], shopping online [2, 3], or operating in a terminal [4]. Agents accomplish these tasks by iteratively choosing actions, invoking tools, and incorporating observations from the environment to make progress toward the user’s goal. Prior work has largely focused on measuring and improving successful task completion, for example through better memory and context management [5], stronger planning and decision-making [6], and more effective agent architectures or scaffolds [7]. Recent work has begun to examine uncertainty [8, 9], inefficient or unsafe tool use [10, 11, 12], and clarification [13, 14, 15, 16] in agent settings, but these lines of work do not study abstention as a decision made by an agent across different scenarios. Existing evaluations therefore leave open how agents should behave when the requested task is infeasible, either due to ambiguity in the user instruction or because a task cannot be completed within the given environment and action space.

Discussion / Conclusion. We introduced Agentic Abstention, the problem of deciding when an LLM agent should stop acting rather than continue interacting with an environment. Across web, terminal, and interactive QA settings, we find that agents are challenged by the decision to abstain. Agents usually abstain too late or not at all, expending many wasteful actions, especially when a task initially appears feasible but becomes clearly unresolvable only after interaction with the environment. We further show that timely abstention can be improved through our method CONVOLVE, which distills interaction trajectories into reusable stopping rules that can be provided as context to the agent without the need to update model parameters. Our findings show that reliable agents require not only stronger task-completion abilities, but also better judgment about when continued action is no longer useful.