SYNTHESIS NOTE
Topics›Design Frameworks›this note

How should AI systems respond when confidence is low?

When an AI feature lacks confidence in its output, what should the interface do? This explores whether low confidence should trigger fallback paths and how to frame uncertainty so users see it as fixable rather than terminal.

Synthesis note · 2026-10-21 · sourced from Design Frameworks

GitLab's Pajamas design system states a specific requirement for what happens when an AI feature is uncertain: "When your system is not certain of the user's intent or has low confidence, make sure there is a path forward that does not rely on AI. Explain why the system was not able to provide a recommendation." The guideline extends the same requirement to breakdowns in live interaction: "When agents cannot understand requests or flows encounter errors, provide clear explanations and alternative paths forward." It then splits the recovery behavior by interaction type — "agent conversations should gracefully handle misunderstandings, while flows should offer users the ability to retry, manually intervene, or fall back to manual processes" — treating a conversational agent's repair and an automated flow's repair as needing different mechanics even though both must leave the user an exit.

The reasoning given is that a confidence gap is informative, not just a failure to suppress. The guideline reframes errors explicitly: "Errors are also opportunities to learn more about your user's mental models and improve the system's ability to make recommendations." It then names the design implication directly — "consider designing a feedback mechanism that presents as a cue for adjustment rather than an error state" — so the interface framing of low confidence is meant to read as an invitation to correct the system's understanding, not as the system failing. This sits downstream of the same document's broader principle that AI should acknowledge uncertainty and defer to user expertise when appropriate, and that users should always "maintain final authority over their work."

This adds a specific, prescriptive behavior to How can users verify what an AI agent actually plans to do?: where that insight argues users need a way to check an agent's goals and actions to build common ground, GitLab's guideline specifies what the agent itself owes the user at the moment common ground breaks down — an explanation of the gap, framed as adjustable rather than terminal, plus a route that works even if the AI component is set aside entirely. It also gives a concrete, product-side instance of the same underlying worry as Do LLM agents misinterpret delegated requests more than formal protocols?: both treat the gap between what a system actually understood and what it claims or implies as something that needs an explicit handling mechanism, though GitLab's answer is an interface pattern for the confidence gap rather than a formal protocol for commitments.

The excerpt does not describe how "low confidence" is computed, what threshold triggers the fallback path, or how GitLab validated that framing errors as "a cue for adjustment" changes user behavior relative to framing them as errors; these are presented as design requirements on a documentation page, not as findings from a usability study. It is also GitLab's own prescriptive stance for its own product surfaces (Duo Chat, Amazon Q, custom agents), not a claim that other AI systems follow or should follow the identical recovery split between conversational agents and automated flows.

Inquiring lines that read this note 3

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How should organizations govern autonomy and access for agentic AI systems? Should AI systems make failures transparent so users can contest and adjust?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
13 direct connections · 129 in 2-hop network ·dense cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

GitLab's Pajamas design system says low-confidence AI should explain the gap and leave a path forward that does not depend on AI