PleaSQLarify: Visual Pragmatic Repair for Natural Language Database Querying

Abstract Natural language database interfaces broaden data access, yet they remain brittle under input ambiguity. Standard approaches often collapse uncertainty into a single query, offering little support for mismatches between user intent and system interpretation. We reframe this challenge through pragmatic inference: while users economize expressions, systems operate on priors over the action space that may not align with the users’. In this view, pragmatic repair—incremental clarification through minimal interaction—is a natural strategy for resolving underspecification. We present PleaSQLarify, which operationalizes pragmatic repair by structuring interaction around interpretable decision variables that enable efficient clarification1. A visual interfacecomplements this by surfacing the action space for exploration, requesting user disambiguation, and making belief updates traceable across turns. In a study with twelve participants, PleaSQLarify helped users recognize alternative interpretations and efficiently resolve ambiguity. Our findings highlight pragmatic repair as a design principle that fosters effective user control in natural language interfaces.

Introduction. Natural language interfaces to data systems allow users to query and interact with databases using everyday language [12, 13, 37, 38]. For instance, a data analyst might type "show me last quarter’s top customers" into a natural language database interface to explore an unfamiliar dataset. These interactions feel natural because they allow people to express intent in their own words, without needing technical commands or formal syntax. Yet the same naturalness also makes them fragile: such requests are often ambiguous, and systems typically sample the most probable interpretation and perform the corresponding system action [13, 23, 39]. This challenge is particularly prominent in natural language programming, such as text-to- SQL [12, 20, 38, 44], where natural utterances must be translated into precise database queries. Especially in such tasks, seemingly straightforward requests often have more than one valid interpretation [35]. Evidence from the CoSQL dataset [41] underscores this point: across the 2’459 interactions where users asked SQL experts to write basic SQL queries conversationally, 31.9% of which contained an instruction requiring clarification of the user’s intent. Although some systems attempt to manage such ambiguity through interactive interfaces [21, 28], current systems still do not expose it effectively or provide robust guiding mechanisms for clarification. Taking a step back, this problem reflects a basic property of language use: underspecification. Pragmatic theories of language use [11, 14, 31] describe humans as economical speakers who deliberately leave details unspecified, relying on listeners to recover the intended meaning from context and shared knowledge. Such underspecification is a natural outcome of human communication; however, in natural language interfaces, it leaves systems with multiple plausible actions and users unaware of which ones the system considered or why. Akin to clarification in human communication, we propose that ambiguity should be interactively engaged with directly through pragmatic repair. Namely, in human conversation, underspecified expressions are routinely clarified incrementally: speakers add just enough information to distinguish their intent from alternatives, and listeners offer targeted requests for clarification. By analogy, natural language interfaces may resolve ambiguity more effectively by surfacing alternative interpretations and asking for clarifications that are both efficient and intelligible to users. To operationalize this idea, we first propose a general framing of pragmatic repair in natural language interfaces, which we then instantiate for text-to-SQL, presenting PleaSQLarify. As showcased in Figure 1, the approach combines an algorithm that selects informative, grouped decision variables with a visual interface that (1) surfaces the model’s action space, (2) supports guided narrowing through contextualized clarifications, and (3) makes updates traceable across turns. Rather than forcing users to accept a single opaque query, the system invites them to explore, compare, and refine interpretations until a satisfactory query is reached. In a study with twelve participants, PleaSQLarify helped users recognize alternative interpretations they had not anticipated and efficiently steer toward satisfactory results. Interaction traces revealed three utterance clarification workflows—broad exploration, targeted prefiltering, and output-driven assembly—that illustrate how users flexibly adapted repair strategies to their goals and the complexity of the task. Overall, this paper makes four contributions:

• A conceptual framing of pragmatic repair for natural language interfaces. We extend theories of pragmatic inference to HCI, showing how underspecification in user utterances can be used as a resource for incremental clarification. • An algorithm for pragmatic repair in natural language interfaces, and an instantiation for text-to-SQL. The algorithm groups atomic features into semantically interpretable decision variables and selects among them by expected information gain, enabling clarification that is both efficient and meaningful. A quantitative evaluation on the AMBROSIA dataset shows that this approach reduces semantic uncertainty in fewer turns than baselines. • A visual interface that operationalizes pragmatic repair for text-to-SQL by making the model’s action space visible, allowing users to surface informative decision variables, and maintaining traceability of updates across turns.

• Empirical insights from a user study that show the effectiveness of the proposed solution and design implications for supporting repair in natural language interfaces more broadly.

Related work. This section surveys the current state of research on the concept of ambiguity in text-to-SQL, as well as algorithm- and user feedbackbased disambiguation approaches.

Ambiguity in Text-to-SQL. A recent survey by Liu et al. [21] identifies ambiguity as a critical unresolved challenge in natural language interfaces for databases. Various forms of ambiguity, such as scope and vagueness, have been identified by researchers, prompting the creation of taxonomies to organize them systematically (e.g., [6, 10]). Several benchmarks have emerged to address the different types of ambiguity, including the CoSQL dataset [41], which contains over 10’000 annotated SQL queries with clarified ambiguities and flagged unanswerable questions; SPLASH [9], which contains utterances with incorrect SQL interpretations and natural language feedback; AmbiQT [2], which was created using a novel decoding algorithm that combines plan-based template generation with constrained infilling; and AMBROSIA [35], a benchmark that includes three distinct types of ambiguity: scope ambiguity, attachment ambiguity, and vagueness, with queries designed to support multiple valid interpretations reflecting realworld complexity. In this work, we employ the AMBROSIA dataset both for the quantitative evaluation of our proposed algorithm and for the extraction of query samples used in the user study.

Algorithm-Based Disambiguation. In recent years, several algorithmic approaches have been proposed for natural language query disambiguation used at different stages of the text-to-SQL pipeline. For instance, Kim et al. [18] introduce APA (Alignment with Perceived Ambiguity) pipeline that aligns LLMs to manage ambiguous queries before their execution. Another example is the “Disambiguate First, Parse Later” paradigm that uses a two-stage approach, initially resolving ambiguity by generating all possible natural language meanings and subsequently parsing each resulting unambiguous interpretation [36]. Wang et al. [40] propose a weakly supervised model, DTE (Detecting-Then-Explaining), to handle ambiguous and unanswerable questions by framing token localization as a sequence labeling task, tagging each token as ambiguous, unanswerable, or other. Ambiguity resolution can also be carried out during SQL translation. For instance, Chen et al. [5] use pre-trained classifiers alongside conformal prediction methods to assess the likelihood of error detection during the translation step. Recent contemporaneous work by Qiu et al. [33] proposes an SQL disambiguation method that utilizes similarity-agnostic, clause-level expected information gain while neglecting the functional equivalence between queries and the interaction between features.

Method. Pragmatic Inference and RSA. First, let u∈U be a natural language utterance chosen by a user from the set of possible utterances U to communicate an intended meaning m∗∈M. In the rational speech act theory (RSA) of probabilistic pragmatics [11], communicative efficiency is captured as Pragmatic Repair. In human–human communication, utterance grounding refers to the interactive process by which speaker and listener establish mutual understanding that the utterance has been correctly interpreted in context, thereby aligning their beliefs about m∗. If there is shared common ground between speaker and listener, their priors are well-aligned. In such cases, even relatively underspecified utterances may yield a low-entropy, peaked posterior pL(m| u). The pragmatic mechanism through which mutual understanding is maintained is pragmatic repair, that is, the incremental clarification that occurs when an utterance is ambiguous or underdetermined. In Clark and Wilkes-Gibbs’s [7] collaborative model of reference, pragmatic repair proceeds incrementally: speakers often begin with a minimal, sometimes general, description of a referent and refine it step-by-step, adding just enough new information to distinguish the intended meaning from remaining alternatives. This coarse-to-fine refinement reflects the pragmatic principle of least collaborative effort [14], in which interlocutors avoid over-specification up front and instead contribute additional details only when needed to resolve remaining uncertainty.

When users issue underspecified commands, natural language interfaces must somehow recover the intended meaning. We argue that this challenge can be understood through the lens of pragmatic inference: users economize their expressions, while systems operate on priors that may not align with user expectations. In this section, we formalize this misalignment and discuss pragmatic repair as a principled strategy for clarification, using the text-to-SQL domain as a running example for clarity.

4.1 Pragmatic Inference in Natural Language Interfaces m∗∉dom(f). In our example in Figure 2, the user may ask for the “product label”, even though no “prod_label” exists in the database. More generally, there may be misalignment between the user’s and the system’s priors over system actions. In other words, a user may be unaware of what system actions the system considers probable and may fail to estimate how well a produced utterance u distinguishes m∗from alternative probable intents. In the example of Figure 2, a system may be generally biased to show all columns, i.e., produce a= “SELECT * FROM sales”. The user is unaware that this is something they need to distinguish against and assumes that their utterance u=“What products do we sell?” yields a peaked posterior pL(m| u) at m∗. However, to the system, the utterance is ambiguous; its resulting posterior pL(m| u) is high-entropy, and sampling from it leads to unexpected behavior. Thus, communicative efficiency in natural language interfaces depends not only on the informativeness of utterances under shared priors, but also on aligning the priors themselves. This alignment can be facilitated by (a) surfacing the action space A to the user, thereby reducing uncertainty about what outcomes are possible, and (b) adapting or calibrating the system’s prior to better reflect the user’s expectations, for instance, through clarification mechanisms.

4.2 Interactive Utterance Grounding as Pragmatic Repair While surfacing the available action space A can help users form more accurate queries, directly exposing the full set of actions is often impractical and cognitively costly, especially when A is large or highly technical. Naturally, the question arises whether— akin to utterance grounding in pragmatics—underspecified user queries can also be interactively grounded in meaning through such clarifying repairs. To this end, a system may leverage the posterior distribution pL(m| u) from the user’s initial utterance to focus only on actionable candidate meanings.

Discussion. Our study of PleaSQLarify not only demonstrates its effectiveness for text-to-SQL repair but also points to broader lessons for the design of interactive natural language interfaces. The patterns we observed suggest several principles for how interfaces can better support users in clarifying underspecified intent and resolving ambiguity.

(1) Understanding ambiguity. In structured natural language tasks such as text-to-SQL, ambiguity is often multi-layered, with several distinct interpretations coexisting without the user’s awareness. As a result, resolving such ambiguity is a decision process that unfolds over multiple interaction turns. Participants could consistently align the system with their intent when the interface surfaced this broader interpretation space, allowing them to recognize alternative meanings of their utterances and compare their outcomes.

(2) Clarifying pragmatically. Participants could steer the system effectively when clarification steps reduced the candidate space in ways that were both efficient and interpretable. Efficiency was supported by prioritizing decision variables, whereas interpretability emerged when decisions were contextualized: participants understood choices best when they were grounded in concrete examples, linked to their role in the query, or surfaced as implicit choices that the system made on their behalf. (3) Accommodating heterogeneous repair strategies. The three workflows discussed in Section 10.4 the complementary strengths of the Action Space, Decision Space, and Predicted Query, as well as the flexibility with which participants appropriated them. The Action Space supported broad exploration, the Decision Space provided structured and prioritized guidance, and the Predicted Query helped participants monitor convergence. Rather than enforcing a single path, users of future systems may benefit from having different strategies available—from open-ended clarification, to efficient targeting, to output-driven reasoning—depending on user goals and task complexity.

More broadly, pragmatic repair extends beyond SQL. The challenge of misaligned priors outlined in Section 3 and the framework of pragmatic repair presented in Section 4.2 apply to most natural language interfaces. By combining algorithmic selection of informative clarifications with visual techniques that reveal alternatives and make consequences observable, systems can leverage ambiguity as a resource for collaborative problem solving—supporting user control even when model interpretations misalign with user intent.

Conclusion. We presented PleaSQLarify, a system that operationalizes pragmatic repair for interactive text-to-SQL disambiguation. The system combines an algorithm that selects informative decision variables with a visual interface that surfaces the model’s action space, requests incremental user clarifications, and makes updates traceable across turns. In a study with twelve participants, it helped users discover interpretations they had not anticipated and feel more in control of the generated outputs by efficiently and understandably guiding them toward satisfactory results. The diversity of usage workflows further highlights the importance of supporting heterogeneous repair strategies. More broadly, this work demonstrates how ambiguity in natural language interfaces can be turned into a resource for collaborative problem solving, strengthening user control even when model interpretations misalign with user intent.

Limitations. 11.2 Limitations and Future Work Experimental Scope. As the scope of the study is about intent clarification, the experimental setup (cf. Section 5) assumes the user has a well-formed intent that can be reasonably mapped to a valid SQL query. Other interesting related scenarios that were observed during the user study—such as when a user is unsure about their intent or changes their intent—are not covered by the experiments.

Limited Candidate Pool. To control latency and computational cost, the candidate set was fixed to the initial set of around 50 language model generations. Although this may be enough to give a user an insight into the model’s probable action space, it failed to adapt more dynamically to user requests. As a result, some participants reported not being able to find their exact intended query (P5, P7, P9). An interesting extension may be to update the candidate pool dynamically, given the user’s input; however, this is not a feasible option with the current setup. As shown in Figure 5, when evaluated on the AMBROSIA dataset, our algorithm—with optimal clarification behavior7 and a fixed candidate pool—takes two to eight clarification steps until converging on the target query.