Guidelines for Human-AI Interaction

Source: Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi et al. (Microsoft); CHI 2019, conference paper · 2019

ABSTRACT Advances in artifcial intelligence (AI) frame opportunities and challenges for user interface design. Principles for human- AI interaction have been discussed in the human-computer interaction community for over two decades, but more study and innovation are needed in light of advances in AI and the growing uses of AI technologies in human-facing applications. We propose 18 generally applicable design guidelines for human-AI interaction. These guidelines are validated through multiple rounds of evaluation including a user study with 49 design practitioners who tested the guidelines against 20 popular AI-infused products. The results verify the relevance of the guidelines over a spectrum of interaction scenarios and reveal gaps in our knowledge, highlighting opportunities for further research. Based on the evaluations, we believe the set of design guidelines can serve as a resource to practitioners working on the design of applications and features that harness AI technologies, and to researchers interested in the further development of guidelines for human-AI interaction design.

Introduction. Advances in artifcial intelligence (AI) are enabling developers to integrate a variety of AI capabilities into user-facing systems. For example, increases in the accuracy of pattern recognition have created opportunities and pressure to integrate speech recognition, translation, object recognition, and face recognition into applications. However, as automated inferences are typically performed under uncertainty, often producing false positives and false negatives, AI-infused systems may demonstrate unpredictable behaviors that can be disruptive, confusing, ofensive, and even dangerous. While some AI technologies are deployed in explicit, interactive uses, other advances are employed behind the scenes in proactive services acting on behalf of users such as automatically fltering content based on inferred relevance or importance. While such attempts at personalization may be delightful when aligned with users’ preferences, automated fltering and routing can be the source of costly information hiding and actions at odds with user goals and expectations.

AI-infused systems1 can violate established usability guidelines of traditional user interface design (e.g., [31, 32]). For example, the principle of consistency advocates for minimizing unexpected changes with a consistent interface appearance and predictable behaviors. However, many AI components are inherently inconsistent due to poorly understood, probabilistic behaviors based on nuances of tasks and settings, and because they change via learning over time. AIinfused systems may react diferently depending on lighting or noise conditions that are not recognized as distinct to end users. Systems may respond diferently to the same text input over time (e.g., autocompletion systems suggesting diferent words after language model updates) or behave diferently from one user to the next (e.g., search engines returning diferent results due to personalization). Inconsistent and unpredictable behaviors can confuse users, erode their confdence, and lead to abandonment of AI technology [7, 22]. Errors are common in AI-infused systems, rendering it difcult to reliably achieve the principle of error prevention. This has contributed to the large and growing body of work on AI explanations and interpretability to support human verifcation of proposed actions aimed at reducing the likelihood of unwarranted or potentially dangerous actions and costly outcomes (e.g., [14, 21, 23, 36, 38, 44]).

For over 20 years, the human-computer interaction (HCI) community has proposed principles, guidelines, and strategies for designing user interfaces and interaction for applications employing AI inferences (e.g., [16, 17, 33]). However, the variability of AI designs (e.g., varying capabilities and interaction styles of commercial conversational agents impacting user engagement and usability [26]) and high-profle reports of failures, ranging from humorous and embarrassing (e.g., autocompletion errors [8]) to more serious harm when users cannot efectively understand or control an AI system (e.g., collaboration with semi-autonomous cars [41]), show that designers and developers continue to struggle with creating intuitive and efective AI-infused systems. Ongoing advances in AI technologies will generate a stream of challenges and opportunities for the HCI community. While such developments will require ongoing studies and vigilance, we also see value in developing reusable guidelines that can be shared, refned, and debated by the HCI community. The development and use of such shared guidelines can help with the design and evaluation of AI-infused systems that people can understand, trust, and can engage with efectively.

In this work, we synthesize over 20 years of learning in AI design into a small set of generally applicable design guidelines for human-AI interaction. Specifcally, our contributions are:

• A codifcation of over 150 AI-related design recommendations collected from academic and industry sources into a set of 18 generally applicable design guidelines for human- AI interaction (see Table 1). • A systematic validation of the 18 guidelines through multiple rounds of iteration and testing.

We hope these guidelines, along with our examination of their applications in AI-infused systems, will serve as a resource for designers working with AI and will facilitate future research into the refnement and development of principles for human-AI interaction.

Related work. For over 20 years, the academic community has proposed numerous guidelines and recommendations for how to design for efective human interaction with AI-infused systems. For example, Norman [33] and Höök [16] both recommended building in safeguards like verifcation steps or controlling levels of autonomy to help prevent unwanted adaptations or actions from intelligent systems. Others recommended managing expectations so as not to mislead or frustrate users during interaction with unpredictable adaptive agents [16, 20, 33]. Horvitz’s formative paper on mixed-initiative systems [17] proposed principles for balancing autonomous actions with direct manipulation constructs, such as supporting user-driven invocation of intelligent services, scoping actions based on inferred goals and confdences, and inferring ideal action in light of costs, benefts, and uncertainties. The latter guideline was operationalized via the introduction of a decision-theoretic methodology to guide decisions about acting on AI inferences versus waiting for user input, based on consideration of expected costs and benefts of performing AI automation under uncertainty.

In some cases, specifc AI design recommendations have received considerable attention within the academic community. For example, a large body of work exists and continues to grow around how to increase transparency or explain the behaviors of AI systems (e.g., [14, 21, 23, 36, 38, 44], to name a few). Similarly, when and how to automatically adapt or personalize interfaces has been studied extensively in a variety of scenarios (e.g., [9, 11–13]).

Others in the community have studied how to design for specifc human-AI interaction scenarios. For example, researchers have been studying how to efectively interact with intelligent agents for many years (e.g., [18, 33]). This scenario has also had a recent resurgence of interest given advances in natural language processing and embedded devices driving the proliferation of conversational agents [26, 29, 35]. Similarly, researchers have for decades studied human interaction with intelligent context-aware computing systems including how to design for understandability and control of the underlying sensing systems [3, 23] and how to support ambiguity resolution [10]. Recent advances in sensing technologies and the widespread availability of commercial ftness and activity trackers have continued to drive interaction research in these domains [37, 45].

Method. 3 PHASE 1: CONSOLIDATING GUIDELINES We gathered AI design recommendations from three sources:

• A review of AI products and guidelines originating from industry. We collected guidelines asserted internally in our company and externally, and grouped them into themes; we audited a sample of AI products within and outside our company against the themes; and cross-referenced themes with internal customer feedback (reviews and bugs reported about our company’s AI products). • Recent public articles and editorials about AI design (e.g., [2, 24, 25, 39]). • Relevant scholarly papers about AI design (see Related Work section).

While we drew AI design guidelines from the academic literature, the list we captured may not be exhaustive because, as discussed in Related Work, potential design guidelines are often not presented explicitly as such, making them difcult to search for via terms or combinations of terms such as “AI”, “machine learning”, “design”, “principle” or “guideline”. Further, as the feld is evolving rapidly, we found the most up to date guidance about AI design in industry sources via articles published in the public domain.

From these sources, we obtained 168 potential AI design guidelines. Three members of our team conducted an asynchronous afnity diagramming process, clustering the guidelines into related concepts. This resulted in 35 concepts which we then fltered by removing concepts we deemed to be either too vague to design for directly (e.g., “build trust”), too specifc to a particular AI scenario (e.g., “establish that the bot is not human”), or not AI specifc (e.g., “display output effectively”). Filtering reduced our set of concepts to 20, each of which we then summarized in a sentence or phrase, forming our frst iteration of the guidelines. We organized the guidelines into four top-level categories based on when during the user’s interaction they applied: “Initially” (Guideline 1 & Guideline 2), “During interaction” (Guideline 3 - Guideline 6), “When wrong” (Guideline 7 - Guideline 11), and “Over time” (Guideline 12 - Guideline 18). Next, we tested the guidelines via a modifed heuristic evaluation.

4 PHASE 2: MODIFIED HEURISTIC EVALUATION We conducted an evaluation to test and iterate on the initial set of 20 AI design guidelines. We modeled our study after a heuristic evaluation [31], a common discount usability testing method where evaluators examine an interface for violations of a given set of usability guidelines. As the primary goal was to evaluate our design guidelines rather than to evaluate an interface, we modifed the heuristic evaluation by asking evaluators to attempt to identify both applications and violations of the proposed guidelines in an interface and to refect on the guidelines themselves during the evaluation.

Eleven members of our team participated in this evaluation. Team members selected AI-infused products or features of their choice and then looked for applications or violations of our initial set of design guidelines over a one-hour period. In total, we inspected 13 AI-infused products or features including: two diferent email products with a feature for fltering unimportant emails, a navigation system, an e-commerce website with product recommendations, two photo organization products, a design assistance feature in a productivity software, a research assistance feature in a productivity application, a social network news feed feature, a web search service, and an image search service. These products were diferent from the products used in Phase 3.

After the modifed heuristic evaluation, we reviewed the fndings and refections about each guideline and discussed issues and revision strategies for conficting interpretations and ambiguities. For example, our initial phrasing of Guideline 9 (“allow efcient correction”) and Guideline 17 (“allow coarse controls”) caused several evaluators to confuse instance-level corrections with global-level settings (several evaluators identifed adjusting settings as applications of Guideline 9 rather than Guideline 17). We subsequently rephrased Guideline 17 to include the term “global”.

Discussion. & FUTURE WORK We synthesized guidance proposed over the past 20 years about the design of human-AI interaction into a set of 18 AI usability guidelines. These guidelines were iteratively refned in four phases by a team of 11 researchers, and were applied or reviewed by an additional 60 designers and usability practitioners. Over the various stages of development, the guidelines were applied to AI-infused products across 10 product categories. These eforts provide evidence for the relevance of the guidelines across a wide range of common AI-infused systems. In terms of utility, we anticipate the guidelines will be useful to evaluate existing products and emerging design ideas. Our evaluation methods show that the guidelines lend themselves well to usability inspection methods such as heuristic evaluation. Future work could examine the uses and value of these guidelines at various stages of design.

Our decisions to optimize for generality, and to focus on observable properties, serve as a reminder that interaction designers routinely encounter these types of trade-ofs. We anticipate situations where there will be interactions and trade-ofs in attempts to employ several of the guidelines. As an example, if a system uses a complex or deep model to achieve a high level of performance, it may be challenging to both convey the consequences of user actions (Guideline 16), while also actively learning from user behavior (Guideline 13). Further research is necessary to understand the implications of these potential interactions and trade-ofs for the design of AI systems and to understand how designers employ these guidelines "in the wild."

Conclusion. We proposed and evaluated 18 generally applicable design guidelines for human-AI interaction. We distilled the guidelines from over 150 AI-related design recommendations and validated them through three rounds of evaluation. We are hopeful that application of these guidelines will result in better, more human-centric AI-infused systems, and that our synthesis can facilitate further research. As the current technology landscape is shifting towards the increasing inclusion of AI in computing applications, we see signifcant value in working to further develop and refne design guidelines for human-AI interaction.

Limitations. We recognize that there is a tradeof between generality and specialization, and that these guidelines might not adequately address all types of AI-infused systems. For example, we reported that some guidelines do not directly apply to AI systems that lack graphical user interfaces (e.g., voice-based virtual assistants and activity trackers). Additional guidelines may be necessary to help designers and developers create intuitive and efective products with these properties or in these product categories. Likewise, specialized guidelines may be required in certain high-risk or highly regulated areas such as semi-autonomous vehicles, robot-assisted surgery, and fnancial systems. We hope the 18 guidelines presented here and their validation process stimulate and inform future research into the development of domain-specifc guidance.

Our work also intentionally focused on AI design guidelines that we believed could be easily evaluated by inspection of a system’s interface. For example, we excluded broad principles such as "build trust", and focused instead on specifc and observable guidelines that are likely to contribute to building trust. Previous work, however, has proposed guidelines that impact the usability of AI-infused systems but must be considered when constructing the AI model.