Principles of Mixed-Initiative User Interfaces

Source: Eric Horvitz (Microsoft Research); CHI 1999, conference paper · 1999

ABSTRACT Recent debate has centered on the relative promise of focusing user-interface research on developing new metaphors and tools that enhance users’ abilities to directly manipulate objects versus directing effort toward developing interface agents that provide automation. In this paper, we review principles that show promise for allowing engineers to enhance human—computer interaction through an elegant coupling of automated services with direct manipulation. Key ideas will be highlighted in terms of the LookOut system for scheduling and meeting management.

Introduction. There has been debate among researchers about where great opportunities lay for innovating in the realm of human— computer interaction [10]. One group of researchers has expressed enthusiasm for the development and application of new kinds of automated services, often referred to as interface “agents.” The efforts of this group center on building machinery for sensing a user’s activity and taking automated actions [4,5,6,8,9]. Other researchers have suggested that effort focused on automation might be better expended on exploring new kinds of metaphors and conventions that enhance a user’s ability to directly manipulate interfaces to access information and invoke services [1,13]. Innovations on both fronts have been fast paced. However, there has been a tendency for a divergence of interests and methodologies versus focused attempts to leverage innovations in both arenas.

We have pursued principles that provide a foundation for integrating research in direct manipulation with work on interface agents. Our goal is to avoid focusing solely on one tack or the other, but to seek valuable synergies between the two areas of investigation. Surely, we should avoid building complex reasoning machinery to patch fundamentally poor designs and metaphors. Likewise, we wish to avoid limiting designs for human–computer interaction to direct manipulation when significant power and efficiencies can be gained with automated reasoning. There is great opportunity for designing innovative user interfaces, and new human–computer interaction modalities by considering, from the ground up, designs that take advantage of the power of direct manipulation and potentially valuable automated reasoning [2].

Related work. In related work in user modeling, probabilistic models of a user’s goals have been employed to continue to perform real-time inference about the probability of alternate feasible goals as a function of observables including the current program context, a user’s sequence of actions and choice of words used in a query [4,6]. Some of this work has leveraged recent successes in building and reasoning with Bayesian network models [7,11].

Method. PRINCIPLES FOR MIXED-INITIATIVE UI Key problems with the use of agents in interfaces include poor guessing about the goals and needs of users, inadequate consideration of the costs and benefits of automated action, poor timing of action, and inadequate attention to opportunities that allow a user to guide the invocation of automated services and to refine potentially suboptimal results of automated analyses. In particular, little effort has been expended on designing for a mixedinitiative approach to solving a user’s problems—where we assume that intelligent services and users may often collaborate efficiently to achieve the user’s goals.

Critical factors for the effective integration of automated services with direct manipulation interfaces include:

(1) Developing significant value-added automation. It is important to provide automated services that provide genuine value over solutions attainable with direct manipulation.

(2) Considering uncertainty about a user’s goals. Computers are often uncertain about the goals and current the focus of attention of a user. In many cases, systems can benefit by employing machinery for inferring and exploiting the uncertainty about a user’s intentions and focus.

(3) Considering the status of a user’s attention in the timing of services. The nature and timing of automated services and alerts can be a critical factor in the costs and benefits of actions. Agents should employ models of the attention of users and consider the costs and benefits of deferring action to a time when action will be less distracting.

(4) Inferring ideal action in light of costs, benefits, and uncertainties. Automated actions taken under uncertainty in a user’s goals and attention are associated with context-dependent costs and benefits.

The value of automated services can be enhanced by guiding their invocation with a consideration of the expected value of taking actions.

(5) Employing dialog to resolve key uncertainties. If a system is uncertain about a user’s intentions, it should be able to engage in an efficient dialog with the user, considering the costs of potentially bothering a user needlessly.

(6) Allowing efficient direct invocation and termination. A system operating under uncertainty will sometimes make poor decisions about invoking— or not invoking—an automated service. The value of agents providing automated services can be enhanced by providing efficient means by which users can directly invoke or terminate the automated services.

(7) Minimizing the cost of poor guesses about action and timing. Designs for services and alerts should be undertaken with an eye to minimizing the cost of poor guesses, including appropriate timing out and natural gestures for rejecting attempts at service.

(8) Scoping precision of service to match uncertainty, variation in goals. We can enhance the value of automation by giving agents the ability to gracefully degrade the precision of service to match current uncertainty. A preference for “doing less” but doing it correctly under uncertainty can provide user’s with a valuable advance towards a solution and minimize the need for costly undoing or backtracking.

(9) Providing mechanisms for efficient agent−user collaboration to refine results. We should design agents with the assumption that users may often wish to complete or refine an analysis provided by an agent.

(10) Employing socially appropriate behaviors for agent−user interaction. An agent should be endowed with tasteful default behaviors and courtesies that match social expectations for a benevolent assistant.

(11) Maintaining working memory of recent interactions. Systems should maintain a memory of recent interactions with users and provide mechanisms that allow users to make efficient and natural references to objects and services included in “shared” short-term experiences.

(12) Continuing to learn by observing. Automated services should be endowed with the ability to continue to become better at working with users by continuing to learn about a user’s goals and needs.

Conclusion. SUMMARY AND CONCLUSIONS We reviewed key challenges and opportunities for building mixed-initiative user interfaces—interfaces that enable users and intelligent agents to collaborate efficiently. We first presented a set of principles for designing mixedinitiative user interfaces that address systematic problems with the use of agents that may often have to guess about a user’s needs. Then, we focused on methods for managing the uncertainties that agents may have about users’ goals the uncertainties that agents may have about users’ goals and focus of attention. We discussed the consideration of uncertainty, as well as the expected costs and benefits of taking autonomous action in different situations. We highlighted methods and design principles with examples drawn from the LookOut system. Research on LookOut has elucidated difficult challenges and promising opportunities for improving human–computer interaction through the elegant combination of reasoning machinery and direct manipulation. We believe continuing efforts to address problems with the design of mixed-initiative user interfaces will likely yield fundamental enhancements in human−computer interaction.