Human Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis

Source: Linda Onnasch, Christopher D. Wickens, Huiyang Li, Dietrich Manzey; Human Factors 56(3), journal article (meta-analysis) · 2014

Objective: We investigated how automationinduced human performance consequences depended on the degree of automation (DOA). Background: Function allocation between human and automation can be represented in terms of the stages and levels taxonomy proposed by Parasuraman, Sheridan, and Wickens. Higher DOAs are achieved both by later stages and higher levels within stages. Method: A meta-analysis based on data of 18 experiments examines the mediating effects of DOA on routine system performance, performance when the automation fails, workload, and situation awareness (SA). The effects of DOA on these measures are summarized by level of statistical significance. Results: We found (a) a clear automation benefit for routine system performance with increasing DOA, (b) a similar but weaker pattern for workload when automation functioned properly, and (c) a negative impact of higher DOA on failure system performance and SA. Most interesting was the finding that negative consequences of automation seem to be most likely when DOA moved across a critical boundary, which was identified between automation supporting information analysis and automation supporting action selection. Conclusion: Results support the proposed cost– benefit trade-off with regard to DOA. It seems that routine performance and workload on one hand, and the potential loss of SA and manual skills on the other hand, directly trade off and that appropriate function allocation can serve only one of the two aspects. Application: Findings contribute to the body of research on adequate function allocation by providing an overall picture through quantitatively combining data from a variety of studies across varying domains.

Introduction. It has been long known that automation can both hurt and benefit human performance (e.g., Bainbridge, 1983; Ephrath & Young, 1981; Kessel & Wickens, 1982; Rasmussen & Rouse, 1981; Sheridan, 2002; Wickens & Kessel, 1979, 1981; Wickens, Mavor, Parasuraman, & McGee, 1998; Wiener & Curry, 1980). This cost–benefit trade-off is particularly prominent when automation is imperfectly reliable. Automation infrequently fails, either due to hardware or software failures, or it fails to achieve desired outcomes simply because a functionality is used in circumstances for which it was not intended. For fielded automation, it is almost always the case that routine or “nonfailure” performance substantially exceeds unaided human performance and/or the automation assistance lowers workload. If it did not, the system would not be fielded or considered useful. However, on those infrequent occasions when automation does fail, the effects on joint humanmachine system performance may be catastrophic. These catastrophic effects may result from human’s reduced monitoring of highly reliable automation at the time it fails, trusting it too much (Parasuraman & Riley, 1997) and losing situation awareness (Endsley & Kiris, 1995). This is sometimes described as a form of complacency (Parasuraman, Molloy, & Singh, 1993) or an automation-induced decision bias (Mosier & Skitka, 1996). Indeed, operators occasionally over-rely on automation and exhibit complacency because the highly (but not perfectly) reliable automation functioned properly for an extended period prior to this first failure (Parasuraman et al., 1993; Parasuraman & Manzey, 2010; Yeh, Merlo, Wickens, & Brandenburg, 2003). Going beyond the issues of highly reliable automation, Endsley and Kiris (1995) and Miller and Parasuraman (2007) have pointed out that also the “competence” of the automation must be considered. The more support an automated system provides, the higher the risk of adverse effects on human performance (e.g., complacency, loss of situation awareness, skill degradation), and the greater the likelihood of catastrophic consequences when it fails. This trade-off, in which more automation yields better human-system performance when all is well but induces increased dependence, which may produce more problematic performance when things fail, will be of critical importance to this review of the performance effects of different degrees of automation. We might refer to this conventional wisdom about automation as the “lumber jack effect”; as applied to trees in the forest, “the higher they are, the farther they fall.” Of importance, the choice of whether or not, and to what degree, to automate a particular function should involve a trade-off between the benefits of reliable automation and the expected costs (true costs × probability of failure) of automation failures (Sheridan & Parasuraman, 2000). The “routine-failure” trade-off is complicated by the fact that “automation” is not an all-ornone concept, as it was often assumed to be in the classic human–machine task allocation analyses (e.g., the “Fitts List”; for a critique of those analyses, see Dekker & Woods, 2002; Parasuraman, Sheridan, & Wickens, 2008). Instead, one can think of varying levels of automation as first put forth by Sheridan and Verplank (1978; see also Endsley & Kiris, 1995). This continuum can be jointly defined by the amount of automation autonomy and responsibility (highest at the highest level) and the amount of human physical and cognitive activity (highest at the lowest level). For example, at the highest level, the automation can perform a decision task completely autonomously; at a lower level, it can choose (and possibly execute) an option unless the human vetoes; and at an even lower level, it may simply offer the human a selection of options. More recently, Kaber and Endsley (2004) and Wickens et al. (1998) put forth the idea that automation could also be categorized according to the stage of information processing that it accomplished. Elaborating on this concept, Parasuraman, Sheridan, and Wickens (2000) and

Related work. Wickens et al. (1998) proposed a concept in which automation could filter information from the environment (Stage 1: information acquisition), integrate this information, as when forming an assessment based on several sources of information (Stage 2: information analysis), choose or decide on an action based on the assessment (Stage 3: decision and action selection), and implement the action via a typically manual activity (Stage 4: action implementation). Within each stage, varying levels could be defined. For example, as described earlier, Sheridan and Verplank (1978) define multiple levels at Stage 3. In so doing, automation can be said to offload, assist, or replace human performance at corresponding stages of human information processing (e.g., automation filtering at Stage 1, can assist human selective attention). As an example, health care automation may (a) alert (call attention to) abnormal patient symptoms, (b) integrate these symptoms to form an intelligent diagnosis of the patient condition, (c) recommend a treatment or course of action based on the diagnosis, and (d) carry out the action as, for example, with an automated infusion pump. In applying this taxonomy, where any given stage can function at various levels, it is important to note the quasi-independence of levels across the various stages. Thus, for example, a totally automated diagnosis may be followed by a fully manual (physician chosen) course of action, just as a fully manual diagnosis may trigger an automated choice of treatment. Considering that “more automation” can be represented both by higher levels within a stage, and, typically, later stages (which, in the literature, are typically preceded by automation at earlier stages), we assume, in the analysis given later, that these two dimensions (higher levels and later stages) increase the degree of automation (DOA; e.g., Manzey, Reichenbach, & Onnasch, 2012). More specifically, it is assumed that differences between automated (support) systems representing automation of different stages and levels can be described on an ordinal scale reflecting the amount of automated support that is provided. The main assumption underlying this concept as we define it asserts that assessment of “more versus less automation” can be based on dominance relationships, early 1990s (for early examples, see, e.g., Crocoll & Coury, 1990; Layton, Smith, & McCoy, 1994). Yet since then at least a limited number of studies have become available that have collected empirical data on effects of two or more different DOAs on workload and/or SA (e.g., Endsley & Kiris, 1995; Kaber, Onal, & Endsley, 2000; Lorenz, Di Nocera, Röttger, & Parasuraman, 2002a; Sarter & Schroeder, 2001).

Method. That is, all other factors held equivalent, (a) a higher level of automation constitutes “more automation,” (b) a later stage of automation constitutes “more automation,” and (c) as a consequence, a combination of higher levels and a greater number of stages at which automation is implemented constitutes “more automation.” As will be shown later, applying this reasoning enables an unambiguous rank ordering of automated systems that have been analyzed and compared in different studies and domains. It further seems to reflect the implicit or explicit assumptions that researchers in the field whose data are employed in the current analysis usually apply when comparing their systems in terms of some concept of “more or less automation.” We illustrate this within Figure 1, which, for simplicity, presents examples of the four-stage model of Parasuraman et al. (2000) with only three levels per stage. Each of the four cases contains two automation systems, A and B, which are compared within one experiment. The automation characteristic of each of these systems is characterized by a profile of levels across the stages. The first three cases also represent the three postulates given earlier. Case 1 (“Pure Levels”) represents different levels within a stage. Case 2 (“Pure Stages”) represents different stages at the same level. Case 3 (“Aggregation”) represents an earlier stage and lower level versus a later stage and higher level. Case 4 (“Confound”) represents an earlier stage and higher level versus a later stage and lower level (i.e., a “trade-off” between stages and levels). We argue that, to the extent that the three postulates given earlier are agreed on, comparisons 1 to 3 clearly represent contrasts between systems with more (System B) versus less (System A) automation, as defined on an ordinal scale. These relationships characterize all of the studies we have reviewed in which authors have invoked a phrase like “more automation.” Case 4 is an important exception. Here, there is a trade-off between later stages and higher levels. It is impossible to assess a relative DOA unless both stages and levels are expressed on an interval or ratio scale, and we have no confidence that this has or even can be done. But none of the studies analyzed later in this article involved such a comparison. Thus, in our analysis, DOA is a useful ordinal metric explicitly available and used to compare two or more systems (or experimental conditions), specifically for the purpose of examining the trade-offs inherent in the lumberjack analogy. Within this DOA concept, the discrete tradeoff described earlier (i.e., automation supports better performance in routine situations but is problematic when automation breaks down) can be expressed as a more continuous trade-off, as illustrated in Figure 2. The two primary performance functions in this figure (heavy lines) indicate that, as the DOA increases, routine performance will improve but performance under failure will decline. This relationship is expressed intuitively by the lumberjack analogy. Prior research has found that the lumberjack analogy appears to apply to the continuum of automation reliability of alerting systems (Wickens & Dixon, 2007). We examine here the extent to which this may also apply to DOA. Furthermore, our interest lies in whether DOA also has a systematic impact on workload and situation awareness (Endsley & Kiris, 1995). Indeed, as discussed later, to the extent that loss of situation awareness may be due to both an increase in automation reliability and an increased DOA, it is plausible to assume that the lumberjack analogy may apply to the latter case (see also Wickens, 2008b). Thus, Figure 2 also depicts the hypothetical trade-off between the two secondary variables, workload and loss of situation awareness (the two lighter lines). With a higher DOA, the workload imposed by the automated task is progressively reduced, almost by definition, since if the automation is doing more cognitive/physical work, the human is doing less.

Discussion. Overall, the results fairly conclusively confirm the lumberjack hypothesis with regard to the DOA. “Conventional wisdom” has now been transformed into “statistical wisdom.” Thus, the pattern underlying the DOA confirms the general pattern that had previously been observed regarding the presence or absence of automation. Automation helps when all goes well, but leaving the user out of the loop can be problematic because it leads to considerable performance impairment if the automation suddenly fails. And this risk appears to increase with increasing DOA. The data presented in Table 1 further suggest that this effect is linked to raised issues of LSA with increasing DOA. However, due to a lack of statistical power, this latter conclusion needs to be treated with caution. The most promising account is suggested by the final post hoc analysis reported earlier. When DOA moves across the critical boundary from information acquisition and information analysis to action selection, the latter alleviating the human from some or all aspects of choosing an action, then the human is much more vulnerable to automation “failures.” Actively choosing actions manually (the generation effect; Slamecka & Graf, 1978) supports SA in a way that supports the manual performance in case of automation breakdown. When that choice is removed, the automation failure response suffers. Thus, the distinction between situation assessment and action support is critically important in automation, just as the simple dichotomy is in other aspects of human factors and cognitive engineering, such as cognitive task analysis (Hoffman, Crandall, & Shadbolt, 1998), predicting multitask performance (Wickens, 2008b), and predicting transfer of training (Osgood, 1949). This finding also qualifies and specifies earlier claims that medium levels of automation would represent an optimum choice with respect to primary performance improvements and workload reductions by, at the same time, reducing unwanted performance consequences in terms of LSA and difficulties of return-to-manual performance (Endsley & Kiris, 1995). The direct trade-off between DOA-related consequences on primary task performance and return-to-manual performance, respectively, suggests that there is no clear optimum of automation support. Each step of increase of DOA seems to be associated with an increase of the risk of return-to-manual performance decrements, meaning that there is no specific DOA below which automationinduced performance benefits can be increased without any performance costs. This renders doubts in any simple design recipes like “medium DOAs are best.” However, the strength of the trade-off is important particularly if the border between information and action support is crossed. That is, the general recommendation of preferring “medium levels of automation” where the human is kept somehow “in the loop” can now be turned into a more specific one: If returnto-manual performance issues are of serious concern, human operators should be kept involved at least to some extent in decision and action selection as well as action implementation. However, even if in this case risks of return to manual might not be fully excluded, they can probably be kept to a comparably low level. One limitation of the present research is the comparably small number of studies available for this analysis and the need to just consider rank data with respect to scaling DOA and performance effects. Our approach of using rank orders based on dominance orderings of three features, that is, stages, levels, and number of stages, allowed neither for quantifying the DOA on a ratio or interval scale nor for resolving trade-offs between stages and levels.

Conclusion. •• Increasing DOA supports routine system performance and workload. •• Increasing DOA negatively affects failure system performance and SA. •• Negative consequences of automation are most likely when DOA moves from Stage 2 to Stage 3 automation.

Limitations. Taking these limitations in mind, the overall pattern of raw effects and statistical results provides a first quantitative summary of the state of knowledge about performance consequences of stages and levels of automation which, together with the remaining questions concerning possible moderating factors, certainly offers an invitation for future research.