Augmenting the algorithm: Emerging human-in-the-loop work configurations

Source: Tor Grønsund, Margunn Aanestad; Journal of Strategic Information Systems 29(2):101614 (open access, CC BY-NC-ND) · 2020

How do configurations of humans and algorithms evolve as firms adopt artificial intelligence (AI) capabilities, and what are the implications for work and organization? We explored these questions through a two-year long case study of an organization in the international maritime trade that introduced automated algorithmic support for data analysis and prediction work. Drawing on a human–machine configuration perspective, we found that humans and the algorithm were configured and reconfigured in multiple ways over time as the organization dealt with the introduction of algorithmic analysis. In contrast to replacing human work, the emergent configurations required new roles and redistribution of extant expertise to augment and improve the accuracy of the algorithm. Our analysis suggests that the new configuration resembled a human-in-the-loop pattern, comprised of both the augmentation work of auditing (i.e. the generation of a ground truth and assessment of the algorithmic output against this) as well as the work of altering the algorithm and the data acquisition architecture. Our research points to the strategic importance of a human-in-the-loop pattern for organizational reflexivity to ensure that the performance of the algorithm meets the organization’s requirements and changes in the environment.

Introduction. Advances in ‘data-driven intelligence engines’ (Baptista et al., 2017a) based on availability of data and artificial intelligence (AI) have the potential to transform the nature of work and organizations (Newell and Marabelli, 2015). Information Systems (IS) researchers have raised fundamental questions as to the ramifications of both datafication processes (Galliers et al., 2017, Günther et al., 2017; Markus, 2017) and intelligent technologies (Bailey and Barley, 2019; Faraj et al., 2018; von Krogh, 2018; Rai et al., 2019), including the implications for organizational strategy (Constantiou and Kallinikos, 2015; Woerner and Wixom, 2015; Bhimani, 2015) and work in settings ranging from education (Marjanovic and Cecez-Kecmanovic, 2017), policing (Waardenburg et al., 2018), recruitment (Broek et al., 2019), and medical diagnosing (Lebovitz et al., 2019). Automation and augmentation figure centrally yet often at either end of the human–machine spectrum in current debates on organizational uptake of AI (Raisch and Krakowski, 2020). Data-powered algorithms allows automation of cognitive, discretionary, and decision-making tasks that humans used to perform. We currently see a significant deployment of simpler types of automation such as Robotic Process Automation tools (based on rule-based engines), and speech recognition and conversational agents used for first line customer interaction (based on Natural Language Processing techniques). More advanced algorithmic automation may include machine learning, computation, and statistical techniques that “rely on large data sets to generate responses, classifications, or dynamic predictions that resemble those of a knowledge worker” (Faraj et al., 2018, p. 62). These are still relatively rarer (Brynjolfsson and Mitchell, 2017), nevertheless, popular debates tend to profess a “replacement” trope, where human jobs are taken over by intelligent machines (McAfee and Brynjolfsson, 2017). More nuanced accounts instead emphasize interrelation and augmentation (Bailey and Barley, 2019; von Krogh, 2018; Markus, 2017; Davenport and Kirby, 2016), where humans and algorithms interact to perform a task. While automation of a human worker’s tasks may result in replacement of the human worker altogether, more often there is a partial automation of specific tasks, resulting in a division of labour between the human and the technology, where novel tasks also emerge and ensure a continued need for the human worker. In addition, one may see indirect effects where not only does automation substitute labour, it also “complements labor, raises output in ways that lead to higher demand for labor, and interacts with adjustments in labor supply” (Autor, 2015, p. 5). For instance, by automating routine manual and cognitive tasks, human capacity is freed to take on non-routine tasks and capacity and productivity can increase, which again drives an increase in demand for non-routine problem-solving (Nedelkoska and Quintini, 2018). In this paper we address these questions with a specific focus on automating/augmenting organizational data analysis and prediction work. Algorithmic decision-making is based on data that are captured through digitized devices and processed by algorithms. The aim is generally towards predicting objects’ (future) behavior based on their current or past behavior (Newell and Marabelli, 2015, p. 4). As a core task underlying such work, prediction refers to taking data or information you do have to generate information you did not previously have (Agrawal et al., 2019, p. 1), including assignment of categorical class labels (classification) and future events (prediction). The capabilities of utilizing data has in itself become a significant strategic asset and an object of strategy formulation (DalleMule and Davenport, 2017; Iansiti and Lakhani, 2020), and data-driven decision-making is increasingly implemented (Brynjolfsson and McElheran, 2016). The strategic value of data may lead organizations to automate and scale up datadriven analysis using algorithmic technologies, as the amounts and complexity of novel data surpass the organization’s existing analysis capacity. At the same time, algorithmic automation requires quality data input to be useful to organizations and thus might require human judgement and verification (von Krogh, 2018). With the possibility of a significant change of work, organizing and strategizing, it is pertinent that IS researchers examine what this entails for work. Such studies could provide research-based knowledge to organizations that consider how to respond strategically to the new influx of data-intensive and algorithmic technologies. We have conducted a qualitative case study of an organization that introduced algorithmic support for its analysis of geospatial satellite data and contextual data sources.

Related work. Automation is not an all-or-nothing phenomenon. Firstly, automation can be applied to different functions of an organization including information acquisition, analysis, decision, and action. Secondly, automation may vary along a continuum from fully manual (i.e., human) performance to fully automatic performance (Parasuraman et al., 2000). Thirdly, across this continuum and these functions, automation may augment human performance of a particular task through increasing the capability of humans to approach a problem situation, gain comprehension, and derive solutions (Engelbart, 1962, p. 1; Licklider, 1960). AI-enabled automation, then, “may range from task substitution (AI substitutes humans) to task augmentation (AI and humans complement one and another) to task assemblage (AI and humans are dynamically brought together to function as an integrated unit)” (Rai et al., 2019, p. iv). von Krogh (2018) asserts that it is in situations of problem-solving (algorithms providing alternative courses of action to resolve a problem), rather than decision-making (conclusions reached by the algorithms based on the data available), that algorithms appear to augment rather than substitute humans in their performance of tasks. Especially in complex situations where solutions may benefit from the relative strengths of humans and algorithms, an “assemblage” of team problem-solving and automated solution generation seems advantageous (von Krogh, 2018, p. 407). Indeed, a widespread assumption is that “computers and human beings have complementary strengths and problem-solving capabilities [...] and human beings generally outperform machines when dealing with ambiguity, vagueness and incomplete information” (Pavlou, 2018, p. 44-45). It has long been suggested that algorithms are more appropriate for structured problem-solving and where outcomes can be clearly described; and human processing is more apt for ill-structured problem-solving, to which solutions tend to require human interpretation and judgment (e.g., Simon, 1973; Shrestha et al., 2019). While these distinctions may be useful heuristics, we do not base the study on a given categorization of humans’ and machines’ attributes, similarities and differences. Rather, we seek to study their interplay and association, prompted by the notions of augmentation and assemblage. To do this, we build on Lucy Suchman’s seminal study at Xerox PARC (Suchman, 1987) which challenged fundamental ideas about humans’ and machines’ cognition and communication; ideas that had guided the design of the intelligent interface of the advanced copying machines used by the scientists she observed.

Method. Research approach We have conducted an in-depth, qualitative case study based on the interpretive research paradigm (Walsham, 1995). The fieldwork was conducted between September 2017 and October 2019 and covered the process of the organization’s design, implementation, and use of algorithm-supported data analysis (Bailey and Barley, 2019). The selection of the site and negotiation of access happened through an extant development project between the shipbroking firm and an enterprise software company. The company delivered cloud and business intelligence (BI) solutions to the maritime sector, and the first author was an employee of the company while doing his PhD. Access was negotiated in spring 2017 on behalf of the software company and with guarantee of confidentiality regarding business-critical information. During the fieldwork, the researcher role and intent was clearly communicated, and observations and interviews were based on informed consent from the informants. Our initial access was negotiated through meetings with the firms’ top management in early fall 2017. As we were granted access to the firm’s headquarters and the research unit, our role of involvement gradually evolved from participating in strategy planning meetings to also taking part in implementation and evaluation meetings. This allowed us to participate in separate and joint meetings with the IT development team, analysts, and brokers from various ‘desks’. In our account we have anonymized the informants.

Data collection The main method was participant observation, including informal interviews and conversation with organizational members. The first author participated in 39 meetings, which, with some exceptions, were on-site. The meetings lasted between thirty minutes and five hours, with an average duration of 104 min. In addition, a two-week-long design workshop was used for data collection in January 2018. The first author was engaged in organizing the workshop, which involved the wider organization in the design of new digital solutions for data acquisition. The Chief Digital Officer (CDO) and brokers were involved throughout the two-week workshop period and engaged particularly in five half-day sessions giving feedback on the evolving designs. The outcome of this process was presented to the executive team. During our observations, we avoided recording so as to ensure openness and trust. Notes were taken during and written out to more comprehensive research notes after the observations. During the fieldwork period, informal interviews and conversation were regularly conducted to follow up and verify our observations, including one-to-one meetings, lunch meetings, face-to-face conversations in the field as well as over Skype and phone. Documentary sources were reviewed including company PowerPoints and reports, spreadsheets, contract excerpts, whitepapers, newspaper articles, web sites, software, and internal databases. We were also given a company chat and email account on which we regularly conversed with our key informants; this contains >200 emails.

Data analysis Our data analysis strategy follows the four-stage approach of Jarzabkowski et al. (2016) to analyzing qualitative process data and everyday activities as they unfold. This approach contrasts to studying formal processes and scripted patterns in that there is no defined sequence to be enacted. Rather, we pay attention to situations where the informants themselves are trying to set forth a direction or order within their activities (p. 244). As such, we assume that it is in the change of practice that new task relations emerge and, eventually, form a pattern of human and machine interplay (Suchman, 2007). This approach aligns with other studies concerned with the relationship between technology and organizational work (e.g., Barley, 1990).

The first of the four stages involved using initial observations from meetings and conversations to confirm that our setting allowed access to phenomena of interest. This was done mainly through participation in the initial meetings with the executive team members at ShipCo and the software provider.

Discussion. Our account and articulation of augmentation work contributes to recent IS and organizational literature concerned with understanding whether and under what conditions intelligent technologies will automate or augment human work practices and tasks (Bailey and Barley, 2019; Faraj et al., 2018; von Krogh, 2018; Markus, 2017). In particular, the human-in-the-loop configuration identified and illustrated in this case is one answer to Markus’ call that “we should focus on the organizational and environmental conditions that promote or inhibit the effective performance of humans in the control loop” (Markus, 2017, p. 237). Addressing our research question and following Baptista et al.’s (2017a) call for strategic perspectives on digital work and organizational transformation, we organize our discussion by three themes: new patterns of work, new forms of organizing, and their strategic significance (see also Günther et al., 2017).

Augmenting the algorithm: Auditing and altering Overall, our study confirms earlier research on the non-trivial nature of organizational value creation from data-driven and algorithmic technologies (see e.g., Watson, 2017; Scheepers et al., 2018). Work and resources are required for developing and training the algorithm, as well as for acquiring and preparing the data sets both for the training phase and for the algorithms’ regular working. Tarafdar et al. (2019), for example, argues that an organization needs five crucial capabilities: data science competence, business domain proficiency, enterprise architecture expertise, operational IT backbone expertise, and digital inquisitiveness. Our empirical account sheds additional light on exactly how certain work tasks and roles were changed upon the introduction of an algorithm. In particular, our study contributes with an empirically based understanding of the nature of “augmentation work”, comprised of auditing and altering the algorithm. Table 1 further describes the two roles of augmentation work, along with their requirements and task association. Following the task flow, we see that the work of auditing and altering are mutually dependent on each another, eventually forming a feedback loop (see, Fig. 6). As such, both roles are crucial parts of the human-in-the-loop configuration of improving the accuracy of the algorithm and transforming data into value. In our analysis, we also asked whether the researcher ultimately would be replaced by algorithms. Traditionally, human classification supported or augmented the work of the analyst. We saw that this human work of classification was being automated through the introduction of the algorithm. However, human work was not discontinued, but repurposed to provide a ground truth and still operated in parallel to the algorithmic classification. This prompts another question: whether the researcher and role of human classification will be kept in the loop as the algorithmic accuracy is improved. It has for a long time been known that continual feedback and interaction is crucial for automation to work (Norman, 1990). Our study underscores this and highlights the roles of augmentation work in providing this feedback. In particular, we saw that human classification played a critical role to provide a ground truth – a component which was necessary not only to enable automation but also to obtain and maintain the feedback loop and overall learning of the system. This suggests that the processes of automation and augmentation influence one another in a reciprocal manner (Raisch and Krakowski, 2020), and that their organizing emerges as a core capability.

Conclusion. Research on algorithmic and intelligent technologies has generated important insights about their potential to replace human work and their relative strengths compared with human capabilities. However, it has largely not investigated the emergent configurations by which humans and algorithmic interplay emerge. Despite the recognition that automating and replacing human work may be more challenging than first expected, detailed research on its processes is still limited. Our two-year long study of an organization that introduced algorithmic support for analysis, is therefore a timely contribution to the building of improved understanding. Upon the introduction of the algorithm, we observed how humans and the algorithm came to augment one another. A set of human work tasks emerged around the automated algorithm. Some tasks, namely auditing and altering the algorithm, were new and constituted the core of augmentation work. Another extant role, the manual work of classifying data, was not replaced, but repurposed to provide the auditing process with a ground truth against which the algorithmic outcome was compared. This emergent human-in-the-loop configuration enabled a feedback mechanism which allowed reflexiveness and scale in the sense of increased accuracy and timeliness. We suggest that the processes of the design and use of the algorithm are tightly coupled as the use feeds into the design mode. Therefore, the organizational choices regarding the division of labour between the algorithm and the humans are crucial. The human-in-the-loop configuration emerges as a strategic capability. It ensures that the performance of the algorithm meets the organization’s requirements, because it provides a feedback loop that supports learning.