AI Adoption and System-Wide Change

Source: Ajay Agrawal, Joshua S. Gans, Avi Goldfarb; NBER w28811; journal version J. Economics & Management Strategy 2023 · 2021

Analyses of AI adoption focus on its adoption at the individual task level. What has received significantly less attention is how AI adoption is shaped by the fact that organisations are composed of many interacting tasks. AI adoption may, therefore, require system-wide change which is both a constraint and an opportunity. We provide the first formal analysis where multiple tasks may be part of a modular or non-modular system. We find that reliance on AI, a prediction tool, increases decision variation which, in turn, raises challenges if decisions across the organisation interact. Modularity, which leads to task independence rather than system-level inter-dependencies, softens that impact. Thus, modularity can facilitate AI adoption. However, it does this at the expense of synergies. By contrast, when there are mechanisms for inter-decision coordination, AI adoption is enhanced when there is a non-modular environment. Consequently, we show that there are important cases where AI adoption will be enhanced when it can be adopted beyond tasks but as part of a designed organisational system.

Introduction. Over the past decade, artificial intelligence (AI) has emerged as a potential general purpose technology (Cockburn et al. (2019)). Spurred on by advances in machine learning, the cost of prediction across various domains has started to fall at an accelerating pace (Agrawal et al. (2018a)). This raises interesting questions of where AI will be adopted and also its potential disruptive impact on employment and businesses (Gans and Leigh (2019), Frey and Osborne (2017), Brynjolfsson and McAfee (2017)). To date, our conception of AI adoption has mainly operated at the level of a task or decision (e.g., Frank et al. (2019); Acemoglu and Restrepo (2018)). To forecast the potential impact of AI on employment, for example, there have been numerous exercises designed to identify jobs at risk from AI, the tasks that comprise jobs that are at risk, and the more general impact of automation on the workplace (Webb (2020); Brynjolfsson and Mitchell (2017); Brynjolfsson et al. (2018); Felten et al. (2018)). That said, some have questioned whether this task-level focus is appropriate. Bresnahan (2020) argues that AI is an information technology and traditionally such technologies have required organisational redesign to be fully adopted. This is readily apparent in patterns of adoption of earlier generations of IT (Bresnahan and Greenstein (1996); Bresnahan et al. (2002); Aral et al. (2012); Dranove et al. (2014)). Bresnahan (2020) challenges the idea that AI adoption can be analysed at the task level independent of the organisational context the task lies in. Bresnahan identifies the degree of modularity in the organisation as a predictor of AI adoption. When an organisation is non-modular, changing the nature of decision-making in one part – as would arise with AI adoption – can require changes in decision-making and practices elsewhere. The need for adjustment throughout a non-modular organisation can, he argues, cause a barrier to the adoption of AI. Instead, Bresnahan forecasts that AI will be primarily adopted in existing modular organisations and might only be adopted in other organisations should those organisations be redesigned to be modular. We build a model of the role of modularity in AI adoption. We consider a firm where value arises from the outcomes of two decisions (akin to tasks). Modularity is the degree to which the firm receives a payofffrom one decision even if the two decisions are not properly aligned. In the baseline model, the decision-makers do not know the external state and hence do not know the correct action. Therefore, they select the action with the highest likelihood of being correct. There is no reason for the decision-makers to communicate, irrespective of the degree of modularity. They both do the same action at all times and are often, but not always, correct.

AI helps decision-making by providing information on the correct choice for one of the decisions. In this respect, AI adoption itself occurs with respect to a single task. With the AI, there can be a benefit to deviating from the focal action. However, if the organisational structure is unchanged with no improvements to intra-organisational communication, we find that AI adoption is more valuable in modular organisations. This result is consistent with Bresnahan’s intuition: It is easier to adopt AI in modular organisations because modularity means less need to coordinate. One decision-maker can respond to incoming information without affecting the outcomes for the other decision-maker. We next consider communication as a type of organisational change.1 With communication, AI is especially valuable to non-modular organisations that can take advantage of the ability to get the first decision right (because of the AI) and then coordinate the second decision. With communication, AI will benefit non-modular organizations and these organisations will benefit even more than modular organisations, with or without communication. We see this framework as informing when to focus on AI adoption with a task versus a systems lens. It is appropriate to focus on AI adoption at the task level when examining modular organisations that do not change. In contrast, a systems focus is appropriate when the organisation can change how it operates, for example by facilitating communication and hence coordination. In this case, AI adoption cannot be analysed without considering the potential for systems-level change to enable non-modular organisations to succeed. Throughout, we provide examples from the retail industry, emphasizing how AI is already enabling modular organisations and how, with better communication, it could also enable non-modular organisations.

Method. The model here is inspired by Van den Steen (2017), although it addresses a distinct research question.2 Suppose that an organisation’s return, R, depends on the outcomes of two decisions, {D1, D2}, indexed by k.3 Each decision results in the choice of an action, ak ∈Ak, a set with Mk elements. D1 is a focal decision whose “correctness” depends upon matching an action with an external state. A stand-alone correct D1 results in an increment to return of α compared with an incorrect decision. D2 is a ”supporting” decision that creates value when aligned with the focal decision. If two decisions, {D1, D2}, are correctly aligned, then this results in an incremental return of γ relative to the case where those decisions are incorrectly aligned (see Figure 1).4 Throughout, we consider the example of a retailer, in which the decisions are what to stock in inventory and what to recommend to the customer. The “correctness” of a decision is modelled here in a reduced form way. We suppose that the correct stand-alone decision is associated with a state, T1 ∈A1, knowledge of which reveals the correct decision. This state could be driven by an assessment of the external environment for a decision and/or an agent’s judgment regarding the trade-offs and risks associated with particular actions. For instance, a retail manager may be considering a decision of how much to stock based on a prediction of future demand as well as the relative costs associated with errors in that forecast (inventory holding costs versus lost sales due to stock outs). Based on that prediction, at a given time, there is an assessment of the state and associated optimal action. If that state is correctly identified and the associated action taken, there is a boost to the organisation’s return of α. If not, there is no such boost.5 We treat the correctness of the supporting decision similarly. We suppose that whether {D1, D2} are aligned is associated with a state T12 ∈{A1, A2}. If {a1, a2} = T12, then the decision is “correct” and it contributes γ to project return. Otherwise, there is no contribution. We assume that T12 is a bijection (i.e., a one-to-one correspondence) where for every a1 there exists an action a2 that creates alignment. Thus, so long as this relationship is known, there is an alignment incentive to choose the actions that selected that state for {D1, D2}. It is assumed that α, γ > 0. The stand-alone and alignment contributions, α and γ, can be realised if D1 and D2 are chosen correctly. We assume that if both D1 and D2 are correct – that is, a1 ∈T1 and {a1, a2} ∈T12 – a contribution of β > α + γ can be generated. We will call β a synergistic contribution. As we will note shortly, this particular opportunity defines both the benefit of non-modular systems as well as driving the difficulty in managing non-modular systems.

What are the characteristics of an organisation that may impact on AI adoption? Bresnahan (2020) argues that a modular organisation will face fewer barriers to adopting AI than one that is not modular. Modularity does not have a commonly agreed upon definition (Campagnolo and Camuffo (2010)). In the study of organisations, its origins are most associated with Simon (1962) and then developed by Baldwin and Clark (2000). In a modular system, actions taken in one component of the system do not impact on those taken in other components and vice versa. At an organisational level, modularity relates to how loosely different parts of the organisation are coupled together (Schilling (2000)). As a general matter, the main trade-offbetween a modular and a non-modular organisation is that the former is easier to manage than the latter; it does this by sacrificing certain opportunities for creating synergistic value. However, it is also the case that modularity makes it easier for decisions to create value by being aligned. In our case, an alignment contribution of γ captures this possibility, while the synergistic contribution of β captures the additional challenge of coordination when the focal decision may itself be varying.

Discussion. If ρ is low, this means that the loss from there being no ability of agent 1 to forecast the correct state of the world is high. In this situation, an organisation that is more modular generates a higher payoffas the challenges to realise synergies are correspondingly higher. This captures the trade-offnoted earlier that modularity makes it easier to realise the value of decisions when coordinating their performance is challenging due to uncertainty. The equilibrium in Proposition 1 is not, however, the only equilibrium outcome. If agent 1 chooses, with certainty, any action, then agent 2 will optimally align to that action. Agent 1’s expected stand-alone contribution will be lower. Thus, all other equilibrium outcomes result in a lower R. For that reason, we will focus on the ”focal point” equilibrium outcome where agent 1 chooses the action most likely to be the correct action.

If agent 1 follows the prediction, this means that their action will vary with the external state. Proposition 2 shows that this makes agent 2’s ability to align with that decision more difficult. This happens both for achieving the alignment contribution in a modular organisation but also for achieving the synergistic contribution, each of which are only realised with probability ρ. To complete our analysis of the equilibrium with prediction, we need to examine whether agent 1 will choose to follow the prediction or not. If agent 1 does follow the prediction, then the organisation will adopt AI. Otherwise it will not.

The proof follows from comparing the various expected returns in Propositions 1 and 2. The benefit of AI adoption is that the stand-alone performance of agent 1 is increased by m(1 −ρ)α. However, the variability this implies for agent 1’s action makes it more difficult for agent 2 to align correctly. Thus, their alignment contribution falls by m(1 −ρ)γ. This implies that, even in a modular organisation as defined here (with m = 1), AI adoption creates alignment challenges. This result arises because the R depends on mI{a1,a2}=T12γ left, where alignment payoffis high and the likelihood that the focal state is correct is low, is similar to traditional retail, where recommendations and inventory choices are largely separate, AI doesn’t provide decision support, and modularity is high. On the right, the focal state is likely to be correct. Both agent 1 and agent 2 benefit from choosing the focal state, getting both the alignment payoffand the stand-alone payoff, regardless of whether AI is adopted. Given that they both choose the focal state, they will choose a non-modular organization so that they capture β > α + γ. AI provides no additional value because agent 1 will always choose the focal state. This business model is similar to the Book of the Month Club, which launched in 1926 and sent the same book to every subscriber. Since the most desired item was consistent across customers, it was possible to integrate the recommendation and shipping decisions with little risk of a mistake. Generally, AI is only useful for improving outcomes in the bottom left quadrant, where the organisation will also choose to be modular. Modularity is best when the likelihood of the focal state is relatively low. If the focal action is relatively unlikely to be the correct action, then the AI will have an impact on the ex post decision more often. This is consistent with the intuition in Bresnahan (2020), AI is useful in modular organisations. In non-modular organizations, it offers no benefit and will not be adopted if it has a cost.

Conclusion. This paper provides a model of AI adoption with potential system level effects. Starting with the understanding that the current excitement around AI is driven by prediction technology, we examine the decision to adopt AI in an organisation with two decisions. When the decisions are coordinated, there is an extra benefit. AI allows more precise decisions, but it makes it more difficult for the two decisions to coordinate. In this case, AI will only be adopted when the benefit of getting the first decision correct outweighs the benefit of coordination. When organisations are modular, the benefit of coordination is relatively small. When better prediction can override the default decision more often, modularity is particularly beneficial.

If, however, the organisation can change the way it operates, by enabling communication between the decision-makers, then AI adoption can have an even larger impact on outcomes in non-modular organisations. In other words, non-modular organisations benefit when the organisation changes. Put in the context of the information technology adoption literature, AI adoption has the biggest benefit in non-modular organisations, but only when there is system-wide change in the way the organisation operates.