Beyond AI Exposure: Which Tasks are Cost-Effective to Automate with Computer Vision?

Source: MIT FutureTech working paper (Maja S. Svanberg, Wensu Li, Martin Fleming, Brian C. Goehring, Neil C. Thompson) · 2024-01-18

The faster AI automation spreads through the economy, the more profound its potential impacts, both positive (improved productivity) and negative (worker displacement). The previous literature on “AI Exposure” cannot predict this pace of automation since it attempts to measure an overall potential for AI to affect an area, not the technical feasibility and economic attractiveness of building such systems. In this article, we present a new type of AI task automation model that is end-to-end, estimating: the level of technical performance needed to do a task, the characteristics of an AI system capable of that performance, and the economic choice of whether to build and deploy such a system. The result is a first estimate of which tasks are technically feasible and economically attractive to automate - and which are not. We focus on computer vision, where cost modeling is more developed. We find that at today’s costs U.S. businesses would choose not to automate most vision tasks that have “AI Exposure,” and that only 23% of worker wages being paid for vision tasks would be attractive to automate. This slower roll-out of AI can be accelerated if costs falls rapidly or if it is deployed via AI-as-a-service platforms that have greater scale than individual firms, both of which we quantify.

Overall, our findings suggest that AI job displacement will be substantial, but also gradual – and therefore there is room for policy and retraining to mitigate unemployment impacts.

Introduction. “Machines will steal our jobs” is a sentiment frequently expressed during times of rapid technological change. Such anxiety has re-emerged with the creation of large language models (e.g. ChatGPT, Bard, GPT-4) that show considerable skill in tasks where previously only human beings showed proficiency. A recent study found that about 50% of tasks could be at least partially automated with large language models (Eloundou et al. 2023). If task automation of that extent were to happen rapidly, it would represent an enormous disruption to the labor force. Conversely, if that amount of automation were to happen slowly then labor might be able to adapt as it did during other economic transformations (e.g. moving from agriculture to manufacturing). So, making good policy and business decisions depends on understanding how rapidly AI task automation will happen.

In this paper, we address three important shortcomings of AI exposure models to construct a more economically-grounded estimate of task automation. First, we survey workers familiar with end-use tasks to understand what performance would be required of an automated system. Second, we model the cost of building AI systems capable of reaching that level of performance. This cost estimate is essential to understanding the deployment of AI, since technically-exacting systems can be enormously expensive. And third, we model the decision about whether AI adoption is economically-attractive. The result is the first end-to-end AI automation model.

The rest of the paper is structured as follows: Section 1 introduces our framework to estimate which tasks are economically attractive to automate, section 2 presents the results, section 3 discusses how laborreplacing AI could proliferate, section 4 discusses the relevance of computer vision automation for other parts of AI, and section 5 concludes.

Related work. While there is already evidence that AI is changing labor demand (Fleming et al. 2019, Acemoglu et al.

2022), most anxieties about AI flow from predictions about “AI Exposure” that classify tasks or abilities by their potential for automation, as measured by various proxies (Arntz et al. 2017, Brynjolfsson et al.

2018, Felten et al. 2018, Webb 2019, Felten et al. 2021, Tolan et al. 2021, Meindl et al. 2021, Zarifhonarvar 2023, Felten et al. 2023). Importantly, nearly all these predictions are vague about the timeline and extent of automation because they do not directly consider the technical feasibility or economic viability of AI systems, but instead use measures of similarity between tasks and AI capabilities to indicate exposure. The only exception in the literature known to us is a McKinsey report (Ellingrud et al. 2023) that estimates AI adoption of between 4% and 55%. With such imprecise predictions, it is unclear what conclusions should follow. AI exposure models also conflate predictions about full task automation, which is more likely to displace workers, with partial automation, which could augment their productivity. Separating these effects is enormously important for understanding the economic and policy implications of automation (Acemoglu and Restrepo 2018).

Method. We develop a task-based approach to our analysis (Autor et al. 2003) that focuses on two key questions: (i) Exposure: might it possible to build an AI model to automate this task, and (ii) Economically-attractive: would it be more attractive to use an AI system for this task than to have human workers continue to do it. To assess exposure, we follow the literature (e.g., Brynjolfsson et al. (2018)) in evaluating task descriptions for whether it might be feasible for an AI systems to perform them. Our main contribution is the second part of the analysis: assessing the economic attractiveness of automation. For reasons discussed later, the economic attractiveness of human labor and AI systems is largely driven by the relative costs of each. Modeling the human cost is straightforward labor accounting. Modeling the cost of AI systems is more complicated, so we draw on the computer science literature on training and doing inference with deep learning,1 as well as 35 case studies that we performed to gather additional data. Central to our comparison of human and AI costs is the concept ofminimum viable scale, which occurs when the AI deployment’s fixed costs are sufficiently amortized that the average cost of using the computer vision system is the same as the cost of human labor of equivalent capability (Borge 2022), as shown in figure 1. AI automation is cost-effective only when the deployment scale is larger than the minimum viable scale.

1 The technique that has been the dominant source of AI progress since 2012.

Concretely, we say that computer vision has an economic advantage EM i,j, where M is for machine, for deployment unit i, over human labor for a given task, j, when computer vision can be used for the task (i.e. it has technology exposure, Tj) and when the cost of the computer vision system, CM i,j, is less than the cost of human labor, CH i,j:

Initially we consider the scope of deployment at the firm level, in other words, i denotes different firms.

Later, we expand our analysis to broader scopes of deployment, namely, changing i to representing industry groups (4-digit NAICS), subsectors (3-digit NAICS), sectors (2-digit NAICS), or the entire U.S. Economy.

In all cases, we focus on U.S. non-farm businesses to align our analysis with the data on firm sizes from the Statistics of U.S. Businesses (U.S. Census Bureau (2023)).

Many tasks currently performed by workers could be carried out by a sufficiently sophisticated computer vision system, e.g., checking products for quality at the end of a factory assembly line or scanning medical imagery for anomalies. Other tasks have little use for vision technologies, e.g., negotiating the salary of subordinates.

3 Examine trays to ensure that they contain required items. – Yes Table 1 Computer Vision Exposure: Examples of O*NET Task - Direct Work Activity pairs, and their classification as vision tasks.

To assess whether a task j has technology exposure, Tj, we need to identify which tasks in the economy are vision tasks and which are not. While prior AI exposure analyses inspired our approach to this paper, we cannot use their data directly. Felten et al. (2018, 2023) base their method on linking AI progress to abilities, which does not translate into our task-based model of replacement.

Discussion. 3. Paths to AI Proliferation Section 2 reveals an important limitation for the proliferation of labor-replacing AI: with today’s technology, many of these systems are unattractive for firms to adopt.

The economic attractiveness of AI could be substantially increased in two important ways. The first is deployment scale, finding ways for AI systems to automate more labor per system. The second is development costs, inventing less expensive ways to build AI systems. Here, we explore how these changes would affect the pace of AI deployment. In particular, to test the impact of scale, we estimate the impact of firms getting larger and of AI-as-a-service being used for more tasks. We also explore the hypothetical pace of computer vision proliferation based on different rates of cost decreases.

3.1. Human Labor is Replaced at Larger Scales In addition to firms custom-building their own systems, there is a possibility of achieving the minimum viable scale by aggregating human labor across firms. While this could hypothetically be done by one firm winning market share from its competitors because of increased efficiency, vision tasks are a small part of firm costs, so an advantage in vision is unlikely to generate large differences in competitive advantage at the firm level in many areas.

We believe that aggregating demand for AI solutions is more likely to happen through AI-as-a-service business models – where, for example, one firm develops the AI system for a task and others that also need the system outsource. Real-world examples of this include a diamond classification tool built by NavTech (Thompson 2021a) and a self-driving platform collaboration by NVIDIA (Thompson 2021b). We define economic advantage for AI-as-a-service as when the aggregate compensation paid to workers performing tasks in a given NAICS code is strictly larger than the cost of developing and running a computer vision system.

We use the same definition of CH i,j as above, with the only difference being that i is a either the overall U.S. private non-farm economy, a sector, subsector, or industry group. We obtain ni,j directly using the imputed OEWS data described earlier.

We find that the median employee works in a firm where close to none of the vision tasks are costeffective to automate.10 Even a firm with 5,000 employees, i.e., larger than 99.9% of firms in the United States, could only cost-effectively automate less than one tenth of their existing vision labor at the current cost structure. This finding helps explain results from McElheran et al. (2023), fewer than 6% of firms use AI-related technologies but that these are disproportionately large firms, representing 18% of employment.

At the extreme end of the firm size spectrum, even a hypothetical firm as big as Walmart lacks the scale to make automating 15% of their vision tasks attractive.11 As shown in our sensitivity analysis, large 10 The median employee works in a firm with between 500-749 employees. We gathered this statistic and the other ones mentioned in the paragraph from the SUSB Data Tables released by U.S. Census Bureau (2023) 11 1,600,000 U.S. employees according to https://corporate.walmart.com/about, Accessed: 2023-05-08 Svanberg et al.: Which Tasks are Cost-Effective to Automate with Computer Vision? Working Paper 19 differences in value creation would be needed to substantially change our results. As such, we expect only minor changes in computer vision attractiveness because of changes in firm size distribution or occupational concentration. Indeed, even a perfectly concentrated economy with exactly one occupation per firm would only have a tenfold multiplier on task value compared to our base assumption.