Why Are There Still So Many Jobs? The History and Future of Workplace Automation

Source: David H. Autor; Journal of Economic Perspectives 29(3) · 2015

In this essay, I begin by identifying the reasons that automation has not wiped out a majority of jobs over the decades and centuries. Automation does indeed substitute for labor—as it is typically intended to do. However, automation also complements labor, raises output in ways that lead to higher demand for labor, and interacts with adjustments in labor supply. Indeed, a key observation of the paper is that journalists and even expert commentators tend to overstate the extent of machine substitution for human labor and ignore the strong complementarities between automation and labor that increase productivity, raise earnings, and augment demand for labor. Changes in technology do alter the types of jobs available and what those jobs pay. In the last few decades, one noticeable change has been “polarization” of the labor market, in which wage gains went disproportionately to those at the top and at the bottom of the income and skill distribution, not to those in the middle. I will offer some evidence on this phenomenon. However, I will also argue that this polarization is unlikely to continue very far into the foreseeable future. The final section of this paper reflects on how recent and future advances in artificial intelligence and robotics should shape our thinking about the likely trajectory of occupational change and employment growth. I argue that the interplay between machine and human comparative advantage allows computers to substitute for workers in performing routine, codifiable tasks while amplifying the comparative advantage of workers in supplying problem-solving skills, adaptability, and creativity. The frontier of automation is rapidly advancing, and the challenges to substituting machines for workers in tasks requiring flexibility, judgment, and common sense remain immense. In many cases, machines both substitute for and complement human labor. Focusing only on what is lost misses a central economic mechanism by which automation affects the demand for labor: raising the value of the tasks that workers uniquely supply.

In 1900, 41 percent of the US workforce was employed in agriculture; by 2000, that share had fallen to 2 percent (Autor 2014), mostly due to a wide range of technologies including automated machinery. The mass-produced automobile drastically reduced demand for many equestrian occupations, including blacksmiths and stable hands. Successive waves of earth-moving equipment and powered tools displaced manual labor from construction. In more recent years, when a computer processes a company’s payroll, alphabetizes a list of names, or tabulates the age distribution of residents in each Census enumeration district, it is replacing a task that a human would have done in a previous era. Broadly speaking, many—perhaps most—workplace technologies are designed to save labor. Whether the technology is tractors, assembly lines, or spreadsheets, the first-order goal is to substitute mechanical power for human musculature, machine-consistency for human handiwork, and digital calculation for slow and error-prone “wetware.”

Given that these technologies demonstrably succeed in their labor saving objective and, moreover, that we invent many more labor-saving technologies all the time, should we not be somewhat surprised that technological change hasn’t already wiped out employment for the vast majority of workers? Why doesn’t automation necessarily reduce aggregate employment, even as it demonstrably reduces labor requirements per unit of output produced? These questions underline an economic reality that is as fundamental as it is overlooked: tasks that cannot be substituted by automation are generally complemented by it. Most work processes draw upon a multifaceted set of inputs: labor and capital; brains and brawn; creativity and rote repetition; technical mastery and intuitive judgment; perspiration and inspiration; adherence to rules and judicious application of discretion. Typically, these inputs each play essential roles; that is, improvements in one do not obviate the need for the other. If so, productivity improvements in one set of tasks almost necessarily increase the economic value of the remaining tasks. An iconic representation of this idea is found in the O‐ring production function studied by Kremer (1993).1 In the O-ring model, failure of any one step in the chain of production leads the entire production process to fail. Conversely, improvements in the reliability of any given link increase the value of improvements in all of the others. Intuitively, if n − 1 links in the chain are reasonably likely to fail, the fact that link n is somewhat unreliable is of little consequence. If the other n − 1 links are made reliable, then the value of making link n more reliable as well rises. Analogously, when automation or computerization makes some steps in a work process more reliable, cheaper, or faster, this increases the value of the remaining human links in the production chain. As a contemporary example, consider the surprising complementarities between information technology and employment in banking, specifically the experience with automated teller machines (ATMs) and bank tellers documented by Bessen (2015). ATMs were introduced in the 1970s, and their numbers in the US economy quadrupled from approximately 100,000 to 400,000 between 1995 and 2010. One might naturally assume that these machines had all but eliminated bank tellers in that interval. But US bank teller employment actually rose modestly from 500,000 to approximately 550,000 over the 30-year period from 1980 to 2010 (although given the growth in the labor force in this time interval, these numbers do imply that bank tellers declined as a share of overall US employment). With the growth of ATMs, what are all of these tellers doing? Bessen observes that two forces worked in opposite directions.

The principle of computer simulation of workplace tasks has not fundamentally changed since the dawn of the computer era—but its cost has. An ingenious 2007 paper by William Nordhaus estimates that the cost of performing a standardized set of computations has fallen by at least 1.7 trillion-fold since the manual computing era, with most of that decline occurring since 1980. Thus, firms have strong economic incentives to substitute ever-cheaper computing power for relatively expensive human labor. What are the effects? One first-order effect is, of course, substitution. As the price of computing power has fallen, computers and their robot cousins have increasingly displaced workers in accomplishing explicit, codifiable tasks. In Autor, Levy, and Murnane (2003), my coauthors and I label these activities as “routine tasks,” not because they are mundane, but because they can be fully codified and hence automated (see Levy and Murnane 2004 for many examples). Routine tasks are characteristic of many middle-skilled cognitive and manual activities: for example, the mathematical calculations involved in simple bookkeeping; the retrieving, sorting, and storing of structured information typical of clerical work; and the precise executing of a repetitive physical operation in an unchanging environment as in repetitive production tasks. Because core tasks of these occupations follow precise, well-understood procedures, they are increasingly codified in computer software and performed by machines. This force has led to a substantial decline in employment in clerical, administrative support, and to a lesser degree, in production and operative employment. But the scope for this kind of substitution is bounded because there are many tasks that people understand tacitly and accomplish effortlessly but for which neither computer programmers nor anyone else can enunciate the explicit “rules” or procedures. I have referred to this constraint as Polanyi’s paradox, named after the economist, philosopher, and chemist who observed in 1966, “We know more than we can tell” (Polanyi 1966; Autor 2015). When we break an egg over the edge of a mixing bowl, identify a distinct species of birds based on a fleeting glimpse, write a persuasive paragraph, or develop a hypothesis to explain a poorly understood phenomenon, we are engaging in tasks that we only tacitly understand how to perform.

Polanyi’s paradox also suggests why high-level reasoning is straightforward to computerize and certain sensorimotor skills are not.

My reading of the evidence suggests otherwise. Indeed, Polanyi’s paradox helps to explain what has not yet been accomplished, and further illuminates the paths by which more will ultimately be accomplished. Specifically, I see two distinct paths that engineering and computer science can seek to traverse to automate tasks for which we “do not know the rules”: environmental control and machine learning. The first path circumvents Polanyi’s paradox by regularizing the environment, so that comparatively inflexible machines can function semi-autonomously. The second approach inverts Polanyi’s paradox: rather than teach machines rules that we do not understand, engineers develop machines that attempt to infer tacit rules from context, abundant data, and applied statistics.

Most automated systems lack flexibility—they are brittle. Modern automobile plants, for example, employ industrial robots to install windshields on new vehicles as they move through the assembly line. But aftermarket windshield replacement companies employ technicians, not robots, to install replacement windshields. Evidently, the tasks of removing a broken windshield, preparing the windshield frame to accept a replacement, and fitting a replacement into that frame demand more real-time adaptability than any contemporary robot can cost-effectively approach. The distinction between assembly line production and the in-situ repair highlights the role of environmental control in enabling automation. Engineers can in some cases radically simplify the environment in which machines work to enable autonomous operation, as in the familiar example of a factory assembly line. Numerous examples of this approach to environmental regularization are so ingrained in daily technology that they escape notice, however.

The ongoing automation of warehouses provides another example. Large online retailers, such as Amazon.com, Zappos.com, and Staples, operate systems of warehouses that have traditionally employed legions of dexterous, athletic “pickers,” who run and climb through shelves of typically non-air-conditioned warehouses to locate, collect, box, label, and ship goods.

Machine learning applies statistics and inductive reasoning to supply best-guess answers where formal procedural rules are unknown. Where engineers are unable to program a machine to “simulate” a nonroutine task by following a scripted procedure, they may nevertheless be able to program a machine to master the task autonomously by studying successful examples of the task being carried out by others. Through a process of exposure, training, and reinforcement, machine learning algorithms may potentially infer how to accomplish tasks that have proved dauntingly challenging to codify with explicit procedures. As a concrete example, consider the task of visually identifying a chair (discussed in Autor, forthcoming). An engineer applying a conventional rules-based programming paradigm might attempt to specify what features of an object qualify an object as a chair—it possesses legs, arms, a seat, and a back, for example. But one would soon discover that many chairs do not possess all of these features (for example, some chairs have no back, or no arms). If the engineer then relaxed the required feature set accordingly (chair back optional), the included set would grow to encompass many objects that are not chairs, such as small tables. The canonical approach to recognizing objects by pre-specifying requisite features—and more sophisticated variants of this approach—would likely have very high misclassification rates. Yet, any grade-school child could perform this task with high accuracy. What does the child know that the rules-based procedure does not? Unfortunately, we cannot enunciate precisely what the child knows—and this is precisely Polanyi’s paradox. Machine learning potentially circumvents this problem. Relying on large databases of so-called “ground truth”—a vast set of curated examples of labeled objects—a machine learning algorithm attempts to infer what attributes of an object make it more or less likely to be designated a chair. This process is called “training.” When training is complete, the machine can apply this statistical model to attempt to identify chairs that are distinct from those in the original dataset. If the statistical model is sufficiently good, it may be able to recognize chairs that are somewhat distinct from those in the original training data, like chairs of different shapes, materials, or dimensions. Machine learning does not require an explicit physical model of “chairness.”

How well does machine learning work in practice? If you use a search engine or Google Translate, operate a smartphone with voice commands, or follow movie suggestions from Netflix, you can assess for yourself how successfully these technologies function. For example, if the majority of users who recently searched for the terms “degrees bacon” clicked on links for Kevin Bacon rather than links for best bacon cooking temperatures, the search engine would tend to place the Kevin Bacon links higher in the list of results.

Moreover, an irony of machine learning algorithms is that they also cannot “tell” programmers why they do what they do. IBM’s Watson computer famously triumphed in the trivia game of Jeopardy against champion human opponents. Yet Watson also produced a spectacularly incorrect answer during its winning match. Under the category of US Cities, the question was, “Its largest airport was named for a World War II hero; its second largest, for a World War II battle.” Watson’s proposed answer was Toronto, a city in Canada. Even leading-edge accomplishments in this domain can appear somewhat underwhelming. A 2012 New York Times article (Markoff 2012) described Google’s X Lab’s recent project (Le et al. 2012) to apply a neural network of 16,000 processors to identify images of cats on YouTube. The article’s headline ruefully poses the question, “How Many Computers to Identify a Cat? 16,000.” Since the underlying technologies—the software, hardware, and training data— are all improving rapidly (Andreopouos and Tsotsos 2013), one should view these examples as prototypes rather than as mature products. Some researchers expect that as computing power rises and training databases grow, the brute force machine learning approach will approach or exceed human capabilities. Others suspect that machine learning will only ever “get it right” on average, while missing many of the most important and informative exceptions. Ultimately, what makes an object a chair is that it is purpose-built for a human being to sit upon. Machine-learning algorithms may have fundamental problems with reasoning about “purposiveness” and intended uses, even given an arbitrarily large training database of images (Grabner, Gall, and Van Gool 2011). One is reminded of Carl Sagan’s (1980, p. 218) remark, “If you wish to make an apple pie from scratch, you must first invent the universe.”

Major newspaper stories offer fresh examples daily of technologies that substitute for human labor in an expanding—although still circumscribed—set of tasks. The offsetting effects of complementarities and rising demand in other areas are, however, far harder to identify as they occur. My own prediction is that employment polarization will not continue indefinitely (as argued in Autor 2013). While some of the tasks in many current middle-skill jobs are susceptible to automation, many middle-skill jobs will continue to demand a mixture of tasks from across the skill spectrum. For example, medical support occupations—radiology technicians, phlebotomists, nurse technicians, and others—are a significant and rapidly growing category of relatively well-remunerated, middle-skill employment. Most of these occupations require mastery of “middle-skill” mathematics, life sciences, and analytical reasoning. They typically require at least two years of postsecondary vocational training, and in some cases a four-year college degree or more.

It also fits a number of modern clerical occupations that provide coordination and decision-making functions, rather than simply typing and filing, like a number of jobs in marketing. There are also cases where technology is enabling workers with less esoteric technical mastery to perform additional tasks: for example, the nurse practitioner occupation that increasingly performs diagnosing and prescribing tasks in lieu of physicians. I expect that a significant stratum of middle-skill jobs combining specific vocational skills with foundational middle-skills levels of literacy, numeracy, adaptability, problem solving, and common sense will persist in coming decades. My conjecture is that many of the tasks currently bundled into these jobs cannot readily be unbundled—with machines performing the middle-skill tasks and workers performing only a low-skill residual—without a substantial drop in quality.