Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality

Source: Organization Science (Dell'Acqua, McFowland, Mollick, Lifshitz, Kellogg, Rajendran, Krayer, Candelon, Lakhani; HBS / Wharton / MIT Sloan / BCG); earlier HBS Working Paper 24-013, SSRN 4573321 · 2026-03-11

Abstract. We introduce and study the concept of a “jagged technology frontier” to describe the uneven impact of artificial intelligence (AI) capabilities, where AI assistance improves performance for some tasks but worsens it for others, even within the same knowledge workflow and with a seemingly similar level of difficulty. In collaboration with the global management consulting firm Boston Consulting Group, we have developed realistic management consulting tasks and examined the human performance implications of using AI to perform complex and knowledge-intensive work. The preregistered experiment involved 758 knowledge workers. After establishing a performance baseline on similar tasks, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview. For each one of a set of 18 realistic knowledge tasks within the frontier of AI capabilities ranging from creative to analytical tasks, subjects using AI outperformed those not using AI, completing 12.2% more tasks and completing them 25.1% more quickly on average while also delivering solutions of significantly improved quality. However, for a complex managerial task selected to be outside the frontier, subjects using AI were 19% less likely to produce correct solutions compared with those without AI, pointing to potential limitations of AI supporting knowledge workers. We discuss the positive and negative implications of AI-aided human performance in knowledge-intensive tasks.

Introduction. The capabilities of artificial intelligence (AI) systems to aid humans in a variety of tasks have improved rapidly, especially since the release of OpenAI’s ChatGPT, which made large language models (LLMs) widely available for public use. Recent research has shown that these systems are unexpectedly capable of augmenting human abilities in the completion of creative, analytical, and writing tasks as well as achieving top scores in academic aptitude tests at graduate and professional levels (Geerling et al. 2023, Noy and Zhang 2023, Boussioux et al. 2024, Brynjolfsson et al. 2025). This represents a significant shift in human-machine augmentation, with hybrid aptitudes increasingly relevant across a broad range of knowledge work—including some highly creative and highly educated professions (Eloundou et al. 2024, Zhou and Lee 2024).

As generative AI’s capabilities increasingly overlap with human skill sets, the integration of AI into human work presents new fundamental challenges, opportunities, and questions, especially in knowledge-intensive scientific and professional fields. In this paper, we investigate the impact of generative AI on management consulting—a cross-disciplinary field where top graduates from leading universities tackle diverse projects across industries. We examine this issue using a randomized, controlled laboratory-in-the-field experiment, in which highly skilled management consultants performed tasks resembling their standard work assignments.

Our results demonstrate that AI capabilities cover an uneven set of knowledge work, and we introduce the concept of a “jagged technological frontier” to characterize the performance of AI when used by knowledge workers. Tasks that appear to human knowledge workers to be of similar difficulty may be performed either better or worse by humans using AI. Within this jagged frontier, AI can complement human work. However, outside the frontier, AI output is inaccurate, is less useful, and can degrade human performance. AI assistance improves human performance only for tasks within current AI capabilities— within the jagged technological frontier—and worsens human performance outside of it. We find that workers who skillfully navigate this frontier in their use of AI systems gain substantial quality and productivity benefits.

Related work. This paper contributes to the emerging literature on the impact of AI on work by showing its nuanced impact on knowledge work. Studies such as Noy and Zhang (2023) have documented that providing AI assistance boosts productivity and efficiency among college-educated professionals. Our paper adds substantially to this finding in several key respects. First, our laboratory-in-the-field experiment conducted with management consultants offers a real-world setting that mirrors the complexities of actual professional workflows, lending high ecological validity to our results. Second, whereas Noy and Zhang (2023) focused primarily on writing tasks, we include a broader range of activities. Third, we designed a dual-task framework with tasks outside current AI capabilities. This “jagged frontier” approach reveals that although AI significantly improves performance on tasks within its capabilities, it can hinder performance on those requiring complex analysis or reasoning.

Other papers have illustrated AI’s heterogeneous effects. For example, Brynjolfsson et al. (2025) have highlighted AI’s varying impact based on skill levels, showing enhancements primarily among lower-skilled workers, whereas Otis et al. (2024b) have found no overall effect from AI usage but noted benefits disproportionately affecting higher-skilled individuals. Our study, the first to measure the impact of AI on knowledge workers in a setting resembling their standard work activities, addresses the tension of the contrasting results in the literature by demonstrating AI’s both positive and negative effects on knowledge work. These enhancements show that our work not only confirms the productivity benefits of introducing AI but also, expands our understanding of AI’s limitations and the best ways to integrate it in professional environments.

Method. For this study, we collaborated with a global management consulting firm (Boston Consulting Group (BCG)), advising them on the design, development, and execution of a preregistered randomized experiment to assess the impact of AI on knowledge workers.2 We specifically worked with them to develop tasks that closely resemble the workflow of their management consultants. Subsequently, the author team received the data that the company collected for this experiment and conducted the analysis presented in this paper. Online Appendix F shows a detailed analysis using Bureau of Labor Statistics data that contextualizes our experimental occupation within the broader landscape of knowledge work, demonstrating how management consulting responsibilities align with the multifaceted demands of modern professional environments.

We tested two distinct sets of tasks: one situated beyond the frontier of AI capabilities and one situated within the bounds of that frontier. The experiment aimed to understand how AI integration with human-performed tasks might reshape the traditional workflows of highly skilled professionals in a knowledge domain. Both sets of tasks were developed to be realistic and were designed by senior professionals at BCG who had experience in the same roles as the participants during their careers and were currently supervising workers in those roles. A senior managing director and partner at the company commented that these tasks were “very much in line with part of the daily activities of the consultants” involved. Notably, some forms of these tasks are also used by the company to screen job applicants, typically from elite academic backgrounds (including PhDs), for their highly coveted positions. Online Appendix G uses a survey of 11 managing directors and partners at BCG, among the most senior executives at the firm, to confirm that the underlying competencies assessed by our experimental tasks are critical for recruiting, job performance, and career progression at BCG.

We collected data from a randomized experiment to assess the causal impact of AI, specifically GPT-4 (then the most capable of the AI models available (June 2023)) on highly qualified professionals otherwise working traditionally without AI.4 We preregistered our study, detailing the design structure, the experimental conditions, the dependent variables, and our main analytical approaches.5 We aimed to determine how introducing this AI into the tasks of highly skilled knowledge workers might augment, disrupt, or influence their performance and workflow. Details about the experimental structure are reported in Online Appendix A.

Results. Our results reveal significant effects, underscoring the prowess of AI even on tasks traditionally executed by highly skilled knowledge workers. Not only did using AI increase the number of subtasks completed by more than 12%, it also enhanced the quality of the responses by an average of 32%. These effects support the view that for tasks that are clearly within its frontier of capabilities, even those that historically demanded intensive human interaction, AI support provides major performance benefits.

Figure 4 presents an important trend; the most significant beneficiaries of using AI are the bottom-half-skill subjects, consistent with findings from Noy and Zhang (2023) and Choi and Schwarcz (2024). By segmenting subjects exposed to one of the two AI conditions into two distinct categories—top-half-skill performers (those ranking in the top 50% on the assessment task) and bottom-half-skill performers (those in the bottom 50%)—we observed performance enhancements in the experimental task for both groups when leveraging GPT-4. Note that the top-half-skill performers also received a significant boost, although not as much as the bottom-half-skill performers.

Our findings gain additional perspective through comparison with recent research in this area. Brynjolfsson et al. (2025) explore customer support roles, identifying productivity gains primarily among lower-skilled workers, with negligible effects observed among their higher-skilled counterparts. Choi and Schwarcz (2024) use law school examinations and find that students at the bottom of the performance distribution benefited enormously from AI assistance, whereas higher-skilled students saw performance declines. Conversely, Otis et al. (2024b) report mixed outcomes from the deployment of generative AI among small-scale entrepreneurs in Kenya. Although high-performing individuals reap the benefits of AI-generated advice, low-performing individuals tend to fare worse when following AI suggestions. In our study, centered on high-end knowledge work, we observe a universal benefit for workers involved in complex, high-level tasks, particularly for tasks within the AI’s capability frontier. Within this group of elite professionals, relatively lower-skilled individuals gained the most, suggesting that generative AI may serve as a significant equalizer in the realm of high-end knowledge work, narrowing the performance gap between different skill levels; however, those who were above average also benefited meaningfully through the use of the AI system. Together, these insights highlight the significance of task-specific and contextual considerations for effectively leveraging generative AI.

Almost every participant (97.6%) assigned to treatment used AI at least once during the experimental task. In separate qualitative work, we examine heterogeneity in how knowledge workers interact with generative AI systems and develop a typology of human-generative AI collaboration modes (Randazzo et al. 2025b). [Footnote 17; Randazzo et al. 2025b is “Cyborgs, centaurs and self-automators: The three modes of human–GenAI knowledge work and their implications for skilling and the future of expertise,” Harvard Business School Working Paper No. 26-036.]

In refining the task within the frontier and recognizing the substantial quality and productivity gains enabled by integrating AI, we sought to develop a task that participants could not have AI easily complete by simply copying and pasting our instructions as a prompt. We designed the beyond-the-frontier task in collaboration with BCG, using as a starting point the type of business cases that BCG uses for its highly competitive job interviews. To be able to solve the task correctly, participants would have to look at the quantitative data using subtle but clear insights from the interviews. Although the spreadsheet data alone were designed to seem to be comprehensive, a careful review of the interview notes revealed crucial details.

For this task outside the frontier, our primary evaluation metric is “correctness,” which is represented as a binary variable, where a value of one indicates that subjects provided an accurate recommendation and zero signifies otherwise. Figure 5 visualizes the correctness percentages for the different groups, highlighting a noticeable dip in performance among the AI treatment groups when juxtaposed with the control group. Subjects in the control group were correct about this exercise about 84.5% of the time, but the AI conditions scored at 60% and 70.6% (an average decrease of 19 percentage points when combining the AI treatment conditions and comparing them with the control condition).

Figure 6 illustrates subjective coherence quality conditional on whether participants ultimately solved the problem correctly. Across both the “correct” and “incorrect” subgroups, AI-supported participants produced recommendations that were more coherent and persuasive, underscoring that AI improves presentation and argumentation even when the underlying analysis is flawed.

Discussion. In this paper, we study the use of AI with a jagged capability frontier by humans performing high-end knowledge work. Using a preregistered, randomized laboratory-in-the-field experiment, we test the potential dual role of AI in augmenting the efforts of knowledge workers, recognizing that it functions as both a booster—enhancing efficiency and productivity on tasks within its capability frontier—and a disruptor, which negatively impacts performance on tasks beyond its frontier. Our findings underscore the transformative potential of AI and offer insights into harnessing its capabilities for optimal outcomes.

We found that the utility of AI can fluctuate across a professional’s workflow, with some tasks falling inside the frontier, whereas others fall outside of it. For tasks inside the frontier, our findings have substantial and positive performance implications. Across 18 realistic business tasks—ranging from creative to analytical tasks—AI significantly improved performance and quality for every model specification, increasing speed by more than 25%, performance by more than 30%, and task completion by more than 12%. However, for a task outside the frontier, subjects using AI were 19 percentage points less likely to produce correct solutions. Although this design highlights a single outside-the-frontier task—an inherent limitation—it underscores that even a single complex challenge beyond AI’s current capabilities can disproportionately reduce overall performance if users rely too heavily on AI’s output. These results suggest a need to understand the frontier’s shape and position as well as how it is perceived by knowledge workers to better determine AI’s influence on work.

A further caveat concerns who actually chooses to adopt generative AI when its use is discretionary rather than strongly encouraged, such as in our setting. Recent evidence shows sizable heterogeneity; early adopters are disproportionately male and more technologically confident, and the resulting performance gains can widen pre-existing gender- and skill-gaps rather than close them (Otis et al. 2024a, Humlum and Vestergaard 2025, Blandin et al. 2026). If such adoption gaps persist, the technology could widen existing promotion differentials. Additionally, beyond unequal gains, AI may substitute some workers entirely.

Although our analysis focuses on individuals, the scale of our experiment within a single organization surfaces implications for how firms might integrate AI into knowledge work. Our findings suggest that organizations should focus on the knowledge workflow and the tasks within it, and for each task, they should evaluate the value of using different configurations of humans and AI.

Our findings also connect to several related literatures. Prior work shows that AI systems tend to generate larger performance gains in ideation and content creation tasks than in judgment or decision-making tasks (e.g., Vaccaro et al. 2024). Although our findings are consistent with this pattern, they show that such divergent effects can arise within the same knowledge workflow and for the same professionals, complicating knowledge workers’ and managers’ ability to anticipate when AI assistance will be beneficial versus harmful. Our findings also relate to research on automation bias, which documents how human performance can deteriorate when automated systems provide imperfect guidance in decision-making contexts (e.g., Skitka et al. 1999, Buçinca et al. 2021). We highlight how such performance losses can emerge when AI systems perform exceptionally well on adjacent tasks, underscoring the challenge posed by AI’s jagged capabilities.

Our findings offer multiple avenues for interpretation in the context of future implementations of human-AI collaboration. First, our results lend support to optimism about AI capabilities for important high-end knowledge work tasks. In our study, the human-AI combination proved surprisingly capable, demonstrating the potential of AI-aided professional work. However, we also show the challenges resulting from AI’s jagged capabilities; experienced and incentivized knowledge professionals, engaged in tasks akin to some of their daily responsibilities, performed worse when given access to AI. This happened even though they did not need to rely on AI’s output to complete the task. The jagged nature of the knowledge frontier suggests that in real-world settings, knowledge-intensive workflows will most likely straddle both sides, creating situations where knowledge workers—unaware of their position relative to AI’s capabilities—perform worse as a result of becoming overly reliant on AI (“falling asleep at the wheel” (Dell’Acqua 2022)), a risk that is reinforced by the persuasive capabilities of generative AI (Randazzo et al. 2025a).

Conclusion. Since the release of our working paper in September 2023, the notion of AI as a jagged technology has been acknowledged by industry leaders and practitioners. As the boundaries of AI capabilities continue to expand, human professionals may need to recalibrate their understanding of the frontier. The frontier is jagged, but it is not fixed, and any given task may move inside with the release of a new model. Overall, AI appears poised to significantly affect human cognition and problem-solving but in uneven ways. AI may lower—and raise—the cognitive costs associated with knowledge acquisition and application across tasks.