Does AI help or harm consultant performance on unfamiliar tasks?

Dell'Acqua, McFowland, Mollick and colleagues' preregistered field experiment randomly assigned 758 Boston Consulting Group consultants to no AI, GPT-4 access, or GPT-4 access with a prompt-engineering overview, then had them complete realistic consulting tasks designed by senior BCG professionals. On 18 tasks "ranging from creative to analytical" and chosen to sit within AI's capabilities, AI users "completed 12.2% more tasks and completed them 25.1% more quickly on average while also delivering solutions of significantly improved quality" — quality rose by about 32% by the paper's measure. On a separate managerial task built to sit outside the frontier, requiring consultants to catch a detail buried in interview notes that contradicted the spreadsheet data, the pattern reversed: the control group was correct "about 84.5% of the time," while the two AI conditions scored "60% and 70.6%," a 19-percentage-point average drop. Even so, on that failed task "AI-supported participants produced recommendations that were more coherent and persuasive," whether or not the analysis was right.

The paper's own term for this is the "jagged technological frontier": AI capability does not track task difficulty as humans perceive it, so tasks that look equally hard can fall on either side of a boundary invisible to the person doing the work. The authors tie the outside-the-frontier failure to automation bias (citing Skitka et al. 1999 and Buçinca et al. 2021) and to the risk of workers "falling asleep at the wheel," which they argue is sharpened by AI's own persuasive polish — the same fluency that helps on in-frontier work also masks wrong answers on out-of-frontier work. Almost every treated subject (97.6%) used the AI at least once; a companion paper, Randazzo et al. 2025b ("Cyborgs, centaurs and self-automators," HBS Working Paper 26-036), develops a typology of how workers actually interact with GenAI across this range.

This sharpens the uniformly positive finding in Can generative AI replace the benefits of having a human teammate?: that result held for a single R&D/Commercial task inside AI's reach, while this experiment shows the same tool reversing effect on a task deliberately chosen to sit outside it. It also bears on Can we measure whether AI erodes independent skill? and How does generative AI actually change worker skills?: the "falling asleep at the wheel" risk described here is exactly the overreliance channel through which skill erosion or undetected capability gaps could form, though this paper doesn't measure skill formation directly. The companion note Does AI help lower-skilled workers catch up to top performers? covers the within-frontier result on who gains most.

The design tests a single out-of-frontier task, which the authors call "an inherent limitation" — the 19-point drop is one data point, not a measured distribution of failure across many out-of-frontier tasks. The paper also doesn't explain the mechanism of AI's persuasiveness beyond citing a companion study (Randazzo et al. 2025a); it asserts the effect rather than isolating why fluency and correctness decouple. The frontier itself is explicitly not fixed — "any given task may move inside with the release of a new model" — so a task outside the frontier today, and the risks that follow from relying on AI there, cannot be assumed stable as models improve.