GPT-4 helped weaker consultants catch up to strong ones more than it helped the strong ones get even better.

This explores which consultants got the biggest boost from GPT-4 (weaker or stronger performers), and whether that pattern holds elsewhere. The short answer from the BCG experiment with 758 consultants is that the bottom half gained the most. Everyone used GPT-4 on the assigned tasks, and lower-skilled consultants improved more than top performers, so the gap between them got smaller Does AI help lower-skilled workers catch up to top performers?. Top performers still improved, just by less. The same pattern shows up outside consulting. In a randomized trial with law students, GPT-4 saved about the same amount of time at every skill level, but the small quality gains went mostly to the weaker students Does GPT-4 actually improve the quality of legal analysis?.

There's an important condition hidden in that result: the consultants were required to use the AI. When people choose for themselves whether and how to use it, the effect can reverse. In a trial with 640 Kenyan entrepreneurs who could ask a GPT-4 assistant for advice on WhatsApp, high performers raised their revenue by 18% and low performers lost 9% Does AI assistance help or hurt less experienced entrepreneurs?. The advice itself wasn't worse for the weaker group. The difference came from which suggestions people picked and how they applied them. So "AI helps weaker workers catch up" may depend on the AI being built into the task rather than left as an option.

The second condition is what kind of task it is. The same BCG study found that on tasks within GPT-4's abilities, consultants worked faster and produced higher-quality work. On a task just outside those abilities, consultants using AI got the right answer 19 percentage points less often than those working without it Does AI help or harm consultant performance on unfamiliar tasks?. GPT-4's wrong answers sounded convincing, and people trusted them. That matters most for the weaker performers who gained the most, because they have less expertise to tell a good answer from a bad one. Checking the output doesn't fully protect against this either. A related BCG study found that when consultants fact-checked or pushed back on GPT-4, it often argued harder instead of admitting it was wrong Does validating AI output make models more defensive?.

What you might not expect is that making weaker workers perform better is different from making them more skilled. In an 8-day experiment, people who learned with ChatGPT scored lower on knowledge tests than people who used Google, especially on critical-thinking questions, because the chatbot handed them answers instead of underlying principles Does ChatGPT harm informal learning compared to Google Search?. A study of analogies points to a similar weakness: GPT-4 came up with plenty of analogies but applied them poorly, while business students offered fewer analogies that fit better Does GPT-4 retrieve analogies differently than humans do?. Put together, the narrower gap among consultants looks real, but it depends on three things: the AI use was required, the tasks were within its abilities, and the result measures how well people performed, not how much they learned.