Does a weak AI productivity number mean AI isn't working, or just that the gains haven't shown up in the data yet?

This explores whether we can separate real AI productivity gains from illusions that come from how we measure them. The corpus's answer is uncomfortable: the measurement problems run in both directions, so the same economy can look like it's undercounting AI and overcounting it at different points in time. The clearest framework is the productivity J-curve Why does AI productivity appear weak when investment is high?. When companies adopt a general-purpose technology, they also spend heavily on things that don't show up as assets: retraining staff, redesigning processes, building internal software. Early on, that spending looks like cost with no output, so measured productivity dips. Later, those hidden assets start producing, and measured productivity jumps by more than the true gain. A weak productivity number today doesn't mean AI isn't working, and a strong one later won't mean everything changed overnight.

The gaps also depend on who you ask. Executives report bigger AI gains than the data shows, partly because better operations take time to show up as revenue Do AI productivity gains feel larger than they actually measure?. Worker surveys of time saved put the aggregate gain around 1.1%, while expert task-based estimates come in near 0.7%. The authors argue the difference comes from method, not from disagreement about what AI does What is generative AI's actual aggregate productivity gain today?. Then there's a twist that pushes the other way: most workers say AI saves them time, yet many hide that they use it because of workplace stigma Why do workers hide productivity gains from AI use?. If people hide the tool, employers can't connect the gains to AI, and some of the saved time may never become extra output at all.

A deeper problem is that "time saved" may be the wrong thing to count. One line of research finds AI doesn't shrink total task time so much as move it from doing the work to writing prompts and checking outputs Does AI really save time, or just change how we spend it?. Narayanan and Kapoor make a similar point at the level of whole jobs: AI compresses the execution step of knowledge work, but the deciding and delivering steps hold steady or grow Does AI really compress all layers of knowledge work equally?. That helps explain why translation and legal employment haven't collapsed despite large task-level speedups. A measurement that zooms in on one step will see a big gain. One that looks at the whole job sees much less.

The same problem shows up in how AI is evaluated, not just in economic statistics. Agents can win benchmark contests and still fail at long, real occupational workflows, because the field has measured contests rather than work Why do agent benchmarks not predict real economic value?. This connects to reward hacking: systems, and the organizations building them, tend to optimize what's literally measured rather than what was meant Why do AIs keep gaming rewards instead of serving intent?. A productivity metric that becomes a target turns into one more score to game.

So how do you tell real gains from artifacts? The most useful lesson comes from an unexpected place, the effort to estimate how fast AI inference is getting more efficient. Researchers there removed hardware price declines and competitive pricing effects, then checked whether an independent method, energy efficiency, gave the same answer How fast is AI inference actually becoming more efficient?. That's the template: strip out known distortions, then see whether separate measurement approaches agree. For productivity, the corpus suggests the gains are probably real but modest so far, unevenly spread, and showing up in places that standard statistics weren't built to see.