Do AI Overviews actually reduce clicks to publisher sites?

Agarwal and Sen built a custom Chrome extension and randomly assigned over 1,000 recruited users to standard Google Search (AI Overviews appear when triggered) or a version with AIOs stripped out in real time, then watched their behavior for two weeks — a randomized field experiment, not an observed correlation. Their "baseline result shows the appearance of an AIO reduced users' organic clicks to third-party sites by 39.8 percent and increased searches where the user clicked on no links at all by 34.5 percent," with "no corresponding change in clicks on sponsored links or in search frequency." Citations embedded inside AIOs generated "about 8% of all clicks" — a small fraction of the traffic the summary itself displaces.

The authors also measured, rather than asked about, click quality: among users who did click through, bounce rates and time-on-site showed "no difference across these click quality measures between the treatment and control groups," which the authors say is "seemingly contradicting Google's public claims that AIOs produced higher quality traffic." A post-experiment survey of the same users found "no measurable improvement in users' perceived search quality or ease of finding information." The treatment effect concentrates in cases where the AIO lands at the very top of the results page, which the authors read as consistent with prior evidence that top-ranked content captures disproportionate attention.

This strengthens, with causal evidence, what the correlational and observational studies in the library already show from different angles. Do AI summaries on Google reduce clicks to actual websites? and How much are AI Overviews actually reducing organic search clicks? both track the same click-suppression pattern in the wild but cannot rule out that AIOs simply appear more often on queries that were already low-click; the randomized removal here closes that gap. It also converges with the quasi-experimental Does AI search summaries divert traffic away from Wikipedia? using a different method on a different source population. Where do searchers look when AI Overviews appear? supplies the attention mechanism this experiment's own top-of-page heterogeneity result points to.

The experiment does not establish that AIOs are substantially similar to the copyrighted content they draw on, nor does it establish an antitrust violation or a specific legal remedy — the authors are explicit that these questions "require substantially more analysis" than the paper provides. What it does establish is narrower and sturdier: the mechanism publishers point to when they object to AIOs (lost traffic, not just lost citation credit) is real and causal, and Google's quality-of-traffic defense does not hold up against click-quality and user-survey measures from the same randomized sample. A policy remedy built only around opt-outs or pay-per-citation, the authors argue, addresses consent but not the traffic loss itself.