SYNTHESIS NOTE
Topics›Knowledge After the Web›this note

Is non-human traffic really half of all internet use?

Cloudflare's network measurements show over 50% of internet traffic is now from AI crawlers and bots rather than humans. Understanding whether this reflects actual internet-wide patterns matters for how publishers, platforms, and regulators should respond.

Synthesis note · 2026-10-09 · sourced from Knowledge After the Web

Cloudflare reports that "more than 50% of traffic on the Internet is now non-human" for the first time, with "52% of crawler requests...now for AI training as of June 2026, up from 22% in Spring 2025" and mixed-use crawlers (blending search, agent use, and training) accounting for "over 36% of activity." It frames this as the collapse of search-driven browsing: "for every hour spent online searching for information, only 15 minutes is spent on the open web." The old exchange — content for search visibility, visibility for referral traffic — is "breaking down" because "content is still being crawled, indexed, and used — but increasingly without corresponding traffic being returned to the source." Cloudflare says some heavily crawled categories "have seen human traffic decline as much as 40% in less than one year," and that many publishers are now bracing for "Google Zero," a world of little to no search-referral traffic.

Cloudflare's own account of the fix is about visibility and enforcement rather than voluntary disclosure. It credits its "attribution, business intelligence, and enforcement tools" with giving publishers network-level visibility into AI consumption, calling this "an enforcement mechanism far more effective than voluntary standards like robots.txt," and ties that visibility directly to "more than 50 publisher-AI agreements...signed since 2023." It singles out Google's "mixed-use crawler," which combines search and AI access in a single bot, as a specific obstacle: because the two uses aren't separated, publishers "cannot tell why Google is accessing their content" and lose the ability to allow or block each independently. Cloudflare's prescribed next step is standardized, machine-readable declarations of "crawl intent" and "programmable, scalable mechanisms for content discovery and monetization."

This lines up with the shift described in Will agents compete for attention just like users do? — both treat agents displacing human clicks as the organizing economic fact of the current web — but this note supplies the measured traffic-share figures (the 50%/52%/36% numbers) behind what that note frames as a forecast. It also extends What security threats emerge when machines read the web?: that note treats agent-mediated reading as a problem of integrity (what agents are made to believe), while here the same shift is read as a problem of compensation (what happens to the people who made the content once agents, not humans, are the ones reading it). Where that note's fix is epistemic, Cloudflare's is commercial — licensing and attribution infrastructure.

Cloudflare's figures describe traffic on its own network, which is large but not the whole internet, and the excerpt gives no sampling detail for "some of the most heavily crawled categories." It also reports the crawling increase and the traffic decline as co-occurring, not as cause and effect. Cloudflare is not a disinterested party here either: it set the crawler-blocking default a year earlier and sells the attribution and enforcement tools it credits with producing publisher-AI licensing deals, so the account of the problem and of the most effective remedy comes from the same commercial source. The traffic-composition numbers are a credible record of what crossed Cloudflare's network; the causal story about publisher revenue, and the claim that licensing infrastructure is the inevitable fix, should be read as Cloudflare's argument for its own product category rather than as independently verified market fact.

Inquiring lines that read this note 1

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Are AI-generated articles systematically disadvantaged in search ranking and user engagement?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 94 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

Cloudflare reports agent traffic has crossed 50 percent of all Internet traffic as publishers shift to licensing deals