AI bots now account for 1 in 31 web visits as human traffic declines
#AI

AI bots now account for 1 in 31 web visits as human traffic declines

Regulation Reporter
3 min read

AI bot traffic is surging while human visits decline, with RAG bots now dominating web scraping and threatening the traditional internet model.

The internet is undergoing a fundamental shift as artificial intelligence bots increasingly dominate web traffic, according to new research from AI traffic tracking firm Tollbit. The data reveals a concerning trend: AI bots now account for approximately one in every 31 web visits, up from just one in 200 visits at the start of 2025.

Featured image

This surge in AI bot activity comes as human web traffic experiences a simultaneous decline. Tollbit's latest State of the Bots report shows that from Q3 to Q4 of 2025, human visitors to websites dropped by five percent. The company warns that these numbers are likely conservative, as AI bots have become increasingly sophisticated at mimicking human behavior online.

"From the tests we ran... many of these web scrapers are indistinguishable from human visitors on sites," Tollbit noted in its report. "In light of this, the data below is conservative; it is likely worse than these numbers."

The shift from training to retrieval

While concerns about AI companies scraping content for model training remain valid, this activity has actually decreased by 15 percent between Q2 and Q4 of last year. Instead, the bot traffic surge is being driven by retrieval augmented generation (RAG) bots.

RAG bots are used by major AI platforms like OpenAI's ChatGPT and Google's Gemini to extract real-time information from the web to answer user queries. This type of bot traffic increased by 33 percent in the same period that training bot traffic declined. AI search indexers, which build indexes used by RAG bots, saw an even more dramatic increase of 59 percent.

OpenAI dominates this space, with its ChatGPT-User bot averaging five times as many scrapes per page as the second-highest scraper from Meta. This shift reflects changing user behavior, with 37 percent of active AI users now starting their searches in AI platforms rather than traditional search engines, according to marketing firm Eight Oh Two.

The publisher crisis

The rise of AI bots presents an existential threat to online publishers. While AI bots are consuming more content than ever, they're sending far less referral traffic back to source websites. Click-through rates from AI apps fell from 0.8 percent in Q2 2025 to just 0.27 percent in Q4 - a nearly threefold decline.

Websites with AI licensing deals haven't been spared either. Their click-through rates dropped to just 1.33 percent in Q4, representing a 6.5x decrease from earlier in the year. This creates a vicious cycle where publishers' content is being scraped extensively but generating minimal revenue in return.

Tollbit cofounder and chief operating officer Olivia Joslin predicts that AI will soon become the primary reader of the internet. "It could be this year that we see AI visitors being the dominant visitors to publisher sites," she estimated. "AI traffic will continue to surge and replace direct human visitors to sites."

The human cost

The shift toward AI-mediated information consumption raises serious concerns about human cognitive development. Studies suggest that AI use has a direct negative impact on critical thinking and skills. Young people, despite mental health warnings about excessive AI use, are frequent users of these tools.

Research on students found that those relying on AI for help writing essays showed poorer knowledge retention than those doing the work independently. This represents another level of curation on top of existing search engine algorithms, potentially affecting what information users receive and how they process it.

As Tollbit's data makes clear, the internet is rapidly transforming from a human-centric information space to one dominated by AI bots. The convenience of AI chatbots comes at the cost of traditional web browsing, threatening the economic model that has sustained online publishing for decades. Whether this represents progress or decline depends largely on one's perspective, but the data suggests the change is both significant and accelerating.

Comments

Loading comments...