Opens in a new tab

Breaking news

Cloudflare Sets New Default To Separate Search Crawlers From AI Bots

Cloudflare has drawn a sharper line between traditional search and artificial intelligence.

Beginning September 15, 2026, the company will change its default settings to block so-called mixed-use crawlers from pages that run ads, unless a site owner chooses otherwise. The policy applies to new Cloudflare customers, new sites created by existing customers, and all current free customers.

A Clearer Divide In Web Access

The shift could materially reshape how AI companies collect web data for model training and agentic products. Cloudflare’s central argument is straightforward: most publishers want their content to remain visible in search and accessible through certain AI services, but they do not want that same material repurposed without compensation.

In Cloudflare’s view, the problem is not crawling itself. It is the blending of three different functions: search, agentic use, and training into a single bot that makes it difficult for website owners to set meaningful boundaries.

The Google Question

Cloudflare pointedly referenced the “world’s largest search engine,” an unmistakable nod to Google, arguing that it has access to roughly twice as much information as rival AI companies because it makes it harder for customers to stay discoverable without also being used for AI.

Google has disputed that framing. The company offers Google Extended, a crawler setting that lets publishers opt out of having content used for training and AI products such as Gemini apps and Vertex AI, without affecting visibility in Google Search. At the same time, Googlebot still crawls for Search and for AI-powered features such as AI Overviews and AI Mode.

Publishers Want Reach, Not Exploitation

Matthew Prince, Cloudflare’s co-founder and chief executive, said the company is moving quickly because the internet is now dominated by machine traffic.

“Now that the majority of traffic on the Internet is non-human, we must go further and act faster so that a sustainable ecosystem can emerge,” Prince said, referring to the recent milestone in which bots surpassed human traffic online sooner than expected.

Prince added that Cloudflare’s tools and partnerships are designed to give publishers more visibility and commercial leverage, while also rewarding AI companies that are transparent about how they use content.

From Pay Per Crawl To Pay Per Use

Cloudflare has increasingly positioned itself as a gatekeeper for publishers looking to assert control in the AI era. The company already offers tools to block AI bots, along with a marketplace called Pay Per Crawl, which lets websites charge AI systems for scraping.

That framework is now expanding into Pay Per Use, which Cloudflare says will allow publishers to charge AI companies when content creates value, not merely when it is fetched. In practical terms, that shifts the economics from extraction to monetization.

Cloudflare says the move may also reduce waste. Its data suggests more than half of crawl traffic from AI bots is spent revisiting pages that have not changed, consuming bandwidth and compute without adding fresh value for either side.

Early Partners Signal The Commercial Model

To launch the new system, Cloudflare is working with Ceramic.ai and You.com. Under the opt-in model, publishers can be paid when their content appears in Ceramic’s AI search results or when You.com accesses premium material.

Cloudflare says other AI companies can adapt the model to fit their own products. The broader message is clear: the era of unrestricted crawling is giving way to one in which access, attribution, and compensation are increasingly negotiated rather than assumed.

Europe’s Deadly Summer Heat Exposed The Cost Of A Warming Continent

Europe experienced its deadliest summer for heat-related deaths on record in 2026, with an estimated 24,400 excess deaths linked to extreme temperatures across 30 countries. Researchers estimate that human-induced climate change was responsible for around 32,000 of the deaths attributed to heat in the study’s broader analysis, accounting for 65% of the total.

The findings highlight the growing public-health consequences of global warming, largely driven by fossil fuel use, as extreme heat becomes more frequent and severe.

A Summer Of Dangerous Heat

Between June and August, heatwaves pushed temperatures in Western Europe to an average of 2.9°C above the 1991–2020 norm, with some days reaching 12°C above average. Nearly a third of estimated heat-related deaths occurred during a particularly intense heatwave between 17 June and 1 July.

“These numbers are a stark reminder that our hotter, more extreme summers are having an increasingly severe impact on public health,” said Dr Clair Barnes, a researcher at Imperial College London and co-author of the study.

The estimated toll exceeded previous peaks of around 29,000 deaths in summer 2022 and 28,000 in 2003 across the region studied.

How Researchers Estimated The Toll

Researchers modelled the relationship between temperature and mortality across 854 urban areas in the EU, UK, Norway and Switzerland, covering around 40% of the population in the 30 countries studied.

The analysis incorporated temperature records, age, socioeconomic conditions, environmental factors and access to healthcare. By comparing actual temperatures with those expected without human-induced warming, the team estimated the additional deaths attributable to climate change.

Final official mortality figures for summer 2026 are not yet available in many countries, and heat-related deaths are often underreported because their effects can be difficult to identify directly.

Older People Face The Greatest Risk

People aged 85 and over accounted for roughly half of the estimated deaths. However, the impact extended across age groups, with around 800 excess deaths among adults aged 20–44 and approximately 6,000 among people under 65.

Southern Europe experienced the highest mortality burden, reflecting hotter temperatures and other vulnerabilities, including air pollution, limited green space and unequal access to healthcare and heat-alert systems.

Italy, France And Spain Record Highest Toll

Italy recorded an estimated 13,303 heat-related excess deaths, followed by France with 7,268 and Spain with 6,824. Germany recorded around 6,340, while the UK saw an estimated 4,354 deaths.

In the UK, the heat also placed significant pressure on healthcare services. Andrew Hartle, Professor of Practice of Anaesthesia at Imperial College Healthcare, said demand during the summer resembled that of a severe winter virus season.

Cities Urged To Strengthen Heat Protection

Researchers say cities can reduce the risks through climate shelters, external shading and building retrofits, alongside measures to help residents avoid exposure during the hottest hours. Checking on vulnerable people, staying hydrated and following heat alerts are also important.

The researchers stress that adaptation must go hand in hand with reducing fossil fuel use to limit further warming. The findings underline that extreme heat is already a major public-health threat across Europe, with thousands of lives at risk each summer.

Aretilaw firm
eCredo
The Future Forbes Realty Global Properties
Uol

Become a Speaker

Become a Speaker

Become a Partner

Subscribe for our weekly newsletter