Cloudflare is introducing a marketplace that enables website owners to monetise the use of their content by AI companies for training purposes. This initiative aims to address the growing issue of AI bots scraping website data without permission or compensation. The marketplace will allow content creators to set their own prices for AI firms accessing their sites.
In conjunction with the marketplace, Cloudflare is launching AI Audit, a free tool that provides website owners with analytics on AI bot activity, including which companies are scraping their content, how frequently, and which site sections are most popular. AI Audit categorises bots into AI Data Scrapers, AI Search Crawlers, and Archivers, allowing website owners to understand the impact of each type. Users can also block all AI crawlers with a single click or create custom rules to manage specific AI providers.
Cloudflare's CEO, Matthew Prince, envisions a future where website owners have greater control over their content, ensuring fair compensation for its use. The goal is to create a sustainable ecosystem where AI companies pay for content access, fostering mutual respect between content creators and AI developers.
