BCP: Bot Consent Protocol a Standard for Regulating Automated Access to Authored Digital Content
An innovation in the field of indexing and searching for copyrighted content. The Bot Consent Protocol (short: BCP), in its original version from December 2025, successfully completed its trial on several test sites a few weeks after its launch and was validated in practice in early January 2026.
BCP as Bot Consent Protocol is a proposed and is already in use Standard for Regulating Automated Access to Authored Digital Content.
Whitepaper on the Legal, Technical, and Economic Framework for Managing Bots, Crawlers, and Automated Traffic
Version: 1.0 (Draft) | Status: Proposed Standard | Acronym: BCP | Date: January 8th, 2026
1. Executive Summary
The Bot Consent Protocol (BCP) is a proposed technical and legal standard for regulating automated access to websites.
Its purpose is to establish a transparent, fair, and measurable relationship between website owners and automated systems accessing their infrastructure—including search engine crawlers, analytics tools, scrapers, AI model crawlers, and other forms of non-human traffic.
In the current web ecosystem, automated traffic accounts for a significant share—often the majority—of total traffic, yet most of this traffic is unregulated, unaccountable, and carries no responsibility for the load it imposes.
The traditional mechanism, robots.txt, introduced in 1994, has become ineffective: it is not legally binding, not technically enforceable, and does not protect against excessive or abusive crawling.
BCP introduces a new approach: automated agents must accept clearly defined terms of use before continuing access, similar to how human users must accept cookies, GDPR notices, or terms of service.
Acceptance is automated and based on continued requests after the display of a Bot Consent Page (BCP Page).
This creates a legal and technical audit trail that enables documentation, regulation, and, where appropriate, billing of the costs caused by automated traffic.
The protocol also defines an economic model that sets a price for automated access (e.g., €0.01 per request), as well as mechanisms for identifying, classifying, and sanctioning bots that violate the rules.
BCP is designed to be technically feasible in existing server environments (nginx, Apache, Cloudflare, Node.js, PHP, Python) and legally compatible with existing regulations, including GDPR, ePrivacy, and the Digital Services Act.
The goal of BCP is not to restrict access, but to establish a fair, transparent, and accountable system in which automated visitors are subject to the same fundamental principles as human users: information, consent, and responsibility.
2. Introduction
Over the past two decades, the web has evolved from an environment dominated by human visitors into one where most traffic is generated by automated systems.
Search engine crawlers, analytics bots, SEO tools, scraping systems, and various automated agents now form the backbone of the digital economy.
However, this backbone operates without clear rules, without accountability, and without economic balance.
Website owners—especially publishers, bloggers, media organizations, and independent creators—bear the full cost of the infrastructure that bots consume.
Excessive crawling, ignoring robots.txt, forged user agents, and aggressive scrapers generate costs that are not compensated by anyone.
At the same time, publishers have no effective mechanism to regulate or limit automated access, beyond the outdated robots.txt, which is merely a non-binding recommendation with no legal or technical weight.
In this context, it becomes clear that the web needs a new standard—one that reflects the reality of the modern internet, where automated systems are the norm rather than the exception.
The Bot Consent Protocol (BCP) is a proposal for such a standard: a system that imposes on automated visitors the same fundamental obligations as on human users—being informed, providing consent, and bearing responsibility.
BCP introduces the concept of a Bot Consent Page, where automated systems are informed about terms of use, pricing, and access rules.
Continuing access after the BCP Page is shown constitutes acceptance of these terms, creating a legal and technical trail that enables regulation and cost accounting.
The protocol also includes technical guidelines for identifying bots, classifying traffic, limiting abuse, and implementing sanctions.
This document presents the overall concept, legal justification, technical specification, and economic model of BCP and serves as a basis for discussion, development, and potential standardization.
