🔍 Read the full analysis: Can One Opt-Out Stop Your Data From Training AI Models? on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
OpenAI, Google, Microsoft and Anthropic have announced support for a shared robots.txt signal that lets website operators state they do not want their content used to train generative AI models. The signal is voluntary, does not block crawling or cover AI answers based on live web content, and is not yet an official internet standard; companies have different implementation timelines.
OpenAI, Google, Microsoft and Anthropic said on October 30 they will support a shared website signal for opting out of generative AI training, giving publishers one proposed robots.txt rule instead of separate training instructions for each company. The signal is a request rather than a technical block, and it does not cover every use of web content by AI services.
The proposal is an individual Internet-Draft titled “Machine-Readable Training Preference Signal,” submitted by Anthropic standards lead Kevin Kelley. It specifies a robots.txt group using User-Agent: __usage_ai-training followed by Disallow: /. A site owner can use that rule to express a preference that participating companies not use the site’s content to train generative AI models.
The companies have not set a common launch date. Anthropic says it has started implementing support, although it expects support across its services to take time. OpenAI says it aims to implement the signal “in the coming months.” Google is aiming for 2027, with its crawler documentation specifying early 2027. Microsoft says it will honor the training preference from February 1, 2027, as it crawls sites.
Microsoft is also adding a separate page-level preference for AI Grounding—the use of web content to inform AI answers in Bing, Copilot and Microsoft 365. That is distinct from the shared training signal. Microsoft says its existing NoArchive and NoCache preferences will stop counting as training or grounding preferences after March 1, 2027. Its announcement says neither choice guarantees inclusion in search or AI answers, traffic, or payment.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
One Rule, Different AI Uses
A shared instruction could make it easier for publishers and website operators to communicate a training preference across participating services. Until now, operators have had to account for company-specific crawler controls, including GPTBot for OpenAI, Google-Extended and ClaudeBot. Microsoft has also used page-level controls. A common signal may reduce the need to maintain a separate training rule for every participating provider.
But the practical effect depends on companies choosing to honor the request, adopting it across their services and interpreting it consistently. The rule does not prevent a crawler from accessing a site, nor does it remove material already used in trained models. Training, search indexing and AI answers based on current web content are separate uses, and a publisher may need different controls for each. The signal also creates no payment or traffic entitlement.
Products Worth Considering
Bose QuietComfort Wireless Noise Cancelling Headphones, Sandstone
Anker SOLIX S2000 Portable Power Station, 2,010Wh, 1,500W
Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black
As an affiliate, we earn on qualifying purchases.
The Draft and Existing Controls
The proposal is associated with the IETF’s AI Preferences working group, which is considering ways for websites to express machine-readable preferences about AI use. However, the text is an individual Internet-Draft, not an adopted IETF standard. At the group’s October 21 meeting, one chair said, “We haven’t declared consensus on anything yet.” That means the announcement of company support should not be confused with formal standards approval.
Company guidance preserves some current controls. OpenAI says its existing crawler rules will continue to apply. Google says that, where a site uses the shared training signal but has no Google-Extended rule, it plans to treat the shared signal as if Google-Extended had been disallowed. Google says Google-Extended does not affect Google Search. Microsoft says its preferences do not change eligibility for Bing search results. Anthropic’s crawler help page, as of October 31, still described its existing per-crawler controls.
Meta’s communications chief, Andy Stone, said the company “strongly support[s] publishers’ ability to express preferences” and called a common signal “an important step.” That is a statement of support, not an implementation commitment. The source report said it had found no statements from Apple, Amazon, xAI or Perplexity about honoring the signal.
““has already started implementing support for this draft””
— Anthropic, describing its implementation
Adoption and Enforcement Remain Open
The draft does not carry the force of an adopted internet standard, and the IETF working group has not declared consensus. Participation remains voluntary; the announcement does not establish what consequences, if any, would follow if a company ignored the signal. The precise scope of implementation at each company, including which products and services will recognize it, is also not fully specified.
Meta has not made an implementation commitment in the material cited, and the position of several other major AI companies remains unreported. It is also not clear how consistently the signal will be interpreted across crawlers or whether later versions of the draft will change its wording. The signal is specifically for training preferences: it does not, by itself, opt a site out of AI answers grounded in live web content.
Company Rollouts and Draft Review
Publishers will need to watch for company documentation and implementation updates. Anthropic says rollout has begun; OpenAI’s stated target is the coming months; Google’s target is early 2027; and Microsoft’s start date is February 1, 2027. Microsoft has set March 1, 2027 as the date after which NoArchive and NoCache will no longer count as training or grounding preferences.
Until services have adopted the shared signal, website operators may need to retain existing company-specific rules and check each provider’s instructions. They should also decide separately whether they want to restrict training, AI grounding or search-related uses. Further developments will depend on company rollout commitments and whether the Internet-Draft advances through the IETF process.
Key Questions
What does the shared AI training opt-out do?
It lets a website publish a robots.txt preference asking participating companies not to use its content to train generative AI models. The request is not a technical block on crawling.
Does one opt-out stop all AI companies from using a site?
No. The signal only applies where a company chooses to honor and implement it. The announcement names OpenAI, Google, Microsoft and Anthropic; Meta expressed support without confirming adoption, and the position of other companies is unclear.
Does it remove content from models already trained?
No. The described rule is a preference about training use going forward. It does not remove material already used in existing models.
Does the rule block AI answers or search results?
Not by itself. The shared signal addresses model training, not AI answers based on live web content. Microsoft’s separate AI Grounding preference addresses some uses in its products. The companies say the training preference does not guarantee search inclusion, traffic or payment.
Is the signal an official internet standard?
No. It is an individual Internet-Draft linked to the IETF AI Preferences working group. The group had not declared consensus as of its October 21 meeting.
Source: ThorstenMeyerAI.com
