🔍 Read the full analysis: A Guide To The AI Training Opt-Out Supported By Four Major Companies on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic say they will support a shared robots.txt signal that lets website operators express a preference against using their content to train generative AI models. The signal is voluntary, does not block crawling or cover AI answers based on live web content, and the companies plan to adopt it on different schedules. Meta voiced support but has not committed to honoring it.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will support a shared robots.txt signal for website operators who do not want their content used to train generative AI models. The move could reduce the need to set separate training preferences for each company, but the signal is voluntary, does not stop sites from being crawled and will take effect on different schedules.
The proposed rule uses the user-agent name __usage_ai-training, followed by Disallow: /. In practice, it tells participating companies that a site prefers its content not be used for generative AI training. The shared signal is set out in an individual Internet-Draft titled “Machine-Readable Training Preference Signal,” written by Anthropic standards lead Kevin Kelley and published on October 30.
Anthropic said it has already started implementing support, though it expects adoption across its services to take time. OpenAI said it aims to implement the signal in the coming months. Microsoft will honor it from February 1, 2027, as it crawls websites, while Google aims to support it in 2027; its crawler documentation specifies early 2027. Those are company timelines, not a single coordinated launch date.
Microsoft is also introducing a separate page-level preference for AI Grounding—the use of web content to inform answers in Bing, Copilot and Microsoft 365. Microsoft says both preferences begin on February 1, 2027. Its existing NoArchive and NoCache tags will stop counting as preferences for training or grounding after March 1, 2027. Existing crawler controls, including rules for GPTBot, Google-Extended and ClaudeBot, remain relevant while companies roll out support.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
One Rule, Different Company Timelines
A shared signal could make it easier for publishers to express a preference across several major AI services without maintaining a separate training rule for each one. That matters to news organizations, other publishers and website operators whose material may be collected for AI model development. The rule offers a common technical vocabulary, but it does not create a technical barrier against access or guarantee that every company will follow the request.
The boundaries matter for anyone deciding how to configure a site. The proposed signal concerns training; it does not, by itself, opt a site out of AI answers generated using live web content. Microsoft has made that distinction explicit with its separate grounding preference. Nor does the signal provide payment or guarantee that opting out will have no effect on a site’s visibility. Microsoft says its choices do not guarantee inclusion, traffic or payment, and says the preferences do not change eligibility for Bing search results. Google says Google-Extended does not affect Google Search.
For publishers, the announcement creates a practical interim task: add the new rule if it reflects their preference, while keeping company-specific rules in place until each service supports the common signal. Operators relying on Microsoft’s NoArchive or NoCache tags will also need to review those settings before the stated March 2027 change.
Products Worth Considering
Ledger Nano Gen5 Crypto Wallet, Black
Ledger Flex + Folio Crypto Hardware Wallet
Logitech MX Keys S Combo Wireless Keyboard and Mouse, Graphite
As an affiliate, we earn on qualifying purchases.
The proposal is connected to the Internet Engineering Task Force’s AI Preferences working group, which considers ways to express machine-readable preferences. However, the document is currently an individual Internet-Draft, not an adopted internet standard. At a working-group meeting on October 21, a chair said, “We haven’t declared consensus on anything yet.” The companies’ announcements indicate support for the draft’s approach, not formal approval through the IETF standards process.
The four companies describe adoption in different terms. OpenAI calls its commitment a voluntary effort. Google says it plans to interpret the shared signal as a disallow rule for Google-Extended when a site has no separate rule for that crawler. Microsoft’s announcement includes a distinct control for AI grounding. Anthropic says implementation is underway, while its crawler help page on October 31 still described its existing per-crawler controls. These details show why the shared line should not be treated as an immediate replacement for every existing setting.
Meta communications chief Andy Stone said the company supports publishers’ ability to express preferences and called a common signal an important step. He did not say that Meta would honor it. The source account also reported finding no public commitment from Apple, Amazon, xAI or Perplexity among the statements reviewed; that is not evidence that those companies have rejected the proposal.
“has already started implementing support for this draft”
— Anthropic, in its announcement of the draft
What the Signal Cannot Guarantee
The draft has no formal standing in the IETF standards process at present, and it is not clear whether or when that status may change. The commitments also differ: some are plans or targets, and Meta has not said whether it will honor the rule. Public statements from other AI companies may emerge, but the companies named in the source material had not announced participation there.
The signal is a request expressed through robots.txt, not a technical lock. The announced plans do not establish how compliance will be monitored or what recourse a publisher would have if a company used content despite the preference. It also does not remove material from models that have already been trained, prevent crawling, cover every use of a website in AI products, or establish compensation.
Specific rollout details remain unsettled as well. Anthropic’s broader service support is still in progress, and the companies’ separate implementation dates mean site operators cannot assume that one line will be recognized by every participant immediately. The draft also does not settle how nonparticipating services will treat it.
Publisher Steps Before Rollout
Website operators can review the draft’s syntax and decide whether to add User-Agent: __usage_ai-training and Disallow: / to their robots.txt file. They should retain existing company-specific crawler rules until they confirm that each service has implemented the shared signal and that its documentation covers the relevant use. Those wishing to limit AI-generated answers based on live content need to check the separate controls available from each service; the training rule alone does not express that preference.
Microsoft’s stated dates give operators a specific point to plan around: its new training and grounding preferences take effect on February 1, 2027, while NoArchive and NoCache stop counting for those purposes after March 1, 2027. Google has set an early-2027 target, and OpenAI has said “in the coming months” without giving a calendar date in the source material. Anthropic says it has started implementation but has not specified a completion date across its services.
The next developments to watch are whether the IETF working group reaches consensus on the draft, whether participating companies publish updated crawler guidance, and whether Meta or other AI providers make clear commitments. Until then, the shared rule is a useful expression of publisher preference, but operators will still need to check each company’s policies and timelines.
Key Questions
What does the shared AI training signal do?
It lets a website operator express a preference in robots.txt that participating companies not use the site’s content to train generative AI models. The proposed rule uses __usage_ai-training with Disallow: /.
Does the rule stop AI companies from crawling a website?
No. The signal is a stated preference about training use, not a technical block on crawling. It also does not remove content from models that have already been trained.
Does the training opt-out cover AI-generated answers?
Not by itself. The draft signal concerns model training, while AI answers based on live web content are a separate use. Microsoft announced a separate AI Grounding preference for certain of its services.
When will the companies honor the signal?
Anthropic says implementation has started; OpenAI aims to implement it in the coming months; Google targets 2027, with its documentation saying early 2027; and Microsoft says it will honor the preference from February 1, 2027. The dates and status are not uniform.
Has Meta committed to honoring the rule?
No commitment was included in the source material. Meta communications chief Andy Stone expressed support for publishers’ ability to state preferences but did not say whether Meta would honor this particular signal.
Source: ThorstenMeyerAI.com
