One line now opts you out of AI training. Most B2B platforms should not add it
Key takeaways
- On 30 October Google said it will support a shared AI training signal in robots.txt from early 2027. Bing, OpenAI and Anthropic made similar moves.
- The signal does not touch Google Search rankings, AI Overviews or AI Mode.
- Google reads it as a Google-Extended block unless you set Google-Extended yourself, and that also covers grounding in Gemini Apps.
- Publishers sell content. B2B platforms sell matches. Blocking by default hides the shop window.
- Decide by path: open discovery pages, protect the data that is your product, and write the decision down before early 2027.
Every few months the AI crawler debate gets a new tool. This one is useful, because it finally lets a site state one preference for several AI companies at once. I still think most B2B platforms will use it the wrong way, and it matters for anyone working on GEO for marketplaces.
Quick Navigation
- What happened
- My take: a marketplace is not a publisher
- Why it matters for marketplaces, agencies, communities and SaaS
- The growth play
- What to do before early 2027
- FAQ
- Related Articles
What happened
On 30 October 2026 Google published Supporting a shared AI training control for websites on the Search Central blog. Google says it will voluntarily adopt a proposal from the AIPREF working group at the IETF and "intends to support" it in early 2027. In Google's words, it "will help websites express their AI training preferences across multiple AI providers in a single step."
The signal is a policy token, not a crawler. You write a robots.txt group for User-Agent: __usage_ai-training and allow or disallow paths under it. Barry Schwartz at Search Engine Roundtable reported the details and the caveats, including Nate Hake's point that "some significant areas of dispute remain" at the IETF level.
Luis Rijo at PPC Land put the four announcements side by side. Microsoft will honor AI training and grounding preferences from 1 February 2027. OpenAI aims to implement the signal "in the coming months". Anthropic "has already started implementing support". A blanket User-Agent: * block does not count as a training preference either way.
Two details matter most for platforms. First, Google says existing controls such as Google-Extended keep working, and John Mueller said that if a site opts out with the new token without setting Google-Extended, Google will treat Google-Extended as disallowed. Second, Google's own documentation says Google-Extended covers both training of future Gemini models and grounding in Gemini Apps, while it "does not impact a site's inclusion in Google Search". AI Overviews and AI Mode sit under Googlebot rules and the Search Console control, not this token.
My take: a marketplace is not a publisher
A standard is good news. Before this, a site that wanted to opt out had to chase a different user agent for every AI company. Now there is one line, and four of the biggest companies say they will read it.
My problem is with the default that will follow. Most of the conversation around AI training comes from publishers, and for a publisher the logic is clear. Their product is the text. If a model learns it for free, they lose something they could license.
A B2B marketplace is a different business. Its product is the match between a buyer and a supplier. Category pages, listing summaries and "how it works" pages are not the product. They are the shop window. When an assistant has learned that your platform exists and what it is for, it can name you when a buyer asks where to find a supplier. Block that and you protect very little while you make yourself harder to find.
At DesignRush we grew a B2B agency marketplace to over a million monthly organic visitors. The listing and category pages were never what we sold. They were how buyers found us. That is the lens I would use for every path in robots.txt.
The data that really is your product is a different case. Pricing history, supplier contact data behind sign-up, proprietary datasets and member content you do not own deserve protection. The new token lets you do that by path, so use it that way.
Why it matters for marketplaces, agencies, communities and SaaS
B2B marketplaces. Split your site in two lists: pages that make you discoverable and data that is your product. Keep the first open and protect the second. Remember the Google mapping: a token block without an explicit Google-Extended line also removes you from Gemini Apps grounding. More on this in our marketplace SEO work.
Agencies. Clients will hear "one line blocks AI" and ask you to add it. Treat training, grounding and search as three separate decisions in every technical audit, and check child properties that can override the parent. It is a short, valuable deliverable for digital agency growth.
B2B communities. Member posts are the hardest call, because the content belongs to members. Ask them, decide, and say publicly what you chose. Named expert answers are still what AI answers quote, so an open choice is a growth choice. See community SEO.
SaaS with directories or marketplaces. Docs, integration pages and comparison pages are how assistants explain and recommend your product. Keep them open. Customer-generated content in your app is a separate decision. See our SaaS SEO work.
The growth play
| Move | Who it is for | First step |
|---|---|---|
| Write an AI access policy by path | All four | List your top-level paths and mark each one discoverable or product |
| Keep the shop window open | Marketplaces, SaaS | Leave category, listing summary, docs and comparison paths allowed for training and grounding |
| Protect the product data | Marketplaces, SaaS | Disallow datasets, price history and gated supplier data under the new token |
| Set Google-Extended explicitly | All four | Add your own Google-Extended group so the token does not decide grounding for you |
| Add AI access to every audit | Agencies | Review training, grounding and search settings, plus Search Console child properties |
| Decide member content openly | Communities | Ask members, publish the choice, keep named expert answers public if you stay in |
What to do before early 2027
Nothing changes in Google's behaviour until early 2027, and Microsoft starts on 1 February 2027. That is time to decide calmly instead of copying a line from a forum. Write the policy, have the people who own sales, product and community sign it off, then update robots.txt once. Keep Search Console on Include for AI Overviews and AI Mode unless you have a strong reason not to; that is a separate control and this token does not change it.
FAQ
What is the new AI training control in robots.txt? It is a policy token from the IETF AIPREF work, written as User-Agent: __usage_ai-training. It lets a site state one AI training preference for every AI company that supports it. Google intends to support it in early 2027.
Does the AI training opt-out affect Google Search or AI Overviews? No. Google says Google-Extended does not affect inclusion in Google Search, and AI Overviews and AI Mode are governed by Googlebot rules and the Search Generative AI control in Search Console.
Does blocking AI training also block Gemini grounding? At Google it can. If you use the new token without setting Google-Extended yourself, Google treats it as a Google-Extended block, and Google-Extended covers grounding in Gemini Apps as well as training.
Should a B2B marketplace block AI training? Not site-wide. Keep the pages that make you discoverable open and protect the data that is your product, path by path.