Short answer
Allow the crawlers that put you in AI search and ads (OAI-SearchBot, OAI-AdsBot, Googlebot, Bingbot), and decide separately whether to allow training crawlers such as GPTBot and Google-Extended. Blocking training crawlers doesn't remove you from ChatGPT search or Google's AI features.
Example
# AI search and ads
User-agent: OAI-SearchBot
Allow: /
User-agent: OAI-AdsBot
Allow: /
# Training (your choice)
User-agent: GPTBot
Allow: /
User-agent: Google-Extended
Allow: /
Sitemap: https://example.com/sitemap.xmlWhat each controls
| Crawler | Controls |
|---|---|
| OAI-SearchBot | ChatGPT search answers |
| OAI-AdsBot | Review of ChatGPT ad landing pages |
| GPTBot | OpenAI model training |
| Googlebot | Google Search, including AI Overviews and AI Mode |
| Google-Extended | Gemini training and grounding in some other Google systems |
Questions
How fast do changes apply?
OpenAI says about 24 hours for OAI-SearchBot.
Facts checked against the sources below on 12 October 2026. Platforms change quickly: if something here no longer matches what you see, tell us and we will update it.
