Identity map
GPTBot vs OAI-SearchBot vs ChatGPT-User
The names belong to one company, but they represent different requests and different policy decisions.
Use the identity’s documented purpose as the decision boundary.
OAI-SearchBot is the relevant automatic crawler for OpenAI search discovery. GPTBot represents model-development crawling. ChatGPT-User identifies certain user-requested visits and should not be treated as an automatic indexing crawler.
OAI-SearchBot: search discovery
Use OAI-SearchBot when deciding whether public pages may be crawled for OpenAI search experiences. Allowing it does not guarantee crawling, inclusion, citation, or traffic; it removes one technical barrier.
Keep canonical URLs, sitemaps, internal links, and page-level indexing directives consistent. Conflicting signals make discovery harder even when the crawler itself is allowed.
GPTBot: model development
GPTBot is a separate policy choice related to improving and training foundation models. A site can allow OAI-SearchBot and decline GPTBot. That distinction is useful for organizations that want search visibility without granting every downstream use.
Document the decision internally so a future security or SEO change does not collapse separate policies into one blanket rule.
ChatGPT-User: a person requested the visit
ChatGPT-User can appear when a person asks ChatGPT to visit or use a page. This is closer to a user following a link than to a crawler building a search index. Vendor documentation should be the source of truth for how robots.txt applies.
Security still matters. Public content can be served normally while private, account, checkout, and administrative routes remain protected by authentication and authorization.
Turn the comparison into policy
A useful policy matrix has one row per identity and one column for purpose, desired access, robots.txt rule, network treatment, and verification method. This makes the decision reviewable by legal, security, content, and engineering teams.
- Do not infer purpose from a bot name alone.
- Do not assume allowing search also allows training, or the reverse.
- Do not use a copied user-agent string as proof that traffic is genuine.
Verification checklist
- Record a separate allow/block decision for all three identities.
- Link the decision to current first-party OpenAI documentation.
- Translate only automatic-crawler decisions into robots.txt rules.
- Keep private routes protected regardless of crawler identity.
- Review the policy when vendor documentation changes.