OpenAI official guide: OAI-SearchBot and ChatGPT search
According to OpenAI's official guidance, a website appears in ChatGPT search answers only if its crawler, OAI-SearchBot, is not opted out. Publishers should allow this bot in robots.txt and let its published IP addresses through their host or CDN. This makes a site eligible, and does not guarantee a citation. Blocking GPTBot is a separate choice that concerns model training.
I build websites and sell SEO, so I have an interest in the subject. I read OpenAI's crawler documentation directly on 1 November 2026. Its publishers and developers FAQ refused my automated reading, so I rely on the excerpts shown by search results, and you should read that page yourself.
The three OpenAI crawlers that publishers confuse
Most of the confusion comes from treating every OpenAI user agent (the name a bot gives when requesting a page) as the same thing. They have different jobs, and blocking the wrong one can cost you visibility without giving you the protection you wanted.
| Agent | Purpose | Effect of robots.txt | Published IP list |
|---|---|---|---|
| OAI-SearchBot | Surfaces websites in ChatGPT search features | Opted-out sites are not shown in search answers, though they may still appear as navigational links | searchbot.json |
| GPTBot | Crawls content that may be used to train foundation models | Disallowing it signals that content should not be used in training | gptbot.json |
| ChatGPT-User | Handles certain actions started by a user, such as visiting a page they asked about | Rules may not apply, because a user starts the action; it does not decide search inclusion | chatgpt-user.json |
The practical point: to control appearing in search, the relevant agent is OAI-SearchBot. OpenAI's own text sends you to that agent for search opt-outs, not to ChatGPT-User.
How to allow OAI-SearchBot as a publisher
A site that wants to be eligible for search while refusing training can use the following lines in robots.txt, the plain text file at the root of a domain:
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /The second block is a business decision and not a requirement. Some publishers are happy to be used for training, and some are not. Decide with your own content rights in mind, and take legal advice if your content is licensed from others. OpenAI says that it can take around 24 hours from a robots.txt update for its search systems to adjust.
Then check the layer people forget: the host. A robots.txt that says yes does not help if a firewall says no. OpenAI recommends allowing requests from its published IP ranges, and security rules at a CDN (the network that sits in front of a site) can block unknown bots by default. Ask your host directly: "Does your firewall block OAI-SearchBot, and do you allow the IP ranges in searchbot.json?" Your server logs will show whether the agent receives a normal page. A request claiming the name but coming from another address is not proof of a visit by OpenAI, which is why the lists exist.
What the official guidance does not tell you
In the crawler documentation I read, access is described, while selection is not. That page does not explain how a source gets picked for an answer, and I would not trust anyone presenting a precise formula as official. Excerpts of the FAQ indicate that any public site can appear, on condition that the crawler is allowed through. That leaves content quality, and your reputation elsewhere on the web, as matters of judgement and outside evidence, which I discuss separately, with the research and its limits.
Eligibility also says little about how often. Bear in mind that people ask ChatGPT about many subjects without any search being run at all, so a good page cannot appear in every related conversation. Promising a citation is a claim nobody outside OpenAI can honestly make.
How to measure ChatGPT referrals
OpenAI's FAQ states that ChatGPT automatically adds utm_source=chatgpt.com to referral URLs. In Google Analytics or a similar tool, filter by that source to see the landing pages and what visitors do next. Clicks, of course, are only part of the picture: being named without a click leaves no trace there. Asking new customers how they found you remains the cheapest way to catch this. Combine the two, and you will have an honest measure, rough as it is.
When this should not be your priority
If your site is not indexed by Google, if your pages carry little real information, or if your customers buy through a local phone call, fix those first. Being reachable by OAI-SearchBot takes minutes and is worth checking, but it gives a weak page nothing to say. And if your content is your product (a paid publication, for instance), blocking may be the wiser decision: a lawyer and a media specialist will advise you better than I can.
A simple next step
Open yourdomain.com/robots.txt and look for the three agents, or for a blanket Disallow: / on all bots. Ask your host about the firewall. Then filter your analytics on the ChatGPT source. If you would rather have someone do this, I offer a free website audit and SEO services. I am an independent developer in Metz, you speak directly to the person who does the work, and I answer within 24 working hours.