AI crawler checker: is your site blocked from ChatGPT?
See whether ChatGPT, Perplexity, Claude and other AI answer engines are allowed to read your site, whether you opted out of Gemini, which robots.txt lines decide it, and the rules to let them in.
Enter a domain or a page URL. We only read your robots.txt, so it takes a second or two.
How the AI crawler checker works
AI assistants answer questions with pages their crawlers are allowed to fetch. Each crawler has its own user-agent name, and your robots.txt decides which ones may read which paths. One blanket Disallow: / can take a site out of ChatGPT, Perplexity and Claude answers without anyone noticing.
The checker fetches your robots.txt and evaluates it for 22 AI bots, the way their own parsers do. Answer engines that can cite you are kept apart from bots that only collect training data and from opt-out tokens such as Google-Extended, which never fetch a page. For every bot you get the User-agent group that applies and the exact Allow or Disallow line that decided the result.
If answer engines are blocked, you get a robots.txt group to paste in that lets them read your public pages while keeping your other rules, and your training-bot blocks, in place.
Frequently asked questions
- How do I know if my site is blocked from ChatGPT?
- ChatGPT reads the web through two bots: OAI-SearchBot for search results and ChatGPT-User for pages it opens while answering someone. If robots.txt disallows them, by name or through a blanket User-agent: * rule, ChatGPT can't fetch your pages to cite them. Enter your domain above to see the result for each bot.
- What's the difference between answer engines and training bots?
- Answer-engine bots fetch pages to answer a question and can link back to you. Training bots such as GPTBot, ClaudeBot and CCBot collect text to train future models. Blocking training bots keeps your content out of training data but doesn't remove you from AI answers. Blocking answer engines does.
- Should I block GPTBot?
- That's a policy choice. Blocking GPTBot keeps your pages out of OpenAI's model training. It doesn't stop ChatGPT search from finding and citing you, because that's OAI-SearchBot.
- Does blocking Google-Extended hurt my Google rankings?
- No. Google-Extended isn't a crawler. It's a robots.txt token that controls whether Gemini can train on your content or ground its answers in it. Googlebot does the crawling, so Google Search, AI Overviews and your rankings aren't affected by it.
- Does this checker crawl my site?
- No. It fetches one file, /robots.txt, and checks its rules for each bot against the page you entered. It reads the rules the same way the bots' own robots.txt parsers do.
- Why can an AI bot be blocked when robots.txt allows it?
- Firewalls and CDN bot protection can refuse AI crawlers before robots.txt is ever read. This checker only reads robots.txt, so check your CDN or hosting settings too.