AI Robots Checker
Find out which AI crawlers your robots.txt allows to access your website.
Reads your public robots.txt. You can also paste a file to check its rules locally.
The essentials
Your website’s first checkpoint.
robots.txt describes which paths a crawler may request. Search crawlers and training crawlers have different purposes: allowing one does not require allowing the other. This check evaluates the path you enter, not every page on the site.
User-agent: * Allow: / User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: OAI-SearchBot Allow: / User-agent: Claude-SearchBot Allow: /
From insight to action
Make your next move clearer.
See the rule behind every result.
Start with a defined input and understand exactly what the tool checks.
Separate AI search from model training.
Review the evidence alongside the result before drawing conclusions.
Catch accidental blocks before publishing.
Save your findings, make a targeted change, and compare a fresh sample.
A few things to know.
How does this tool work?+
Reads your public robots.txt. You can also paste a file to check its rules locally.
Is this free to use?+
Public website scans have a daily limit. Local analysis of pasted content is free and does not require an account.
Does this measure all AI platforms?+
Results cover only the data or technical signals explicitly shown in the report. They do not represent all AI conversations, search volume, or a guarantee of recommendations.
Can I save the results?+
Yes. Copy the report or download it as JSON. The export includes its data source so you can keep the analysis in context.
Keep exploring
Your next useful tool.
GEO Score Checker
Check the technical signals that help AI systems discover and understand your website.
TechnicalKnowledge Freshness Checker
Surface dated references in AI answers so you know what to verify next.
ResearchAI Search Volume Checker
Understand demand for a prompt using the search observations you collect.
Turn better insights into
your next growth move.
Bring your research, content, and growth workflow together.