Robots.txt Checker
Check, test, and validate robots.txt syntax and crawl rules.
About this tool
Check and validate robots.txt syntax before deploy. Paste your file content and get immediate feedback on missing User-agent directives, malformed lines, and unknown directives.
Useful when debugging Search Console crawl errors, auditing staging vs production robots files, or learning robots.txt format. Pair with the Sitemap.xml Validator and read the robots.txt checker guide on the blog.
Your robots.txt content is processed locally — nothing is uploaded.
How to use
- Paste the contents of a robots.txt file.
- Validate directive syntax (User-agent, Allow, Disallow, Sitemap, etc.).
- Review reported issues and fix typos or conflicting rules.
- Re-check after edits before deploying to your site root.
Example
Block an admin path
Input
User-agent: * Disallow: /admin
Output
Valid robots.txt syntax Rules: * → Disallow /admin
Limitations
- Performs syntax and rule-structure checks only — it does not simulate a live Googlebot crawl.
- Crawl-budget, JavaScript rendering, and robots meta/X-Robots-Tag behavior are out of scope.
- Host-specific crawler quirks beyond the common directives may not be flagged.
Related guides
1Related tools
3Frequently asked questions
How do I check robots.txt?
Paste your robots.txt content into the checker and click Check. It validates User-agent directives, line format, and flags unknown directives — all in your browser.
What does the robots.txt checker test?
Syntax and structure: missing User-agent groups, malformed lines, and invalid directive names. It also warns about common issues like Disallow: / blocking all crawling.
Is this a robots.txt tester or validator?
Both — it checks syntax the same way a tester or validator would. Paste content from your site or a draft file before deploy.
Is the Robots.txt Checker free?
Yes. No signup or usage limits.
Is my robots.txt uploaded to a server?
No. Validation runs entirely in your browser. For a full walkthrough, see the robots.txt checker guide on the codefunc blog.