Validate a robots.txt file

Paste the contents of your robots.txt file to check its syntax and spot invalid directives or a missing User-agent rule. 100% local processing (WebAssembly).

Example:

How it works

A syntax error in robots.txt can cause crawlers to ignore some rules, with consequences ranging from accidentally blocking important pages to crawling areas that should stay private. Paste the contents of your file before deploying it to check that every line follows a recognized syntax.

Which directives are recognized as valid?

The tool recognizes User-agent, Disallow, Allow, Sitemap and Crawl-delay (case-insensitive). Blank lines and comments starting with # are ignored; everything else is flagged as potentially invalid.

Why is a User-agent directive required?

Without a User-agent directive, the Disallow or Allow rules that follow have no target crawler and may be ignored by search engines. Each block of rules must start with a User-agent line indicating which crawler it applies to (* for all).

100% local processing: validation runs entirely in your browser via WebAssembly — your file never leaves your device.