Free Online Robots.txt Generator

Allow / Disallow Paths

The Best Free Robots.txt Generator

The robots.txt file tells search engine crawlers which pages or sections of your site they can and cannot access. Getting it wrong can hurt your SEO or expose private areas. The GadgetsFocus free Robots.txt Generator creates a properly formatted robots.txt file with common WordPress rules built in.

How to Use

  1. Enter your website URL (for the Sitemap directive).
  2. Check the paths you want to disallow (wp-admin, search, tags, etc.).
  3. Add any custom disallow paths.
  4. Optionally set a crawl delay.
  5. Click “Generate” and copy the output to your site’s root directory.