🧩 SEO Builders

Robots.txt Generator

List the paths you want kept out of the crawl and get correctly formatted robots.txt rules back — for any bot you name.

100% free, no limits
No sign-up
Your data never leaves your browser
Results in under a second

Use the Robots.txt Generator now ↓ Open it in the full Notepad

Free tool Paste your list — the Robots.txt Generator does the rest

The example below is already loaded, so you can see exactly what you get. Replace it with your own list and the result updates as you type.

🧩 Doing the whole job? Generate SEO markup → every fix for it, ticked and in the right order

Got a bigger job? The full Notepad runs the same engine on lists of any size — chain this with dedupe, sort and filter, undo any step, and save the whole routine as a recipe for next time.

Example Robots.txt Generator: before and after

Before

/wp-admin/
https://example.com/cart/
search

After

User-agent: *
Disallow: /wp-admin/
Disallow: /cart/
Disallow: /search

Sitemap: https://example.com/sitemap.xml

Settings used: Sitemap URL (optional): https://example.com/sitemap.xml

How it works What the Robots.txt Generator does

Turns a list of paths or URLs into robots.txt Disallow (or Allow) rules for the user-agent you choose, with an optional Sitemap line.

List the paths or URLs you want to block (or allow) and get a robots.txt group back: the User-agent line, one Disallow or Allow rule per path, and an optional Sitemap line. Full URLs are reduced to their paths and missing leading slashes are added.

Test the result with the Robots.txt Tester before uploading. Remember that robots.txt stops crawling, not indexing; use noindex for pages that must not appear in search.

3 steps How to use the Robots.txt Generator

Paste your list into the box above — one item per line. Keyword exports, crawl exports and spreadsheet columns all work.
Check the settings and adjust them to your job — the result updates as you type.
Copy the result and paste it wherever it needs to go. Nothing is uploaded, so client and confidential data stay on your machine.

Use it to When to use the Robots.txt Generator

Block cart, checkout and internal search URLs from being crawled
Write a group of rules for one specific bot such as GPTBot
Add a Sitemap line to a new site’s robots.txt

FAQ Robots.txt Generator questions

Is Build robots.txt Rules free to use?

Yes. Build robots.txt Rules runs entirely in your browser, on this page and in the full Notepad, with no sign-up, no limits, and nothing sent to a server.

Does Disallow remove pages from Google?

No. It stops crawling, but a blocked URL can still be indexed if other sites link to it. Use a noindex tag instead.

How do I target one crawler?

Enter its user-agent name, e.g. GPTBot or Googlebot-Image, instead of *.

Pairs well with More SEO Builders tools, all free

Stop fixing lists by hand

The Robots.txt Generator is one of the free tools inside Seowolf’s Notepad. Paste a list once, then dedupe, filter, sort, compare and format it in seconds — every step undoable, nothing uploaded, no account.

Open the Notepad — it’s free

← Browse all free tools