robots.txt is a text file placed at the root of your website that tells search engine crawlers which pages or sections they can or cannot access.

Can robots.txt block pages from appearing in Google?

robots.txt prevents crawling but not indexing. If other sites link to a blocked page, Google may still index the URL. Use a noindex meta tag to prevent indexing.

robots.txt is a text file placed at the root of your website that tells search engine crawlers which pages or sections they can or cannot access.

Can robots.txt block pages from appearing in Google?

robots.txt prevents crawling but not indexing. If other sites link to a blocked page, Google may still index the URL. Use a noindex meta tag to prevent indexing.

免费

robots.txt生成器

使用可视化编辑器构建robots.txt文件。选择预设或手动自定义规则。

100%免费无需注册在浏览器中运行

User-Agent

Allow

Disallow

站点地图URL

Host（可选）

测试URL路径

robots.txt预览

User-agent: *
Allow: /

关于robots.txt Generator

Create a properly formatted robots.txt file using a visual editor. Add allow and disallow rules for different user agents, set crawl delays, and link your sitemap. Download the file or copy the output directly.

robots.txt生成器

robots.txt预览

关于robots.txt Generator

常见问题

更多SEO & Meta工具

Sitemap XML Generator

Website SEO Checker

Meta Tag / OG Preview