Learn · Business
What is robots.txt?
robots.txt is a file that tells search engine crawlers which parts of a site they should not crawl.
Why it matters
A wrong rule can hide your whole site from Google.
Practical tip
Never block CSS or JS files that your pages need to render.
Frequently asked questions
What is robots.txt?
robots.txt is a file that tells search engine crawlers which parts of a site they should not crawl.
Why does robots.txt matter for my website?
A wrong rule can hide your whole site from Google.
Any practical tip about robots.txt?
Never block CSS or JS files that your pages need to render.