What are robots.txt files?



The robots.txt is a file at the root a domain indicates parts of the site should not be accessed by search engine crawlers.

https://www.irradiah.com/robots.txt


Comments