How to use Robots.txt File
Everything you wanted to know about the Robots.txt and how to use it. A robots.txt is a utility that can be used to wield further control on the Search Engine bots. A crucial aspect of any Search Engine Optimization strategy is controlling what the search engine spiders can see or cannot see on your website….