Author Topic: What is robots.txt used for?  (Read 2850 times)

Offline danielnash

  • Jr. Member
  • Posts: 25
  • digital marketing expert and blogger
    • View Profile
    • Road Transport in UK
What is robots.txt used for?
« on: March 09, 2016, 05:50:59 AM »
I learnt here about sitemap.xml file. Now I'm little bit confused for what is robots.txt used for?

Please guys clear it.



Offline snginc

  • Moderator
  • Posts: 3,615
  • Nothing is impossible to an open mind!
    • View Profile
Re: What is robots.txt used for?
« Reply #1 on: March 09, 2016, 06:37:49 AM »
You can read about it here http://www.robotstxt.org/robotstxt.html