A crawler, or web crawler, is a computer program that "reads" or indexes data on the World Wide Web. Search engines use crawlers to provide up-to-date search results [1]. Also known as a spider [2].
A crawler, or web crawler, is a computer program that "reads" or indexes data on the World Wide Web. Search engines use crawlers to provide up-to-date search results [1]. Also known as a spider [2].
Links:
[1] http://www.wordstream.com/search-results
[2] http://www.wordstream.com/spider