web crawler
noun
variants:
or Web crawler
Definition of web crawler
: a computer program that automatically and systematically searches web pages for certain keywords
Each search engine has its own proprietary computation (called an "algorithm") that ranks websites for each keyword or combination of keywords. These algorithms use Web crawlers … that collect data from your website to determine where it ranks with respect to search terms.— Julie Brinton
Keep scrolling for more