A program that automatically explores the internet by retrieving a document and recursively retrieving some or all the documents that are referenced in it. These programs are commonly employed by Search Engines in order to create results for queries sent to the search engine by web surfers. Spiders can be programmed to search for and gather all the information they encounters or for very narrow search parameters.