More hints important link
https://cloudlinks1.blob.core.windows.net/custombusinessintelligenceservices/Web-Scraping-Services/web-scraping/internet-scraping-vs-web-crawling-whats-the39851.html
So as you can see, web spiders are vital in producing accurate results. Without web crawling, you would not have Google providing you search results page in a progressively more precise and also effective way.