Top 3 web-crawler-python open source projects

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Explore a website recursively and download all the wanted documents (PDF, ODT…)

Compares price of the product entered by the user from e-commerce sites Amazon and Flipkart 💰 📊