今日头条科技新闻接口爬虫

百度贴吧爬虫(基于scrapy和mysql)

swiss army knife for hackers

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

Uno Crawler por https://douban.com

A set of spiders and scrapers to extract location information from places that post their location on the internet.

API for scrapping common meme sites

🚀电商网站爬虫合集，淘宝京东亚马逊等

TripAdvisor scraper

🎊 Design and implement of lightweight crawler framework.

PTT 圖片下載器 (抓取整個看板的圖片，並用文章標題作為資料夾的名稱 ) (使用Scrapy)

Generating spiders dynamically to crawl and check those free proxy ip on the internet with scrapy.

Crawlera middleware for Scrapy

a simple & tiny scrapy clustering solution, considered a drop-in replacement for scrapyd

A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

Building a model to recognize incentives for landscape restoration in environmental policies from Latin America, the US and India. Bringing NLP to the world of policy analysis through an extensible framework that includes scraping, preprocessing, active learning and text analysis pipelines.

🔧 🔩 🔨 收集整理了爬虫相关的工具、模拟登陆技术、代理IP、scrapy模板代码等内容。

The Zipru scraper developed in the Advanced Web Scraping Tutorial.

Build and deploy a machine learning app from scratch 🚀

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy