289 Open source projects that are alternatives of or similar to Scrapydouban

Crawlera middleware for Scrapy

Get movie info from douban(豆瓣) and display in your terminal

PTT 網路版爬蟲

Document Search Engine Tool

Visualize your reading history.

Docker microservice & Crawler by scrapy

知名網站avgle下載器

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

A set of spiders and scrapers to extract location information from places that post their location on the internet.

豆瓣电影JSON数据

爬虫管理平台

🎊 Design and implement of lightweight crawler framework.

🚀电商网站爬虫合集，淘宝京东亚马逊等

163music spider by scrapy.

🕷️some python spiders with BeautifulSoup or scarpy

Extentions I made for PopClip.

百度贴吧爬虫(基于scrapy和mysql)

白熊咖啡馆的豆瓣广播

小密圈备份，圈子/话题/图片/文件。

The source code for animecenter

🎊 Design and implement of lightweight crawler framework.

A Web Crawler that scrapes using YAML and python code.

豆瓣电影-react版本

Simple and friendly Bot for Instagram, using Selenium and Scrapy with Python.

Uno Crawler por https://douban.com

A simple robot for Douban.com

A django admin site for scrapy

NScrapy is a .net core corss platform Distributed Spider Framework which provide an easy way to write your own Spider

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

IMDb + Rotten Tomatoes + Wikipedia on Douban Movie

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

Scrapy项目（mysql+mongodb豆瓣top250电影）

Generating spiders dynamically to crawl and check those free proxy ip on the internet with scrapy.

Python分布式爬虫打造搜索引擎

V2EX爬虫

👍 京东爬虫（大量注释，对刚入门爬虫者极度友好）

微信小程序-豆瓣电影

爬取拉勾、BOSS直聘、智联招聘、51job、赶集招聘、58招聘等职位信息

Build a Flask web application to help users retrieve key restaurant information and feature-based reviews (generated by applying market-basket model – Apriori algorithm and NLP on user reviews).

UFC bout winner prediction using neural nets.

PTT 圖片下載器 (抓取整個看板的圖片，並用文章標題作為資料夾的名稱 ) (使用Scrapy)

show full stack technology applications : Scrapy + webservice[restful] + websocket + VueJS + MongoDB

Use Scrapy crawl foreign exchange rate from BOC (Bank of China)

日常爬虫

A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

bing、google、baidu搜索引擎爬虫。python3.6 and scrapy

基于Scrapy+Elasticsearch+Django搭建的分布式电影搜索

简单、免费、高效的百度地图poi采集和分析工具。

a simple & tiny scrapy clustering solution, considered a drop-in replacement for scrapyd

虎扑步行街爬虫

收集各种免费的 Python 爬虫项目

With Selenium headless browsing and CAPTCHA solving

基于 mpvue 实现豆瓣电影微信小程序 @zce

Personal Data Engineering Projects

基于 scrapy-redis 的通用分布式爬虫框架

The Zipru scraper developed in the Advanced Web Scraping Tutorial.

swiss army knife for hackers

🍰 Everything is RSSible

Building a model to recognize incentives for landscape restoration in environmental policies from Latin America, the US and India. Bringing NLP to the world of policy analysis through an extensible framework that includes scraping, preprocessing, active learning and text analysis pipelines.

豆瓣租房小组多线程爬虫。爬取后自动按时间排序生成markdown文件。