288 Open source projects that are alternatives of or similar to ancient_chinese

Using scrapy to scrape tripadvisor in order to get users' reviews.

码云仓库链接:AJay13/ECommerceCrawlers Github 仓库链接:DropsDevopsOrg/ECommerceCrawlers 项目展示平台链接:http://wechat.doonsec.com

A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

Dynamic configurable crawl (动态可配置化爬虫)

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

Scrapy downloader middleware that stores response HTMLs to disk.

World’s single largest Internet domains dataset

A Scrapy extension to log items coverage when the spider shuts down

《Python爬虫开发 从入门到实战》配套源代码。

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

Iota is a web scraper which can find all of the images and links/suburls on a webpage

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

Inventus is a spider designed to find subdomains of a specific domain by crawling it and any subdomains it discovers.

Send reddit directly to your kindle

Japanese - English dictionary for Kindle based on the JMdict / EDICT database

python+scrapy+splash 爬取拉勾全站职位信息

Classificador de poemas do Fernando Pessoa de acordo com os seus heterônimos

腾讯新闻、知乎话题、微博粉丝，Tumblr爬虫、斗鱼弹幕、妹子图爬虫、分布式设计等

A Scrapy middleware to bypass the CloudFlare's anti-bot protection

Scrapy+Splash for JavaScript integration

新浪博客文章/wenku8轻小说文库爬虫，可抓取图片保存，一键制作电子书。kindle读书党的神器！

crawler framework, distributed crawler extractor

Raspagem de dados para iniciante usando Scrapy e outras libs básicas

Github 仓库及用户分析爬虫

Araignée LeBonCoin avec Scrapy et ElasticSearch

使用asyncio和aiohttp开发的轻量级异步协程web爬虫框架

直播网站数据采集

With Selenium headless browsing and CAPTCHA solving

Export Vocabulary Builder to Anki APKG

日常爬虫

python based software to unpack kindlegen generated ebooks

scrapy mysql pipeline

A scrapy middleware to use rotated proxy ip list.

show full stack technology applications : Scrapy + webservice[restful] + websocket + VueJS + MongoDB

Analyze scraped data

A calibre plugin that converts your comics into a readable format for kindle.

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

bing、google、baidu搜索引擎爬虫。python3.6 and scrapy

Scraping the real estate agencies for up-to-date house listings as soon as they arrive!

Common interface for data container classes

招聘网数据爬虫

爬取拉勾、BOSS直聘、智联招聘、51job、赶集招聘、58招聘等职位信息

Dynamic file detection tool based on crawler 基于爬虫的动态敏感文件探测工具

Crawling zhihu, jobbole, lagou by Scrapy, and using Elasticsearch+Django to build a Search Engine website --- README_zh.md (including: implementation roadmap, distributed-crawler and coping with anti-crawling strategies).

admin ui for scrapy/open source scrapinghub

简单、免费、高效的百度地图poi采集和分析工具。

Scrape, standardize and share public meetings from local government websites

Command line tool to access your kindle clippings

抖音、快手、火山、皮皮虾，视频去水印程序

公众号文章抓取&生成kindle电子书

基于python语言开发的千万级别搜索引擎

My code for scraping financial data in Vietnam

新闻抓取（微信、微博、头条...）

虎扑步行街爬虫

Crawling the data from lazada, websosanh, compare.vn, cdiscount and cungmua with flexible configs

Use your Kindle KT3 (8. generation) for a weather station with weather underground api and homematic weather sensor.

UFC bout winner prediction using neural nets.

Descarga manga en español en diferentes formatos (PNG, PDF, EPUB, MOBI)

Distributed crawling/scraping, Kafka And Redis based components for Scrapy

python爬虫项目集合