zorlan / Skycaiji
蓝天采集器是一款免费的数据采集发布爬虫软件，采用php+mysql开发，可部署在云服务器，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统
Stars: ✭ 1,514
Programming Languages
Projects that are alternatives of or similar to Skycaiji
Linkedin Profile Scraper
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-88.71%)
Mutual labels: crawler, spider, crawling
Crawlab
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Stars: ✭ 8,392 (+454.29%)
Mutual labels: crawler, spider, webcrawler
Arachnid
Powerful web scraping framework for Crystal
Stars: ✭ 68 (-95.51%)
Mutual labels: crawler, spider, crawling
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+926.09%)
Mutual labels: crawler, spider, crawling
flink-crawler
Continuous scalable web crawler built on top of Flink and crawler-commons
Stars: ✭ 48 (-96.83%)
Mutual labels: crawler, spider, crawling
Webster
a reliable high-level web crawling & scraping framework for Node.js.
Stars: ✭ 364 (-75.96%)
Mutual labels: crawler, spider, crawling
Gopa
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Stars: ✭ 277 (-81.7%)
Mutual labels: crawler, spider, crawling
Crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (-70.94%)
Mutual labels: crawler, spider, crawling
Avbook
AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+437.19%)
Mutual labels: crawler, spider
Awesome Python Primer
自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向
Stars: ✭ 57 (-96.24%)
Mutual labels: crawler, spider
Beanbun
Beanbun 是用 PHP 编写的多进程网络爬虫框架，具有良好的开放性、高可扩展性，基于 Workerman。
Stars: ✭ 1,096 (-27.61%)
Mutual labels: crawler, spider
Photon
Incredibly fast crawler designed for OSINT.
Stars: ✭ 8,332 (+450.33%)
Mutual labels: crawler, spider
Nodespider
[DEPRECATED] Simple, flexible, delightful web crawler/spider package
Stars: ✭ 33 (-97.82%)
Mutual labels: crawler, spider
Gopa Abandoned
GOPA, a spider written in Go.（NOTE: this project moved to https://github.com/infinitbyte/gopa ）
Stars: ✭ 98 (-93.53%)
Mutual labels: crawler, spider
Crawler examples
Some classic web crawler projects.一些经典的爬虫
Stars: ✭ 74 (-95.11%)
Mutual labels: crawler, spider
将本压缩包上传至您的服务器，如果根目录有站点建议放在子目录里，解压后打开浏览器输入您的服务器域名或ip地址（存放在子目录则加上子目录的名称）进入安装界面。 使用手册：https://www.skycaiji.com/manual 安装协议：https://www.skycaiji.com/licenses
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].