804 Open source projects that are alternatives of or similar to Zhihuspider

A php crawler that finds emails on the internets

拼多多爬虫，爬取所有商品、评论等信息

一个java版本的分布式的通用爬虫，可以插拔各个组件（提供默认的）

（已跑路）百度贴吧吧务管理工具，自动扫描帖子并处理违规帖

네이버 뉴스 수집을 위한 도구

根据关键词抓取微博数据，再生成词云

Xenomorph Crawler, a Concise, Declarative and Observable Distributed Crawler(Node / Go / Java / Rust) For Web, RDB, OS, also can act as a Monitor(with Prometheus) or ETL for Infrastructure 💫 多语言执行器，分布式爬虫

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉

用 node.js 爬你自己的 leetcode 解题源码

Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池

A Moodle downloader that downloads course content fast from Moodle (eg. lecture pdfs)

用爬虫爬取小说网站上所有小说，存储到数据库中，并用爬到的数据构建自己的小说网站

破解JS反爬虫加密参数，已破解中国裁判文书网（2020-06-30更新），淘宝密码，天安保险登录，b站登录，房天下登录，WPS登录，微博登录，有道翻译，网易登录，微信公众号登录，空中网登录，今目标登录，学生信息管理系统登录，共赢金融登录，重庆科技资源共享平台登录，网易云音乐下载，一键解析视频链接，财联社登录。

Dynamic meta tags in your AngularJS single page application

Sample of using proxies to crawl baidu search results.

房价预测完整项目：1.爬取链家网数据 2.处理后，用sklearn中几个逻辑回归机器学习模型和keras神经网络搭建模型预测房价 最终结果神经网络效果更好，R^2值0.75左右

😊😊😊 知乎问题爬虫

java framework for prerender

A React-based web video player

Viewers for statistics and dashboarding of Domain Search Engine data

Python爬虫和Flask实现小说网站

📢 Ptt 文章通知機器人！Notify Ptt Article in Realtime

🎬 电影资源爬虫,电影图片抓取脚本,Flask|Nginx|wsgi

2019新型冠状病毒疫情实时爬虫及API | COVID-19/2019-nCoV Realtime Infection Crawler and API

scrapy专利爬虫（停止维护）

滑动验证码，希望对你们有所帮助❤️

CoCrawler is a versatile web crawler built using modern tools and concurrency.

抖音API、抖音数据、抖音直播数据、抖音直播Api、抖音视频Api、抖音爬虫、抖音去水印、抖音视频下载、抖音视频解析、抖音直播监控、抖音数据采集

Unmaintained 🐳 ☕️ 🕷 Scala crawler(spider) framework, inspired by scrapy, created by @gaocegege

抓取豆瓣，bilibili等中的电视剧、电影、动漫演员等信息

📖 爬取 Stackoverflow 100万 条问答并简单分析

dynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites

Blazing fast admin panel finder with asyncio and aiohttp

A collection of awesome web scaper, crawler.

一只优雅的正方教务系统爬虫。

又一个 java 内容（pa）获取（chong）工具

owllook-小说搜索引擎

一些爬虫的代码

GraphQuery is a query language and execution engine tied to any backend service.

netease-music-spider is a sipder that you can find beautiful girlfriend or handsome boyfriend.

Node.js scraper to get data from Google Play

portia-dashboard is a visual web crawler based on scrapinghub/portia

Headless Chrome For Java （Java 爬虫）

🕷招聘网站爬虫合集，不定期更新分支

简单、易用、高效 一个有态度的开源.Net Http请求框架!可以用制作爬虫，api请求等等。

lots of spider (很多爬虫）

💻 Quickly crawl the information (e.g. followers, tags, etc...) of an instagram profile. No login required!

Touhou Project random music video generator/player, crawling image and video from websites to generate MV.

pylinkvalidator is a standalone and pure python link validator and crawler that traverses a web site and reports errors (e.g., 500 and 404 errors) encountered.

Cross-platform persistent and distributed web crawler 🔗

My Python Demo

Enjoy driving on a Javascriptive (originally Pythonic) way to Japanese AV!

An automated website accessibility scanner and cli

Fawkes is a tool to search for targets vulnerable to SQL Injection. Performs the search using Google search engine.

Go process used to crawl websites

Scrapy, a fast high-level web crawling & scraping framework for Python.

An image crawler written in Python.

😩Tool For Taobao/Tmall| 儿时玩具已经过时