2403 Open source projects that are alternatives of or similar to Ppspider

a puppeteer walker 🕷 🕸

dynamic crawler for web vulnerability scanner

💐Marmot | Web Crawler/HTTP protocol Download Package 🐭

Python爬虫代理IP池(proxy pool)

🍿爬虫代理IP池(proxy pool) python🍟一个还ok的IP代理池

获取免费socks/https/http代理的网站集合

🥄 A package for building specific Proxy Pool for different Sites.

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

稳健高效的评分制-针对性- IP代理池 + API服务，可以自己插入采集器进行代理IP的爬取，针对你的爬虫的一个或多个目标网站分别生成有效的IP代理数据库，支持MongoDB 4.0 使用 Python3.7（Scored IP proxy pool ,customise proxy data crawler can be added anytime）

a reliable high-level web crawling & scraping framework for Node.js.

Selenium alternative for Crystal. Browser manipulation without the Java overhead.

APIs for loginning some websites by using requests.

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

A proper monster for front-end automated testing

[https://www.sofineday.com], golang项目开发脚手架,集成最佳实践(gin+gorm+go-redis+mongo+cors+jwt+json日志库zap(支持日志收集到kafka或mongo)+消息队列kafka+微信支付宝支付gopay+api加密+api反向代理+go modules依赖管理+headless爬虫chromedp+makefile+二进制压缩+livereload热加载)

yspider -- 轻量级爬虫系统

all kinds of scrapy demo

😱Full Automatic Amazon Distributed Spider | 亚马逊分布式四国际站采集选款产品|账号admin,密码adminadmin

中国大陆大学列表爬虫

dynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites

Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池

微博超级话题爬虫，微博词频统计+情感分析+简单分类，新增肺炎超话爬取数据

😴 Attends classes for you.

The web browser that's built for scraping.

一款分布式爬虫平台，帮助你更好的管理和开发爬虫。 内置一套爬虫定义规则（模版），可使用模版快速定义爬虫，也可当作框架手动开发爬虫。(兴趣使然的项目，用的不爽了就更新)

Web crawling framework based on asyncio.

A Laravel wrapper for CrawlerDetect - the web crawler detection library

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS 🎭

疫情数据爬虫，2019新型冠状病毒数据仓库，轨迹数据，同乘数据，报道

Sample of using proxies to crawl baidu search results.

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

简单易用的Python爬虫框架，QQ交流群：597510560

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

B站用户爬虫 好耶~是爬虫

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.

MM131网站图片爬取 🚨

Mailchimp Open Commerce is an API-first, headless commerce platform built using Node.js, React, GraphQL. Deployed via Docker and Kubernetes.

Digger is a powerful and flexible web crawler implemented by pure golang

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

A lite distributed Java spider framework :-)

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

Golang爬虫 爬取豆瓣电影Top250

链家网和贝壳网房价爬虫，采集北京上海广州深圳等21个中国主要城市的房价数据（小区，二手房，出租房，新房），稳定可靠快速！支持csv,MySQL, MongoDB,Excel, json存储，支持Python2和3，图表展示数据，注释丰富 ，点星支持，仅供学习参考，请勿用于商业用途，后果自负。

Crawl some picture for fun

A Lightweight High Performance ESMTP email server

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

book.zi5.me全站kindle电子书籍爬取，按照作者书籍名分类，每本书有mobi和equb两种格式，采用分布式进行全站爬取

A simple distributed crawler for zhihu && data analysis

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

多线程知乎用户爬虫，基于python3

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

Getting started with Puppeteer and Chrome Headless for Web Scraping

JavaScript + BeautifulSoup = JSSoup

Elegant Scraper and Crawler Framework for Golang

京东口罩库存监控爬虫(非selenium)，扫码登录、查价、加购、下单、秒杀

BaiduSpider，一个爬取百度搜索结果的爬虫，目前支持百度网页搜索，百度图片搜索，百度知道搜索，百度视频搜索，百度资讯搜索，百度文库搜索，百度经验搜索和百度百科搜索。

爬取北大法宝网http://www.pkulaw.cn/Case/

JS逆向研究

Headless Chrome For Java （Java 爬虫）