1491 Open source projects that are alternatives of or similar to Puppeteer Walker

web spider built by puppeteer, support task-queue and task-scheduling by decorators，support nedb / mongodb, support data visualization; 基于puppeteer的web爬虫框架，提供灵活的任务队列管理调度方案，提供便捷的数据保存方案（nedb/mongodb），提供数据可视化和用户交互的实现方案

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

Web page PDF/PNG rendering done right. Self-hosted service for rendering receipts, invoices, or any content.

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

Serverless Framework + AWS Lambda Layers + Puppeteer = ❤️

Headless Chrome For Java （Java 爬虫）

利用chrome的headless模式，模拟用户注册进行短信轰炸机

dynamic crawler for web vulnerability scanner

Distributed crawler powered by Headless Chrome

📖 Puppeteer中文文档（官方指定的中文文档）

a reliable high-level web crawling & scraping framework for Node.js.

Puppeteer (Headless Chrome Node API)-based rendering solution.

Declarative web scraping

A collection of awesome web crawler,spider in different languages

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

🕸 Modular, multithreaded, puppeteer-based crawler

带你了解一下Golang的市场行情

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

Starter Kit for running Headless-Chrome by Puppeteer on AWS Lambda.

INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱🧰，旨在安全快捷的帮助用户拿回自己的数据，工具代码开源，流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。

HTML to PDF converter via Chrome/Chromium

Some classic web crawler projects.一些经典的爬虫

☕️ Run Mocha tests using headless Google Chrome

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.

🐾 Creeper - The Next Generation Crawler Framework (Go)

Powerful web scraping framework for Crystal

A Python Package for the Google Chrome Dev Protocol [threading base]

QQ空间导出助手，用于备份QQ空间的说说、日志、私密日记、相册、视频、留言板、QQ好友、收藏夹、分享、最近访客为文件，便于迁移与保存

A Facebook crawler

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

Python的基础练习代码与各种爬虫代码

Dark Web OSINT Tool

一个超级轻量的百度图片爬虫

Free Web Scraping Tool with Java

A multi-thread crawler framework with many builtin image crawlers provided.

API of DouYin for Humans used to Crawl Popular Videos and Musics

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

python crawler spider

Headless Chrome/Chromium driver for Capybara

Crawly, a high-level web crawling & scraping framework for Elixir.

Cross Platform C# Web crawler framework, headless browser, parallel crawler. Please star this project! +1.

Scraping scripts for various websites.

Gospider - Fast web spider written in Go

A bullet-proof, fast, and reliable headless browser API

[DEPRECATED] Simple, flexible, delightful web crawler/spider package

Plugin framework for PuppeteerSharp

Rust Web Crawler saving pages on Redis

🦊 Node.js API to control Firefox

Crawl websites for accessibility issues from the command line.

🔍 gowitness - a golang, web screenshot utility using Chrome Headless

💐 Full Amazon Automatic Download

Incredibly fast crawler designed for OSINT.

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

Puppeteer, Headless Chrome；爬取《es6标准入门》、自动推文到掘金、站点性能分析；高级爬虫、自动化UI测试、性能分析；

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

网站签到脚本