1306 Open source projects that are alternatives of or similar to Webster

Distributed crawler powered by Headless Chrome

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

dynamic crawler for web vulnerability scanner

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

A curated list of awesome puppeteer resources.

a puppeteer walker 🕷 🕸

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

🎪 End-to-end testing made simple (using Jest and Puppeteer)

Puppeteer as a service hosted on Saasify.

Elegant Scraper and Crawler Framework for Golang

蓝天采集器是一款免费的数据采集发布爬虫软件，采用php+mysql开发，可部署在云服务器，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统

No description or website provided.

Crawly, a high-level web crawling & scraping framework for Elixir.

A powerful dynamic crawler for web vulnerability scanners

Headless Chromium-based web performance metrics collector and monitoring tool

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Powerful web scraping framework for Crystal

web spider built by puppeteer, support task-queue and task-scheduling by decorators，support nedb / mongodb, support data visualization; 基于puppeteer的web爬虫框架，提供灵活的任务队列管理调度方案，提供便捷的数据保存方案（nedb/mongodb），提供数据可视化和用户交互的实现方案

Continuous scalable web crawler built on top of Flink and crawler-commons

Webrecorders DevTools Protocol Automation Library

[DEPRECATED] CLI for automated tests in web projects.

🔎Search LinkedIn profile by email address📧

知乎模拟登录，支持提取验证码和保存 Cookies

A Docker container running headless Chrome

Puppeteer (Headless Chrome) provider for Nest.js

☕️ TDD with Browserify, Mocha, Headless Chrome and WebDriver

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

基于Electron爬虫程序

Handle infinite scroll on websites by puppeteer

Use Puppeteer to download pens from Codepen.io as single html pages

Instagram automation driven by headless chrome.

Quickstart for Express on Nanobox

A web spider framework

Apify command-line interface helps you create, develop, build and run Apify actors, and manage the Apify cloud platform.

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

A browser extension that generates Cypress, Playwright and Puppeteer test scripts from your interactions 🖱 ⌨

Handy API for Headless Chromium

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

FlareSolverr .Net / Proxy server to bypass Cloudflare protection

Email automation driven by headless chrome.

Every web site provides APIs.

talospider - A simple,lightweight scraping micro-framework

Search for Slack Webhooks token publicly exposed on Github

A simple and flexible web crawler framework for java.

Template-based Web Application Framework

⚠️ 🚧 Install precompiled versions of the Chromium/Chrome headless shell using npm or yarn

A binding to puppeteer to drive headless Chrome.

An interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL

GitHub automation driven by headless chrome.

一个轻量级、快速、多线程、多管道、灵活配置的网络爬虫。

译文：Puppeteer 与 Chrome Headless —— 从入门到爬虫

Convert HTML to PDF Server by headless chrome with TypeScript. The new version of hcep-pdf-server.

🍰 一个可视化的爬虫平台

🐛一只登录正方教务管理系统，爬取数据的小爬虫

快速、简洁且强大的PHP爬虫框架

A fast tool to fetch URLs from HTML attributes by crawl-in.

Man in the middle using Playwright

The simple, easy to use command line web crawler.

微信公众号爬虫，公众号历史文章，文章评论，文章阅读及在看数据，可视化web页面，可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现，高效微信爬虫，微信公众号爬虫，历史文章，文章评论，数据更新。

High performance crawler for Nodejs