1191 Open source projects that are alternatives of or similar to Squidwarc

Scrapy, a fast high-level web crawling & scraping framework for Python.

News, full-text, and article metadata extraction in Python 3. Advanced docs:

Chrome Devtools Protocol client for PHP

Simple but useful Python web scraping tutorial code.

A Headless Chrome rendering solution

Elegant Scraper and Crawler Framework for Golang

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

A minimal Docker image for Puppeteer

📖 Puppeteer中文文档（官方指定的中文文档）

HTML to PDF converter via Chrome/Chromium

A browser driver on top of puppeteer, ready for production scenarios.

Headless Chrome/Chromium driver for Capybara

web spider built by puppeteer, support task-queue and task-scheduling by decorators，support nedb / mongodb, support data visualization; 基于puppeteer的web爬虫框架，提供灵活的任务队列管理调度方案，提供便捷的数据保存方案（nedb/mongodb），提供数据可视化和用户交互的实现方案

本仓库收集整理爬虫相关资源，开发语言以Java为主

蓝天采集器是一款免费的数据采集发布爬虫软件，采用php+mysql开发，可部署在云服务器，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统

🖥 Chrome automation made simple. Runs locally or headless on AWS Lambda.

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

Use Puppeteer to download pens from Codepen.io as single html pages

Handle infinite scroll on websites by puppeteer

📷 BrowserBox - Remote isolated browser API for security, automation visibility and interactivity. Run on our cloud, or bring your own. Full scope double reverse web proxy with multi-tab, mobile-ready browser UI frontend. Plus co-browsing, advanced adaptive streaming, secure document viewing and more! But only in the Pro version. Get BB today! Se…

Handy API for Headless Chromium

Puppeteer (Headless Chrome) provider for Nest.js

Automating Web Performance testing with Puppeteer 🎪

Apify command-line interface helps you create, develop, build and run Apify actors, and manage the Apify cloud platform.

GitHub automation driven by headless chrome.

An interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL

Search for Slack Webhooks token publicly exposed on Github

Minimal container for Chrome's headless shell, useful for automating / driving the web

A bullet-proof, fast, and reliable headless browser API

☕️ TDD with Browserify, Mocha, Headless Chrome and WebDriver

Continuously building Chrome Docker image for Linux.

🔳 WhatsApp chat from commandline/console/cli using GoogleChrome puppeteer

Crawl a website and run it through Google lighthouse

Headless Chrome Ruby API

Crawly, a high-level web crawling & scraping framework for Elixir.

Puppeteer as a service

Web Extension for Firefox/Chrome/MS Edge and CLI tool to save a faithful copy of an entire web page in a single HTML file

Differencify is a library for visual regression testing

A powerful dynamic crawler for web vulnerability scanners

Chromy is a library for operating headless chrome. 🍺🍺🍺

A Puppeteer bridge for PHP, supporting the entire API.

Google Puppeteer 文档的中文版本 , 目标版本 1.9.0, 翻译中...

☕️ Run Mocha tests using headless Google Chrome

Run Puppeteer code in the cloud

HTTP API for Scrapy spiders

Run your tests using Jest & Puppeteer 🎪✨

Getting headless chrome running on heroku

📷 ViewFinder - NodeJS product to make the browser into a web app. WTF RBI. CBII. Remote browser isolation, embeddable browserview, secure chrome saas. Licenses, managed, self-hosted. Like S2, WebGap, Bromium, Authentic8, Menlo Security and Broadcom, but open source with free live demos available now! Also, integrated RBI/CDR with CDR from https://github.com/dosyago/p2%2e

Powerful web scraping framework for Crystal

Puppeteer Firebase Functions demo

Microservice to generate screenshot from a webpage and upload it to a AWS S3 Bucket.

📷 page2image is a npm package for taking screenshots which also provides CLI command

[Unmaintained] A simple and clean video/music/image downloader 👾

Headless Chrome as a Service

Chrome extension that records your browser interactions and generates a Playwright or Puppeteer script.

Web Screenshots at scale based on headless chrome

Chrome Headless docker images built upon alpine official image

🕸 Modular, multithreaded, puppeteer-based crawler

Private API to enable API access for Roam Research. Now you can connect Roam to your other projects.

A Dart library to automate the Chrome browser over the DevTools Protocol. This is a port of the Puppeteer API