2588 Open source projects that are alternatives of or similar to Headless Chrome Crawler

Easily crawl news portals or blog sites using Storm Crawler.

Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other things like capture a screenshot, generate pdf, extract content or execute custom Puppeteer, Playwright functions.

Open bank data for Leumi bank and Leumi card credit card

Instagram automation driven by headless chrome.

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

Handle infinite scroll on websites by puppeteer

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

No description or website provided.

Handy API for Headless Chromium

Puppeteer (Headless Chrome) provider for Nest.js

⚠️ 🚧 Install precompiled versions of the Chromium/Chrome headless shell using npm or yarn

A browser extension that generates Cypress, Playwright and Puppeteer test scripts from your interactions 🖱 ⌨

Web service and CLI tool for SEO site audit: crawl site, lighthouse all pages, view public reports in browser. Also output to console, json, csv, xlsx, Google Drive.

Rubium is a lightweight alternative to Selenium/Capybara/Watir if you need to perform some operations (like web scraping) using Headless Chromium and Ruby

Web crawling and scraping framework for Golang

[DEPRECATED] CLI for automated tests in web projects.

A PHP Scraping Class

Command line program to download documents from web portals.

FlareSolverr .Net / Proxy server to bypass Cloudflare protection

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

Scrapes nft floor prices and additional information from opensea. Used for https://nftfloorprice.info

Crawl sites for RSS, Atom, and JSON feeds.

Email automation driven by headless chrome.

🕸 List of mini projects that involve web scraping 🕸

All-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha and Anticaptcha API's!

📉 Uptime and error monitoring CLI

A Python Package for the Google Chrome Dev Protocol [threading base]

A binding to puppeteer to drive headless Chrome.

Screen scraping and web crawling framework

Search for Slack Webhooks token publicly exposed on Github

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

An interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL

A Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)

High performance crawler for Nodejs

利用chrome的headless模式，模拟用户注册进行短信轰炸机

A framework for creating semi-automatic web content extractors

Continuous scalable web crawler built on top of Flink and crawler-commons

A Scraper, Downloader, & Recorder for static open directories.

Crawler for Nintendo Switch eShop

Az összes számlád még egy helyen :)

译文：Puppeteer 与 Chrome Headless —— 从入门到爬虫

.NET version of the Playwright testing and automation library.

A Facebook crawler

The simplest demo of chrome automation by python and selenium in AWS Lambda

A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.

GitHub automation driven by headless chrome.

Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.

🔒 Chromium's HSTS preload list submission website.

QQ空间导出助手，用于备份QQ空间的说说、日志、私密日记、相册、视频、留言板、QQ好友、收藏夹、分享、最近访客为文件，便于迁移与保存

Simple Weibo Scraper

A script made to buy any out-of-stock product off spanish stores

我的爬虫合集

🍭 epub generator for (light)novels (轻) 小说 epub 生成器，支持站点：轻之国度、轻小说文库

A collection of tools to (semi-)automatically collect and analyze data from online discussions on Facebook groups and pages.

Man in the middle using Playwright

The simple, easy to use command line web crawler.

Web Rendering Proxy: Use vintage, historical, legacy browsers on modern web

LinkedIn Scraper (currently working 2020)

Convert HTML to PDF Server by headless chrome with TypeScript. The new version of hcep-pdf-server.