2136 Open source projects that are alternatives of or similar to Apify Js

A curated list of awesome puppeteer resources.

Distributed crawler powered by Headless Chrome

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

Apify command-line interface helps you create, develop, build and run Apify actors, and manage the Apify cloud platform.

Headless Chromium-based web performance metrics collector and monitoring tool

Puppeteer as a service hosted on Saasify.

Use Puppeteer to download pens from Codepen.io as single html pages

Web Scraping Framework

💯 Teach puppeteer new tricks through plugins.

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

No description or website provided.

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

Contains all the public APIs listed in Phantombuster's API store. Pull requests welcome!

📷 page2image is a npm package for taking screenshots which also provides CLI command

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

⚡️ Ayakashi.io - The next generation web scraping framework

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

a reliable high-level web crawling & scraping framework for Node.js.

A Puppeteer bridge for PHP, supporting the entire API.

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

A port of puppeteer running on Deno

An open source web crawling platform

Generate a network graph of connections from your WhatsApp groups data

SimFin's open source PDF crawler

Module for using Headless-Chrome by Puppeteer on AWS Lambda.

Headless Chrome Node.js API

A node.js library for testing modern web applications

This is a Puppeteer+AngleSharp crawler console app samples, used C# 7.1 coding and dotnet core build.

cdp4j - Chrome DevTools Protocol for Java

Serverless selenium which dynamically execute any given code.

AI-powered Smart Robotic Process Automation 🤖

Basic setup with random user agents and IP addresses for Python Scrapy Framework.

Tutorial for web scraping / crawling with Node.js.

The complete web scraping toolkit for PHP.

You can use this act to monitor any page's content and get a notification when content changes.

👨‍👩‍👦 Social account detection and extraction in Python, e.g. for crawling/scraping.

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments

A Scrapy extension to log items coverage when the spider shuts down

Convert HTML to PDF Server by headless chrome with TypeScript. The new version of hcep-pdf-server.

Easily crawl news portals or blog sites using Storm Crawler.

Handle infinite scroll on websites by puppeteer

A library to read a YML file with Xpath or CSS Selectors and extract data from HTML pages using them

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

[DEPRECATED] CLI for automated tests in web projects.

🔎Search LinkedIn profile by email address📧

🕵‍♂ Bot detection tests for Puppeteer. Hide and seek!

Instagram automation driven by headless chrome.

ProxyCrawl Python library for scraping and crawling

Puppeteer (Headless Chrome) provider for Nest.js

🎪 End-to-end testing made simple (using Jest and Puppeteer)

Handy API for Headless Chromium

Web crawling and scraping framework for Golang

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

译文：Puppeteer 与 Chrome Headless —— 从入门到爬虫

Crawl sites for RSS, Atom, and JSON feeds.

Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other things like capture a screenshot, generate pdf, extract content or execute custom Puppeteer, Playwright functions.

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.