browser-poolA Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.
Stars: ✭ 71 (+195.83%)
bots-zooNo description or website provided.
Stars: ✭ 59 (+145.83%)
mugshotFramework independent visual testing library
Stars: ✭ 126 (+425%)
Gazpacho🥫 The simple, fast, and modern web scraping library
Stars: ✭ 525 (+2087.5%)
Tinking🧶 Extract data from any website without code, just clicks.
Stars: ✭ 331 (+1279.17%)
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (+612.5%)
Secret AgentThe web browser that's built for scraping.
Stars: ✭ 151 (+529.17%)
screenshotA screenshot API to convert web to image or PDF. Supports desktop and mobile views.
Stars: ✭ 108 (+350%)
pappetA command-line tool to crawl websites using puppeteer.
Stars: ✭ 95 (+295.83%)
Chromdaλ 🖼️ Chromda is an AWS Lambda function for capturing screenshots of websites.
Stars: ✭ 481 (+1904.17%)
anime-scraper[partially working] Scrape and add anime episode stream URLs to uGet (Linux) or IDM (Windows) ~ Python3
Stars: ✭ 21 (-12.5%)
chesfCHeSF is the Chrome Headless Scraping Framework, a very very alpha code to scrape javascript intensive web pages
Stars: ✭ 18 (-25%)
Apify JsApify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
Stars: ✭ 3,154 (+13041.67%)
AutoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+16887.5%)
DotnetcrawlerDotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c
Stars: ✭ 100 (+316.67%)
Awesome PuppeteerA curated list of awesome puppeteer resources.
Stars: ✭ 1,728 (+7100%)
puppeteer-botcheck🕵♂ Bot detection tests for Puppeteer. Hide and seek!
Stars: ✭ 42 (+75%)
PuppetronPuppeteer (Headless Chrome Node API)-based rendering solution.
Stars: ✭ 429 (+1687.5%)
Page2image📷 page2image is a npm package for taking screenshots which also provides CLI command
Stars: ✭ 66 (+175%)
Puppeteer DartA Dart library to automate the Chrome browser over the DevTools Protocol. This is a port of the Puppeteer API
Stars: ✭ 92 (+283.33%)
Site ScanCLI for capturing website screenshots, powered by puppeteer.
Stars: ✭ 137 (+470.83%)
puppet-masterPuppeteer as a service hosted on Saasify.
Stars: ✭ 25 (+4.17%)
TrackPurchase단 몇줄의 코드로 다양한 쇼핑 플랫폼에서 결제 내역을 긁어오자!
Stars: ✭ 19 (-20.83%)
playwright-demosplaywright for scrapping and UI testing / automate testing workflows
Stars: ✭ 65 (+170.83%)
double-agentA test suite of common scraper detection techniques. See how detectable your scraper stack is.
Stars: ✭ 123 (+412.5%)
naos📉 Uptime and error monitoring CLI
Stars: ✭ 30 (+25%)
PyppeteerHeadless chrome/chromium automation library (unofficial port of puppeteer)
Stars: ✭ 3,480 (+14400%)
ARGUSARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9
Stars: ✭ 68 (+183.33%)
Email ExtractorThe main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Stars: ✭ 81 (+237.5%)
Educative.io Downloader📖 This tool is to download course from educative.io for offline usage. It uses your login credentials and download the course.
Stars: ✭ 139 (+479.17%)
ConfigsPublic, free to use, repository with diggers configs for scraping / extracting data from various e-commerce websites and online stores
Stars: ✭ 37 (+54.17%)
jest-puppe-shotsA Jest plugin for creating screenshots of React components with a little help of Puppeteer
Stars: ✭ 86 (+258.33%)
ThalGetting started with Puppeteer and Chrome Headless for Web Scraping
Stars: ✭ 2,345 (+9670.83%)
screenie-serverA Node server with a pool of Puppeteer (Chrome headless) instances for scalable screenshot generation.
Stars: ✭ 19 (-20.83%)
test-real-styles(test-)framework agnostic utilities to test real styling of (virtual) dom elements
Stars: ✭ 37 (+54.17%)
Docker Puppeteerdocker image with Google Puppeteer installed
Stars: ✭ 415 (+1629.17%)
SinglefileWeb Extension for Firefox/Chrome/MS Edge and CLI tool to save a faithful copy of an entire web page in a single HTML file
Stars: ✭ 4,417 (+18304.17%)
StorycapA Storybook Addon, Save the screenshot image of your stories 📷 via puppeteer.
Stars: ✭ 451 (+1779.17%)
Webshot FactoryWeb Screenshots at scale based on headless chrome
Stars: ✭ 288 (+1100%)
Dark Mode ScreenshotThis Puppeteer script takes a 📷 screenshot of a webpage in 🌞 Light and 🌒 Dark Mode.
Stars: ✭ 47 (+95.83%)
Chart To AwsMicroservice to generate screenshot from a webpage and upload it to a AWS S3 Bucket.
Stars: ✭ 43 (+79.17%)
Lancia网页转PDF渲染服务。提供收据、发票、报告或任何网页内容转PDF的微服务
Stars: ✭ 108 (+350%)
puppeteer-screenshot-testerSmall library that allows us to compare screenshots generated by puppeteer in our tests.
Stars: ✭ 50 (+108.33%)
DhalangGenerate PDFs and make screenshots of HTML using Puppeteer in Ruby
Stars: ✭ 41 (+70.83%)
Whatsapp-NetGenerate a network graph of connections from your WhatsApp groups data
Stars: ✭ 75 (+212.5%)
Viewfinder📷 BrowserBox - Remote isolated browser API for security, automation visibility and interactivity. Run on our cloud, or bring your own. Full scope double reverse web proxy with multi-tab, mobile-ready browser UI frontend. Plus co-browsing, advanced adaptive streaming, secure document viewing and more! But only in the Pro version. Get BB today! Se…
Stars: ✭ 1,741 (+7154.17%)
aws-pdf-textract-pipeline🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
Stars: ✭ 141 (+487.5%)
Headless RecorderChrome extension that records your browser interactions and generates a Playwright or Puppeteer script.
Stars: ✭ 13,786 (+57341.67%)
Qawolf🐺 Create browser tests 10x faster
Stars: ✭ 2,912 (+12033.33%)
vrt-reactTake a screenshot 📸 of React component. Push it and compare images in pull request.
Stars: ✭ 19 (-20.83%)
SerpscrapSEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.
Stars: ✭ 153 (+537.5%)
simplechromeWebrecorders DevTools Protocol Automation Library
Stars: ✭ 16 (-33.33%)
iowebWeb Scraping Framework
Stars: ✭ 31 (+29.17%)