All Projects → berstend → Puppeteer Extra

berstend / Puppeteer Extra

Licence: mit
💯 Teach puppeteer new tricks through plugins.

Programming Languages

javascript
184084 projects - #8 most used programming language
typescript
32286 projects

Projects that are alternatives of or similar to Puppeteer Extra

Puppeteer Sharp Extra
Plugin framework for PuppeteerSharp
Stars: ✭ 39 (-98.85%)
Mutual labels:  recaptcha, chrome, puppeteer, headless-chrome
Puphpeteer
A Puppeteer bridge for PHP, supporting the entire API.
Stars: ✭ 1,014 (-70.15%)
Mutual labels:  automation, puppeteer, headless-chrome
Navalia
A bullet-proof, fast, and reliable headless browser API
Stars: ✭ 950 (-72.03%)
Mutual labels:  automation, chrome, headless-chrome
Puppeteer Deep
Puppeteer, Headless Chrome;爬取《es6标准入门》、自动推文到掘金、站点性能分析;高级爬虫、自动化UI测试、性能分析;
Stars: ✭ 1,033 (-69.59%)
Mutual labels:  chrome, puppeteer, headless-chrome
Puppeteer Lambda Starter Kit
Starter Kit for running Headless-Chrome by Puppeteer on AWS Lambda.
Stars: ✭ 563 (-83.43%)
Mutual labels:  chrome, puppeteer, headless-chrome
Puppeteer Api Zh cn
📖 Puppeteer中文文档(官方指定的中文文档)
Stars: ✭ 697 (-79.48%)
Mutual labels:  automation, chrome, puppeteer
Ferrum
Headless Chrome Ruby API
Stars: ✭ 1,009 (-70.3%)
Mutual labels:  automation, chrome, headless-chrome
Api
API that uncovers the technologies used on websites and generates thumbnail from screenshot of website
Stars: ✭ 189 (-94.44%)
Mutual labels:  automation, chrome, headless-chrome
Phantomas
Headless Chromium-based web performance metrics collector and monitoring tool
Stars: ✭ 2,191 (-35.5%)
Mutual labels:  automation, puppeteer, headless-chrome
Awesome Puppeteer
A curated list of awesome puppeteer resources.
Stars: ✭ 1,728 (-49.13%)
Mutual labels:  automation, puppeteer, headless-chrome
Headless Chrome Crawler
Distributed crawler powered by Headless Chrome
Stars: ✭ 5,129 (+50.99%)
Mutual labels:  chrome, puppeteer, headless-chrome
Element
💦Load test your app using real web browsers
Stars: ✭ 204 (-93.99%)
Mutual labels:  automation, chrome, puppeteer
Apify Js
Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
Stars: ✭ 3,154 (-7.15%)
Mutual labels:  automation, puppeteer, headless-chrome
Url To Pdf Api
Web page PDF/PNG rendering done right. Self-hosted service for rendering receipts, invoices, or any content.
Stars: ✭ 6,544 (+92.64%)
Mutual labels:  chrome, puppeteer, headless-chrome
mono.ts
A minimal example of a typescript mono repo with pre-build support (tests, vscode).
Stars: ✭ 58 (-98.29%)
Mutual labels:  lerna, monorepo, yarn-workspaces
vscode-monorepo-workspace
📦✨Manage monorepos with multi-root workspaces. Supports Lerna, Yarn, Pnpm, Rushjs and recursive package directories.
Stars: ✭ 93 (-97.26%)
Mutual labels:  lerna, monorepo, yarn-workspaces
cra-monorepo-demo
Monorepo example using create-react-app and common component library structure with yarn workspaces
Stars: ✭ 37 (-98.91%)
Mutual labels:  lerna, monorepo, yarn-workspaces
Chrome Devtools Protocol
Chrome Devtools Protocol client for PHP
Stars: ✭ 112 (-96.7%)
Mutual labels:  automation, chrome, headless-chrome
Squidwarc
Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
Stars: ✭ 125 (-96.32%)
Mutual labels:  chrome, puppeteer, headless-chrome
Deno Puppeteer
A port of puppeteer running on Deno
Stars: ✭ 128 (-96.23%)
Mutual labels:  automation, puppeteer, headless-chrome

puppeteer-extra

This is the monorepo for puppeteer-extra, a modular plugin framework for puppeteer. :-)

🌟 For the main documentation, please head over to the puppeteer-extra package.

In case you're interested in the available plugins, check out the packages folder.

Monorepo

Contributing

Contributing

PRs and new plugins are welcome! The plugin API for puppeteer-extra is clean and fun to use. Have a look the PuppeteerExtraPlugin base class documentation to get going and check out the existing plugins (minimal example is the anonymize-ua plugin) for reference.

We use a monorepo powered by Lerna (and yarn workspaces), ava for testing, the standard style for linting and JSDoc heavily to auto-generate markdown documentation based on code. :-)

Lerna

Lerna

This monorepo is powered by Lerna and yarn workspaces.

Initial setup

# Install deps
yarn

# Bootstrap the packages in the current Lerna repo.
# Installs all of their dependencies and links any cross-dependencies.
yarn bootstrap

# Build all TypeScript sources
yarn build

Development flow

# Install debug in all packages
yarn lerna add debug

# Install fs-extra to puppeteer-extra-plugin-user-data-dir
yarn lerna add fs-extra --scope=puppeteer-extra-plugin-user-data-dir

# Remove dependency
# https://github.com/lerna/lerna/issues/833
yarn lerna exec --concurrency 1 'yarn remove fs-extra; echo 0'

# Run test in all packages
yarn test

# Update JSDoc based documentation in markdown files
yarn docs

# Upgrade project wide deps like puppeteer
# (We keep the devDependency version blurry)
rm -rf node_modules
rm -rf yarn.lock
yarn
yarn lerna bootstrap

# Update deps within packages (interactive)
yarn lernaupdate

# If in doubt :-(
yarn lerna exec "rm -f yarn.lock; rm -rf node_modules; echo 0"
rm -f yarn.lock &&  rm -rf node_modules && yarn cache clean


# Test a local monorepo package in an outside folder as it would've been installed from the registry
# Change PACKAGE_DIR to the path of this monorepo and PACKAGE to the package you wish to install
PACKAGE=puppeteer-extra PACKAGE_DIR=/Users/foo/puppeteer-extra/packages && yarn remove $(echo $PACKAGE); true && rm -f $(pwd)/$(echo $PACKAGE)-latest.tgz && yarn --cwd $(echo $PACKAGE_DIR)/$(echo $PACKAGE) pack --filename $(pwd)/$(echo $PACKAGE)-latest.tgz && YARN_CACHE_FOLDER=/tmp/yarn yarn add file:$(pwd)/$(echo $PACKAGE)-latest.tgz && rm -rf /tmp/yarn

Publishing

# make sure you're signed into npm before publishing
# yarn publishing is broken so lerna uses npm
npm whoami

# ensure everything is up2date and peachy
yarn
yarn bootstrap
yarn lerna link
yarn build
yarn test

# Phew, let's publish these packages!
# - Will publish all changed packages
# - Will ask for new pkg version per package
# - Will updated inter-package dependency versions automatically
yarn lerna publish

# Fix new dependency version symlinks
yarn bootstrap && yarn lerna link

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].