All Projects → go-rod → Rod

go-rod / Rod

Licence: mit
A Devtools driver for web automation and scraping

Programming Languages

go
31211 projects - #10 most used programming language
golang
3204 projects

Projects that are alternatives of or similar to Rod

Cdp4j
cdp4j - Chrome DevTools Protocol for Java
Stars: ✭ 232 (-83.33%)
Mutual labels:  automation, chrome-devtools, chrome-headless
Sillynium
Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (-92.82%)
Mutual labels:  automation, scraper, web-scraping
Autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+192.89%)
Mutual labels:  automation, scraper, web-scraping
Chrome Devtools Protocol
Chrome Devtools Protocol client for PHP
Stars: ✭ 112 (-91.95%)
Mutual labels:  automation, headless, chrome-devtools
Go Chrome
A golang library for interacting with the Chrome DevTools Protocol. https://chromedevtools.github.io/devtools-protocol/
Stars: ✭ 96 (-93.1%)
Mutual labels:  devtools, headless, chrome-devtools
Webdriverio
Next-gen browser and mobile automation test framework for Node.js
Stars: ✭ 7,214 (+418.25%)
Mutual labels:  automation, devtools
Triflejs
Headless automation for Internet Explorer
Stars: ✭ 815 (-41.45%)
Mutual labels:  automation, headless
Huginn
Create agents that monitor and act on your behalf. Your agents are standing by!
Stars: ✭ 33,694 (+2320.55%)
Mutual labels:  automation, scraper
Gowitness
🔍 gowitness - a golang, web screenshot utility using Chrome Headless
Stars: ✭ 996 (-28.45%)
Mutual labels:  headless, chrome-headless
Spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (-52.87%)
Mutual labels:  scraper, web-scraping
Navalia
A bullet-proof, fast, and reliable headless browser API
Stars: ✭ 950 (-31.75%)
Mutual labels:  automation, headless
Ferrum
Headless Chrome Ruby API
Stars: ✭ 1,009 (-27.51%)
Mutual labels:  automation, headless
Chromedp
A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.
Stars: ✭ 7,057 (+406.97%)
Mutual labels:  headless, chrome-devtools
Phantomjs
Scriptable Headless Browser
Stars: ✭ 28,654 (+1958.48%)
Mutual labels:  automation, headless
Puppeteer Api Zh cn
📖 Puppeteer中文文档(官方指定的中文文档)
Stars: ✭ 697 (-49.93%)
Mutual labels:  automation, headless
Cljs Devtools
A collection of Chrome DevTools enhancements for ClojureScript developers
Stars: ✭ 969 (-30.39%)
Mutual labels:  devtools, chrome-devtools
Spam Bot 3000
Social media research and promotion, semi-autonomous CLI bot
Stars: ✭ 79 (-94.32%)
Mutual labels:  automation, scraper
Artemis Dev Tool
An Apollo GraphQL Query Schema Testing Tool
Stars: ✭ 66 (-95.26%)
Mutual labels:  devtools, chrome-devtools
Puppeteer Webperf
Automating Web Performance testing with Puppeteer 🎪
Stars: ✭ 1,392 (+0%)
Mutual labels:  devtools, chrome-devtools
Rpa
UI.Vision: Open-Source RPA Software (formerly Kantu) - Modern Robotic Process Automation with Selenium IDE++
Stars: ✭ 477 (-65.73%)
Mutual labels:  automation, web-scraping

Overview

Go Reference Discord Chat

Documentation | API reference

Rod is a high-level driver directly based on DevTools Protocol. It's designed for web automation and scraping. Rod is designed for both high-level and low-level use, senior programmers can use the low-level packages and functions to easily customize or build up their own version of Rod, the high-level functions are just examples to build a default version of Rod.

Features

  • Chained context design, intuitive to timeout or cancel the long-running task
  • Auto-wait elements to be ready
  • Debugging friendly, auto input tracing, remote monitoring headless browser
  • Thread-safe for all operations
  • Automatically find or download browser
  • Lightweight, no third-party dependencies, CI tested on Linux, Mac, and Windows
  • High-level helpers like WaitStable, WaitRequestIdle, HijackRequests, WaitDownload, etc
  • Two-step WaitEvent design, never miss an event (how it works)
  • Correctly handles nested iframes or shadow DOMs
  • No zombie browser process after the crash (how it works)

Examples

Please check the examples_test.go file first, then check the examples folder.

For more detailed examples, please search the unit tests. Such as the usage of method HandleAuth, you can search all the *_test.go files that contain HandleAuth, for example, use Github online search in repository. You can also search the GitHub issues, they contain a lot of usage examples too.

Here is a comparison of the examples between rod and Chromedp.

If you have questions, please raise an issue or join the chat room.

How it works

Here's the common start process of rod:

  1. Try to connect to a Devtools endpoint (WebSocket), if not found try to launch a local browser, if still not found try to download one, then connect again. The lib to handle it is launcher.

  2. Use the JSON-RPC to talk to the Devtools endpoint to control the browser. The lib handles it is cdp.

  3. Use the type definitions of the JSON-RPC to perform high-level actions. The lib handles it is proto.

Object model:

object model

Join us

Your help is more than welcome! Even just open an issue to ask a question may greatly help others.

Please read How To Ask Questions The Smart Way before you ask questions.

We use Github Projects to manage tasks, you can see the priority and progress of the issues here.

If you want to contribute code for this project please read the Contributor Guide.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].