1244 Open source projects that are alternatives of or similar to Awesome Java Crawler

📦 Currency mask module for Angular

Selenium Java Code for all selenium sessions - WebDriver, TestNG, POI, etc...

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

Typescript, React, Redux, Styled-Component and Webpack based sample extension boilerplate. Runs on Chrome and Firefox. Sample chrome extension.

Webtau (short for web test automation) is a testing API, command line tool and a framework to write unit, integration and end-to-end tests. Test across REST-API, Graph QL, Browser, Database, CLI and Business Logic with consistent set of matchers and concepts. REPL mode speeds-up tests development. Rich reporting cuts down investigation time.

🎬 电影资源爬虫,电影图片抓取脚本,Flask|Nginx|wsgi

An open source tab manager for Chrome

scrapy专利爬虫（停止维护）

Time Travel Debugger for React useReducer

爬取北大法宝网http://www.pkulaw.cn/Case/

🌐 Run headless Chrome/Chromium on AWS Lambda

一只优雅的正方教务系统爬虫。

一款分布式爬虫平台，帮助你更好的管理和开发爬虫。 内置一套爬虫定义规则（模版），可使用模版快速定义爬虫，也可当作框架手动开发爬虫。(兴趣使然的项目，用的不爽了就更新)

A Go package for working with headless Chrome. Run interactive JavaScript commands on web pages with Go and Chrome.

Github 仓库及用户分析爬虫

GraphQuery is a query language and execution engine tied to any backend service.

Selenium web driver library written in Go

🎨 An extension that can run Prettier in the browser

Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS 🎭

A Tumblr Backup Application

golang light-weight image crawler

🥄 A package for building specific Proxy Pool for different Sites.

News, full-text, and article metadata extraction in Python 3. Advanced docs:

Node.js scraper to get data from Google Play

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

Cross-platform persistent and distributed web crawler 🔗

Write Selenium and Appium tests in Python using the Page Object pattern. This Pythonic GUI and API test automation framework will help you get started with QA automation quickly. It comes with many useful integrations like - email, BrowserStack, Slack, TestRail, etc. This repository is developed and maintained by Qxf2 Services (https://qxf2.com).

Scrapy, a fast high-level web crawling & scraping framework for Python.

An easy to use, powerful crawler implemented in PHP. Can execute Javascript.

🎨 Color Pickers from Sketch, Photoshop, Chrome, Github, Twitter & more

Automated Chrome tracing for benchmarking.

🕶️ A curated list of resources around the topic: visual regression testing

A chrome extension for reloading unpacked extensions

Framework for writing maintainable Selenium and REST API tests in Java.

Elegant Scraper and Crawler Framework for Golang

A web crawler (for bug hunting) that gathers more than you can imagine.

WebDriver implementation for Qt

Exposed Unity Editor WebView API

A browser extension (Chrome, Firefox) that toggles concealment of sensitive information found in the Azure Portal web page such as Subscription Id's

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

End-to-end testing framework written in Node.js and using the Webdriver API

WikipediaP2P.org Chrome Extension

Dynamic meta tags in your AngularJS single page application

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

模拟登陆基本采用的是直接登录或者使用selenium+webdriver的方式，有的网站直接登录难度很大，比如qq空间，bilibili等如果采用selenium就相对轻松一些。

In this project we will discuss working Selenium with cucumber

A lite distributed Java spider framework :-)

Automating Web Performance testing with Puppeteer 🎪

It is a Chrome extension for printing JSON and JSONP.

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

Send bulk messages right from your WhatsApp Android Client or WhatsApp Web

A 1Password-compatible command-line and web-based password manager

Zebrunner Reporting Tool

Crawler for LinkedIn full profiles 2019

Chrome Extension boilerplate with ReactJS and vanilla JS examples

Better code navigation in github

A browser extension for Chrome, Opera & Firefox that adds Mermaid language support to Github

🎹🐻 Nook is a Chrome/Firefox extension that plays Animal Crossing hourly themes.