WebmagicA scalable web crawler framework for Java.
Stars: ✭ 10,186 (+5405.95%)
Frameworkium CoreFramework for writing maintainable Selenium and REST API tests in Java.
Stars: ✭ 107 (-42.16%)
HeadlessCreate a virtual X screen from Ruby, record videos and take screenshots.
Stars: ✭ 951 (+414.05%)
Selenium extensionsTools that will make writing tests, bots and scrapers using Selenium much easier
Stars: ✭ 140 (-24.32%)
Zhihu ApiZhihu API for Humans
Stars: ✭ 911 (+392.43%)
Tradingview TrainerA lightweight app for practicing your trading on Tradingview
Stars: ✭ 106 (-42.7%)
Tumblr CrawlerEasily download all the photos/videos from tumblr blogs. 下载指定的 Tumblr 博客中的图片,视频
Stars: ✭ 1,118 (+504.32%)
SinglefilezWeb Extension for Firefox/Chrome/MS Edge and CLI tool to save a faithful copy of an entire web page in a self-extracting HTML/ZIP polyglot file
Stars: ✭ 882 (+376.76%)
Crawler爬虫, http代理, 模拟登陆!
Stars: ✭ 106 (-42.7%)
PypergrabberFetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.
Stars: ✭ 14 (-92.43%)
SearchAn Open Source Search Engine
Stars: ✭ 139 (-24.86%)
Crawlselenium异步爬取网页图片
Stars: ✭ 13 (-92.97%)
TaurusAutomation-friendly framework for Continuous Testing by
Stars: ✭ 1,566 (+746.49%)
Sina Stock CrawlerSina stock options crawler with CSV output 新浪上证ETF期权数据爬虫
Stars: ✭ 12 (-93.51%)
Go SeleniumSelenium web driver library written in Go
Stars: ✭ 157 (-15.14%)
DisecDistributed Image Search Engine Crawler
Stars: ✭ 11 (-94.05%)
E2e ExperimentA demo project with Spring Boot / Angular application and e2e tests
Stars: ✭ 9 (-95.14%)
WhatsappbotSend messages to any person in any time how much you want.
Stars: ✭ 104 (-43.78%)
Sqlivmassive SQL injection vulnerability scanner
Stars: ✭ 840 (+354.05%)
Automationtest自动化测试,支持接口自动化、WEB UI自动化、APP UI自动化、性能测试
Stars: ✭ 171 (-7.57%)
Gisaid ScrapperScrapping tool for GISAID data regarding SARS-CoV-2
Stars: ✭ 25 (-86.49%)
OnegramThis repository is no longer maintained.
Stars: ✭ 137 (-25.95%)
TumblthreeA Tumblr Blog Backup Application
Stars: ✭ 923 (+398.92%)
FscrawlerElasticsearch File System Crawler (FS Crawler)
Stars: ✭ 906 (+389.73%)
AbotCross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.
Stars: ✭ 1,961 (+960%)
FinalreconThe Last Web Recon Tool You'll Need
Stars: ✭ 888 (+380%)
PulsarTurn large Web sites into tables and charts using simple SQLs.
Stars: ✭ 100 (-45.95%)
BitextorBitextor generates translation memories from multilingual websites.
Stars: ✭ 168 (-9.19%)
Th Music Video GeneratorTouhou Project random music video generator/player, crawling image and video from websites to generate MV.
Stars: ✭ 146 (-21.08%)
Docs《数据采集从入门到放弃》源码。内容简介:爬虫介绍、就业情况、爬虫工程师面试题 ;HTTP协议介绍; Requests使用 ;解析器Xpath介绍; MongoDB与MySQL; 多线程爬虫; Scrapy介绍 ;Scrapy-redis介绍; 使用docker部署; 使用nomad管理docker集群; 使用EFK查询docker日志
Stars: ✭ 118 (-36.22%)
Hproxyhproxy - Asynchronous IP proxy pool, aims to make getting proxy as convenient as possible.(异步爬虫代理池)
Stars: ✭ 62 (-66.49%)
4chan DownloaderPython3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation
Stars: ✭ 136 (-26.49%)
Selendroid"Selenium for Android" (Test automate native or hybrid Android apps and the mobile web with Selendroid.) Join us on IRC #selendroid on freenode. Also confirm you have signed the CLA http://goo.gl/pAvxEI when making a Pull Request.
Stars: ✭ 828 (+347.57%)
AetAET - a system that detects visual changes on web sites and performs basic page health checks
Stars: ✭ 100 (-45.95%)
TorbotDark Web OSINT Tool
Stars: ✭ 821 (+343.78%)
Instagram CrawlerCrawl instagram photos, posts and videos for download.
Stars: ✭ 178 (-3.78%)
SillyniumAutomate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (-45.95%)
GospiderGospider - Fast web spider written in Go
Stars: ✭ 785 (+324.32%)
Goclone Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.
Stars: ✭ 134 (-27.57%)
Foxr🦊 Node.js API to control Firefox
Stars: ✭ 783 (+323.24%)
PxerA tool for pixiv.net. 人人可用的P站爬虫
Stars: ✭ 776 (+319.46%)
Qxf2 Page Object ModelWrite Selenium and Appium tests in Python using the Page Object pattern. This Pythonic GUI and API test automation framework will help you get started with QA automation quickly. It comes with many useful integrations like - email, BrowserStack, Slack, TestRail, etc. This repository is developed and maintained by Qxf2 Services (https://qxf2.com).
Stars: ✭ 155 (-16.22%)
Moodle Downloader 2A Moodle downloader that downloads course content fast from Moodle (eg. lecture pdfs)
Stars: ✭ 118 (-36.22%)
PyleniumioBring the best of Selenium and Cypress into a single Python package
Stars: ✭ 61 (-67.03%)
JavpyEnjoy driving on a Javascriptive (originally Pythonic) way to Japanese AV!
Stars: ✭ 147 (-20.54%)
SeleniumcrawlerAn example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117 (-36.76%)
Boj AutocommitWhen you solve the problem of Baekjoon Online Judge, it automatically commits and pushes to the remote repository.
Stars: ✭ 60 (-67.57%)