htmlunit🕸🧰☕️Tools to Scrape Dynamic Web Content via the 'HtmlUnit' Java Library
Stars: ✭ 39 (+77.27%)
MarionetteSelenium alternative for Crystal. Browser manipulation without the Java overhead.
Stars: ✭ 119 (+440.91%)
wayback⏪ Tools to Work with the Various Internet Archive Wayback Machine APIs
Stars: ✭ 52 (+136.36%)
Dappeteer🏌🏼E2E testing for dApps using Puppeteer + MetaMask
Stars: ✭ 117 (+431.82%)
Data-Wrangling-with-PythonSimplify your ETL processes with these hands-on data sanitation tips, tricks, and best practices
Stars: ✭ 90 (+309.09%)
Query Selector Shadow DomquerySelector that can pierce Shadow DOM roots without knowing the path through nested shadow roots. Useful for automated testing of Web Components. Production use is not advised, this is for test environments/tools such as Web Driver, Playwright, Puppeteer
Stars: ✭ 115 (+422.73%)
PulsarTurn large Web sites into tables and charts using simple SQLs.
Stars: ✭ 100 (+354.55%)
Pwa Asset GeneratorAutomates PWA asset generation and image declaration. Automatically generates icon and splash screen images, favicons and mstile images. Updates manifest.json and index.html files with the generated images according to Web App Manifest specs and Apple Human Interface guidelines.
Stars: ✭ 1,787 (+8022.73%)
rymscraperPython API to extract data from rateyourmusic.com.
Stars: ✭ 63 (+186.36%)
coreThe complete web scraping toolkit for PHP.
Stars: ✭ 1,110 (+4945.45%)
Node Spidernodejs爬取西瓜视频(今日头条视频)
Stars: ✭ 105 (+377.27%)
faexportThe API for Furaffinity you wish existed
Stars: ✭ 61 (+177.27%)
Puppeteer WebperfAutomating Web Performance testing with Puppeteer 🎪
Stars: ✭ 1,392 (+6227.27%)
Splashr💦 Tools to Work with the 'Splash' JavaScript Rendering Service in R
Stars: ✭ 93 (+322.73%)
Review reverse👋2019年末总结下今年做过的逆向,整理代码,复习思路。🙏拼夕夕Web端anti_content参数逆向分析👺 WEB淘宝sign逆向分析;😺努比亚Cookie生成逆向分析;🙌百度指数data加密逆向分析 👣今日头条WEB端_signature、as、cp参数逆向分析🎶知乎登录formdata加密逆向分析 🤡KNN猫眼字体反爬👅Boss直聘Cookie加密字段__zp_stoken__逆向分析
Stars: ✭ 1,302 (+5818.18%)
TikTokDownloader PyWebIO🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音|TikTok数据爬取工具,支持API调用,在线批量解析及下载。
Stars: ✭ 919 (+4077.27%)
PyppeteerHeadless chrome/chromium automation library (unofficial port of puppeteer)
Stars: ✭ 1,286 (+5745.45%)
insta-feed.jsShow your Instagram profile on your website. 🌈
Stars: ✭ 25 (+13.64%)
Perftools RunnerGoogle Performance Tools runner using Puppeteer
Stars: ✭ 79 (+259.09%)
iowebWeb Scraping Framework
Stars: ✭ 31 (+40.91%)
MostlyThey mostly come at night; mostly.
Stars: ✭ 78 (+254.55%)
HumanoidNode.js package to bypass CloudFlare's anti-bot JavaScript challenges
Stars: ✭ 88 (+300%)
Puppeteer ContainerA GitHub Action / Docker image for Puppeteer, the Headless Chrome Node API
Stars: ✭ 72 (+227.27%)
Stock-Market-PredictorStock Market Predictor with LSTM network. Web scraping and analyzing tools (ohlc, mean)
Stars: ✭ 28 (+27.27%)
KjuKju — Improved waiting time for the adidas.com splash page ❯❯❯_
Stars: ✭ 68 (+209.09%)
PythonScrapyBasicSetupBasic setup with random user agents and IP addresses for Python Scrapy Framework.
Stars: ✭ 57 (+159.09%)
TibEasy e2e browser testing in Node
Stars: ✭ 64 (+190.91%)
scrapy-wayback-machineA Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
Stars: ✭ 92 (+318.18%)
RvestSimple web scraping for R
Stars: ✭ 1,253 (+5595.45%)
Angular Puppeteer DemoA demonstration repository explains how to using Puppeteer in unit testing
Stars: ✭ 59 (+168.18%)
trafilaturaPython & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
Stars: ✭ 711 (+3131.82%)
codebookOpen-source learning tool that allows to users to follow textbook contents while practicing HTML, CSS and JS using a live code editor.
Stars: ✭ 20 (-9.09%)
PuppeteerHeadless Chrome Node.js API
Stars: ✭ 75,197 (+341704.55%)
Dark Mode ScreenshotThis Puppeteer script takes a 📷 screenshot of a webpage in 🌞 Light and 🌒 Dark Mode.
Stars: ✭ 47 (+113.64%)
ReaderExtract clean(er), readable text from web pages via Mercury Web Parser.
Stars: ✭ 75 (+240.91%)
PaperScraperA web scraping tool to systematically extract the text of scientific papers and corresponding metadata from university accessible journals.
Stars: ✭ 63 (+186.36%)
generateDeviceScreenshotsA simple command line tool to generate device screenshots of different webpages.
Stars: ✭ 69 (+213.64%)
Puppeteer Request SpyWatch, fake or block requests from puppeteer matching patterns
Stars: ✭ 42 (+90.91%)
Mibuy小米商城抢购
Stars: ✭ 35 (+59.09%)
Serverless Chrome🌐 Run headless Chrome/Chromium on AWS Lambda
Stars: ✭ 2,625 (+11831.82%)
Devtools Protocol DemosDemos of the Chrome DevTools Protocol for various testing/automation techniques
Stars: ✭ 21 (-4.55%)
ArachnidPowerful web scraping framework for Crystal
Stars: ✭ 68 (+209.09%)
BlazarPre-Render Pages on the Fly in Laravel
Stars: ✭ 14 (-36.36%)
node-headless-chrome⚠️ 🚧 Install precompiled versions of the Chromium/Chrome headless shell using npm or yarn
Stars: ✭ 20 (-9.09%)
codeframeThe fastest, easiest way to build and deploy quick static webpages
Stars: ✭ 107 (+386.36%)
LighthousebotRun Lighthouse in CI, as a web service, using Docker. Pass/Fail GH pull requests.
Stars: ✭ 2,251 (+10131.82%)
uzenWebsite crawler with YARA detection
Stars: ✭ 84 (+281.82%)
web-poetWeb scraping Page Objects core library
Stars: ✭ 67 (+204.55%)
Viewfinder📷 BrowserBox - Remote isolated browser API for security, automation visibility and interactivity. Run on our cloud, or bring your own. Full scope double reverse web proxy with multi-tab, mobile-ready browser UI frontend. Plus co-browsing, advanced adaptive streaming, secure document viewing and more! But only in the Pro version. Get BB today! Se…
Stars: ✭ 1,741 (+7813.64%)
playwright-demosplaywright for scrapping and UI testing / automate testing workflows
Stars: ✭ 65 (+195.45%)
pupflareA webpage proxy that request through Chromium (puppeteer) - can be used to bypass Cloudflare anti bot / anti ddos on any application (like curl)
Stars: ✭ 183 (+731.82%)
Netflix CloneNetflix like full-stack application with SPA client and backend implemented in service oriented architecture
Stars: ✭ 156 (+609.09%)