SquidwarcSquidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
Stars: ✭ 125 (-35.23%)
Lancia网页转PDF渲染服务。提供收据、发票、报告或任何网页内容转PDF的微服务
Stars: ✭ 108 (-44.04%)
FerretDeclarative web scraping
Stars: ✭ 4,837 (+2406.22%)
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-11.4%)
Sms Boom利用chrome的headless模式,模拟用户注册进行短信轰炸机
Stars: ✭ 507 (+162.69%)
Rendoradynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites
Stars: ✭ 1,853 (+860.1%)
bots-zooNo description or website provided.
Stars: ✭ 59 (-69.43%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+239.9%)
Goribot[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-1.55%)
ApiAPI that uncovers the technologies used on websites and generates thumbnail from screenshot of website
Stars: ✭ 189 (-2.07%)
Lulu[Unmaintained] A simple and clean video/music/image downloader 👾
Stars: ✭ 789 (+308.81%)
ScrapitScraping scripts for various websites.
Stars: ✭ 25 (-87.05%)
Instagram CrawlerCrawl instagram photos, posts and videos for download.
Stars: ✭ 178 (-7.77%)
Chart To AwsMicroservice to generate screenshot from a webpage and upload it to a AWS S3 Bucket.
Stars: ✭ 43 (-77.72%)
Puppeteer DeepPuppeteer, Headless Chrome;爬取《es6标准入门》、自动推文到掘金、站点性能分析;高级爬虫、自动化UI测试、性能分析;
Stars: ✭ 1,033 (+435.23%)
Viewfinderjs📷 ViewFinder - NodeJS product to make the browser into a web app. WTF RBI. CBII. Remote browser isolation, embeddable browserview, secure chrome saas. Licenses, managed, self-hosted. Like S2, WebGap, Bromium, Authentic8, Menlo Security and Broadcom, but open source with free live demos available now! Also, integrated RBI/CDR with CDR from https://github.com/dosyago/p2%2e
Stars: ✭ 1,175 (+508.81%)
PypergrabberFetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.
Stars: ✭ 14 (-92.75%)
Social ScraperTổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt
Stars: ✭ 47 (-75.65%)
Jd AutobuyPython爬虫,京东自动登录,在线抢购商品
Stars: ✭ 1,174 (+508.29%)
Headless RecorderChrome extension that records your browser interactions and generates a Playwright or Puppeteer script.
Stars: ✭ 13,786 (+7043.01%)
LightcrawlerCrawl a website and run it through Google lighthouse
Stars: ✭ 1,339 (+593.78%)
SillyniumAutomate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (-48.19%)
Node FrontendNode.js Docker image with all Puppeteer dependencies installed for frontend Chrome Headless testing and default Nginx config, for multi-stage Docker building
Stars: ✭ 104 (-46.11%)
FbcrawlA Facebook crawler
Stars: ✭ 536 (+177.72%)
ScrapyrtHTTP API for Scrapy spiders
Stars: ✭ 637 (+230.05%)
CrawlerA high performance web crawler in Elixir.
Stars: ✭ 781 (+304.66%)
Alpine ChromeChrome Headless docker images built upon alpine official image
Stars: ✭ 754 (+290.67%)
Url To Pdf ApiWeb page PDF/PNG rendering done right. Self-hosted service for rendering receipts, invoices, or any content.
Stars: ✭ 6,544 (+3290.67%)
Awesome CrawlerA collection of awesome web crawler,spider in different languages
Stars: ✭ 4,793 (+2383.42%)
AvbookAV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+4113.99%)
Gowitness🔍 gowitness - a golang, web screenshot utility using Chrome Headless
Stars: ✭ 996 (+416.06%)
RodA Devtools driver for web automation and scraping
Stars: ✭ 1,392 (+621.24%)
Sushi Browser Sushi Browser is the next generation browser which mounts the multi-panel and the video support function and so on. Its goal is to be as fantastic as sushi. 🍣
Stars: ✭ 116 (-39.9%)
ScrapoxyScrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!
Stars: ✭ 1,322 (+584.97%)
Roam Research Private ApiPrivate API to enable API access for Roam Research. Now you can connect Roam to your other projects.
Stars: ✭ 88 (-54.4%)
Puppeteer WebperfAutomating Web Performance testing with Puppeteer 🎪
Stars: ✭ 1,392 (+621.24%)
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+545.6%)
Youtube ProjectsThis repository contains all the code I use in my YouTube tutorials.
Stars: ✭ 144 (-25.39%)
HeadlesschromeA Go package for working with headless Chrome. Run interactive JavaScript commands on web pages with Go and Chrome.
Stars: ✭ 112 (-41.97%)
WombatLightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
Stars: ✭ 1,220 (+532.12%)
Google Play ScraperGoogle play scraper for Python inspired by <facundoolano/google-play-scraper>
Stars: ✭ 143 (-25.91%)
OnegramThis repository is no longer maintained.
Stars: ✭ 137 (-29.02%)
RizeHigh-level, fluent and chainable API provided library for puppeteer.
Stars: ✭ 147 (-23.83%)
NewspaperNews, full-text, and article metadata extraction in Python 3. Advanced docs:
Stars: ✭ 11,545 (+5881.87%)
Chromdaλ 🖼️ Chromda is an AWS Lambda function for capturing screenshots of websites.
Stars: ✭ 481 (+149.22%)
GoscraperGolang pkg to quickly return a preview of a webpage (title/description/images)
Stars: ✭ 72 (-62.69%)
Datmusic ApiAlternative for VK Audio API
Stars: ✭ 160 (-17.1%)
Instagram Scraperscrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot
Stars: ✭ 2,209 (+1044.56%)