Skrape.itA Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
Stars: ✭ 231 (+28.33%)
Hquery.phpAn extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.
Stars: ✭ 295 (+63.89%)
Cumcomic updater, mangafied
Stars: ✭ 117 (-35%)
Google Play ScraperGoogle play scraper for Python inspired by <facundoolano/google-play-scraper>
Stars: ✭ 143 (-20.56%)
FlokiFloki is a simple HTML parser that enables search for nodes using CSS selectors.
Stars: ✭ 1,642 (+812.22%)
MwofflinerScrape any online Mediawiki motorised wiki (like Wikipedia) to your local filesystem
Stars: ✭ 121 (-32.78%)
Scraperwiki PythonScraperWiki Python library for scraping and saving data
Stars: ✭ 146 (-18.89%)
Save For OfflineAndroid app for saving webpages for offline reading.
Stars: ✭ 114 (-36.67%)
PywebcopyPython library to mirror webpage and websites.
Stars: ✭ 156 (-13.33%)
RodA Devtools driver for web automation and scraping
Stars: ✭ 1,392 (+673.33%)
Awesome Dl This is a list of repositories and libraries that allow for scripted downloading of online content.
Stars: ✭ 93 (-48.33%)
ProxyscrapePython library for retrieving free proxies (HTTP, HTTPS, SOCKS4, SOCKS5).
Stars: ✭ 134 (-25.56%)
MinimizeMinimize HTML
Stars: ✭ 150 (-16.67%)
Youtube Comment SuiteDownload YouTube comments from numerous videos, playlists, and channels for archiving, general search, and showing activity.
Stars: ✭ 120 (-33.33%)
Covid19 mobilityCOVID-19 Mobility Data Aggregator. Scraper of Google, Apple, Waze and TomTom COVID-19 Mobility Reports🚶🚘🚉
Stars: ✭ 156 (-13.33%)
Lua GumboMoved to https://gitlab.com/craigbarnes/lua-gumbo
Stars: ✭ 116 (-35.56%)
Google2csvGoogle2Csv a simple google scraper that saves the results on a csv/xlsx/jsonl file
Stars: ✭ 145 (-19.44%)
JobfunnelScrape job websites into a single spreadsheet with no duplicates.
Stars: ✭ 1,528 (+748.89%)
Scrape Twitter🐦 Access Twitter data without an API key. [DEPRECATED]
Stars: ✭ 166 (-7.78%)
AutocserAutoCSer is a high-performance RPC framework. AutoCSer 是一个以高效率为目标向导的整体开发框架。主要包括 TCP 接口服务框架、TCP 函数服务框架、远程表达式链组件、前后端一体 WEB 视图框架、ORM 内存索引缓存框架、日志流内存数据库缓存组件、消息队列组件、二进制 / JSON / XML 数据序列化 等一系列无缝集成的高性能组件。
Stars: ✭ 140 (-22.22%)
Laravel ScavengerThe most integrated web scraper package for Laravel.
Stars: ✭ 91 (-49.44%)
DemeterDemeter is a tool for scraping the calibre web ui
Stars: ✭ 155 (-13.89%)
Hockey ScraperPython Package for scraping NHL Play-by-Play and Shift data
Stars: ✭ 93 (-48.33%)
UdemycoursegrabberYour will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!
Stars: ✭ 137 (-23.89%)
Sax WasmThe first streamable, fixed memory XML, HTML, and JSX parser for WebAssembly.
Stars: ✭ 89 (-50.56%)
NewspaperNews, full-text, and article metadata extraction in Python 3. Advanced docs:
Stars: ✭ 11,545 (+6313.89%)
ScraperA scraper that switches between normal mode and gentleman mode, built on Eletron, React
Stars: ✭ 127 (-29.44%)
OpensanctionsAn open database of international sanctions data, persons of interest and politically exposed persons
Stars: ✭ 157 (-12.78%)
ArxivscraperA python module to scrape arxiv.org for specific date range and categories
Stars: ✭ 121 (-32.78%)
PhpscraperPHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (-17.78%)
SeleniumcrawlerAn example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117 (-35%)
Novel基于 Laravel 5.2 的小说网站
Stars: ✭ 172 (-4.44%)
Ridereceipts🚕 Simple automation desktop app to download and organize your receipts from Uber/Lyft. Try out our new Ride Receipts PRO !
Stars: ✭ 117 (-35%)
NsoupNSoup is a .NET port of the jsoup (http://jsoup.org) HTML parser and sanitizer originally written in Java
Stars: ✭ 145 (-19.44%)
Instagram Python ScraperA instagram scraper wrote in python. Similar to instagram-php-scraper.Usages are in example.py. Enjoy it!
Stars: ✭ 115 (-36.11%)
Instagram Scraperscrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot
Stars: ✭ 2,209 (+1127.22%)
HeadlesschromeA Go package for working with headless Chrome. Run interactive JavaScript commands on web pages with Go and Chrome.
Stars: ✭ 112 (-37.78%)
Youtube ProjectsThis repository contains all the code I use in my YouTube tutorials.
Stars: ✭ 144 (-20%)
MyhtmlFast C/C++ HTML 5 Parser. Using threads.
Stars: ✭ 1,512 (+740%)
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-5%)
ZillowZillow Scraper for Python using Selenium
Stars: ✭ 141 (-21.67%)
DidomSimple and fast HTML and XML parser
Stars: ✭ 1,939 (+977.22%)
SillyniumAutomate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (-44.44%)
Go Jd京东自动登录,在线商品自动下单
Stars: ✭ 139 (-22.78%)
ScrapoxyScrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!
Stars: ✭ 1,322 (+634.44%)
Scrapelib⛏ a library for scraping things
Stars: ✭ 164 (-8.89%)
OnegramThis repository is no longer maintained.
Stars: ✭ 137 (-23.89%)
Wxmlify一个轻量快速的插件,帮助你在微信小程序中显示富文本编辑器生成的HTML。
Stars: ✭ 93 (-48.33%)
Html Agility PackHtml Agility Pack (HAP) is a free and open-source HTML parser written in C# to read/write DOM and supports plain XPATH or XSLT. It is a .NET code library that allows you to parse "out of the web" HTML files.
Stars: ✭ 2,014 (+1018.89%)
Wxparse微信小程序富文本解析
Stars: ✭ 135 (-25%)
Instagram CrawlerCrawl instagram photos, posts and videos for download.
Stars: ✭ 178 (-1.11%)
ReadablewebproxyRewriting web proxy and archival tool. At this point, it just tries to download all the things.
Stars: ✭ 172 (-4.44%)
Datmusic ApiAlternative for VK Audio API
Stars: ✭ 160 (-11.11%)