Strong Web Crawler基于C#.NET+PhantomJS+Sellenium的高级网络爬虫程序。可执行Javascript代码、触发各类事件、操纵页面Dom结构。
Stars: ✭ 238 (-25.62%)
Skrape.itA Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
Stars: ✭ 231 (-27.81%)
FilesensorDynamic file detection tool based on crawler 基于爬虫的动态敏感文件探测工具
Stars: ✭ 227 (-29.06%)
spiderable-middleware🤖 Prerendering for JavaScript powered websites. Great solution for PWAs (Progressive Web Apps), SPAs (Single Page Applications), and other websites based on top of front-end JavaScript frameworks
Stars: ✭ 29 (-90.94%)
Annie👾 Fast and simple video download library and CLI tool written in Go
Stars: ✭ 16,369 (+5015.31%)
SelenopsA Swift Web Crawler 🕷
Stars: ✭ 225 (-29.69%)
ProxybrokerProxy [Finder | Checker | Server]. HTTP(S) & SOCKS 🎭
Stars: ✭ 2,767 (+764.69%)
WeiboCrawler无cookie版微博爬虫,可以连续爬取一个或多个新浪微博用户信息、用户微博及其微博评论转发。
Stars: ✭ 45 (-85.94%)
GetVideoBotTwitter bot to help you download videos and gifs
Stars: ✭ 34 (-89.37%)
img-cliAn interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL
Stars: ✭ 15 (-95.31%)
TumblthreeA Tumblr Backup Application
Stars: ✭ 211 (-34.06%)
Algoliasearch NetlifyOfficial Algolia Plugin for Netlify. Index your website to Algolia when deploying your project to Netlify with the Algolia Crawler
Stars: ✭ 208 (-35%)
dy抖音数据抓取
Stars: ✭ 22 (-93.12%)
Tianyanchapip安装的天眼查爬虫API,指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.
Stars: ✭ 206 (-35.62%)
Bili-dlA Video downloader for Bilibili 用来下载b站视频的工具 使用的是b站旧版flash播放api 暂无更新计划
Stars: ✭ 67 (-79.06%)
WoidSimple news aggregator displaying top stories in real time
Stars: ✭ 204 (-36.25%)
papercutPapercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.
Stars: ✭ 15 (-95.31%)
GooglescraperA Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.
Stars: ✭ 2,363 (+638.44%)
redvidSmart downloader for Reddit hosted videos
Stars: ✭ 83 (-74.06%)
2017 PyConTW Talktw.pycon.org/2017/events/talk/314386410792550475/
Stars: ✭ 18 (-94.37%)
Laosjgolang light-weight image crawler
Stars: ✭ 199 (-37.81%)
JvppeteerHeadless Chrome For Java (Java 爬虫)
Stars: ✭ 193 (-39.69%)
chu music网页音乐播放器,仅1.4K,支持咪咕等等,随心更新中~~支持小程序
Stars: ✭ 16 (-95%)
medium-stat-boxPractical pinned gist which show your latest medium status 📌
Stars: ✭ 29 (-90.94%)
Goribot[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-40.62%)
ComiccrawlerAn image crawler written in Python.
Stars: ✭ 185 (-42.19%)
glinkShort video direct link acquisition 短视频直连去水印工具(支持抖音、快手、TikTok、皮皮虾、火山、开眼、陌陌等十几种)
Stars: ✭ 112 (-65%)
Web Bee🐝 Web vertical crawler framework for fun
Stars: ✭ 184 (-42.5%)
ByteDanceOpenBytedance(TikTok, Toutiao)third party open platform backend SDK, provides easy-to-use authorization, template management, code package management and other APIs.字节跳动(抖音, 头条)第三方开放平台小程序sdk, 提供了简单易用的授权,模版管理,代码包管理等api.
Stars: ✭ 67 (-79.06%)
NosmokeA cross platform UI crawler which scans view trees then generate and execute UI test cases.
Stars: ✭ 178 (-44.37%)
N2h4네이버 뉴스 수집을 위한 도구
Stars: ✭ 177 (-44.69%)
tele-tube-riderTelegram Bot 🤖 to download videos from Youtube, Reddit and Twitter and send it back to you.
Stars: ✭ 36 (-88.75%)
Spoon🥄 A package for building specific Proxy Pool for different Sites.
Stars: ✭ 173 (-45.94%)
SharinganWe will try to find your visible basic footprint from social media as much as possible - 😤 more sites is comming soon
Stars: ✭ 13 (-95.94%)
PoliteBe nice on the web
Stars: ✭ 253 (-20.94%)
YoutubeDownloaderA Music downloader with Spotify integration for automatic metadata application. [Depricated]
Stars: ✭ 13 (-95.94%)
CrawlBoxEasy way to brute-force web directory.
Stars: ✭ 118 (-63.12%)
arachnodHigh performance crawler for Nodejs
Stars: ✭ 17 (-94.69%)