BeanbunBeanbun 是用 PHP 编写的多进程网络爬虫框架,具有良好的开放性、高可扩展性,基于 Workerman。
Stars: ✭ 1,096 (+544.71%)
ScyllaIntelligent proxy pool for Humans™ (Maintainer needed)
Stars: ✭ 3,409 (+1905.29%)
Google Play ScraperGoogle play scraper for Python inspired by <facundoolano/google-play-scraper>
Stars: ✭ 143 (-15.88%)
LinkedinLinkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
Stars: ✭ 309 (+81.76%)
CrawlergoA powerful dynamic crawler for web vulnerability scanners
Stars: ✭ 1,088 (+540%)
ToapiEvery web site provides APIs.
Stars: ✭ 3,209 (+1787.65%)
Laravel Socialite Social OAuth Authentication for Laravel 5. drivers: facebook, github, google, linkedin, weibo, qq, wechat and douban
Stars: ✭ 296 (+74.12%)
Awesome Python Primer自学入门 Python 优质中文资源索引,包含 书籍 / 文档 / 视频,适用于 爬虫 / Web / 数据分析 / 机器学习 方向
Stars: ✭ 57 (-66.47%)
GhcrawlerCrawl GitHub APIs and store the discovered orgs, repos, commits, ...
Stars: ✭ 293 (+72.35%)
GainWeb crawling framework based on asyncio.
Stars: ✭ 2,002 (+1077.65%)
Weixin Spider微信公众号爬虫,公众号历史文章,文章评论,文章阅读及在看数据,可视化web页面,可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现,高效微信爬虫,微信公众号爬虫,历史文章,文章评论,数据更新。
Stars: ✭ 287 (+68.82%)
Linkedin BotJS script for automatic invitations to add to the network of contacts
Stars: ✭ 52 (-69.41%)
Gospidergolang实现的爬虫框架,使用者只需关心页面规则,提供web管理界面。基于colly开发。
Stars: ✭ 285 (+67.65%)
Jianso movie🎬 电影资源爬虫,电影图片抓取脚本,Flask|Nginx|wsgi
Stars: ✭ 114 (-32.94%)
Hacker News Digest📰 A responsive interface of Hacker News with summaries and thumbnails.
Stars: ✭ 278 (+63.53%)
OddishTo crawl all csgo skins from website.
Stars: ✭ 139 (-18.24%)
Gopa[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Stars: ✭ 277 (+62.94%)
Lyrics CrawlerGet the lyrics for the song currently playing on Spotify
Stars: ✭ 49 (-71.18%)
RcrawlerAn R web crawler and scraper
Stars: ✭ 274 (+61.18%)
DatahubThe Metadata Platform for the Modern Data Stack
Stars: ✭ 4,232 (+2389.41%)
Social ScraperTổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt
Stars: ✭ 47 (-72.35%)
Python3 SpiderPython爬虫实战 - 模拟登陆各大网站 包含但不限于:滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝,如果喜欢请start ❤️
Stars: ✭ 2,129 (+1152.35%)
Weibo terminator workflowUpdate Version of weibo_terminator, This is Workflow Version aim at Get Job Done!
Stars: ✭ 259 (+52.35%)
PixevalA Strong, Fast and Flexible Pixiv Client based on .NET Core and WPF
Stars: ✭ 1,031 (+506.47%)
SpidyThe simple, easy to use command line web crawler.
Stars: ✭ 257 (+51.18%)
Lcrawl一只优雅的正方教务系统爬虫。
Stars: ✭ 112 (-34.12%)
galerA fast tool to fetch URLs from HTML attributes by crawl-in.
Stars: ✭ 138 (-18.82%)
Weibo Crawler新浪微博爬虫,用python爬取新浪微博数据,并下载微博图片和微博视频
Stars: ✭ 1,019 (+499.41%)
octopusRecursive and multi-threaded broken link checker
Stars: ✭ 19 (-88.82%)
Instagram BotAn Instagram bot developed using the Selenium Framework
Stars: ✭ 138 (-18.82%)
AvbookAV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+4684.12%)
Data-mining-python-scriptIt contain various script on web crawling/ data mining of social web(RSS,facebook,twitter,Linkedin)
Stars: ✭ 24 (-85.88%)
BaiduspiderBaiduSpider,一个爬取百度搜索结果的爬虫,目前支持百度网页搜索,百度图片搜索,百度知道搜索,百度视频搜索,百度资讯搜索,百度文库搜索,百度经验搜索和百度百科搜索。
Stars: ✭ 105 (-38.24%)
eastmoneypython requests + Django+ nodejs koa+ mysql to crawl eastmoney fund and stock data,for data analysis and visualiaztion .
Stars: ✭ 56 (-67.06%)
Vulnxvulnx 🕷️ is an intelligent bot auto shell injector that detect vulnerabilities in multiple types of cms { `wordpress , joomla , drupal , prestashop .. `}
Stars: ✭ 1,009 (+493.53%)
rankr🇰🇷 Realtime integrated information analysis service
Stars: ✭ 21 (-87.65%)
GocrawlPolite, slim and concurrent web crawler.
Stars: ✭ 1,962 (+1054.12%)
html-queryA fluent and functional approach to querying HTML
Stars: ✭ 48 (-71.76%)
Instagram Profilecrawl💻 Quickly crawl the information (e.g. followers, tags, etc...) of an instagram profile. No login required!
Stars: ✭ 110 (-35.29%)
snapcrawlCrawl a website and take screenshots
Stars: ✭ 37 (-78.24%)
TumblTwoTumblTwo, an Improved Fork of TumblOne, a Tumblr Downloader.
Stars: ✭ 57 (-66.47%)
Go spider[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.
Stars: ✭ 1,745 (+926.47%)
XsrfprobeThe Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.
Stars: ✭ 532 (+212.94%)
React Native Linkedin SdkReact Native Wrapper for Latest LinkedIn Mobile SDK for Sign-In / Auth and API Access.
Stars: ✭ 37 (-78.24%)
Proxy poolPython爬虫代理IP池(proxy pool)
Stars: ✭ 13,964 (+8114.12%)
Douyin crawler 抖音爬虫,tiktok crawler,抖音数据采集接口,抖音视频去水印,百分百成功,不需要服务器,不需要代理 IP。
Stars: ✭ 169 (-0.59%)
Magento 2 Social LoginMagento 2 Social Login extension is designed for quick login to your Magento 2 store without procesing complex register steps
Stars: ✭ 156 (-8.24%)
CrawlerGo process used to crawl websites
Stars: ✭ 147 (-13.53%)
Crawlab LiteLite version of Crawlab. 轻量版 Crawlab 爬虫管理平台
Stars: ✭ 122 (-28.24%)