Douyinsdk抖音 SDK,数据采集,爬虫抓取不是梦
Stars: ✭ 99 (-53.74%)
Jd mask robot京东口罩库存监控爬虫(非selenium),扫码登录、查价、加购、下单、秒杀
Stars: ✭ 216 (+0.93%)
Yispider一款分布式爬虫平台,帮助你更好的管理和开发爬虫。 内置一套爬虫定义规则(模版),可使用模版快速定义爬虫,也可当作框架手动开发爬虫。(兴趣使然的项目,用的不爽了就更新)
Stars: ✭ 158 (-26.17%)
BaiduspiderBaiduSpider,一个爬取百度搜索结果的爬虫,目前支持百度网页搜索,百度图片搜索,百度知道搜索,百度视频搜索,百度资讯搜索,百度文库搜索,百度经验搜索和百度百科搜索。
Stars: ✭ 105 (-50.93%)
crawlerA simple and flexible web crawler framework for java.
Stars: ✭ 20 (-90.65%)
Magic googleGoogle search results crawler, get google search results that you need
Stars: ✭ 247 (+15.42%)
arachnodHigh performance crawler for Nodejs
Stars: ✭ 17 (-92.06%)
WebCrawler一个轻量级、快速、多线程、多管道、灵活配置的网络爬虫。
Stars: ✭ 39 (-81.78%)
Lianjia Beike Spider链家网和贝壳网房价爬虫,采集北京上海广州深圳等21个中国主要城市的房价数据(小区,二手房,出租房,新房),稳定可靠快速!支持csv,MySQL, MongoDB,Excel, json存储,支持Python2和3,图表展示数据,注释丰富 ,点星支持,仅供学习参考,请勿用于商业用途,后果自负。
Stars: ✭ 2,257 (+954.67%)
RuiaAsync Python 3.6+ web scraping micro-framework based on asyncio
Stars: ✭ 1,366 (+538.32%)
Gospidergolang实现的爬虫框架,使用者只需关心页面规则,提供web管理界面。基于colly开发。
Stars: ✭ 285 (+33.18%)
Crawlertutorial爬蟲極簡教學(fetch, parse, search, multiprocessing, API)- PTT 為例
Stars: ✭ 282 (+31.78%)
JlitespiderA lite distributed Java spider framework :-)
Stars: ✭ 151 (-29.44%)
Hacker News Digest📰 A responsive interface of Hacker News with summaries and thumbnails.
Stars: ✭ 278 (+29.91%)
Ttbot今日头条机器人,支持用户登陆、关注、取消关注、获取关注粉丝、发文、发悟空问答、点赞、评论、采集各种类型新闻讯息等,使用今日头条网页版API实现
Stars: ✭ 338 (+57.94%)
91porn Api🌭💦 91porn爬虫在线无限制API接口(永久有效,口令每日更新) 及 在线web预览
Stars: ✭ 341 (+59.35%)
Fictiondown小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对
Stars: ✭ 362 (+69.16%)
Gopa[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Stars: ✭ 277 (+29.44%)
GosintOSINT Swiss Army Knife
Stars: ✭ 401 (+87.38%)
Bilili🍻 bilibili video (including bangumi) and danmaku downloader | B站视频(含番剧)、弹幕下载器
Stars: ✭ 379 (+77.1%)
LearnpythonPython的基础练习代码与各种爬虫代码
Stars: ✭ 451 (+110.75%)
Signature algorithm各种App、小程序、网站的请求签名或加密算法。 现已有:自如、小红书、蛋壳公寓、luckin coffee(瑞幸咖啡)、bangkokair(曼谷航空)
Stars: ✭ 380 (+77.57%)
XsrfprobeThe Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.
Stars: ✭ 532 (+148.6%)
Go jobs带你了解一下Golang的市场行情
Stars: ✭ 526 (+145.79%)
NetdiscoveryNetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。
Stars: ✭ 573 (+167.76%)
Skycaiji蓝天采集器是一款免费的数据采集发布爬虫软件,采用php+mysql开发,可部署在云服务器,几乎能采集所有类型的网页,无缝对接各类CMS建站程序,免登录实时发布数据,全自动无需人工干预!是网页大数据采集软件中完全跨平台的云端爬虫系统
Stars: ✭ 1,514 (+607.48%)
Creeper🐾 Creeper - The Next Generation Crawler Framework (Go)
Stars: ✭ 762 (+256.07%)
Grab SiteThe archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Stars: ✭ 680 (+217.76%)
TorbotDark Web OSINT Tool
Stars: ✭ 821 (+283.64%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+206.54%)
Nodespider[DEPRECATED] Simple, flexible, delightful web crawler/spider package
Stars: ✭ 33 (-84.58%)
ScrapitScraping scripts for various websites.
Stars: ✭ 25 (-88.32%)
Lizard💐 Full Amazon Automatic Download
Stars: ✭ 41 (-80.84%)
IcrawlerA multi-thread crawler framework with many builtin image crawlers provided.
Stars: ✭ 629 (+193.93%)
Awesome Python Primer自学入门 Python 优质中文资源索引,包含 书籍 / 文档 / 视频,适用于 爬虫 / Web / 数据分析 / 机器学习 方向
Stars: ✭ 57 (-73.36%)
PhotonIncredibly fast crawler designed for OSINT.
Stars: ✭ 8,332 (+3793.46%)
Amazonbigspider😱Full Automatic Amazon Distributed Spider | 亚马逊分布式四国际站采集选款产品|账号admin,密码adminadmin
Stars: ✭ 140 (-34.58%)
AvbookAV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+3700.47%)
Spiderpython crawler spider
Stars: ✭ 70 (-67.29%)
ArachnidPowerful web scraping framework for Crystal
Stars: ✭ 68 (-68.22%)
NewcrawlerFree Web Scraping Tool with Java
Stars: ✭ 589 (+175.23%)
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+482.24%)
DecryptloginAPIs for loginning some websites by using requests.
Stars: ✭ 1,861 (+769.63%)
Crawlab LiteLite version of Crawlab. 轻量版 Crawlab 爬虫管理平台
Stars: ✭ 122 (-42.99%)
Fun crawlerCrawl some picture for fun
Stars: ✭ 169 (-21.03%)
BitextorBitextor generates translation memories from multilingual websites.
Stars: ✭ 168 (-21.5%)
Mooc Dl👨🎓 中国大学MOOC全课件(视频、文档、附件)下载器
Stars: ✭ 163 (-23.83%)
ScrapydwebWeb app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉
Stars: ✭ 2,385 (+1014.49%)
GocrawlPolite, slim and concurrent web crawler.
Stars: ✭ 1,962 (+816.82%)
Videospider抓取豆瓣,bilibili等中的电视剧、电影、动漫演员等信息
Stars: ✭ 186 (-13.08%)
Datmusic ApiAlternative for VK Audio API
Stars: ✭ 160 (-25.23%)
GooglescraperA Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.
Stars: ✭ 2,363 (+1004.21%)
FinkPHP Link Checker
Stars: ✭ 157 (-26.64%)
ComiccrawlerAn image crawler written in Python.
Stars: ✭ 185 (-13.55%)