Nodespider[DEPRECATED] Simple, flexible, delightful web crawler/spider package
Stars: ✭ 33 (-52.86%)
BtletSome toolkits implements part of BT Protocol, like DHT spider.
Stars: ✭ 54 (-22.86%)
BlackwidowA Python based web application scanner to gather OSINT and fuzz for OWASP vulnerabilities on a target website.
Stars: ✭ 887 (+1167.14%)
Boj AutocommitWhen you solve the problem of Baekjoon Online Judge, it automatically commits and pushes to the remote repository.
Stars: ✭ 60 (-14.29%)
ScrapitScraping scripts for various websites.
Stars: ✭ 25 (-64.29%)
Zhihu Crawlerzhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目
Stars: ✭ 890 (+1171.43%)
Awesome Python Primer自学入门 Python 优质中文资源索引,包含 书籍 / 文档 / 视频,适用于 爬虫 / Web / 数据分析 / 机器学习 方向
Stars: ✭ 57 (-18.57%)
App comments spider爬取百度贴吧、TapTap、appstore、微博官方博主上的游戏评论(基于redis_scrapy),过滤器采用了bloomfilter。
Stars: ✭ 38 (-45.71%)
Phantomjs Maven PluginA maven plugin for installing the phantomjs binary on your system automatically.
Stars: ✭ 62 (-11.43%)
PhantomjsScriptable Headless Browser
Stars: ✭ 28,654 (+40834.29%)
BlazarPre-Render Pages on the Fly in Laravel
Stars: ✭ 14 (-80%)
AntcolonyNodejs实现的一个磁力链接爬虫 http://findit.keenwon.com (原域名http://findit.so )
Stars: ✭ 1,151 (+1544.29%)
Go spiderA golang spider
Stars: ✭ 25 (-64.29%)
Reptile🏀 Python3 网络爬虫实战(部分含详细教程)猫眼 腾讯视频 豆瓣 研招网 微博 笔趣阁小说 百度热点 B站 CSDN 网易云阅读 阿里文学 百度股票 今日头条 微信公众号 网易云音乐 拉勾 有道 unsplash 实习僧 汽车之家 英雄联盟盒子 大众点评 链家 LPL赛程 台风 梦幻西游、阴阳师藏宝阁 天气 牛客网 百度文库 睡前故事 知乎 Wish
Stars: ✭ 1,048 (+1397.14%)
Test demoTesting Using Python Demo. 使用Python测试脚本demo。
Stars: ✭ 60 (-14.29%)
Anti Anti Spider越来越多的网站具有反爬虫特性,有的用图片隐藏关键数据,有的使用反人类的验证码,建立反反爬虫的代码仓库,通过与不同特性的网站做斗争(无恶意)提高技术。(欢迎提交难以采集的网站)(因工作原因,项目暂停)
Stars: ✭ 6,907 (+9767.14%)
PhotonIncredibly fast crawler designed for OSINT.
Stars: ✭ 8,332 (+11802.86%)
GospiderGospider - Fast web spider written in Go
Stars: ✭ 785 (+1021.43%)
Lizard💐 Full Amazon Automatic Download
Stars: ✭ 41 (-41.43%)
Car PricesGolang爬虫 爬取汽车之家 二手车产品库
Stars: ✭ 57 (-18.57%)
Lambda PacksPrecompiled packages for AWS Lambda
Stars: ✭ 997 (+1324.29%)
Novel Plus小说精品屋-plus是一个多端(PC、WAP)阅读、功能完善的原创文学CMS系统,由前台门户系统、作家后台管理系统、平台后台管理系统、爬虫管理系统等多个子系统构成,支持多模版、会员充值、订阅模式、新闻发布和实时统计报表等功能,新书自动入库,老书自动更新。
Stars: ✭ 1,122 (+1502.86%)
MamanRust Web Crawler saving pages on Redis
Stars: ✭ 39 (-44.29%)
NetcloudNetCloud Web Spider
Stars: ✭ 37 (-47.14%)
SpiderA configurable web spider with a easy-to-use web console
Stars: ✭ 954 (+1262.86%)
Gotoolscreate some tools use go lang.
Stars: ✭ 54 (-22.86%)
JspiderJSpider会每周更新至少一个网站的JS解密方式,欢迎 Star,交流微信:13298307816
Stars: ✭ 914 (+1205.71%)
T66y spiderPython多线程下载 草榴(t66y.com) 网站【新時代的我們】和【達蓋爾的旗幟】两个板块帖子内的图片
Stars: ✭ 62 (-11.43%)
Go DemoGo语言实例教程从入门到进阶,包括基础库使用、设计模式、面试易错点、工具类、对接第三方等
Stars: ✭ 881 (+1158.57%)
Serverless Html PdfConvert HTML to PDF thru a lambda function using PhantomJS.
Stars: ✭ 51 (-27.14%)
PholcusPholcus is a distributed high-concurrency crawler software written in pure golang
Stars: ✭ 6,990 (+9885.71%)
Api StoreContains all the public APIs listed in Phantombuster's API store. Pull requests welcome!
Stars: ✭ 69 (-1.43%)
EasyloginA python3 package for writing spider more easily.
Stars: ✭ 26 (-62.86%)
Wish常用javaweb技术,不定期更新,欢迎讨论。
Stars: ✭ 60 (-14.29%)
Super Spider根据腾讯安全应急响应中心的架构编写的一款超强爬虫(广度优先搜索)
Stars: ✭ 48 (-31.43%)
WallabyConcurrent browser tests with elixir
Stars: ✭ 1,143 (+1532.86%)
SeekerSeeker - another job board aggregator.
Stars: ✭ 16 (-77.14%)
FbiwarningNode.js seed downloader (Node.js 种子神器)
Stars: ✭ 44 (-37.14%)
TorbotDark Web OSINT Tool
Stars: ✭ 821 (+1072.86%)
GlyphhangerYour web font utility belt. It can subset web fonts. It can find unicode-ranges for you automatically. It makes julienne fries.
Stars: ✭ 1,099 (+1470%)
FunpyspidersearchengineWord2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索
Stars: ✭ 782 (+1017.14%)
AvbookAV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+11518.57%)
Alipayspider ScrapyAlipaySpider on Scrapy(use chrome driver); 支付宝爬虫(基于Scrapy)
Stars: ✭ 70 (+0%)
ArachnidPowerful web scraping framework for Crystal
Stars: ✭ 68 (-2.86%)
AbotxCross Platform C# Web crawler framework, headless browser, parallel crawler. Please star this project! +1.
Stars: ✭ 63 (-10%)