All Projects → zhangslob → Awesome_crawl

zhangslob / Awesome_crawl

腾讯新闻、知乎话题、微博粉丝,Tumblr爬虫、斗鱼弹幕、妹子图爬虫、分布式设计等

Programming Languages

python
139335 projects - #7 most used programming language
golang
3204 projects

Projects that are alternatives of or similar to Awesome crawl

Articlespider
慕课网python分布式爬虫源码-长期更新维护
Stars: ✭ 40 (-83.74%)
Mutual labels:  zhihu, scrapy, redis
Rsshub
🍰 Everything is RSSible
Stars: ✭ 18,111 (+7262.2%)
Mutual labels:  zhihu, douyu, weibo
Weibospider
This is a sina weibo spider built by scrapy [微博爬虫/持续维护]
Stars: ✭ 2,408 (+878.86%)
Mutual labels:  scrapy, redis, weibo
Decryptlogin
APIs for loginning some websites by using requests.
Stars: ✭ 1,861 (+656.5%)
Mutual labels:  zhihu, tencent, weibo
Funpyspidersearchengine
Word2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索
Stars: ✭ 782 (+217.89%)
Mutual labels:  zhihu, scrapy, redis
Sacrificial Socket
A Go websocket library with an API similar to Socket.IO... but not Socket.IO
Stars: ✭ 96 (-60.98%)
Mutual labels:  redis, websockets
Qewd
Quick and Easy Web Development
Stars: ✭ 106 (-56.91%)
Mutual labels:  redis, websockets
Weibo hot search
微博爬虫:每天定时爬取微博热搜榜的内容,留下互联网人的记忆。
Stars: ✭ 113 (-54.07%)
Mutual labels:  scrapy, weibo
Datamining And Social Sentiment Analysis Based On Weibo
基于微博的数据挖掘与社交舆情分析
Stars: ✭ 149 (-39.43%)
Mutual labels:  scrapy, redis
Omega
Real-time issue tracker optimized for small teams
Stars: ✭ 115 (-53.25%)
Mutual labels:  redis, websockets
Python3 Spider
Python爬虫实战 - 模拟登陆各大网站 包含但不限于:滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝,如果喜欢请start ❤️
Stars: ✭ 2,129 (+765.45%)
Mutual labels:  scrapy, crawl
Laravel Blog
Laravel 8.0 blog application with Vue.js, Homestead, Horizon, Telescope and Pusher
Stars: ✭ 1,248 (+407.32%)
Mutual labels:  redis, websockets
Django Channels React Multiplayer
turn based strategy game using django channels, redux, and react hooks
Stars: ✭ 52 (-78.86%)
Mutual labels:  redis, websockets
Entangled
Rails in real time
Stars: ✭ 108 (-56.1%)
Mutual labels:  redis, websockets
Socketshark
A WebSocket message router based on Python/Redis/asyncio
Stars: ✭ 51 (-79.27%)
Mutual labels:  redis, websockets
Spring Boot Examples
个人学习 SpringBoot2.x 写的一些示例程序,目前正在持续更新中.....
Stars: ✭ 159 (-35.37%)
Mutual labels:  redis, websockets
Nodebb
Node.js based forum software built for the modern web
Stars: ✭ 12,303 (+4901.22%)
Mutual labels:  redis, websockets
Django instagram
Photo sharing social media site built with Python/Django. Based on Instagram's design.
Stars: ✭ 165 (-32.93%)
Mutual labels:  redis, websockets
Scrapy Cluster
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
Stars: ✭ 921 (+274.39%)
Mutual labels:  scrapy, redis
Kuzzle
Open-source Back-end, self-hostable & ready to use - Real-time, storage, advanced search - Web, Apps, Mobile, IoT -
Stars: ✭ 991 (+302.85%)
Mutual labels:  redis, websockets

awesome_crawl Awesome


爬虫 说明 技术栈
腾讯新闻 采集所有腾讯新闻的链接和新闻详情 scrapy,mongo,redis
知乎话题 从话题广场出发,先采集子话题ID,再采集ID下所有问题 scrapy,mongo,redis
微博粉丝 采集大V的所有粉丝 scrapy,mongo,redis
Tumblr爬虫 下载指定Tumblr博主的资源 requests,concurrent
妹子图爬虫 go语言下载妹子图 goroutine,goquery
全站爬虫 分布式全站爬虫——以"搜狗电视剧"为例 scrapy,redis,分布式

如果能帮助你,那就最好了。欢迎关注公众号:

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].