GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → changba → nsqproxy

changba / nsqproxy

Licence: MIT license
NSQProxy是NSQ和Worker之间的中间件，根据配置负责消息转发。然后通过HTTP/FastCGI/CBNSQ等协议转发给Worker机执行。

Programming Languages

go
31211 projects - #10 most used programming language
Vue
7211 projects
javascript
184084 projects - #8 most used programming language
SCSS
7915 projects

Labels

mysqlhttpvuensqfastcginsq-clientcbnsqnsqproxy-nsq-worker

Projects that are alternatives of or similar to nsqproxy

gobroker
golang wrapper for all (to-be) kinds of message brokers
Stars: ✭ 15 (-81.01%)
Mutual labels:  nsq, nsq-client
nsqclient-php
Yet another PHP client for NSQ
Stars: ✭ 19 (-75.95%)
Mutual labels:  nsq
Fastcgipp
fastcgi++: A C++ FastCGI and Web development platform:
Stars: ✭ 236 (+198.73%)
Mutual labels:  fastcgi
go-graphql-subscription-example
☝️ go-graphql subscription over Kafka/Redis/NSQ example
Stars: ✭ 34 (-56.96%)
Mutual labels:  nsq
public
util toolkit for go.golang 通用函数包
Stars: ✭ 135 (+70.89%)
Mutual labels:  nsq
verticegateway
REST API server with built in auth, interface to ScyllaDB/Cassandra
Stars: ✭ 25 (-68.35%)
Mutual labels:  nsq
Kcgi
minimal CGI and FastCGI library for C/C++
Stars: ✭ 194 (+145.57%)
Mutual labels:  fastcgi
pan
Pan is a high performance mq proxy,support kafka,rabbit-mq,rocketmq,nsq and other mq
Stars: ✭ 27 (-65.82%)
Mutual labels:  nsq
fcgi-function
A cross-platform module to writing C/C++ service for nginx.
Stars: ✭ 33 (-58.23%)
Mutual labels:  fastcgi
Go Nsq
The official Go package for NSQ
Stars: ✭ 2,045 (+2488.61%)
Mutual labels:  nsq
Nsq
A realtime distributed messaging platform
Stars: ✭ 20,663 (+26055.7%)
Mutual labels:  nsq
go-nsq
No description or website provided.
Stars: ✭ 57 (-27.85%)
Mutual labels:  nsq
funboost
pip install funboost，python全功能分布式函数调度框架,。支持python所有类型的并发模式和全球一切知名消息队列中间件，python函数加速器，框架包罗万象，一统编程思维，兼容50% python编程业务场景，适用范围广。只需要一行代码即可分布式执行python一切函数。旧名字是function_scheduling_distributed_framework
Stars: ✭ 351 (+344.3%)
Mutual labels:  nsq
http2fcgi
a quick & tiny http to fast-cgi reverse proxy, let's serve our php,python ... etc apps with no hassle!
Stars: ✭ 57 (-27.85%)
Mutual labels:  fastcgi
lua-resty-nsq
lua-resty-nsq - Lua nsq client driver for the ngx_lua based on the cosocket API
Stars: ✭ 18 (-77.22%)
Mutual labels:  nsq
Criollo
A powerful Cocoa web framework and HTTP server for macOS, iOS and tvOS.
Stars: ✭ 229 (+189.87%)
Mutual labels:  fastcgi
go-starter
Go 服务框架脚手架. 整合 echo、swag、viper、nsq、logrus、fx、xorm、cobra 等第三方库
Stars: ✭ 145 (+83.54%)
Mutual labels:  nsq
nsq-0.3.7
nsq注释版基于0.3.7版本
Stars: ✭ 45 (-43.04%)
Mutual labels:  nsq
gofcgi
golang client for fastcgi
Stars: ✭ 14 (-82.28%)
Mutual labels:  fastcgi
nsqworker
A skeletal guide of one way to implement a nsq consumer
Stars: ✭ 43 (-45.57%)
Mutual labels:  nsq
View All Similar Projects ➔

NSQProxy

NSQProxy是Golang开发的NSQ和Worker之间的中间件，根据数据库配置，负责消息转发。NSQProxy启动后，接受NSQD队列内容，然后通过HTTP/FastCGI/CBNSQ等协议转发给Worker机执行。在唱吧内部使用2年，高效稳定的处理着每日数十亿条消息。

go report card MIT license Downloads Release

解决的问题

  • 各Topic执行机器可配
  • 各Topic消费速度可配
  • 各Worker机协议可配
    • HTTP：将消息发送给配好的URL。
    • FastCGI：将消息发送给配置的服务端，如PHP-FPM。
    • CBNSQ：自定义的基于TCP的文本协议。
  • 可视化界面管理
  • 队列积压超出阈值报警
  • 散乱在各处的消费者集中化管理
  • 通过网络分发，无需安装.so等扩展库，因此无需修改线上环境

有图有真相

流程图

消费者管理

worker机管理

使用

请先部署好NSQLookupd、NSQd、MySQL

启动NSQLookupd nsqlookupd -broadcast-address="0.0.0.0" -http-address="0.0.0.0:4161" -tcp-address="0.0.0.0:4160"

启动NSQD nsqd -broadcast-address="0.0.0.0" -lookupd-tcp-address="0.0.0.0:4160" -tcp-address="0.0.0.0:4150" -http-address="0.0.0.0:4151"

启动MySQL

安装

二进制安装

  • 下载最新版本的压缩包 https://github.com/changba/nsqproxy/releases
  • 解压
  • 启动（注意替换为自己的MySQL信息） ./nsqproxy -dbHost=127.0.0.1 -dbPort=3306 -dbUsername=root -dbPassword=rootpsd -dbName=nsqproxy -logLevel=debug -nsqlookupdHTTP=127.0.0.1:4161
  • 命令行 curl http://0.0.0.0:19421/status 输出ok
  • 浏览器打开 http://0.0.0.0:19421/admin

源码安装

  • 要求Go1.13及以上
  • 下载本项目 go get github.com/changba/nsqproxy
  • cd nsqproxy
  • export GO111MODULE=on
  • 编译 make build
  • 启动（注意替换为自己的MySQL信息） ./bin/nsqproxy -dbHost=127.0.0.1 -dbPort=3306 -dbUsername=root -dbPassword=rootpsd -dbName=nsqproxy -logLevel=debug -nsqlookupdHTTP=127.0.0.1:4161
  • 命令行 curl http://0.0.0.0:19421/status 输出ok
  • 浏览器打开 http://0.0.0.0:19421/admin

快速开始

二次开发

前端

使用VUE开发，所有源码均在/web/vue-admin目录中，开发完成后需要编译，编译后的文件存放在/web/public/目录中。使用开源项目statik将静态文件/web/public/变成一个go文件internal/statik/statik.go，这样前端的静态文件也会被我们编译到同一个二进制文件中了。

  • 启动go服务 make run
  • 安装VUE make vue-install（如果国内被墙可以使用淘宝的源进行安装：make vue-install-taobao）
  • 开启VUE开发环境 make vue-dev
  • 浏览器打开 http://0.0.0.0:9528/admin
  • 开发前端相关功能
  • 编译VUE make vue-build
  • 前段文件转换为一个go文件 make statik
  • 编译go服务 make build
  • 浏览器打开 http://0.0.0.0:19421/admin

接口文档

TODO LIST

  • 协议增加protobuf
  • 后台增加用户权限管理
  • 报警HOOK
  • 日志按天分割

License

© Changba.com, 2020~time.Now

Released under the MIT License

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].