GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → zhuangjinjin → piccolo

zhuangjinjin / piccolo

Licence: Apache-2.0 license
Netty4长连接网关

Programming Languages

java
68154 projects - #9 most used programming language
HTML
75241 projects
shell
77523 projects

Labels

rediskafkanettygatewaysnowflakeprometheuszookeepernetty4piccolonacos

Projects that are alternatives of or similar to piccolo

twjitm-core
采用Netty信息加载实现长连接实时通讯系统，客户端可以值任何场景，支持实时http通讯、webSocket通讯、tcp协议通讯、和udp协议通讯、广播协议等 通过http协议，rpc协议。 采用自定义网络数据包结构， 实现自定义网络栈。
Stars: ✭ 98 (+415.79%)
Mutual labels:  netty, zookeeper, netty4
Books Recommendation
程序员进阶书籍（视频），持续更新（Programmer Books）
Stars: ✭ 558 (+2836.84%)
Mutual labels:  netty, prometheus, zookeeper
ship-gate
Use SpringWebFlux,Netty and Nacos to build high performance,responsive API gateways.
Stars: ✭ 138 (+626.32%)
Mutual labels:  netty, gateway, nacos
rpc-spring-boot-starter
自定义rpc框架，支持Java序列化和protobuf序列化协议，多种负载均衡算法
Stars: ✭ 75 (+294.74%)
Mutual labels:  netty, zookeeper
eagle
Eagle分布式rpc调用，借助Zookeeper实现服务注册和发现，基于AQS实现高性能连接池，支持分布式追踪、监控、过载保护等配置。提供Spring和SpringBoot插件，方便与Spring和SpringBoot集成。
Stars: ✭ 77 (+305.26%)
Mutual labels:  netty, zookeeper
xmutca-rpc
Xmutca-rpc是一个基于netty开发的分布式服务框架，提供稳定高性能的RPC远程服务调用功能，支持注册中心，服务治理，负载均衡等特性，开箱即用。
Stars: ✭ 18 (-5.26%)
Mutual labels:  netty, nacos
codes-scratch-zookeeper-netty
zk + netty 实现集群节点文件同步服务
Stars: ✭ 29 (+52.63%)
Mutual labels:  netty, zookeeper
Xproxy
reverse proxy implement in java
Stars: ✭ 20 (+5.26%)
Mutual labels:  gateway, netty4
aquiver
🚀 The aquifer is a java web framework based on Java8 and netty
Stars: ✭ 38 (+100%)
Mutual labels:  netty, netty4
gateway
API proxy gateway base on netty and okHttp.
Stars: ✭ 21 (+10.53%)
Mutual labels:  netty, gateway
taotao-cloud-project
微服务开发脚手架,包括大数据模块、微服务模块、前端模块。基于Spring Cloud Alibaba的微服务架构。提供技术框架的基础能力的封装，减少开发工作，只关注业务,包含了工作以来的工作总结和技术沉淀
Stars: ✭ 76 (+300%)
Mutual labels:  zookeeper, nacos
itstack-naive-chat-server
💞 《服务端》| 服务端同样使用Netty4.x作为socket的通信框架，同时在服务端使用Layui作为管理后台的页面，并且我们的服务端采用偏向于DDD领域驱动设计的方式与Netty集合，以此来达到我们的框架结构整洁干净易于扩展。同时我们的通信协议也是在服务端进行定义的，并对外提供可引入的Jar包，这样来保证客户端与服务端共同协议下进行通信。
Stars: ✭ 21 (+10.53%)
Mutual labels:  netty, netty4
Mini Rpc
Spring + Netty + Protostuff + ZooKeeper 实现了一个轻量级 RPC 框架，使用 Spring 提供依赖注入与参数配置，使用 Netty 实现 NIO 方式的数据传输，使用 Protostuff 实现对象序列化，使用 ZooKeeper 实现服务注册与发现。使用该框架，可将服务部署到分布式环境中的任意节点上，客户端通过远程接口来调用服务端的具体实现，让服务端与客户端的开发完全分离，为实现大规模分布式应用提供了基础支持
Stars: ✭ 205 (+978.95%)
Mutual labels:  netty, zookeeper
Thunder
⚡️ Nepxion Thunder is a distribution RPC framework based on Netty + Hessian + Kafka + ActiveMQ + Tibco + Zookeeper + Redis + Spring Web MVC + Spring Boot + Docker 多协议、多组件、多序列化的分布式RPC调用框架
Stars: ✭ 204 (+973.68%)
Mutual labels:  netty, zookeeper
toy-rpc
Java基于Netty,Protostuff和Zookeeper实现分布式RPC框架
Stars: ✭ 55 (+189.47%)
Mutual labels:  netty, zookeeper
Netty 4 User Guide
Chinese translation of Netty 4.x User Guide. 中文翻译《Netty 4.x 用户指南》
Stars: ✭ 2,061 (+10747.37%)
Mutual labels:  netty, netty4
Mango
A high-performance, open-source java RPC framework.
Stars: ✭ 150 (+689.47%)
Mutual labels:  netty, zookeeper
distributed-id
基于netty4+twitter-snowFlake分布式Id生成之服务实现
Stars: ✭ 18 (-5.26%)
Mutual labels:  netty, snowflake
Fpush
📶即时消息推送服务(即时通讯)，基于Netty+protobuf-- Instant Messaging push service based on Netty+protobuf
Stars: ✭ 134 (+605.26%)
Mutual labels:  netty, netty4
Nettyrpc
A simple RPC framework based on Netty, ZooKeeper and Spring
Stars: ✭ 1,975 (+10294.74%)
Mutual labels:  netty, zookeeper
View All Similar Projects ➔

piccolo

License img

简介

piccolo是一个长连接网关服务器。

架构图

技术

项目主要是基于Netty4.x开发的。

  • Jdk 8+
  • Netty 4.1.45
  • ZooKeeper 3.5.5
  • Kafka 2.0.0
  • RocketMQ 4.9.1
  • Nacos 1.1.3
  • Redis
  • Prometheus
  • Apache Commons
  • Guava
  • Fastjson
  • Protobuf

模块说明

模块名称 用途
piccolo-api API接口模块（包含缓存，配置，传输，负载均衡，消息队列，路由，插件等抽象接口）
piccolo-cache 缓存模块（目前仅支持redis缓存）
piccolo-client 客户端模块（供网关服务器的下游业务系统调用）
piccolo-commons 通用组件模块（包含事件总线，负载均衡实现，自定义消息，安全，线程池）
piccolo-config 配置模块（配置松散绑定）
piccolo-configcenter 配置中心 (目前仅支持Nacos)
piccolo-core 服务端核心处理模块
piccolo-distribution 发布模块
piccolo-monitor 监控模块（基于JMX，监控系统CPU负载，系统内存，JVM内存，线程池，JVM GC等）
piccolo-mq 消息模块（目前仅支持Kafka）
piccolo-registry 服务注册/发现模块（目前仅支持Nacos，ZooKeeper）
piccolo-server 服务端模块
piccolo-transport 传输模块（基于Netty 4开发，可配置粗粒度的流控，自定义流式解析）

使用

Server端

Step1: GitHub下载源代码并编译

git clone https://github.com/zhuangjinjin/piccolo.git

Step2: 构建项目

cd piccolo
mvn clean install -Dmaven.test.skip=true -P release-server
export PICCOLO_HOME=${piccolo_path}/piccolo-distribution/target/piccolo/piccolo

Step3: 修改配置文件

复制$PICCOLO_HOME/conf/piccolo-server.properties.example并重新命名为piccolo-server.properties，修改配置如下：

piccolo.redis.host=${redis.host}
piccolo.zookeeper.host=${zookeeper.host}
piccolo.kafka.producer.bootstrap-servers=${kafka.bootstrap-servers}
piccolo.kafka.consumer.bootstrap-servers=${kafka.bootstrap-servers}
piccolo.kafka.admin-client.bootstrap-servers=${kafka.bootstrap-servers}
piccolo.nacos.server-address=${nacos.server-addr}
piccolo.nacos.endpoint=${nacos.endpoint}

Step4: 启动网关服务器

cd $PICCOLO_HOME
sh bin/startup.sh -m standalone

运行成功后会出现piccolo的LOGO

Client端

Step1:GitHub下载源代码并编译

git clone https://github.com/zhuangjinjin/piccolo.git

Step2: 构建项目

cd piccolo
mvn clean install -Dmaven.test.skip=true

Step3: 在项目pom.xml引入依赖

<dependency>
  <groupId>io.github.ukuz</groupId>
  <artifactId>piccolo-client</artifactId>
  <version>0.1.0</version>
</dependency>

注册消息路由

RouteLocatorBuilder.routes()
  .route("/foo1", "bar")
  .route("/foo2", "bar");

注册消息回调

PushClient pushClient = new PushClient();
pushClient.registerHandler("bar", (dispatchMessage) -> {
  //todo some logic process
  ...
  //manual ack mode
  dispatchMessage.completeConsume();
});

消息回发

PushClient pushClient = new PushClient();
PushContext ret = PushContext.builder().userId("user-0").context("msg".getBytes()).build();
pushClient.push(ret);

消息广播

PushClient pushClient = new PushClient();
PushContext ret = PushContext.builder().context("msg".getBytes()).build();
pushClient.push(ret);

分布式ID

IdGen idGen = new IdGenBuilder().build();
long xid = idGen.genXid();

配置

服务端配置

在服务端conf目录下新建一个piccolo-server.properties，配置可分如下几类：

网络配置

属性名 描述
piccolo.net.ws-path WebSocket的根路径
piccolo.net.connect-server.bind-port ConnectServer的端口
piccolo.net.connect-server.register-ip ConnectServer的IP(可选填)
piccolo.net.connect-server.snd-buf ConnectServer套接字的发送缓存区的大小(默认32K)
piccolo.net.connect-server.rcv-buf ConnectServer套接字的接收缓存区的大小(默认32K)
piccolo.net.connect-server.write-water-mark-low ConnectServer的写低水位(默认128K)
piccolo.net.connect-server.write-water-mark-high ConnectServer的写高水位(默认256K)
piccolo.net.connect-server-traffic.enabled ConnectServer的流控开关(默认开启)
piccolo.net.connect-server-traffic.check-interval-ms ConnectServer流控的检测间隔时间(默认1000ms)
piccolo.net.connect-server-traffic.write-global-limit ConnectServer所有连接的在检测间隔时间内写的流量上限(单位Byte)
piccolo.net.connect-server-traffic.read-global-limit ConnectServer所有连接的在检测间隔时间内读的流量上限(单位Byte)
piccolo.net.connect-server-traffic.write-channel-limit ConnectServer单个连接的在检测间隔时间内写的流量上限(单位Byte)
piccolo.net.connect-server-traffic.read-channel-limit ConnectServer单个连接的在检测间隔时间内读的流量上限(单位Byte)
piccolo.net.ws-server.bind-port
piccolo.net.ws-server.register-ip
piccolo.net.ws-server.snd-buf
piccolo.net.ws-server.rcv-buf
piccolo.net.ws-server.write-water-mark-low
piccolo.net.ws-server.write-water-mark-high
piccolo.net.gateway-server.bind-port
piccolo.net.gateway-server.register-ip
piccolo.net.gateway-server.snd-buf
piccolo.net.gateway-server.rcv-buf
piccolo.net.gateway-server.write-water-mark-low
piccolo.net.gateway-server.write-water-mark-high
piccolo.net.gateway-server-traffic.enabled
piccolo.net.gateway-server-traffic.check-interval-ms
piccolo.net.gateway-server-traffic.write-global-limit
piccolo.net.gateway-server-traffic.read-global-limit
piccolo.net.gateway-server-traffic.write-channel-limit
piccolo.net.gateway-server-traffic.read-channel-limit

核心配置

属性名 描述
piccolo.core.epoll-provider 填netty表示使用epoll， 填nio表示使用nio
piccolo.core.max-heartbeat-time
piccolo.core.min-heartbeat-time
piccolo.core.max-hb-timeout-ms
piccolo.core.max-packet-size
piccolo.core.session-expire-time

线程配置

属性名 描述
piccolo.thread.connect-worker-thread-num
piccolo.thread.gateway-worker-thread-num
piccolo.thread.ws-worker-thread-num

线程池配置

属性名 描述
piccolo.thread-pool.event-bus.core-size
piccolo.thread-pool.event-bus.max-size
piccolo.thread-pool.event-bus.queue-size
piccolo.thread-pool.event-bus.keep-alive-seconds
piccolo.thread-pool.mq.core-size
piccolo.thread-pool.mq.max-size
piccolo.thread-pool.mq.queue-size
piccolo.thread-pool.mq.keep-alive-seconds
piccolo.thread-pool.id-gen.core-size
piccolo.thread-pool.id-gen.max-size
piccolo.thread-pool.id-gen.queue-size
piccolo.thread-pool.id-gen.keep-alive-seconds

Redis配置

属性名 描述
piccolo.redis.host
piccolo.redis.mode
piccolo.redis.database
piccolo.redis.sentinel-master
piccolo.redis.password
piccolo.redis.pool-config.test-while-idle
piccolo.redis.pool-config.test-on-borrow
piccolo.redis.pool-config.min-evictable-idle-time-millis
piccolo.redis.pool-config.time-between-eviction-runs-millis
piccolo.redis.pool-config.num-tests-per-eviction-run

ZooKeeper配置

属性名 描述
piccolo.zookeeper.host
piccolo.zookeeper.ns
piccolo.zookeeper.session-timeout-ms
piccolo.zookeeper.connection-timeout-ms
piccolo.zookeeper.digest
piccolo.zookeeper.max-retries
piccolo.zookeeper.base-sleep-time-ms
piccolo.zookeeper.max-sleep-ms

Kafka配置

属性名 描述
piccolo.kafka.producer.bootstrap-servers
piccolo.kafka.producer.linger-ms
piccolo.kafka.producer.batch-size
piccolo.kafka.producer.compress-type
piccolo.kafka.producer.max-in-flight-requests-per-connection
piccolo.kafka.producer.acks
piccolo.kafka.producer.retries
piccolo.kafka.producer.retries-backoff-ms
piccolo.kafka.producer.request-timeout-ms
piccolo.kafka.consumer.bootstrap-servers
piccolo.kafka.consumer.group-id
piccolo.kafka.consumer.auto-offset-reset
piccolo.kafka.consumer.session-timeout
piccolo.kafka.consumer.max-poll-interval-ms
piccolo.kafka.consumer.enable-auto-commit
piccolo.kafka.consumer.auto-commit-interval-ms
piccolo.kafka.consumer.fetch-max-bytes
piccolo.kafka.consumer.max-poll-records
piccolo.kafka.consumer.heartbeat-interval-ms
piccolo.kafka.consumer.connections-interval-ms
piccolo.kafka.admin-client.bootstrap-servers

Nacos配置

属性名 描述
piccolo.nacos.server-address
piccolo.nacos.endpoint
piccolo.nacos.namespace
piccolo.nacos.access-key
piccolo.nacos.secret-key

消息队列配置

属性名 描述
piccolo.mq-topic.dispatch.num-partitions
piccolo.mq-topic.dispatch.replication-factor
piccolo.mq-topic.online.num-partitions
piccolo.mq-topic.online.replication-factor
piccolo.mq-topic.offline.num-partitions
piccolo.mq-topic.offline.replication-factor

下游业务系统消息队列配置

属性名 描述
piccolo.mq-topic.routes[0].topic
piccolo.mq-topic.routes[0].num-partitions
piccolo.mq-topic.routes[0].replication-factor

安全配置

属性名 描述
piccolo.security.public-key
piccolo.security.private-key

客户端配置

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].