All Projects → yejiayu → Cluster Dispatch

yejiayu / Cluster Dispatch

解决Node.js在Cluster模式下的连接/资源复用问题

Programming Languages

javascript
184084 projects - #8 most used programming language

Projects that are alternatives of or similar to Cluster Dispatch

node-advanced
Node Advanced Courseware
Stars: ✭ 80 (+515.38%)
Mutual labels:  cluster, node-module
Kudo
Kubernetes Universal Declarative Operator (KUDO)
Stars: ✭ 849 (+6430.77%)
Mutual labels:  cluster
Open C Book
开源书籍:《C语言编程透视》,配套视频课程《360° 剖析 Linux ELF》已上线,视频讲解更为系统和深入,欢迎订阅:https://www.cctalk.com/m/group/88089283
Stars: ✭ 715 (+5400%)
Mutual labels:  process
Horde
Horde is a distributed Supervisor and Registry backed by DeltaCrdt
Stars: ✭ 834 (+6315.38%)
Mutual labels:  cluster
Hmq
High performance mqtt broker
Stars: ✭ 722 (+5453.85%)
Mutual labels:  cluster
Cv4pve Api Java
Proxmox VE Client API JAVA
Stars: ✭ 17 (+30.77%)
Mutual labels:  cluster
Immortal
⭕ A *nix cross-platform (OS agnostic) supervisor
Stars: ✭ 701 (+5292.31%)
Mutual labels:  process
Docx Builder
NPM Module for creating or merging .docx files
Stars: ✭ 11 (-15.38%)
Mutual labels:  node-module
Blast
Blast is a full text search and indexing server, written in Go, built on top of Bleve.
Stars: ✭ 934 (+7084.62%)
Mutual labels:  cluster
Gonet
go分布式服务器,基于内存mmo
Stars: ✭ 804 (+6084.62%)
Mutual labels:  cluster
Fkill Cli
Fabulously kill processes. Cross-platform.
Stars: ✭ 6,418 (+49269.23%)
Mutual labels:  process
V2ray Web Manager
v2ray-web-manager 是一个v2ray的面板,也是一个集群的解决方案;同时增加了流量控制/账号管理/限速等功能。key: admin , panel ,web,cluster,集群,proxy
Stars: ✭ 738 (+5576.92%)
Mutual labels:  cluster
Orcinus
Agnostic Orchestration Tools
Stars: ✭ 17 (+30.77%)
Mutual labels:  cluster
Iodine
iodine - HTTP / WebSockets Server for Ruby with Pub/Sub support
Stars: ✭ 720 (+5438.46%)
Mutual labels:  cluster
Process
The Process component executes commands in sub-processes.
Stars: ✭ 6,942 (+53300%)
Mutual labels:  process
Conf
Simple config handling for your app or module
Stars: ✭ 707 (+5338.46%)
Mutual labels:  node-module
Kubernetes Gpu Guide
This guide should help fellow researchers and hobbyists to easily automate and accelerate there deep leaning training with their own Kubernetes GPU cluster.
Stars: ✭ 740 (+5592.31%)
Mutual labels:  cluster
Zato
ESB, SOA, REST, APIs and Cloud Integrations in Python
Stars: ✭ 889 (+6738.46%)
Mutual labels:  cluster
Clusterws
💥 Lightweight, fast and powerful framework for building scalable WebSocket applications in Node.js
Stars: ✭ 868 (+6576.92%)
Mutual labels:  cluster
Terra Aws Core Kube
Terraform configuration to bootstrap a Kubernetes Cluster on top of CoreOS using AWS-EC2 instances
Stars: ✭ 10 (-23.08%)
Mutual labels:  cluster

cluster-dispatch

解决Node.js在Cluster模式下的连接/资源复用问题

NPM version build status Test coverage David deps node version npm download npm license

你会不会有这么一种场景, 比如创建一个TCP连接并且监听一个configserver的配置变动, 当变动的时候同时调整配置, 当你的应用运行在cluster模式下这个TCP连接就会是N个, 但是你并不需要N个因为浪费系统和你的服务方资源, 你只需要有一个TCP连接就够了.

这个模块就是解决这个问题诞生的.

他用了这样一个进程模型

        Master  -> fork Library
          |           |       
          |         Agent
       Cluster        |
  Wokrer, Wokrer, Woker....

你可以把需要复用的资源放在Library进程中, Agent来负责跨进程的调用

Installation

$ npm i cluster-dispatch --save

Example

$ git clone

$ npm i

$ npm run example

API

Master

master作为守护进程也是应用的入口, 负责启动appWokrer和libraryWorker

const Master = require('cluster-dispatch').Master;

/*
  * @param {Object} options
  *   - {String} baseDir - 工程根路径
  *   - {String} appPath - app进程入口文件, 可以是一个相对路径
  *   - {String} libraryPath - 第三方库入口文件, 具体参照exmaple/lib/index.js, 可以是一个相对路径
  *   - {Number} appWorkerCount - 需要启动的app进程数
  *   - {Funcion} logging - log
  *   - {Boolean} needLibrary - 是否需要启动library进程
  *   - {Boolean} needAgent - 是否需要自动启动代理
 */
const master = new Master({
  baseDir: __dirname,
  logging: debug,
  appWorkerCount,
});

(async function initMaster() {
  await master.init(); // 初始化

  master.on('error', error => debug(error));
})().catch(debug)

Agent

agent负责跨进程的调度, 这个agent实例每个appWorker都有, 默认会在appWorker初始化时创建好

在appWorker中通过Agent.getAgent()就可以拿到agent实例

// agent/index.js
'use strict';

const Agent = require('cluster-dispatch').Agent;

module.exports = Agent.getAgent();

通过agent调用和正常调用是一样的, 只是得通过agent对象调用

eg

// agent/lib/demo.js
module.exports = {
  getName() {
    return new Promise((resolve) => {
      resolve('get name')
    });
  }
}

// index.js
const agent = require('./agent')

(async function eg() {
  const name = await agent.demo.getName()
  console.log(name) // "get name"
})().catch(console.error)

不过因为是跨进程的所以任何调用都是异步的, 即使你调用的原本是个同步方法

启动流程

  1. 启动master进程
  2. master fork 出 library worker
  3. library worker 根据实例化master传递的libraryPath参数拿到下面对象签名, 然后触发ready事件给master
  4. master接到ready事件后开始已cluster模式启动app worker
  5. app worker初始化完成后会向library worker请求对象签名, 拿到对象签名后开始初始化agent
  6. agent遍历对象签名, 并且把对象的每个key重写, 类似这样的逻辑
agent.invoke = function(objName, methodName, ...rest) {
  return client.setTo('Library').send({
    objName,
    methodName,
    args: rest,
  })
}

等于拿到你的调用方法后, 把参数原样返回回去, 实际执行还是在library worker中

  1. done
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].