All Projects → UncP → Mushroom

UncP / Mushroom

Licence: other
Distributed In-Memory Index 分布式内存索引

Labels

Projects that are alternatives of or similar to Mushroom

FISCO-BCOS
FISCO BCOS是由微众牵头的金链盟主导研发、对外开源、安全可控的企业级金融区块链底层技术平台。 单链配置下,性能TPS可达万级。提供群组架构、并行计算、分布式存储、可插拔的共识机制、隐私保护算法、支持全链路国密算法等诸多特性。 经过多个机构、多个应用,长时间在生产环境中的实践检验,具备金融级的高性能、高可用性及高安全性。FISCO BCOS is a secure and reliable financial-grade open-source blockchain platform. The platform provides rich features including group architecture, cross-chain communication protoc…
Stars: ✭ 1,603 (+472.5%)
Mutual labels:  raft, rpc
Bifrost
Pure rust building block for distributed systems
Stars: ✭ 118 (-57.86%)
Mutual labels:  rpc, raft
awesome-list-rpc-nodes-providers
A curated list of awesome Node providers and public PRC endpoints for Ethereum, BSC, xDAI, Fantom, Avalanche and more :)
Stars: ✭ 39 (-86.07%)
Mutual labels:  rpc
Permazen
Language-Natural Persistence Layer for Java
Stars: ✭ 265 (-5.36%)
Mutual labels:  raft
Rpcx Examples
examples for the latest rpcx
Stars: ✭ 256 (-8.57%)
Mutual labels:  rpc
wislove
Java 服务端api解决方案,处理部分通用业务逻辑。包括用户中心,支付,订单,IM相关,内容管理,角色权限授权。以及分库分表的处理,RPC分布式的解决方案等。一个人慢慢做
Stars: ✭ 14 (-95%)
Mutual labels:  rpc
Tarsgo
Tarsgo is high performance RPC framework in Golang programing language using the tars protocol. Go has become popular for programming with the rise of containerization technology such as docker, k8s, and etcd. Go's goroutine concurrency mechanism means Go is very suitable for large-scale high-concurrency back-end server program development. The Go language has nearly C/C++ performance and near Python productivity. In Tencent, part of the existing C++ development team has gradually turned into Go developers. Tars, a widely used RPC framework, supports C++, Java, NodeJS, and PHP, and now Go. The combination with Go language has become a general trend. Therefore, in the voice of users, we launched Tarsgo, and we have applied to Tencent map application, YingYongbao application, Internet plus and other projects. Learn more about the whole Tars architecture and design at Introduction.
Stars: ✭ 2,931 (+946.79%)
Mutual labels:  rpc
raft-rocks
A simple database based on raft and rocksdb
Stars: ✭ 38 (-86.43%)
Mutual labels:  raft
Shineframe
高性能超轻量级C++开发库及服务器编程框架
Stars: ✭ 274 (-2.14%)
Mutual labels:  raft
Paxos raft protocol
分布式一致性协议相关论文及中文译文,涵盖Paxos、Raft、Zab
Stars: ✭ 255 (-8.93%)
Mutual labels:  raft
Erpc
Embedded RPC
Stars: ✭ 262 (-6.43%)
Mutual labels:  rpc
idealingua-v1
IdeaLingua RPC for Scala, TypeScript, C#, Go
Stars: ✭ 13 (-95.36%)
Mutual labels:  rpc
quanta
A Game Server Engine based on Lua!
Stars: ✭ 17 (-93.93%)
Mutual labels:  rpc
Floyd
A raft consensus implementation that is simply and understandable
Stars: ✭ 259 (-7.5%)
Mutual labels:  raft
transmission-rpc-java
Java implementation of the Transmission RPC API.
Stars: ✭ 33 (-88.21%)
Mutual labels:  rpc
Mu Scala
Mu is a purely functional library for building RPC endpoint based services with support for RPC and HTTP/2
Stars: ✭ 266 (-5%)
Mutual labels:  rpc
Jazminserver
Java based application,rpc,message,rtmp,game,sip,rtp,relay,webrtc,webssh,proxy server,message queue,mysql proxy server
Stars: ✭ 256 (-8.57%)
Mutual labels:  rpc
kerala
Distributed KV Streams
Stars: ✭ 16 (-94.29%)
Mutual labels:  raft
Kotlin Cn
【已下线】https://discuss.kotliner.cn 的第一个实验版本,尝试使用Kotlin编写构建的 Kotlin China 论坛,etcd+自研tpc协议RPC
Stars: ✭ 276 (-1.43%)
Mutual labels:  rpc
Saluki
Spring Boot starter module for gRPC framework.
Stars: ✭ 267 (-4.64%)
Mutual labels:  rpc

这个 repo 已弃用,请移步 aili 查看最新进展


Mushroom(蘑菇):分布式内存索引

English Version of README

Author Version Build License

Mushroom是一个C++11编写、不依赖第三方库的轻量级Linux环境分布式内存索引,它由并发Blink树索引,TCP通信库,RPC框架,Raft一致性算法组成。

Behold, the power of Mushroom!

尝试

首先运行脚本gen_test_data.sh生成测试数据,然后进入src目录
./run blink queue 100,测试多线程索引(通过队列),数量可以是1到1000万
./run blink thread 100,测试多线程索引,数量可以是1到1000万
./run art 100,测试Adaptive Radix Tree,数量可以是1到1000万
【失效】./run index 100,单机测试分布式索引,100是索引数量(因为Raft没有深入优化过,所以请将数量控制在1-1000)

版本信息

版本 备注
0.1.0
0.2.0 读写锁并发索引
0.2.1 锁管理器优化
0.3.0 引入前缀压缩,Blink树占用内存减少约 9.1 %
0.4.0 实现映射队列,减少程序使用内存超过 50 %
0.4.1 合并锁管理器与页面管理器,使每次操作减少1把锁
0.4.2 修改根节点分裂方式
0.4.3 增加测试策略,多线程不经过队列直接进行任务
0.4.4 重构锁管理器
0.5.0 修复从版本0.4.1到0.4.4一直存在的bug(原子操作bug)
0.6.0 共享内存映射支持多进程,修复搜索bug,正确实现并发Blink
0.6.1 二次哈希页面管理器,实现页面的懒惰分配
0.6.2 减少对标准库的依赖,加快编译速度,减少程序体积约42.1%
0.6.4 使用posix自旋锁,优化MushroomDB和BLinkTree结构
0.7.0 日志结构合并树(LSM Tree)
0.8.0 TCP通信库、RPC框架
0.9.0 Raft
0.9.1 ACID (批操作)、修改加锁策略
0.9.2 优化Raft状态变化,提高Liveness
1.0.0 分布式内存索引
1.1.0 Blink树结点从1/2满提高到2/3满,降低内存使用率约12%
1.1.1 优化Raft内存使用(牺牲可读性)
1.2.0 Adaptive Radix Tree
1.2.1 移除锁管理器,每个页面一把锁,性能提升约10%
1.2.2 引入布隆过滤器进行读写优化(然并卵)

其他

  • 你可以在这个知乎专栏里找到Mushroom的介绍
  • 分布式索引只是完成了单机功能,我写完就放弃优化了,所以较新的版本里./run index这个命令并不支持

PS: Mushroom 是一次致力于获得极致存储性能的激进尝试,在阅读具体代码之前,请注意:

  1. 编程规范、代码可读性、内存使用、接口是否友好等等一切无条件为性能做出让步
  2. 但是核心 B link 树的并发算法实现很简洁
  3. 对于内存的使用很极端,比如映射队列,介绍文章以及
  4. 有一些指针的 Hack 操作,比如柔性数组
  5. 删除和插入都是写操作,所以只有 Put(写)和 Get(读)操作(即不支持物理删除)
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].