Top 35 distributed-storage open source projects

An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.

MySQL 学习笔记

A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

Low-Level Versioned Store

ChubaoFS (abbrev. CBFS) is a cloud native distributed file system and object store.

distributed key - value persisted storage system

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

A collection of Salt files for deploying, managing and automating Ceph.

Pravega is 100% open source and community-driven. All components are available under Apache 2 License on GitHub.

The LeoFS Storage System

Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

The distributed object storage server used by PitchPoint Solutions to securely store billions of large and small files using minimal resources. Object data is stored in replicated volumes implemented like Facebooks Haystack Object Store. Object metadata which essentially maps an object name to a volume position is stored in an elasticsearch index.

Kertish-dfs is a simple distributed storage platform, implements file storage on a single distributed computer cluster, and provides interfaces for file/folder handling. Kertish-dfs aims primarily for completely distributed operation without a single point of failure, scalable to the exabyte level.

Distributed "massively parallel" SQL query engine

File-system fault injection framework for distributed storage systems

MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

Distributed Storage System for QEMU

High-performance Distributed Storage

Raft Java implementation which is simple and easy to understand.

Awesome list of distributed systems resources

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

Dragonboat is a high performance multi-group Raft consensus library in pure Go.

Distributed object storage server from Douban Inc.

DAOS Storage Engine

Filestorage.js is client library for SKALE sidechain decentralized file storage. Used to transfer files between a browser and a SKALE sidechain.

DTC is a high performance Distributed Table Cache system designed by JD.com that offering hotspot data cache for databases in order to reduce pressure of database and improve QPS.

A decentralized (p2p) file storage system built atop Kademlia DHT that enforces data integrity, privacy, and availability through sharding, proofs of retrievability, redundancy, and encryption, with smart-contract powered incentive scheme

Distributed KV Storage System based on Raft and RocksDB, can be use to store small files, like images.

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

Distributed Message Queue based on Raft

IOTA meets BitTorrent: An algorithm to persist any file onto the tangle of IOTA

CubeFS is a cloud native distributed storage platform.

Mirror of official Lustre development repository http://git.whamcloud.com/

电子科技大学分布式存储与计算实验室2018新生学习资料整理

XML/Document DB on top of distributed cache