All Projects → hhu-bsinfo → dxram

hhu-bsinfo / dxram

Licence: GPL-3.0 license
A distributed in-memory key-value storage for billions of small objects.

Programming Languages

java
68154 projects - #9 most used programming language
shell
77523 projects

Projects that are alternatives of or similar to dxram

Hazelcast
Open-source distributed computation and storage platform
Stars: ✭ 4,662 (+18548%)
Mutual labels:  big-data, distributed, in-memory, low-latency
Hazelcast Go Client
Hazelcast IMDG Go Client
Stars: ✭ 140 (+460%)
Mutual labels:  big-data, distributed, in-memory
Hazelcast Cpp Client
Hazelcast IMDG C++ Client
Stars: ✭ 67 (+168%)
Mutual labels:  big-data, distributed, in-memory
Hazelcast Python Client
Hazelcast IMDG Python Client
Stars: ✭ 92 (+268%)
Mutual labels:  big-data, distributed, in-memory
cachegrand
cachegrand is an open-source fast, scalable and secure Key-Value store, also fully compatible with Redis protocol, designed from the ground up to take advantage of modern hardware vertical scalability, able to provide better performance and a larger cache at lower cost, without losing focus on distributed systems.
Stars: ✭ 87 (+248%)
Mutual labels:  distributed, low-latency, key-value-store
Hazelcast Nodejs Client
Hazelcast IMDG Node.js Client
Stars: ✭ 124 (+396%)
Mutual labels:  big-data, distributed, in-memory
insightedge
InsightEdge Core
Stars: ✭ 22 (-12%)
Mutual labels:  big-data, distributed, in-memory
Olric
Distributed cache and in-memory key/value data store. It can be used both as an embedded Go library and as a language-independent service.
Stars: ✭ 2,067 (+8168%)
Mutual labels:  distributed, in-memory, key-value-store
hazelcast-csharp-client
Hazelcast .NET Client
Stars: ✭ 98 (+292%)
Mutual labels:  big-data, distributed, in-memory
Titanoboa
Titanoboa makes complex workflows easy. It is a low-code workflow orchestration platform for JVM - distributed, highly scalable and fault tolerant.
Stars: ✭ 787 (+3048%)
Mutual labels:  big-data, distributed
Hazelcast Jet
Distributed Stream and Batch Processing
Stars: ✭ 855 (+3320%)
Mutual labels:  big-data, low-latency
H2o 3
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
Stars: ✭ 5,656 (+22524%)
Mutual labels:  big-data, distributed
Scanner
Efficient video analysis at scale
Stars: ✭ 569 (+2176%)
Mutual labels:  big-data, distributed
Halodb
A fast, log structured key-value store.
Stars: ✭ 370 (+1380%)
Mutual labels:  big-data, key-value-store
Attaca
Robust, distributed version control for large files.
Stars: ✭ 41 (+64%)
Mutual labels:  big-data, distributed
Arcus
ARCUS is the NAVER memcached with lists, sets, maps and b+trees. http://naver.github.io/arcus
Stars: ✭ 273 (+992%)
Mutual labels:  memcached, key-value-store
Social-Network-Analysis-in-Python
Social Network Facebook Analysis (Python, Networkx)
Stars: ✭ 26 (+4%)
Mutual labels:  big-data, social-network
Cucache
Fast PUT/GET/DELETE in-memory key-value store for lookaside caching
Stars: ✭ 63 (+152%)
Mutual labels:  memcached, in-memory
soundstorm
The Federated Social Audio Platform
Stars: ✭ 26 (+4%)
Mutual labels:  social-network, distributed
Crate
CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of data in real-time.
Stars: ✭ 3,254 (+12916%)
Mutual labels:  big-data, distributed

DXRAM: A distributed in-memory key-value storage

Build Status Build Status

Developed by the operating systems group of the department of computer science of the Heinrich-Heine-University Düsseldorf, DXRAM is a distributed in-memory key-value store for low-latency cloud applications, e.g. social networks, search engines and I/O-bound long running scientific computations running in a data center. It is designed to efficiently manage billions of small data objects which are typical for graph based applications.

For example, Facebook runs approx. 1,000 memcached servers keeping around 75% of its data always in RAM because back-end database accesses are too slow. We argue that it is possible to go one step further by keeping all data always in RAM relieving developers from cache management and synchronization of caches with secondary storage.

Important

DXRAM is a research project that's under development. We do not recommend using the system in a production environment or with production data without having an external backup. Expect to encounter bugs. However, we are looking forward to bug reports and code contributions.

Features

  • Distributed object lookup and scalable meta-data management with super-peer overlay
  • Optimized for up to 10 GBit/s Ethernet network and up to 56 GBit/s InfiniBand
  • All objects always in RAM
  • Custom, highly efficient and performant memory management for billions of small objects (< 64 bytes) per node
  • Smart asynchronous logging to disk (optimized for SSD) for persistency
  • Replication of logs to remote nodes for fault tolerance
  • Crash-recovery failure-model
  • Object migrations for storage nodes to handle hot spots
  • Run computations on storage nodes using lightweight jobs or tasks submitted to coordinated compute groups

Contents

License

Copyright (C) 2018 Heinrich-Heine-Universitaet Duesseldorf, Institute of Computer Science, Department Operating Systems. Licensed under the GNU General Public License.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].