All Categories → Software Architecture → distributed-computing

Top 167 distributed-computing open source projects

Hydra

A light-weight library for building distributed applications such as microservices

✭ 611

javascript redis microservices backend distributed-computing services hydra

Finn

Fast Raft framework using the Redis protocol for Go

✭ 534

go golang redis distributed-computing raft

Thrill

Thrill - An EXPERIMENTAL Algorithmic Distributed Big Data Batch Processing Framework in C++

✭ 528

big-data distributed-computing

Fedml

A Research-oriented Federated Learning Library. Supporting distributed computing, mobile/IoT on-device training, and standalone simulation. Best Paper Award at NeurIPS 2020 Federated Learning workshop. Join our Slack Community:(https://join.slack.com/t/fedml/shared_invite/zt-havwx1ee-a1xfOUrATNfc9DFqU~r34w)

✭ 512

python machine-learning pytorch distributed-computing

Awesome Distributed Systems

Awesome list of distributed systems resources

✭ 512

awesome awesome-list distributed-systems distributed computer-science distributed-computing distributed-database distributed-storage

Atm

Auto Tune Models - A multi-tenant, multi-data system for automated machine learning (model selection and tuning).

✭ 504

python machine-learning data-science automl distributed-computing hyperparameter-optimization

Easylambda

distributed dataflows with functional list operations for data processing with C++14

✭ 475

cpp14 functional-programming parallel hpc distributed-computing mpi dataflow-programming

Couler

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

✭ 405

python machine-learning kubernetes cloud-native scheduler distributed-computing workflow-engine

Metorikku

A simplified, lightweight ETL Framework based on Apache Spark

✭ 361

scala sql spark big-data etl distributed-computing etl-framework

Protoactor Go

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

✭ 3,934

go cross-platform grpc protobuf clustering akka distributed-computing actor-model actors

Diplomat

A HTTP Ruby API for Consul

✭ 358

ruby api rails distributed-systems distributed api-client cluster consul distributed-computing key-value lock

Paracel

Distributed training framework with parameter server

✭ 335

machine-learning graph distributed-computing

Platon Go

Golang implementation of the PlatON protocol

✭ 331

go distributed-systems wasm distributed-computing distributed-ledger

Sleuth

A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services

✭ 331

go golang distributed-systems rpc distributed-computing discovery zeromq

Datafuse

Datafuse is a free Cloud-Native Analytics DBMS(Inspired by ClickHouse) implemented in Rust

✭ 327

rust database sql simd distributed-computing clickhouse parallelism olap

Fishnet

Distributed Stockfish analysis for lichess.org

✭ 306

rust distributed-computing chess

Awesome Federated Computing

📚 👓 A collection of research papers, codes, tutorials and blogs on Federated Computing/Learning.

✭ 314

deep-learning machine-learning awesome-list decentralized distributed-computing

Python Seminar

Python for Data Science (Seminar Course at UC Berkeley; AY 250)

✭ 302

python jupyter-notebook machine-learning data-science visualization distributed-computing

Federated Learning Pytorch

Implementation of Communication-Efficient Learning of Deep Networks from Decentralized Data

✭ 286

python deep-learning pytorch distributed-computing

Awesome Distributed Deep Learning

A curated list of awesome Distributed Deep Learning resources.

✭ 277

deep-learning machine-learning awesome awesome-list computer-vision nlp data-science natural-language-processing deep-neural-networks neural-networks gpu distributed-systems asynchronous distributed-computing hyperparameter-optimization

Tdigest

t-Digest data structure in Python. Useful for percentiles and quantiles, including distributed enviroments like PySpark

✭ 274

python distributed-computing pyspark mapreduce

Gleam

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

✭ 2,949

go Yacc distributed-systems distributed-computing map-reduce

Charm4py

Parallel Programming with Python and Charm++

✭ 259

python runtime hpc distributed-computing asynchronous-tasks

Hazelcast

Open-source distributed computation and storage platform

✭ 4,662

java FreeMarker shell HCL python c hacktoberfest big-data distributed caching scalability in-memory distributed-systems hazelcast distributed-computing stream-processing low-latency

data-parallelism

juliafolds.github.io/data-parallelism/

✭ 22

julia CSS HTML Makefile high-performance parallel distributed-computing map-reduce iterators transducers franklin

interbit

To the end of servers

✭ 23

javascript CSS HTML redux cryptography redux-saga functional-programming smart-contracts blockchain virtualization distributed-computing hyperconverged

SciFlow

Scientific workflow management

✭ 49

haskell distributed-computing workflow-management cloud-haskell computational-pipelines

mobius

Mobius is an AI infra platform including realtime computing and training.

✭ 22

java C++python cython Starlark c shell training machine-learning streaming realtime distributed-computing fusion

Awesome-Federated-Machine-Learning

Everything about federated learning, including research papers, books, codes, tutorials, videos and beyond

✭ 190

security machine-learning computer-vision deep-learning distributed-computing differential-privacy edge-computing federated-learning privacy-preserving-machine-learning

SadlyDistributed

Distributing your code(soul), in almost any language(state), among a cluster of idle browsers(voids)

✭ 20

javascript go Dockerfile HTML python C++Makefile websockets distributed-computing webassembly volunteer-computing

CipherCompute

The free EAP version of the Cosmian Collaborative Confidential Computing platform. Try it!

✭ 20

cryptography distributed-computing multiparty-computation confidential-computing

server

Hashtopolis - A Hashcat wrapper for distributed hashcracking

✭ 954

PHP HTML javascript CSS shell TeX distributed-computing hash distributed pentesting passwords hashes cracking hashcat hashlist hashtopussy hashtopolis

frovedis

Framework of vectorized and distributed data analytics

✭ 59

C++python scala assembly java c machine-learning spark scikit-learn mpi distributed-computing vectorization sx-aurora-tsubasa

bloomfilter

Bloomfilter written in Golang, includes rotation and RPC

✭ 61

go bloom-filter distributed-computing rpc crdt probabilistic-data-structures g-set

optimism-v2

ARCHIVE of monorepo implementing Boba, an L2 Compute solution built on Optimistic Ethereum - active repo is at https://github.com/bobanetwork/boba

✭ 34

go javascript typescript c solidity python ethereum distributed-computing

easyFL

An experimental platform to quickly realize and compare with popular centralized federated learning algorithms. A realization of federated learning algorithm on fairness (FedFV, Federated Learning with Fair Averaging, https://fanxlxmu.github.io/publication/ijcai2021/) was accepted by IJCAI-21 (https://www.ijcai.org/proceedings/2021/223).

✭ 104

python machine-learning deep-learning distributed-computing pytorch fairness federated-learning

PFL-Non-IID

The origin of the Non-IID phenomenon is the personalization of users, who generate the Non-IID data. With Non-IID (Not Independent and Identically Distributed) issues existing in the federated learning setting, a myriad of approaches has been proposed to crack this hard nut. In contrast, the personalized federated learning may take the advantage…

✭ 58