GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → HewlettPackard → swarm-learning

HewlettPackard / swarm-learning

Licence: other
A simplified library for decentralized, privacy preserving machine learning

Programming Languages

shell
77523 projects

Labels

machine-learningdeep-learningdistributed-computingdistributed-ledgerprivacy-enhancing-technologiesdistributed-controlprivacy-by-design

Projects that are alternatives of or similar to swarm-learning

privapi
Detect Sensitive REST API communication using Deep Neural Networks
Stars: ✭ 42 (-70.42%)
Mutual labels:  privacy-enhancing-technologies, privacy-by-design
Platon Go
Golang implementation of the PlatON protocol
Stars: ✭ 331 (+133.1%)
Mutual labels:  distributed-computing, distributed-ledger
Helib
HElib is an open-source software library that implements homomorphic encryption. It supports the BGV scheme with bootstrapping and the Approximate Number CKKS scheme. HElib also includes optimizations for efficient homomorphic evaluation, focusing on effective use of ciphertext packing techniques and on the Gentry-Halevi-Smart optimizations.
Stars: ✭ 2,749 (+1835.92%)
Mutual labels:  privacy-enhancing-technologies, privacy-by-design
HElib
HElib is an open-source software library that implements homomorphic encryption. It supports the BGV scheme with bootstrapping and the Approximate Number CKKS scheme. HElib also includes optimizations for efficient homomorphic evaluation, focusing on effective use of ciphertext packing techniques and on the Gentry-Halevi-Smart optimizations.
Stars: ✭ 2,913 (+1951.41%)
Mutual labels:  privacy-enhancing-technologies, privacy-by-design
indyscan
Hyperldger Indy Transaction Explorer
Stars: ✭ 52 (-63.38%)
Mutual labels:  distributed-ledger
JOLI.jl
Julia Operators LIbrary
Stars: ✭ 14 (-90.14%)
Mutual labels:  distributed-computing
adblock2privoxy
Convert adblock config files to privoxy format
Stars: ✭ 78 (-45.07%)
Mutual labels:  privacy-enhancing-technologies
decentralized-ml
Full stack service enabling decentralized machine learning on private data
Stars: ✭ 50 (-64.79%)
Mutual labels:  privacy-enhancing-technologies
Distributed-Data-Structures
[GSoC] Distributed Data Structures - Collections Framework for Chapel language
Stars: ✭ 13 (-90.85%)
Mutual labels:  distributed-computing
Windows-On-Reins
Wor is a Powershell script to harden, debloat, optimize, enhance privacy, avoid fingerprinting and improve performance on Windows 10 and 11.
Stars: ✭ 170 (+19.72%)
Mutual labels:  privacy-enhancing-technologies
sawtooth-docs
Documentation source for Sawtooth Lake. Published docs at the link.
Stars: ✭ 10 (-92.96%)
Mutual labels:  distributed-ledger
asyncoro
Python framework for asynchronous, concurrent, distributed, network programming with coroutines
Stars: ✭ 50 (-64.79%)
Mutual labels:  distributed-computing
BMW-Anonymization-API
This repository allows you to anonymize sensitive information in images/videos. The solution is fully compatible with the DL-based training/inference solutions that we already published/will publish for Object Detection and Semantic Segmentation.
Stars: ✭ 121 (-14.79%)
Mutual labels:  privacy-enhancing-technologies
flowchain-ledger
A distributed ledger for the p2p and decentralized IoT devices in JavaScript.
Stars: ✭ 58 (-59.15%)
Mutual labels:  distributed-ledger
distex
Distributed process pool for Python
Stars: ✭ 101 (-28.87%)
Mutual labels:  distributed-computing
pat-helland-and-me
Materials related to my talk "Pat Helland and Me"
Stars: ✭ 14 (-90.14%)
Mutual labels:  distributed-computing
protoactor-go
Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
Stars: ✭ 4,138 (+2814.08%)
Mutual labels:  distributed-computing
IoTPy
Python for streams
Stars: ✭ 24 (-83.1%)
Mutual labels:  distributed-computing
hyperqueue
Scheduler for sub-node tasks for HPC systems with batch scheduling
Stars: ✭ 48 (-66.2%)
Mutual labels:  distributed-computing
vocdoni-node
A set of libraries and tools for the Vocdoni decentralized backend infrastructure, the main ground of our universally verifiable, privacy-centric and scalable digital voting protocol
Stars: ✭ 58 (-59.15%)
Mutual labels:  privacy-by-design
View All Similar Projects ➔

SWARM LEARNING

Swarm Learning is a decentralized, privacy-preserving Machine Learning framework. This framework utilizes the computing power at, or near, the distributed data sources to run the Machine Learning algorithms that train the models. It uses the security of a blockchain platform to share learnings with peers in a safe and secure manner. In Swarm Learning, training of the model occurs at the edge, where data is most recent, and where prompt, data-driven decisions are mostly necessary. In this completely decentralized architecture, only the insights learned are shared with the collaborating ML peers, not the raw data. This tremendously enhances data security and privacy.

Swarm Learning has five components, connected to form a network:

  • Swarm Learning (SL) nodes – These nodes run a user-defined Machine Learning algorithm. This algorithm is called the Swarm Learning ML Program. This program is responsible for training and updating the model in an iterative fashion. The Swarm Learning ML program should be either a Keras (TensorFlow 2 backend) or PyTorch based Machine Learning algorithm that is implemented using Python3. It can also be configured to run on NVIDIA GPUs.

  • Swarm Network (SN) nodes – These nodes form the blockchain network. The current version of Swarm Learning uses an open-source version of Ethereum as the underlying blockchain platform. The Swarm Network nodes interact with each other using this blockchain platform to maintain global state information about the model that is being trained and to track progress (note that only metadata is written to the blockchain. The model itself is not stored in the blockchain.) The Swarm Network nodes use this state and progress information to coordinate the working of the Swarm Learning nodes. Each Swarm Learning node registers itself with a Swarm Network node as a part of its startup and initialization.

    • Sentinel node: This is a special Swarm Network node. The Sentinel node is responsible for initializing the blockchain network. This should be the first node to start.

  • Swarm Learning Command Interface node (SWCI) - SWCI node is the command interface tool to the Swarm Learning framework. It is used to view the status, control and manage the swarm learning framework. It uses a secure link to connect to the Swarm Network node, using the API port. SWCI node can connect to any of the SN nodes in a given Swarm Learning framework to manage the framework.

  • SPIFFE SPIRE Server nodes – These nodes provide the security for the whole network. The platform can run one or more SPIRE Server nodes that are connected together to form a federation. The platform includes a SPIRE Agent Workload Attestor plugin (not shown in the figure) that communicates with the SPIRE Servers to attest the identities of the Swarm Network and Swarm Learning nodes, acquire and manage a SPIFFE Verifiable Identity Document (SVID). For an overview SPIFFE, SPIRE and their capabilities refer to https://spiffe.io/docs/latest/spiffe-about/overview/

  • License Server node – The license to run the Swarm Learning platform is installed and managed by the License Server node.

NOTE: All the Swarm Learning nodes must use the same ML platform – either Keras (based on TensorFlow 2) or PyTorch. Using Keras for some of the nodes and PyTorch for the other nodes is not supported.

Swarm Learning nodes works in collaboration with other Swarm Learning nodes in the network. It regularly shares its learnings with the other nodes and incorporates their insights. This process continues until the Swarm Learning nodes train the model to desired state.

You can transform any Keras or PyTorch based ML program that has been written using Python3 into a Swarm Learning ML program by making a few simple changes to the model training code, such as updating the paths from where the program reads and writes data; and including the SwarmCallback object. See the examples included with the Swarm Learning package for sample code.

Getting Started

Documentation

References

Acronyms and Abbreviations

Refer to Acronyms and Abbreviations for more information.

Getting in touch

Feedback and questions are appreciated. You can use the issue tracker to report bugs on GitHub.

or

Join below Slack channel to communicate with us.

hpe-ai-swarm-learning

Contributing

Refer to Contributing for more information.

License

The distribution of Swarm Learning in this repository is for non-commercial and experimental use under this license.

See ATTRIBUTIONS and DATA LICENSE for terms and conditions for using the datasets included in this repository.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].