All Projects → kaizhang → SciFlow

kaizhang / SciFlow

Licence: other
Scientific workflow management

Programming Languages

haskell
3896 projects

Projects that are alternatives of or similar to SciFlow

dlsa
Distributed least squares approximation (dlsa) implemented with Apache Spark
Stars: ✭ 25 (-48.98%)
Mutual labels:  distributed-computing
realtimemap-dotnet
A showcase for Proto.Actor - an ultra-fast distributed actors solution for Go, C#, and Java/Kotlin.
Stars: ✭ 47 (-4.08%)
Mutual labels:  distributed-computing
server
Hashtopolis - A Hashcat wrapper for distributed hashcracking
Stars: ✭ 954 (+1846.94%)
Mutual labels:  distributed-computing
goobi-workflow
Goobi workflow - Workflow management software for digitisation projects used in more than 70 cultural heritage institutions in at least 17 countries.
Stars: ✭ 43 (-12.24%)
Mutual labels:  workflow-management
Distributed-System-Algorithms-Implementation
Algorithms for implementation of Clock Synchronization, Consistency, Mutual Exclusion, Leader Election
Stars: ✭ 39 (-20.41%)
Mutual labels:  distributed-computing
easyFL
An experimental platform to quickly realize and compare with popular centralized federated learning algorithms. A realization of federated learning algorithm on fairness (FedFV, Federated Learning with Fair Averaging, https://fanxlxmu.github.io/publication/ijcai2021/) was accepted by IJCAI-21 (https://www.ijcai.org/proceedings/2021/223).
Stars: ✭ 104 (+112.24%)
Mutual labels:  distributed-computing
Distributed-Data-Structures
[GSoC] Distributed Data Structures - Collections Framework for Chapel language
Stars: ✭ 13 (-73.47%)
Mutual labels:  distributed-computing
Awesome-Federated-Machine-Learning
Everything about federated learning, including research papers, books, codes, tutorials, videos and beyond
Stars: ✭ 190 (+287.76%)
Mutual labels:  distributed-computing
blockchain-reading-list
A reading list on blockchain and related technologies, targeted at technical people who want a deep understanding of those topics.
Stars: ✭ 93 (+89.8%)
Mutual labels:  distributed-computing
frovedis
Framework of vectorized and distributed data analytics
Stars: ✭ 59 (+20.41%)
Mutual labels:  distributed-computing
dicodile
Experiments for "Distributed Convolutional Dictionary Learning (DiCoDiLe): Pattern Discovery in Large Images and Signals"
Stars: ✭ 15 (-69.39%)
Mutual labels:  distributed-computing
DevOps
DevOps code to deploy eScience services
Stars: ✭ 19 (-61.22%)
Mutual labels:  distributed-computing
optimism-v2
ARCHIVE of monorepo implementing Boba, an L2 Compute solution built on Optimistic Ethereum - active repo is at https://github.com/bobanetwork/boba
Stars: ✭ 34 (-30.61%)
Mutual labels:  distributed-computing
kar
KAR: A Runtime for the Hybrid Cloud
Stars: ✭ 18 (-63.27%)
Mutual labels:  distributed-computing
CipherCompute
The free EAP version of the Cosmian Collaborative Confidential Computing platform. Try it!
Stars: ✭ 20 (-59.18%)
Mutual labels:  distributed-computing
swarm-learning
A simplified library for decentralized, privacy preserving machine learning
Stars: ✭ 142 (+189.8%)
Mutual labels:  distributed-computing
PFL-Non-IID
The origin of the Non-IID phenomenon is the personalization of users, who generate the Non-IID data. With Non-IID (Not Independent and Identically Distributed) issues existing in the federated learning setting, a myriad of approaches has been proposed to crack this hard nut. In contrast, the personalized federated learning may take the advantage…
Stars: ✭ 58 (+18.37%)
Mutual labels:  distributed-computing
mobius
Mobius is an AI infra platform including realtime computing and training.
Stars: ✭ 22 (-55.1%)
Mutual labels:  distributed-computing
SadlyDistributed
Distributing your code(soul), in almost any language(state), among a cluster of idle browsers(voids)
Stars: ✭ 20 (-59.18%)
Mutual labels:  distributed-computing
bloomfilter
Bloomfilter written in Golang, includes rotation and RPC
Stars: ✭ 61 (+24.49%)
Mutual labels:  distributed-computing

Scientific workflow management system

SciFlow is a DSL for building type-safe computational workflows. SciFlow is implemented based on the Free Arrow and is heavily inspired by the funflow package. The differences between SciFlow and funflow are:

  1. SciFlow uses Template Haskell to ease the process of workflow specification and to allow composition and reuse of defined workflows.

  2. SciFlow supports distributed computing thanks to Cloud Haskell!

Design

Features

  1. Easy to use and safe: Provide a simple and flexible way to design type safe computational pipelines in Haskell.

  2. Automatic Checkpointing: The states of intermediate steps are automatically logged, allowing easy restart upon failures.

  3. Parallelism and distributed computing support.

Examples

See examples in the "examples" directory for more details.

Featured applications

Here are some bioinformatics pipelines built with SciFlow.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].