Top 83 pipelines open source projects

Platform for Situated Intelligence

Make + Docker + Shell = CI Pipeline

Fast, flexible, and version-tolerant serializer for .NET

MLOps Platform

Train and run Pytorch models on Apache Spark.

High performance data aggregating storage

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

🔀 FluentMediator is an unobtrusive library that allows developers to build custom pipelines for Commands, Queries and Events.

Jenkins plugin for pipeline visualisation, perfect for Continuous Delivery

⚡♠️ Assemble bacterial isolate genomes from Illumina paired-end reads

Dataflow Programming for Machine Learning in R

Hop Orchestration Platform

Translates GitHub Actions into Tekton and Knative Objects

Collective Knowledge components for TensorFlow (code, data sets, models, packages, workflows):

The Darkest Pipeline - Multithreaded pipelines for modern C++

Continuous Integration Tool for Swift Projects

Pipelines - Run Bitbucket Pipelines Wherever They Dock

Demonstration of a Kubernetes-centric CI/CD workflow

Primitives for machine learning and data science.

ZenML 🙏: MLOps framework to create reproducible ML pipelines for production machine learning.

A Julia machine learning framework

this code is now in https://github.com/heroku/cli

IceCI is a continuous integration system designed for Kubernetes from the ground up.

Elyra extends JupyterLab Notebooks with an AI centric approach.

Cross platform sign language translation with mediapipe

Event-driven workflow automation framework

A new kind of IDE for Data Science.

Concourse is a container-based continuous thing-doer written in Go.

Framework for large distributed pipelines

A lightweight stream processing library for Go

🦊 Test your Gitlab CI Pipelines changes locally using Docker.

The open and extensible integrated development environment (IDE) for computer vision with built-in modules for model building, automated labeling, data processing, model training, hyperparameter tuning and workflow orchestration.

Azure DevOps Extension for Azure CLI

Zero-copy I/O primitives and pipelines for Go. Linux-specific.

An experimental programming language for data flow

A web-based monitor dashboard for GitLab CI

Command Line Tools for Drone CI

A framework for pluggable business logic components

An extensible Java framework for building XML and non-XML streaming applications

Event triggering with Tekton!

Machine Learning Platform for Kubernetes (MLOps tools for experimentation and automation)

Repository to host tool-specific module files for the Nextflow DSL2 community!

A library for composing end-to-end tunable machine learning pipelines.

Machines and people collaborating together through Jupyter notebooks.

Cloud Pipelines Editor is a web app that allows the users to build and run Machine Learning pipelines without having to set up development environment.

Serves as documentation, starter code, and companion guide for a DevOps 101 workshop using the JFrog platform.

Kubeflow pipelines built on top of Tensorflow TFX library

A collection of pipelines for Scrapy

"Integrating Julia in real-world, distributed pipelines" for JuliaCon 2017

The Refreshingly Simple Cross-Platform C++ Dataflow / Pipelining / Stream Processing / Reactive Programming Framework

A cookiecutter for projects with continuous delivery baked in.

Simplifying the definition and execution, scaling and deployment of pipelines on the cloud.

System.IO.Pipelines API adapter for System.Net.WebSockets

Tibanna helps you run your genomic pipelines on Amazon cloud (AWS). It is used by the 4DN DCIC (4D Nucleome Data Coordination and Integration Center) to process data. Tibanna supports CWL/WDL (w/ docker), Snakemake (w/ conda) and custom Docker/shell command.

🏵 Gee is tool of stdin to each files and stdout. It is similar to the tee command, but there are more functions for convenience. In addition, it was written as go

A task for Azure DevOps Pipelines to run newman tests.

ADENINE: A Data ExploratioN PipelINE

Build container images with Knative + Gitlab + Harbor inside Kops cluster running on AWS

Open resource exchange platform for non-human primate neuroimaging