All Categories → Data Processing → pipeline

Top 371 pipeline open source projects

A library for creating and curating reproducible pipelines for scientific and industrial machine learning

✭ 27

Jupyter Notebook python pipeline reproducible-research scikit-learn grid-search data-preparation reproducible-experiments prepare-data

multibranch-action-triggers-plugin

MultiBranch Actions Trigger Plugin

✭ 29

java HTML pipeline declarative trigger trigger-events pipeline-job multibranch declarative-pipeline multibranch-pipeline pipeline-trigger

pipeline

Spline is a tool that is capable of running locally as well as part of well known pipelines like Jenkins (Jenkinsfile), Travis CI (.travis.yml) or similar ones.

✭ 29

python shell Jinja docker yaml workflow pipeline matrix

redundans

Redundans is a pipeline that assists an assembly of heterozygous/polymorphic genomes.

✭ 90

perl python shell bioinformatics pipeline genomics docker-image polymorphic closing fasta scaffolding assembled-contigs genome-assembly paired-end contigs heterozygous mate-pairs

piper-nf

RNA mapping pipeline

✭ 18

Nextflow perl shell pipeline

ngs pipeline

Exome/Capture/RNASeq Pipeline Implementation using snakemake

✭ 40

python perl r shell snakemake pipeline panel ngs tophat gatk rnaseq fastq exome ngs-pipeline annovar snpeff

streamalg

Extensible stream pipelines with object algebras.

✭ 26

java C#scala shell benchmark pipeline iteration streams extensible-pipelines tagless

ImcSegmentationPipeline

A pixel classification based multiplexed image segmentation pipeline

✭ 62

Jupyter Notebook python pipeline ubuntu cellprofiler segmentation imc pixel-classification

dagshub-branch-source-plugin

Jenkins plugin for using DAGsHub as a source for multibranch projects

✭ 12

git pipeline scm scm-related

needlestack

Multi-sample somatic variant caller

✭ 45

r Nextflow shell Dockerfile TeX docker bam-files pipeline ngs regression variants somatic-mutations samtools negative-binomial-regression

ngs-preprocess

A pipeline for preprocessing NGS data from Illumina, Nanopore and PacBio technologies

✭ 22

Nextflow Dockerfile workflow bioinformatics pipeline reproducible-research ngs reproducible-science pacbio illumina nanopack trimgalore ngs-preprocess bax2bam porechop pacbio-ccs

cmdc

Maya Python API 3.0

✭ 98

c C++python pipeline animation maya autodesk rigging

pachinko

modular pluggable media sorter

✭ 27

go Makefile Dockerfile trakt pipeline processor plex tmdb tvdb hacktoberfest subtypes datastream intra pachinko media-sorter postprocessors options-pachinko

rmonad

Pipelines you can compute on

✭ 66

r TeX Makefile workflow pipeline

bactmap

A mapping-based pipeline for creating a phylogeny from bacterial whole genome sequences

✭ 36

Nextflow groovy python HTML workflow tree pipeline genomics mapping bacteria phylogeny bacterial nf-core bacterial-genome-analysis

nifi

Deploy a secured, clustered, auto-scaling NiFi service in AWS.

✭ 37

HCL shell python aws ansible automation cloud big-data kms ec2 pipeline ubuntu terraform s3 iam apache zookeeper rhel iac nifi apache-nifi

unix-programming-and-regular-expressions-workshop

A workshop on Unix Programming Principles using tools such as grep, sed, awk, shell programming and regular expressions

✭ 25

shell awk nodejs workshop networking pipeline makefile build-automation regular-expression subshell text-substitution unix-programming

artifact-promotion-plugin

A simple Jenkins plugin to promote artifacts.

✭ 29

java HTML jenkins pipeline continuous-delivery jenkins-plugin jenkins-ci nexus-oss adopt-this-plugin

tpack

Pack a Go workflow/function as a Unix-style pipeline command

✭ 55

go stream pipeline command command-line pipe pipes unix-command

naas

⚙️ Schedule notebooks, run them like APIs, expose securely your assets: Jupyter as a viable ⚡️ Production environment

✭ 219

python Jupyter Notebook HTML typescript shell Dockerfile open-source data-science data binder ai integration jupyter pipeline etl engine data-transformation jupyterlab notebooks

dolphinnext

A graphical user interface for distributed data processing of high throughput genomics

✭ 92

PHP javascript Nextflow CSS hack Twig bioinformatics rna-seq pipeline drag-and-drop pipelines chip-seq atac-seq clusters workflows amazon-cloud dolphinnext share-pipelines

human genomics pipeline

A Snakemake workflow to process single samples or cohorts of paired-end sequencing data (WGS or WES) using trim galore/bwa/GATK4/parabricks.

✭ 19

python shell CSS pipeline genomics exome-pipeline illumina nvidia-gpu cohort-analysis gatk4 snakemake-pipeline gatk-bestpractices nvidia-clara-parabricks

poco

Interactive pipeline filtering in PowerShell (a port of peco).

✭ 16

powershell pipeline

rocket-pipes

Powerful pipes for TypeScript, that chain Promise and ADT for you 🚌 -> ⛰️ -> 🚠 -> 🏂 -> 🚀

✭ 18

typescript javascript mock validation pipeline promise pipe composition ramda context aop adt either compose exit kleisli monet chain-promise fp-libraries

pipeline-as-yaml-plugin

Jenkins Pipeline As Yaml Plugin

✭ 111

java HTML yaml jenkins pipeline jenkins-plugin jenkins-pipeline pipeline-as-code multibranch-pipeline pipeline-as-yaml

cherry-on-py

Cloud computing is a game changer for developers. What can you do in a couple hundred lines of code?

✭ 67

python javascript HTML CSS Procfile machine-learning serverless pipeline image-processing video-processing

etran

Erlang Parse Transforms Including Fold (MapReduce) comprehension, Elixir-like Pipeline, and default function arguments

✭ 19

erlang Makefile HTML map parser pipeline pipe arguments function fold transform mapreduce default

pipeline-github-plugin

Pipeline: GitHub

✭ 151

github pipeline jenkins-plugin jenkins-pipeline

saisoku

Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.

✭ 40

python sync pipeline scheduling s3 orchestration-framework rclone luigi tornado data-transfer file-transfer transfer-files data-synchronization data-pipeline sync-directories luigi-pipeline transfer-server directory-transfer

zenhub-pipeline

Automatically transfer issues in pipeline by commit message

✭ 14

javascript Dockerfile hooks pipeline zenhub move commit

TensorflowDeepSortTracking

Tensorflow object detection with tracking based on the DeepSort algorithm

✭ 60

python Jupyter Notebook algorithm pipeline tensorflow realtime-tracking bounding-boxes

vector

A high-performance observability data pipeline.

✭ 12,138

rust CUE HTML typescript javascript shell parser events router pipeline metrics vector logs stream-processing forwarder observability

LTRpred

De novo annotation of young retrotransposons

✭ 35

r UnrealScript TeX Dockerfile pipeline genome evolution ltr diversification ltr-transposons ltr-retrotransposons

adonis-bundler

Blazing fast, zero configuration assets bundler for AdonisJS

✭ 19

javascript bundler pipeline assets zero-configuration adonis

MegFlow

Efficient ML solution for long-tailed demands.

✭ 372

rust python javascript HTML CSS shell computer-vision deep-learning pipeline pipeline-framework megengine megenginelite

runiac

Run IaC Anywhere With Ease

✭ 18

go Dockerfile shell docker aws pipeline azure terraform container gcp iac arm-templates armtemplates regional-deployments

cmakebuilder-plugin

Jenkins CMake plugin

✭ 16

java HTML cmake pipeline jenkins-plugin jenkins-builder

BACTpipe

BACTpipe: An assembly and annotation pipeline for bacterial genomics

✭ 19

groovy Nextflow python shell Makefile bioinformatics pipeline annotation bacteria whole-genome-sequencing

pipelineRD

A chain of responsability pattern implementation in .NET that supports retry policy, sync and async steps, rollback, pipeline recovery by cache and visual documentation using diagrams.

✭ 19

C#Vim Snippet pipeline dotnet diagram rollback policy chain-of-responsibility

grape-nf

An automated RNA-seq pipeline using Nextflow

✭ 30

shell Nextflow Dockerfile python Makefile awk rna-seq pipeline ngs crg guigo

HTGS

The Hybrid Task Graph Scheduler API

✭ 36

C++CMake c CSS cpu pipeline gpu task-scheduler hybrid-taskgraph data-flow-graph

cpp-from-zero-to-one

Learning C++ from zero to one

✭ 52

pipeline

A framework (comand line tool + libraries) for creating flexible compute pipelines

✭ 54

python workflow data-science pipeline notebook research-tool

CloudConductor

CloudConductor is a workflow management system that generates and executes bioinformatics pipelines

✭ 13

python Dockerfile workflow cloud bioinformatics pipeline google-cloud

gawn

Genome Annotation Without Nightmares

✭ 35

perl python shell pipeline genomics genome-annotation transcriptome

eidos-audition

Collection of auditory models.

✭ 25

C++Starlark python shell pipeline algorithms signal-processing speech perception auditory

rna-seq-snakemake

Snakemake based pipeline for RNA-Seq analysis

✭ 29

r python PowerBuilder shell bioinformatics snakemake pipeline rna-seq-analysis rna-seq-snakemake

ember-pipeline

Railway oriented programming in Ember

✭ 17

javascript HTML ember experimental pipeline ember-addon railway-oriented-programming not-production-ready

langx-java

Java tools, helper, common utilities. A replacement of guava, apache-commons, hutool

✭ 50

java security diff helper csv collection tools cron stream repository pipeline timer reflect pinyin aop common idgenerator classpath

hic

Analysis of Chromosome Conformation Capture data (Hi-C)

✭ 45

Nextflow python groovy C++workflow pipeline hi-c chromosome-conformation-capture nf-core

XProc-Z

A platform for running XProc pipelines as web applications in a Java servlet container

✭ 20

XSLT XProc java webservice pipeline xml webserver

pipeline

Pipelines using goroutines

✭ 46

go pipeline concurrency

functions

An Open Source Serverless Platform

✭ 44

javascript Makefile nodejs lambda serverless pipeline lambda-functions

STOCK-RETURN-PREDICTION-USING-KNN-SVM-GUASSIAN-PROCESS-ADABOOST-TREE-REGRESSION-AND-QDA

Forecast stock prices using machine learning approach. A time series analysis. Employ the Use of Predictive Modeling in Machine Learning to Forecast Stock Return. Approach Used by Hedge Funds to Select Tradeable Stocks

✭ 94