Top 6 luigi open source projects

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

✭ 15,226

python javascript HTML hadoop scheduling orchestration-framework luigi

osm-data-classification

Migrated to: https://gitlab.com/Oslandia/osm-data-classification

✭ 23

python TSQL machine-learning statistics openstreetmap osm luigi pca data-analysis kmeans data-quality

docker-luigi

A project to help develop Luigi pipelines using Docker ✳️

✭ 73

shell python Dockerfile docker data-science data machine-learning luigi

saisoku

Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.

✭ 40

python sync pipeline scheduling s3 orchestration-framework rclone luigi tornado data-transfer file-transfer transfer-files data-synchronization data-pipeline sync-directories luigi-pipeline transfer-server directory-transfer

ClarityNLP

An NLP framework for clinical phenotyping. Docker | Python | Solr | OMOP. http://claritynlp.readthedocs.io/en/latest/

✭ 100

Jupyter Notebook python CSS C++C#shell nlp docker natural-language-processing mongodb solr luigi phenotype phenotypes clinical-research clinical-quality-language nlpql clinical-phenotyping

gluish

Utils around luigi.

✭ 63

python Makefile luigi

1-6 of 6 luigi projects