339 Open source projects that are alternatives of or similar to Luigi

Resource scheduling and cluster management for AI

🔜 Angular Idle Preload for preloading async routes via @TipeIO

HBase客户端数据管理软件

基于wifi抓取信息的大数据查询分析系统

实践源码库：https://github.com/jast90/bigdata 。 微信搜索Jast关注公众号，获取最新技术分享😯。

Python client for Hadoop® YARN API

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

Time Table generation using Genetic Algorithms ( Java-Struts2)

Spark、Flink等离线任务的调度以及实时任务的监控

Distributed,multi-tenancy,job-flow scheduling application(分布式多租户的支持流程编排的任务调度平台) QQ群:1029617143

A large-scale entity and relation database supporting aggregation of properties

Multiple node cluster on Docker for self development.

Facebook's Hive UDFs

Road to Machine Learning

Ephemeral Hadoop clusters using Google Compute Platform

Mirror of Apache Chukwa

📚 Notebook storage and publishing workflows for the masses

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Kubernetes Scheduler for Deep Learning

Educational notes,Hands on problems w/ solutions for hadoop ecosystem

Apache Nutch is an extensible and scalable web crawler

A scheduling and time utilities module that doesn't waste your time

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Axibase Time Series Database Documentation

Cylc: a workflow engine for cycling systems. Repository master branch: core meta-scheduler component of cylc-8 (in development); Repository 7.8.x branch: full cylc-7 system.

Runs, manages, and autoscales web services on Mesos and Kubernetes

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Asynchronous non-blocking running many tasks in JavaScript. Demo https://codesandbox.io/s/admiring-ride-jdoq0

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

A Spark cluster setup running on Docker containers

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

SpringBoot 整合各类框架和应用

A pandas-like deferred expression system, with first-class SQL support

Automate and relate multiple processes.

Quickly and easily install, uninstall, and set up automatic updates for any of Steven Black's unified hosts files.

MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

Web application for solving Employee Rostering using OptaPlanner

Golang (as of 1.12.5) runtime internals that gives you an access to internal scheduling primitives. Park Gs, read IDs. (for learning purposes)

Fundamentals of Spark with Python (using PySpark), code examples

A Python tool to assist the task of scheduling a conference http://conference-scheduler.readthedocs.org

Asakusa Framework

在動漫花園訂閱並排程下載磁鏈，支援 Linux & Windows 10 & Docker

Java开发者或者大数据开发者面试知识点整理

零基础学习spark，大数据学习

Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Presto and AWS Athena.

A Java NIO file system provider for HDFS

Apache Hadoop

xxl-job is a lightweight distributed task scheduling framework, and this package provide a dotnet executor client for it

liteflow是一个基于任务版本来实现的分布式任务流调度系统

MapReduce, Spark, Java, and Scala for Data Algorithms Book

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Concept showcase for "Appointment Scheduling System for Small and Medium-Sized Medical Facilities"

🐘 Source code for assignments of Udacity course "Introduction to Hadoop and MapReduce"

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

Apache Calcite

推荐项目（实时推荐和离线推荐）

The vip.com's distributed job scheduling platform.

Big Data Toolkit for the JVM