185 Open source projects that are alternatives of or similar to Airflow Scheduler Failover Controller

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]

afctl helps to manage and deploy Apache Airflow projects faster and smoother.

Udacity Data Engineering Nano Degree (DEND)

Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

Airflow workflow management platform chef cookbook.

🐳📊🤓Cookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)

🐘 Ansible Role for Patroni

A microservices example developed with Spring Cloud and Vaadin

Tars is a high-performance RPC framework based on name service and Tars protocol, also integrated administration platform, and implemented hosting-service via flexible schedule.

Automating Your Data Pipeline with Apache Airflow

A web-based GUI for managing and monitoring the Pacemaker High-Availability cluster resource manager

Example end to end data engineering project.

Airflow plugin to export dag and task based metrics to Prometheus.

[EOL] A command-line tool to declaratively manage Kubernetes clusters on AWS

Airflow configuration for Telemetry

A distributed workflow runner focusing on performance and simplicity.

A Node.js library for building zero-configuration microservices.

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

Fast iterative local development and testing of Apache Airflow workflows

Bond-cni is for fail-over and high availability of networking in cloudnative orchestration

A Helm chart to install Apache Airflow on Kubernetes

Elyra extends JupyterLab Notebooks with an AI centric approach.

Build AWX clustering on Docker Standalone Installation

Bitnami Docker Image for Apache Airflow

🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)

This is the repository of ZEVENET Load Balancer (next generation of Zen Load Balancer)

HAProxy Load Balancer's development branch (mirror of git.haproxy.org)

mod_cluster is an intelligent native Apache httpd-based and pure-Java Undertow-based load-balancer

Production PostgreSQL for Kubernetes, from high availability Postgres clusters to full-scale database-as-a-service.

Airflow training for the crunch conf

Airflow Unit Tests and Integration Tests

Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor

Deploy a Production Ready Kubernetes Cluster

ETLy is an add-on dashboard service on top of Apache Airflow.

A modern, distributed monitoring system written in Go

Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.

MoHA(Mobike High Availability): A MySQL/Postgres high availability supervisor

tns provides distributed solutions for thrift, support service discovery, high availability, load balancing, the gray release, horizontal scaling, and so on.

A free, open source GSLB (Global Server Load Balancing) solution.

Workflow engine for Kubernetes

Implementation of RAFT distributed consensus algorithm among TCP Peers on .NET / .NETStandard / .NETCore / dotnet

MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

Deploying and managing production-grade etcd clusters on cloud providers: failure recovery, disaster recovery, backups and resizing.

Bare minimal Airflow on Kubernetes (Local, EKS, AKS)

Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)

Continuously synchronize directories from remote object store to local filesystem

📖 [译] Airflow 中文文档

A series of DAGs/Workflows to help maintain the operation of Airflow

Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks

A facebook for data

A docker-compose example of HA Consul + Vault + Vault UI

Let's Make DNS Outage Suck Less

🔥 Fast, Secure and Reliable System Backup, Set up in Minutes.

Metric collection, data testing and monitoring for SQL accessible data

High Availability Framework for Happy Data

Airflow Deployment on AWS ECS Fargate Using Cloudformation

MySQL active-active replication solution.