102 Open source projects that are alternatives of or similar to fairflow

Identifies and collects data on cc-licensed content across web crawl data and public apis.

Export Airflow metrics (from mysql) in prometheus format

A complete development environment setup for working with Airflow

Apache Airflow - OpenApi Client for Python

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

A plugin for Apache Airflow that allows you to manage the users that can login

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

A plugin for Apache Airflow that allows you to edit DAGs in browser

Curated list of resources about Apache Airflow

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

📖 [译] Airflow 中文文档

A distributed workflow runner focusing on performance and simplicity.

Example end to end data engineering project.

Metric collection, data testing and monitoring for SQL accessible data

ETLy is an add-on dashboard service on top of Apache Airflow.

Blog post on ETL pipelines with Airflow

Workflow engine for Kubernetes

🐳📊🤓Cookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)

Bare minimal Airflow on Kubernetes (Local, EKS, AKS)

Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/

A way for home buyers to know about factors affecting a state

PipelineAI Kubeflow Distribution

Airflow Deployment on AWS ECS Fargate Using Cloudformation

Elyra extends JupyterLab Notebooks with an AI centric approach.

Automating Your Data Pipeline with Apache Airflow

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Udacity Data Engineering Nano Degree (DEND)

A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability

Airflow training for the crunch conf

Kedro Plugin to support running pipelines on Kubernetes using Airflow.

Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor

Airflow Unit Tests and Integration Tests

Airflow workflow management platform chef cookbook.

Apache Airflow Website

Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]

Airflow plugin to export dag and task based metrics to Prometheus.

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

💨🥫 A Data Factory system for running data processing pipelines built on AirFlow and tailored to CKAN. Includes evolution of DataPusher and Xloader for loading data to DataStore.

Continuously synchronize directories from remote object store to local filesystem

A Helm chart to install Apache Airflow on Kubernetes

A series of DAGs/Workflows to help maintain the operation of Airflow

Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'

A facebook for data

🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.

Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available out of box.

Airflow configuration for Telemetry

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.

Example DAGs using hooks and operators from Airflow Plugins

afctl helps to manage and deploy Apache Airflow projects faster and smoother.

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Dynamically generate Apache Airflow DAGs from YAML configuration files

Fast iterative local development and testing of Apache Airflow workflows

Turbine: the bare metals that gets you Airflow

Airflow basics tutorial

A web frontend for scheduling Jupyter notebook reports

Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)