Top 99 airflow open source projects

triggering a DAG run multiple times

Identifies and collects data on cc-licensed content across web crawl data and public apis.

Apache Airflow integration for dbt

Use Airflow to move data from multiple MySQL databases to BigQuery

Astro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Air Data Computer

Export Airflow metrics (from mysql) in prometheus format

Personal Data Engineering Projects

JobAnalytics system consumes data from multiple sources and provides valuable information to both job hunters and recruiters.

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

Observatory Platform Package

Get your MLOps (Level 1) platform started and going fast.

Build DataOps platform with Apache Airflow and dbt on AWS

DBT Cloud Plugin for Airflow

Visualize Airflow's schedule by exporting future DAG runs as events to Google Calendar.

Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.

Apache Airflow CI pipeline

Configuration and definitions of Airflow for OpenTrials

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

Plugins do Airflow para implementação de pipelines de dados

A complete development environment setup for working with Airflow

Execution of DBT models using Apache Airflow through Docker Compose

Apache Airflow - OpenApi Client for Python

Testbed of AI Systems Quality Management

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

Parcel for Apache Airflow

A lightweight tool to get an AI Infrastructure Stack up in minutes not days. K3ai will take care of setup K8s for You, deploy the AI tool of your choice and even run your code on it.

Functional Airflow DAG definitions.

Real Time Twitter Sentiment Analysis Product

Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'

Apache Airflow Website

A way for home buyers to know about factors affecting a state

Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.

Flask-AppBuilder SecurityManager for OpenIDConnect

Kedro Plugin to support running pipelines on Kubernetes using Airflow.

Blog post on ETL pipelines with Airflow

💨🥫 A Data Factory system for running data processing pipelines built on AirFlow and tailored to CKAN. Includes evolution of DataPusher and Xloader for loading data to DataStore.

PipelineAI Kubeflow Distribution

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.