122 Open source projects that are alternatives of or similar to airflow-dbt

Get your MLOps (Level 1) platform started and going fast.

🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)

Testbed of AI Systems Quality Management

Airflow configuration for Telemetry

The Clickhouse plugin for dbt (data build tool)

Fast iterative local development and testing of Apache Airflow workflows

Parcel for Apache Airflow

Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks

🎣 List of `pre-commit` hooks to ensure the quality of your `dbt` projects.

Udacity Data Engineering Nano Degree (DEND)

A lightweight tool to get an AI Infrastructure Stack up in minutes not days. K3ai will take care of setup K8s for You, deploy the AI tool of your choice and even run your code on it.

Air Data Computer

dbt-sugar is a CLI tool that allows users of dbt to have fun and ease performing actions around dbt models

Metric collection, data testing and monitoring for SQL accessible data

Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor

Functional Airflow DAG definitions.

Airflow workflow management platform chef cookbook.

Utility functions for dbt projects running on Spark

Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]

Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

JobAnalytics system consumes data from multiple sources and provides valuable information to both job hunters and recruiters.

Continuously synchronize directories from remote object store to local filesystem

Containerized end-to-end analytics of Spotify data using Python, dbt, Postgres, and Metabase

A series of DAGs/Workflows to help maintain the operation of Airflow

Visualize Airflow's schedule by exporting future DAG runs as events to Google Calendar.

A facebook for data

Helper code to interact with Rasgo via our SDK, PyRasgo

Astro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available out of box.

Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Apache Airflow CI pipeline

Dynamically generate Apache Airflow DAGs from YAML configuration files

Flask-AppBuilder SecurityManager for OpenIDConnect

Airflow basics tutorial

Make dbt docs and Apache Superset talk to one another

A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces

Blog post on ETL pipelines with Airflow

A Data Engineering & Machine Learning Knowledge Hub

Configuration and definitions of Airflow for OpenTrials

PipelineAI Kubeflow Distribution

Example of an ETL Pipeline using Airflow

Export Airflow metrics (from mysql) in prometheus format

Astronomer Core Docker Images

Airflow Unit Tests and Integration Tests

A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract data from S3, apply a series of transformations and load into S3 and Redshift.

Plugins do Airflow para implementação de pipelines de dados

Cloudformation templates for deploying Airflow in ECS

A web frontend for scheduling Jupyter notebook reports

Identifies and collects data on cc-licensed content across web crawl data and public apis.

A CLI for creating, updating, and deleting dbt property files

A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability

Use Airflow to move data from multiple MySQL databases to BigQuery

Lightweight and performant dynamic binary translation for RISC–V code on x86–64

An open source alternative to Looker built using dbt. Made for analysts ❤️

Observatory Platform Package

Generate lookml for views from dbt models

📖 [译] Airflow 中文文档