122 Open source projects that are alternatives of or similar to dbt-on-airflow

Use Airflow to move data from multiple MySQL databases to BigQuery

💨🥫 A Data Factory system for running data processing pipelines built on AirFlow and tailored to CKAN. Includes evolution of DataPusher and Xloader for loading data to DataStore.

Kubernetes custom controller and CRDs to managing Airflow

A _simple_ starter template for Snowflake Cloud Data Platform

Observatory Platform Package

Helper code to interact with Rasgo via our SDK, PyRasgo

Example DAGs using hooks and operators from Airflow Plugins

do more with dbt. fal helps you run Python alongside dbt, so you can send Slack alerts, detect anomalies and build machine learning models.

Curated list of resources about Apache Airflow

Lightweight and performant dynamic binary translation for RISC–V code on x86–64

Metric collection, data testing and monitoring for SQL accessible data

Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.

📖 [译] Airflow 中文文档

A plugin for Apache Airflow that allows you to manage the users that can login

🐳📊🤓Cookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)

Formatting for dbt jinja-flavored sql

Airflow Deployment on AWS ECS Fargate Using Cloudformation

Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data sc…

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

afctl helps to manage and deploy Apache Airflow projects faster and smoother.

A plugin for Apache Airflow that allows you to edit DAGs in browser

Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)

A complete development environment setup for working with Airflow

Bitnami Docker Image for Apache Airflow

An open source alternative to Looker built using dbt. Made for analysts ❤️

Example end to end data engineering project.

Generate lookml for views from dbt models

Tellery lets you build metrics using SQL and bring them to your team. As easy as using a document. As powerful as a data modeling tool.

ETLy is an add-on dashboard service on top of Apache Airflow.

Apache Airflow - OpenApi Client for Python

A distributed workflow runner focusing on performance and simplicity.

Personal Data Engineering Projects

Workflow engine for Kubernetes

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

Bare minimal Airflow on Kubernetes (Local, EKS, AKS)

triggering a DAG run multiple times

Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/

Package for dbt that allows users to train, audit and use BigQuery ML models.

Elyra extends JupyterLab Notebooks with an AI centric approach.

A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.

Automating Your Data Pipeline with Apache Airflow

Fivetran's ad reporting dbt package. Combine your Facebook, Google, Pinterest, Linkedin, Twitter, Snapchat and Microsoft advertising spend using this package.

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Astronomer Core Docker Images

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Real Time Twitter Sentiment Analysis Product

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

Turbine: the bare metals that gets you Airflow

The metrics layer for your data. Join us at https://metriql.com/slack

Docker Apache Airflow

re_data - fix data issues before your users & CEO would discover them 😊

Apache Airflow Website

Minikube for big data with Scala and Spark

Example of an ETL Pipeline using Airflow

Bitnami Docker Image for Apache Airflow Scheduler

A way for home buyers to know about factors affecting a state

Get your MLOps (Level 1) platform started and going fast.

Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.