98 Open source projects that are alternatives of or similar to observatory-platform

Airflow Deployment on AWS ECS Fargate Using Cloudformation

Bitnami Docker Image for Apache Airflow

Flask-AppBuilder SecurityManager for OpenIDConnect

📖 [译] Airflow 中文文档

A distributed workflow runner focusing on performance and simplicity.

Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'

afctl helps to manage and deploy Apache Airflow projects faster and smoother.

Execution of DBT models using Apache Airflow through Docker Compose

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

PipelineAI Kubeflow Distribution

Metric collection, data testing and monitoring for SQL accessible data

Bare minimal Airflow on Kubernetes (Local, EKS, AKS)

Functional Airflow DAG definitions.

🐳📊🤓Cookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)

Plugins do Airflow para implementação de pipelines de dados

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

A way for home buyers to know about factors affecting a state

Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)

Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.

Example end to end data engineering project.

Blog post on ETL pipelines with Airflow

ETLy is an add-on dashboard service on top of Apache Airflow.

Testbed of AI Systems Quality Management

Workflow engine for Kubernetes

Example DAGs using hooks and operators from Airflow Plugins

A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability

Continuously synchronize directories from remote object store to local filesystem

A lightweight tool to get an AI Infrastructure Stack up in minutes not days. K3ai will take care of setup K8s for You, deploy the AI tool of your choice and even run your code on it.

Airflow Unit Tests and Integration Tests

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

Airflow plugin to export dag and task based metrics to Prometheus.

Real Time Twitter Sentiment Analysis Product

A Helm chart to install Apache Airflow on Kubernetes

Visualize Airflow's schedule by exporting future DAG runs as events to Google Calendar.

🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)

Apache Airflow Website

Airflow configuration for Telemetry

A complete development environment setup for working with Airflow

Fast iterative local development and testing of Apache Airflow workflows

Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.

Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks

Build DataOps platform with Apache Airflow and dbt on AWS

Udacity Data Engineering Nano Degree (DEND)

Kedro Plugin to support running pipelines on Kubernetes using Airflow.

Airflow training for the crunch conf

Apache Airflow - OpenApi Client for Python

Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor

💨🥫 A Data Factory system for running data processing pipelines built on AirFlow and tailored to CKAN. Includes evolution of DataPusher and Xloader for loading data to DataStore.

Airflow workflow management platform chef cookbook.

Apache Airflow CI pipeline

Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

A web frontend for scheduling Jupyter notebook reports

Get your MLOps (Level 1) platform started and going fast.

DBT Cloud Plugin for Airflow

Configuration and definitions of Airflow for OpenTrials

Parcel for Apache Airflow

Curated list of resources about Apache Airflow