128 Open source projects that are alternatives of or similar to Whirl

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

A Data Engineering & Machine Learning Knowledge Hub

Continuously synchronize directories from remote object store to local filesystem

Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available out of box.

Bitnami Docker Image for Apache Airflow Scheduler

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

Airflow basics tutorial

Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor

Minikube for big data with Scala and Spark

Local amazee.io Docker Drupal Development Environment https://rubygems.org/gems/pygmy/

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Cloudformation templates for deploying Airflow in ECS

Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

🐬 Quickly toggle between different VVV provisioning scripts

Local development against a remote Kubernetes or OpenShift cluster

My local WordPress development built with Docker and Docker Compose.

A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces

Bitnami Docker Image for Apache Airflow

FWD - easy environments for web applications

A simple zero-config tool to make locally trusted development certificates with any names you'd like.

A plugin for Apache Airflow that allows you to manage the users that can login

Mage2click Docker-based development environment for Magento with mutagen.io sync for files on macOS

A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract data from S3, apply a series of transformations and load into S3 and Redshift.

A facebook for data

📦 WPacked is a WordPress development starter kit with portability and immediate local development in mind.

A distributed workflow runner focusing on performance and simplicity.

Local HTTPS development made easy

Airflow training for the crunch conf

HTTPS domains for localhost. 🏠

Workflow engine for Kubernetes

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

A docker based multi-site setup for local PHP development. Inspired by Laravel Valet, Homestead and Vessel.

Dynamically generate Apache Airflow DAGs from YAML configuration files

🌐 Variable VVV - a VVV Site Creation Wizard.‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ❌ This project is no longer maintained. Please update your copy of VVV , which has most of the vv features built in.

Turbine: the bare metals that gets you Airflow

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Docker Apache Airflow

Bare minimal Airflow on Kubernetes (Local, EKS, AKS)

Kubernetes custom controller and CRDs to managing Airflow

Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks

Yet another helpdesk based on multiple providers

Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/

No description or website provided.

🔄 Link your NPM projects automatically, for sophisticated / modular local development.

Example of an ETL Pipeline using Airflow

A series of DAGs/Workflows to help maintain the operation of Airflow

Astronomer Core Docker Images

Udacity Data Engineering Nano Degree (DEND)

A template for new Symfony applications using Docker: ./install.sh && 🚀

Elyra extends JupyterLab Notebooks with an AI centric approach.

A plugin for Apache Airflow that allows you to edit DAGs in browser

ETLy is an add-on dashboard service on top of Apache Airflow.

DDEV-Local: a local PHP development environment system

Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)

A Front-end template for building web apps or sites using Pug(Jade) and Sass

Example end to end data engineering project.

Airflow workflow management platform chef cookbook.

Automating Your Data Pipeline with Apache Airflow