394 Open source projects that are alternatives of or similar to T-Watch

Bitnami Docker Image for Apache Airflow

AWS S3 Video Thumbnailer with Lambda

Example end to end data engineering project.

Blog post on ETL pipelines with Airflow

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'

ETLy is an add-on dashboard service on top of Apache Airflow.

💨🥫 A Data Factory system for running data processing pipelines built on AirFlow and tailored to CKAN. Includes evolution of DataPusher and Xloader for loading data to DataStore.

A distributed workflow runner focusing on performance and simplicity.

AWS SSM Session manager client

Workflow engine for Kubernetes

A general purpose metrics monitor implemented with Apache Spark. Kafka source, Elastic sink, aggregate metrics, different analysis, notifications, actions, live configuration update, missing metrics, ...

Bare minimal Airflow on Kubernetes (Local, EKS, AKS)

Counting Tweets Per User in Real-Time

Automatically create and configure Amazon CloudWatch alarms for EC2 instances, RDS, and AWS Lambda using tags for standard and custom CloudWatch Metrics.

Elyra extends JupyterLab Notebooks with an AI centric approach.

Apache Airflow Website

Automating Your Data Pipeline with Apache Airflow

akka-persistence journal/snapshot plugin for AWS S3(support aws sdk for java v2)

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Set of tools to provide automation of tasks for creating STIG-partitioned EL7 AMIs

Air Data Computer

🖥 A feature rich terminal UI file transfer and explorer with support for SCP/SFTP/FTP/S3

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

A lightweight tool for analyzing the performance and data integrity of object-based storage services

Turbine: the bare metals that gets you Airflow

TrackIt helps you to optimize your AWS cloud

API providing AWS EC2 Instance Type Data

A Helm chart to install Apache Airflow on Kubernetes

Personal Data Engineering Projects

Kubernetes custom controller and CRDs to managing Airflow

Example DAGs using hooks and operators from Airflow Plugins

Yet another helpdesk based on multiple providers

Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.

Minikube for big data with Scala and Spark

Curated list of resources about Apache Airflow

A plugin for Apache Airflow that allows you to manage the users that can login

基于spark-ml,spark-mllib,spark-streaming的推荐算法实现

Bitnami Docker Image for Apache Airflow Scheduler

Metric collection, data testing and monitoring for SQL accessible data

A plugin for Apache Airflow that allows you to edit DAGs in browser

JavaScript Image Uploader Library for use with Amazon S3

triggering a DAG run multiple times

📖 [译] Airflow 中文文档

Apache Airflow integration for dbt

Multithreaded recursive directory upload to S3 using FOG

Astro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

🐳📊🤓Cookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)

Export Airflow metrics (from mysql) in prometheus format

Go serverless website on AWS Lambda.

JobAnalytics system consumes data from multiple sources and provides valuable information to both job hunters and recruiters.

Airflow Deployment on AWS ECS Fargate Using Cloudformation

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

Observatory Platform Package

[*No longer maintained*] Nexus Repository S3 Blobstores

Upload multiple files to AWS S3, make them public, and get their URLs easily from the command line.

🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)

Get your MLOps (Level 1) platform started and going fast.

Build DataOps platform with Apache Airflow and dbt on AWS

Flask-AppBuilder SecurityManager for OpenIDConnect