136 Open source projects that are alternatives of or similar to incremental_training

A lightweight tool to get an AI Infrastructure Stack up in minutes not days. K3ai will take care of setup K8s for You, deploy the AI tool of your choice and even run your code on it.

Airflow Unit Tests and Integration Tests

Airflow training for the crunch conf

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks

A web frontend for scheduling Jupyter notebook reports

Airflow workflow management platform chef cookbook.

Kedro Plugin to support running pipelines on Kubernetes using Airflow.

A Helm chart to install Apache Airflow on Kubernetes

A series of DAGs/Workflows to help maintain the operation of Airflow

Fast iterative local development and testing of Apache Airflow workflows

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.

Udacity Data Engineering Nano Degree (DEND)

Flask-AppBuilder SecurityManager for OpenIDConnect

Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor

A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability

Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]

Apache Airflow Website

Continuously synchronize directories from remote object store to local filesystem

Airflow plugin to export dag and task based metrics to Prometheus.

A facebook for data

💨🥫 A Data Factory system for running data processing pipelines built on AirFlow and tailored to CKAN. Includes evolution of DataPusher and Xloader for loading data to DataStore.

🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)

Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available out of box.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

afctl helps to manage and deploy Apache Airflow projects faster and smoother.

支持百度竞赛数据的中文事件抽取，支持ace2005数据的英文事件抽取，本人将苏神的三元组抽取算法中的DGCNN改成了事件抽取任务，并将karas改成了本人习惯使用的pytorch，在数据加载处考虑了各种语言的扩展

Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)

Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.

Bitnami Docker Image for Apache Airflow

Example DAGs using hooks and operators from Airflow Plugins

Example end to end data engineering project.

FuseML aims to provide an MLOps framework as the medium dynamically integrating together the AI/ML tools of your choice. It's an extensible tool built through collaboration, where Data Engineers and DevOps Engineers can come together and contribute with reusable integration code.

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Curated list of resources about Apache Airflow

ETLy is an add-on dashboard service on top of Apache Airflow.

Projects developed by Domino's R&D team

A distributed workflow runner focusing on performance and simplicity.

Metric collection, data testing and monitoring for SQL accessible data

Workflow engine for Kubernetes

Comet-For-MLFlow Extension

Bare minimal Airflow on Kubernetes (Local, EKS, AKS)

📖 [译] Airflow 中文文档

Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/

Blog post on ETL pipelines with Airflow

Elyra extends JupyterLab Notebooks with an AI centric approach.

🐳📊🤓Cookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)

Automating Your Data Pipeline with Apache Airflow

Evaluate wandb, tensorboard, neptune, mlflow, etc

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Airflow Deployment on AWS ECS Fargate Using Cloudformation

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

PipelineAI Kubeflow Distribution

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

The Tornado 🌪️ framework, designed and implemented for adaptive online learning and data stream mining in Python.

GoCD plugins to work with MLFlow as model repository in a CD flow

A way for home buyers to know about factors affecting a state

Framework for Analysis of Class-Incremental Learning with 12 state-of-the-art methods and 3 baselines.