115 Open source projects that are alternatives of or similar to ml-in-production

Airflow Deployment on AWS ECS Fargate Using Cloudformation

PipelineX: Python package to build ML pipelines for experimentation with Kedro, MLflow, and more

A tool for building feature stores.

Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive

Accelerate data science

📊 📋 Dashboards using YAML or JSON files

Apache Superset is a Data Visualization and Data Exploration Platform

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

Example end to end data engineering project.

A simple Spark-powered ETL framework that just works 🍺

Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

Ansible playbook to deploy distributed technologies

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

Quilt is a self-organizing data hub for S3

dbt adapter for SQL Server and Azure SQL

Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017

Git-like capabilities for your object storage

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

The easiest way to automate your data

Clean APIs for data cleaning. Python implementation of R package Janitor

Example project implementing best practices for PySpark ETL jobs and applications.

Data validation and organization of metadata for data frames and database tables

Accumulated knowledge and experience in the field of Data Engineering

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Always know what to expect from your data.

Turn complex requirements to workflows without leaving the comfort of your technology stack.

An Awesome List of Open-Source Data Engineering Projects

📝 A compilation of everything that I learn; Computer Science, Software Development, Engineering, Math, and Coding in General. Read the rendered results here ->

Dataform is a framework for managing SQL based data operations in BigQuery, Snowflake, and Redshift

Open Metadata and Governance

Fancy stream processing made operationally mundane

A Data Engineering & Machine Learning Knowledge Hub

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

Feature Store for Machine Learning

The Data Engineering Cookbook

A python package to create a database on the platform using our moj data warehousing framework

Automate building ML classification pipelines in .NET

Open Source Feature Flagging and A/B Testing Platform

ELT Code for your Data Warehouse

Python library for building Pangeo Forge recipes.

A comprehensive list of 180+ YouTube Channels for Data Science, Data Engineering, Machine Learning, Deep learning, Computer Science, programming, software engineering, etc.

Summary of my projects on kaggle

Deep Neural Network architecture as a predictive optimal controller for {HVAC+Solar cell + battery} disturbance afflicted system vs classic Model Predictive Control

This repo contains commands that data engineers use in day to day work.

An orchestration platform for the development, production, and observation of data assets.

Dataset format for AI. Build, manage, & visualize datasets for deep learning. Stream data real-time to PyTorch/TensorFlow & version-control it. https://activeloop.ai

ARAKAT - Big Data Analysis and Business Intelligence Application Development Platform

Spark-Transformers: Library for exporting Apache Spark MLLIB models to use them in any Java application with no other dependencies.

Curated list of resources about Apache Airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

A plugin for Apache Airflow that allows you to manage the users that can login

A plugin for Apache Airflow that allows you to edit DAGs in browser

Identifies and collects data on cc-licensed content across web crawl data and public apis.

Export Airflow metrics (from mysql) in prometheus format