526 Open source projects that are alternatives of or similar to Data Engineering Howto

The easiest way to automate your data

A comprehensive list of 180+ YouTube Channels for Data Science, Data Engineering, Machine Learning, Deep learning, Computer Science, programming, software engineering, etc.

Distributed Big Data Orchestration Service

Hybrid Logical Clocks for Elixir

高性能golang网络库，游戏开发脚手架

Auklet is a high performance storage engine based on Openstack Swift

Simple to use but full configurable p2p framework

Deep Neural Network architecture as a predictive optimal controller for {HVAC+Solar cell + battery} disturbance afflicted system vs classic Model Predictive Control

open source training courses about distributed database and distributed systemes

🔖 Guides, Articles, Podcasts, Videos and Notes to Build Reliable Large-Scale Distributed Systems.

MIT6.824分布式系统（2018秋）

Actor based reactive java framework for microservices in local and distributed environment

Mysterium Network Node - official implementation of distributed VPN network (dVPN) protocol

This repo contains commands that data engineers use in day to day work.

A simple Spark-powered ETL framework that just works 🍺

Instant Kubernetes-Native Application Observability

The enterprise-grade behavioral data engine (web, mobile, server-side, webhooks), running cloud-natively on AWS and GCP

file_management_sys 是一个文件共享系统，包括前端文件展示系统和后台管理系统，基于SpringBoot + MyBatis实现。前端文件展示系统包括文件分类和展示界面，文件搜索和文件上传等模块。后台管理系统包含文件管理，权限管理等模块。

Pegasus Workflow Management System - Automate, recover, and debug scientific computations.

A system design tool that allows you to simulate data flow of distributed systems.

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

distributed system;blokchain;filecoin/ipfs,...

Keybase Filesystem (KBFS)

The practical use-cases of how to make your Machine Learning Pipelines robust and reliable using Apache Airflow.

High-performance, high-scalability distributed computing with Erlang and Elixir.

A Chinese version of MIT 6.824 (Distributed System)

Python Stream Processing. A Faust fork

Augmentation pipeline for rendering synthetic paper printing, faxing, scanning and copy machine processes

Lightweight and Scalable framework that combines mainstream algorithms of Click-Through-Rate prediction based computational DAG, philosophy of Parameter Server and Ring-AllReduce collective communication.

NKN shell daemon

已作为 Hasor 的子项目，迁移到：http://git.oschina.net/zycgit/hasor

A Node.js and JavaScript synchronous data pipeline processing, data sharing and stream processing library. Actionable & Transformable Pipeline data processing.

项目停止更新，新项目：https://github.com/RayTale/Vertex

Personal Data Engineering Projects

This is the demo application that i created for my talk 'Microservice Architecture & Implementation with Asp.Net Core' at Dotnet İstanbul Meetup Group.

Python Stream Processing

Multi backend queues for Golang

Fancy stream processing made operationally mundane

Distributed LMS for automating Computing Science Courses From NYU

Decentralized trust ratings using signed claims

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

Kubernetes-native Deep Learning Framework

A curated list of all the Awesome --Topic Name-- lists I've found till date relevant to Data lifecycle, ML and DL.

Examples for the Manning "Vert.x in Action" book

nact ⇒ node.js + actors ⇒ your services have never been so µ

Instant Kubernetes-Native Application Observability

Scalable, fault-tolerant, highly-available cache

The Data Engineering Cookbook

Farm operated by bots to grow and harvest new zim files

Golimit is Uber ringpop based distributed and decentralized rate limiter

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

Distributed reliable key-value store for the most critical data of a distributed system

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Facebook, Amazon, Apple, Netflix and Google (FAANG) Job preparation.

Orleans Membership provider for Kubernetes

The Accelerator is a tool for fast and reproducible processing of large amounts of data.

A graph-based distributed in-memory store that leverages efficient graph exploration to provide highly concurrent and low-latency queries over big linked data

Lightweight, facility, high performance golang based game server framework

Raft distributed consensus algorithm implemented in Rust.

Distributed Rails with commands, events and projections.