Top 12 datalake open source projects

Upserts, Deletes And Incremental Processing on Big Data.

The LeoFS Storage System

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Apache Spark Course Material

A custom extractor designed to read parquet for Azure Data Lake Analytics

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi

Apiary provides modules which can be combined to create a federated cloud data lake

Python idiomatic SDK for Cortex™ Data Lake.

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

Terraform scripts for deploying Apiary Data Lake

Dinky is an out of the box one-stop real-time computing platform dedicated to the construction and practice of Unified Streaming & Batch and Unified Data Lake & Data Warehouse. Based on Apache Flink, Dinky provides the ability to connect many big data frameworks including OLAP and Data Lake.