Top 231 hadoop open source projects

Hadoop utility to compact small files

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Teradata Distribution of Presto -- A Distributed SQL Query Engine for Big Data

Visualizations of the Hadoop Ecosystem

Liquibase extension to add Impala Database support

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL

General Architecture for Text Engineering

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)

rastercube is a python library for big data analysis of georeferenced time series data (e.g. MODIS NDVI)

Tidy up Spark and Hadoop tutorials.

Terraform module to deploy Cloudera on Oracle Cloud Infrastructure (OCI)

Prometheus jmx_exporter configurations for Cloudera Hadoop

A tool and library for easily deploying applications on Apache YARN

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

A library for manipulating bioinformatics sequencing formats in Apache Spark

An ORC File Scheme for the Cascading data processing platform.

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

Pyspark in Google Colab: A simple machine learning (Linear Regression) model

基于hadoop+hbase+springboot实现分布式网盘系统

[Archive] Intern project - Big Data Exploration using MongoDB - This Repository is NOT a supported MongoDB product

论坛日志分析系统清洗程序(包含IP规则库，UDF开发，MapReduce程序，日志数据)

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

大数据生态圈学习

Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )

Data pipeline project

All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above

Hive Storage Handler for interoperability between BigQuery and Apache Hive

Information Inference Service of the OpenAIRE system

同步Hive数据仓库数据到Elasticsearch的小工具

基于Hadoop的分布式云存储系统 🌴

Smart Automation Tool for building modern Data Lakes and Data Pipelines

Companion to Learning Hadoop and Learning Spark courses on Linked In Learning

大数据学习笔记，学习路线，技术案例整理。

Open Source Phasor Data Concentrator

Node.js WebHDFS REST API client

大数据相关内容汇总，包括分布式存储引擎、分布式计算引擎、数仓建设等。关键词：Hadoop、HBase、ES、Kudu、Hive、Presto、Spark、Flink、Kylin、ClickHouse

TonY is a framework to natively run deep learning frameworks on Apache Hadoop.

Export Hadoop YARN (resource-manager) metrics in prometheus format

gomrjob - a Go Framework for Hadoop Map Reduce Jobs

Scalable data processing pipelines in JavaScript

Simple Java Framework,designed for easily develop Spring based java program.Support Bigdata And metadata management.A common elasticsearch comm query tool and so on.

Distributed Java micro-services using ZooKeeper

Management and automation platform for Stateful Distributed Systems

Install hadoop cluster with ansible

Source code and dataset for paper "CBMR: An optimized MapReduce for item‐based collaborative filtering recommendation algorithm with empirical analysis"

Dockerfiles and Docker Compose for HDP 2.6 with Blueprints

Kafka Connect FileSystem Connector

Apache Phoenix / Hbase Spring Boot Microservices

bigdatatutorial

Docker image for main Apache Hadoop components (Yarn/Hdfs)