231 Open source projects that are alternatives of or similar to Hadoop

A pandas-like deferred expression system, with first-class SQL support

Docker containers for Hadoop.

Multiple node cluster on Docker for self development.

ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows

Road to Machine Learning

TonY is a framework to natively run deep learning frameworks on Apache Hadoop.

Asakusa Framework

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Mirror of Apache Chukwa

Alluxio, data orchestration for analytics and machine learning in the cloud

HBase客户端数据管理软件

💎🔥大数据学习笔记

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.

Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Presto and AWS Athena.

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Educational notes,Hands on problems w/ solutions for hadoop ecosystem

Generic Data Ingestion & Dispersal Library for Hadoop

Griffon Data Science Virtual Machine

Kafka Connect HDFS connector

Axibase Time Series Database Documentation

Iceberg is a table format for large, slow-moving tabular data

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

Apache Ignite

Jumbune, an open source BigData APM & Data Quality Management Platform for Data Clouds. Enterprise feature offering is available at http://jumbune.com. More details of open source offering are at,

Apache Hive

Big Data Toolkit for the JVM

Ytk-learn is a distributed machine learning library which implements most of popular machine learning algorithms(GBDT, GBRT, Mixture Logistic Regression, Gradient Boosting Soft Tree, Factorization Machines, Field-aware Factorization Machines, Logistic Regression, Softmax).

An implementation of locality sensitive hashing with Hadoop

Gathers scalable tensorflow and infrastructure deployment

Production Ready Data Integration Product, documentation：

Apache Tez

A Docker container with a full Hadoop cluster setup with Spark and Zeppelin

Data Lineage Tracking And Visualization Solution

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Create clusters of VMs on the cloud and configure them with Ansible.

https://www.researchgate.net/profile/Rajah_Iyer

Lightweight, simple structured NoSQL database for Android

hadoop-mini-clusters provides an easy way to test Hadoop projects directly in your IDE

450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...

phData Pulse application log aggregation and monitoring

Mirror of Apache Calcite - Avatica

Deprecated, please use https://github.com/jcrist/skein or https://github.com/dask/dask-yarn instead

零基础学习spark，大数据学习

A complete (distributed) BigData stack, running in containers

AntsDB is a low latency, high concurrency, MySQL compliant SQL layer for HBase

qihoo360 xlearning with GPU support; AI on Hadoop

An easy way to deploy your Akka services to a distributed environment.

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

Real-time analysis and visualization with Storm-AMQ-Camel-Websockets-Highcharts integration.

Apache Parquet implementation in Rust

AI on Hadoop

A large-scale entity and relation database supporting aggregation of properties

Apache Drill is a distributed MPP query layer for self describing data

Python client for Hadoop® YARN API

🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结