795 Open source projects that are alternatives of or similar to big_data

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

大数据入门指南 ⭐

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Apache Spark Course Material

Fundamentals of Spark with Python (using PySpark), code examples

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

大数据学习笔记，学习路线，技术案例整理。

bigdatatutorial

ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结

大数据生态圈学习

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Asakusa Framework

Big Data for Data Engineers Coursera Specialization from Yandex

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.

Apache Drill is a distributed MPP query layer for self describing data

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

A large-scale entity and relation database supporting aggregation of properties

🐳 big data study

Mirror of Apache Calcite - Avatica

The official home of the Presto distributed SQL query engine for big data

Apache Calcite

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Full featured TypeScript Twitter archive reader and browser

可视化大屏解决方案, 提供一套可视化编辑引擎, 助力个人或企业轻松定制自己的可视化大屏应用.

Griffon Data Science Virtual Machine

Distributed Big Data Orchestration Service

Big Data Toolkit for the JVM

clusterdock is a framework for creating Docker-based container clusters

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

R on Apache Spark (SparkR) tutorials for Big Data analysis and Machine Learning as IPython / Jupyter notebooks

Ultimate Tennis Statistics and Tennis Crystal Ball - Tennis Big Data Analysis and Prediction

C4E, a JVM friendly library written in Scala for both local and distributed (Spark) Clustering.

Data validation library for PySpark 3.0.0

Companion to Learning Hadoop and Learning Spark courses on Linked In Learning

A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.

gomrjob - a Go Framework for Hadoop Map Reduce Jobs

Big Data Processing Framework - Unified Data API or SQL on Any Storage

编程路上加油站！------【持续更新中...欢迎star,欢迎常回来看看......】【内容：编程/学习/阅读资源，开源项目,面试题,网站,书,博客,教程等等】

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

Documentation placeholder and utilities for all the other containers.

SQL Parsers for BigData, built with antlr4.

No description or website provided.

Mapreduce project by Hadoop, Nutch, AWS EMR, Pig, Tez, Hive

🍉基于Hadoop和HBase的大规模海量数据去重

Information Inference Service of the OpenAIRE system

网站点击流离线日志分析

Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )

Data pipeline project

Anovos - An Open Source Library for Scalable feature engineering Using Apache-Spark

All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above

Samples to the Oracle Big Data Lite VM

rastercube is a python library for big data analysis of georeferenced time series data (e.g. MODIS NDVI)

Tidy up Spark and Hadoop tutorials.

HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

Pyspark in Google Colab: A simple machine learning (Linear Regression) model