2100 Open source projects that are alternatives of or similar to Bigdata Notes

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

Moonbox is a DVtaaS (Data Virtualization as a Service) Platform

Server for the ListenBrainz project

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

A simplified, lightweight ETL Framework based on Apache Spark

🎉🎉🎉JAVA高级架构师技术栈==任何技能通过 “刻意练习” 都可以达到融会贯通的境界，就像烹饪一样，这里有一份JAVA开发技术手册，只需要增加自己练习的次数。🏃🏃🏃

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Web UI for Trino, Presto, Hive, Elasticsearch, SparkSQL

Example end to end data engineering project.

Train TensorFlow models on YARN in just a few lines of code!

A Docker container with a full Hadoop cluster setup with Spark and Zeppelin

Write your Spark data to Kafka seamlessly

MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

550+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Kafka, Docker, APIs, Hadoop, SQL, PostgreSQL, MySQL, Hive, Impala, Travis CI, Jenkins, Concourse, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, .tmux.conf, .psqlrc ...

Java源码分析 【源码笔记】专注于Java后端系列框架的源码分析，每周持续推出Java后端系列框架的源码分析文章。

A serverless cluster computing system for the Go programming language

编程电子书，电子书，编程书籍，包括C，C#，Docker，Elasticsearch，Git，Hadoop，HeadFirst，Java，Javascript，jvm，Kafka，Linux，Maven，MongoDB，MyBatis，MySQL，Netty，Nginx，Python，RabbitMQ，Redis，Scala，Solr，Spark，Spring，SpringBoot，SpringCloud，TCPIP，Tomcat，Zookeeper，人工智能，大数据类，并发编程，数据库类，数据挖掘，新面试题，架构设计，算法系列，计算机类，设计模式，软件测试，重构优化，等更多分类

Superman是什么：构建Java 高级开发技术的知识体系，从基础不断打怪升级成为超人之路（更新中.......）

Java生态研究(Spring Boot + Redis + Dubbo + RocketMQ + Elasticsearch)🔥🔥🔥🔥🔥

Kafka Zookeeper RESTful API to perform topic/consumer group administration/metric(offset\lag\message) collection and monitor

docker compose files to create a fully working kafka stack

An open source framework for building data analytic applications.

Geo Spatial Data Analytics on Spark

✍️ Java 工程师必备架构体系知识总结：涵盖分布式、微服务、RPC等互联网公司常用架构，以及数据存储、缓存、搜索等必备技能

⚡️ Nepxion Thunder is a distribution RPC framework based on Netty + Hessian + Kafka + ActiveMQ + Tibco + Zookeeper + Redis + Spring Web MVC + Spring Boot + Docker 多协议、多组件、多序列化的分布式RPC调用框架

Serverless Platform for the stateful services

Kafka Web UI

大数据博客、笔试题、教程、项目、面经的整理

程序员进阶书籍（视频），持续更新（Programmer Books）

Alluxio, data orchestration for analytics and machine learning in the cloud

A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.

Collection of Kubernetes Operators built with KUDO.

基于分布式的云笔记（参考某道云笔记），数据存储在redis与hbase中

gomrjob - a Go Framework for Hadoop Map Reduce Jobs

A cohesive & pragmatic framework of FP centric Scala libraries

C4E, a JVM friendly library written in Scala for both local and distributed (Spark) Clustering.

Export Hadoop YARN (resource-manager) metrics in prometheus format

equivalent to kafka-streams 🐙 for nodejs ✨🐢🚀✨

Apache Spark Website

🍉基于Hadoop和HBase的大规模海量数据去重

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

Smart Automation Tool for building modern Data Lakes and Data Pipelines

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

电商用户行为分析大数据平台

A framework for writing Spark 2.x applications in a pretty way

Deploy a secured, clustered, auto-scaling NiFi service in AWS.

学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、网站、工具。涉及大数据几大组件、Python机器学习和数据分析、Linux、操作系统、算法、网络等

type-class based data cleansing library for Apache Spark SQL

Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)

Streaming reference architecture for ETL with Kafka and Kafka-Connect. You can find more on http://lenses.io on how we provide a unified solution to manage your connectors, most advanced SQL engine for Kafka and Kafka Streams, cluster monitoring and alerting, and more.

Python client for Hadoop® YARN API

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.

Computes and visualizes the sentiment analysis of tweets of US States in real-time using Storm.

Kubernetes Universal Declarative Operator (KUDO)

SiteWhere is an industrial strength open-source application enablement platform for the Internet of Things (IoT). It provides a multi-tenant microservice-based infrastructure that includes device/asset management, data ingestion, big-data storage, and integration through a modern, scalable architecture. SiteWhere provides REST APIs for all system functionality. SiteWhere provides SDKs for many common device platforms including Android, iOS, Arduino, and any Java-capable platform such as Raspberry Pi rapidly accelerating the speed of innovation.

Distributed Stream and Batch Processing

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.