2100 Open source projects that are alternatives of or similar to Bigdata Notes

An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset

flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

Spring Boot 2.x code examples

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

A pandas-like deferred expression system, with first-class SQL support

大数据分析可视化实践

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

C# and F# language binding and extensions to Apache Spark

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

推荐项目（实时推荐和离线推荐）

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

💥 🚀 封装sparkstreaming动态调节batch time(有数据就执行计算)；🚀 支持运行过程中增删topic；🚀 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。

A daily digest of the articles or videos I've found interesting, that I want to share with you.

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

大数据相关内容汇总，包括分布式存储引擎、分布式计算引擎、数仓建设等。关键词：Hadoop、HBase、ES、Kudu、Hive、Presto、Spark、Flink、Kylin、ClickHouse

Docker image for main Apache Hadoop components (Yarn/Hdfs)

MapReduce, Spark, Java, and Scala for Data Algorithms Book

All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above

同步Hive数据仓库数据到Elasticsearch的小工具

A big data platform for analyzing web access logs

HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)

汇总java生态圈常用技术框架、开源中间件，系统架构、数据库、大公司架构案例、常用三方类库、项目管理、线上问题排查、个人成长、思考等知识

A Spark cluster setup running on Docker containers

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

📁 Extract, Transform, Load (ETL) 👷 refers to a process in database usage and especially in data warehousing. This repository contains a starter kit featuring ETL related work.

Mapreduce project by Hadoop, Nutch, AWS EMR, Pig, Tez, Hive

ACID Data Source for Apache Spark based on Hive ACID

DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。

Apache Phoenix / Hbase Spring Boot Microservices

OLAP and ETL of Big Data

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Data Lineage Tracking And Visualization Solution

Docker build project to setup a lightweight hadoop cluster containing hadoop, pig, zookeeper, hbase, phoenix, storm, kafka, kafka manager

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

Kyuubi is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark

Building Large-Scale AI Applications for Distributed Big Data

Apache Hive

基于 scrapy-redis 的通用分布式爬虫框架

一个大数据架构师应该掌握的技能

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

Server for the ListenBrainz project

Examples for Deep Learning/Feature Store/Spark/Flink/Hive/Kafka jobs and Jupyter notebooks on Hops

Generic Data Ingestion & Dispersal Library for Hadoop

Moonbox is a DVtaaS (Data Virtualization as a Service) Platform

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

🎉🎉🎉JAVA高级架构师技术栈==任何技能通过 “刻意练习” 都可以达到融会贯通的境界，就像烹饪一样，这里有一份JAVA开发技术手册，只需要增加自己练习的次数。🏃🏃🏃

d3 library to build circular graphs

关于kafka的一些相关使用示例代码。

Web UI for Trino, Presto, Hive, Elasticsearch, SparkSQL

大数据博客、笔试题、教程、项目、面经的整理

Java源码分析 【源码笔记】专注于Java后端系列框架的源码分析，每周持续推出Java后端系列框架的源码分析文章。

A serverless cluster computing system for the Go programming language

编程电子书，电子书，编程书籍，包括C，C#，Docker，Elasticsearch，Git，Hadoop，HeadFirst，Java，Javascript，jvm，Kafka，Linux，Maven，MongoDB，MyBatis，MySQL，Netty，Nginx，Python，RabbitMQ，Redis，Scala，Solr，Spark，Spring，SpringBoot，SpringCloud，TCPIP，Tomcat，Zookeeper，人工智能，大数据类，并发编程，数据库类，数据挖掘，新面试题，架构设计，算法系列，计算机类，设计模式，软件测试，重构优化，等更多分类

Geo Spatial Data Analytics on Spark

An open source framework for building data analytic applications.

程序员进阶书籍（视频），持续更新（Programmer Books）