1537 Open source projects that are alternatives of or similar to Bigdata

Kafka Connect HDFS connector

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Management and automation platform for Stateful Distributed Systems

gomrjob - a Go Framework for Hadoop Map Reduce Jobs

A large-scale entity and relation database supporting aggregation of properties

Mirror of Linkedin's Camus

The official home of the Presto distributed SQL query engine for big data

基于 scrapy-redis 的通用分布式爬虫框架

Fundamentals of Spark with Python (using PySpark), code examples

【Java面试+Java学习指南】 一份涵盖大部分Java程序员所需要掌握的核心知识。

✍️ Java 工程师必备架构体系知识总结：涵盖分布式、微服务、RPC等互联网公司常用架构，以及数据存储、缓存、搜索等必备技能

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Educational notes,Hands on problems w/ solutions for hadoop ecosystem

Docker image for main Apache Hadoop components (Yarn/Hdfs)

Smart Automation Tool for building modern Data Lakes and Data Pipelines

Companion to Learning Hadoop and Learning Spark courses on Linked In Learning

基于Hadoop的分布式云存储系统 🌴

基于微服务架构的在线电影购票平台

基于hadoop+hbase+springboot实现分布式网盘系统

涵盖大部分Java进阶需要掌握的知识，包括【微服务】【中间件】【缓存】【数据库优化】【搜索引擎】【分布式】等等，欢迎Star~

Repair hbase metadata table from hdfs.

🎉🎉🎉JAVA高级架构师技术栈==任何技能通过 “刻意练习” 都可以达到融会贯通的境界，就像烹饪一样，这里有一份JAVA开发技术手册，只需要增加自己练习的次数。🏃🏃🏃

Kafka Connect FileSystem Connector

Scalable data processing pipelines in JavaScript

Apache Phoenix / Hbase Spring Boot Microservices

HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)

All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL

Liquibase extension to add Impala Database support

大数据生态圈学习

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Data pipeline project

An alternative to the "hive standalone" jar for connecting Java applications to Apache Hive via JDBC

Fast desktop client for Hadoop Distributed File System

Core utility library & data connectors designed for simpler usage in Scala

A tool and library for easily deploying applications on Apache YARN

A set of commands for managing CDH clusters using Cloudera Manager REST API.

Hadoop utility to compact small files

Manage Apache Atlas and Ranger configuration for your Hadoop environment.

Mount HDFS with fuse, works with kerberos!

编程电子书，电子书，编程书籍，包括C，C#，Docker，Elasticsearch，Git，Hadoop，HeadFirst，Java，Javascript，jvm，Kafka，Linux，Maven，MongoDB，MyBatis，MySQL，Netty，Nginx，Python，RabbitMQ，Redis，Scala，Solr，Spark，Spring，SpringBoot，SpringCloud，TCPIP，Tomcat，Zookeeper，人工智能，大数据类，并发编程，数据库类，数据挖掘，新面试题，架构设计，算法系列，计算机类，设计模式，软件测试，重构优化，等更多分类

Avro Schema Evolution made easy

Hadoop splittable InputFormat for ROS. Process rosbag with Hadoop Spark and other HDFS compatible systems.

와이빅타 엔지니어링팀의 자료를 정리해두는 곳입니다.

Hive Storage Handler for interoperability between BigQuery and Apache Hive

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

Today I Learned

Hadoop WebHDFS output plugin for Fluentd

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Perl Utility Library for my other repos

Addax is an open source universal ETL tool that supports most of those RDBMS and NoSQLs on the planet, helping you transfer data from any one place to another.

Docker build project to setup a lightweight hadoop cluster containing hadoop, pig, zookeeper, hbase, phoenix, storm, kafka, kafka manager

Apache Hive

Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.

Cascading is a feature rich API for defining and executing complex and fault tolerant data processing flows locally or on a cluster. See https://github.com/Cascading/cascading for the release repository.

Axibase Time Series Database Documentation

Hive UDFs for funnel analysis

最好的大数据项目。《Titan数据运营系统》，本项目是一个全栈闭环系统，我们有用作数据可视化的web系统，然后用flume-kafaka-flume进行日志的读取，在hive设计数仓，编写spark代码进行数仓表之间的转化以及ads层表到mysql的迁移，使用azkaban进行定时任务的调度，使用技术：Java/Scala语言，Hadoop、Spark、Hive、Kafka、Flume、Azkaban、SpringBoot，Bootstrap， Echart等；