571 Open source projects that are alternatives of or similar to bigdata-doc

懒松鼠Flink-Boot 脚手架让Flink全面拥抱Spring生态体系，使得开发者可以以Java WEB开发模式开发出分布式运行的流处理程序，懒松鼠让跨界变得更加简单。懒松鼠旨在让开发者以更底上手成本（不需要理解分布式计算的理论知识和Flink框架的细节）便可以快速编写业务代码实现。为了进一步提升开发者使用懒松鼠脚手架开发大型项目的敏捷的度，该脚手架默认集成Spring框架进行Bean管理，同时将微服务以及WEB开发领域中经常用到的框架集成进来，进一步提升开发速度。比如集成Mybatis ORM框架，Hibernate Validator校验框架,Spring Retry重试框架等，具体见下面的脚手架特性。

基于Hadoop的分布式云存储系统 🌴

Based on Apache Flink. support data synchronization/integration and streaming SQL computation.

Tidy up Spark and Hadoop tutorials.

商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark

Hadoop splittable InputFormat for ROS. Process rosbag with Hadoop Spark and other HDFS compatible systems.

Documentation placeholder and utilities for all the other containers.

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Spark、Flink等离线任务的调度以及实时任务的监控

Apache Hive

实践源码库：https://github.com/jast90/bigdata 。 微信搜索Jast关注公众号，获取最新技术分享😯。

An end-to-end machine learning and data mining framework on Hadoop

Hive JDBC "uber" or "standalone" jar based on the latest Apache Hive version

Examples for Deep Learning/Feature Store/Spark/Flink/Hive/Kafka jobs and Jupyter notebooks on Hops

Kafka Connect FileSystem Connector

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Hive UDFs for funnel analysis

一个大数据架构师应该掌握的技能

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.

A Java NIO file system provider for HDFS

MapReduce, Spark, Java, and Scala for Data Algorithms Book

Easy parsing of Apache HTTPD and NGINX access logs with Java, Hadoop, Hive, Pig, Flink, Beam, Storm, Drill, ...

Apache Drill is a distributed MPP query layer for self describing data

基于Spark2.2新闻网大数据实时系统项目

flink学习笔记

Production Ready Data Integration Product, documentation：

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Hive Storage Handler for interoperability between BigQuery and Apache Hive

Addax is an open source universal ETL tool that supports most of those RDBMS and NoSQLs on the planet, helping you transfer data from any one place to another.

Fundamentals of Spark with Python (using PySpark), code examples

A pandas-like deferred expression system, with first-class SQL support

Facebook's Hive UDFs

Java开发者或者大数据开发者面试知识点整理

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

Asakusa Framework

Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具

Smart Automation Tool for building modern Data Lakes and Data Pipelines

A collection of pentest tools and resources targeting Hadoop environments

An alternative to the "hive standalone" jar for connecting Java applications to Apache Hive via JDBC

大数据分析可视化实践

The official home of the Presto distributed SQL query engine for big data

Open-source distribute workflow schedule tools, also support streaming task.

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

Liquibase extension to add Impala Database support

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Today I Learned

Manage Apache Atlas and Ranger configuration for your Hadoop environment.

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

와이빅타 엔지니어링팀의 자료를 정리해두는 곳입니다.

云计算之hadoop、hive、hue、oozie、sqoop、hbase、zookeeper环境搭建及配置文件

最好的大数据项目。《Titan数据运营系统》，本项目是一个全栈闭环系统，我们有用作数据可视化的web系统，然后用flume-kafaka-flume进行日志的读取，在hive设计数仓，编写spark代码进行数仓表之间的转化以及ads层表到mysql的迁移，使用azkaban进行定时任务的调度，使用技术：Java/Scala语言，Hadoop、Spark、Hive、Kafka、Flume、Azkaban、SpringBoot，Bootstrap， Echart等；

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

📁 Extract, Transform, Load (ETL) 👷 refers to a process in database usage and especially in data warehousing. This repository contains a starter kit featuring ETL related work.

Big Data Toolkit for the JVM

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources