343 Open source projects that are alternatives of or similar to fluent-plugin-webhdfs

Fluentd plugin to collect windows event logs

The input plugin of fluentd to pull log from rest api.

A Java NIO file system provider for HDFS

Kafka Connect FileSystem Connector

Hadoop utility to compact small files

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

Fundamentals of Spark with Python (using PySpark), code examples

Scalable data processing pipelines in JavaScript

大数据入门指南 ⭐

A tool and library for easily deploying applications on Apache YARN

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

基于wifi抓取信息的大数据查询分析系统

Docker image for main Apache Hadoop components (Yarn/Hdfs)

Mount HDFS with fuse, works with kerberos!

A complete (distributed) BigData stack, running in containers

🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结

ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.

A pandas-like deferred expression system, with first-class SQL support

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

Azure Linux monitoring agent (mdsd) output plugin for fluentd

Big Data Ecosystem Docker

同步Hive数据仓库数据到Elasticsearch的小工具

Fluentd output plugin to add Amazon EC2 metadata into messages

一个对用户行为日志进行分析的大数据项目

Hadoop splittable InputFormat for ROS. Process rosbag with Hadoop Spark and other HDFS compatible systems.

Google Cloud Storage output plugin for Fluentd.

个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。

零基础学习spark，大数据学习

Fast desktop client for Hadoop Distributed File System

大数据学习笔记，学习路线，技术案例整理。

Fluentd's Grok parser

💎🔥大数据学习笔记

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Multiprocess agent plugin for Fluentd

Kafka Connect HDFS connector

基于Hadoop的分布式云存储系统 🌴

Mirror of Linkedin's Camus

Redis output plugin for Fluent event collector

Fluent input/output plugin for RabbitMQ.

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

Manage Apache Atlas and Ranger configuration for your Hadoop environment.

最好的大数据项目。《Titan数据运营系统》，本项目是一个全栈闭环系统，我们有用作数据可视化的web系统，然后用flume-kafaka-flume进行日志的读取，在hive设计数仓，编写spark代码进行数仓表之间的转化以及ads层表到mysql的迁移，使用azkaban进行定时任务的调度，使用技术：Java/Scala语言，Hadoop、Spark、Hive、Kafka、Flume、Azkaban、SpringBoot，Bootstrap， Echart等；

clusterdock is a framework for creating Docker-based container clusters

Documentation placeholder and utilities for all the other containers.

🐘 A local Hadoop cluster bootstrapper using Vagrant, Ansible, and Ambari.

와이빅타 엔지니어링팀의 자료를 정리해두는 곳입니다.

Bitnami Docker Image for Fluentd

Learn by doing projects

Code for the deployment of Hadoop clusters, written in Bourne or Bourne Again shell.

OLAP and ETL of Big Data

A yet another Fluentd logger for Rust. a.k.a. verification of Fluentd's forward protocol playground.

网站点击流离线日志分析

This repository contains Machine-Learning MapReduce codes for Hadoop which are written from scratch (without using any package or library). E.g. Prediction (Linear and Logistic Regression), Clustering (K-Means), Classification (KNN) etc.

Import data from clickhouse to hadoop with pure SQL

tools for bigData

Stackdriver Nozzle for Cloud Foundry Loggregator, Host Monitoring Agents BOSH Release

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

A structured logger implementation in Scala.