1714 Open source projects that are alternatives of or similar to fastdata-cluster

🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结

大数据入门指南 ⭐

个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive

Fundamentals of Spark with Python (using PySpark), code examples

Big Data Ecosystem Docker

A pandas-like deferred expression system, with first-class SQL support

Vagrant projects for various use-cases with Spark, Zeppelin, IPython / Jupyter, SparkR

Production Ready Data Integration Product, documentation：

A complete (distributed) BigData stack, running in containers

Create clusters of VMs on the cloud and configure them with Ansible.

Docker image for main Apache Hadoop components (Yarn/Hdfs)

A tool and library for easily deploying applications on Apache YARN

Spark、Flink等离线任务的调度以及实时任务的监控

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

50+ DockerHub public images for Docker & Kubernetes - Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak, TeamCity and DevOps tools built on the major Linux distros: Alpine, CentOS, Debian, Fedora, Ubuntu

零基础学习spark，大数据学习

深圳地铁大数据客流分析系统🚇🚄🌟

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

大数据学习笔记，学习路线，技术案例整理。

Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

DataFibers Data Service

Spark Streaming监控平台，支持任务部署与告警、自启动

Elassandra = Elasticsearch + Apache Cassandra

A large-scale entity and relation database supporting aggregation of properties

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

🐘 A local Hadoop cluster bootstrapper using Vagrant, Ansible, and Ambari.

extremely distributed machine learning

商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark

DataStax Spark Cassandra Connector

Spark Structured Streaming / Kafka / Cassandra / Elastic

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learni…

Java开发者或者大数据开发者面试知识点整理

💥 🚀 封装sparkstreaming动态调节batch time(有数据就执行计算)；🚀 支持运行过程中增删topic；🚀 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

Getting started with Docker swarm

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Compile-time Language Integrated Queries for Scala

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Kubernetes cluster (for testing purposes) made easy with Vagrant and CoreOS.

A Kubernetes Vagrant Multi node environment using kubeadm.

Distributed Java micro-services using ZooKeeper

从本地IDEA提交Flink/Spark任务到Yarn/k8s集群

Mount HDFS with fuse, works with kerberos!

Export Hadoop YARN (resource-manager) metrics in prometheus format

Kafka Connect FileSystem Connector

Scalable data processing pipelines in JavaScript

DEPRECATED Simple Node.js Development VM using Vagrant + VirtualBox + Ansible

基于Hadoop的分布式云存储系统 🌴

同步Hive数据仓库数据到Elasticsearch的小工具