637 Open source projects that are alternatives of or similar to Spark On K8s Operator

RabbitMQ Kubernetes operator

A connector for Spark that allows reading and writing to/from Redis cluster

An ad hoc query service based on the spark sql engine.(基于spark sql引擎的即席查询服务)

Ytk-learn is a distributed machine learning library which implements most of popular machine learning algorithms(GBDT, GBRT, Mixture Logistic Regression, Gradient Boosting Soft Tree, Factorization Machines, Field-aware Factorization Machines, Logistic Regression, Softmax).

A Flexible and Powerful Parameter Server for large-scale machine learning

Apache Spark examples exclusively in Java

An improved implementation of the classical feature selection method: minimum Redundancy and Maximum Relevance (mRMR).

Apache Zeppelin notebooks for Recommendation Engines using Keras and Machine Learning on Apache Spark

The Metering Operator is responsible for collecting metrics and other information about what's happening in a Kubernetes cluster, and providing a way to create reports on the collected data.

TPC-H queries in Apache Spark SQL using native DataFrames API

An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset

prototype kubernetes operator for couchDB

hadoop各组件使用，持续更新

Kubernetes Operator to automatically delete completed Jobs and their Pods

Jupyter magics and kernels for working with remote Spark clusters

A boilerplate for writing PySpark Jobs

Apache Spark docker container image (Standalone mode)

Expressive types for Spark.

人生苦短 我用Python

A simple Spark-powered ETL framework that just works 🍺

Visualize column-level data lineage in Spark SQL

Use Jupyter Notebooks to demonstrate how to build a Recommender with Apache Spark & Elasticsearch

Kafka Consumer Operator. Kubernetes operator to manage consumers of unbalanced kafka topics with per-partition vertical autoscaling based on Prometheus metrics

scala、spark使用过程中，各种测试用例以及相关资料整理

Autoscaling for Google Cloud Dataproc

电商用户行为分析大数据平台

Online search advertisement platform & Realtime Campaign Monitoring [Maybe Deprecated]

个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。

A cohesive & pragmatic framework of FP centric Scala libraries

low-level helpers for Apache Spark libraries and tests

A curated list of awesome Apache Spark packages and resources.

Map Reduce Implementation of Connected Component on Apache Spark

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Operator allows you to run stream processing logic directly on a Kubernetes cluster

Singularity implementation of k8s operator for interacting with SLURM.

Sparkline BI Accelerator provides fast ad-hoc query capability over Logical Cubes. This has been folded into our SNAP Platform(http://bit.ly/2oBJSpP) an Integrated BI platform on Apache Spark.

DataFusion has now been donated to the Apache Arrow project

Open-source distribute workflow schedule tools, also support streaming task.

A Kubernetes Operator for RabbitMQ

Resources for Spark AR

The MongoDB Spark Connector

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

A tool for monitoring and tuning Spark jobs for efficiency.

Writing application logic for Spark jobs that can be unit-tested without a SparkContext

Bucketing and partitioning system for Parquet

Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark

Spark functions to run popular phonetic and string matching algorithms

A command-line tool for launching Apache Spark clusters.

Dependencies verification system with Kubernetes Operator

(Deprecated) Scikit-learn integration package for Apache Spark

Streaming System 相关的论文读物

Utilities for writing Kubernetes controllers & operators

Apache Spark Website

MapReduce, Spark, Java, and Scala for Data Algorithms Book

ClickHouse Native Protocol JDBC implementation

Shell-operator is a tool for running event-driven scripts in a Kubernetes cluster

High performance HBase / Spark SQL engine

Simple framework agnostic UI router for SPAs

An open-source storage layer that brings scalable, ACID transactions to Apache Spark™ and big data workloads.