399 Open source projects that are alternatives of or similar to Spark

When Apache Pulsar meets Apache Spark

dllib is a distributed deep learning library running on Apache Spark

SparklingGraph provides easy to use set of features that will give you ability to proces large scala graphs using Spark and GraphX.

尚硅谷大数据Spark-2019版最新 Spark 学习

scala、spark使用过程中，各种测试用例以及相关资料整理

Prosto is a data processing toolkit radically changing how data is processed by heavily relying on functions and operations with functions - an alternative to map-reduce and join-groupby

Applying Lambda Architecture with Spark, Kafka, and Cassandra.

Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.

blog entries

Spark Structured Streaming / Kafka / Cassandra / Elastic

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

The Internals of Apache Spark

A library that provides useful extensions to Apache Spark and PySpark.

java 进阶之路：面试高频算法、akka、多线程、NIO、Netty、SpringBoot、Spark&&Flink 等

A compiler for automatically re-targeting sequential Java code to Apache Spark.

This application comes as Spark2.1-as-Service-Provider using an embedded, Reactive-Streams-based, fully asynchronous HTTP server

Type System for Data Analysis in Python

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Streaming data changes to a Data Lake with Debezium and Delta Lake pipeline

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Spark SQL index for Parquet tables

Spark Shuffle Optimization with RDMA+AEP

Official code repository for GATK versions 4 and up

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Julia binding for Apache Spark

Apache Spark docker container image (Standalone mode)

Python Helper library for Jupyter Notebooks

人生苦短 我用Python

🐍 Quick reference guide to common patterns & functions in PySpark.

Visualize column-level data lineage in Spark SQL

Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster

spark on kubernetes

Node.js bindings for Apache Spark DataFrame APIs

Online search advertisement platform & Realtime Campaign Monitoring [Maybe Deprecated]

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

Apache Spark in your IDE with gradle

flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

Identifies and collects data on cc-licensed content across web crawl data and public apis.

零基础学习spark，大数据学习

❄️ 🐳 Awesome tools and libs for AI, Deep Learning, Machine Learning, Computer Vision, Data Science, Data Analytics and Cognitive Computing that are baked in the oven to be Native on Kubernetes and Docker with Python, R, Scala, Java, C#, Go, Julia, C++ etc

Big Data Ecosystem Docker

My presentation at ODSC India 2018 about Deep Learning with Apache Spark

An extensive non-reactive Typescript framework that eases the development experience in Spark AR

A Linux based software package to control led's on Logitech G910, G810, G610 and G410.

Code examples for my blog posts

Jupyter magics and kernels for working with remote Spark clusters

Implementation of Fellegi-Sunter's canonical model of record linkage in Apache Spark, including EM algorithm to estimate parameters

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

MapReduce, Spark, Java, and Scala for Data Algorithms Book

State of the Art Natural Language Processing

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

Apache Spark Connector for Azure Cosmos DB

HandySpark - bringing pandas-like capabilities to Spark dataframes

汇总java生态圈常用技术框架、开源中间件，系统架构、数据库、大公司架构案例、常用三方类库、项目管理、线上问题排查、个人成长、思考等知识

Boiler plate framework to use Spark and ZIO together.

Optimize and improve the Label propagation algorithm

Tools for reading data from Solr as a Spark RDD and indexing objects from Spark into Solr using SolrJ.