399 Open source projects that are alternatives of or similar to Spark

A Spark/Scala implementation of the isolation forest unsupervised outlier detection algorithm.

An encrypted data analytics platform

spark ml 算法原理剖析以及具体的源码实现分析

A repository to keep track of all the code that I end up writing for my blog posts.

The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.

Compile-time Language Integrated Queries for Scala

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

Process Common Crawl data with Python and Spark

A Spark SQL extension which provides SQL Standard Authorization for Apache Spark

There are pieces of scala code that explain Scala syntax and related things - like what you can do with all this

LearningApacheSpark

商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark

Profile and monitor your ML data pipeline end-to-end

Apache Spark on AWS Lambda

Live-coding the cluster!

Isolation Forest on Spark

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

AZTK powered by Azure Batch: On-demand, Dockerized, Spark Jobs on Azure

🌿 基于springboot的快速学习示例,整合自己遇到的开源框架,如：rabbitmq(延迟队列)、Kafka、jpa、redies、oauth2、swagger、jsp、docker、spring-batch、异常处理、日志输出、多模块开发、多环境打包、缓存cache、爬虫、jwt、GraphQL、dubbo、zookeeper和Async等等📌

An open-source toolkit for large-scale genomic analysis

Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive

Pandas and Spark DataFrame comparison for humans

Fast, Scientific and Numerical Computing for the JVM (NDArrays)

This package contains a generic implementation of greedy Information Theoretic Feature Selection (FS) methods. The implementation is based on the common theoretic framework presented by Gavin Brown. Implementations of mRMR, InfoGain, JMI and other commonly used FS filters are provided.

A Clojure dataframe library that runs on Spark

Geospatial Raster support for Spark DataFrames

Spark、Flink等离线任务的调度以及实时任务的监控

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Monitor Apache Spark from Jupyter Notebook

SparklingGraph provides easy to use set of features that will give you ability to proces large scala graphs using Spark and GraphX.

Spark Structured Streaming / Kafka / Cassandra / Elastic

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

Julia binding for Apache Spark

Node.js bindings for Apache Spark DataFrame APIs

Big Data Ecosystem Docker

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

Distributed t-SNE via Apache Spark

Avro SerDe for Apache Spark structured APIs.

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learni…

Jupyter kernel for scala and spark

A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).

A large-scale entity and relation database supporting aggregation of properties

Simple and beautiful card component with an elegant spark line, for VueJS.

Feature Store for Machine Learning

Fundamentals of Spark with Python (using PySpark), code examples

A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with luv by Data Scientists & Security Analysts from the Cyber Security Industry. [PRE-ALPHA]

GeoTrellis for PySpark

Collaborate on privacy-preserving policy for data science projects in Pandas and Apache Spark

Updated repository

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Content for architecting a data science platform for products using Luigi, Spark & Flask.

DataStax Spark Cassandra Connector

State of the Art Natural Language Processing

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

Apache Spark Connector for Azure Cosmos DB

HandySpark - bringing pandas-like capabilities to Spark dataframes

汇总java生态圈常用技术框架、开源中间件，系统架构、数据库、大公司架构案例、常用三方类库、项目管理、线上问题排查、个人成长、思考等知识