420 Open source projects that are alternatives of or similar to Apache Spark Node

A Spark plugin for reading Excel files via Apache POI

Data validation and organization of metadata for data frames and database tables

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.

java 进阶之路：面试高频算法、akka、多线程、NIO、Netty、SpringBoot、Spark&&Flink 等

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

This package contains a generic implementation of greedy Information Theoretic Feature Selection (FS) methods. The implementation is based on the common theoretic framework presented by Gavin Brown. Implementations of mRMR, InfoGain, JMI and other commonly used FS filters are provided.

🌿 基于springboot的快速学习示例,整合自己遇到的开源框架,如：rabbitmq(延迟队列)、Kafka、jpa、redies、oauth2、swagger、jsp、docker、spring-batch、异常处理、日志输出、多模块开发、多环境打包、缓存cache、爬虫、jwt、GraphQL、dubbo、zookeeper和Async等等📌

🐍 Quick reference guide to common patterns & functions in PySpark.

pyspark🍒🥭 is delicious，just eat it!😋😋

A Linux based software package to control led's on Logitech G910, G810, G610 and G410.

Spark Terasort

extremely distributed machine learning

There are pieces of scala code that explain Scala syntax and related things - like what you can do with all this

Applying Lambda Architecture with Spark, Kafka, and Cassandra.

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

Spark SQL index for Parquet tables

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

An encrypted data analytics platform

Source code for James Lee's Aparch Spark with Java course

Spark Streaming监控平台，支持任务部署与告警、自启动

The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.

Elassandra = Elasticsearch + Apache Cassandra

A Scala kernel for Jupyter

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

🌟 ✨ Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Data science and Big Data with Python

A large-scale entity and relation database supporting aggregation of properties

Distributed Deep learning with Keras & Spark

Python based framework to retreive Global Database of Events, Language, and Tone (GDELT) version 1.0 and version 2.0 data.

Production Ready Data Integration Product, documentation：

Avro SerDe for Apache Spark structured APIs.

Two-day workshop that covers how to use R to interact databases and Spark

Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive

A distributed data processing framework in Haskell.

Feature Store for Machine Learning

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

Boiler plate framework to use Spark and ZIO together.

Machine Learning Platform and Recommendation Engine built on Kubernetes

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Apache Spark and Apache Kafka integration example

Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange

A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with luv by Data Scientists & Security Analysts from the Cyber Security Industry. [PRE-ALPHA]

FFM (Field-Awared Factorization Machine) on Spark

Kinesis Connector for Structured Streaming

大数据入门指南 ⭐

Jupyter kernel for scala and spark

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

A pandas-like deferred expression system, with first-class SQL support

Collaborate on privacy-preserving policy for data science projects in Pandas and Apache Spark

Relation Extraction using Deep learning(CNN)

📊 Cube — Open-Source Analytics API for Building Data Apps

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。

Spark RDD with Lucene's query and entity linkage capabilities

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.