415 Open source projects that are alternatives of or similar to Spark-PMoF

blog entries

An extensive non-reactive Typescript framework that eases the development experience in Spark AR

Alternate firmware for Mutable Instruments Branches synthesizer module

A daily digest of the articles or videos I've found interesting, that I want to share with you.

Celeborn provides an elastic and high-performance service for shuffle and spilled data.

❄️ 🐳 Awesome tools and libs for AI, Deep Learning, Machine Learning, Computer Vision, Data Science, Data Analytics and Cognitive Computing that are baked in the oven to be Native on Kubernetes and Docker with Python, R, Scala, Java, C#, Go, Julia, C++ etc

Validator for RDMA Configuration and Best Practices

Autoscaling for Google Cloud Dataproc

Implementation of Fellegi-Sunter's canonical model of record linkage in Apache Spark, including EM algorithm to estimate parameters

Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs

基于Docker构建的Hadoop开发测试环境，包含Hadoop，Hive，HBase，Spark

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

Some class materials for a data processing course using PySpark

Kubernetes Rdma SRIOV device plugin

A parallel implementation of word2vec based on Spark

ksmbd kernel server(SMB/CIFS server)

My presentation at ODSC India 2018 about Deep Learning with Apache Spark

Koalas: pandas API on Apache Spark

Apache Spark Sentry Integration

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

Code examples for my blog posts

Book recommender system using collaborative filtering based on Spark

Code Library for My Blog

A Jupyter notebook using some standard techniques for data science and data engineering to analyze data for the 2017 flooding in Houston, TX.

The Internals of Spark SQL

Applying Data Science and Machine Learning to Solve Real World Business Problems

DevOps tools for Microsoft Dynamics 365

Identifies and collects data on cc-licensed content across web crawl data and public apis.

Big Compute Learning Labs

Visualize column-level data lineage in Spark SQL

DaRPC: Data Center Remote Procedure Call

ksmbd kernel server(SMB/CIFS server)

Passive Disaggregated Persistent Memory at USENIX ATC 2020.

Apache Spark docker container image (Standalone mode)

Automatic library-wide shuffle for mpd.

Sparkline BI Accelerator provides fast ad-hoc query capability over Logical Cubes. This has been folded into our SNAP Platform(http://bit.ly/2oBJSpP) an Integrated BI platform on Apache Spark.

Framework providing operating system abstractions and a range of shared networking (RDMA, TCP/IP) and memory services to common modern heterogeneous platforms.

spark on kubernetes

Transpose: SIMD Integer+Floating Point Compression Filter

Open-source distribute workflow schedule tools, also support streaming task.

REST job server for Apache Spark

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)

Resources for Spark AR

An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.

Online search advertisement platform & Realtime Campaign Monitoring [Maybe Deprecated]

Python clone of Spark, a MapReduce alike framework in Python

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

人生苦短 我用Python

☁️ Cloud-scale telemetry ingestion from any stream of data with Azure Event Hubs

Spark functions to run popular phonetic and string matching algorithms

Installations for Data Science. Anaconda, RStudio, Spark, TensorFlow, AWS (Amazon Web Services).

Apache Spark in your IDE with gradle

▁▂▄▆▇█▇▆▄▂▁

RLib is a header-only library for easier usage of RDMA.

基于Spark2.2新闻网大数据实时系统项目

🎼 Docker compose files for various kafka stacks

ACID Data Source for Apache Spark based on Hive ACID

low-level helpers for Apache Spark libraries and tests