180 Open source projects that are alternatives of or similar to spark-utils

C# and F# language binding and extensions to Apache Spark

酷玩 Spark: Spark 源代码解析、Spark 类库等

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Custom state store providers for Apache Spark

Streaming System 相关的论文读物

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

Plot live-stats as graph from ApacheSpark application using Lightning-viz

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Companion to Learning Hadoop and Learning Spark courses on Linked In Learning

A converter for the OSM PBFs to Parquet files

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

JupyterLab extension that enables monitoring launched Apache Spark jobs from within a notebook

curated list of awesome tools and libraries for specific domains

Bitnami Docker Image for Apache Spark

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

Real-world Spark pipelines examples

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Methods for the parallel and distributed analysis and mining of the Protein Data Bank using MMTF and Apache Spark.

Astronomy Broker based on Apache Spark

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

大数据生态圈学习

Routines and data structures for using isarn-sketches idiomatically in Apache Spark

A general purpose metrics monitor implemented with Apache Spark. Kafka source, Elastic sink, aggregate metrics, different analysis, notifications, actions, live configuration update, missing metrics, ...

Big Data RDF Processing and Analytics Stack built on Apache Spark and Apache Jena http://sansa-stack.github.io/SANSA-Stack/

Overlapping community detection in Large-Scale Networks using BigCLAM model build on Apache Spark

Old repo for R interface for GraphFrames

Extensible streaming ingestion pipeline on top of Apache Spark

The Proxima platform.

Spark in Action, 2nd edition - chapter 7 - Ingestion from files

基于spark-ml,spark-mllib,spark-streaming的推荐算法实现

An Apache Spark standalone application using the Spark API in Scala. The application uses Simple Build Tool(SBT) for building the project.

Repository for integration with Apache Kafka

Powerful rapid automatic EDA and feature engineering library with a very easy to use API 🌟

Big Data Management and Analysis Final Project

Emacs mode that moves to modern keybindings

🐧基于spark streaming+flume+kafka+hbase的实时日志处理分析系统(分为控制台版本和基于springboot、Echarts等的Web UI可视化版本)

Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.

An umbrella project for multiple implementations of model serving

🐬 Apache Parquet for modern .Net

Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteorology, …

Plasmoid for KDE Plasma to control docker containers

Spark cloud integration: tests, cloud committers and more

A connector for Apache Spark to access Exasol

a one-stop repo to lookup for code snippets of core java concepts, sql, data structures as well as big data. It also consists of interview questions asked in real-life.

No description or website provided.

seatunnel plugin developing examples.

bigdatatutorial

The Internals of Spark SQL

👨‍💻Zaak's 🧩(missing) 🏛Standard 🔬Fortran 📚Library 🚧(WIP)

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

This repository contains the code base for the Open Stream Processing Benchmark.

A pure Python implementation of Apache Spark's RDD and DStream interfaces.

Infrastructures™ for Machine Learning Training/Inference in Production.

Real Time Twitter Sentiment Analysis Product

Apache Spark™ and Scala Workshops

pyspark methods to enhance developer productivity 📣 👯 🎉

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !