549 Open source projects that are alternatives of or similar to Spark

Collection of Databricks and Jupyter Notebooks

The Apache Kafka C/C++ library

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

The Apache Kafka Client SDK

Albis: High-Performance File Format for Big Data Systems

Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.

Another app which used to monitor the progress of Kafka Producer and Consumer

Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.

A configurable RabbitMQ consumer made in Rust, useful for a stable and reliable CLI commands processor.

A project with examples of using few commonly used data manipulation/processing/transformation APIs in Apache Spark 2.0.0

A real time analytics dashboard to analyze the trending hashtags and @ mentions at any location using kafka and spark streaming.

This repository houses the interactions, consultations and work management to support the maintenance of baselined components of the Consumer Data Right API Standards and Information Security profile.

A complete queue consumer for SQS

A collection of Apache Parquet add-on modules

MJPEG webcam network streamer for linux

A command line tool to query an ODBC data source and write the result into a parquet file.

Services in Ember are scoped to the app as a whole and are singletons. Sometimes you don't want that. :) This addon provides ephemeral route-based services.

How to use Parquet in Flink

CSVs sliced, diced & analyzed.

A Flutter app with live cryptocurrency updates, powered by Ably

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Library to read a subset of Parquet files

youtube download queue websocket server with login for private NAS.

BunnyMQ is an amqp.node wrapper to ease common AMQP usages (RPC, pub/sub, channel/connection handling etc.).

Rust Kafka HTTP proxy

A collection of tools for extracting FHIR resources and analytics services on top of that data.

An encrypted data analytics platform

🔆 spark自学手册，包含了例如spark core、spark sql、spark streaming、spark-kafka、delta-lake，以及scala基础练习，还有一些例如master、shuffle源码分析，总结及翻译。

Hyperconverged cloud-edge native database

The magic message bus

SQL Parsers for BigData, built with antlr4.

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Consume an AWS Kinesis Data Stream to look over the records from a terminal.

A simple app that lets you stream every live and archived NHL and MLB game from any of your iOS devices.

bring sf to spark in production

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

Chrome extension which lets you watch WWDC Developer Videos in Google Chrome (thus not having to use Safari)

RabbitMQ Messaging Polyglot with Java, ColdFusion, CommandBox, Groovy and more

Enables extraction of measurement data from binary files with extension 'raw' used by proprietary software imcFAMOS/imcSTUDIO and facilitates its storage in open source file formats

dvbtee: a digital television streamer / parser / service information aggregator supporting various interfaces including telnet CLI & http control

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi

rejected is a consumer framework for RabbitMQ

Platform for Situated Intelligence

Consumer Banking Application

PHP Kafka producer / consumer library with PHP Avro support, based on php-rdkafka

Implementing a Web Based solution through which farmers can participate in a commodity exchange market

A collection of examples of continuous analytics.

🌀 .NET Event Processing library

Spark VCF data source implementation for Dataframes

bigdatatutorial

Spark Streaming ETL jobs for Mozilla Telemetry

Apache Pulsar Go Client Library

Pulsar Flex is a modern Apache Pulsar client for Node.js, developed to be independent of C++.

Simple self-hosted music streaming server

betfairlightweight - python wrapper for Betfair API-NG (with streaming)

Make record oriented data to columnar format.

Apache Pulsar Manager

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

This code creates a Kinesis Firehose in AWS to send CloudWatch log data to S3.