The micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API.

✭ 462

javascript parser stream-processing

Ksql

The database purpose-built for stream processing applications.

✭ 4,668

java ANTLR python shell HTML javascript sql kafka real-time interactive stream-processing kafka-connect materialized-views ksqldb ksqldb-documentation ksqldb-tutorials streaming-queries event-streaming-database

Kasper

Kasper is a lightweight library for processing Kafka topics.

✭ 413

go kafka stream-processing

Awesome Kafka

A list about Apache Kafka

✭ 397

kafka infrastructure apache-spark stream-processing apache-kafka kafka-streams data-processing streaming-data

Awesome System Design

A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

✭ 4,999

microservices distributed-systems interview nosql stream-processing microservices-architecture relational-database message-broker hadoop-ecosystem

Json Machine

Efficient, easy-to-use, and fast PHP JSON stream parser

✭ 376

parsing stream-processing json-parser

Bistro

A general-purpose data analysis engine radically changing the way batch and stream data is processed

✭ 333

iot analytics stream-processing

Yomo

🦖 Streaming-Serverless Framework for Low-latency Edge Computing applications, running atop QUIC protocol, engaging 5G technology.

✭ 279

go serverless iot networking realtime stream-processing low-latency functional-reactive-programming quic edge-computing

Smooks

An extensible Java framework for building XML and non-XML streaming applications

✭ 293

java xml big-data etl event-driven stream-processing pipelines

Benthos

Fancy stream processing made operationally mundane

✭ 3,705

go javascript CSS HTML shell Makefile kafka rabbitmq cqrs event-sourcing etl stream-processing amqp message-queue logs data-engineering nats message-bus streaming-data stream-processor data-ops

Hazelcast

Open-source distributed computation and storage platform

✭ 4,662

java FreeMarker shell HCL python c hacktoberfest big-data distributed caching scalability in-memory distributed-systems hazelcast distributed-computing stream-processing low-latency

kerala

Distributed KV Streams

✭ 16

kotlin java Dockerfile distributed-systems raft streams stream-processing

JPStream

JPStream: JSONPath Stream Processing in Parallel

✭ 19

c C++Yacc Lex CMake json stream-processing json-parser jsonpath path-query jsonpath-query

Akka-Streams-custom-stream-processing-examples

Demos of how to do custom stream processing using the Akka Streams GraphStages API

✭ 13

scala groovy akka stream-processing akka-streams

transform-hub

Flexible and efficient data processing engine and an evolution of the popular Scramjet Framework based on node.js. Our Transform Hub was designed specifically for data processing and has its own unique algorithms included.

✭ 38

typescript python javascript Gherkin nodejs streaming real-time application-server data-acquisition stream-processing transform-streams live-data transform-as-a-service

theodolite

Theodolite is a framework for benchmarking the horizontal and vertical scalability of cloud-native applications.

✭ 20

java kotlin Jupyter Notebook python shell Dockerfile Smarty benchmarking scalability stream-processing event-driven-microservices cloud-native

data processing course

Some class materials for a data processing course using PySpark

✭ 50

python ruby shell Makefile Dockerfile HTML Jupyter Notebook course spark bigdata stream-processing pyspark apache-beam data-processing

dspatch

The Refreshingly Simple Cross-Platform C++ Dataflow / Pipelining / Stream Processing / Reactive Programming Framework

✭ 124

C++data-driven pipelines reactive-programming stream-processing dataflow flow-based-programming dataflow-programming pipelining

ottla

An opinionated clojure framework for writing kafka machines

✭ 14

clojure kafka stream-processing

traffic

Massively real-time traffic streaming application

swirl

High-Performance Erlang Stream Processor

✭ 52

erlang Makefile high-performance stream-processing

streamdb-readings

Readings in Stream Processing

✭ 62

database research stream-processing papers

stream-registry

Stream Discovery and Stream Orchestration

✭ 105

java streams stream-processing metadata-management

EsperIoT

Small and simple stream-based CEP tool for IoT devices connected to an MQTT broker

✭ 18

java Dockerfile mqtt iot spring cep internet-of-things stream-processing event-driven complex-event-processing esper

storm-ml

an online learning algorithm library for Storm

✭ 18

java shell big-data storm stream-processing

mage

MAGE - Memgraph Advanced Graph Extensions 🔮

✭ 89

C++python rust c Cuda CMake open-source graph-algorithms stream-processing graph-database cypher real-time-analytics memgraph

artml

ARTML- Real time learning

✭ 20

Jupyter Notebook python data-science machine-learning real-time incremental stream-processing adaptive-learning

godsend

A simple and eloquent workflow for streaming messages to micro-services.

✭ 15

javascript microservices stream microservice messaging bus filter message-bus streams stream-processing broker streaming-api microservices-architecture message-passing streaming-data message-broker

talaria

TalariaDB is a distributed, highly available, and low latency time-series database for Presto

✭ 148

go python Thrift real-time database big-data time-series stream-processing column-store prestodb

ripple

Simple shared surface streaming application

awesome-bigdata

A curated list of awesome big data frameworks, ressources and other awesomeness.

✭ 11,093

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

gostream

Stream Processing Library for Go

✭ 51

go Makefile stream-processing complex-event-processing

flink-connectors

Apache Flink connectors for Pravega.

✭ 84

java python stream-processing flink pravega

kafka-shell

⚡A supercharged, interactive Kafka shell built on top of the existing Kafka CLI tools.

✭ 107

python shell cli terminal kafka apache prompt streams stream-processing confluent kafka-cli kafka-tools

ramen

A stream processing language and compiler for small-scale monitoring

✭ 14

ocaml PHP c Makefile Gherkin shell stream-processing network-monitoring monitoring-systems

kafka-workers

Kafka Workers is a client library which unifies records consuming from Kafka and processing them by user-defined WorkerTasks.

✭ 30

java kafka parallel multithreading stream-processing kafka-consumer partitioning backpressure kafka-workers parallel-consumer

csvplus

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

✭ 67

go Makefile csv etl stream-processing fluent-interface csv-format go-csv etl-framework etl-pipeline

product-sp

An open source, cloud-native streaming data integration and analytics product optimized for agile digital businesses

✭ 80

python java shell HTML Batchfile CSS analytics stream-processing complex-event-processing real-time-processing streaming-data-integration

go-rivers

Collection of stream processing / multiplexing / networking libs in Go

✭ 35

go ruby Makefile streaming stream-processing link-collection

blockchain-etl-streaming

Streaming Ethereum and Bitcoin blockchain data to Google Pub/Sub or Postgres in Kubernetes

gretel-python-client

The Gretel Python Client allows you to interact with the Gretel REST API.

✭ 28

python machine-learning privacy stream-processing datascience privacy-enhancing-technologies synthetic-data

mediapipe plus

The purpose of this project is to apply mediapipe to more AI chips.

makinage

Stream Processing Made Easy

✭ 31

python distributed-systems machine-learning streaming kafka reactive-programming stream-processing streaming-data reactive-systems reactive-machine-learning

mxfactorial

a payment application intended for deployment by the united states treasury

Perfect-Kafka

An Express Swift Client of Apache Kafka 0.8, the Stream Processing Platform

✭ 20

swift shell kafka stream-processing message-queue

open-stream-processing-benchmark

This repository contains the code base for the Open Stream Processing Benchmark.

✭ 37

Jupyter Notebook scala shell Dockerfile distributed-systems benchmark real-time kafka spark distributed-computing stream-processing spark-streaming flink kafka-streams benchmark-suite real-time-processing structured-streaming

kafka-spark-streaming-example

Simple examle for Spark Streaming over Kafka topic

✭ 102

java HTML scala kafka spark stream-processing

distogram

A library to compute histograms on distributed environments, on streaming data

✭ 19

python streaming statistics histogram stream-processing quantile-estimation data-summary percentile

vector

A high-performance observability data pipeline.

✭ 12,138

rust CUE HTML typescript javascript shell parser events router pipeline metrics vector logs stream-processing forwarder observability

dagger

Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data.

✭ 238

java framework influxdb prometheus stream-processing dataops apache-flink apache-kafka real-time-analytics real-time-processing

openPDC

Open Source Phasor Data Concentrator

streamsx.kafka

Repository for integration with Apache Kafka

✭ 13

java shell kafka apache-spark toolkit messaging stream-processing ibm-streams

beepbeep-3

An event stream processor anyone can use

✭ 20

java log-analysis event-stream cep stream-processing beepbeep stream-processing-engine

football-events

Event-Driven microservices with Kafka Streams

✭ 57

java HTML Dockerfile demo microservices kafka cqrs example event-sourcing stream-processing event-driven-microservices event-driven kafka-connect microservices-architecture kafka-streams

61-120 of 126 stream-processing projects

‹

›