793 Open source projects that are alternatives of or similar to Spark Streaming With Kafka

Java SDK for CloudEvents

MongoDB Kafka Connector

Examples for Deep Learning/Feature Store/Spark/Flink/Hive/Kafka jobs and Jupyter notebooks on Hops

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

以Kafka为存储介质，提供异步Http RPC的中间件

商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark

Apache Spark - A unified analytics engine for large-scale data processing

syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL.

weiboAd Infrastructure Data Processor : kafka consumer embedded Lua scripting language in data process framework

A Clojure dataframe library that runs on Spark

A Time Series Library for Apache Spark

An opinionated Elixir wrapper around brod, the Erlang Kafka client, that supports encrypted connections to Heroku Kafka out of the box.

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

A Spark/Scala implementation of the isolation forest unsupervised outlier detection algorithm.

MySQL to NoSQL real time dataflow

This is a Platform that collects all the data accuring in your Application and shows the data in real time by using Kibana or other tools.

Feature engineering toolkit for Spark MLlib.

Kafka 한국 사용자 모임에서 운영하는 meetup repository

Simple learning project pushing CSV data into Kafka then indexing the data in ElasticSearch

Apache Kafka bridge

MaRe leverages the power of Docker and Spark to run and scale your serial tools in MapReduce fashion.

A CLI and Go client for Kafka Schema Registry

LearningApacheSpark

Distributed Stream and Batch Processing

Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange

关于kafka的一些相关使用示例代码。

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

Spring Boot 2.x code examples

Feature Store for Machine Learning

.NET event stream projection and scheduling platform with EventStore, CosmosDb, Equinox and Kafka integrations

Scala toolchain for InfluxDB

Isolation Forest on Spark

Kafka connect Elastic sink connector, with just in time index/delete behaviour.

Java生态研究(Spring Boot + Redis + Dubbo + RocketMQ + Elasticsearch)🔥🔥🔥🔥🔥

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

WaterDrop is a standalone Karafka component library for generating Kafka messages

Kafka Connect Elasticsearch Source

a Rocket-fast SYStem for LOG processing

A resilient and scalable shopping cart system designed using Event Sourcing (ES) and Command Query Responsibility Segregation (CQRS)

Monitor Apache Spark from Jupyter Notebook

Spark Terasort

KafkaCenter is a unified platform for Kafka cluster management and maintenance, producer / consumer monitoring, and use of ecological components.

Secor is a service implementing Kafka log persistence

Parquet file generator

OpenTracing Instrumentation for Apache Kafka Client

Collection of Kubernetes Operators built with KUDO.

Distributed t-SNE via Apache Spark

🌿 基于springboot的快速学习示例,整合自己遇到的开源框架,如：rabbitmq(延迟队列)、Kafka、jpa、redies、oauth2、swagger、jsp、docker、spring-batch、异常处理、日志输出、多模块开发、多环境打包、缓存cache、爬虫、jwt、GraphQL、dubbo、zookeeper和Async等等📌

Karapace - Your Kafka essentials in one tool

👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/

Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go.

A bank simulation application using mainly Clojure, which can be used to end-to-end test and show some graphs.

A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with luv by Data Scientists & Security Analysts from the Cyber Security Industry. [PRE-ALPHA]

Cookbook to install Hadoop 2.0+ using Chef

Spark job for dependency links

大数据分析可视化实践

Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more...

The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.

Mirror of Linkedin's Camus

MLeap: Deploy ML Pipelines to Production