Top 766 kafka open source projects

Streaming data changes to a Data Lake with Debezium and Delta Lake pipeline

Kafka Akka.Streams connectors - part of the Alpakka project

SparkStreaming中利用MySQL保存Kafka偏移量保证0数据丢失

💥 A simple multi-purpose connection pool client (Kafka & Hbase & Redis & RMDB & Socket & Http)

A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.

450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...

A library provides consumer/producer to work with kafka, avro and schema registry

Kafka exporter for Prometheus

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

GraphQL Rust Demo

Another app which used to monitor the progress of Kafka Producer and Consumer

Tool to ease and automate Apache Kafka cluster configuration management

基于utgard的OPC DA客户端

[DEPRECATED] Docker images for Confluent Platform.

Cluster extensions for Sarama, the Go client library for Apache Kafka 0.9 [DEPRECATED]

Files needed to try out KSQL Recipes for yourself

De-/Serialization wrapper for Kafka to accomplish end to end encryption

This toolkit is focused on interacting with popular messaging systems such as Kafka, JMS, XMS, and MQTT. After release v5.4.2 the complete toolkit will be deprecated. See the README.md file for hints to alternative toolkits.

A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud

weiboAd Infrastructure Data Processor : kafka consumer embedded Lua scripting language in data process framework

Sarama is a Go library for Apache Kafka 0.8, and up.

MySQL to NoSQL real time dataflow

CLI tools for monitoring and managing Apache Kafka

必看！java后端，亮剑诛仙。java发展路线技术要点。

Simple learning project pushing CSV data into Kafka then indexing the data in ElasticSearch

Apache Kafka support for Ruby Object Mapper

🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结

Distributed Stream and Batch Processing

Kubernetes Universal Declarative Operator (KUDO)

50+ DockerHub public images for Docker & Kubernetes - Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak, TeamCity and DevOps tools built on the major Linux distros: Alpine, CentOS, Debian, Fedora, Ubuntu

Kafka connect Elastic sink connector, with just in time index/delete behaviour.

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

Kafka Connect Elasticsearch Source

A resilient and scalable shopping cart system designed using Event Sourcing (ES) and Command Query Responsibility Segregation (CQRS)

Read Kafka topics with WebSockets

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

Kafka Development Tool

KafkaCenter is a unified platform for Kafka cluster management and maintenance, producer / consumer monitoring, and use of ecological components.

Very simplified shop sales system made in a microservices architecture using quarkus

深圳地铁大数据客流分析系统🚇🚄🌟

大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/

Kafka command line tool that likes JSON

Spring Boot 学习教程

SiteWhere is an industrial strength open-source application enablement platform for the Internet of Things (IoT). It provides a multi-tenant microservice-based infrastructure that includes device/asset management, data ingestion, big-data storage, and integration through a modern, scalable architecture. SiteWhere provides REST APIs for all system functionality. SiteWhere provides SDKs for many common device platforms including Android, iOS, Arduino, and any Java-capable platform such as Raspberry Pi rapidly accelerating the speed of innovation.

🐂基于springcloud +dubbo构建的模拟秒杀项目，模块化设计，集成了分库分表、elasticsearch🔍、gateway、mybatis-plus、spring-session等常用开源组件

Enterprise Network Flow Collector (IPFIX, sFlow, Netflow) from Verizon Media

Streaming reference architecture for ETL with Kafka and Kafka-Connect. You can find more on http://lenses.io on how we provide a unified solution to manage your connectors, most advanced SQL engine for Kafka and Kafka Streams, cluster monitoring and alerting, and more.

Open source Observability Platform. 👉 SigNoz helps developers find issues in their deployed applications & solve them quickly

A distributed event bus that implements a RESTful API abstraction on top of Kafka-like queues

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

High performance mqtt broker

Kafka Connect connector for JDBC-compatible databases

a fast and simple distributed publish-subscribe messaging system (mq)

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].

Free and open source log management

An extensible distributed system for reliable nearline data streaming at scale

Fast and reliable message broker built on top of Kafka.

This project contains examples which demonstrate how to deploy analytic models to mission-critical, scalable production environments leveraging Apache Kafka and its Streams API. Models are built with Python, H2O, TensorFlow, Keras, DeepLearning4 and other technologies.

Asp.Net Core 5 Clean Architecture (Microservices, Modular Monolith, Monolith) samples (+Blazor, Angular 11, React 17, Vue 2.6), Domain-Driven Design, CQRS, Event Sourcing, SOLID, Asp.Net Core Identity Custom Storage, Identity Server 4 Admin UI, Entity Framework Core, Selenium E2E Testing, SignalR Notification, Hangfire Tasks Scheduling, Health Checks, Security Headers, ...