Top 766 kafka open source projects

Curated by Lenses.io

A daily digest of the articles or videos I've found interesting, that I want to share with you.

spring cloud 版分布式电商项目，全力打造顶级多模块，高可用，高扩展电商项目

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

DeviceHive Java Server

Open-Source Web GUI for Apache Kafka Management

Building event-driven applications the easy way in Go.

docker-compose.yml files for cp-all-in-one , cp-all-in-one-community, cp-all-in-one-cloud

Kafka-based message queue

Yivnet is a microservice game server base on go-kit

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Streaming reactive and dataflow graphs in Python

550+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Kafka, Docker, APIs, Hadoop, SQL, PostgreSQL, MySQL, Hive, Impala, Travis CI, Jenkins, Concourse, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, .tmux.conf, .psqlrc ...

syncClient，数据实时同步中间件（同步mysql到kafka、redis、elasticsearch、httpmq）！

Learn Clean Architecture with .NET Core 3.0 🔥

Materialize lets you ask questions of your live data, which it answers and then maintains for you as your data continue to change. The moment you need a refreshed answer, you can get it in milliseconds. Materialize is designed to help you interactively explore your streaming data, perform data warehousing analytics against live relational data, or just increase the freshness and reduce the load of your dashboard and monitoring tasks.

DDD. Event sourcing. CQRS. REST. Modular. Microservices. Kotlin. Spring. Axon platform. Apache Kafka. RabbitMQ

An actor framework for Go

Deep Learning UDF for KSQL for Streaming Anomaly Detection of MQTT IoT Sensor Data

以教学为目的的电商系统。包含ToB复杂业务、互联网高并发业务、缓存应用；DDD、微服务指导。模型驱动、数据驱动。了解大型服务进化路线，编码技巧、学习Linux，性能调优。Docker/k8s助力、监控、日志收集、中间件学习。前端技术、后端实践等。主要技术：SpringBoot+JPA+Mybatis-plus+Antd+Vue3。

Big Data Processing Framework - Unified Data API or SQL on Any Storage

The Kafka CLI for humans

Health Check for Kafka Brokers.

Asky开源架构：极简、轻量、极致性能《Asky零基础1小时学编程 dnc+vue+tidb+redis+rabbitMQ+ES》QQ群 779699538

Go driver for Kafka

基于Spring Cloud（Greenwich.SR2）搭建的微服务脚手架（适用于在线系统），已集成注册中心（Nacos Config）、配置中心（Nacos Discovery）、认证授权（Oauth 2 + JWT）、日志处理（ELK + Kafka）、限流熔断（AliBaba Sentinel）、应用指标监控（Prometheus + Grafana）、调用链监控（Pinpoint）、以及Spring Boot Admin。

Go client library for Apache Kafka

Sample Application DDD, Reactive Microservices, CQRS Event Sourcing Powered by DERMAYON LIBRARY

⚡️ Nepxion Thunder is a distribution RPC framework based on Netty + Hessian + Kafka + ActiveMQ + Tibco + Zookeeper + Redis + Spring Web MVC + Spring Boot + Docker 多协议、多组件、多序列化的分布式RPC调用框架

我的个人技术博客（Python、Django、Docker、Go、Redis、ElasticSearch、Kafka、Linux）

Real Time Big Data / IoT Machine Learning (Model Training and Inference) with HiveMQ (MQTT), TensorFlow IO and Apache Kafka - no additional data store like S3, HDFS or Spark required

QMQ是去哪儿网内部广泛使用的消息中间件，自2012年诞生以来在去哪儿网所有业务场景中广泛的应用，包括跟交易息息相关的订单场景； 也包括报价搜索等高吞吐量场景。

💪 example of common used libraries and frameworks, programming required, don't fork man.

Sync data from the other DB to ClickHouse(cluster)

Apache Kafka running on Kubernetes

♒︎ [WIP] An experimental ~distributed~ commit-log

franz-go contains a high performance, pure Go library for interacting with Kafka from 0.8.0 through 2.7.0+. Producing, consuming, transacting, administrating, etc.

推荐项目（实时推荐和离线推荐）

amazonriver 是一个将postgresql的实时数据同步到es或kafka的服务

Serverless Platform for the stateful services

istio 微服务示例代码 grpc+protobuf+echo+websocket+mysql+redis+kafka+docker-compose

Reactive (RSocket/gRPC) Gateway for the event-based systems

C# 开发辅助类库，和士官长一样身经百战且越战越勇的战争机器，能力无人能出其右。

Thin Scala wrapper around Kafka Streams Java API

Kafka Web UI

Proxy connections to Kafka cluster. Connect through SOCKS Proxy, HTTP Proxy or to cluster running in Kubernetes.

后端面试题汇总（Python、Redis、MySQL、PostgreSQL、Kafka、数据结构、算法、编程、网络）

High throughput asynchronous task processing on Apache Kafka

mq transaction, tcc, eventually consistency. tx life cycle: all listeners handled, if necessary, produce next message

Schema Registry

A simple way to publish messages to Kafka from Ruby applications

Scala DSL for Unit-Testing Processing Topologies in Kafka Streams

OpenMessaging Benchmark Framework

Kafka consumer lag-checking application for monitoring, written in Scala and Akka HTTP; a wrap around the Kafka consumer group command. Integrations with Cloudwatch and Datadog. Authentication recently added

Apache Kafka load testing "...basically a cloth bag filled with small jagged pieces of scrap iron"

Self-contained examples of Apache Spark streaming integrated with Apache Kafka.

基于 Springboot + Redis + Kafka 的秒杀系统，乐观锁 + 缓存 + 限流 + 异步，TPS 从 500 优化到 3000

💥 🚀 封装sparkstreaming动态调节batch time(有数据就执行计算)；🚀 支持运行过程中增删topic；🚀 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。

Command Line Tool for managing Apache Kafka