761 Open source projects that are alternatives of or similar to Spark Kafka Writer

Use Kafka as a remote storage database for Prometheus (remote write only)

Everything about Apache Kafka

Confluent's Apache Kafka .NET client

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

State of the Art Natural Language Processing

Fast, Scientific and Numerical Computing for the JVM (NDArrays)

A Clojure dataframe library that runs on Spark

Geospatial Raster support for Spark DataFrames

Apache Spark Connector for Azure Cosmos DB

LearningApacheSpark

WaterDrop is a standalone Karafka component library for generating Kafka messages

Big Data Toolkit for the JVM

Firely's open source FHIR server

Playground for Kafka/Confluent Docker experimentations

Monitor Apache Spark from Jupyter Notebook

SparklingGraph provides easy to use set of features that will give you ability to proces large scala graphs using Spark and GraphX.

大数据分析可视化实践

Syslog Collector written in Go, streams to Kafka 0.8

Redpanda is the real-time engine for modern apps. Kafka API Compatible; 10x faster 🚀 See more at vectorized.io/redpanda

**Unofficial / Community** Kafka Connect MongoDB Sink Connector - Find the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learni…

Apache Spark on AWS Lambda

Julia binding for Apache Spark

Profile and monitor your ML data pipeline end-to-end

Node.js bindings for Apache Spark DataFrame APIs

Kafka Connect suite of connectors for Cloud storage (Amazon S3)

Secor is a service implementing Kafka log persistence

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Live-coding the cluster!

Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go.

Node.js bindings for librdkafka

A RPC framework leveraging Spark RPC module

Java SDK for CloudEvents

一站式Apache Kafka集群指标监控与运维管控平台

Big Data Ecosystem Docker

Modern and performant Kafka client library for Ruby based on librdkafka

🦄 开源社区系统：基于 SpringBoot + MyBatis + MySQL + Redis + Kafka + Elasticsearch + Spring Security + ... 并提供详细的开发文档和配套教程。包含帖子、评论、私信、系统通知、点赞、关注、搜索、用户设置、数据统计等模块。

StreamLine - Streaming Analytics

Samsara is a real-time analytics platform

DC/OS SDK is a collection of tools, libraries, and documentation for easy integration of technologies such as Kafka, Cassandra, HDFS, Spark, and TensorFlow with DC/OS.

Distributed t-SNE via Apache Spark

Community driven, reusable components for distributed apps

Metronome is a distributed and fault-tolerant event scheduler

Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more...

This library wraps Kafka's embedded test cluster, allowing you to more easily create and run integration tests using JUnit against a "real" kafka server running within the context of your tests. No need to stand up an external kafka cluster!

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

spark ml 算法原理剖析以及具体的源码实现分析

An encrypted data analytics platform

Jupyter kernel for scala and spark

A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

A large-scale entity and relation database supporting aggregation of properties

Kafka-on-Pulsar - A protocol handler that brings native Kafka protocol to Apache Pulsar

AZTK powered by Azure Batch: On-demand, Dockerized, Spark Jobs on Azure

all kinds of scrapy demo

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

Fundamentals of Spark with Python (using PySpark), code examples

Feature Store for Machine Learning

.NET Stream Processing Library for Apache Kafka 🚀