505 Open source projects that are alternatives of or similar to confluent-spark-avro

A PHP 7.3+ API client for the Confluent Schema Registry REST API based on Guzzle 6 - http://docs.confluent.io/current/schema-registry/docs/index.html

A convenient Elixir library to work with Avro schemas and Confluent® Schema Registry

Confluent Schema Registry for Kafka

Avro Serialisation/Deserialisation (SerDe) library for PHP 7.3+ & 8.0 with a Symfony Serializer integration

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Standalone alternatives to Kafka Connect Connectors

Avro SerDe for Apache Spark structured APIs.

🎼 Docker compose files for various kafka stacks

A CLI and Go client for Kafka Schema Registry

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Goal: create a Spring Boot application that handles users using Event Sourcing. So, whenever a user is created, updated, or deleted, an event informing this change is sent to Kafka. Also, we will implement another application that listens to those events and saves them in Cassandra. Finally, we will use Testcontainers for integration testing.

Golang Client for Schema Registry

Kafka De/Serializer using avro and Confluent's Schema Registry

Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.

Confluent REST Proxy for Kafka

An Avro implementation for .NET

Puppet Module for installing and configuring the Confluent Platform

Demo applications and code examples for Confluent Platform and Apache Kafka

A bank simulation application using mainly Clojure, which can be used to end-to-end test and show some graphs.

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

Iceberg is a table format for large, slow-moving tabular data

📙 json & avro http schema registry backed by Kafka

A library that makes it easier to use the Avro serialization format from Ruby.

Manage Confluent Schema Registry subjects through Infrastructure as code

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Autoscaling for Google Cloud Dataproc

Online search advertisement platform & Realtime Campaign Monitoring [Maybe Deprecated]

Code Library for My Blog

A complete (distributed) BigData stack, running in containers

Framework of vectorized and distributed data analytics

Apache Spark in your IDE with gradle

low-level helpers for Apache Spark libraries and tests

Apache Avro serializer for .NET

Identifies and collects data on cc-licensed content across web crawl data and public apis.

Library for generating avro schema files (.avsc) based on DB tables structure

A compiler for automatically re-targeting sequential Java code to Apache Spark.

Spark Shuffle Optimization with RDMA+AEP

Some class materials for a data processing course using PySpark

❄️ 🐳 Awesome tools and libs for AI, Deep Learning, Machine Learning, Computer Vision, Data Science, Data Analytics and Cognitive Computing that are baked in the oven to be Native on Kubernetes and Docker with Python, R, Scala, Java, C#, Go, Julia, C++ etc

基于Spark2.2新闻网大数据实时系统项目

Sparkline BI Accelerator provides fast ad-hoc query capability over Logical Cubes. This has been folded into our SNAP Platform(http://bit.ly/2oBJSpP) an Integrated BI platform on Apache Spark.

A client UI to inspect Kafka topics, consume, produce and much more

HL7 Apache Spark Datasource

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

My presentation at ODSC India 2018 about Deep Learning with Apache Spark

Internet of Things Integration Example => Apache Kafka + Kafka Connect + MQTT Connector + Sensor Data

Open-source distribute workflow schedule tools, also support streaming task.

blog entries

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

Type System for Data Analysis in Python

An extensive non-reactive Typescript framework that eases the development experience in Spark AR

Resources for Spark AR

Code examples for my blog posts

Apache Spark docker container image (Standalone mode)

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

Implementation of Fellegi-Sunter's canonical model of record linkage in Apache Spark, including EM algorithm to estimate parameters

Collection of different demo applications using Apache Spark

Apache Spark Sentry Integration