87 Open source projects that are alternatives of or similar to fast-avro-write

Avro for JavaScript ⚡️

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

The metrics pipeline for DC/OS 1.9-1.11

Library for generating avro schema files (.avsc) based on DB tables structure

A fast Go Avro codec

Avro schema generation and serialization / deserialization for Scala

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Gradle plugin to interact with Confluent Schema-Registry.

A collection of Magnolia add-on modules

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

🎼 Docker compose files for various kafka stacks

**Unofficial / Community** Kafka Connect MongoDB Sink Connector - Find the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

A library provides consumer/producer to work with kafka, avro and schema registry

Mu (μ) is a purely functional framework for building micro services.

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

A PHP 7.3+ API client for the Confluent Schema Registry REST API based on Guzzle 6 - http://docs.confluent.io/current/schema-registry/docs/index.html

A tool for data sampling, data generation, and data diffing

A CLI and Go client for Kafka Schema Registry

Showcases how to build a small Event-sourced application using Spring Boot, Spring Kafka, Apache Avro and Apache Kafka

MongoDB Kafka Connector

Apache Avro serializer for .NET

Avro support for kotlinx.serialization

Ruby DSL to create Avro schemas

TypeScript Code Generator for Apache Avro Schema Types

Flexible, Fast & Compact Serialization with RPC

GCS support for avro-tools, parquet-tools and protobuf

Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile

Demo applications and code examples for Confluent Platform and Apache Kafka

Record Query - A tool for doing record analysis and transformation

Avrocado is a convenience library to handle Avro in Golang

Avro Serialisation/Deserialisation (SerDe) library for PHP 7.3+ & 8.0 with a Symfony Serializer integration

☀️ Avro, Protobuf, Thrift on Swagger

Avro SerDe for Apache Spark structured APIs.

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].

Command Line Tool for managing Apache Kafka

Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++

Scala library to eliminate boilerplate

Iceberg is a table format for large, slow-moving tabular data

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

Web tool for Avro Schema Registry |

Confluent Schema Registry for Kafka

Divolte Collector

A Gradle plugin to allow easily performing Java code generation for Apache Avro. It supports JSON schema declaration files, JSON protocol declaration files, and Avro IDL files.

A SQL-like language for performing ETL transformations.

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

Goal: create a Spring Boot application that handles users using Event Sourcing. So, whenever a user is created, updated, or deleted, an event informing this change is sent to Kafka. Also, we will implement another application that listens to those events and saves them in Cassandra. Finally, we will use Testcontainers for integration testing.

Kafka backed service library.

An Avro implementation for .NET

Apache Avro is a data serialization system.

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

How to use Parquet in Flink

Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

☀️ A tool for validating data using JSON Schema and converting JSON Schema documents into different data-interchange formats

A bank simulation application using mainly Clojure, which can be used to end-to-end test and show some graphs.